server load
Friday, 5 July 2019 · 5 messages · last reply 10 Jul 2019
ВВ
Ветошкин Владимир5 Jul 2019 09:06 #
Hi BaseX team,
I have uploaded about 3700 xml-files (450mb) to db.
CPU usage at that time was about 100%.
Is there any way to reduce the server load?
May be basex has some kind of settings?
Thank you
--
С уважением,
Ветошкин Владимир Владимирович
CG
Christian Grün5 Jul 2019 21:03 #
Quoted text · Ветошкин Владимир · 4 lines
> I have uploaded about 3700 xml-files (450mb) to db. > CPU usage at that time was about 100%. > Is there any way to reduce the server load? > May be basex has some kind of settings?
Usually the I/O usage should be higher than the CPU usage. How much RAM do you have, and how many CPUs cores are available on your system (I guess only 1 of the cores was used)?
ВВ
Ветошкин Владимир7 Jul 2019 08:09 #
Yes, it's test server. It has only 1 core and 500mb ram + 500mp swap.
Quoted text · Christian Grün · 9 lines
05.07.2019, 22:03, "Christian Grün" <christian.gruen at gmail.com>: > > I have uploaded about 3700 xml-files (450mb) to db. > > CPU usage at that time was about 100%. > > Is there any way to reduce the server load? > > May be basex has some kind of settings? > > Usually the I/O usage should be higher than the CPU usage. How much > RAM do you have, and how many CPUs cores are available on your system > (I guess only 1 of the cores was used)?
--
С уважением,
Ветошкин Владимир Владимирович
ВВ
Ветошкин Владимир10 Jul 2019 10:58 #
Do you have any suggestions?
I have attached log from vmstat.
Quoted text · Ветошкин Владимир · 19 lines
07.07.2019, 09:09, "Ветошкин Владимир" <en-trance at yandex.ru>: > Yes, it's test server. It has only 1 core and 500mb ram + 500mp swap. > > 05.07.2019, 22:03, "Christian Grün" <christian.gruen at gmail.com>: > > > > I have uploaded about 3700 xml-files (450mb) to db. > > > CPU usage at that time was about 100%. > > > Is there any way to reduce the server load? > > > May be basex has some kind of settings? > > > > Usually the I/O usage should be higher than the CPU usage. How much > > RAM do you have, and how many CPUs cores are available on your system > > (I guess only 1 of the cores was used)? > > > -- > С уважением, > Ветошкин Владимир Владимирович >
--
С уважением,
Ветошкин Владимир Владимирович
CG
Christian Grün10 Jul 2019 13:38 #
Quoted text · Ветошкин Владимир · 1 line
> Yes, it's test server. It has only 1 core and 500mb ram + 500mp swap.
500 mb doesn’t sound that much. It may actually be sufficient for BaseX itself, but I’m not sure how much space is occupied by other processes?
Two things to try:
1. Use our internal XML parser (see [1] for more details),
2. Add the -XX:StartFlightRecording,filename=dump.jfr flag to the BaseX JVM call and send us the dump.
Best,
Christian
[1] http://docs.basex.org/wiki/Options#INTPARSE
Quoted text · Ветошкин Владимир · 26 lines
> > 05.07.2019, 22:03, "Christian Grün" <christian.gruen at gmail.com>: > > I have uploaded about 3700 xml-files (450mb) to db. > CPU usage at that time was about 100%. > Is there any way to reduce the server load? > May be basex has some kind of settings? > > > Usually the I/O usage should be higher than the CPU usage. How much > RAM do you have, and how many CPUs cores are available on your system > (I guess only 1 of the cores was used)? > > > > -- > С уважением, > Ветошкин Владимир Владимирович > > > > > -- > С уважением, > Ветошкин Владимир Владимирович >
Written for BaseX 9.2. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery