Out of Main Memory issues when opening a tiny database

On Tue, Aug 31, 2010 at 8:35 PM, Sandra Maria Silcot <ssilcot at unimelb.edu.au > wrote: > Hi all, > > This I feel sure is "pilot error"; I am messing something up in my perl > script to create the db. Have a tiny collection of 6 documents 75kb when > loaded. It seems to load ok but when I go to use it I am getting Out of > Main Memory errors. The JVM is 4gb using -Xms and -Xmx options on the java > command to start the basex server (on an 8gb VM). > > On reviewing the logs I can see that my client is not doing a CLOSE for > the database -- yesterday's log showed 175 OPENs and only 31 closes -- > OOPS!! Ok, so I start getting this: > > OPEN chaingang Error: Out of Main Memory. 11286.05 ms > I can see in top the JVMs ram just being gobbled up rapidly until it runs > out and this message appears. > > Yes, I know I need to fix my clients but is it possible that even when the > clients session closes, somehow the database is being kept open, and > eventually we run out of memory? But that doesn;t really fit with the > "gobbling ram" I am seeing, unless the former situation triggers the > latter. > > IF this is the issue, could the basex server be made to nicely close any > databases open in a session when that session is closed or the pipe gets > broken? > > # when trying to recover > 2010-08-31.log:22:24:11.781 > DROP DB chaingang Error: Database 'chaingang' is currently locked. > > Or of a db has become corrupted somehow, could it cause such message? Are > there any integrity check commands on the horizon? > > I feel a bit nervous about these kinds of memory errors as I operate in a > VSphere v4 (VMWARE) environment and my JVM monitoring skills are > rudimentary -- the JVM does look like it has a nice big contiguous slab of > ram to work with. Clearly something bad/silly I have done is triggering > this behaviour: basex just "runs away" with ram. > > Any thoughts? Has the api interface changed -- my perl client dates back > to late March? > > Sorry to be so vague. > > Any similarly vague clues most welcome. > > On a similar vien, my web interface/query controller is modperl2 under > apache, which means the script sits resident in ram. I have it > opening/closing a basex session on each request cycle (and am about to go > check what I do when it closes!). Perhaps it would be far better to have > the basex connection done in the apache initialisation stage (when apache2 > launches the modperl2 modules) so the session just sits there until apache > restarts. Then each request cycle should just open/close the database? I > ask here as it could possibly be related to my grief above. If not my > apologies for confusing the matter. > > I am using this Sun jdk: /opt/java/jdk1.6.0_18/bin/java. Kind of > regretting not using openSolaris as my OS platform rather than Linux. I do > so love its trace facilities and its a great Java platform. I'm using > 64bit Linux: 2.6.27-17-server #1 SMP Fri Mar 12 02:48:52 UTC 2010 x86_64 > GNU/Linux ; and BaseX 6.2.6 [Server] > > Best wishes all, > > Sandra > > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

> Download Eclipse MAT http://www.eclipse.org/mat/ > <http://www.eclipse.org/mat/>and run the JVM with > http://stackoverflow.com/questions/542979/using-heapdumponoutofmemoryerror-parameter-for-heap-dump-for-jboss > > <http://stackoverflow.com/questions/542979/using-heapdumponoutofmemoryerror-parameter-for-heap-dump-for-jboss>Take the snapshot and open it in MAT and tell BaseX's developers why it's running > out of memory (or put the report MAT creates on your webserver - it's just > a > bunch of .html files) > > - Godmar > > On Tue, Aug 31, 2010 at 8:35 PM, Sandra Maria Silcot > <ssilcot at unimelb.edu.au >> wrote: > >> Hi all, >> This I feel sure is "pilot error"; I am messing something up in my perl script to create the db. Have a tiny collection of 6 documents 75kb when >> loaded. It seems to load ok but when I go to use it I am getting Out of Main Memory errors. The JVM is 4gb using -Xms and -Xmx options on the java >> command to start the basex server (on an 8gb VM). >> On reviewing the logs I can see that my client is not doing a CLOSE for

>> OPEN chaingang Error: Out of Main Memory. 11286.05 ms >> I can see in top the JVMs ram just being gobbled up rapidly until it runs >> out and this message appears. >> Yes, I know I need to fix my clients but is it possible that even when the >> clients session closes, somehow the database is being kept open, and

>> IF this is the issue, could the basex server be made to nicely close any >> databases open in a session when that session is closed or the pipe gets >> broken? >> # when trying to recover >> 2010-08-31.log:22:24:11.781 >> DROP DB chaingang Error: Database 'chaingang' is currently

>> Or of a db has become corrupted somehow, could it cause such message? Are >> there any integrity check commands on the horizon? >> I feel a bit nervous about these kinds of memory errors as I operate in

>> VSphere v4 (VMWARE) environment and my JVM monitoring skills are rudimentary -- the JVM does look like it has a nice big contiguous slab of >> ram to work with. Clearly something bad/silly I have done is triggering

>> Any thoughts? Has the api interface changed -- my perl client dates back >> to late March? >> Sorry to be so vague. >> Any similarly vague clues most welcome. >> On a similar vien, my web interface/query controller is modperl2 under apache, which means the script sits resident in ram. I have it >> opening/closing a basex session on each request cycle (and am about to go >> check what I do when it closes!). Perhaps it would be far better to have >> the basex connection done in the apache initialisation stage (when apache2 >> launches the modperl2 modules) so the session just sits there until apache >> restarts. Then each request cycle should just open/close the database? I >> ask here as it could possibly be related to my grief above. If not my

>> I am using this Sun jdk: /opt/java/jdk1.6.0_18/bin/java. Kind of regretting not using openSolaris as my OS platform rather than Linux. I do >> so love its trace facilities and its a great Java platform. I'm using

>> GNU/Linux ; and BaseX 6.2.6 [Server] >> Best wishes all, >> Sandra >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

On Wed, Sep 1, 2010 at 8:24 AM, Sandra Maria Silcot <ssilcot at unimelb.edu.au> wrote: > Godmar, > > I'm not running Jboss -- I running the server as a jar like this: > > nohup java -Xmx4200m -cp basex-current.jar org.basex.BaseXServer -p 8888 > (basex-current.jar symlinks to -> basex-6.2.6.jar) > > Is Jboss required for MAT? > > Please note: > > * I have gone through my modperl and perl admin clients > * I have checked I issue a 'close' for every open and that I close the > connection -- several corrections were made where this was NOT happening > * I have corrected my clients so I'm not sending empty/null commands (a > carriage return or a null was slipping through) > * I have closely monitored usage today and basex has been very well > behaved - no problems at all > * I have cycled through various scripted full creates and all seems well > * the JVM is using ram, as I would expect it to: > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 8823 fas 20 0 4697m 3.7g 8976 S 0 49.3 16:22.60 java > > This may be coincidence but it looks very much to me as if my sloppy > coding in not "close"(ing) databases was triggering the runaway ram > problem. > > If I can do the MAT thing without jboss - please let me know, and I will > roll back my changes and try to replicate, but I'm feeling fairly sure you > could be looking at your server code and perhaps making it a little more > defensive against sloppy clients! > > I'll continue to monitor. If you like I can send you yesterday's log > privately - its too embarassing to post it to the list!!! > > Many thanks, > > Sandra > >> Download Eclipse MAT http://www.eclipse.org/mat/ >> <http://www.eclipse.org/mat/>and run the JVM with >> http://stackoverflow.com/questions/542979/using-heapdumponoutofmemoryerror-parameter-for-heap-dump-for-jboss >> >> <http://stackoverflow.com/questions/542979/using-heapdumponoutofmemoryerror-parameter-for-heap-dump-for-jboss>Take > the snapshot and open it in MAT and tell BaseX's developers why it's running >> out of memory (or put the report MAT creates on your webserver - it's > just >> a >> bunch of .html files) >> >> - Godmar >> >> On Tue, Aug 31, 2010 at 8:35 PM, Sandra Maria Silcot >> <ssilcot at unimelb.edu.au >>> wrote: >> >>> Hi all, >>> This I feel sure is "pilot error"; I am messing something up in my perl > script to create the db. Have a tiny collection of 6 documents 75kb when >>> loaded. It seems to load ok but when I go to use it I am getting Out of > Main Memory errors. The JVM is 4gb using -Xms and -Xmx options on the java >>> command to start the basex server (on an 8gb VM). >>> On reviewing the logs I can see that my client is not doing a CLOSE for > the database -- yesterday's log showed 175 OPENs and only 31 closes -- > OOPS!! Ok, so I start getting this: >>> OPEN chaingang Error: Out of Main Memory. 11286.05 ms >>> I can see in top the JVMs ram just being gobbled up rapidly until it > runs >>> out and this message appears. >>> Yes, I know I need to fix my clients but is it possible that even when > the >>> clients session closes, somehow the database is being kept open, and > eventually we run out of memory? But that doesn;t really fit with the > "gobbling ram" I am seeing, unless the former situation triggers the > latter. >>> IF this is the issue, could the basex server be made to nicely close > any >>> databases open in a session when that session is closed or the pipe > gets >>> broken? >>> # when trying to recover >>> 2010-08-31.log:22:24:11.781 >>> DROP DB chaingang Error: Database 'chaingang' is currently > locked. >>> Or of a db has become corrupted somehow, could it cause such message? > Are >>> there any integrity check commands on the horizon? >>> I feel a bit nervous about these kinds of memory errors as I operate in > a >>> VSphere v4 (VMWARE) environment and my JVM monitoring skills are > rudimentary -- the JVM does look like it has a nice big contiguous slab of >>> ram to work with. Clearly something bad/silly I have done is triggering > this behaviour: basex just "runs away" with ram. >>> Any thoughts? Has the api interface changed -- my perl client dates > back >>> to late March? >>> Sorry to be so vague. >>> Any similarly vague clues most welcome. >>> On a similar vien, my web interface/query controller is modperl2 under > apache, which means the script sits resident in ram. I have it >>> opening/closing a basex session on each request cycle (and am about to > go >>> check what I do when it closes!). Perhaps it would be far better to > have >>> the basex connection done in the apache initialisation stage (when > apache2 >>> launches the modperl2 modules) so the session just sits there until > apache >>> restarts. Then each request cycle should just open/close the database? > I >>> ask here as it could possibly be related to my grief above. If not my > apologies for confusing the matter. >>> I am using this Sun jdk: /opt/java/jdk1.6.0_18/bin/java. Kind of > regretting not using openSolaris as my OS platform rather than Linux. I do >>> so love its trace facilities and its a great Java platform. I'm using > 64bit Linux: 2.6.27-17-server #1 SMP Fri Mar 12 02:48:52 UTC 2010 > x86_64 >>> GNU/Linux ; and BaseX 6.2.6 [Server] >>> Best wishes all, >>> Sandra >>> _______________________________________________ >>> BaseX-Talk mailing list >>> BaseX-Talk at mailman.uni-konstanz.de >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > > > > > > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >