Main Page » Mailing List » October 2010

Out of memory bug

Thursday, 28 October 2010 · 9 messages · last reply 31 Jul 2013
IL
Ivan Lagunov28 Oct 2010 13:07 #
Hello, I've got 'Out of memory' error and corrupted database again, this time it's for 6.3 version. Steps: 1. Started server, started terminal client, checked there are no text and attribute indexes. 2. Started my Java application that runs several tests on database: querying, adding, removing. 3. Then I forgot to reopen terminal client and used the one that I opened in Step 1. 4. Run optimize command for the collection and got error. 5. Cannot open the collection any more as I'm getting 'Out of memory'. It's clear that I shouldn't have used the terminal client that I opened before execution of my Java application. But sometimes it's hard to remember. And it's completely unacceptable that these actions result in the corrupted database. You really should catch ArrayIndexOutOfBoundsException everywhere in order to save database from being corrupted. Terminal client stacktrace:
Quoted text · 1 line
> optimize
Possible bug? Feedback is welcome: basex-talk at mailman.uni-konstanz.de BaseX 6.3: java.lang.ArrayIndexOutOfBoundsException: -1 org.basex.data.PathSummary.add(PathSummary.java:68) org.basex.core.cmd.Optimize.stats(Optimize.java:68) org.basex.core.cmd.Optimize.run(Optimize.java:34) org.basex.core.Command.run(Command.java:236) org.basex.core.Command.exec(Command.java:218) org.basex.core.Command.execute(Command.java:66) org.basex.server.ServerProcess.run(ServerProcess.java:161)
Quoted text · 1 line
> open products
Out of Main Memory. Best regards, Ivan
CG
Christian Grün28 Oct 2010 13:34 #
Hi Ivan,
Quoted text · Ivan Lagunov · 5 lines
> Steps: > 1. Started server, started terminal client, checked there are no text and > attribute indexes. > 2. Started my Java application that runs several tests on database: > querying, adding, removing.
So, if I got it right, your Java application wasn't communicating with the BaseX server instance from 1? – In fact, the standalone and client/server mode should never be mixed. Some time ago, we played around with global locking mechanisms to avoid access to databases that are currently opened by a server instance; we might reintroduce this one day to prevent cases like yours. If you're only working with clients accessing the same server instance, however, you shouldn't encounter any problems. If I'm wrong, this would be sth. we should have another look at.
Quoted text · Ivan Lagunov · 2 lines
> 3. Then I forgot to reopen terminal client and used the one that I opened in > Step 1.
...indeed that's completely ok – Step 2 should rather be avoided (if it didn't access the server from 1, as I mentioned above) Hope this helps, Christian
Quoted text · Ivan Lagunov · 32 lines
> 4. Run optimize command for the collection and got error. > 5. Cannot open the collection any more as I'm getting 'Out of memory'. > > It's clear that I shouldn't have used the terminal client that I opened > before execution of my Java application. But sometimes it's hard to > remember. And it's completely unacceptable that these actions result in the > corrupted database. You really should catch ArrayIndexOutOfBoundsException > everywhere in order to save database from being corrupted. > > Terminal client stacktrace: >> optimize > Possible bug? Feedback is welcome: basex-talk at mailman.uni-konstanz.de > BaseX 6.3: > java.lang.ArrayIndexOutOfBoundsException: -1 > org.basex.data.PathSummary.add(PathSummary.java:68) > org.basex.core.cmd.Optimize.stats(Optimize.java:68) > org.basex.core.cmd.Optimize.run(Optimize.java:34) > org.basex.core.Command.run(Command.java:236) > org.basex.core.Command.exec(Command.java:218) > org.basex.core.Command.execute(Command.java:66) > org.basex.server.ServerProcess.run(ServerProcess.java:161) >> open products > Out of Main Memory. > > Best regards, > Ivan > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >
IL
Ivan Lagunov28 Oct 2010 13:46 #
Hi Christian, You're totally right. My Java application didn't communicate with the running BaseX server. That's the cause of problem. However, it'd be great to have global locking mechanisms or at least an ability to erase existing indexes that are causing 'Out of memory' without opening broken database. Best regards, Ivan Lagunov
Quoted text · Christian Grün · 62 lines
On 10/28/2010 03:34 PM, Christian Grün wrote: > Hi Ivan, > >> Steps: >> 1. Started server, started terminal client, checked there are no text and >> attribute indexes. >> 2. Started my Java application that runs several tests on database: >> querying, adding, removing. > So, if I got it right, your Java application wasn't communicating with > the BaseX server instance from 1? – In fact, the standalone and > client/server mode should never be mixed. Some time ago, we played > around with global locking mechanisms to avoid access to databases > that are currently opened by a server instance; we might reintroduce > this one day to prevent cases like yours. > > If you're only working with clients accessing the same server > instance, however, you shouldn't encounter any problems. If I'm wrong, > this would be sth. we should have another look at. > >> 3. Then I forgot to reopen terminal client and used the one that I opened in >> Step 1. > ...indeed that's completely ok – Step 2 should rather be avoided (if > it didn't access the server from 1, as I mentioned above) > > Hope this helps, > Christian > > > >> 4. Run optimize command for the collection and got error. >> 5. Cannot open the collection any more as I'm getting 'Out of memory'. >> >> It's clear that I shouldn't have used the terminal client that I opened >> before execution of my Java application. But sometimes it's hard to >> remember. And it's completely unacceptable that these actions result in the >> corrupted database. You really should catch ArrayIndexOutOfBoundsException >> everywhere in order to save database from being corrupted. >> >> Terminal client stacktrace: >>> optimize >> Possible bug? Feedback is welcome: basex-talk at mailman.uni-konstanz.de >> BaseX 6.3: >> java.lang.ArrayIndexOutOfBoundsException: -1 >> org.basex.data.PathSummary.add(PathSummary.java:68) >> org.basex.core.cmd.Optimize.stats(Optimize.java:68) >> org.basex.core.cmd.Optimize.run(Optimize.java:34) >> org.basex.core.Command.run(Command.java:236) >> org.basex.core.Command.exec(Command.java:218) >> org.basex.core.Command.execute(Command.java:66) >> org.basex.server.ServerProcess.run(ServerProcess.java:161) >>> open products >> Out of Main Memory. >> >> Best regards, >> Ivan >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> >
AW
Andreas Weilerin reply to Pascal Heus, 28 Oct 2010, 21:3628 Oct 2010 20:06 #
Hi Pascal, i dont know about xquery but in Java you can do the following...
     Context ctx = new Context();
     try {
       new Open("Database").execute(ctx);
       System.out.println(Token.string(ctx.data.info(IndexType.PATH)));
     } catch(Exception e) {
       e.printStackTrace();
     }
Kind regards, Andreas
Quoted text · Pascal Heus · 13 lines
Am 28.10.2010 19:38, schrieb Pascal Heus: > Hi: > the BaseX UI provide a Path Summary tab in the database file properties > from where I can export the underlying XML. Can this XML be retrieved > directly in a xquery statement (I assume as a basex: function)? > Otherwise from Java? > thanks > *P > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
PH
Pascal Heusin reply to Christian Grün, 28 Oct 2010, 13:3428 Oct 2010 21:36 #
Hi: the BaseX UI provide a Path Summary tab in the database file properties from where I can export the underlying XML. Can this XML be retrieved directly in a xquery statement (I assume as a basex: function)? Otherwise from Java? thanks *P
CG
Christian Grünin reply to Andreas Weiler, 28 Oct 2010, 20:0629 Oct 2010 09:10 #
Hi Pascal, you might also use the (undocumented) database command "info index path" to get a plain text representation of the path index. It's a good idea to offer additional basex functions for database meta data; I've put this on our list. Best, Christian
Quoted text · Andreas Weiler · 37 lines
On Thu, Oct 28, 2010 at 8:06 PM, Andreas Weiler <andreas.weiler at uni-konstanz.de> wrote: > Hi Pascal, > > i dont know about xquery but in Java you can do the following... > > Context ctx = new Context(); > try { > new Open("Database").execute(ctx); > System.out.println(Token.string(ctx.data.info(IndexType.PATH))); > } catch(Exception e) { > e.printStackTrace(); > } > > Kind regards, > Andreas > > Am 28.10.2010 19:38, schrieb Pascal Heus: >> >> Hi: >> the BaseX UI provide a Path Summary tab in the database file properties >> from where I can export the underlying XML. Can this XML be retrieved >> directly in a xquery statement (I assume as a basex: function)? >> Otherwise from Java? >> thanks >> *P >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >
PH
Pascal Heus29 Oct 2010 20:17 #
Chritsian: Thanks for the tip. BTW, in the generated XML, it would also be useful to have the element namespace (if this is available). best Pascal
Quoted text · Christian Grün · 48 lines
On 10/29/10 3:10 AM, Christian Grün wrote: > Hi Pascal, > > you might also use the (undocumented) database command "info index > path" to get a plain text representation of the path index. It's a > good idea to offer additional basex functions for database meta data; > I've put this on our list. > > Best, > Christian > > > > On Thu, Oct 28, 2010 at 8:06 PM, Andreas Weiler > <andreas.weiler at uni-konstanz.de> wrote: >> Hi Pascal, >> >> i dont know about xquery but in Java you can do the following... >> >> Context ctx = new Context(); >> try { >> new Open("Database").execute(ctx); >> System.out.println(Token.string(ctx.data.info(IndexType.PATH))); >> } catch(Exception e) { >> e.printStackTrace(); >> } >> >> Kind regards, >> Andreas >> >> Am 28.10.2010 19:38, schrieb Pascal Heus: >>> Hi: >>> the BaseX UI provide a Path Summary tab in the database file properties >>> from where I can export the underlying XML. Can this XML be retrieved >>> directly in a xquery statement (I assume as a basex: function)? >>> Otherwise from Java? >>> thanks >>> *P >>> >>> _______________________________________________ >>> BaseX-Talk mailing list >>> BaseX-Talk at mailman.uni-konstanz.de >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>
PH
Pascal Heusin reply to Christian Grün, 29 Oct 2010, 09:1031 Jul 2013 19:01 #
Christian: Reviving an old question.... Was the "info index path" command ever turned into an xquery function? Don't seem to see an equivalent in http://docs.basex.org/wiki/Index_Module thanks *P
Quoted text · Christian Grün · 48 lines
On 10/29/10 9:10 AM, Christian Grün wrote: > Hi Pascal, > > you might also use the (undocumented) database command "info index > path" to get a plain text representation of the path index. It's a > good idea to offer additional basex functions for database meta data; > I've put this on our list. > > Best, > Christian > > > > On Thu, Oct 28, 2010 at 8:06 PM, Andreas Weiler > <andreas.weiler at uni-konstanz.de> wrote: >> Hi Pascal, >> >> i dont know about xquery but in Java you can do the following... >> >> Context ctx = new Context(); >> try { >> new Open("Database").execute(ctx); >> System.out.println(Token.string(ctx.data.info(IndexType.PATH))); >> } catch(Exception e) { >> e.printStackTrace(); >> } >> >> Kind regards, >> Andreas >> >> Am 28.10.2010 19:38, schrieb Pascal Heus: >>> Hi: >>> the BaseX UI provide a Path Summary tab in the database file properties >>> from where I can export the underlying XML. Can this XML be retrieved >>> directly in a xquery statement (I assume as a basex: function)? >>> Otherwise from Java? >>> thanks >>> *P >>> >>> _______________________________________________ >>> BaseX-Talk mailing list >>> BaseX-Talk at mailman.uni-konstanz.de >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>
CG
Christian Grün31 Jul 2013 19:08 #
Hi Pascal, you are probably looking for the index:facets function. I have slightly revamped the Wiki page to make the relationship more obvious. Christian ______________________________ 2013/7/31 Pascal Heus <pascal.heus at gmail.com>:
Quoted text · Pascal Heus · 60 lines
> Christian: > Reviving an old question.... Was the "info index path" command ever > turned into an xquery function? > Don't seem to see an equivalent in http://docs.basex.org/wiki/Index_Module > thanks > *P > > On 10/29/10 9:10 AM, Christian Grün wrote: >> Hi Pascal, >> >> you might also use the (undocumented) database command "info index >> path" to get a plain text representation of the path index. It's a >> good idea to offer additional basex functions for database meta data; >> I've put this on our list. >> >> Best, >> Christian >> >> >> >> On Thu, Oct 28, 2010 at 8:06 PM, Andreas Weiler >> <andreas.weiler at uni-konstanz.de> wrote: >>> Hi Pascal, >>> >>> i dont know about xquery but in Java you can do the following... >>> >>> Context ctx = new Context(); >>> try { >>> new Open("Database").execute(ctx); >>> System.out.println(Token.string(ctx.data.info(IndexType.PATH))); >>> } catch(Exception e) { >>> e.printStackTrace(); >>> } >>> >>> Kind regards, >>> Andreas >>> >>> Am 28.10.2010 19:38, schrieb Pascal Heus: >>>> Hi: >>>> the BaseX UI provide a Path Summary tab in the database file properties >>>> from where I can export the underlying XML. Can this XML be retrieved >>>> directly in a xquery statement (I assume as a basex: function)? >>>> Otherwise from Java? >>>> thanks >>>> *P >>>> >>>> _______________________________________________ >>>> BaseX-Talk mailing list >>>> BaseX-Talk at mailman.uni-konstanz.de >>>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>> _______________________________________________ >>> BaseX-Talk mailing list >>> BaseX-Talk at mailman.uni-konstanz.de >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>> > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Written for BaseX 6, 7.6. BaseX may have changed since: please check the current documentation.

« How to output plain text (CSV) from XQueryJoins »
⚡Generated with XQuery