Bugreport: org.basex.server.ClientQuery.cache(ClientQuery.java:81)

On Fri, Mar 8, 2013 at 3:06 PM, Bodo Hüsemann <bodo.huesemann at gmx.de> wrote: > Hi all, > > I apologize for the german post on this english mailinglist...So here we go > in english: > > I have recently migrated my BaseX application from version 7.2 to 7.6 (and > 7.6.1 Beta). The BaseX lib throws exceptions while running concurrent XQuery > queries and updates (based on a threadpool), see the exceptions attached > below (the specific numeric error-code changes slightly but the error occurs > every time on the same line of code). The error itself seems to happen unter > heavy concurrency (sequential queries have no problem) and not every time > query executes. > > The application code is as follows (class > de.infabswissarmy.control.XMLDBManagerBaseX.query): > > public ArrayList<String> query(String inQuery, String[][] inArgs){ > ClientSession mySession = null; ClientQuery myQuery = null; > ArrayList<String> tmpString = new ArrayList<String>(); > try { > mySession = getClientSession(); > myQuery = mySession.query(inQuery); > > // bind variable > if (inArgs != null){ > for (int i=0; i < inArgs.length; i++){ > myQuery.bind("$" + inArgs[i][0], inArgs[i][1]); > } > } > > while (myQuery.more()){ // **exception is in this line ** > tmpString.add(myQuery.next()); > } > } catch (IOException e) { > // TODO Auto-generated catch block > e.printStackTrace(); > } finally { > if (myQuery != null){ > try { > myQuery.close(); > } catch (BaseXException e) { > // ignore > } catch (IOException e) { > // TODO Auto-generated catch block > e.printStackTrace(); > } > } > } > return tmpString; > } > > To be complete this is the used helper function: > > private static ClientSession getClientSession() throws IOException { > ClientSession mySession = new ClientSession("localhost", 1984, > "admin", "admin"); > return mySession; > } > > Version 7.2 is running without problems. In version 7.6 I have seen some > rare Java heap space errors at the same location, and once I even saw a > massive growth of the DB > 1GB (normal size would be 1MB), but this may be > another error and I cannot reproduce it. I will try to deliver a > selfcontained min-example, but this could be quite an effort to do (it will > take some time to produce it). So I hope that you might find the problem > with the delivered information. > > Many thanks once again for this great XML-DB and thanks in advance for your > help! > Bodo > > > -- attached error stack with version 7.6.1 (in 7.6 the error occurs in row > 79 instead of 81) > org.basex.core.BaseXException: -146032 > at org.basex.server.ClientQuery.cache(ClientQuery.java:81) > at org.basex.server.Query.more(Query.java:74) > at > de.infabswissarmy.control.XMLDBManagerBaseX.query(XMLDBManagerBaseX.java:86) > at > de.infabswissarmy.control.XMLDBManagerBaseX.queryDoc(XMLDBManagerBaseX.java: > 121) > at de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:180) > at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at java.lang.Thread.run(Unknown Source) > org.basex.core.BaseXException: 0 > at org.basex.server.ClientQuery.cache(ClientQuery.java:81) > at org.basex.server.Query.more(Query.java:74) > at > de.infabswissarmy.control.XMLDBManagerBaseX.query(XMLDBManagerBaseX.java:86) > at > de.infabswissarmy.control.XMLDBManagerBaseX.queryDoc(XMLDBManagerBaseX.java: > 121) > at de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:180) > at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at java.lang.Thread.run(Unknown Source) > org.basex.core.BaseXException: java.lang.ArrayIndexOutOfBoundsException > at org.basex.server.ClientQuery.cache(ClientQuery.java:81) > at org.basex.server.Query.more(Query.java:74) > at > de.infabswissarmy.control.XMLDBManagerBaseX.query(XMLDBManagerBaseX.java:86) > at > de.infabswissarmy.control.XMLDBManagerBaseX.queryDoc(XMLDBManagerBaseX.java: > 121) > at de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:180) > at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at java.lang.Thread.run(Unknown Source) > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

-----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen at gmail.com] Gesendet: Sonntag, 10. März 2013 13:33 An: Bodo Hüsemann Cc: BaseX Betreff: Re: [basex-talk] Bugreport: org.basex.server.ClientQuery.cache(ClientQuery.java:81) Hi Bodo, thanks for your report. I would be interested to know a) if the errors also occurs if you set the GLOBALLOCK flag in your .basex configuration file to "true" b) if the error persists if you close your client sessions via mySession.close() (in both cases, this call is always recommendable) c) which queries you are running? your query strings may be pretty important to find out what causes the problems, in particular if the problems only occur for GLOBALLOCK=false Best, Christian On Fri, Mar 8, 2013 at 3:06 PM, Bodo Hüsemann <bodo.huesemann at gmx.de> wrote: > Hi all, > > I apologize for the german post on this english mailinglist...So here > we go in english: > > I have recently migrated my BaseX application from version 7.2 to 7.6 > (and > 7.6.1 Beta). The BaseX lib throws exceptions while running concurrent > XQuery queries and updates (based on a threadpool), see the exceptions > attached below (the specific numeric error-code changes slightly but > the error occurs every time on the same line of code). The error > itself seems to happen unter heavy concurrency (sequential queries > have no problem) and not every time query executes. > > The application code is as follows (class > de.infabswissarmy.control.XMLDBManagerBaseX.query): > > public ArrayList<String> query(String inQuery, String[][] inArgs){ > ClientSession mySession = null; ClientQuery myQuery = null; > ArrayList<String> tmpString = new ArrayList<String>(); > try { > mySession = getClientSession(); > myQuery = mySession.query(inQuery); > > // bind variable > if (inArgs != null){ > for (int i=0; i < inArgs.length; i++){ > myQuery.bind("$" + inArgs[i][0], inArgs[i][1]); > } > } > > while (myQuery.more()){ // **exception is in this line ** > tmpString.add(myQuery.next()); > } > } catch (IOException e) { > // TODO Auto-generated catch block > e.printStackTrace(); > } finally { > if (myQuery != null){ > try { > myQuery.close(); > } catch (BaseXException e) { > // ignore > } catch (IOException e) { > // TODO Auto-generated catch block > e.printStackTrace(); > } > } > } > return tmpString; > } > > To be complete this is the used helper function: > > private static ClientSession getClientSession() throws IOException { > ClientSession mySession = new ClientSession("localhost", 1984, > "admin", "admin"); > return mySession; > } > > Version 7.2 is running without problems. In version 7.6 I have seen > some rare Java heap space errors at the same location, and once I even > saw a massive growth of the DB > 1GB (normal size would be 1MB), but > this may be another error and I cannot reproduce it. I will try to > deliver a selfcontained min-example, but this could be quite an effort > to do (it will take some time to produce it). So I hope that you might > find the problem with the delivered information. > > Many thanks once again for this great XML-DB and thanks in advance for > your help! > Bodo > > > -- attached error stack with version 7.6.1 (in 7.6 the error occurs in > row > 79 instead of 81) > org.basex.core.BaseXException: -146032 > at org.basex.server.ClientQuery.cache(ClientQuery.java:81) > at org.basex.server.Query.more(Query.java:74) > at > de.infabswissarmy.control.XMLDBManagerBaseX.query(XMLDBManagerBaseX.java:86) > at > de.infabswissarmy.control.XMLDBManagerBaseX.queryDoc(XMLDBManagerBaseX.java: > 121) > at de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:180) > at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at java.lang.Thread.run(Unknown Source) > org.basex.core.BaseXException: 0 > at org.basex.server.ClientQuery.cache(ClientQuery.java:81) > at org.basex.server.Query.more(Query.java:74) > at > de.infabswissarmy.control.XMLDBManagerBaseX.query(XMLDBManagerBaseX.java:86) > at > de.infabswissarmy.control.XMLDBManagerBaseX.queryDoc(XMLDBManagerBaseX.java: > 121) > at de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:180) > at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at java.lang.Thread.run(Unknown Source) > org.basex.core.BaseXException: java.lang.ArrayIndexOutOfBoundsException > at org.basex.server.ClientQuery.cache(ClientQuery.java:81) > at org.basex.server.Query.more(Query.java:74) > at > de.infabswissarmy.control.XMLDBManagerBaseX.query(XMLDBManagerBaseX.java:86) > at > de.infabswissarmy.control.XMLDBManagerBaseX.queryDoc(XMLDBManagerBaseX.java: > 121) > at de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:180) > at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at java.lang.Thread.run(Unknown Source) > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

On Wed, Mar 13, 2013 at 9:06 PM, Bodo Hüsemann <bodo.huesemann at gmx.de> wrote: > Hi Christian, > > I've tried to set GLOBALLOCK to TRUE today and this prevents the error. > > a) I performed some more tests and found some more other errors (without > global locking, see attached log). Anyhow, I suspect an encoding issue for > specific input data so included the documents I have saved in the DB. > > b) I've added mySession.close() to the finally clause but the error still > persists. > > c) Following a query which reproduces the problem (as you can see, I declare > a parameter variable and call xq:getResult). Every call of the query will be > concurrently executed to corresponding parallel uploads of documents that > are read from an internet database meanwhile. > > module namespace xq = "xq"; > import module namespace html = 'html' at 'html.xq'; > > declare function xq:insertTable($inGuild as xs:anyAtomicType) as element()* > { > let $guild_sheet := collection('default')/apiresponse[guild/@name=$inGuild] > return if ($guild_sheet) then <tr id='{string($inGuild)}'> > <td>{string($guild_sheet/guild/@name)}</td> > <td>{string($guild_sheet/guild/@memberCount)}</td> > <td>{if (not(empty($guild_sheet))) then > string($guild_sheet/cache_info/@cached_until_gmt) else ()}</td> > </tr> > else > (<error id='{string($inGuild)}'/>)}; > > declare function xq:getResult($inGuild as xs:anyAtomicType) as element()*{ > (xq:insertTable($inGuild)) > }; > > For analysis purposes I've sent the documents to you separately. > > How can we proceed (I know a selfcontained example would be nice and of > great use, but this will take some time for me). Any other things we could > try? > > Many thanks for your help, > Bodo > > > > --------------------------- > WARNING 2013.03.13 20:42:39 Error while loading document: > 60284427BAD70B0E537DC3214A56091E3FC9DFF4 > org.basex.core.BaseXException: Improper use? Potential bug? Your feedback is > welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 7.6 > Java: Oracle Corporation, 1.7.0_15 > OS: Windows 7, amd64 > Stack Trace: > java.lang.ArrayIndexOutOfBoundsException > java.lang.System.arraycopy(Native Method) > org.basex.io.random.DataAccess.writeToken(DataAccess.java:347) > org.basex.io.random.DataAccess.writeToken(DataAccess.java:329) > org.basex.data.DiskData.index(DiskData.java:394) > org.basex.data.Data.attr(Data.java:1027) > org.basex.data.Data.insert(Data.java:811) > org.basex.core.cmd.Add.run(Add.java:118) > org.basex.core.Command.run(Command.java:342) > org.basex.core.Command.exec(Command.java:321) > org.basex.core.Command.execute(Command.java:78) > org.basex.core.Command.execute(Command.java:90) > org.basex.server.ClientListener.execute(ClientListener.java:386) > org.basex.server.ClientListener.add(ClientListener.java:357) > org.basex.server.ClientListener.run(ClientListener.java:100) > > at org.basex.server.ClientSession.receive(ClientSession.java:259) > at org.basex.server.ClientSession.send(ClientSession.java:246) > at org.basex.server.ClientSession.send(ClientSession.java:284) > at org.basex.server.ClientSession.add(ClientSession.java:134) > at > de.infabswissarmy.control.XMLDBManagerBaseX.putDoc(XMLDBManagerBaseX.java:24 > 2) > at > de.infabswissarmy.control.XMLDBManagerBaseX.putDoc(XMLDBManagerBaseX.java:56 > ) > at > de.infabswissarmy.control.UpdateManager.getFile(UpdateManager.java:204) > at de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:144) > at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at java.lang.Thread.run(Unknown Source) > > --- > > org.basex.core.BaseXException: Improper use? Potential bug? Your feedback is > welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 7.6 > Java: Oracle Corporation, 1.7.0_15 > OS: Windows 7, amd64 > Stack Trace: > java.lang.RuntimeException: Data Access out of bounds: > - pre value: 4517 > - #used blocks: 18 > - #total locks: 18 > - access: 17 (18 > 17] > org.basex.util.Util.notexpected(Util.java:53) > org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:488) > org.basex.io.random.TableDiskAccess.read5(TableDiskAccess.java:211) > org.basex.data.Data.textOff(Data.java:422) > org.basex.data.DiskData.text(DiskData.java:234) > org.basex.index.resource.Docs.paths(Docs.java:107) > org.basex.index.resource.Docs.docs(Docs.java:230) > org.basex.index.resource.Resources.docs(Resources.java:110) > org.basex.query.QueryResources.collection(QueryResources.java:178) > org.basex.query.func.FNGen.collection(FNGen.java:121) > org.basex.query.func.FNGen.value(FNGen.java:74) > org.basex.query.func.StandardFunc.compile(StandardFunc.java:48) > org.basex.query.expr.Arr.compile(Arr.java:38) > org.basex.query.func.StandardFunc.compile(StandardFunc.java:46) > org.basex.query.QueryContext.analyze(QueryContext.java:225) > org.basex.query.QueryContext.compile(QueryContext.java:209) > org.basex.query.QueryProcessor.compile(QueryProcessor.java:66) > org.basex.core.cmd.AQuery.query(AQuery.java:69) > org.basex.core.cmd.XQuery.run(XQuery.java:22) > org.basex.core.Command.run(Command.java:342) > org.basex.core.Command.exec(Command.java:321) > org.basex.core.Command.execute(Command.java:78) > org.basex.server.ClientListener.run(ClientListener.java:145) > > at org.basex.server.ClientSession.receive(ClientSession.java:259) > at org.basex.server.ClientSession.execute(ClientSession.java:162) > at org.basex.server.Session.execute(Session.java:50) > at > de.infabswissarmy.control.XMLDBManagerBaseX.executeCommand(XMLDBManagerBaseX > .java:172) > at > de.infabswissarmy.control.XMLDBManagerBaseX.executeXQuery(XMLDBManagerBaseX. > java:163) > at > de.infabswissarmy.control.XMLDBManagerBaseX.executeXQuery(XMLDBManagerBaseX. > java:159) > at > de.infabswissarmy.control.XMLDBManagerBaseX.existsDoc(XMLDBManagerBaseX.java > :145) > at > de.infabswissarmy.control.XMLDBManagerBaseX.putDoc(XMLDBManagerBaseX.java:23 > 5) > at > de.infabswissarmy.control.XMLDBManagerBaseX.putDoc(XMLDBManagerBaseX.java:56 > ) > at > de.infabswissarmy.control.UpdateManager.getFile(UpdateManager.java:204) > at de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:144) > at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at java.lang.Thread.run(Unknown Source) > -----Ursprüngliche Nachricht----- > Von: Christian Grün [mailto:christian.gruen at gmail.com] > Gesendet: Sonntag, 10. März 2013 13:33 > An: Bodo Hüsemann > Cc: BaseX > Betreff: Re: [basex-talk] Bugreport: > org.basex.server.ClientQuery.cache(ClientQuery.java:81) > > Hi Bodo, > > thanks for your report. I would be interested to know… > > a) if the errors also occurs if you set the GLOBALLOCK flag in your .basex > configuration file to "true" > b) if the error persists if you close your client sessions via > mySession.close() (in both cases, this call is always recommendable) > c) which queries you are running? your query strings may be pretty important > to find out what causes the problems, in particular if the problems only > occur for GLOBALLOCK=false > > Best, > Christian > ___________________________ > > On Fri, Mar 8, 2013 at 3:06 PM, Bodo Hüsemann <bodo.huesemann at gmx.de> wrote: >> Hi all, >> >> I apologize for the german post on this english mailinglist...So here >> we go in english: >> >> I have recently migrated my BaseX application from version 7.2 to 7.6 >> (and >> 7.6.1 Beta). The BaseX lib throws exceptions while running concurrent >> XQuery queries and updates (based on a threadpool), see the exceptions >> attached below (the specific numeric error-code changes slightly but >> the error occurs every time on the same line of code). The error >> itself seems to happen unter heavy concurrency (sequential queries >> have no problem) and not every time query executes. >> >> The application code is as follows (class >> de.infabswissarmy.control.XMLDBManagerBaseX.query): >> >> public ArrayList<String> query(String inQuery, String[][] inArgs){ >> ClientSession mySession = null; ClientQuery myQuery = null; >> ArrayList<String> tmpString = new ArrayList<String>(); >> try { >> mySession = getClientSession(); >> myQuery = mySession.query(inQuery); >> >> // bind variable >> if (inArgs != null){ >> for (int i=0; i < inArgs.length; i++){ >> myQuery.bind("$" + inArgs[i][0], inArgs[i][1]); >> } >> } >> >> while (myQuery.more()){ // **exception is in this line ** >> tmpString.add(myQuery.next()); >> } >> } catch (IOException e) { >> // TODO Auto-generated catch block >> e.printStackTrace(); >> } finally { >> if (myQuery != null){ >> try { >> myQuery.close(); >> } catch (BaseXException e) { >> // ignore >> } catch (IOException e) { >> // TODO Auto-generated catch block >> e.printStackTrace(); >> } >> } >> } >> return tmpString; >> } >> >> To be complete this is the used helper function: >> >> private static ClientSession getClientSession() throws IOException { >> ClientSession mySession = new ClientSession("localhost", 1984, >> "admin", "admin"); >> return mySession; >> } >> >> Version 7.2 is running without problems. In version 7.6 I have seen >> some rare Java heap space errors at the same location, and once I even >> saw a massive growth of the DB > 1GB (normal size would be 1MB), but >> this may be another error and I cannot reproduce it. I will try to >> deliver a selfcontained min-example, but this could be quite an effort >> to do (it will take some time to produce it). So I hope that you might >> find the problem with the delivered information. >> >> Many thanks once again for this great XML-DB and thanks in advance for >> your help! >> Bodo >> >> >> -- attached error stack with version 7.6.1 (in 7.6 the error occurs in >> row >> 79 instead of 81) >> org.basex.core.BaseXException: -146032 >> at org.basex.server.ClientQuery.cache(ClientQuery.java:81) >> at org.basex.server.Query.more(Query.java:74) >> at >> > de.infabswissarmy.control.XMLDBManagerBaseX.query(XMLDBManagerBaseX.java:86) >> at >> > de.infabswissarmy.control.XMLDBManagerBaseX.queryDoc(XMLDBManagerBaseX.java: >> 121) >> at > de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:180) >> at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown > Source) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown >> Source) >> at java.lang.Thread.run(Unknown Source) >> org.basex.core.BaseXException: 0 >> at org.basex.server.ClientQuery.cache(ClientQuery.java:81) >> at org.basex.server.Query.more(Query.java:74) >> at >> > de.infabswissarmy.control.XMLDBManagerBaseX.query(XMLDBManagerBaseX.java:86) >> at >> > de.infabswissarmy.control.XMLDBManagerBaseX.queryDoc(XMLDBManagerBaseX.java: >> 121) >> at > de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:180) >> at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown > Source) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown >> Source) >> at java.lang.Thread.run(Unknown Source) >> org.basex.core.BaseXException: java.lang.ArrayIndexOutOfBoundsException >> at org.basex.server.ClientQuery.cache(ClientQuery.java:81) >> at org.basex.server.Query.more(Query.java:74) >> at >> > de.infabswissarmy.control.XMLDBManagerBaseX.query(XMLDBManagerBaseX.java:86) >> at >> > de.infabswissarmy.control.XMLDBManagerBaseX.queryDoc(XMLDBManagerBaseX.java: >> 121) >> at > de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:180) >> at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown > Source) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown >> Source) >> at java.lang.Thread.run(Unknown Source) >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

-----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen at gmail.com] Gesendet: Mittwoch, 13. März 2013 23:36 An: Bodo Hüsemann Cc: BaseX Betreff: Re: [basex-talk] Bugreport: org.basex.server.ClientQuery.cache(ClientQuery.java:81) Bodo, thanks for your in-depth bug report, which allowed me to track down and fix the bug. Could you check out the latest stable snapshot [1] and give us some feedback if the problem is resolved? Best, Christian [1] http://files.basex.org/releases/latest/ On Wed, Mar 13, 2013 at 9:06 PM, Bodo Hüsemann <bodo.huesemann at gmx.de> wrote: > Hi Christian, > > I've tried to set GLOBALLOCK to TRUE today and this prevents the error. > > a) I performed some more tests and found some more other errors (without > global locking, see attached log). Anyhow, I suspect an encoding issue for > specific input data so included the documents I have saved in the DB. > > b) I've added mySession.close() to the finally clause but the error still > persists. > > c) Following a query which reproduces the problem (as you can see, I declare > a parameter variable and call xq:getResult). Every call of the query will be > concurrently executed to corresponding parallel uploads of documents that > are read from an internet database meanwhile. > > module namespace xq = "xq"; > import module namespace html = 'html' at 'html.xq'; > > declare function xq:insertTable($inGuild as xs:anyAtomicType) as element()* > { > let $guild_sheet := collection('default')/apiresponse[guild/@name=$inGuild] > return if ($guild_sheet) then <tr id='{string($inGuild)}'> > <td>{string($guild_sheet/guild/@name)}</td> > <td>{string($guild_sheet/guild/@memberCount)}</td> > <td>{if (not(empty($guild_sheet))) then > string($guild_sheet/cache_info/@cached_until_gmt) else ()}</td> > </tr> > else > (<error id='{string($inGuild)}'/>)}; > > declare function xq:getResult($inGuild as xs:anyAtomicType) as element()*{ > (xq:insertTable($inGuild)) > }; > > For analysis purposes I've sent the documents to you separately. > > How can we proceed (I know a selfcontained example would be nice and of > great use, but this will take some time for me). Any other things we could > try? > > Many thanks for your help, > Bodo > > > > --------------------------- > WARNING 2013.03.13 20:42:39 Error while loading document: > 60284427BAD70B0E537DC3214A56091E3FC9DFF4 > org.basex.core.BaseXException: Improper use? Potential bug? Your feedback is > welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 7.6 > Java: Oracle Corporation, 1.7.0_15 > OS: Windows 7, amd64 > Stack Trace: > java.lang.ArrayIndexOutOfBoundsException > java.lang.System.arraycopy(Native Method) > org.basex.io.random.DataAccess.writeToken(DataAccess.java:347) > org.basex.io.random.DataAccess.writeToken(DataAccess.java:329) > org.basex.data.DiskData.index(DiskData.java:394) > org.basex.data.Data.attr(Data.java:1027) > org.basex.data.Data.insert(Data.java:811) > org.basex.core.cmd.Add.run(Add.java:118) > org.basex.core.Command.run(Command.java:342) > org.basex.core.Command.exec(Command.java:321) > org.basex.core.Command.execute(Command.java:78) > org.basex.core.Command.execute(Command.java:90) > org.basex.server.ClientListener.execute(ClientListener.java:386) > org.basex.server.ClientListener.add(ClientListener.java:357) > org.basex.server.ClientListener.run(ClientListener.java:100) > > at org.basex.server.ClientSession.receive(ClientSession.java:259) > at org.basex.server.ClientSession.send(ClientSession.java:246) > at org.basex.server.ClientSession.send(ClientSession.java:284) > at org.basex.server.ClientSession.add(ClientSession.java:134) > at > de.infabswissarmy.control.XMLDBManagerBaseX.putDoc(XMLDBManagerBaseX.java:24 > 2) > at > de.infabswissarmy.control.XMLDBManagerBaseX.putDoc(XMLDBManagerBaseX.java:56 > ) > at > de.infabswissarmy.control.UpdateManager.getFile(UpdateManager.java:204) > at de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:144) > at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at java.lang.Thread.run(Unknown Source) > > --- > > org.basex.core.BaseXException: Improper use? Potential bug? Your feedback is > welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 7.6 > Java: Oracle Corporation, 1.7.0_15 > OS: Windows 7, amd64 > Stack Trace: > java.lang.RuntimeException: Data Access out of bounds: > - pre value: 4517 > - #used blocks: 18 > - #total locks: 18 > - access: 17 (18 > 17] > org.basex.util.Util.notexpected(Util.java:53) > org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:488) > org.basex.io.random.TableDiskAccess.read5(TableDiskAccess.java:211) > org.basex.data.Data.textOff(Data.java:422) > org.basex.data.DiskData.text(DiskData.java:234) > org.basex.index.resource.Docs.paths(Docs.java:107) > org.basex.index.resource.Docs.docs(Docs.java:230) > org.basex.index.resource.Resources.docs(Resources.java:110) > org.basex.query.QueryResources.collection(QueryResources.java:178) > org.basex.query.func.FNGen.collection(FNGen.java:121) > org.basex.query.func.FNGen.value(FNGen.java:74) > org.basex.query.func.StandardFunc.compile(StandardFunc.java:48) > org.basex.query.expr.Arr.compile(Arr.java:38) > org.basex.query.func.StandardFunc.compile(StandardFunc.java:46) > org.basex.query.QueryContext.analyze(QueryContext.java:225) > org.basex.query.QueryContext.compile(QueryContext.java:209) > org.basex.query.QueryProcessor.compile(QueryProcessor.java:66) > org.basex.core.cmd.AQuery.query(AQuery.java:69) > org.basex.core.cmd.XQuery.run(XQuery.java:22) > org.basex.core.Command.run(Command.java:342) > org.basex.core.Command.exec(Command.java:321) > org.basex.core.Command.execute(Command.java:78) > org.basex.server.ClientListener.run(ClientListener.java:145) > > at org.basex.server.ClientSession.receive(ClientSession.java:259) > at org.basex.server.ClientSession.execute(ClientSession.java:162) > at org.basex.server.Session.execute(Session.java:50) > at > de.infabswissarmy.control.XMLDBManagerBaseX.executeCommand(XMLDBManagerBaseX > .java:172) > at > de.infabswissarmy.control.XMLDBManagerBaseX.executeXQuery(XMLDBManagerBaseX. > java:163) > at > de.infabswissarmy.control.XMLDBManagerBaseX.executeXQuery(XMLDBManagerBaseX. > java:159) > at > de.infabswissarmy.control.XMLDBManagerBaseX.existsDoc(XMLDBManagerBaseX.java > :145) > at > de.infabswissarmy.control.XMLDBManagerBaseX.putDoc(XMLDBManagerBaseX.java:23 > 5) > at > de.infabswissarmy.control.XMLDBManagerBaseX.putDoc(XMLDBManagerBaseX.java:56 > ) > at > de.infabswissarmy.control.UpdateManager.getFile(UpdateManager.java:204) > at de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:144) > at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at java.lang.Thread.run(Unknown Source) > -----Ursprüngliche Nachricht----- > Von: Christian Grün [mailto:christian.gruen at gmail.com] > Gesendet: Sonntag, 10. März 2013 13:33 > An: Bodo Hüsemann > Cc: BaseX > Betreff: Re: [basex-talk] Bugreport: > org.basex.server.ClientQuery.cache(ClientQuery.java:81) > > Hi Bodo, > > thanks for your report. I would be interested to know > > a) if the errors also occurs if you set the GLOBALLOCK flag in your .basex > configuration file to "true" > b) if the error persists if you close your client sessions via > mySession.close() (in both cases, this call is always recommendable) > c) which queries you are running? your query strings may be pretty important > to find out what causes the problems, in particular if the problems only > occur for GLOBALLOCK=false > > Best, > Christian > ___________________________ > > On Fri, Mar 8, 2013 at 3:06 PM, Bodo Hüsemann <bodo.huesemann at gmx.de> wrote: >> Hi all, >> >> I apologize for the german post on this english mailinglist...So here >> we go in english: >> >> I have recently migrated my BaseX application from version 7.2 to 7.6 >> (and >> 7.6.1 Beta). The BaseX lib throws exceptions while running concurrent >> XQuery queries and updates (based on a threadpool), see the exceptions >> attached below (the specific numeric error-code changes slightly but >> the error occurs every time on the same line of code). The error >> itself seems to happen unter heavy concurrency (sequential queries >> have no problem) and not every time query executes. >> >> The application code is as follows (class >> de.infabswissarmy.control.XMLDBManagerBaseX.query): >> >> public ArrayList<String> query(String inQuery, String[][] inArgs){ >> ClientSession mySession = null; ClientQuery myQuery = null; >> ArrayList<String> tmpString = new ArrayList<String>(); >> try { >> mySession = getClientSession(); >> myQuery = mySession.query(inQuery); >> >> // bind variable >> if (inArgs != null){ >> for (int i=0; i < inArgs.length; i++){ >> myQuery.bind("$" + inArgs[i][0], inArgs[i][1]); >> } >> } >> >> while (myQuery.more()){ // **exception is in this line ** >> tmpString.add(myQuery.next()); >> } >> } catch (IOException e) { >> // TODO Auto-generated catch block >> e.printStackTrace(); >> } finally { >> if (myQuery != null){ >> try { >> myQuery.close(); >> } catch (BaseXException e) { >> // ignore >> } catch (IOException e) { >> // TODO Auto-generated catch block >> e.printStackTrace(); >> } >> } >> } >> return tmpString; >> } >> >> To be complete this is the used helper function: >> >> private static ClientSession getClientSession() throws IOException { >> ClientSession mySession = new ClientSession("localhost", 1984, >> "admin", "admin"); >> return mySession; >> } >> >> Version 7.2 is running without problems. In version 7.6 I have seen >> some rare Java heap space errors at the same location, and once I even >> saw a massive growth of the DB > 1GB (normal size would be 1MB), but >> this may be another error and I cannot reproduce it. I will try to >> deliver a selfcontained min-example, but this could be quite an effort >> to do (it will take some time to produce it). So I hope that you might >> find the problem with the delivered information. >> >> Many thanks once again for this great XML-DB and thanks in advance for >> your help! >> Bodo >> >> >> -- attached error stack with version 7.6.1 (in 7.6 the error occurs in >> row >> 79 instead of 81) >> org.basex.core.BaseXException: -146032 >> at org.basex.server.ClientQuery.cache(ClientQuery.java:81) >> at org.basex.server.Query.more(Query.java:74) >> at >> > de.infabswissarmy.control.XMLDBManagerBaseX.query(XMLDBManagerBaseX.java:86) >> at >> > de.infabswissarmy.control.XMLDBManagerBaseX.queryDoc(XMLDBManagerBaseX.java: >> 121) >> at > de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:180) >> at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown > Source) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown >> Source) >> at java.lang.Thread.run(Unknown Source) >> org.basex.core.BaseXException: 0 >> at org.basex.server.ClientQuery.cache(ClientQuery.java:81) >> at org.basex.server.Query.more(Query.java:74) >> at >> > de.infabswissarmy.control.XMLDBManagerBaseX.query(XMLDBManagerBaseX.java:86) >> at >> > de.infabswissarmy.control.XMLDBManagerBaseX.queryDoc(XMLDBManagerBaseX.java: >> 121) >> at > de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:180) >> at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown > Source) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown >> Source) >> at java.lang.Thread.run(Unknown Source) >> org.basex.core.BaseXException: java.lang.ArrayIndexOutOfBoundsException >> at org.basex.server.ClientQuery.cache(ClientQuery.java:81) >> at org.basex.server.Query.more(Query.java:74) >> at >> > de.infabswissarmy.control.XMLDBManagerBaseX.query(XMLDBManagerBaseX.java:86) >> at >> > de.infabswissarmy.control.XMLDBManagerBaseX.queryDoc(XMLDBManagerBaseX.java: >> 121) >> at > de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:180) >> at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown > Source) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown >> Source) >> at java.lang.Thread.run(Unknown Source) >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >