BaseX weird error
Wednesday, 9 May 2018 · 9 messages · last reply 18 May 2018
HT
halit tiryaki9 May 2018 06:50 #
Hello,
got a serious problem here. any ideas? thanks
Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk at mailman.uni-konstanz.de
Version: BaseX 9.0.2 beta
Java: Oracle Corporation, 1.8.0_151
OS: Linux, amd64
Stack Trace:
java.lang.ArrayIndexOutOfBoundsException: 4288
at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:151)
at org.basex.data.Data.kind(Data.java:294)
at org.basex.query.value.node.DBNode$4.next(DBNode.java:332)
at org.basex.query.value.node.DBNode$4.next(DBNode.java:323)
at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38)
at org.basex.query.expr.path.IterStep$1.next(IterStep.java:32)
at org.basex.query.QueryContext.next(QueryContext.java:392)
at org.basex.query.expr.path.IterPath$1.next(IterPath.java:50)
at org.basex.query.expr.path.IterPath$1.next(IterPath.java:34)
at org.basex.query.expr.ParseExpr.item(ParseExpr.java:58)
at org.basex.query.expr.ParseExpr.atomItem(ParseExpr.java:84)
at org.basex.query.func.fn.FnConcat.item(FnConcat.java:20)
at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71)
at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177)
at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72)
at org.basex.query.expr.path.CachedPath.nodeIter(CachedPath.java:36)
at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:69)
at org.basex.query.expr.gflwor.For$1.next(For.java:107)
at org.basex.query.expr.gflwor.OrderBy$1.sort(OrderBy.java:73)
at org.basex.query.expr.gflwor.OrderBy$1.next(OrderBy.java:54)
at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72)
at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177)
at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:87)
at org.basex.query.QueryContext.next(QueryContext.java:392)
at org.basex.query.scope.MainModule$1.next(MainModule.java:122)
at org.basex.core.cmd.AQuery.query(AQuery.java:94)
at org.basex.core.cmd.XQuery.run(XQuery.java:22)
at org.basex.core.Command.run(Command.java:257)
at org.basex.core.Command.execute(Command.java:93)
at org.basex.server.ClientListener.run(ClientListener.java:140)AH
Alexander Holupirek9 May 2018 08:44 #
Hi,
could you please provide some more details?
The best would be a reproducible example.
Thanks,
Alex
Quoted text · halit tiryaki · 43 lines
> On 9. May 2018, at 06:50, halit tiryaki <halit.tiryaki at gmail.com> wrote: > > Hello, > > got a serious problem here. any ideas? thanks > > Improper use? Potential bug? Your feedback is welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 9.0.2 beta > Java: Oracle Corporation, 1.8.0_151 > OS: Linux, amd64 > Stack Trace: > java.lang.ArrayIndexOutOfBoundsException: 4288 > at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:151) > at org.basex.data.Data.kind(Data.java:294) > at org.basex.query.value.node.DBNode$4.next(DBNode.java:332) > at org.basex.query.value.node.DBNode$4.next(DBNode.java:323) > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38) > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:32) > at org.basex.query.QueryContext.next(QueryContext.java:392) > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:50) > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:34) > at org.basex.query.expr.ParseExpr.item(ParseExpr.java:58) > at org.basex.query.expr.ParseExpr.atomItem(ParseExpr.java:84) > at org.basex.query.func.fn.FnConcat.item(FnConcat.java:20) > at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71) > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) > at org.basex.query.expr.path.CachedPath.nodeIter(CachedPath.java:36) > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:69) > at org.basex.query.expr.gflwor.For$1.next(For.java:107) > at org.basex.query.expr.gflwor.OrderBy$1.sort(OrderBy.java:73) > at org.basex.query.expr.gflwor.OrderBy$1.next(OrderBy.java:54) > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) > at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:87) > at org.basex.query.QueryContext.next(QueryContext.java:392) > at org.basex.query.scope.MainModule$1.next(MainModule.java:122) > at org.basex.core.cmd.AQuery.query(AQuery.java:94) > at org.basex.core.cmd.XQuery.run(XQuery.java:22) > at org.basex.core.Command.run(Command.java:257) > at org.basex.core.Command.execute(Command.java:93) > at org.basex.server.ClientListener.run(ClientListener.java:140)
HT
halit tiryaki9 May 2018 16:28 #
I'm using the C# driver. but still, it doesn't look like a high level problem. neither a query problem. thats why I didn't post some code snippets. To make sure, I just tried to execute the INSPECT command in the CLI. same exception. Also the
db:optimize fails with the same exception, although with a different stacktrace, which I will post below.
Altough the race condition point of two processes interfering is a good point, I'm currently having trouble to do anything useful with a session from which a race condition would matter. If, then it must have happened while the import was running along with the other, non-importing process.
As I mentioned in the first post, while the import process was still ongoing, the other process already started to have the exception at a very late progress of the import. After the import was done, I was still able to operate on the db as normal, but only with the session in the process which did the long-run import.
To restore the second process its session state to normal, I restarted the webapplication, which closes both sessions. After that, I'm no longer to operate on the db. Not even directly from the CLI.
Seems it is a sort of a persisted data state which the low level basex code can't handle when trying to reread it into to the memory.
I 've already googled around for the same exception in BaseX, seems it was an issue in some older versions too.
here the stacktrace when running db:optimize:
Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk at mailman.uni-konstanz.de
Version: BaseX 9.0.2 beta
Java: Oracle Corporation, 1.8.0_151
OS: Linux, amd64
Stack Trace:java.lang.ArrayIndexOutOfBoundsException: 52
at org.basex.util.hash.TokenSet.key(TokenSet.java:128)
at org.basex.data.Data.name(Data.java:388)
at org.basex.io.serial.Serializer.node(Serializer.java:414)
at org.basex.io.serial.Serializer.node(Serializer.java:158)
at org.basex.io.serial.Serializer.node(Serializer.java:345)
at org.basex.io.serial.Serializer.node(Serializer.java:158)
at org.basex.io.serial.Serializer.serialize(Serializer.java:109)
at org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll.java:200)
at org.basex.build.Builder.parse(Builder.java:77)
at org.basex.build.DiskBuilder.build(DiskBuilder.java:77)
at org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:122)
at org.basex.query.up.primitives.db.DBOptimize.apply(DBOptimize.java:124)
at org.basex.query.up.DataUpdates.apply(DataUpdates.java:175)
at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120)
at org.basex.query.up.Updates.apply(Updates.java:157)
at org.basex.query.QueryContext.iter(QueryContext.java:341)
at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90)
at org.basex.core.cmd.AQuery.query(AQuery.java:92)
at org.basex.core.cmd.XQuery.run(XQuery.java:22)
at org.basex.core.Command.run(Command.java:257)
at org.basex.core.Command.execute(Command.java:93)
at org.basex.server.ClientListener.run(ClientListener.java:140)Quoted text · Alexander Holupirek · 130 lines
On Wed, May 9, 2018 at 11:45 AM, Alexander Holupirek <alex at holupirek.de> wrote: > Hi, > > (please also respond to the list. Others might have suggestions as well > ;-) > > it still would be great if you could provide even more details. For > instance, what programming language do you use?Ideally include a Short, > Self Contained, Correct (Compilable), Example (SSCCE) [1] in order to let > people reproduce the behaviour. > > From a high level perspective, are you sure the sessions do not > interfere? Do you additionally work with BaseXGUI on the database, while > your program is importing? > > Cheers, > Alex > > > [1] http://sscce.org/ > > > On 9. May 2018, at 09:26, halit tiryaki <halit.tiryaki at gmail.com> wrote: > > > > hello, thanks for the fast response. > > > > It happened in the process of a large Import. There were two processes > with each a respective Session to BaseX. One process with the Open session > which did the Import was doing fine. The Second process however Starter > having the mentioned exception. After stopping Both processes and > restarting the app, no basex Session is now able to operate normal. > > > > The ArrayIndexOutOfBoundsException already comes in executing the > command INSPECT. > > > > The db-Info returns following: > > > > > > info db: > > Database Properties > > NAME: XXX > > SIZE: 5877 kB > > NODES: 124051 > > DOCUMENTS: 5497 > > BINARIES: 0 > > TIMESTAMP: 2018-05-09T05:20:44.000Z > > UPTODATE: false > > > > Resource Properties > > INPUTPATH: > > INPUTSIZE: 0 b > > INPUTDATE: 2018-05-09T04:32:10.539Z > > > > Indexes > > TEXTINDEX: true > > ATTRINDEX: true > > TOKENINDEX: false > > FTINDEX: false > > TEXTINCLUDE: /item_User/node()/User:Email > > ATTRINCLUDE: /item_UserImage/node()/UserImage:Id > > TOKENINCLUDE: > > FTINCLUDE: /item_XXX/node()/XXX:[german] > > LANGUAGE: English > > STEMMING: false > > CASESENS: false > > DIACRITICS: false > > STOPWORDS: > > UPDINDEX: true > > AUTOOPTIMIZE: false > > MAXCATS: 100 > > MAXLEN: 96 > > SPLITSIZE: 0 > > Alexander Holupirek <alex at holupirek.de> schrieb am Mi., 9. Mai 2018 > 08:44: > > Hi, > > > > could you please provide some more details? > > The best would be a reproducible example. > > > > Thanks, > > Alex > > > > > > > On 9. May 2018, at 06:50, halit tiryaki <halit.tiryaki at gmail.com> > wrote: > > > > > > Hello, > > > > > > got a serious problem here. any ideas? thanks > > > > > > Improper use? Potential bug? Your feedback is welcome: > > > Contact: basex-talk at mailman.uni-konstanz.de > > > Version: BaseX 9.0.2 beta > > > Java: Oracle Corporation, 1.8.0_151 > > > OS: Linux, amd64 > > > Stack Trace: > > > java.lang.ArrayIndexOutOfBoundsException: 4288 > > > at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:151) > > > at org.basex.data.Data.kind(Data.java:294) > > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:332) > > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:323) > > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38) > > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:32) > > > at org.basex.query.QueryContext.next(QueryContext.java:392) > > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:50) > > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:34) > > > at org.basex.query.expr.ParseExpr.item(ParseExpr.java:58) > > > at org.basex.query.expr.ParseExpr.atomItem(ParseExpr.java:84) > > > at org.basex.query.func.fn.FnConcat.item(FnConcat.java:20) > > > at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71) > > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) > > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) > > > at org.basex.query.expr.path.CachedPath.nodeIter(CachedPath.java:36) > > > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:69) > > > at org.basex.query.expr.gflwor.For$1.next(For.java:107) > > > at org.basex.query.expr.gflwor.OrderBy$1.sort(OrderBy.java:73) > > > at org.basex.query.expr.gflwor.OrderBy$1.next(OrderBy.java:54) > > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) > > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) > > > at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:87) > > > at org.basex.query.QueryContext.next(QueryContext.java:392) > > > at org.basex.query.scope.MainModule$1.next(MainModule.java:122) > > > at org.basex.core.cmd.AQuery.query(AQuery.java:94) > > > at org.basex.core.cmd.XQuery.run(XQuery.java:22) > > > at org.basex.core.Command.run(Command.java:257) > > > at org.basex.core.Command.execute(Command.java:93) > > > at org.basex.server.ClientListener.run(ClientListener.java:140) > > > >
CG
Christian Grün14 May 2018 16:10 #
Hi Halit,
Thanks for reporting the stack trace. The only thing that I can derive from your error message is that something has gone wrong with your database in an earlier state. Unfortunately, it’s difficult to guess what might have been the reason. Feel free to give us an update if you should manage to get the issue reproduced and reproducible for us.
Best,
Christian
Quoted text · halit tiryaki · 194 lines
On Wed, May 9, 2018 at 4:28 PM, halit tiryaki <halit.tiryaki at gmail.com> wrote: > I'm using the C# driver. but still, it doesn't look like a high level > problem. neither a query problem. thats why I didn't post some code > snippets. To make sure, I just tried to execute the INSPECT command in the > CLI. same exception. Also the db:optimize fails with the same exception, > although with a different stacktrace, which I will post below. > > Altough the race condition point of two processes interfering is a good > point, I'm currently having trouble to do anything useful with a session > from which a race condition would matter. If, then it must have happened > while the import was running along with the other, non-importing process. > > As I mentioned in the first post, while the import process was still > ongoing, the other process already started to have the exception at a very > late progress of the import. After the import was done, I was still able to > operate on the db as normal, but only with the session in the process which > did the long-run import. > > To restore the second process its session state to normal, I restarted the > webapplication, which closes both sessions. After that, I'm no longer to > operate on the db. Not even directly from the CLI. > > Seems it is a sort of a persisted data state which the low level basex code > can't handle when trying to reread it into to the memory. > > I 've already googled around for the same exception in BaseX, seems it was > an issue in some older versions too. > > > here the stacktrace when running db:optimize: > > Improper use? Potential bug? Your feedback is welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 9.0.2 beta > Java: Oracle Corporation, 1.8.0_151 > OS: Linux, amd64 > Stack Trace: > java.lang.ArrayIndexOutOfBoundsException: 52 > at org.basex.util.hash.TokenSet.key(TokenSet.java:128) > at org.basex.data.Data.name(Data.java:388) > at org.basex.io.serial.Serializer.node(Serializer.java:414) > at org.basex.io.serial.Serializer.node(Serializer.java:158) > at org.basex.io.serial.Serializer.node(Serializer.java:345) > at org.basex.io.serial.Serializer.node(Serializer.java:158) > at org.basex.io.serial.Serializer.serialize(Serializer.java:109) > at org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll.java:200) > at org.basex.build.Builder.parse(Builder.java:77) > at org.basex.build.DiskBuilder.build(DiskBuilder.java:77) > at org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:122) > at org.basex.query.up.primitives.db.DBOptimize.apply(DBOptimize.java:124) > at org.basex.query.up.DataUpdates.apply(DataUpdates.java:175) > at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120) > at org.basex.query.up.Updates.apply(Updates.java:157) > at org.basex.query.QueryContext.iter(QueryContext.java:341) > at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) > at org.basex.core.cmd.AQuery.query(AQuery.java:92) > at org.basex.core.cmd.XQuery.run(XQuery.java:22) > at org.basex.core.Command.run(Command.java:257) > at org.basex.core.Command.execute(Command.java:93) > at org.basex.server.ClientListener.run(ClientListener.java:140) > > > On Wed, May 9, 2018 at 11:45 AM, Alexander Holupirek <alex at holupirek.de> > wrote: >> >> Hi, >> >> (please also respond to the list. Others might have suggestions as well >> ;-) >> >> it still would be great if you could provide even more details. For >> instance, what programming language do you use?Ideally include a Short, Self >> Contained, Correct (Compilable), Example (SSCCE) [1] in order to let people >> reproduce the behaviour. >> >> From a high level perspective, are you sure the sessions do not interfere? >> Do you additionally work with BaseXGUI on the database, while your program >> is importing? >> >> Cheers, >> Alex >> >> >> [1] http://sscce.org/ >> >> > On 9. May 2018, at 09:26, halit tiryaki <halit.tiryaki at gmail.com> wrote: >> > >> > hello, thanks for the fast response. >> > >> > It happened in the process of a large Import. There were two processes >> > with each a respective Session to BaseX. One process with the Open session >> > which did the Import was doing fine. The Second process however Starter >> > having the mentioned exception. After stopping Both processes and restarting >> > the app, no basex Session is now able to operate normal. >> > >> > The ArrayIndexOutOfBoundsException already comes in executing the >> > command INSPECT. >> > >> > The db-Info returns following: >> > >> > >> > info db: >> > Database Properties >> > NAME: XXX >> > SIZE: 5877 kB >> > NODES: 124051 >> > DOCUMENTS: 5497 >> > BINARIES: 0 >> > TIMESTAMP: 2018-05-09T05:20:44.000Z >> > UPTODATE: false >> > >> > Resource Properties >> > INPUTPATH: >> > INPUTSIZE: 0 b >> > INPUTDATE: 2018-05-09T04:32:10.539Z >> > >> > Indexes >> > TEXTINDEX: true >> > ATTRINDEX: true >> > TOKENINDEX: false >> > FTINDEX: false >> > TEXTINCLUDE: /item_User/node()/User:Email >> > ATTRINCLUDE: /item_UserImage/node()/UserImage:Id >> > TOKENINCLUDE: >> > FTINCLUDE: /item_XXX/node()/XXX:[german] >> > LANGUAGE: English >> > STEMMING: false >> > CASESENS: false >> > DIACRITICS: false >> > STOPWORDS: >> > UPDINDEX: true >> > AUTOOPTIMIZE: false >> > MAXCATS: 100 >> > MAXLEN: 96 >> > SPLITSIZE: 0 >> >> > Alexander Holupirek <alex at holupirek.de> schrieb am Mi., 9. Mai 2018 >> > 08:44: >> > Hi, >> > >> > could you please provide some more details? >> > The best would be a reproducible example. >> > >> > Thanks, >> > Alex >> > >> > >> > > On 9. May 2018, at 06:50, halit tiryaki <halit.tiryaki at gmail.com> >> > > wrote: >> > > >> > > Hello, >> > > >> > > got a serious problem here. any ideas? thanks >> > > >> > > Improper use? Potential bug? Your feedback is welcome: >> > > Contact: basex-talk at mailman.uni-konstanz.de >> > > Version: BaseX 9.0.2 beta >> > > Java: Oracle Corporation, 1.8.0_151 >> > > OS: Linux, amd64 >> > > Stack Trace: >> > > java.lang.ArrayIndexOutOfBoundsException: 4288 >> > > at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:151) >> > > at org.basex.data.Data.kind(Data.java:294) >> > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:332) >> > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:323) >> > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38) >> > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:32) >> > > at org.basex.query.QueryContext.next(QueryContext.java:392) >> > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:50) >> > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:34) >> > > at org.basex.query.expr.ParseExpr.item(ParseExpr.java:58) >> > > at org.basex.query.expr.ParseExpr.atomItem(ParseExpr.java:84) >> > > at org.basex.query.func.fn.FnConcat.item(FnConcat.java:20) >> > > at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71) >> > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) >> > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) >> > > at org.basex.query.expr.path.CachedPath.nodeIter(CachedPath.java:36) >> > > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:69) >> > > at org.basex.query.expr.gflwor.For$1.next(For.java:107) >> > > at org.basex.query.expr.gflwor.OrderBy$1.sort(OrderBy.java:73) >> > > at org.basex.query.expr.gflwor.OrderBy$1.next(OrderBy.java:54) >> > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) >> > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) >> > > at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:87) >> > > at org.basex.query.QueryContext.next(QueryContext.java:392) >> > > at org.basex.query.scope.MainModule$1.next(MainModule.java:122) >> > > at org.basex.core.cmd.AQuery.query(AQuery.java:94) >> > > at org.basex.core.cmd.XQuery.run(XQuery.java:22) >> > > at org.basex.core.Command.run(Command.java:257) >> > > at org.basex.core.Command.execute(Command.java:93) >> > > at org.basex.server.ClientListener.run(ClientListener.java:140) >> > >> >
HT
halit tiryaki17 May 2018 22:23 #
Hi Christian,
I think the cause of the problem was the usage of AutoFlush:false in the import process. The Flush() invoke in longer periods during the process might have been corrupted the state in race conditions.
Restored the db from a backup and restarted the import with AutoFlush:true, this time I got:
Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk at mailman.uni-konstanz.de
Version: BaseX 9.0.2 beta
Java: Oracle Corporation, 1.8.0_151
OS: Linux, amd64
Stack Trace:
java.lang.RuntimeException: Key does not exist:
'86295d851a084fb2bde4181e80760099'
at org.basex.util.Util.notExpected(Util.java:61)
at
org.basex.index.value.UpdatableDiskValues.delete(UpdatableDiskValues.java:99) at org.basex.data.Data.indexDelete(Data.java:1060)
at org.basex.data.Data.delete(Data.java:663)
at org.basex.query.up.atomic.Delete.apply(Delete.java:39)
at
org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(AtomicUpdateCache.java:296) at
org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUpdateCache.java:280) at org.basex.query.up.DataUpdates.apply(DataUpdates.java:168) at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120) at org.basex.query.up.Updates.apply(Updates.java:157)
at org.basex.query.QueryContext.iter(QueryContext.java:341)
at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) at org.basex.core.cmd.AQuery.query(AQuery.java:92)
at org.basex.core.cmd.XQuery.run(XQuery.java:22)
at org.basex.core.Command.run(Command.java:257)
at org.basex.core.Command.execute(Command.java:93)
at org.basex.server.ClientListener.run(ClientListener.java:140)
Quoted text · Christian Grün · 229 lines
On Mon, May 14, 2018 at 4:10 PM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Halit, > > Thanks for reporting the stack trace. The only thing that I can derive > from your error message is that something has gone wrong with your > database in an earlier state. Unfortunately, it’s difficult to guess > what might have been the reason. Feel free to give us an update if you > should manage to get the issue reproduced and reproducible for us. > > Best, > Christian > > > > On Wed, May 9, 2018 at 4:28 PM, halit tiryaki <halit.tiryaki at gmail.com> > wrote: > > I'm using the C# driver. but still, it doesn't look like a high level > > problem. neither a query problem. thats why I didn't post some code > > snippets. To make sure, I just tried to execute the INSPECT command in > the > > CLI. same exception. Also the db:optimize fails with the same exception, > > although with a different stacktrace, which I will post below. > > > > Altough the race condition point of two processes interfering is a good > > point, I'm currently having trouble to do anything useful with a session > > from which a race condition would matter. If, then it must have happened > > while the import was running along with the other, non-importing process. > > > > As I mentioned in the first post, while the import process was still > > ongoing, the other process already started to have the exception at a > very > > late progress of the import. After the import was done, I was still able > to > > operate on the db as normal, but only with the session in the process > which > > did the long-run import. > > > > To restore the second process its session state to normal, I restarted > the > > webapplication, which closes both sessions. After that, I'm no longer to > > operate on the db. Not even directly from the CLI. > > > > Seems it is a sort of a persisted data state which the low level basex > code > > can't handle when trying to reread it into to the memory. > > > > I 've already googled around for the same exception in BaseX, seems it > was > > an issue in some older versions too. > > > > > > here the stacktrace when running db:optimize: > > > > Improper use? Potential bug? Your feedback is welcome: > > Contact: basex-talk at mailman.uni-konstanz.de > > Version: BaseX 9.0.2 beta > > Java: Oracle Corporation, 1.8.0_151 > > OS: Linux, amd64 > > Stack Trace: > > java.lang.ArrayIndexOutOfBoundsException: 52 > > at org.basex.util.hash.TokenSet.key(TokenSet.java:128) > > at org.basex.data.Data.name(Data.java:388) > > at org.basex.io.serial.Serializer.node(Serializer.java:414) > > at org.basex.io.serial.Serializer.node(Serializer.java:158) > > at org.basex.io.serial.Serializer.node(Serializer.java:345) > > at org.basex.io.serial.Serializer.node(Serializer.java:158) > > at org.basex.io.serial.Serializer.serialize(Serializer.java:109) > > at org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll.java:200) > > at org.basex.build.Builder.parse(Builder.java:77) > > at org.basex.build.DiskBuilder.build(DiskBuilder.java:77) > > at org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:122) > > at org.basex.query.up.primitives.db.DBOptimize.apply( > DBOptimize.java:124) > > at org.basex.query.up.DataUpdates.apply(DataUpdates.java:175) > > at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120) > > at org.basex.query.up.Updates.apply(Updates.java:157) > > at org.basex.query.QueryContext.iter(QueryContext.java:341) > > at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) > > at org.basex.core.cmd.AQuery.query(AQuery.java:92) > > at org.basex.core.cmd.XQuery.run(XQuery.java:22) > > at org.basex.core.Command.run(Command.java:257) > > at org.basex.core.Command.execute(Command.java:93) > > at org.basex.server.ClientListener.run(ClientListener.java:140) > > > > > > On Wed, May 9, 2018 at 11:45 AM, Alexander Holupirek <alex at holupirek.de> > > wrote: > >> > >> Hi, > >> > >> (please also respond to the list. Others might have suggestions as well > >> ;-) > >> > >> it still would be great if you could provide even more details. For > >> instance, what programming language do you use?Ideally include a Short, > Self > >> Contained, Correct (Compilable), Example (SSCCE) [1] in order to let > people > >> reproduce the behaviour. > >> > >> From a high level perspective, are you sure the sessions do not > interfere? > >> Do you additionally work with BaseXGUI on the database, while your > program > >> is importing? > >> > >> Cheers, > >> Alex > >> > >> > >> [1] http://sscce.org/ > >> > >> > On 9. May 2018, at 09:26, halit tiryaki <halit.tiryaki at gmail.com> > wrote: > >> > > >> > hello, thanks for the fast response. > >> > > >> > It happened in the process of a large Import. There were two processes > >> > with each a respective Session to BaseX. One process with the Open > session > >> > which did the Import was doing fine. The Second process however > Starter > >> > having the mentioned exception. After stopping Both processes and > restarting > >> > the app, no basex Session is now able to operate normal. > >> > > >> > The ArrayIndexOutOfBoundsException already comes in executing the > >> > command INSPECT. > >> > > >> > The db-Info returns following: > >> > > >> > > >> > info db: > >> > Database Properties > >> > NAME: XXX > >> > SIZE: 5877 kB > >> > NODES: 124051 > >> > DOCUMENTS: 5497 > >> > BINARIES: 0 > >> > TIMESTAMP: 2018-05-09T05:20:44.000Z > >> > UPTODATE: false > >> > > >> > Resource Properties > >> > INPUTPATH: > >> > INPUTSIZE: 0 b > >> > INPUTDATE: 2018-05-09T04:32:10.539Z > >> > > >> > Indexes > >> > TEXTINDEX: true > >> > ATTRINDEX: true > >> > TOKENINDEX: false > >> > FTINDEX: false > >> > TEXTINCLUDE: /item_User/node()/User:Email > >> > ATTRINCLUDE: /item_UserImage/node()/UserImage:Id > >> > TOKENINCLUDE: > >> > FTINCLUDE: /item_XXX/node()/XXX:[german] > >> > LANGUAGE: English > >> > STEMMING: false > >> > CASESENS: false > >> > DIACRITICS: false > >> > STOPWORDS: > >> > UPDINDEX: true > >> > AUTOOPTIMIZE: false > >> > MAXCATS: 100 > >> > MAXLEN: 96 > >> > SPLITSIZE: 0 > >> > >> > Alexander Holupirek <alex at holupirek.de> schrieb am Mi., 9. Mai 2018 > >> > 08:44: > >> > Hi, > >> > > >> > could you please provide some more details? > >> > The best would be a reproducible example. > >> > > >> > Thanks, > >> > Alex > >> > > >> > > >> > > On 9. May 2018, at 06:50, halit tiryaki <halit.tiryaki at gmail.com> > >> > > wrote: > >> > > > >> > > Hello, > >> > > > >> > > got a serious problem here. any ideas? thanks > >> > > > >> > > Improper use? Potential bug? Your feedback is welcome: > >> > > Contact: basex-talk at mailman.uni-konstanz.de > >> > > Version: BaseX 9.0.2 beta > >> > > Java: Oracle Corporation, 1.8.0_151 > >> > > OS: Linux, amd64 > >> > > Stack Trace: > >> > > java.lang.ArrayIndexOutOfBoundsException: 4288 > >> > > at org.basex.io.random.TableDiskAccess.read1( > TableDiskAccess.java:151) > >> > > at org.basex.data.Data.kind(Data.java:294) > >> > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:332) > >> > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:323) > >> > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38) > >> > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:32) > >> > > at org.basex.query.QueryContext.next(QueryContext.java:392) > >> > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:50) > >> > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:34) > >> > > at org.basex.query.expr.ParseExpr.item(ParseExpr.java:58) > >> > > at org.basex.query.expr.ParseExpr.atomItem(ParseExpr.java:84) > >> > > at org.basex.query.func.fn.FnConcat.item(FnConcat.java:20) > >> > > at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71) > >> > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) > >> > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) > >> > > at org.basex.query.expr.path.CachedPath.nodeIter( > CachedPath.java:36) > >> > > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:69) > >> > > at org.basex.query.expr.gflwor.For$1.next(For.java:107) > >> > > at org.basex.query.expr.gflwor.OrderBy$1.sort(OrderBy.java:73) > >> > > at org.basex.query.expr.gflwor.OrderBy$1.next(OrderBy.java:54) > >> > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) > >> > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) > >> > > at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:87) > >> > > at org.basex.query.QueryContext.next(QueryContext.java:392) > >> > > at org.basex.query.scope.MainModule$1.next(MainModule.java:122) > >> > > at org.basex.core.cmd.AQuery.query(AQuery.java:94) > >> > > at org.basex.core.cmd.XQuery.run(XQuery.java:22) > >> > > at org.basex.core.Command.run(Command.java:257) > >> > > at org.basex.core.Command.execute(Command.java:93) > >> > > at org.basex.server.ClientListener.run(ClientListener.java:140) > >> > > >> > > >
CG
Christian Grün17 May 2018 22:28 #
Hi Halit,
Some more questions:
Quoted text · halit tiryaki · 1 line
> The Flush() invoke in longer periods during the process might have been
corrupted the state in race conditions.
How did you invoke flush? Did you use the client/server architecture?
Quoted text · halit tiryaki · 3 lines
Restored the db from a backup and restarted the import with AutoFlush:true, > this time I got: >
The stack trace seems to indicate that you were deleting data. Is it part of your import step?
Best,
Christian
Quoted text · halit tiryaki · 267 lines
> > > Improper use? Potential bug? Your feedback is welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 9.0.2 beta > Java: Oracle Corporation, 1.8.0_151 > OS: Linux, amd64 > Stack Trace: > java.lang.RuntimeException: Key does not exist: > '86295d851a084fb2bde4181e80760099' > at org.basex.util.Util.notExpected(Util.java:61) > at > org.basex.index.value.UpdatableDiskValues.delete(UpdatableDiskValues.java:99) > at org.basex.data.Data.indexDelete(Data.java:1060) > at org.basex.data.Data.delete(Data.java:663) > at org.basex.query.up.atomic.Delete.apply(Delete.java:39) > at > org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(AtomicUpdateCache.java:296) > at > org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUpdateCache.java:280) > at org.basex.query.up.DataUpdates.apply(DataUpdates.java:168) > at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120) > at org.basex.query.up.Updates.apply(Updates.java:157) > at org.basex.query.QueryContext.iter(QueryContext.java:341) > at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) > at org.basex.core.cmd.AQuery.query(AQuery.java:92) > at org.basex.core.cmd.XQuery.run(XQuery.java:22) > at org.basex.core.Command.run(Command.java:257) > at org.basex.core.Command.execute(Command.java:93) > at org.basex.server.ClientListener.run(ClientListener.java:140) > > On Mon, May 14, 2018 at 4:10 PM, Christian Grün <christian.gruen at gmail.com > > wrote: > >> Hi Halit, >> >> Thanks for reporting the stack trace. The only thing that I can derive >> from your error message is that something has gone wrong with your >> database in an earlier state. Unfortunately, it’s difficult to guess >> what might have been the reason. Feel free to give us an update if you >> should manage to get the issue reproduced and reproducible for us. >> >> Best, >> Christian >> >> >> >> On Wed, May 9, 2018 at 4:28 PM, halit tiryaki <halit.tiryaki at gmail.com> >> wrote: >> > I'm using the C# driver. but still, it doesn't look like a high level >> > problem. neither a query problem. thats why I didn't post some code >> > snippets. To make sure, I just tried to execute the INSPECT command in >> the >> > CLI. same exception. Also the db:optimize fails with the same exception, >> > although with a different stacktrace, which I will post below. >> > >> > Altough the race condition point of two processes interfering is a good >> > point, I'm currently having trouble to do anything useful with a session >> > from which a race condition would matter. If, then it must have happened >> > while the import was running along with the other, non-importing >> process. >> > >> > As I mentioned in the first post, while the import process was still >> > ongoing, the other process already started to have the exception at a >> very >> > late progress of the import. After the import was done, I was still >> able to >> > operate on the db as normal, but only with the session in the process >> which >> > did the long-run import. >> > >> > To restore the second process its session state to normal, I restarted >> the >> > webapplication, which closes both sessions. After that, I'm no longer to >> > operate on the db. Not even directly from the CLI. >> > >> > Seems it is a sort of a persisted data state which the low level basex >> code >> > can't handle when trying to reread it into to the memory. >> > >> > I 've already googled around for the same exception in BaseX, seems it >> was >> > an issue in some older versions too. >> > >> > >> > here the stacktrace when running db:optimize: >> > >> > Improper use? Potential bug? Your feedback is welcome: >> > Contact: basex-talk at mailman.uni-konstanz.de >> > Version: BaseX 9.0.2 beta >> > Java: Oracle Corporation, 1.8.0_151 >> > OS: Linux, amd64 >> > Stack Trace: >> > java.lang.ArrayIndexOutOfBoundsException: 52 >> > at org.basex.util.hash.TokenSet.key(TokenSet.java:128) >> > at org.basex.data.Data.name(Data.java:388) >> > at org.basex.io.serial.Serializer.node(Serializer.java:414) >> > at org.basex.io.serial.Serializer.node(Serializer.java:158) >> > at org.basex.io.serial.Serializer.node(Serializer.java:345) >> > at org.basex.io.serial.Serializer.node(Serializer.java:158) >> > at org.basex.io.serial.Serializer.serialize(Serializer.java:109) >> > at org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll.java:200) >> > at org.basex.build.Builder.parse(Builder.java:77) >> > at org.basex.build.DiskBuilder.build(DiskBuilder.java:77) >> > at org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:122) >> > at >> org.basex.query.up.primitives.db.DBOptimize.apply(DBOptimize.java:124) >> > at org.basex.query.up.DataUpdates.apply(DataUpdates.java:175) >> > at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120) >> > at org.basex.query.up.Updates.apply(Updates.java:157) >> > at org.basex.query.QueryContext.iter(QueryContext.java:341) >> > at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) >> > at org.basex.core.cmd.AQuery.query(AQuery.java:92) >> > at org.basex.core.cmd.XQuery.run(XQuery.java:22) >> > at org.basex.core.Command.run(Command.java:257) >> > at org.basex.core.Command.execute(Command.java:93) >> > at org.basex.server.ClientListener.run(ClientListener.java:140) >> > >> > >> > On Wed, May 9, 2018 at 11:45 AM, Alexander Holupirek <alex at holupirek.de >> > >> > wrote: >> >> >> >> Hi, >> >> >> >> (please also respond to the list. Others might have suggestions as >> well >> >> ;-) >> >> >> >> it still would be great if you could provide even more details. For >> >> instance, what programming language do you use?Ideally include a >> Short, Self >> >> Contained, Correct (Compilable), Example (SSCCE) [1] in order to let >> people >> >> reproduce the behaviour. >> >> >> >> From a high level perspective, are you sure the sessions do not >> interfere? >> >> Do you additionally work with BaseXGUI on the database, while your >> program >> >> is importing? >> >> >> >> Cheers, >> >> Alex >> >> >> >> >> >> [1] http://sscce.org/ >> >> >> >> > On 9. May 2018, at 09:26, halit tiryaki <halit.tiryaki at gmail.com> >> wrote: >> >> > >> >> > hello, thanks for the fast response. >> >> > >> >> > It happened in the process of a large Import. There were two >> processes >> >> > with each a respective Session to BaseX. One process with the Open >> session >> >> > which did the Import was doing fine. The Second process however >> Starter >> >> > having the mentioned exception. After stopping Both processes and >> restarting >> >> > the app, no basex Session is now able to operate normal. >> >> > >> >> > The ArrayIndexOutOfBoundsException already comes in executing the >> >> > command INSPECT. >> >> > >> >> > The db-Info returns following: >> >> > >> >> > >> >> > info db: >> >> > Database Properties >> >> > NAME: XXX >> >> > SIZE: 5877 kB >> >> > NODES: 124051 >> >> > DOCUMENTS: 5497 >> >> > BINARIES: 0 >> >> > TIMESTAMP: 2018-05-09T05:20:44.000Z >> >> > UPTODATE: false >> >> > >> >> > Resource Properties >> >> > INPUTPATH: >> >> > INPUTSIZE: 0 b >> >> > INPUTDATE: 2018-05-09T04:32:10.539Z >> >> > >> >> > Indexes >> >> > TEXTINDEX: true >> >> > ATTRINDEX: true >> >> > TOKENINDEX: false >> >> > FTINDEX: false >> >> > TEXTINCLUDE: /item_User/node()/User:Email >> >> > ATTRINCLUDE: /item_UserImage/node()/UserImage:Id >> >> > TOKENINCLUDE: >> >> > FTINCLUDE: /item_XXX/node()/XXX:[german] >> >> > LANGUAGE: English >> >> > STEMMING: false >> >> > CASESENS: false >> >> > DIACRITICS: false >> >> > STOPWORDS: >> >> > UPDINDEX: true >> >> > AUTOOPTIMIZE: false >> >> > MAXCATS: 100 >> >> > MAXLEN: 96 >> >> > SPLITSIZE: 0 >> >> >> >> > Alexander Holupirek <alex at holupirek.de> schrieb am Mi., 9. Mai 2018 >> >> > 08:44: >> >> > Hi, >> >> > >> >> > could you please provide some more details? >> >> > The best would be a reproducible example. >> >> > >> >> > Thanks, >> >> > Alex >> >> > >> >> > >> >> > > On 9. May 2018, at 06:50, halit tiryaki <halit.tiryaki at gmail.com> >> >> > > wrote: >> >> > > >> >> > > Hello, >> >> > > >> >> > > got a serious problem here. any ideas? thanks >> >> > > >> >> > > Improper use? Potential bug? Your feedback is welcome: >> >> > > Contact: basex-talk at mailman.uni-konstanz.de >> >> > > Version: BaseX 9.0.2 beta >> >> > > Java: Oracle Corporation, 1.8.0_151 >> >> > > OS: Linux, amd64 >> >> > > Stack Trace: >> >> > > java.lang.ArrayIndexOutOfBoundsException: 4288 >> >> > > at >> org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:151) >> >> > > at org.basex.data.Data.kind(Data.java:294) >> >> > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:332) >> >> > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:323) >> >> > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38) >> >> > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:32) >> >> > > at org.basex.query.QueryContext.next(QueryContext.java:392) >> >> > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:50) >> >> > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:34) >> >> > > at org.basex.query.expr.ParseExpr.item(ParseExpr.java:58) >> >> > > at org.basex.query.expr.ParseExpr.atomItem(ParseExpr.java:84) >> >> > > at org.basex.query.func.fn.FnConcat.item(FnConcat.java:20) >> >> > > at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71) >> >> > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) >> >> > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) >> >> > > at >> org.basex.query.expr.path.CachedPath.nodeIter(CachedPath.java:36) >> >> > > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:69) >> >> > > at org.basex.query.expr.gflwor.For$1.next(For.java:107) >> >> > > at org.basex.query.expr.gflwor.OrderBy$1.sort(OrderBy.java:73) >> >> > > at org.basex.query.expr.gflwor.OrderBy$1.next(OrderBy.java:54) >> >> > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) >> >> > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) >> >> > > at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:87) >> >> > > at org.basex.query.QueryContext.next(QueryContext.java:392) >> >> > > at org.basex.query.scope.MainModule$1.next(MainModule.java:122) >> >> > > at org.basex.core.cmd.AQuery.query(AQuery.java:94) >> >> > > at org.basex.core.cmd.XQuery.run(XQuery.java:22) >> >> > > at org.basex.core.Command.run(Command.java:257) >> >> > > at org.basex.core.Command.execute(Command.java:93) >> >> > > at org.basex.server.ClientListener.run(ClientListener.java:140) >> >> > >> >> >> > >> > >
HT
halit tiryaki17 May 2018 23:15 #
Quoted text · Christian Grün · 1 line
> How did you invoke flush? Did you use the client/server architecture?
I'm using the C# client.
Simply executing the commands on the session:
- SET
AUTOFLUSH true
- FLUSH (after ~200 docs of insert)Quoted text · Christian Grün · 1 line
> The stack trace seems to indicate that you were deleting data. Is it part
of your import step?
yes, outdated documents are dropped in the import process
Thanks
Halit
Quoted text · Christian Grün · 295 lines
On Thu, May 17, 2018 at 10:28 PM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Halit, > > > Some more questions: > > > The Flush() invoke in longer periods during the process might have been > corrupted the state in race conditions. > > How did you invoke flush? Did you use the client/server architecture? > > Restored the db from a backup and restarted the import with >> AutoFlush:true, this time I got: >> > > The stack trace seems to indicate that you were deleting data. Is it part > of your import step? > > Best, > Christian > > > >> >> >> Improper use? Potential bug? Your feedback is welcome: >> Contact: basex-talk at mailman.uni-konstanz.de >> Version: BaseX 9.0.2 beta >> Java: Oracle Corporation, 1.8.0_151 >> OS: Linux, amd64 >> Stack Trace: >> java.lang.RuntimeException: Key does not exist: ' >> 86295d851a084fb2bde4181e80760099' >> at org.basex.util.Util.notExpected(Util.java:61) >> at org.basex.index.value.UpdatableDiskValues.delete( >> UpdatableDiskValues.java:99) >> at org.basex.data.Data.indexDelete(Data.java:1060) >> at org.basex.data.Data.delete(Data.java:663) >> at org.basex.query.up.atomic.Delete.apply(Delete.java:39) >> at org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates( >> AtomicUpdateCache.java:296) >> at org.basex.query.up.atomic.AtomicUpdateCache.execute( >> AtomicUpdateCache.java:280) >> at org.basex.query.up.DataUpdates.apply(DataUpdates.java:168) >> at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120) >> at org.basex.query.up.Updates.apply(Updates.java:157) >> at org.basex.query.QueryContext.iter(QueryContext.java:341) >> at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) >> at org.basex.core.cmd.AQuery.query(AQuery.java:92) >> at org.basex.core.cmd.XQuery.run(XQuery.java:22) >> at org.basex.core.Command.run(Command.java:257) >> at org.basex.core.Command.execute(Command.java:93) >> at org.basex.server.ClientListener.run(ClientListener.java:140) >> >> On Mon, May 14, 2018 at 4:10 PM, Christian Grün < >> christian.gruen at gmail.com> wrote: >> >>> Hi Halit, >>> >>> Thanks for reporting the stack trace. The only thing that I can derive >>> from your error message is that something has gone wrong with your >>> database in an earlier state. Unfortunately, it’s difficult to guess >>> what might have been the reason. Feel free to give us an update if you >>> should manage to get the issue reproduced and reproducible for us. >>> >>> Best, >>> Christian >>> >>> >>> >>> On Wed, May 9, 2018 at 4:28 PM, halit tiryaki <halit.tiryaki at gmail.com> >>> wrote: >>> > I'm using the C# driver. but still, it doesn't look like a high level >>> > problem. neither a query problem. thats why I didn't post some code >>> > snippets. To make sure, I just tried to execute the INSPECT command in >>> the >>> > CLI. same exception. Also the db:optimize fails with the same >>> exception, >>> > although with a different stacktrace, which I will post below. >>> > >>> > Altough the race condition point of two processes interfering is a good >>> > point, I'm currently having trouble to do anything useful with a >>> session >>> > from which a race condition would matter. If, then it must have >>> happened >>> > while the import was running along with the other, non-importing >>> process. >>> > >>> > As I mentioned in the first post, while the import process was still >>> > ongoing, the other process already started to have the exception at a >>> very >>> > late progress of the import. After the import was done, I was still >>> able to >>> > operate on the db as normal, but only with the session in the process >>> which >>> > did the long-run import. >>> > >>> > To restore the second process its session state to normal, I restarted >>> the >>> > webapplication, which closes both sessions. After that, I'm no longer >>> to >>> > operate on the db. Not even directly from the CLI. >>> > >>> > Seems it is a sort of a persisted data state which the low level basex >>> code >>> > can't handle when trying to reread it into to the memory. >>> > >>> > I 've already googled around for the same exception in BaseX, seems it >>> was >>> > an issue in some older versions too. >>> > >>> > >>> > here the stacktrace when running db:optimize: >>> > >>> > Improper use? Potential bug? Your feedback is welcome: >>> > Contact: basex-talk at mailman.uni-konstanz.de >>> > Version: BaseX 9.0.2 beta >>> > Java: Oracle Corporation, 1.8.0_151 >>> > OS: Linux, amd64 >>> > Stack Trace: >>> > java.lang.ArrayIndexOutOfBoundsException: 52 >>> > at org.basex.util.hash.TokenSet.key(TokenSet.java:128) >>> > at org.basex.data.Data.name(Data.java:388) >>> > at org.basex.io.serial.Serializer.node(Serializer.java:414) >>> > at org.basex.io.serial.Serializer.node(Serializer.java:158) >>> > at org.basex.io.serial.Serializer.node(Serializer.java:345) >>> > at org.basex.io.serial.Serializer.node(Serializer.java:158) >>> > at org.basex.io.serial.Serializer.serialize(Serializer.java:109) >>> > at org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll.java:200) >>> > at org.basex.build.Builder.parse(Builder.java:77) >>> > at org.basex.build.DiskBuilder.build(DiskBuilder.java:77) >>> > at org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:122) >>> > at org.basex.query.up.primitives.db.DBOptimize.apply( >>> DBOptimize.java:124) >>> > at org.basex.query.up.DataUpdates.apply(DataUpdates.java:175) >>> > at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120) >>> > at org.basex.query.up.Updates.apply(Updates.java:157) >>> > at org.basex.query.QueryContext.iter(QueryContext.java:341) >>> > at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) >>> > at org.basex.core.cmd.AQuery.query(AQuery.java:92) >>> > at org.basex.core.cmd.XQuery.run(XQuery.java:22) >>> > at org.basex.core.Command.run(Command.java:257) >>> > at org.basex.core.Command.execute(Command.java:93) >>> > at org.basex.server.ClientListener.run(ClientListener.java:140) >>> > >>> > >>> > On Wed, May 9, 2018 at 11:45 AM, Alexander Holupirek < >>> alex at holupirek.de> >>> > wrote: >>> >> >>> >> Hi, >>> >> >>> >> (please also respond to the list. Others might have suggestions as >>> well >>> >> ;-) >>> >> >>> >> it still would be great if you could provide even more details. For >>> >> instance, what programming language do you use?Ideally include a >>> Short, Self >>> >> Contained, Correct (Compilable), Example (SSCCE) [1] in order to let >>> people >>> >> reproduce the behaviour. >>> >> >>> >> From a high level perspective, are you sure the sessions do not >>> interfere? >>> >> Do you additionally work with BaseXGUI on the database, while your >>> program >>> >> is importing? >>> >> >>> >> Cheers, >>> >> Alex >>> >> >>> >> >>> >> [1] http://sscce.org/ >>> >> >>> >> > On 9. May 2018, at 09:26, halit tiryaki <halit.tiryaki at gmail.com> >>> wrote: >>> >> > >>> >> > hello, thanks for the fast response. >>> >> > >>> >> > It happened in the process of a large Import. There were two >>> processes >>> >> > with each a respective Session to BaseX. One process with the Open >>> session >>> >> > which did the Import was doing fine. The Second process however >>> Starter >>> >> > having the mentioned exception. After stopping Both processes and >>> restarting >>> >> > the app, no basex Session is now able to operate normal. >>> >> > >>> >> > The ArrayIndexOutOfBoundsException already comes in executing the >>> >> > command INSPECT. >>> >> > >>> >> > The db-Info returns following: >>> >> > >>> >> > >>> >> > info db: >>> >> > Database Properties >>> >> > NAME: XXX >>> >> > SIZE: 5877 kB >>> >> > NODES: 124051 >>> >> > DOCUMENTS: 5497 >>> >> > BINARIES: 0 >>> >> > TIMESTAMP: 2018-05-09T05:20:44.000Z >>> >> > UPTODATE: false >>> >> > >>> >> > Resource Properties >>> >> > INPUTPATH: >>> >> > INPUTSIZE: 0 b >>> >> > INPUTDATE: 2018-05-09T04:32:10.539Z >>> >> > >>> >> > Indexes >>> >> > TEXTINDEX: true >>> >> > ATTRINDEX: true >>> >> > TOKENINDEX: false >>> >> > FTINDEX: false >>> >> > TEXTINCLUDE: /item_User/node()/User:Email >>> >> > ATTRINCLUDE: /item_UserImage/node()/UserImage:Id >>> >> > TOKENINCLUDE: >>> >> > FTINCLUDE: /item_XXX/node()/XXX:[german] >>> >> > LANGUAGE: English >>> >> > STEMMING: false >>> >> > CASESENS: false >>> >> > DIACRITICS: false >>> >> > STOPWORDS: >>> >> > UPDINDEX: true >>> >> > AUTOOPTIMIZE: false >>> >> > MAXCATS: 100 >>> >> > MAXLEN: 96 >>> >> > SPLITSIZE: 0 >>> >> >>> >> > Alexander Holupirek <alex at holupirek.de> schrieb am Mi., 9. Mai 2018 >>> >> > 08:44: >>> >> > Hi, >>> >> > >>> >> > could you please provide some more details? >>> >> > The best would be a reproducible example. >>> >> > >>> >> > Thanks, >>> >> > Alex >>> >> > >>> >> > >>> >> > > On 9. May 2018, at 06:50, halit tiryaki <halit.tiryaki at gmail.com> >>> >> > > wrote: >>> >> > > >>> >> > > Hello, >>> >> > > >>> >> > > got a serious problem here. any ideas? thanks >>> >> > > >>> >> > > Improper use? Potential bug? Your feedback is welcome: >>> >> > > Contact: basex-talk at mailman.uni-konstanz.de >>> >> > > Version: BaseX 9.0.2 beta >>> >> > > Java: Oracle Corporation, 1.8.0_151 >>> >> > > OS: Linux, amd64 >>> >> > > Stack Trace: >>> >> > > java.lang.ArrayIndexOutOfBoundsException: 4288 >>> >> > > at org.basex.io.random.TableDiskAccess.read1( >>> TableDiskAccess.java:151) >>> >> > > at org.basex.data.Data.kind(Data.java:294) >>> >> > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:332) >>> >> > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:323) >>> >> > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38) >>> >> > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:32) >>> >> > > at org.basex.query.QueryContext.next(QueryContext.java:392) >>> >> > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:50) >>> >> > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:34) >>> >> > > at org.basex.query.expr.ParseExpr.item(ParseExpr.java:58) >>> >> > > at org.basex.query.expr.ParseExpr.atomItem(ParseExpr.java:84) >>> >> > > at org.basex.query.func.fn.FnConcat.item(FnConcat.java:20) >>> >> > > at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71) >>> >> > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) >>> >> > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) >>> >> > > at org.basex.query.expr.path.CachedPath.nodeIter( >>> CachedPath.java:36) >>> >> > > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:69) >>> >> > > at org.basex.query.expr.gflwor.For$1.next(For.java:107) >>> >> > > at org.basex.query.expr.gflwor.OrderBy$1.sort(OrderBy.java:73) >>> >> > > at org.basex.query.expr.gflwor.OrderBy$1.next(OrderBy.java:54) >>> >> > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) >>> >> > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) >>> >> > > at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:87) >>> >> > > at org.basex.query.QueryContext.next(QueryContext.java:392) >>> >> > > at org.basex.query.scope.MainModule$1.next(MainModule.java:122) >>> >> > > at org.basex.core.cmd.AQuery.query(AQuery.java:94) >>> >> > > at org.basex.core.cmd.XQuery.run(XQuery.java:22) >>> >> > > at org.basex.core.Command.run(Command.java:257) >>> >> > > at org.basex.core.Command.execute(Command.java:93) >>> >> > > at org.basex.server.ClientListener.run(ClientListener.java:140) >>> >> > >>> >> >>> > >>> >> >>
HT
halit tiryaki17 May 2018 23:16 #
correction:
ofcourse executing the command: SET
AUTOFLUSH falseQuoted text · halit tiryaki · 317 lines
On Thu, May 17, 2018 at 11:15 PM, halit tiryaki <halit.tiryaki at gmail.com> wrote: > > How did you invoke flush? Did you use the client/server architecture? > I'm using the C# client. > Simply executing the commands on the session: > - SET AUTOFLUSH true > - FLUSH (after ~200 docs of insert) > > > > The stack trace seems to indicate that you were deleting data. Is it > part of your import step? > yes, outdated documents are dropped in the import process > > Thanks > Halit > > On Thu, May 17, 2018 at 10:28 PM, Christian Grün < > christian.gruen at gmail.com> wrote: > >> Hi Halit, >> >> >> Some more questions: >> >> > The Flush() invoke in longer periods during the process might have >> been corrupted the state in race conditions. >> >> How did you invoke flush? Did you use the client/server architecture? >> >> Restored the db from a backup and restarted the import with >>> AutoFlush:true, this time I got: >>> >> >> The stack trace seems to indicate that you were deleting data. Is it part >> of your import step? >> >> Best, >> Christian >> >> >> >>> >>> >>> Improper use? Potential bug? Your feedback is welcome: >>> Contact: basex-talk at mailman.uni-konstanz.de >>> Version: BaseX 9.0.2 beta >>> Java: Oracle Corporation, 1.8.0_151 >>> OS: Linux, amd64 >>> Stack Trace: >>> java.lang.RuntimeException: Key does not exist: >>> '86295d851a084fb2bde4181e80760099' >>> at org.basex.util.Util.notExpected(Util.java:61) >>> at org.basex.index.value.UpdatableDiskValues.delete(UpdatableDi >>> skValues.java:99) >>> at org.basex.data.Data.indexDelete(Data.java:1060) >>> at org.basex.data.Data.delete(Data.java:663) >>> at org.basex.query.up.atomic.Delete.apply(Delete.java:39) >>> at org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(Ato >>> micUpdateCache.java:296) >>> at org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUp >>> dateCache.java:280) >>> at org.basex.query.up.DataUpdates.apply(DataUpdates.java:168) >>> at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120) >>> at org.basex.query.up.Updates.apply(Updates.java:157) >>> at org.basex.query.QueryContext.iter(QueryContext.java:341) >>> at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) >>> at org.basex.core.cmd.AQuery.query(AQuery.java:92) >>> at org.basex.core.cmd.XQuery.run(XQuery.java:22) >>> at org.basex.core.Command.run(Command.java:257) >>> at org.basex.core.Command.execute(Command.java:93) >>> at org.basex.server.ClientListener.run(ClientListener.java:140) >>> >>> On Mon, May 14, 2018 at 4:10 PM, Christian Grün < >>> christian.gruen at gmail.com> wrote: >>> >>>> Hi Halit, >>>> >>>> Thanks for reporting the stack trace. The only thing that I can derive >>>> from your error message is that something has gone wrong with your >>>> database in an earlier state. Unfortunately, it’s difficult to guess >>>> what might have been the reason. Feel free to give us an update if you >>>> should manage to get the issue reproduced and reproducible for us. >>>> >>>> Best, >>>> Christian >>>> >>>> >>>> >>>> On Wed, May 9, 2018 at 4:28 PM, halit tiryaki <halit.tiryaki at gmail.com> >>>> wrote: >>>> > I'm using the C# driver. but still, it doesn't look like a high level >>>> > problem. neither a query problem. thats why I didn't post some code >>>> > snippets. To make sure, I just tried to execute the INSPECT command >>>> in the >>>> > CLI. same exception. Also the db:optimize fails with the same >>>> exception, >>>> > although with a different stacktrace, which I will post below. >>>> > >>>> > Altough the race condition point of two processes interfering is a >>>> good >>>> > point, I'm currently having trouble to do anything useful with a >>>> session >>>> > from which a race condition would matter. If, then it must have >>>> happened >>>> > while the import was running along with the other, non-importing >>>> process. >>>> > >>>> > As I mentioned in the first post, while the import process was still >>>> > ongoing, the other process already started to have the exception at a >>>> very >>>> > late progress of the import. After the import was done, I was still >>>> able to >>>> > operate on the db as normal, but only with the session in the process >>>> which >>>> > did the long-run import. >>>> > >>>> > To restore the second process its session state to normal, I >>>> restarted the >>>> > webapplication, which closes both sessions. After that, I'm no longer >>>> to >>>> > operate on the db. Not even directly from the CLI. >>>> > >>>> > Seems it is a sort of a persisted data state which the low level >>>> basex code >>>> > can't handle when trying to reread it into to the memory. >>>> > >>>> > I 've already googled around for the same exception in BaseX, seems >>>> it was >>>> > an issue in some older versions too. >>>> > >>>> > >>>> > here the stacktrace when running db:optimize: >>>> > >>>> > Improper use? Potential bug? Your feedback is welcome: >>>> > Contact: basex-talk at mailman.uni-konstanz.de >>>> > Version: BaseX 9.0.2 beta >>>> > Java: Oracle Corporation, 1.8.0_151 >>>> > OS: Linux, amd64 >>>> > Stack Trace: >>>> > java.lang.ArrayIndexOutOfBoundsException: 52 >>>> > at org.basex.util.hash.TokenSet.key(TokenSet.java:128) >>>> > at org.basex.data.Data.name(Data.java:388) >>>> > at org.basex.io.serial.Serializer.node(Serializer.java:414) >>>> > at org.basex.io.serial.Serializer.node(Serializer.java:158) >>>> > at org.basex.io.serial.Serializer.node(Serializer.java:345) >>>> > at org.basex.io.serial.Serializer.node(Serializer.java:158) >>>> > at org.basex.io.serial.Serializer.serialize(Serializer.java:109) >>>> > at org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll. >>>> java:200) >>>> > at org.basex.build.Builder.parse(Builder.java:77) >>>> > at org.basex.build.DiskBuilder.build(DiskBuilder.java:77) >>>> > at org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:122) >>>> > at org.basex.query.up.primitives.db.DBOptimize.apply(DBOptimize >>>> .java:124) >>>> > at org.basex.query.up.DataUpdates.apply(DataUpdates.java:175) >>>> > at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120) >>>> > at org.basex.query.up.Updates.apply(Updates.java:157) >>>> > at org.basex.query.QueryContext.iter(QueryContext.java:341) >>>> > at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) >>>> > at org.basex.core.cmd.AQuery.query(AQuery.java:92) >>>> > at org.basex.core.cmd.XQuery.run(XQuery.java:22) >>>> > at org.basex.core.Command.run(Command.java:257) >>>> > at org.basex.core.Command.execute(Command.java:93) >>>> > at org.basex.server.ClientListener.run(ClientListener.java:140) >>>> > >>>> > >>>> > On Wed, May 9, 2018 at 11:45 AM, Alexander Holupirek < >>>> alex at holupirek.de> >>>> > wrote: >>>> >> >>>> >> Hi, >>>> >> >>>> >> (please also respond to the list. Others might have suggestions as >>>> well >>>> >> ;-) >>>> >> >>>> >> it still would be great if you could provide even more details. For >>>> >> instance, what programming language do you use?Ideally include a >>>> Short, Self >>>> >> Contained, Correct (Compilable), Example (SSCCE) [1] in order to let >>>> people >>>> >> reproduce the behaviour. >>>> >> >>>> >> From a high level perspective, are you sure the sessions do not >>>> interfere? >>>> >> Do you additionally work with BaseXGUI on the database, while your >>>> program >>>> >> is importing? >>>> >> >>>> >> Cheers, >>>> >> Alex >>>> >> >>>> >> >>>> >> [1] http://sscce.org/ >>>> >> >>>> >> > On 9. May 2018, at 09:26, halit tiryaki <halit.tiryaki at gmail.com> >>>> wrote: >>>> >> > >>>> >> > hello, thanks for the fast response. >>>> >> > >>>> >> > It happened in the process of a large Import. There were two >>>> processes >>>> >> > with each a respective Session to BaseX. One process with the Open >>>> session >>>> >> > which did the Import was doing fine. The Second process however >>>> Starter >>>> >> > having the mentioned exception. After stopping Both processes and >>>> restarting >>>> >> > the app, no basex Session is now able to operate normal. >>>> >> > >>>> >> > The ArrayIndexOutOfBoundsException already comes in executing the >>>> >> > command INSPECT. >>>> >> > >>>> >> > The db-Info returns following: >>>> >> > >>>> >> > >>>> >> > info db: >>>> >> > Database Properties >>>> >> > NAME: XXX >>>> >> > SIZE: 5877 kB >>>> >> > NODES: 124051 >>>> >> > DOCUMENTS: 5497 >>>> >> > BINARIES: 0 >>>> >> > TIMESTAMP: 2018-05-09T05:20:44.000Z >>>> >> > UPTODATE: false >>>> >> > >>>> >> > Resource Properties >>>> >> > INPUTPATH: >>>> >> > INPUTSIZE: 0 b >>>> >> > INPUTDATE: 2018-05-09T04:32:10.539Z >>>> >> > >>>> >> > Indexes >>>> >> > TEXTINDEX: true >>>> >> > ATTRINDEX: true >>>> >> > TOKENINDEX: false >>>> >> > FTINDEX: false >>>> >> > TEXTINCLUDE: /item_User/node()/User:Email >>>> >> > ATTRINCLUDE: /item_UserImage/node()/UserImage:Id >>>> >> > TOKENINCLUDE: >>>> >> > FTINCLUDE: /item_XXX/node()/XXX:[german] >>>> >> > LANGUAGE: English >>>> >> > STEMMING: false >>>> >> > CASESENS: false >>>> >> > DIACRITICS: false >>>> >> > STOPWORDS: >>>> >> > UPDINDEX: true >>>> >> > AUTOOPTIMIZE: false >>>> >> > MAXCATS: 100 >>>> >> > MAXLEN: 96 >>>> >> > SPLITSIZE: 0 >>>> >> >>>> >> > Alexander Holupirek <alex at holupirek.de> schrieb am Mi., 9. Mai >>>> 2018 >>>> >> > 08:44: >>>> >> > Hi, >>>> >> > >>>> >> > could you please provide some more details? >>>> >> > The best would be a reproducible example. >>>> >> > >>>> >> > Thanks, >>>> >> > Alex >>>> >> > >>>> >> > >>>> >> > > On 9. May 2018, at 06:50, halit tiryaki <halit.tiryaki at gmail.com >>>> > >>>> >> > > wrote: >>>> >> > > >>>> >> > > Hello, >>>> >> > > >>>> >> > > got a serious problem here. any ideas? thanks >>>> >> > > >>>> >> > > Improper use? Potential bug? Your feedback is welcome: >>>> >> > > Contact: basex-talk at mailman.uni-konstanz.de >>>> >> > > Version: BaseX 9.0.2 beta >>>> >> > > Java: Oracle Corporation, 1.8.0_151 >>>> >> > > OS: Linux, amd64 >>>> >> > > Stack Trace: >>>> >> > > java.lang.ArrayIndexOutOfBoundsException: 4288 >>>> >> > > at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess. >>>> java:151) >>>> >> > > at org.basex.data.Data.kind(Data.java:294) >>>> >> > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:332) >>>> >> > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:323) >>>> >> > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38) >>>> >> > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:32) >>>> >> > > at org.basex.query.QueryContext.next(QueryContext.java:392) >>>> >> > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:50) >>>> >> > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:34) >>>> >> > > at org.basex.query.expr.ParseExpr.item(ParseExpr.java:58) >>>> >> > > at org.basex.query.expr.ParseExpr.atomItem(ParseExpr.java:84) >>>> >> > > at org.basex.query.func.fn.FnConcat.item(FnConcat.java:20) >>>> >> > > at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71) >>>> >> > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) >>>> >> > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) >>>> >> > > at org.basex.query.expr.path.CachedPath.nodeIter(CachedPath. >>>> java:36) >>>> >> > > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:69) >>>> >> > > at org.basex.query.expr.gflwor.For$1.next(For.java:107) >>>> >> > > at org.basex.query.expr.gflwor.OrderBy$1.sort(OrderBy.java:73) >>>> >> > > at org.basex.query.expr.gflwor.OrderBy$1.next(OrderBy.java:54) >>>> >> > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) >>>> >> > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) >>>> >> > > at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:87) >>>> >> > > at org.basex.query.QueryContext.next(QueryContext.java:392) >>>> >> > > at org.basex.query.scope.MainModule$1.next(MainModule.java:122) >>>> >> > > at org.basex.core.cmd.AQuery.query(AQuery.java:94) >>>> >> > > at org.basex.core.cmd.XQuery.run(XQuery.java:22) >>>> >> > > at org.basex.core.Command.run(Command.java:257) >>>> >> > > at org.basex.core.Command.execute(Command.java:93) >>>> >> > > at org.basex.server.ClientListener.run(ClientListener.java:140) >>>> >> > >>>> >> >>>> > >>>> >>> >>> >
CG
If the "restored database" (which you are updating in the second step) is still ok, we might be able to trace this down; but you'd probably need to provide us with your data.
Quoted text · halit tiryaki · 315 lines
halit tiryaki <halit.tiryaki at gmail.com> schrieb am Do., 17. Mai 2018, 23:15: > > How did you invoke flush? Did you use the client/server architecture? > I'm using the C# client. > Simply executing the commands on the session: > - SET AUTOFLUSH true > - FLUSH (after ~200 docs of insert) > > > > The stack trace seems to indicate that you were deleting data. Is it > part of your import step? > yes, outdated documents are dropped in the import process > > Thanks > Halit > > On Thu, May 17, 2018 at 10:28 PM, Christian Grün < > christian.gruen at gmail.com> wrote: > >> Hi Halit, >> >> >> Some more questions: >> >> > The Flush() invoke in longer periods during the process might have >> been corrupted the state in race conditions. >> >> How did you invoke flush? Did you use the client/server architecture? >> >> Restored the db from a backup and restarted the import with >>> AutoFlush:true, this time I got: >>> >> >> The stack trace seems to indicate that you were deleting data. Is it part >> of your import step? >> >> Best, >> Christian >> >> >> >>> >>> >>> Improper use? Potential bug? Your feedback is welcome: >>> Contact: basex-talk at mailman.uni-konstanz.de >>> Version: BaseX 9.0.2 beta >>> Java: Oracle Corporation, 1.8.0_151 >>> OS: Linux, amd64 >>> Stack Trace: >>> java.lang.RuntimeException: Key does not exist: >>> '86295d851a084fb2bde4181e80760099' >>> at org.basex.util.Util.notExpected(Util.java:61) >>> at >>> org.basex.index.value.UpdatableDiskValues.delete(UpdatableDiskValues.java:99) >>> at org.basex.data.Data.indexDelete(Data.java:1060) >>> at org.basex.data.Data.delete(Data.java:663) >>> at org.basex.query.up.atomic.Delete.apply(Delete.java:39) >>> at >>> org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(AtomicUpdateCache.java:296) >>> at >>> org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUpdateCache.java:280) >>> at org.basex.query.up.DataUpdates.apply(DataUpdates.java:168) >>> at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120) >>> at org.basex.query.up.Updates.apply(Updates.java:157) >>> at org.basex.query.QueryContext.iter(QueryContext.java:341) >>> at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) >>> at org.basex.core.cmd.AQuery.query(AQuery.java:92) >>> at org.basex.core.cmd.XQuery.run(XQuery.java:22) >>> at org.basex.core.Command.run(Command.java:257) >>> at org.basex.core.Command.execute(Command.java:93) >>> at org.basex.server.ClientListener.run(ClientListener.java:140) >>> >>> On Mon, May 14, 2018 at 4:10 PM, Christian Grün < >>> christian.gruen at gmail.com> wrote: >>> >>>> Hi Halit, >>>> >>>> Thanks for reporting the stack trace. The only thing that I can derive >>>> from your error message is that something has gone wrong with your >>>> database in an earlier state. Unfortunately, it’s difficult to guess >>>> what might have been the reason. Feel free to give us an update if you >>>> should manage to get the issue reproduced and reproducible for us. >>>> >>>> Best, >>>> Christian >>>> >>>> >>>> >>>> On Wed, May 9, 2018 at 4:28 PM, halit tiryaki <halit.tiryaki at gmail.com> >>>> wrote: >>>> > I'm using the C# driver. but still, it doesn't look like a high level >>>> > problem. neither a query problem. thats why I didn't post some code >>>> > snippets. To make sure, I just tried to execute the INSPECT command >>>> in the >>>> > CLI. same exception. Also the db:optimize fails with the same >>>> exception, >>>> > although with a different stacktrace, which I will post below. >>>> > >>>> > Altough the race condition point of two processes interfering is a >>>> good >>>> > point, I'm currently having trouble to do anything useful with a >>>> session >>>> > from which a race condition would matter. If, then it must have >>>> happened >>>> > while the import was running along with the other, non-importing >>>> process. >>>> > >>>> > As I mentioned in the first post, while the import process was still >>>> > ongoing, the other process already started to have the exception at a >>>> very >>>> > late progress of the import. After the import was done, I was still >>>> able to >>>> > operate on the db as normal, but only with the session in the process >>>> which >>>> > did the long-run import. >>>> > >>>> > To restore the second process its session state to normal, I >>>> restarted the >>>> > webapplication, which closes both sessions. After that, I'm no longer >>>> to >>>> > operate on the db. Not even directly from the CLI. >>>> > >>>> > Seems it is a sort of a persisted data state which the low level >>>> basex code >>>> > can't handle when trying to reread it into to the memory. >>>> > >>>> > I 've already googled around for the same exception in BaseX, seems >>>> it was >>>> > an issue in some older versions too. >>>> > >>>> > >>>> > here the stacktrace when running db:optimize: >>>> > >>>> > Improper use? Potential bug? Your feedback is welcome: >>>> > Contact: basex-talk at mailman.uni-konstanz.de >>>> > Version: BaseX 9.0.2 beta >>>> > Java: Oracle Corporation, 1.8.0_151 >>>> > OS: Linux, amd64 >>>> > Stack Trace: >>>> > java.lang.ArrayIndexOutOfBoundsException: 52 >>>> > at org.basex.util.hash.TokenSet.key(TokenSet.java:128) >>>> > at org.basex.data.Data.name(Data.java:388) >>>> > at org.basex.io.serial.Serializer.node(Serializer.java:414) >>>> > at org.basex.io.serial.Serializer.node(Serializer.java:158) >>>> > at org.basex.io.serial.Serializer.node(Serializer.java:345) >>>> > at org.basex.io.serial.Serializer.node(Serializer.java:158) >>>> > at org.basex.io.serial.Serializer.serialize(Serializer.java:109) >>>> > at org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll.java:200) >>>> > at org.basex.build.Builder.parse(Builder.java:77) >>>> > at org.basex.build.DiskBuilder.build(DiskBuilder.java:77) >>>> > at org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:122) >>>> > at >>>> org.basex.query.up.primitives.db.DBOptimize.apply(DBOptimize.java:124) >>>> > at org.basex.query.up.DataUpdates.apply(DataUpdates.java:175) >>>> > at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120) >>>> > at org.basex.query.up.Updates.apply(Updates.java:157) >>>> > at org.basex.query.QueryContext.iter(QueryContext.java:341) >>>> > at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) >>>> > at org.basex.core.cmd.AQuery.query(AQuery.java:92) >>>> > at org.basex.core.cmd.XQuery.run(XQuery.java:22) >>>> > at org.basex.core.Command.run(Command.java:257) >>>> > at org.basex.core.Command.execute(Command.java:93) >>>> > at org.basex.server.ClientListener.run(ClientListener.java:140) >>>> > >>>> > >>>> > On Wed, May 9, 2018 at 11:45 AM, Alexander Holupirek < >>>> alex at holupirek.de> >>>> > wrote: >>>> >> >>>> >> Hi, >>>> >> >>>> >> (please also respond to the list. Others might have suggestions as >>>> well >>>> >> ;-) >>>> >> >>>> >> it still would be great if you could provide even more details. For >>>> >> instance, what programming language do you use?Ideally include a >>>> Short, Self >>>> >> Contained, Correct (Compilable), Example (SSCCE) [1] in order to let >>>> people >>>> >> reproduce the behaviour. >>>> >> >>>> >> From a high level perspective, are you sure the sessions do not >>>> interfere? >>>> >> Do you additionally work with BaseXGUI on the database, while your >>>> program >>>> >> is importing? >>>> >> >>>> >> Cheers, >>>> >> Alex >>>> >> >>>> >> >>>> >> [1] http://sscce.org/ >>>> >> >>>> >> > On 9. May 2018, at 09:26, halit tiryaki <halit.tiryaki at gmail.com> >>>> wrote: >>>> >> > >>>> >> > hello, thanks for the fast response. >>>> >> > >>>> >> > It happened in the process of a large Import. There were two >>>> processes >>>> >> > with each a respective Session to BaseX. One process with the Open >>>> session >>>> >> > which did the Import was doing fine. The Second process however >>>> Starter >>>> >> > having the mentioned exception. After stopping Both processes and >>>> restarting >>>> >> > the app, no basex Session is now able to operate normal. >>>> >> > >>>> >> > The ArrayIndexOutOfBoundsException already comes in executing the >>>> >> > command INSPECT. >>>> >> > >>>> >> > The db-Info returns following: >>>> >> > >>>> >> > >>>> >> > info db: >>>> >> > Database Properties >>>> >> > NAME: XXX >>>> >> > SIZE: 5877 kB >>>> >> > NODES: 124051 >>>> >> > DOCUMENTS: 5497 >>>> >> > BINARIES: 0 >>>> >> > TIMESTAMP: 2018-05-09T05:20:44.000Z >>>> >> > UPTODATE: false >>>> >> > >>>> >> > Resource Properties >>>> >> > INPUTPATH: >>>> >> > INPUTSIZE: 0 b >>>> >> > INPUTDATE: 2018-05-09T04:32:10.539Z >>>> >> > >>>> >> > Indexes >>>> >> > TEXTINDEX: true >>>> >> > ATTRINDEX: true >>>> >> > TOKENINDEX: false >>>> >> > FTINDEX: false >>>> >> > TEXTINCLUDE: /item_User/node()/User:Email >>>> >> > ATTRINCLUDE: /item_UserImage/node()/UserImage:Id >>>> >> > TOKENINCLUDE: >>>> >> > FTINCLUDE: /item_XXX/node()/XXX:[german] >>>> >> > LANGUAGE: English >>>> >> > STEMMING: false >>>> >> > CASESENS: false >>>> >> > DIACRITICS: false >>>> >> > STOPWORDS: >>>> >> > UPDINDEX: true >>>> >> > AUTOOPTIMIZE: false >>>> >> > MAXCATS: 100 >>>> >> > MAXLEN: 96 >>>> >> > SPLITSIZE: 0 >>>> >> >>>> >> > Alexander Holupirek <alex at holupirek.de> schrieb am Mi., 9. Mai >>>> 2018 >>>> >> > 08:44: >>>> >> > Hi, >>>> >> > >>>> >> > could you please provide some more details? >>>> >> > The best would be a reproducible example. >>>> >> > >>>> >> > Thanks, >>>> >> > Alex >>>> >> > >>>> >> > >>>> >> > > On 9. May 2018, at 06:50, halit tiryaki <halit.tiryaki at gmail.com >>>> > >>>> >> > > wrote: >>>> >> > > >>>> >> > > Hello, >>>> >> > > >>>> >> > > got a serious problem here. any ideas? thanks >>>> >> > > >>>> >> > > Improper use? Potential bug? Your feedback is welcome: >>>> >> > > Contact: basex-talk at mailman.uni-konstanz.de >>>> >> > > Version: BaseX 9.0.2 beta >>>> >> > > Java: Oracle Corporation, 1.8.0_151 >>>> >> > > OS: Linux, amd64 >>>> >> > > Stack Trace: >>>> >> > > java.lang.ArrayIndexOutOfBoundsException: 4288 >>>> >> > > at >>>> org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:151) >>>> >> > > at org.basex.data.Data.kind(Data.java:294) >>>> >> > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:332) >>>> >> > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:323) >>>> >> > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38) >>>> >> > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:32) >>>> >> > > at org.basex.query.QueryContext.next(QueryContext.java:392) >>>> >> > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:50) >>>> >> > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:34) >>>> >> > > at org.basex.query.expr.ParseExpr.item(ParseExpr.java:58) >>>> >> > > at org.basex.query.expr.ParseExpr.atomItem(ParseExpr.java:84) >>>> >> > > at org.basex.query.func.fn.FnConcat.item(FnConcat.java:20) >>>> >> > > at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71) >>>> >> > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) >>>> >> > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) >>>> >> > > at >>>> org.basex.query.expr.path.CachedPath.nodeIter(CachedPath.java:36) >>>> >> > > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:69) >>>> >> > > at org.basex.query.expr.gflwor.For$1.next(For.java:107) >>>> >> > > at org.basex.query.expr.gflwor.OrderBy$1.sort(OrderBy.java:73) >>>> >> > > at org.basex.query.expr.gflwor.OrderBy$1.next(OrderBy.java:54) >>>> >> > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) >>>> >> > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) >>>> >> > > at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:87) >>>> >> > > at org.basex.query.QueryContext.next(QueryContext.java:392) >>>> >> > > at org.basex.query.scope.MainModule$1.next(MainModule.java:122) >>>> >> > > at org.basex.core.cmd.AQuery.query(AQuery.java:94) >>>> >> > > at org.basex.core.cmd.XQuery.run(XQuery.java:22) >>>> >> > > at org.basex.core.Command.run(Command.java:257) >>>> >> > > at org.basex.core.Command.execute(Command.java:93) >>>> >> > > at org.basex.server.ClientListener.run(ClientListener.java:140) >>>> >> > >>>> >> >>>> > >>>> >>> >>> >
Written for BaseX 9.0. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery