Database corrupted on updating with RestXQ

On 26/07/19 16:29, Marco Lettere wrote: > Hi all, > > starting with 9.2.1 we experience a strange error with a RESTXQ API > of ours that we have been using for years. > > The typical pattern is lookup a document update it and store it back. > > We have done this milions of time and also all the tests work neatly. > > But when using it from inside the web application, at around the tenth > operation we get the DB corrupted with the stack trace [1]. This seems > to happen only when the database is nearly empty and, for sure, it > doesn't appear on 8.x series. > > It feels like some concurrency flaw is introduced somewhere but it's > hard to say because the API is eight years in age and it never changed > significantly. > > I know it's hard without an SSCE... but while working to insulate this > byzantine behaviour (I've been trying for days now) maybe someone in > the list has a clue or is able to interpret the stack trace? > > Thanks and have a nice weekend, > > Marco. > > > [1] > > Improper use? Potential bug? Your feedback is welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 9.3 beta > Java: Ubuntu, 13 > OS: Linux, amd64 > Stack Trace: > java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for > length 1 > at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) > at > org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) > at > org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) > at org.basex.data.Data.kind(Data.java:304) > at org.basex.query.value.node.DBNode.<init>(DBNode.java:50) > at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) > at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) > at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) > at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) > at org.basex.query.expr.Arr.inline(Arr.java:69) > at org.basex.query.expr.path.Path.inline(Path.java:962) > at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) > at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) > at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) > at org.basex.query.expr.Try.inline(Try.java:106) > at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) > at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) > at > org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) > at org.basex.query.scope.MainModule.comp(MainModule.java:81) > at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) > at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) > at org.basex.query.QueryContext.compile(QueryContext.java:312) > at org.basex.query.QueryContext.iter(QueryContext.java:331) > at > org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) > at org.basex.http.web.WebResponse.create(WebResponse.java:63) > at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) > at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) > at > org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at org.eclipse.jetty.server.Server.handle(Server.java:505) > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) > at > org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > at > org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > at > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) > at java.base/java.lang.Thread.run(Thread.java:835) >

On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere <m.lettere at gmail.com> wrote: > > Hi all, > > starting with 9.2.1 we experience a strange error with a RESTXQ API of > ours that we have been using for years. > > The typical pattern is lookup a document update it and store it back. > > We have done this milions of time and also all the tests work neatly. > > But when using it from inside the web application, at around the tenth > operation we get the DB corrupted with the stack trace [1]. This seems > to happen only when the database is nearly empty and, for sure, it > doesn't appear on 8.x series. > > It feels like some concurrency flaw is introduced somewhere but it's > hard to say because the API is eight years in age and it never changed > significantly. > > I know it's hard without an SSCE... but while working to insulate this > byzantine behaviour (I've been trying for days now) maybe someone in the > list has a clue or is able to interpret the stack trace? > > Thanks and have a nice weekend, > > Marco. > > > [1] > > Improper use? Potential bug? Your feedback is welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 9.3 beta > Java: Ubuntu, 13 > OS: Linux, amd64 > Stack Trace: > java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 > at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) > at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) > at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) > at org.basex.data.Data.kind(Data.java:304) > at org.basex.query.value.node.DBNode.<init>(DBNode.java:50) > at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) > at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) > at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) > at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) > at org.basex.query.expr.Arr.inline(Arr.java:69) > at org.basex.query.expr.path.Path.inline(Path.java:962) > at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) > at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) > at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) > at org.basex.query.expr.Try.inline(Try.java:106) > at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) > at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) > at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) > at org.basex.query.scope.MainModule.comp(MainModule.java:81) > at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) > at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) > at org.basex.query.QueryContext.compile(QueryContext.java:312) > at org.basex.query.QueryContext.iter(QueryContext.java:331) > at > org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) > at org.basex.http.web.WebResponse.create(WebResponse.java:63) > at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) > at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) > at > org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at org.eclipse.jetty.server.Server.handle(Server.java:505) > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) > at > org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > at > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) > at java.base/java.lang.Thread.run(Thread.java:835) >

On 26/07/19 18:41, Christian Grün wrote: > Hi Marco, > > Back in February, a user reported a database inconsistency [1] – which > happened, too, if a database was nearly empty – and we managed to > build a little text case to get this reproduced [2]. After that, 9.2 > was released. Maybe this fix introduced another irregularity for this > special case? > > Maybe we can get this reproduced by taking the script from issue 1662 > and modify it a little? That would be great. > > Sorry to the inconvenience, > Christian > > PS: You can safely ignore the "Creating database..." output. It may > just indicate that an XML document is parsed, and that an internal > main-memory database instance is created, possibly for your users.xml > file in the data directory. > > [1] https://www.mail-archive.com/basex-talk at mailman.uni-konstanz.de/msg11459.html > [2] https://github.com/BaseXdb/basex/issues/1662 > > > > On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere <m.lettere at gmail.com> wrote: >> Hi all, >> >> starting with 9.2.1 we experience a strange error with a RESTXQ API of >> ours that we have been using for years. >> >> The typical pattern is lookup a document update it and store it back. >> >> We have done this milions of time and also all the tests work neatly. >> >> But when using it from inside the web application, at around the tenth >> operation we get the DB corrupted with the stack trace [1]. This seems >> to happen only when the database is nearly empty and, for sure, it >> doesn't appear on 8.x series. >> >> It feels like some concurrency flaw is introduced somewhere but it's >> hard to say because the API is eight years in age and it never changed >> significantly. >> >> I know it's hard without an SSCE... but while working to insulate this >> byzantine behaviour (I've been trying for days now) maybe someone in the >> list has a clue or is able to interpret the stack trace? >> >> Thanks and have a nice weekend, >> >> Marco. >> >> >> [1] >> >> Improper use? Potential bug? Your feedback is welcome: >> Contact: basex-talk at mailman.uni-konstanz.de >> Version: BaseX 9.3 beta >> Java: Ubuntu, 13 >> OS: Linux, amd64 >> Stack Trace: >> java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 >> at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) >> at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) >> at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) >> at org.basex.data.Data.kind(Data.java:304) >> at org.basex.query.value.node.DBNode.<init>(DBNode.java:50) >> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) >> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) >> at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) >> at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) >> at org.basex.query.expr.Arr.inline(Arr.java:69) >> at org.basex.query.expr.path.Path.inline(Path.java:962) >> at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) >> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) >> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) >> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) >> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) >> at org.basex.query.expr.Try.inline(Try.java:106) >> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) >> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) >> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) >> at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) >> at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) >> at org.basex.query.scope.MainModule.comp(MainModule.java:81) >> at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) >> at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) >> at org.basex.query.QueryContext.compile(QueryContext.java:312) >> at org.basex.query.QueryContext.iter(QueryContext.java:331) >> at >> org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) >> at org.basex.http.web.WebResponse.create(WebResponse.java:63) >> at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) >> at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) >> at >> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) >> at >> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) >> at >> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) >> at >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) >> at >> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) >> at >> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) >> at >> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) >> at >> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) >> at >> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) >> at >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) >> at org.eclipse.jetty.server.Server.handle(Server.java:505) >> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) >> at >> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) >> at >> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) >> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) >> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) >> at >> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) >> at >> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) >> at >> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) >> at >> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) >> at >> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) >> at >> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) >> at >> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) >> at java.base/java.lang.Thread.run(Thread.java:835) >>

On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere <m.lettere at gmail.com> wrote: > > Thanks Christian, > I haven't been able to reproduce the bug with your SSCE. > Nevertheless I spent some time in tracing the operations with the actual > data I'm currently storing into the DB and a sequence of > db:add/db:replace close enough to actual app flow. > I attach the query script which should be rather self explaining. > You may run the script changing always $f with $f + 1 starting (from 0). > By uncommenting the different configurations of the variable $input you > can see how the misbehaviour depends somehow on the data because only > $d8 is causing the crash. > I stared at the diffs from $d8 to the other but there really isn't any > significant difference so now it's very hard for me to understand. > > Thank you in advance for all support that you can provide as usual. > Regards, > Marco. > > On 26/07/19 18:41, Christian Grün wrote: > > Hi Marco, > > > > Back in February, a user reported a database inconsistency [1] – which > > happened, too, if a database was nearly empty – and we managed to > > build a little text case to get this reproduced [2]. After that, 9.2 > > was released. Maybe this fix introduced another irregularity for this > > special case? > > > > Maybe we can get this reproduced by taking the script from issue 1662 > > and modify it a little? That would be great. > > > > Sorry to the inconvenience, > > Christian > > > > PS: You can safely ignore the "Creating database..." output. It may > > just indicate that an XML document is parsed, and that an internal > > main-memory database instance is created, possibly for your users.xml > > file in the data directory. > > > > [1] https://www.mail-archive.com/basex-talk at mailman.uni-konstanz.de/msg11459.html > > [2] https://github.com/BaseXdb/basex/issues/1662 > > > > > > > > On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere <m.lettere at gmail.com> wrote: > >> Hi all, > >> > >> starting with 9.2.1 we experience a strange error with a RESTXQ API of > >> ours that we have been using for years. > >> > >> The typical pattern is lookup a document update it and store it back. > >> > >> We have done this milions of time and also all the tests work neatly. > >> > >> But when using it from inside the web application, at around the tenth > >> operation we get the DB corrupted with the stack trace [1]. This seems > >> to happen only when the database is nearly empty and, for sure, it > >> doesn't appear on 8.x series. > >> > >> It feels like some concurrency flaw is introduced somewhere but it's > >> hard to say because the API is eight years in age and it never changed > >> significantly. > >> > >> I know it's hard without an SSCE... but while working to insulate this > >> byzantine behaviour (I've been trying for days now) maybe someone in the > >> list has a clue or is able to interpret the stack trace? > >> > >> Thanks and have a nice weekend, > >> > >> Marco. > >> > >> > >> [1] > >> > >> Improper use? Potential bug? Your feedback is welcome: > >> Contact: basex-talk at mailman.uni-konstanz.de > >> Version: BaseX 9.3 beta > >> Java: Ubuntu, 13 > >> OS: Linux, amd64 > >> Stack Trace: > >> java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 > >> at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) > >> at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) > >> at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) > >> at org.basex.data.Data.kind(Data.java:304) > >> at org.basex.query.value.node.DBNode.<init>(DBNode.java:50) > >> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) > >> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) > >> at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) > >> at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) > >> at org.basex.query.expr.Arr.inline(Arr.java:69) > >> at org.basex.query.expr.path.Path.inline(Path.java:962) > >> at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) > >> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) > >> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > >> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > >> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) > >> at org.basex.query.expr.Try.inline(Try.java:106) > >> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) > >> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > >> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > >> at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) > >> at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) > >> at org.basex.query.scope.MainModule.comp(MainModule.java:81) > >> at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) > >> at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) > >> at org.basex.query.QueryContext.compile(QueryContext.java:312) > >> at org.basex.query.QueryContext.iter(QueryContext.java:331) > >> at > >> org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) > >> at org.basex.http.web.WebResponse.create(WebResponse.java:63) > >> at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) > >> at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) > >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) > >> at > >> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) > >> at > >> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) > >> at > >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > >> at > >> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > >> at > >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > >> at > >> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > >> at > >> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) > >> at > >> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > >> at > >> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) > >> at > >> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > >> at > >> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) > >> at > >> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) > >> at > >> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > >> at > >> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) > >> at > >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > >> at > >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > >> at org.eclipse.jetty.server.Server.handle(Server.java:505) > >> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) > >> at > >> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) > >> at > >> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) > >> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > >> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > >> at > >> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > >> at > >> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > >> at > >> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > >> at > >> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > >> at > >> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > >> at > >> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) > >> at > >> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) > >> at java.base/java.lang.Thread.run(Thread.java:835) > >> >

On Wed, Jul 31, 2019 at 10:20 AM Christian Grün <christian.gruen at gmail.com> wrote: > > Marco, thanks for the attachment. I have created a script that > triggers the error [1]. Most probably, the bug is related to the > previous bug issue. We’ll have a look at this soon. – Christian > > [1] https://github.com/BaseXdb/basex/issues/1711 > > > > On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere <m.lettere at gmail.com> wrote: > > > > Thanks Christian, > > I haven't been able to reproduce the bug with your SSCE. > > Nevertheless I spent some time in tracing the operations with the actual > > data I'm currently storing into the DB and a sequence of > > db:add/db:replace close enough to actual app flow. > > I attach the query script which should be rather self explaining. > > You may run the script changing always $f with $f + 1 starting (from 0). > > By uncommenting the different configurations of the variable $input you > > can see how the misbehaviour depends somehow on the data because only > > $d8 is causing the crash. > > I stared at the diffs from $d8 to the other but there really isn't any > > significant difference so now it's very hard for me to understand. > > > > Thank you in advance for all support that you can provide as usual. > > Regards, > > Marco. > > > > On 26/07/19 18:41, Christian Grün wrote: > > > Hi Marco, > > > > > > Back in February, a user reported a database inconsistency [1] – which > > > happened, too, if a database was nearly empty – and we managed to > > > build a little text case to get this reproduced [2]. After that, 9.2 > > > was released. Maybe this fix introduced another irregularity for this > > > special case? > > > > > > Maybe we can get this reproduced by taking the script from issue 1662 > > > and modify it a little? That would be great. > > > > > > Sorry to the inconvenience, > > > Christian > > > > > > PS: You can safely ignore the "Creating database..." output. It may > > > just indicate that an XML document is parsed, and that an internal > > > main-memory database instance is created, possibly for your users.xml > > > file in the data directory. > > > > > > [1] https://www.mail-archive.com/basex-talk at mailman.uni-konstanz.de/msg11459.html > > > [2] https://github.com/BaseXdb/basex/issues/1662 > > > > > > > > > > > > On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere <m.lettere at gmail.com> wrote: > > >> Hi all, > > >> > > >> starting with 9.2.1 we experience a strange error with a RESTXQ API of > > >> ours that we have been using for years. > > >> > > >> The typical pattern is lookup a document update it and store it back. > > >> > > >> We have done this milions of time and also all the tests work neatly. > > >> > > >> But when using it from inside the web application, at around the tenth > > >> operation we get the DB corrupted with the stack trace [1]. This seems > > >> to happen only when the database is nearly empty and, for sure, it > > >> doesn't appear on 8.x series. > > >> > > >> It feels like some concurrency flaw is introduced somewhere but it's > > >> hard to say because the API is eight years in age and it never changed > > >> significantly. > > >> > > >> I know it's hard without an SSCE... but while working to insulate this > > >> byzantine behaviour (I've been trying for days now) maybe someone in the > > >> list has a clue or is able to interpret the stack trace? > > >> > > >> Thanks and have a nice weekend, > > >> > > >> Marco. > > >> > > >> > > >> [1] > > >> > > >> Improper use? Potential bug? Your feedback is welcome: > > >> Contact: basex-talk at mailman.uni-konstanz.de > > >> Version: BaseX 9.3 beta > > >> Java: Ubuntu, 13 > > >> OS: Linux, amd64 > > >> Stack Trace: > > >> java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 > > >> at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) > > >> at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) > > >> at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) > > >> at org.basex.data.Data.kind(Data.java:304) > > >> at org.basex.query.value.node.DBNode.<init>(DBNode.java:50) > > >> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) > > >> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) > > >> at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) > > >> at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) > > >> at org.basex.query.expr.Arr.inline(Arr.java:69) > > >> at org.basex.query.expr.path.Path.inline(Path.java:962) > > >> at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) > > >> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) > > >> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > > >> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > > >> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) > > >> at org.basex.query.expr.Try.inline(Try.java:106) > > >> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) > > >> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > > >> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > > >> at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) > > >> at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) > > >> at org.basex.query.scope.MainModule.comp(MainModule.java:81) > > >> at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) > > >> at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) > > >> at org.basex.query.QueryContext.compile(QueryContext.java:312) > > >> at org.basex.query.QueryContext.iter(QueryContext.java:331) > > >> at > > >> org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) > > >> at org.basex.http.web.WebResponse.create(WebResponse.java:63) > > >> at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) > > >> at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) > > >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) > > >> at > > >> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) > > >> at > > >> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) > > >> at > > >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > > >> at > > >> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > > >> at > > >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > >> at > > >> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > > >> at > > >> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) > > >> at > > >> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > > >> at > > >> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) > > >> at > > >> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > > >> at > > >> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) > > >> at > > >> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) > > >> at > > >> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > > >> at > > >> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) > > >> at > > >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > > >> at > > >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > >> at org.eclipse.jetty.server.Server.handle(Server.java:505) > > >> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) > > >> at > > >> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) > > >> at > > >> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) > > >> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > > >> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > > >> at > > >> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > > >> at > > >> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > > >> at > > >> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > > >> at > > >> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > > >> at > > >> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > > >> at > > >> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) > > >> at > > >> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) > > >> at java.base/java.lang.Thread.run(Thread.java:835) > > >> > >

On 02/08/19 15:04, Christian Grün wrote: > Hi Marco, > > A little update on the status quo: > > • I noticed that the bug you observed was basically exposed by the > previous bug fix (it existed before, but it didn’t occur in your > particular setting). > • It happens only in certain cases (i.e., for specific document sizes) > and if the added/replaced document has namespaces. > • Another good thing: It happens only if your database is empty. > > It’s not exactly an elegant proposal, but as long as we haven’t fixed > the bug, you could add an initial <dummy/> document to your database. > > We are doing our best, though. > Christian > > > > On Wed, Jul 31, 2019 at 10:20 AM Christian Grün > <christian.gruen at gmail.com> wrote: >> Marco, thanks for the attachment. I have created a script that >> triggers the error [1]. Most probably, the bug is related to the >> previous bug issue. We’ll have a look at this soon. – Christian >> >> [1] https://github.com/BaseXdb/basex/issues/1711 >> >> >> >> On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere <m.lettere at gmail.com> wrote: >>> Thanks Christian, >>> I haven't been able to reproduce the bug with your SSCE. >>> Nevertheless I spent some time in tracing the operations with the actual >>> data I'm currently storing into the DB and a sequence of >>> db:add/db:replace close enough to actual app flow. >>> I attach the query script which should be rather self explaining. >>> You may run the script changing always $f with $f + 1 starting (from 0). >>> By uncommenting the different configurations of the variable $input you >>> can see how the misbehaviour depends somehow on the data because only >>> $d8 is causing the crash. >>> I stared at the diffs from $d8 to the other but there really isn't any >>> significant difference so now it's very hard for me to understand. >>> >>> Thank you in advance for all support that you can provide as usual. >>> Regards, >>> Marco. >>> >>> On 26/07/19 18:41, Christian Grün wrote: >>>> Hi Marco, >>>> >>>> Back in February, a user reported a database inconsistency [1] – which >>>> happened, too, if a database was nearly empty – and we managed to >>>> build a little text case to get this reproduced [2]. After that, 9.2 >>>> was released. Maybe this fix introduced another irregularity for this >>>> special case? >>>> >>>> Maybe we can get this reproduced by taking the script from issue 1662 >>>> and modify it a little? That would be great. >>>> >>>> Sorry to the inconvenience, >>>> Christian >>>> >>>> PS: You can safely ignore the "Creating database..." output. It may >>>> just indicate that an XML document is parsed, and that an internal >>>> main-memory database instance is created, possibly for your users.xml >>>> file in the data directory. >>>> >>>> [1] https://www.mail-archive.com/basex-talk at mailman.uni-konstanz.de/msg11459.html >>>> [2] https://github.com/BaseXdb/basex/issues/1662 >>>> >>>> >>>> >>>> On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere <m.lettere at gmail.com> wrote: >>>>> Hi all, >>>>> >>>>> starting with 9.2.1 we experience a strange error with a RESTXQ API of >>>>> ours that we have been using for years. >>>>> >>>>> The typical pattern is lookup a document update it and store it back. >>>>> >>>>> We have done this milions of time and also all the tests work neatly. >>>>> >>>>> But when using it from inside the web application, at around the tenth >>>>> operation we get the DB corrupted with the stack trace [1]. This seems >>>>> to happen only when the database is nearly empty and, for sure, it >>>>> doesn't appear on 8.x series. >>>>> >>>>> It feels like some concurrency flaw is introduced somewhere but it's >>>>> hard to say because the API is eight years in age and it never changed >>>>> significantly. >>>>> >>>>> I know it's hard without an SSCE... but while working to insulate this >>>>> byzantine behaviour (I've been trying for days now) maybe someone in the >>>>> list has a clue or is able to interpret the stack trace? >>>>> >>>>> Thanks and have a nice weekend, >>>>> >>>>> Marco. >>>>> >>>>> >>>>> [1] >>>>> >>>>> Improper use? Potential bug? Your feedback is welcome: >>>>> Contact: basex-talk at mailman.uni-konstanz.de >>>>> Version: BaseX 9.3 beta >>>>> Java: Ubuntu, 13 >>>>> OS: Linux, amd64 >>>>> Stack Trace: >>>>> java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 >>>>> at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) >>>>> at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) >>>>> at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) >>>>> at org.basex.data.Data.kind(Data.java:304) >>>>> at org.basex.query.value.node.DBNode.<init>(DBNode.java:50) >>>>> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) >>>>> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) >>>>> at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) >>>>> at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) >>>>> at org.basex.query.expr.Arr.inline(Arr.java:69) >>>>> at org.basex.query.expr.path.Path.inline(Path.java:962) >>>>> at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) >>>>> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) >>>>> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) >>>>> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) >>>>> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) >>>>> at org.basex.query.expr.Try.inline(Try.java:106) >>>>> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) >>>>> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) >>>>> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) >>>>> at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) >>>>> at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) >>>>> at org.basex.query.scope.MainModule.comp(MainModule.java:81) >>>>> at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) >>>>> at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) >>>>> at org.basex.query.QueryContext.compile(QueryContext.java:312) >>>>> at org.basex.query.QueryContext.iter(QueryContext.java:331) >>>>> at >>>>> org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) >>>>> at org.basex.http.web.WebResponse.create(WebResponse.java:63) >>>>> at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) >>>>> at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) >>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) >>>>> at >>>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) >>>>> at >>>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) >>>>> at >>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) >>>>> at >>>>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) >>>>> at >>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) >>>>> at >>>>> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) >>>>> at >>>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) >>>>> at >>>>> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) >>>>> at >>>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) >>>>> at >>>>> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) >>>>> at >>>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) >>>>> at >>>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) >>>>> at >>>>> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) >>>>> at >>>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) >>>>> at >>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) >>>>> at >>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) >>>>> at org.eclipse.jetty.server.Server.handle(Server.java:505) >>>>> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) >>>>> at >>>>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) >>>>> at >>>>> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) >>>>> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) >>>>> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) >>>>> at >>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) >>>>> at >>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) >>>>> at >>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) >>>>> at >>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) >>>>> at >>>>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) >>>>> at >>>>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) >>>>> at >>>>> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) >>>>> at java.base/java.lang.Thread.run(Thread.java:835) >>>>>

On Sat, Aug 3, 2019 at 10:02 AM Marco Lettere <m.lettere at gmail.com> wrote: > > Hi Christian, > I'm currently preparing a deployment based on Docker for one of our > customers. Here in Italy it's holiday time in August so I have a bit of > time and I can coope with your suggested workaround for the moment. > Just a question to be prepared ... are intermediate bug-fix releases > also available in the form of docker containers? > Thanks for your support as usual. > Regards, > Marco. > > On 02/08/19 15:04, Christian Grün wrote: > > Hi Marco, > > > > A little update on the status quo: > > > > • I noticed that the bug you observed was basically exposed by the > > previous bug fix (it existed before, but it didn’t occur in your > > particular setting). > > • It happens only in certain cases (i.e., for specific document sizes) > > and if the added/replaced document has namespaces. > > • Another good thing: It happens only if your database is empty. > > > > It’s not exactly an elegant proposal, but as long as we haven’t fixed > > the bug, you could add an initial <dummy/> document to your database. > > > > We are doing our best, though. > > Christian > > > > > > > > On Wed, Jul 31, 2019 at 10:20 AM Christian Grün > > <christian.gruen at gmail.com> wrote: > >> Marco, thanks for the attachment. I have created a script that > >> triggers the error [1]. Most probably, the bug is related to the > >> previous bug issue. We’ll have a look at this soon. – Christian > >> > >> [1] https://github.com/BaseXdb/basex/issues/1711 > >> > >> > >> > >> On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere <m.lettere at gmail.com> wrote: > >>> Thanks Christian, > >>> I haven't been able to reproduce the bug with your SSCE. > >>> Nevertheless I spent some time in tracing the operations with the actual > >>> data I'm currently storing into the DB and a sequence of > >>> db:add/db:replace close enough to actual app flow. > >>> I attach the query script which should be rather self explaining. > >>> You may run the script changing always $f with $f + 1 starting (from 0). > >>> By uncommenting the different configurations of the variable $input you > >>> can see how the misbehaviour depends somehow on the data because only > >>> $d8 is causing the crash. > >>> I stared at the diffs from $d8 to the other but there really isn't any > >>> significant difference so now it's very hard for me to understand. > >>> > >>> Thank you in advance for all support that you can provide as usual. > >>> Regards, > >>> Marco. > >>> > >>> On 26/07/19 18:41, Christian Grün wrote: > >>>> Hi Marco, > >>>> > >>>> Back in February, a user reported a database inconsistency [1] – which > >>>> happened, too, if a database was nearly empty – and we managed to > >>>> build a little text case to get this reproduced [2]. After that, 9.2 > >>>> was released. Maybe this fix introduced another irregularity for this > >>>> special case? > >>>> > >>>> Maybe we can get this reproduced by taking the script from issue 1662 > >>>> and modify it a little? That would be great. > >>>> > >>>> Sorry to the inconvenience, > >>>> Christian > >>>> > >>>> PS: You can safely ignore the "Creating database..." output. It may > >>>> just indicate that an XML document is parsed, and that an internal > >>>> main-memory database instance is created, possibly for your users.xml > >>>> file in the data directory. > >>>> > >>>> [1] https://www.mail-archive.com/basex-talk at mailman.uni-konstanz.de/msg11459.html > >>>> [2] https://github.com/BaseXdb/basex/issues/1662 > >>>> > >>>> > >>>> > >>>> On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere <m.lettere at gmail.com> wrote: > >>>>> Hi all, > >>>>> > >>>>> starting with 9.2.1 we experience a strange error with a RESTXQ API of > >>>>> ours that we have been using for years. > >>>>> > >>>>> The typical pattern is lookup a document update it and store it back. > >>>>> > >>>>> We have done this milions of time and also all the tests work neatly. > >>>>> > >>>>> But when using it from inside the web application, at around the tenth > >>>>> operation we get the DB corrupted with the stack trace [1]. This seems > >>>>> to happen only when the database is nearly empty and, for sure, it > >>>>> doesn't appear on 8.x series. > >>>>> > >>>>> It feels like some concurrency flaw is introduced somewhere but it's > >>>>> hard to say because the API is eight years in age and it never changed > >>>>> significantly. > >>>>> > >>>>> I know it's hard without an SSCE... but while working to insulate this > >>>>> byzantine behaviour (I've been trying for days now) maybe someone in the > >>>>> list has a clue or is able to interpret the stack trace? > >>>>> > >>>>> Thanks and have a nice weekend, > >>>>> > >>>>> Marco. > >>>>> > >>>>> > >>>>> [1] > >>>>> > >>>>> Improper use? Potential bug? Your feedback is welcome: > >>>>> Contact: basex-talk at mailman.uni-konstanz.de > >>>>> Version: BaseX 9.3 beta > >>>>> Java: Ubuntu, 13 > >>>>> OS: Linux, amd64 > >>>>> Stack Trace: > >>>>> java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 > >>>>> at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) > >>>>> at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) > >>>>> at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) > >>>>> at org.basex.data.Data.kind(Data.java:304) > >>>>> at org.basex.query.value.node.DBNode.<init>(DBNode.java:50) > >>>>> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) > >>>>> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) > >>>>> at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) > >>>>> at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) > >>>>> at org.basex.query.expr.Arr.inline(Arr.java:69) > >>>>> at org.basex.query.expr.path.Path.inline(Path.java:962) > >>>>> at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) > >>>>> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) > >>>>> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > >>>>> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > >>>>> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) > >>>>> at org.basex.query.expr.Try.inline(Try.java:106) > >>>>> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) > >>>>> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > >>>>> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > >>>>> at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) > >>>>> at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) > >>>>> at org.basex.query.scope.MainModule.comp(MainModule.java:81) > >>>>> at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) > >>>>> at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) > >>>>> at org.basex.query.QueryContext.compile(QueryContext.java:312) > >>>>> at org.basex.query.QueryContext.iter(QueryContext.java:331) > >>>>> at > >>>>> org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) > >>>>> at org.basex.http.web.WebResponse.create(WebResponse.java:63) > >>>>> at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) > >>>>> at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) > >>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) > >>>>> at > >>>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) > >>>>> at > >>>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) > >>>>> at > >>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > >>>>> at > >>>>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > >>>>> at > >>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > >>>>> at > >>>>> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > >>>>> at > >>>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) > >>>>> at > >>>>> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > >>>>> at > >>>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) > >>>>> at > >>>>> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > >>>>> at > >>>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) > >>>>> at > >>>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) > >>>>> at > >>>>> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > >>>>> at > >>>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) > >>>>> at > >>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > >>>>> at > >>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > >>>>> at org.eclipse.jetty.server.Server.handle(Server.java:505) > >>>>> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) > >>>>> at > >>>>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) > >>>>> at > >>>>> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) > >>>>> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > >>>>> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > >>>>> at > >>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > >>>>> at > >>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > >>>>> at > >>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > >>>>> at > >>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > >>>>> at > >>>>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > >>>>> at > >>>>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) > >>>>> at > >>>>> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) > >>>>> at java.base/java.lang.Thread.run(Thread.java:835) > >>>>> >

On Tue, Aug 6, 2019 at 9:38 AM Christian Grün <christian.gruen at gmail.com> wrote: > > Hi Marco, > > I think that the bug fix (which is still on my todo list) will be made > available with BaseX 9.3; so, for now, it’s probably better to choose > the workaround. > > Cheers, > Christian > > > > On Sat, Aug 3, 2019 at 10:02 AM Marco Lettere <m.lettere at gmail.com> wrote: > > > > Hi Christian, > > I'm currently preparing a deployment based on Docker for one of our > > customers. Here in Italy it's holiday time in August so I have a bit of > > time and I can coope with your suggested workaround for the moment. > > Just a question to be prepared ... are intermediate bug-fix releases > > also available in the form of docker containers? > > Thanks for your support as usual. > > Regards, > > Marco. > > > > On 02/08/19 15:04, Christian Grün wrote: > > > Hi Marco, > > > > > > A little update on the status quo: > > > > > > • I noticed that the bug you observed was basically exposed by the > > > previous bug fix (it existed before, but it didn’t occur in your > > > particular setting). > > > • It happens only in certain cases (i.e., for specific document sizes) > > > and if the added/replaced document has namespaces. > > > • Another good thing: It happens only if your database is empty. > > > > > > It’s not exactly an elegant proposal, but as long as we haven’t fixed > > > the bug, you could add an initial <dummy/> document to your database. > > > > > > We are doing our best, though. > > > Christian > > > > > > > > > > > > On Wed, Jul 31, 2019 at 10:20 AM Christian Grün > > > <christian.gruen at gmail.com> wrote: > > >> Marco, thanks for the attachment. I have created a script that > > >> triggers the error [1]. Most probably, the bug is related to the > > >> previous bug issue. We’ll have a look at this soon. – Christian > > >> > > >> [1] https://github.com/BaseXdb/basex/issues/1711 > > >> > > >> > > >> > > >> On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere <m.lettere at gmail.com> wrote: > > >>> Thanks Christian, > > >>> I haven't been able to reproduce the bug with your SSCE. > > >>> Nevertheless I spent some time in tracing the operations with the actual > > >>> data I'm currently storing into the DB and a sequence of > > >>> db:add/db:replace close enough to actual app flow. > > >>> I attach the query script which should be rather self explaining. > > >>> You may run the script changing always $f with $f + 1 starting (from 0). > > >>> By uncommenting the different configurations of the variable $input you > > >>> can see how the misbehaviour depends somehow on the data because only > > >>> $d8 is causing the crash. > > >>> I stared at the diffs from $d8 to the other but there really isn't any > > >>> significant difference so now it's very hard for me to understand. > > >>> > > >>> Thank you in advance for all support that you can provide as usual. > > >>> Regards, > > >>> Marco. > > >>> > > >>> On 26/07/19 18:41, Christian Grün wrote: > > >>>> Hi Marco, > > >>>> > > >>>> Back in February, a user reported a database inconsistency [1] – which > > >>>> happened, too, if a database was nearly empty – and we managed to > > >>>> build a little text case to get this reproduced [2]. After that, 9.2 > > >>>> was released. Maybe this fix introduced another irregularity for this > > >>>> special case? > > >>>> > > >>>> Maybe we can get this reproduced by taking the script from issue 1662 > > >>>> and modify it a little? That would be great. > > >>>> > > >>>> Sorry to the inconvenience, > > >>>> Christian > > >>>> > > >>>> PS: You can safely ignore the "Creating database..." output. It may > > >>>> just indicate that an XML document is parsed, and that an internal > > >>>> main-memory database instance is created, possibly for your users.xml > > >>>> file in the data directory. > > >>>> > > >>>> [1] https://www.mail-archive.com/basex-talk at mailman.uni-konstanz.de/msg11459.html > > >>>> [2] https://github.com/BaseXdb/basex/issues/1662 > > >>>> > > >>>> > > >>>> > > >>>> On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere <m.lettere at gmail.com> wrote: > > >>>>> Hi all, > > >>>>> > > >>>>> starting with 9.2.1 we experience a strange error with a RESTXQ API of > > >>>>> ours that we have been using for years. > > >>>>> > > >>>>> The typical pattern is lookup a document update it and store it back. > > >>>>> > > >>>>> We have done this milions of time and also all the tests work neatly. > > >>>>> > > >>>>> But when using it from inside the web application, at around the tenth > > >>>>> operation we get the DB corrupted with the stack trace [1]. This seems > > >>>>> to happen only when the database is nearly empty and, for sure, it > > >>>>> doesn't appear on 8.x series. > > >>>>> > > >>>>> It feels like some concurrency flaw is introduced somewhere but it's > > >>>>> hard to say because the API is eight years in age and it never changed > > >>>>> significantly. > > >>>>> > > >>>>> I know it's hard without an SSCE... but while working to insulate this > > >>>>> byzantine behaviour (I've been trying for days now) maybe someone in the > > >>>>> list has a clue or is able to interpret the stack trace? > > >>>>> > > >>>>> Thanks and have a nice weekend, > > >>>>> > > >>>>> Marco. > > >>>>> > > >>>>> > > >>>>> [1] > > >>>>> > > >>>>> Improper use? Potential bug? Your feedback is welcome: > > >>>>> Contact: basex-talk at mailman.uni-konstanz.de > > >>>>> Version: BaseX 9.3 beta > > >>>>> Java: Ubuntu, 13 > > >>>>> OS: Linux, amd64 > > >>>>> Stack Trace: > > >>>>> java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 > > >>>>> at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) > > >>>>> at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) > > >>>>> at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) > > >>>>> at org.basex.data.Data.kind(Data.java:304) > > >>>>> at org.basex.query.value.node.DBNode.<init>(DBNode.java:50) > > >>>>> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) > > >>>>> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) > > >>>>> at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) > > >>>>> at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) > > >>>>> at org.basex.query.expr.Arr.inline(Arr.java:69) > > >>>>> at org.basex.query.expr.path.Path.inline(Path.java:962) > > >>>>> at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) > > >>>>> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) > > >>>>> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > > >>>>> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > > >>>>> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) > > >>>>> at org.basex.query.expr.Try.inline(Try.java:106) > > >>>>> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) > > >>>>> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > > >>>>> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > > >>>>> at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) > > >>>>> at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) > > >>>>> at org.basex.query.scope.MainModule.comp(MainModule.java:81) > > >>>>> at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) > > >>>>> at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) > > >>>>> at org.basex.query.QueryContext.compile(QueryContext.java:312) > > >>>>> at org.basex.query.QueryContext.iter(QueryContext.java:331) > > >>>>> at > > >>>>> org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) > > >>>>> at org.basex.http.web.WebResponse.create(WebResponse.java:63) > > >>>>> at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) > > >>>>> at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) > > >>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) > > >>>>> at > > >>>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) > > >>>>> at > > >>>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) > > >>>>> at > > >>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > > >>>>> at > > >>>>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > > >>>>> at > > >>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > >>>>> at > > >>>>> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > > >>>>> at > > >>>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) > > >>>>> at > > >>>>> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > > >>>>> at > > >>>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) > > >>>>> at > > >>>>> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > > >>>>> at > > >>>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) > > >>>>> at > > >>>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) > > >>>>> at > > >>>>> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > > >>>>> at > > >>>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) > > >>>>> at > > >>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > > >>>>> at > > >>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > >>>>> at org.eclipse.jetty.server.Server.handle(Server.java:505) > > >>>>> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) > > >>>>> at > > >>>>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) > > >>>>> at > > >>>>> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) > > >>>>> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > > >>>>> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > > >>>>> at > > >>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > > >>>>> at > > >>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > > >>>>> at > > >>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > > >>>>> at > > >>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > > >>>>> at > > >>>>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > > >>>>> at > > >>>>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) > > >>>>> at > > >>>>> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) > > >>>>> at java.base/java.lang.Thread.run(Thread.java:835) > > >>>>> > >

On 26/09/19 13:55, Christian Grün wrote: > Hi Marco, > > Finally some news on the storage bug you reported: It is fixed with > the latest snapshot [1]. > > Looking forward to your tests (and I hope you received my response on > parallel querying), > Christian > > [1] http://files.basex.org/releases/latest/ > > > > On Tue, Aug 6, 2019 at 9:38 AM Christian Grün <christian.gruen at gmail.com> wrote: >> Hi Marco, >> >> I think that the bug fix (which is still on my todo list) will be made >> available with BaseX 9.3; so, for now, it’s probably better to choose >> the workaround. >> >> Cheers, >> Christian >> >> >> >> On Sat, Aug 3, 2019 at 10:02 AM Marco Lettere <m.lettere at gmail.com> wrote: >>> Hi Christian, >>> I'm currently preparing a deployment based on Docker for one of our >>> customers. Here in Italy it's holiday time in August so I have a bit of >>> time and I can coope with your suggested workaround for the moment. >>> Just a question to be prepared ... are intermediate bug-fix releases >>> also available in the form of docker containers? >>> Thanks for your support as usual. >>> Regards, >>> Marco. >>> >>> On 02/08/19 15:04, Christian Grün wrote: >>>> Hi Marco, >>>> >>>> A little update on the status quo: >>>> >>>> • I noticed that the bug you observed was basically exposed by the >>>> previous bug fix (it existed before, but it didn’t occur in your >>>> particular setting). >>>> • It happens only in certain cases (i.e., for specific document sizes) >>>> and if the added/replaced document has namespaces. >>>> • Another good thing: It happens only if your database is empty. >>>> >>>> It’s not exactly an elegant proposal, but as long as we haven’t fixed >>>> the bug, you could add an initial <dummy/> document to your database. >>>> >>>> We are doing our best, though. >>>> Christian >>>> >>>> >>>> >>>> On Wed, Jul 31, 2019 at 10:20 AM Christian Grün >>>> <christian.gruen at gmail.com> wrote: >>>>> Marco, thanks for the attachment. I have created a script that >>>>> triggers the error [1]. Most probably, the bug is related to the >>>>> previous bug issue. We’ll have a look at this soon. – Christian >>>>> >>>>> [1] https://github.com/BaseXdb/basex/issues/1711 >>>>> >>>>> >>>>> >>>>> On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere <m.lettere at gmail.com> wrote: >>>>>> Thanks Christian, >>>>>> I haven't been able to reproduce the bug with your SSCE. >>>>>> Nevertheless I spent some time in tracing the operations with the actual >>>>>> data I'm currently storing into the DB and a sequence of >>>>>> db:add/db:replace close enough to actual app flow. >>>>>> I attach the query script which should be rather self explaining. >>>>>> You may run the script changing always $f with $f + 1 starting (from 0). >>>>>> By uncommenting the different configurations of the variable $input you >>>>>> can see how the misbehaviour depends somehow on the data because only >>>>>> $d8 is causing the crash. >>>>>> I stared at the diffs from $d8 to the other but there really isn't any >>>>>> significant difference so now it's very hard for me to understand. >>>>>> >>>>>> Thank you in advance for all support that you can provide as usual. >>>>>> Regards, >>>>>> Marco. >>>>>> >>>>>> On 26/07/19 18:41, Christian Grün wrote: >>>>>>> Hi Marco, >>>>>>> >>>>>>> Back in February, a user reported a database inconsistency [1] – which >>>>>>> happened, too, if a database was nearly empty – and we managed to >>>>>>> build a little text case to get this reproduced [2]. After that, 9.2 >>>>>>> was released. Maybe this fix introduced another irregularity for this >>>>>>> special case? >>>>>>> >>>>>>> Maybe we can get this reproduced by taking the script from issue 1662 >>>>>>> and modify it a little? That would be great. >>>>>>> >>>>>>> Sorry to the inconvenience, >>>>>>> Christian >>>>>>> >>>>>>> PS: You can safely ignore the "Creating database..." output. It may >>>>>>> just indicate that an XML document is parsed, and that an internal >>>>>>> main-memory database instance is created, possibly for your users.xml >>>>>>> file in the data directory. >>>>>>> >>>>>>> [1] https://www.mail-archive.com/basex-talk at mailman.uni-konstanz.de/msg11459.html >>>>>>> [2] https://github.com/BaseXdb/basex/issues/1662 >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere <m.lettere at gmail.com> wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> starting with 9.2.1 we experience a strange error with a RESTXQ API of >>>>>>>> ours that we have been using for years. >>>>>>>> >>>>>>>> The typical pattern is lookup a document update it and store it back. >>>>>>>> >>>>>>>> We have done this milions of time and also all the tests work neatly. >>>>>>>> >>>>>>>> But when using it from inside the web application, at around the tenth >>>>>>>> operation we get the DB corrupted with the stack trace [1]. This seems >>>>>>>> to happen only when the database is nearly empty and, for sure, it >>>>>>>> doesn't appear on 8.x series. >>>>>>>> >>>>>>>> It feels like some concurrency flaw is introduced somewhere but it's >>>>>>>> hard to say because the API is eight years in age and it never changed >>>>>>>> significantly. >>>>>>>> >>>>>>>> I know it's hard without an SSCE... but while working to insulate this >>>>>>>> byzantine behaviour (I've been trying for days now) maybe someone in the >>>>>>>> list has a clue or is able to interpret the stack trace? >>>>>>>> >>>>>>>> Thanks and have a nice weekend, >>>>>>>> >>>>>>>> Marco. >>>>>>>> >>>>>>>> >>>>>>>> [1] >>>>>>>> >>>>>>>> Improper use? Potential bug? Your feedback is welcome: >>>>>>>> Contact: basex-talk at mailman.uni-konstanz.de >>>>>>>> Version: BaseX 9.3 beta >>>>>>>> Java: Ubuntu, 13 >>>>>>>> OS: Linux, amd64 >>>>>>>> Stack Trace: >>>>>>>> java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 >>>>>>>> at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) >>>>>>>> at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) >>>>>>>> at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) >>>>>>>> at org.basex.data.Data.kind(Data.java:304) >>>>>>>> at org.basex.query.value.node.DBNode.<init>(DBNode.java:50) >>>>>>>> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) >>>>>>>> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) >>>>>>>> at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) >>>>>>>> at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) >>>>>>>> at org.basex.query.expr.Arr.inline(Arr.java:69) >>>>>>>> at org.basex.query.expr.path.Path.inline(Path.java:962) >>>>>>>> at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) >>>>>>>> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) >>>>>>>> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) >>>>>>>> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) >>>>>>>> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) >>>>>>>> at org.basex.query.expr.Try.inline(Try.java:106) >>>>>>>> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) >>>>>>>> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) >>>>>>>> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) >>>>>>>> at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) >>>>>>>> at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) >>>>>>>> at org.basex.query.scope.MainModule.comp(MainModule.java:81) >>>>>>>> at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) >>>>>>>> at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) >>>>>>>> at org.basex.query.QueryContext.compile(QueryContext.java:312) >>>>>>>> at org.basex.query.QueryContext.iter(QueryContext.java:331) >>>>>>>> at >>>>>>>> org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) >>>>>>>> at org.basex.http.web.WebResponse.create(WebResponse.java:63) >>>>>>>> at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) >>>>>>>> at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) >>>>>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) >>>>>>>> at >>>>>>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) >>>>>>>> at >>>>>>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) >>>>>>>> at >>>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) >>>>>>>> at >>>>>>>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) >>>>>>>> at >>>>>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) >>>>>>>> at >>>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) >>>>>>>> at >>>>>>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) >>>>>>>> at >>>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) >>>>>>>> at >>>>>>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) >>>>>>>> at >>>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) >>>>>>>> at >>>>>>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) >>>>>>>> at >>>>>>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) >>>>>>>> at >>>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) >>>>>>>> at >>>>>>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) >>>>>>>> at >>>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) >>>>>>>> at >>>>>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) >>>>>>>> at org.eclipse.jetty.server.Server.handle(Server.java:505) >>>>>>>> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) >>>>>>>> at >>>>>>>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) >>>>>>>> at >>>>>>>> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) >>>>>>>> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) >>>>>>>> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) >>>>>>>> at >>>>>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) >>>>>>>> at >>>>>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) >>>>>>>> at >>>>>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) >>>>>>>> at >>>>>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) >>>>>>>> at >>>>>>>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) >>>>>>>> at >>>>>>>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) >>>>>>>> at >>>>>>>> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) >>>>>>>> at java.base/java.lang.Thread.run(Thread.java:835) >>>>>>>>

> > Is it automatically available shipped in docker? >

The current scenario where to put it immediately at test is based on the > docker distribution and it's taking too much time at the moment to setup > a different test env. > I've received your answer about concurrent XQuery execution. I'm > currently elaborating my observations on this. Hop to be able to > formulate them during WE. > Cheers, > Marco. > > On 26/09/19 13:55, Christian Grün wrote: > > Hi Marco, > > > > Finally some news on the storage bug you reported: It is fixed with > > the latest snapshot [1]. > > > > Looking forward to your tests (and I hope you received my response on > > parallel querying), > > Christian > > > > [1] http://files.basex.org/releases/latest/ > > > > > > > > On Tue, Aug 6, 2019 at 9:38 AM Christian Grün <christian.gruen at gmail.com> > wrote: > >> Hi Marco, > >> > >> I think that the bug fix (which is still on my todo list) will be made > >> available with BaseX 9.3; so, for now, it’s probably better to choose > >> the workaround. > >> > >> Cheers, > >> Christian > >> > >> > >> > >> On Sat, Aug 3, 2019 at 10:02 AM Marco Lettere <m.lettere at gmail.com> > wrote: > >>> Hi Christian, > >>> I'm currently preparing a deployment based on Docker for one of our > >>> customers. Here in Italy it's holiday time in August so I have a bit of > >>> time and I can coope with your suggested workaround for the moment. > >>> Just a question to be prepared ... are intermediate bug-fix releases > >>> also available in the form of docker containers? > >>> Thanks for your support as usual. > >>> Regards, > >>> Marco. > >>> > >>> On 02/08/19 15:04, Christian Grün wrote: > >>>> Hi Marco, > >>>> > >>>> A little update on the status quo: > >>>> > >>>> • I noticed that the bug you observed was basically exposed by the > >>>> previous bug fix (it existed before, but it didn’t occur in your > >>>> particular setting). > >>>> • It happens only in certain cases (i.e., for specific document sizes) > >>>> and if the added/replaced document has namespaces. > >>>> • Another good thing: It happens only if your database is empty. > >>>> > >>>> It’s not exactly an elegant proposal, but as long as we haven’t fixed > >>>> the bug, you could add an initial <dummy/> document to your database. > >>>> > >>>> We are doing our best, though. > >>>> Christian > >>>> > >>>> > >>>> > >>>> On Wed, Jul 31, 2019 at 10:20 AM Christian Grün > >>>> <christian.gruen at gmail.com> wrote: > >>>>> Marco, thanks for the attachment. I have created a script that > >>>>> triggers the error [1]. Most probably, the bug is related to the > >>>>> previous bug issue. We’ll have a look at this soon. – Christian > >>>>> > >>>>> [1] https://github.com/BaseXdb/basex/issues/1711 > >>>>> > >>>>> > >>>>> > >>>>> On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere <m.lettere at gmail.com> > wrote: > >>>>>> Thanks Christian, > >>>>>> I haven't been able to reproduce the bug with your SSCE. > >>>>>> Nevertheless I spent some time in tracing the operations with the > actual > >>>>>> data I'm currently storing into the DB and a sequence of > >>>>>> db:add/db:replace close enough to actual app flow. > >>>>>> I attach the query script which should be rather self explaining. > >>>>>> You may run the script changing always $f with $f + 1 starting > (from 0). > >>>>>> By uncommenting the different configurations of the variable $input > you > >>>>>> can see how the misbehaviour depends somehow on the data because > only > >>>>>> $d8 is causing the crash. > >>>>>> I stared at the diffs from $d8 to the other but there really isn't > any > >>>>>> significant difference so now it's very hard for me to understand. > >>>>>> > >>>>>> Thank you in advance for all support that you can provide as usual. > >>>>>> Regards, > >>>>>> Marco. > >>>>>> > >>>>>> On 26/07/19 18:41, Christian Grün wrote: > >>>>>>> Hi Marco, > >>>>>>> > >>>>>>> Back in February, a user reported a database inconsistency [1] – > which > >>>>>>> happened, too, if a database was nearly empty – and we managed to > >>>>>>> build a little text case to get this reproduced [2]. After that, > 9.2 > >>>>>>> was released. Maybe this fix introduced another irregularity for > this > >>>>>>> special case? > >>>>>>> > >>>>>>> Maybe we can get this reproduced by taking the script from issue > 1662 > >>>>>>> and modify it a little? That would be great. > >>>>>>> > >>>>>>> Sorry to the inconvenience, > >>>>>>> Christian > >>>>>>> > >>>>>>> PS: You can safely ignore the "Creating database..." output. It may > >>>>>>> just indicate that an XML document is parsed, and that an internal > >>>>>>> main-memory database instance is created, possibly for your > users.xml > >>>>>>> file in the data directory. > >>>>>>> > >>>>>>> [1] > https://www.mail-archive.com/basex-talk at mailman.uni-konstanz.de/msg11459.html > >>>>>>> [2] https://github.com/BaseXdb/basex/issues/1662 > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere <m.lettere at gmail.com> > wrote: > >>>>>>>> Hi all, > >>>>>>>> > >>>>>>>> starting with 9.2.1 we experience a strange error with a RESTXQ > API of > >>>>>>>> ours that we have been using for years. > >>>>>>>> > >>>>>>>> The typical pattern is lookup a document update it and store it > back. > >>>>>>>> > >>>>>>>> We have done this milions of time and also all the tests work > neatly. > >>>>>>>> > >>>>>>>> But when using it from inside the web application, at around the > tenth > >>>>>>>> operation we get the DB corrupted with the stack trace [1]. This > seems > >>>>>>>> to happen only when the database is nearly empty and, for sure, it > >>>>>>>> doesn't appear on 8.x series. > >>>>>>>> > >>>>>>>> It feels like some concurrency flaw is introduced somewhere but > it's > >>>>>>>> hard to say because the API is eight years in age and it never > changed > >>>>>>>> significantly. > >>>>>>>> > >>>>>>>> I know it's hard without an SSCE... but while working to insulate > this > >>>>>>>> byzantine behaviour (I've been trying for days now) maybe someone > in the > >>>>>>>> list has a clue or is able to interpret the stack trace? > >>>>>>>> > >>>>>>>> Thanks and have a nice weekend, > >>>>>>>> > >>>>>>>> Marco. > >>>>>>>> > >>>>>>>> > >>>>>>>> [1] > >>>>>>>> > >>>>>>>> Improper use? Potential bug? Your feedback is welcome: > >>>>>>>> Contact: basex-talk at mailman.uni-konstanz.de > >>>>>>>> Version: BaseX 9.3 beta > >>>>>>>> Java: Ubuntu, 13 > >>>>>>>> OS: Linux, amd64 > >>>>>>>> Stack Trace: > >>>>>>>> java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds > for length 1 > >>>>>>>> at > org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) > >>>>>>>> at > org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) > >>>>>>>> at > org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) > >>>>>>>> at org.basex.data.Data.kind(Data.java:304) > >>>>>>>> at > org.basex.query.value.node.DBNode.<init>(DBNode.java:50) > >>>>>>>> at > org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) > >>>>>>>> at > org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) > >>>>>>>> at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) > >>>>>>>> at > org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) > >>>>>>>> at org.basex.query.expr.Arr.inline(Arr.java:69) > >>>>>>>> at org.basex.query.expr.path.Path.inline(Path.java:962) > >>>>>>>> at > org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) > >>>>>>>> at > org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) > >>>>>>>> at > org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > >>>>>>>> at > org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > >>>>>>>> at > org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) > >>>>>>>> at org.basex.query.expr.Try.inline(Try.java:106) > >>>>>>>> at > org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) > >>>>>>>> at > org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > >>>>>>>> at > org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > >>>>>>>> at > org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) > >>>>>>>> at > org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) > >>>>>>>> at > org.basex.query.scope.MainModule.comp(MainModule.java:81) > >>>>>>>> at > org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) > >>>>>>>> at > org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) > >>>>>>>> at > org.basex.query.QueryContext.compile(QueryContext.java:312) > >>>>>>>> at > org.basex.query.QueryContext.iter(QueryContext.java:331) > >>>>>>>> at > >>>>>>>> > org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) > >>>>>>>> at > org.basex.http.web.WebResponse.create(WebResponse.java:63) > >>>>>>>> at > org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) > >>>>>>>> at > org.basex.http.BaseXServlet.service(BaseXServlet.java:65) > >>>>>>>> at > javax.servlet.http.HttpServlet.service(HttpServlet.java:790) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > >>>>>>>> at org.eclipse.jetty.server.Server.handle(Server.java:505) > >>>>>>>> at > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) > >>>>>>>> at > >>>>>>>> org.eclipse.jetty.io > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) > >>>>>>>> at org.eclipse.jetty.io > .FillInterest.fillable(FillInterest.java:103) > >>>>>>>> at org.eclipse.jetty.io > .ChannelEndPoint$2.run(ChannelEndPoint.java:117) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) > >>>>>>>> at java.base/java.lang.Thread.run(Thread.java:835) > >>>>>>>> > >

On 26/09/19 14:37, Christian Grün wrote: > > Is it automatically available shipped in docker? > > > I guess so ("Updated 38 minutes ago"): > > https://hub.docker.com/u/basex > > > > The current scenario where to put it immediately at test is based > on the > docker distribution and it's taking too much time at the moment to > setup > a different test env. > I've received your answer about concurrent XQuery execution. I'm > currently elaborating my observations on this. Hop to be able to > formulate them during WE. > Cheers, > Marco. > > On 26/09/19 13:55, Christian Grün wrote: > > Hi Marco, > > > > Finally some news on the storage bug you reported: It is fixed with > > the latest snapshot [1]. > > > > Looking forward to your tests (and I hope you received my > response on > > parallel querying), > > Christian > > > > [1] http://files.basex.org/releases/latest/ > > > > > > > > On Tue, Aug 6, 2019 at 9:38 AM Christian Grün > <christian.gruen at gmail.com <mailto:christian.gruen at gmail.com>> wrote: > >> Hi Marco, > >> > >> I think that the bug fix (which is still on my todo list) will > be made > >> available with BaseX 9.3; so, for now, it’s probably better to > choose > >> the workaround. > >> > >> Cheers, > >> Christian > >> > >> > >> > >> On Sat, Aug 3, 2019 at 10:02 AM Marco Lettere > <m.lettere at gmail.com <mailto:m.lettere at gmail.com>> wrote: > >>> Hi Christian, > >>> I'm currently preparing a deployment based on Docker for one > of our > >>> customers. Here in Italy it's holiday time in August so I have > a bit of > >>> time and I can coope with your suggested workaround for the > moment. > >>> Just a question to be prepared ... are intermediate bug-fix > releases > >>> also available in the form of docker containers? > >>> Thanks for your support as usual. > >>> Regards, > >>> Marco. > >>> > >>> On 02/08/19 15:04, Christian Grün wrote: > >>>> Hi Marco, > >>>> > >>>> A little update on the status quo: > >>>> > >>>> • I noticed that the bug you observed was basically exposed > by the > >>>> previous bug fix (it existed before, but it didn’t occur in your > >>>> particular setting). > >>>> • It happens only in certain cases (i.e., for specific > document sizes) > >>>> and if the added/replaced document has namespaces. > >>>> • Another good thing: It happens only if your database is empty. > >>>> > >>>> It’s not exactly an elegant proposal, but as long as we > haven’t fixed > >>>> the bug, you could add an initial <dummy/> document to your > database. > >>>> > >>>> We are doing our best, though. > >>>> Christian > >>>> > >>>> > >>>> > >>>> On Wed, Jul 31, 2019 at 10:20 AM Christian Grün > >>>> <christian.gruen at gmail.com > <mailto:christian.gruen at gmail.com>> wrote: > >>>>> Marco, thanks for the attachment. I have created a script that > >>>>> triggers the error [1]. Most probably, the bug is related to the > >>>>> previous bug issue. We’ll have a look at this soon. – Christian > >>>>> > >>>>> [1] https://github.com/BaseXdb/basex/issues/1711 > >>>>> > >>>>> > >>>>> > >>>>> On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere > <m.lettere at gmail.com <mailto:m.lettere at gmail.com>> wrote: > >>>>>> Thanks Christian, > >>>>>> I haven't been able to reproduce the bug with your SSCE. > >>>>>> Nevertheless I spent some time in tracing the operations > with the actual > >>>>>> data I'm currently storing into the DB and a sequence of > >>>>>> db:add/db:replace close enough to actual app flow. > >>>>>> I attach the query script which should be rather self > explaining. > >>>>>> You may run the script changing always $f with $f + 1 > starting (from 0). > >>>>>> By uncommenting the different configurations of the > variable $input you > >>>>>> can see how the misbehaviour depends somehow on the data > because only > >>>>>> $d8 is causing the crash. > >>>>>> I stared at the diffs from $d8 to the other but there > really isn't any > >>>>>> significant difference so now it's very hard for me to > understand. > >>>>>> > >>>>>> Thank you in advance for all support that you can provide > as usual. > >>>>>> Regards, > >>>>>> Marco. > >>>>>> > >>>>>> On 26/07/19 18:41, Christian Grün wrote: > >>>>>>> Hi Marco, > >>>>>>> > >>>>>>> Back in February, a user reported a database inconsistency > [1] – which > >>>>>>> happened, too, if a database was nearly empty – and we > managed to > >>>>>>> build a little text case to get this reproduced [2]. After > that, 9.2 > >>>>>>> was released. Maybe this fix introduced another > irregularity for this > >>>>>>> special case? > >>>>>>> > >>>>>>> Maybe we can get this reproduced by taking the script from > issue 1662 > >>>>>>> and modify it a little? That would be great. > >>>>>>> > >>>>>>> Sorry to the inconvenience, > >>>>>>> Christian > >>>>>>> > >>>>>>> PS: You can safely ignore the "Creating database..." > output. It may > >>>>>>> just indicate that an XML document is parsed, and that an > internal > >>>>>>> main-memory database instance is created, possibly for > your users.xml > >>>>>>> file in the data directory. > >>>>>>> > >>>>>>> [1] > https://www.mail-archive.com/basex-talk at mailman.uni-konstanz.de/msg11459.html > >>>>>>> [2] https://github.com/BaseXdb/basex/issues/1662 > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere > <m.lettere at gmail.com <mailto:m.lettere at gmail.com>> wrote: > >>>>>>>> Hi all, > >>>>>>>> > >>>>>>>> starting with 9.2.1 we experience a strange error with a > RESTXQ API of > >>>>>>>> ours that we have been using for years. > >>>>>>>> > >>>>>>>> The typical pattern is lookup a document update it and > store it back. > >>>>>>>> > >>>>>>>> We have done this milions of time and also all the tests > work neatly. > >>>>>>>> > >>>>>>>> But when using it from inside the web application, at > around the tenth > >>>>>>>> operation we get the DB corrupted with the stack trace > [1]. This seems > >>>>>>>> to happen only when the database is nearly empty and, for > sure, it > >>>>>>>> doesn't appear on 8.x series. > >>>>>>>> > >>>>>>>> It feels like some concurrency flaw is introduced > somewhere but it's > >>>>>>>> hard to say because the API is eight years in age and it > never changed > >>>>>>>> significantly. > >>>>>>>> > >>>>>>>> I know it's hard without an SSCE... but while working to > insulate this > >>>>>>>> byzantine behaviour (I've been trying for days now) maybe > someone in the > >>>>>>>> list has a clue or is able to interpret the stack trace? > >>>>>>>> > >>>>>>>> Thanks and have a nice weekend, > >>>>>>>> > >>>>>>>> Marco. > >>>>>>>> > >>>>>>>> > >>>>>>>> [1] > >>>>>>>> > >>>>>>>> Improper use? Potential bug? Your feedback is welcome: > >>>>>>>> Contact: basex-talk at mailman.uni-konstanz.de > <mailto:basex-talk at mailman.uni-konstanz.de> > >>>>>>>> Version: BaseX 9.3 beta > >>>>>>>> Java: Ubuntu, 13 > >>>>>>>> OS: Linux, amd64 > >>>>>>>> Stack Trace: > >>>>>>>> java.lang.ArrayIndexOutOfBoundsException: Index 1 out of > bounds for length 1 > >>>>>>>> at > org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) > >>>>>>>> at > org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) > >>>>>>>> at > org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) > >>>>>>>> at org.basex.data.Data.kind(Data.java:304) > >>>>>>>> at > org.basex.query.value.node.DBNode.<init>(DBNode.java:50) > >>>>>>>> at > org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) > >>>>>>>> at > org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) > >>>>>>>> at > org.basex.query.func.db.DbOpen.value(DbOpen.java:19) > >>>>>>>> at > org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) > >>>>>>>> at org.basex.query.expr.Arr.inline(Arr.java:69) > >>>>>>>> at > org.basex.query.expr.path.Path.inline(Path.java:962) > >>>>>>>> at > org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) > >>>>>>>> at > org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) > >>>>>>>> at > org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > >>>>>>>> at > org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > >>>>>>>> at > org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) > >>>>>>>> at org.basex.query.expr.Try.inline(Try.java:106) > >>>>>>>> at > org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) > >>>>>>>> at > org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > >>>>>>>> at > org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > >>>>>>>> at > org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) > >>>>>>>> at > org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) > >>>>>>>> at > org.basex.query.scope.MainModule.comp(MainModule.java:81) > >>>>>>>> at > org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) > >>>>>>>> at > org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) > >>>>>>>> at > org.basex.query.QueryContext.compile(QueryContext.java:312) > >>>>>>>> at > org.basex.query.QueryContext.iter(QueryContext.java:331) > >>>>>>>> at > >>>>>>>> > org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) > >>>>>>>> at > org.basex.http.web.WebResponse.create(WebResponse.java:63) > >>>>>>>> at > org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) > >>>>>>>> at > org.basex.http.BaseXServlet.service(BaseXServlet.java:65) > >>>>>>>> at > javax.servlet.http.HttpServlet.service(HttpServlet.java:790) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > >>>>>>>> at > org.eclipse.jetty.server.Server.handle(Server.java:505) > >>>>>>>> at > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) > >>>>>>>> at > >>>>>>>> org.eclipse.jetty.io > <http://org.eclipse.jetty.io>.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) > >>>>>>>> at org.eclipse.jetty.io > <http://org.eclipse.jetty.io>.FillInterest.fillable(FillInterest.java:103) > >>>>>>>> at org.eclipse.jetty.io > <http://org.eclipse.jetty.io>.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) > >>>>>>>> at > >>>>>>>> > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) > >>>>>>>> at java.base/java.lang.Thread.run(Thread.java:835) > >>>>>>>> >