Database currupted bug when trying to re-insert documents
Thursday, 7 February 2019 · 10 messages · last reply 11 Feb 2019
HM
Herbert Mertig7 Feb 2019 11:05 #
I stumbled over this while running some JUnit tests of our Applications which uses the embedded version of BaseX. It always seemed to happen when inserting, deleting and then inserting the same documents again. I attached 4 simple xquery files to reproduce it using basexgui, just follow these steps:
1. open basexgui
2. create an empty database
3. run insert_pigs.xq to insert the first document
4. run insert_plants.xq to insert the second document
5. run delete_pigs.xq to delete the first document
6. run delete_plants.xq to delete the second document
now insert the same 2 documents in the opposite order
7: run insert_pigs.xq
8. run insert_plants.xq <-- this will trigger the error, the database is now garbage
Some more observations:
It does not trigger when using only 1 document. Not sure if it happens with more than 2, I think I have observed that as well. Sometimes closing the database and running optimize on it seems to fix it.
Hope that helps to find and fiy the bug, its pretty nasty.
BR
Herbert
CG
Christian Grün7 Feb 2019 12:36 #
Hi Herbert,
First, thanks for the step-by-step procedure.
Quoted text · Herbert Mertig · 3 lines
> now insert the same 2 documents in the opposite order > 7: run insert_pigs.xq > 8. run insert_plants.xq <-- this will trigger the error, the database is now garbage
My database is still valid after these steps (I can run queries and I can export the full database). I also tried to swap step 7 and 8 (that would probably be the opposite order you mentioned?).
This is what I did:
1a. download and unzip BaseX91.zip
1b. start bin/basexgui from the unzipped directory
2. create an empty TEST database via the GUI dialog (and, in the second run, by running
db:create('TEST').
3. run ...etc
Some question in return:
• How do you detect that your database is garbage after the last step?
• Does this also happen with a fresh installation of BaseX (did you change the default options)?
• I made no difference in my tests, but how did you create the database? Did you use the GUI features, or did you run
db:create('TEST') ?
• Could you experience the same behavior on command line?
Cheers
ChristianHM
Herbert Mertig7 Feb 2019 14:25 #
Hello,
well I am using the latest version 9.1.2 which I downloaded a couple of days ago. I tried it both on Windows 10 and on Ubuntu 18.04, it shows the same behaviour. (sorry, I should have added these detail in the original email already...). The error first appeared way back with version 8.something, but I was unable to pinpoint it as it was very sporadic. I attached a screenshot of how it looks like in basexgui right after step
8. I can reproduce it here anytime. I also tried creating the database via the GUI as well as using xquery, I also played with
UPDINDEX and AUTOOPTIMIZE - it did not make any difference.
I also attached the broken databases, each with different indexing options as described.
From our Java application the error looks looks this, it is always at the very first query attempt after the databasse has corrupted:
Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk at mailman.uni-konstanz.de
Version: BaseX 9.1.2
Java: Oracle Corporation, 1.8.0_162
OS: Windows 10, amd64
Stack Trace:java.lang.ArrayIndexOutOfBoundsException: 2147483647
at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:502)
at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:453)
at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:151)
at org.basex.data.Data.kind(Data.java:304)
at org.basex.query.value.node.DBNode.<init>(DBNode.java:50)
at org.basex.query.value.seq.DBNodeSeq.itemAt(DBNodeSeq.java:70)
at org.basex.query.value.seq.DBNodeSeq.itemAt(DBNodeSeq.java:1)
at org.basex.query.value.seq.Seq$1.get(Seq.java:95)
at org.basex.query.iter.BasicIter.next(BasicIter.java:38)
at org.basex.query.expr.path.CachedPath.nodeIter(CachedPath.java:42)
at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:66)
at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:37)
at org.basex.query.func.fn.FnCount.item(FnCount.java:25)
at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:48)
at org.basex.query.expr.constr.Constr.add(Constr.java:70)
at org.basex.query.expr.constr.CElem.item(CElem.java:93)
at org.basex.query.expr.constr.CElem.item(CElem.java:1)
at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:48)
at org.basex.query.expr.constr.Constr.add(Constr.java:70)
at org.basex.query.expr.constr.CElem.item(CElem.java:93)
at org.basex.query.expr.constr.CElem.item(CElem.java:1)
at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71)
at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:175)
at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:88)
at org.basex.query.QueryContext.next(QueryContext.java:400)
at org.basex.query.scope.MainModule$1.next(MainModule.java:122)
at org.basex.core.cmd.AQuery.query(AQuery.java:111)
at org.basex.core.cmd.XQuery.run(XQuery.java:22)
at org.basex.core.Command.run(Command.java:257)
at org.basex.core.Command.execute(Command.java:93)
at org.basex.core.Command.execute(Command.java:116)Am Do., 7. Feb. 2019 um 12:36 Uhr schrieb Christian Grün <
christian.gruen at gmail.com>:
Quoted text · Christian Grün · 33 lines
> Hi Herbert, > > First, thanks for the step-by-step procedure. > > > now insert the same 2 documents in the opposite order > > 7: run insert_pigs.xq > > 8. run insert_plants.xq <-- this will trigger the error, the database is > now garbage > > My database is still valid after these steps (I can run queries and I > can export the full database). I also tried to swap step 7 and 8 (that > would probably be the opposite order you mentioned?). > > This is what I did: > 1a. download and unzip BaseX91.zip > 1b. start bin/basexgui from the unzipped directory > 2. create an empty TEST database via the GUI dialog (and, in the > second run, by running db:create('TEST'). > 3. run ...etc > > Some question in return: > > • How do you detect that your database is garbage after the last step? > • Does this also happen with a fresh installation of BaseX (did you > change the default options)? > • I made no difference in my tests, but how did you create the > database? Did you use the GUI features, or did you run > db:create('TEST') ? > • Could you experience the same behavior on command line? > > Cheers > Christian >
CG
Christian Grün7 Feb 2019 17:07 #
I still have no chance to get this reproduced. I would assume that the attached BaseX command script fails on your system, too. Could you execute it and see what happens?
Quoted text · Herbert Mertig · 1 line
> basex script.bxs
You can either run it on command line or in the BaseX GUI (you may need to update the file paths if you launch BaseX from a different directory).
If the script runs successfully, we may have tested different things. Could you rewrite the script and send me a version that fails on your system?
Quoted text · Herbert Mertig · 84 lines
On Thu, Feb 7, 2019 at 2:25 PM Herbert Mertig <h.mertig at gmail.com> wrote: > > Hello, > > well I am using the latest version 9.1.2 which I downloaded a couple of days ago. I tried it both on Windows 10 and on Ubuntu 18.04, it shows the same behaviour. (sorry, I should have added these detail in the original email already...). The error first appeared way back with version 8.something, but I was unable to pinpoint it as it was very sporadic. > I attached a screenshot of how it looks like in basexgui right after step 8. I can reproduce it here anytime. I also tried creating the database via the GUI as well as using xquery, I also played with UPDINDEX and AUTOOPTIMIZE - it did not make any difference. > I also attached the broken databases, each with different indexing options as described. > > From our Java application the error looks looks this, it is always at the very first query attempt after the databasse has corrupted: > > Improper use? Potential bug? Your feedback is welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 9.1.2 > Java: Oracle Corporation, 1.8.0_162 > OS: Windows 10, amd64 > Stack Trace: > java.lang.ArrayIndexOutOfBoundsException: 2147483647 > at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:502) > at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:453) > at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:151) > at org.basex.data.Data.kind(Data.java:304) > at org.basex.query.value.node.DBNode.<init>(DBNode.java:50) > at org.basex.query.value.seq.DBNodeSeq.itemAt(DBNodeSeq.java:70) > at org.basex.query.value.seq.DBNodeSeq.itemAt(DBNodeSeq.java:1) > at org.basex.query.value.seq.Seq$1.get(Seq.java:95) > at org.basex.query.iter.BasicIter.next(BasicIter.java:38) > at org.basex.query.expr.path.CachedPath.nodeIter(CachedPath.java:42) > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:66) > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:37) > at org.basex.query.func.fn.FnCount.item(FnCount.java:25) > at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:48) > at org.basex.query.expr.constr.Constr.add(Constr.java:70) > at org.basex.query.expr.constr.CElem.item(CElem.java:93) > at org.basex.query.expr.constr.CElem.item(CElem.java:1) > at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:48) > at org.basex.query.expr.constr.Constr.add(Constr.java:70) > at org.basex.query.expr.constr.CElem.item(CElem.java:93) > at org.basex.query.expr.constr.CElem.item(CElem.java:1) > at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71) > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:175) > at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:88) > at org.basex.query.QueryContext.next(QueryContext.java:400) > at org.basex.query.scope.MainModule$1.next(MainModule.java:122) > at org.basex.core.cmd.AQuery.query(AQuery.java:111) > at org.basex.core.cmd.XQuery.run(XQuery.java:22) > at org.basex.core.Command.run(Command.java:257) > at org.basex.core.Command.execute(Command.java:93) > at org.basex.core.Command.execute(Command.java:116) > > > > Am Do., 7. Feb. 2019 um 12:36 Uhr schrieb Christian Grün <christian.gruen at gmail.com>: >> >> Hi Herbert, >> >> First, thanks for the step-by-step procedure. >> >> > now insert the same 2 documents in the opposite order >> > 7: run insert_pigs.xq >> > 8. run insert_plants.xq <-- this will trigger the error, the database is now garbage >> >> My database is still valid after these steps (I can run queries and I >> can export the full database). I also tried to swap step 7 and 8 (that >> would probably be the opposite order you mentioned?). >> >> This is what I did: >> 1a. download and unzip BaseX91.zip >> 1b. start bin/basexgui from the unzipped directory >> 2. create an empty TEST database via the GUI dialog (and, in the >> second run, by running db:create('TEST'). >> 3. run ...etc >> >> Some question in return: >> >> • How do you detect that your database is garbage after the last step? >> • Does this also happen with a fresh installation of BaseX (did you >> change the default options)? >> • I made no difference in my tests, but how did you create the >> database? Did you use the GUI features, or did you run >> db:create('TEST') ? >> • Could you experience the same behavior on command line? >> >> Cheers >> Christian
HM
Herbert Mertig7 Feb 2019 17:19 #
I tried the commandline basex both manually and with your script, it works fine there no matter how often I try, not sure why...
BR
Herbert
Am Do., 7. Feb. 2019 um 17:07 Uhr schrieb Christian Grün <
christian.gruen at gmail.com>:
Quoted text · Christian Grün · 112 lines
> I still have no chance to get this reproduced. I would assume that the > attached BaseX command script fails on your system, too. Could you > execute it and see what happens? > > > basex script.bxs > > You can either run it on command line or in the BaseX GUI (you may > need to update the file paths if you launch BaseX from a different > directory). > > If the script runs successfully, we may have tested different things. > Could you rewrite the script and send me a version that fails on your > system? > > > > On Thu, Feb 7, 2019 at 2:25 PM Herbert Mertig <h.mertig at gmail.com> wrote: > > > > Hello, > > > > well I am using the latest version 9.1.2 which I downloaded a couple of > days ago. I tried it both on Windows 10 and on Ubuntu 18.04, it shows the > same behaviour. (sorry, I should have added these detail in the original > email already...). The error first appeared way back with version > 8.something, but I was unable to pinpoint it as it was very sporadic. > > I attached a screenshot of how it looks like in basexgui right after > step 8. I can reproduce it here anytime. I also tried creating the database > via the GUI as well as using xquery, I also played with UPDINDEX and > AUTOOPTIMIZE - it did not make any difference. > > I also attached the broken databases, each with different indexing > options as described. > > > > From our Java application the error looks looks this, it is always at > the very first query attempt after the databasse has corrupted: > > > > Improper use? Potential bug? Your feedback is welcome: > > Contact: basex-talk at mailman.uni-konstanz.de > > Version: BaseX 9.1.2 > > Java: Oracle Corporation, 1.8.0_162 > > OS: Windows 10, amd64 > > Stack Trace: > > java.lang.ArrayIndexOutOfBoundsException: 2147483647 > > at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:502) > > at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:453) > > at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:151) > > at org.basex.data.Data.kind(Data.java:304) > > at org.basex.query.value.node.DBNode.<init>(DBNode.java:50) > > at org.basex.query.value.seq.DBNodeSeq.itemAt(DBNodeSeq.java:70) > > at org.basex.query.value.seq.DBNodeSeq.itemAt(DBNodeSeq.java:1) > > at org.basex.query.value.seq.Seq$1.get(Seq.java:95) > > at org.basex.query.iter.BasicIter.next(BasicIter.java:38) > > at org.basex.query.expr.path.CachedPath.nodeIter(CachedPath.java:42) > > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:66) > > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:37) > > at org.basex.query.func.fn.FnCount.item(FnCount.java:25) > > at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:48) > > at org.basex.query.expr.constr.Constr.add(Constr.java:70) > > at org.basex.query.expr.constr.CElem.item(CElem.java:93) > > at org.basex.query.expr.constr.CElem.item(CElem.java:1) > > at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:48) > > at org.basex.query.expr.constr.Constr.add(Constr.java:70) > > at org.basex.query.expr.constr.CElem.item(CElem.java:93) > > at org.basex.query.expr.constr.CElem.item(CElem.java:1) > > at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71) > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:175) > > at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:88) > > at org.basex.query.QueryContext.next(QueryContext.java:400) > > at org.basex.query.scope.MainModule$1.next(MainModule.java:122) > > at org.basex.core.cmd.AQuery.query(AQuery.java:111) > > at org.basex.core.cmd.XQuery.run(XQuery.java:22) > > at org.basex.core.Command.run(Command.java:257) > > at org.basex.core.Command.execute(Command.java:93) > > at org.basex.core.Command.execute(Command.java:116) > > > > > > > > Am Do., 7. Feb. 2019 um 12:36 Uhr schrieb Christian Grün < > christian.gruen at gmail.com>: > >> > >> Hi Herbert, > >> > >> First, thanks for the step-by-step procedure. > >> > >> > now insert the same 2 documents in the opposite order > >> > 7: run insert_pigs.xq > >> > 8. run insert_plants.xq <-- this will trigger the error, the database > is now garbage > >> > >> My database is still valid after these steps (I can run queries and I > >> can export the full database). I also tried to swap step 7 and 8 (that > >> would probably be the opposite order you mentioned?). > >> > >> This is what I did: > >> 1a. download and unzip BaseX91.zip > >> 1b. start bin/basexgui from the unzipped directory > >> 2. create an empty TEST database via the GUI dialog (and, in the > >> second run, by running db:create('TEST'). > >> 3. run ...etc > >> > >> Some question in return: > >> > >> • How do you detect that your database is garbage after the last step? > >> • Does this also happen with a fresh installation of BaseX (did you > >> change the default options)? > >> • I made no difference in my tests, but how did you create the > >> database? Did you use the GUI features, or did you run > >> db:create('TEST') ? > >> • Could you experience the same behavior on command line? > >> > >> Cheers > >> Christian >
CG
Christian Grün7 Feb 2019 17:21 #
Quoted text · Herbert Mertig · 1 line
> I tried the commandline basex both manually and with your script, it works fine there no matter how often I try, not sure why...
Good to know! Once we have a script that fails on one of your systems,it will be easy to get your use case reproduced on other systems.
HM
Ok, this version of the script you sent me triggers it both in the gui as well as in the commandline version.
(I just changed the order a bit)
BR
Herbert
Am Do., 7. Feb. 2019 um 17:19 Uhr schrieb Herbert Mertig <h.mertig at gmail.com
Quoted text · Herbert Mertig · 123 lines
>: > I tried the commandline basex both manually and with your script, it works > fine there no matter how often I try, not sure why... > > BR > Herbert > > Am Do., 7. Feb. 2019 um 17:07 Uhr schrieb Christian Grün < > christian.gruen at gmail.com>: > >> I still have no chance to get this reproduced. I would assume that the >> attached BaseX command script fails on your system, too. Could you >> execute it and see what happens? >> >> > basex script.bxs >> >> You can either run it on command line or in the BaseX GUI (you may >> need to update the file paths if you launch BaseX from a different >> directory). >> >> If the script runs successfully, we may have tested different things. >> Could you rewrite the script and send me a version that fails on your >> system? >> >> >> >> On Thu, Feb 7, 2019 at 2:25 PM Herbert Mertig <h.mertig at gmail.com> wrote: >> > >> > Hello, >> > >> > well I am using the latest version 9.1.2 which I downloaded a couple of >> days ago. I tried it both on Windows 10 and on Ubuntu 18.04, it shows the >> same behaviour. (sorry, I should have added these detail in the original >> email already...). The error first appeared way back with version >> 8.something, but I was unable to pinpoint it as it was very sporadic. >> > I attached a screenshot of how it looks like in basexgui right after >> step 8. I can reproduce it here anytime. I also tried creating the database >> via the GUI as well as using xquery, I also played with UPDINDEX and >> AUTOOPTIMIZE - it did not make any difference. >> > I also attached the broken databases, each with different indexing >> options as described. >> > >> > From our Java application the error looks looks this, it is always at >> the very first query attempt after the databasse has corrupted: >> > >> > Improper use? Potential bug? Your feedback is welcome: >> > Contact: basex-talk at mailman.uni-konstanz.de >> > Version: BaseX 9.1.2 >> > Java: Oracle Corporation, 1.8.0_162 >> > OS: Windows 10, amd64 >> > Stack Trace: >> > java.lang.ArrayIndexOutOfBoundsException: 2147483647 >> > at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:502) >> > at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:453) >> > at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:151) >> > at org.basex.data.Data.kind(Data.java:304) >> > at org.basex.query.value.node.DBNode.<init>(DBNode.java:50) >> > at org.basex.query.value.seq.DBNodeSeq.itemAt(DBNodeSeq.java:70) >> > at org.basex.query.value.seq.DBNodeSeq.itemAt(DBNodeSeq.java:1) >> > at org.basex.query.value.seq.Seq$1.get(Seq.java:95) >> > at org.basex.query.iter.BasicIter.next(BasicIter.java:38) >> > at org.basex.query.expr.path.CachedPath.nodeIter(CachedPath.java:42) >> > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:66) >> > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:37) >> > at org.basex.query.func.fn.FnCount.item(FnCount.java:25) >> > at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:48) >> > at org.basex.query.expr.constr.Constr.add(Constr.java:70) >> > at org.basex.query.expr.constr.CElem.item(CElem.java:93) >> > at org.basex.query.expr.constr.CElem.item(CElem.java:1) >> > at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:48) >> > at org.basex.query.expr.constr.Constr.add(Constr.java:70) >> > at org.basex.query.expr.constr.CElem.item(CElem.java:93) >> > at org.basex.query.expr.constr.CElem.item(CElem.java:1) >> > at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71) >> > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:175) >> > at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:88) >> > at org.basex.query.QueryContext.next(QueryContext.java:400) >> > at org.basex.query.scope.MainModule$1.next(MainModule.java:122) >> > at org.basex.core.cmd.AQuery.query(AQuery.java:111) >> > at org.basex.core.cmd.XQuery.run(XQuery.java:22) >> > at org.basex.core.Command.run(Command.java:257) >> > at org.basex.core.Command.execute(Command.java:93) >> > at org.basex.core.Command.execute(Command.java:116) >> > >> > >> > >> > Am Do., 7. Feb. 2019 um 12:36 Uhr schrieb Christian Grün < >> christian.gruen at gmail.com>: >> >> >> >> Hi Herbert, >> >> >> >> First, thanks for the step-by-step procedure. >> >> >> >> > now insert the same 2 documents in the opposite order >> >> > 7: run insert_pigs.xq >> >> > 8. run insert_plants.xq <-- this will trigger the error, the >> database is now garbage >> >> >> >> My database is still valid after these steps (I can run queries and I >> >> can export the full database). I also tried to swap step 7 and 8 (that >> >> would probably be the opposite order you mentioned?). >> >> >> >> This is what I did: >> >> 1a. download and unzip BaseX91.zip >> >> 1b. start bin/basexgui from the unzipped directory >> >> 2. create an empty TEST database via the GUI dialog (and, in the >> >> second run, by running db:create('TEST'). >> >> 3. run ...etc >> >> >> >> Some question in return: >> >> >> >> • How do you detect that your database is garbage after the last step? >> >> • Does this also happen with a fresh installation of BaseX (did you >> >> change the default options)? >> >> • I made no difference in my tests, but how did you create the >> >> database? Did you use the GUI features, or did you run >> >> db:create('TEST') ? >> >> • Could you experience the same behavior on command line? >> >> >> >> Cheers >> >> Christian >> >
CG
Christian Grün7 Feb 2019 18:08 #
Dear Herbert,
Thanks for your patience. I created a new issue, we’ll look at this soon [1].
Best,
Christian
[1] https://github.com/BaseXdb/basex/issues/1662
Quoted text · Herbert Mertig · 117 lines
On Thu, Feb 7, 2019 at 5:29 PM Herbert Mertig <h.mertig at gmail.com> wrote: > > Ok, this version of the script you sent me triggers it both in the gui as well as in the commandline version. > (I just changed the order a bit) > > BR > Herbert > > Am Do., 7. Feb. 2019 um 17:19 Uhr schrieb Herbert Mertig <h.mertig at gmail.com>: >> >> I tried the commandline basex both manually and with your script, it works fine there no matter how often I try, not sure why... >> >> BR >> Herbert >> >> Am Do., 7. Feb. 2019 um 17:07 Uhr schrieb Christian Grün <christian.gruen at gmail.com>: >>> >>> I still have no chance to get this reproduced. I would assume that the >>> attached BaseX command script fails on your system, too. Could you >>> execute it and see what happens? >>> >>> > basex script.bxs >>> >>> You can either run it on command line or in the BaseX GUI (you may >>> need to update the file paths if you launch BaseX from a different >>> directory). >>> >>> If the script runs successfully, we may have tested different things. >>> Could you rewrite the script and send me a version that fails on your >>> system? >>> >>> >>> >>> On Thu, Feb 7, 2019 at 2:25 PM Herbert Mertig <h.mertig at gmail.com> wrote: >>> > >>> > Hello, >>> > >>> > well I am using the latest version 9.1.2 which I downloaded a couple of days ago. I tried it both on Windows 10 and on Ubuntu 18.04, it shows the same behaviour. (sorry, I should have added these detail in the original email already...). The error first appeared way back with version 8.something, but I was unable to pinpoint it as it was very sporadic. >>> > I attached a screenshot of how it looks like in basexgui right after step 8. I can reproduce it here anytime. I also tried creating the database via the GUI as well as using xquery, I also played with UPDINDEX and AUTOOPTIMIZE - it did not make any difference. >>> > I also attached the broken databases, each with different indexing options as described. >>> > >>> > From our Java application the error looks looks this, it is always at the very first query attempt after the databasse has corrupted: >>> > >>> > Improper use? Potential bug? Your feedback is welcome: >>> > Contact: basex-talk at mailman.uni-konstanz.de >>> > Version: BaseX 9.1.2 >>> > Java: Oracle Corporation, 1.8.0_162 >>> > OS: Windows 10, amd64 >>> > Stack Trace: >>> > java.lang.ArrayIndexOutOfBoundsException: 2147483647 >>> > at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:502) >>> > at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:453) >>> > at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:151) >>> > at org.basex.data.Data.kind(Data.java:304) >>> > at org.basex.query.value.node.DBNode.<init>(DBNode.java:50) >>> > at org.basex.query.value.seq.DBNodeSeq.itemAt(DBNodeSeq.java:70) >>> > at org.basex.query.value.seq.DBNodeSeq.itemAt(DBNodeSeq.java:1) >>> > at org.basex.query.value.seq.Seq$1.get(Seq.java:95) >>> > at org.basex.query.iter.BasicIter.next(BasicIter.java:38) >>> > at org.basex.query.expr.path.CachedPath.nodeIter(CachedPath.java:42) >>> > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:66) >>> > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:37) >>> > at org.basex.query.func.fn.FnCount.item(FnCount.java:25) >>> > at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:48) >>> > at org.basex.query.expr.constr.Constr.add(Constr.java:70) >>> > at org.basex.query.expr.constr.CElem.item(CElem.java:93) >>> > at org.basex.query.expr.constr.CElem.item(CElem.java:1) >>> > at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:48) >>> > at org.basex.query.expr.constr.Constr.add(Constr.java:70) >>> > at org.basex.query.expr.constr.CElem.item(CElem.java:93) >>> > at org.basex.query.expr.constr.CElem.item(CElem.java:1) >>> > at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71) >>> > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:175) >>> > at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:88) >>> > at org.basex.query.QueryContext.next(QueryContext.java:400) >>> > at org.basex.query.scope.MainModule$1.next(MainModule.java:122) >>> > at org.basex.core.cmd.AQuery.query(AQuery.java:111) >>> > at org.basex.core.cmd.XQuery.run(XQuery.java:22) >>> > at org.basex.core.Command.run(Command.java:257) >>> > at org.basex.core.Command.execute(Command.java:93) >>> > at org.basex.core.Command.execute(Command.java:116) >>> > >>> > >>> > >>> > Am Do., 7. Feb. 2019 um 12:36 Uhr schrieb Christian Grün <christian.gruen at gmail.com>: >>> >> >>> >> Hi Herbert, >>> >> >>> >> First, thanks for the step-by-step procedure. >>> >> >>> >> > now insert the same 2 documents in the opposite order >>> >> > 7: run insert_pigs.xq >>> >> > 8. run insert_plants.xq <-- this will trigger the error, the database is now garbage >>> >> >>> >> My database is still valid after these steps (I can run queries and I >>> >> can export the full database). I also tried to swap step 7 and 8 (that >>> >> would probably be the opposite order you mentioned?). >>> >> >>> >> This is what I did: >>> >> 1a. download and unzip BaseX91.zip >>> >> 1b. start bin/basexgui from the unzipped directory >>> >> 2. create an empty TEST database via the GUI dialog (and, in the >>> >> second run, by running db:create('TEST'). >>> >> 3. run ...etc >>> >> >>> >> Some question in return: >>> >> >>> >> • How do you detect that your database is garbage after the last step? >>> >> • Does this also happen with a fresh installation of BaseX (did you >>> >> change the default options)? >>> >> • I made no difference in my tests, but how did you create the >>> >> database? Did you use the GUI features, or did you run >>> >> db:create('TEST') ? >>> >> • Could you experience the same behavior on command line? >>> >> >>> >> Cheers >>> >> Christian
CG
Hi Herbert,
The problem you encountered has been fixed. If all documents were deleted from a database in a specific order, the emptied disk block mapping was not correctly reinitialized.
This has been fixed with the latest snapshot [1]. Don’t hesitate and come back to us if you should still encounter unexpected surprises.
Thanks,
Christian
[1] http://files.basex.org/releases/latest/
Quoted text · Herbert Mertig · 23 lines
On Thu, Feb 7, 2019 at 11:06 AM Herbert Mertig <h.mertig at gmail.com> wrote: > > I stumbled over this while running some JUnit tests of our Applications which uses the embedded version of BaseX. It always seemed to happen when inserting, deleting and then inserting the same documents again. > I attached 4 simple xquery files to reproduce it using basexgui, just follow these steps: > > 1. open basexgui > 2. create an empty database > 3. run insert_pigs.xq to insert the first document > 4. run insert_plants.xq to insert the second document > 5. run delete_pigs.xq to delete the first document > 6. run delete_plants.xq to delete the second document > now insert the same 2 documents in the opposite order > 7: run insert_pigs.xq > 8. run insert_plants.xq <-- this will trigger the error, the database is now garbage > > Some more observations: > It does not trigger when using only 1 document. Not sure if it happens with more than 2, I think I have observed that as well. Sometimes closing the database and running optimize on it seems to fix it. > > Hope that helps to find and fiy the bug, its pretty nasty. > > BR > Herbert >
HM
Herbert Mertig11 Feb 2019 14:53 #
Great, I will test the snapshot right away.
Good work!
BR
Herbert Mertig
Am Mo., 11. Feb. 2019 um 12:11 Uhr schrieb Christian Grün <
christian.gruen at gmail.com>:
Quoted text · Christian Grün · 46 lines
> Hi Herbert, > > The problem you encountered has been fixed. If all documents were > deleted from a database in a specific order, the emptied disk block > mapping was not correctly reinitialized. > > This has been fixed with the latest snapshot [1]. Don’t hesitate and > come back to us if you should still encounter unexpected surprises. > > Thanks, > Christian > > [1] http://files.basex.org/releases/latest/ > > > > On Thu, Feb 7, 2019 at 11:06 AM Herbert Mertig <h.mertig at gmail.com> wrote: > > > > I stumbled over this while running some JUnit tests of our Applications > which uses the embedded version of BaseX. It always seemed to happen when > inserting, deleting and then inserting the same documents again. > > I attached 4 simple xquery files to reproduce it using basexgui, just > follow these steps: > > > > 1. open basexgui > > 2. create an empty database > > 3. run insert_pigs.xq to insert the first document > > 4. run insert_plants.xq to insert the second document > > 5. run delete_pigs.xq to delete the first document > > 6. run delete_plants.xq to delete the second document > > now insert the same 2 documents in the opposite order > > 7: run insert_pigs.xq > > 8. run insert_plants.xq <-- this will trigger the error, the database is > now garbage > > > > Some more observations: > > It does not trigger when using only 1 document. Not sure if it happens > with more than 2, I think I have observed that as well. Sometimes closing > the database and running optimize on it seems to fix it. > > > > Hope that helps to find and fiy the bug, its pretty nasty. > > > > BR > > Herbert > > >
Written for BaseX 9.1. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery