Corrupt data base not recognized by inspect
Thursday, 11 January 2024 · 4 messages · last reply 15 Jan 2024
JB
Jürgen Schwenk Büro11 Jan 2024 18:07 #
Hello,
in our application we "produced" a corrupt data base caused by a power fail. On restart we check data base using the "inspect" command to detect failures, creating new empty data base if found one. The attached data base seems OK using "inspect", but accessing the data base we get an exception. Perhaps this may help to make "inspect" more reliable.
Thanks a lot,
Jürgen Schwenk
Client output:
Quoted text · 1 line
> list
Name Resources Size Input Path ----------------------------------------------------------------------------------- VentilutionOnlineDataBaseCorrupt 1 1170
VentilutionOnlineDataBase.xml
1 database(s).
Quoted text · 1 line
> open VentilutionOnlineDataBaseCorrupt
Database 'VentilutionOnlineDataBaseCorrupt' was opened in 0.03 ms.
Quoted text · 1 line
> inspect
Checking main table (152 nodes):
- 0 invalid node kinds
- 0 invalid parent references
- 0 wrong parent/descendant relationships
No inconsistencies found.
'VentilutionOnlineDataBaseCorrupt' inspected in 0.06 ms.
Quoted text · 1 line
> xquery //*
Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk at mailman.uni-konstanz.de
Version: BaseX 10.1
Java: Eclipse Adoptium, 17.0.4
OS: Windows 10, amd64
Stack Trace:
java.lang.RuntimeException: Data Access out of bounds:
- pre value: 152
- table size: 152
- first/next pre value: 0/152
- #total/used pages: 1/1
- accessed page: 0 (1 > 0]
at org.basex.util.Util.notExpected(Util.java:64)
at
org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:477) at
org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:158) at org.basex.data.Data.kind(Data.java:312)
at org.basex.query.value.node.DBNode$6.next(DBNode.java:402) at org.basex.query.value.node.DBNode$6.next(DBNode.java:1) at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38) at org.basex.query.expr.path.IterStep$1.next(IterStep.java:1) at org.basex.query.QueryContext.next(QueryContext.java:375) at org.basex.query.expr.path.IterPath$1.next(IterPath.java:48) at org.basex.query.scope.MainModule$1.next(MainModule.java:55) at org.basex.core.cmd.AQuery.run(AQuery.java:88)
at org.basex.core.Command.run(Command.java:233)
at org.basex.core.Command.execute(Command.java:93)
at org.basex.server.ClientListener.run(ClientListener.java:141)
Query "softwareProjectVentilution" executed in 0.31 ms.
--
--
Jürgen Schwenk
- R&D Engineer -
Gründler GmbH Tel: +49 (0)7441 40761-14
Jaspisstrasse 23 Fax: +49 (0)7441 40761-99
72250 Freudenstadt E-Mail: juergen.schwenk at gruendler.de
--
--
Jürgen Schwenk
- R&D Engineer -
Gründler GmbH Tel: +49 (0)7441 40761-14
Jaspisstrasse 23 Fax: +49 (0)7441 40761-99
72250 Freudenstadt E-Mail: juergen.schwenk at gruendler.de
CG
Christian Grün12 Jan 2024 14:30 #
Dear Jürgen,
Thanks for the observation.
In fact, the existing consistency checks won’t allow you to completely rule out inconsistencies in databases. If your data is not strictly confidential, we invite you to share the corrupt database instance with us, it may help us to improve our checks. In addition, it could be interesting to learn more about the update operation that was running during the power fail (provided it was logged, e.g. in the data/.logs directory).
All the best,
Christian
Quoted text · Jürgen Schwenk Büro · 100 lines
On Thu, Jan 11, 2024 at 6:07 PM Jürgen Schwenk Büro < juergen.schwenk at gruendler.de> wrote: > Hello, > > in our application we "produced" a corrupt data base caused by a power > fail. On restart we check data base using the "inspect" command to detect > failures, creating new empty data base if found one. The attached data base > seems OK using "inspect", but accessing the data base we get an exception. > Perhaps this may help to make "inspect" more reliable. > > Thanks a lot, > > Jürgen Schwenk > > > Client output: > > > list > Name Resources Size Input Path > > ----------------------------------------------------------------------------------- > VentilutionOnlineDataBaseCorrupt 1 1170 > VentilutionOnlineDataBase.xml > > 1 database(s). > > > open VentilutionOnlineDataBaseCorrupt > Database 'VentilutionOnlineDataBaseCorrupt' was opened in 0.03 ms. > > > inspect > Checking main table (152 nodes): > - 0 invalid node kinds > - 0 invalid parent references > - 0 wrong parent/descendant relationships > No inconsistencies found. > 'VentilutionOnlineDataBaseCorrupt' inspected in 0.06 ms. > > > xquery //* > Improper use? Potential bug? Your feedback is welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 10.1 > Java: Eclipse Adoptium, 17.0.4 > OS: Windows 10, amd64 > Stack Trace: > java.lang.RuntimeException: Data Access out of bounds: > - pre value: 152 > - table size: 152 > - first/next pre value: 0/152 > - #total/used pages: 1/1 > - accessed page: 0 (1 > 0] > at org.basex.util.Util.notExpected(Util.java:64) > at > org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:477) > at > org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:158) > at org.basex.data.Data.kind(Data.java:312) > at org.basex.query.value.node.DBNode$6.next(DBNode.java:402) > at org.basex.query.value.node.DBNode$6.next(DBNode.java:1) > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38) > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:1) > at org.basex.query.QueryContext.next(QueryContext.java:375) > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:48) > at org.basex.query.scope.MainModule$1.next(MainModule.java:55) > at org.basex.core.cmd.AQuery.run(AQuery.java:88) > at org.basex.core.Command.run(Command.java:233) > at org.basex.core.Command.execute(Command.java:93) > at org.basex.server.ClientListener.run(ClientListener.java:141) > > Query "softwareProjectVentilution" executed in 0.31 ms. > > > -- > > > -- > > Jürgen Schwenk > - R&D Engineer - > > > > > Gründler GmbH Tel: +49 (0)7441 40761-14 > Jaspisstrasse 23 Fax: +49 (0)7441 40761-99 > 72250 Freudenstadt E-Mail: juergen.schwenk at gruendler.de > -- > > > -- > > Jürgen Schwenk > - R&D Engineer - > > > > > Gründler GmbH Tel: +49 (0)7441 40761-14 > Jaspisstrasse 23 Fax: +49 (0)7441 40761-99 > 72250 Freudenstadt E-Mail: juergen.schwenk at gruendler.de >
CG
Dear Jürgen,
It turned out that the invocation of the INSPECT command raised an exception that was not caught by the command itself, but was only sent to stderr. We’ve revised this with the latest snapshot [1].
Best (and good luck for your power connection in 2024)
Christian
[1] https://files.basex.org/releases/latest/
Quoted text · Jürgen Schwenk Büro · 100 lines
On Thu, Jan 11, 2024 at 6:07 PM Jürgen Schwenk Büro < juergen.schwenk at gruendler.de> wrote: > Hello, > > in our application we "produced" a corrupt data base caused by a power > fail. On restart we check data base using the "inspect" command to detect > failures, creating new empty data base if found one. The attached data base > seems OK using "inspect", but accessing the data base we get an exception. > Perhaps this may help to make "inspect" more reliable. > > Thanks a lot, > > Jürgen Schwenk > > > Client output: > > > list > Name Resources Size Input Path > > ----------------------------------------------------------------------------------- > VentilutionOnlineDataBaseCorrupt 1 1170 > VentilutionOnlineDataBase.xml > > 1 database(s). > > > open VentilutionOnlineDataBaseCorrupt > Database 'VentilutionOnlineDataBaseCorrupt' was opened in 0.03 ms. > > > inspect > Checking main table (152 nodes): > - 0 invalid node kinds > - 0 invalid parent references > - 0 wrong parent/descendant relationships > No inconsistencies found. > 'VentilutionOnlineDataBaseCorrupt' inspected in 0.06 ms. > > > xquery //* > Improper use? Potential bug? Your feedback is welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 10.1 > Java: Eclipse Adoptium, 17.0.4 > OS: Windows 10, amd64 > Stack Trace: > java.lang.RuntimeException: Data Access out of bounds: > - pre value: 152 > - table size: 152 > - first/next pre value: 0/152 > - #total/used pages: 1/1 > - accessed page: 0 (1 > 0] > at org.basex.util.Util.notExpected(Util.java:64) > at > org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:477) > at > org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:158) > at org.basex.data.Data.kind(Data.java:312) > at org.basex.query.value.node.DBNode$6.next(DBNode.java:402) > at org.basex.query.value.node.DBNode$6.next(DBNode.java:1) > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38) > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:1) > at org.basex.query.QueryContext.next(QueryContext.java:375) > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:48) > at org.basex.query.scope.MainModule$1.next(MainModule.java:55) > at org.basex.core.cmd.AQuery.run(AQuery.java:88) > at org.basex.core.Command.run(Command.java:233) > at org.basex.core.Command.execute(Command.java:93) > at org.basex.server.ClientListener.run(ClientListener.java:141) > > Query "softwareProjectVentilution" executed in 0.31 ms. > > > -- > > > -- > > Jürgen Schwenk > - R&D Engineer - > > > > > Gründler GmbH Tel: +49 (0)7441 40761-14 > Jaspisstrasse 23 Fax: +49 (0)7441 40761-99 > 72250 Freudenstadt E-Mail: juergen.schwenk at gruendler.de > -- > > > -- > > Jürgen Schwenk > - R&D Engineer - > > > > > Gründler GmbH Tel: +49 (0)7441 40761-14 > Jaspisstrasse 23 Fax: +49 (0)7441 40761-99 > 72250 Freudenstadt E-Mail: juergen.schwenk at gruendler.de >
CG
Christian Grün15 Jan 2024 14:51 #
Dear Jürgen,
The safest/easiest option for now may be to consider a database corrupt if the output does not contain the string “No inconsistencies found”. For more details, feel free to check out the source code [1].
Hope this helps,
Christian
[1]
https://github.com/BaseXdb/basex/blob/main/basex-core/src/main/java/org/basex/core/cmd/Inspect.java
Quoted text · Jürgen Schwenk Büro · 150 lines
On Mon, Jan 15, 2024 at 2:44 PM Jürgen Schwenk Büro < juergen.schwenk at gruendler.de> wrote: > Dear Christian, > > thanks a lot for your work. I tried it out and got the answer "Access to > storage failed (java.lang.RuntimeException: java.io.EOFException)." That is > fine. > > One more, perhaps last :-), question: To avoid a crash when opening > database in our application I send the "inspect" command via C-Adapter > basexdbc.c and check the resulting string. This is because the query > command itself is OK, so I have to check the content. Up to now I check the > resulting string if it contains either "Warning: Database is inconsistent." > or "RuntimeException". I hope, these are the only possible failure > responses. Or are there any other? > > Regards, > > Jürgen > Am 15.01.2024 um 11:04 schrieb Christian Grün: > > Dear Jürgen, > > It turned out that the invocation of the INSPECT command raised an > exception that was not caught by the command itself, but was only sent to > stderr. We’ve revised this with the latest snapshot [1]. > > Best (and good luck for your power connection in 2024) > Christian > > [1] https://files.basex.org/releases/latest/ > > > > On Thu, Jan 11, 2024 at 6:07 PM Jürgen Schwenk Büro < > juergen.schwenk at gruendler.de> wrote: > >> Hello, >> >> in our application we "produced" a corrupt data base caused by a power >> fail. On restart we check data base using the "inspect" command to detect >> failures, creating new empty data base if found one. The attached data base >> seems OK using "inspect", but accessing the data base we get an exception. >> Perhaps this may help to make "inspect" more reliable. >> >> Thanks a lot, >> >> Jürgen Schwenk >> >> >> Client output: >> >> > list >> Name Resources Size Input Path >> >> ----------------------------------------------------------------------------------- >> VentilutionOnlineDataBaseCorrupt 1 1170 >> VentilutionOnlineDataBase.xml >> >> 1 database(s). >> >> > open VentilutionOnlineDataBaseCorrupt >> Database 'VentilutionOnlineDataBaseCorrupt' was opened in 0.03 ms. >> >> > inspect >> Checking main table (152 nodes): >> - 0 invalid node kinds >> - 0 invalid parent references >> - 0 wrong parent/descendant relationships >> No inconsistencies found. >> 'VentilutionOnlineDataBaseCorrupt' inspected in 0.06 ms. >> >> > xquery //* >> Improper use? Potential bug? Your feedback is welcome: >> Contact: basex-talk at mailman.uni-konstanz.de >> Version: BaseX 10.1 >> Java: Eclipse Adoptium, 17.0.4 >> OS: Windows 10, amd64 >> Stack Trace: >> java.lang.RuntimeException: Data Access out of bounds: >> - pre value: 152 >> - table size: 152 >> - first/next pre value: 0/152 >> - #total/used pages: 1/1 >> - accessed page: 0 (1 > 0] >> at org.basex.util.Util.notExpected(Util.java:64) >> at >> org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:477) >> at >> org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:158) >> at org.basex.data.Data.kind(Data.java:312) >> at org.basex.query.value.node.DBNode$6.next(DBNode.java:402) >> at org.basex.query.value.node.DBNode$6.next(DBNode.java:1) >> at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38) >> at org.basex.query.expr.path.IterStep$1.next(IterStep.java:1) >> at org.basex.query.QueryContext.next(QueryContext.java:375) >> at org.basex.query.expr.path.IterPath$1.next(IterPath.java:48) >> at org.basex.query.scope.MainModule$1.next(MainModule.java:55) >> at org.basex.core.cmd.AQuery.run(AQuery.java:88) >> at org.basex.core.Command.run(Command.java:233) >> at org.basex.core.Command.execute(Command.java:93) >> at org.basex.server.ClientListener.run(ClientListener.java:141) >> >> Query "softwareProjectVentilution" executed in 0.31 ms. >> >> >> -- >> >> >> -- >> >> Jürgen Schwenk >> - R&D Engineer - >> >> >> >> >> Gründler GmbH Tel: +49 (0)7441 40761-14 >> Jaspisstrasse 23 Fax: +49 (0)7441 40761-99 >> 72250 Freudenstadt E-Mail: juergen.schwenk at gruendler.de >> -- >> >> >> -- >> >> Jürgen Schwenk >> - R&D Engineer - >> >> >> >> >> Gründler GmbH Tel: +49 (0)7441 40761-14 >> Jaspisstrasse 23 Fax: +49 (0)7441 40761-99 >> 72250 Freudenstadt E-Mail: juergen.schwenk at gruendler.de >> > -- > > > -- > > Jürgen Schwenk > - R&D Engineer - > > > > > Gründler GmbH Tel: +49 (0)7441 40761-14 > Jaspisstrasse 23 Fax: +49 (0)7441 40761-99 > 72250 Freudenstadt E-Mail: juergen.schwenk at gruendler.de >
Written for BaseX 10.7. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery