Main Page » Mailing List » July 2020

Possible bug in database size (bytes) after database optimize from BaseX Database Administration

Thursday, 16 July 2020 · 9 messages
CG
Christian Grünin reply to Geoff Alexander, 16 Jul 2020, 20:0616 Jul 2020 19:10 #
Hi Geoff, Did you run OPTIMIZE ALL or db:optimize(..., true()) ? What do you mean by "record count"? Best, Christian
Quoted text · Geoff Alexander · 15 lines
Geoff Alexander <gdlxn at us.ibm.com> schrieb am Do., 16. Juli 2020, 18:06: > I've found that when I perform a database optimize on an unoptimized > database that while the record count decreases as expected, the database > size (bytes column) stay the same. Is this a bug in reporting the database > size (less severe problem) or a bug in the database not reducing on > optimize (more severe problem)? I've encountered this with both BaseX 8.6.7 > and 9.3.3 running on Windows 10. > > Thanks, > Geoff Alexander, Ph.D. > Software Engineer, Corporate Tools Development > IBM Corporation > Charlotte, NC > >
GA
Geoff Alexander16 Jul 2020 20:06 #
I've found that when I perform a database optimize on an unoptimized database that while the record count decreases as expected, the database size (bytes column) stay the same. Is this a bug in reporting the database size (less severe problem) or a bug in the database not reducing on optimize (more severe problem)? I've encountered this with both BaseX 8.6.7 and 9.3.3 running on Windows 10. Thanks, Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC
CG
Christian Grünin reply to Geoff Alexander, 16 Jul 2020, 22:0416 Jul 2020 20:11 #
It's surprising that the count value changed, as it should represent the number of resources (documents, binary files) in your database – and this value shouldn't change if your data is optimized. Feel free to provide us with a little reproducible example. The size of the database may stay the same, though. The DBA provides no way to trigger a full optimization, but you can e.g. use the query panel for that.
Quoted text · Geoff Alexander · 59 lines
Geoff Alexander <gdlxn at us.ibm.com> schrieb am Do., 16. Juli 2020, 20:04: > On the BaseX Database Administration's Database page at > https://localhost:10443/BaseX/dba/databases, I selected a database I knew > was unoptimized and pressed the Optimize button. The database's COUNT > column decreased to the number entries in the database as expected. > However, the database's BYTES column did not change, even after I logged > off and back on to BaseX Database Administration. > > Geoff Alexander, Ph.D. > Software Engineer, Corporate Tools Development > IBM Corporation > Charlotte, NC > > > [image: Inactive hide details for "Christian Grün" ---07/16/2020 01:10:38 > PM---Hi Geoff, Did you run OPTIMIZE ALL or db:optimize(..., t]"Christian > Grün" ---07/16/2020 01:10:38 PM---Hi Geoff, Did you run OPTIMIZE ALL or > db:optimize(..., true()) ? What do you mean by > > From: "Christian Grün" <christian.gruen at gmail.com> > To: Geoff Alexander <gdlxn at us.ibm.com> > Cc: BaseX <basex-talk at mailman.uni-konstanz.de> > Date: 07/16/2020 01:10 PM > Subject: [EXTERNAL] Re: [basex-talk] Possible bug in database size > (bytes) after database optimize from BaseX Database Administration > ------------------------------ > > > > Hi Geoff, > > Did you run OPTIMIZE ALL or db:optimize(..., true()) ? What do you mean by > "record count"? > > Best, > Christian > > > > Geoff Alexander <*gdlxn at us.ibm.com* <gdlxn at us.ibm.com>> schrieb am Do., > 16. Juli 2020, 18:06: > > I've found that when I perform a database optimize on an unoptimized > database that while the record count decreases as expected, the database > size (bytes column) stay the same. Is this a bug in reporting the database > size (less severe problem) or a bug in the database not reducing on > optimize (more severe problem)? I've encountered this with both BaseX 8.6.7 > and 9.3.3 running on Windows 10. > > Thanks, > Geoff Alexander, Ph.D. > Software Engineer, Corporate Tools Development > IBM Corporation > Charlotte, NC > > > > >
CG
Christian Grünin reply to Geoff Alexander, 16 Jul 2020, 22:4516 Jul 2020 21:06 #
Quoted text · Geoff Alexander · 3 lines
> > (2) Update one or of the database's entries. >
If I replace the document via the DBA, or if I run an update expression via the Queries Panel, the count always reflects the number of resources, it doesn’t change. How did you update the document? Is it an XML document or a binary file you updated?
Quoted text · Geoff Alexander · 3 lines
> Maybe I have a misundertanding in what the Optimize button on the BaseX > Database Administration's Database page actually does. >
Feel free to have a look into our documentation [1]. [1] https://docs.basex.org/wiki/Commands#OPTIMIZE
CG
Christian Grünin reply to Geoff Alexander, 16 Jul 2020, 23:0916 Jul 2020 21:10 #
Quoted text · Geoff Alexander · 4 lines
> > We use the BaseX REST API from a Java problem to add and update documents > in BaseX. >
Do you think it’s reproducible for us?
Quoted text · Geoff Alexander · 1 line
>
GA
Geoff Alexanderin reply to Christian Grün, 16 Jul 2020, 19:1016 Jul 2020 22:04 #
On the BaseX Database Administration's Database page at https://localhost:10443/BaseX/dba/databases, I selected a database I knew was unoptimized and pressed the Optimize button. The database's COUNT column decreased to the number entries in the database as expected. However, the database's BYTES column did not change, even after I logged off and back on to BaseX Database Administration. Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC From: "Christian Grün" <christian.gruen at gmail.com> To: Geoff Alexander <gdlxn at us.ibm.com> Cc: BaseX <basex-talk at mailman.uni-konstanz.de> Date: 07/16/2020 01:10 PM Subject: [EXTERNAL] Re: [basex-talk] Possible bug in database size (bytes) after database optimize from BaseX Database Administration Hi Geoff, Did you run OPTIMIZE ALL or db:optimize(..., true()) ? What do you mean by "record count"? Best, Christian
Quoted text · Christian Grün · 1 line
Geoff Alexander <gdlxn at us.ibm.com> schrieb am Do., 16. Juli 2020, 18:06:
I've found that when I perform a database optimize on an unoptimized database that while the record count decreases as expected, the database size (bytes column) stay the same. Is this a bug in reporting the database size (less severe problem) or a bug in the database not reducing on optimize (more severe problem)? I've encountered this with both BaseX 8.6.7 and 9.3.3 running on Windows 10. Thanks, Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC
GA
Geoff Alexanderin reply to Christian Grün, 16 Jul 2020, 20:1116 Jul 2020 22:45 #
Here are steps to recreate the problem: (1) Add one or more entries to an new (empty) database. One the BaseX Database Administration's Database page, you'll find that the database's COUNT column shows the number of entries added and that the database's BYTES column shows the database size. (2) Update one or of the database's entries. After refreshing BaseX Database Administration's Database page, you should find that the database's COUNT and BYTES columns both increased. (3) On the BaseX Database Administration's Database page, select the database and press the Optimize button. You should find that the database's COUNT column decreases back to the number of entries in the database. However, the database's BYTES column doesn't decrease to reflect a reduction in the database size. Maybe I have a misundertanding in what the Optimize button on the BaseX Database Administration's Database page actually does. Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC From: "Christian Grün" <christian.gruen at gmail.com> To: Geoff Alexander <gdlxn at us.ibm.com> Cc: BaseX <basex-talk at mailman.uni-konstanz.de> Date: 07/16/2020 02:11 PM Subject: [EXTERNAL] Re: Possible bug in database size (bytes) after database optimize from BaseX Database Administration It's surprising that the count value changed, as it should represent the number of resources (documents, binary files) in your database – and this value shouldn't change if your data is optimized. Feel free to provide us with a little reproducible example. The size of the database may stay the same, though. The DBA provides no way to trigger a full optimization, but you can e.g. use the query panel for that.
Quoted text · Christian Grün · 1 line
Geoff Alexander <gdlxn at us.ibm.com> schrieb am Do., 16. Juli 2020, 20:04:
On the BaseX Database Administration's Database page at https://localhost:10443/BaseX/dba/databases, I selected a database I knew was unoptimized and pressed the Optimize button. The database's COUNT column decreased to the number entries in the database as expected. However, the database's BYTES column did not change, even after I logged off and back on to BaseX Database Administration. Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC "Christian Grün" ---07/16/2020 01:10:38 PM---Hi Geoff, Did you run OPTIMIZE ALL or db:optimize(..., true()) ? What do you mean by From: "Christian Grün" <christian.gruen at gmail.com> To: Geoff Alexander <gdlxn at us.ibm.com> Cc: BaseX <basex-talk at mailman.uni-konstanz.de> Date: 07/16/2020 01:10 PM Subject: [EXTERNAL] Re: [basex-talk] Possible bug in database size (bytes) after database optimize from BaseX Database Administration Hi Geoff, Did you run OPTIMIZE ALL or db:optimize(..., true()) ? What do you mean by "record count"? Best, Christian
Quoted text · Christian Grün · 1 line
Geoff Alexander <gdlxn at us.ibm.com> schrieb am Do., 16. Juli 2020, 18:06:
I've found that when I perform a database optimize on an unoptimized database that while the record count decreases as expected, the database size (bytes column) stay the same. Is this a bug in reporting the database size (less severe problem) or a bug in the database not reducing on optimize (more severe problem)? I've encountered this with both BaseX 8.6.7 and 9.3.3 running on Windows 10. Thanks, Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC [attachment "graycol.gif" deleted by Geoff Alexander/Raleigh/IBM]
GA
Geoff Alexanderin reply to Christian Grün, 16 Jul 2020, 21:0616 Jul 2020 23:09 #
We use the BaseX REST API from a Java problem to add and update documents in BaseX. Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC From: "Christian Grün" <christian.gruen at gmail.com> To: Geoff Alexander <gdlxn at us.ibm.com> Cc: BaseX <basex-talk at mailman.uni-konstanz.de> Date: 07/16/2020 03:07 PM Subject: [EXTERNAL] Re: Possible bug in database size (bytes) after database optimize from BaseX Database Administration (2) Update one or of the database's entries. If I replace the document via the DBA, or if I run an update expression via the Queries Panel, the count always reflects the number of resources, it doesn’t change. How did you update the document? Is it an XML document or a binary file you updated? Maybe I have a misundertanding in what the Optimize button on the BaseX Database Administration's Database page actually does. Feel free to have a look into our documentation [1]. [1] https://docs.basex.org/wiki/Commands#OPTIMIZE
GA
Geoff Alexanderin reply to Christian Grün, 16 Jul 2020, 21:1016 Jul 2020 23:15 #
I would think so. Let me see if I can create a small example using the BaseX WEB API that recreates the problem. Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC From: "Christian Grün" <christian.gruen at gmail.com> To: Geoff Alexander <gdlxn at us.ibm.com> Cc: BaseX <basex-talk at mailman.uni-konstanz.de> Date: 07/16/2020 03:10 PM Subject: [EXTERNAL] Re: Possible bug in database size (bytes) after database optimize from BaseX Database Administration We use the BaseX REST API from a Java problem to add and update documents in BaseX. Do you think it’s reproducible for us?

Written for BaseX 9.4. BaseX may have changed since: please check the current documentation.

⚡Generated with XQuery