Main Page » Mailing List » January 2015

Pruned, optimized DB shows same document count, db size

Monday, 12 January 2015 · 8 messages · last reply 16 Jan 2015
HE
Hondros, Constantine (ELS-AMS)12 Jan 2015 14:28 #
Hello all, I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize. The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same. I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious? Thanks in advance, Constantine Hondros ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
FE
Fabrice Etanchaud12 Jan 2015 16:14 #
Dear Constantine, From what I know, optimize only rebuilt indexes and metadata. Did you try "optimize all" to compact db files ? But documents' number should be updated. How do you get metadata information ? Best regards, Fabrice Etanchaud Questel/Orbit http://docs.basex.org/wiki/Database_Module#db:optimize
Quoted text · Hondros, Constantine (ELS-AMS) · 11 lines
De : basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 14:29 À : basex-talk at mailman.uni-konstanz.de Objet : [basex-talk] Pruned, optimized DB shows same document count, db size Hello all, I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize. The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same. I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious? Thanks in advance, Constantine Hondros Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
HE
Hondros, Constantine (ELS-AMS)12 Jan 2015 16:32 #
Aha, of course, the all boolean. I did not try that, and will. DB metadata I get via the GUI menu : Database -> Open and Manage Cheers, C.
Quoted text · Fabrice Etanchaud · 25 lines
From: Fabrice Etanchaud [mailto:fetanchaud at questel.com] Sent: 12 January 2015 16:14 To: Hondros, Constantine (ELS-AMS); basex-talk at mailman.uni-konstanz.de Subject: RE: Pruned, optimized DB shows same document count, db size Dear Constantine, From what I know, optimize only rebuilt indexes and metadata. Did you try "optimize all" to compact db files ? But documents' number should be updated. How do you get metadata information ? Best regards, Fabrice Etanchaud Questel/Orbit http://docs.basex.org/wiki/Database_Module#db:optimize De : basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de> [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 14:29 À : basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Objet : [basex-talk] Pruned, optimized DB shows same document count, db size Hello all, I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize. The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same. I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious? Thanks in advance, Constantine Hondros Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
FE
Fabrice Etanchaud12 Jan 2015 17:00 #
Constantine, Did you mean db :delete or delete nodes ? I'm using 8.0 769b53b and after db:delete, even without optimize, I get correct documents' count in the GUI window. Best, Fabrice
Quoted text · Hondros, Constantine (ELS-AMS) · 33 lines
De : basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 16:32 À : basex-talk at mailman.uni-konstanz.de Objet : Re: [basex-talk] Pruned, optimized DB shows same document count, db size Aha, of course, the all boolean. I did not try that, and will. DB metadata I get via the GUI menu : Database -> Open and Manage Cheers, C. From: Fabrice Etanchaud [mailto:fetanchaud at questel.com] Sent: 12 January 2015 16:14 To: Hondros, Constantine (ELS-AMS); basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Subject: RE: Pruned, optimized DB shows same document count, db size Dear Constantine, From what I know, optimize only rebuilt indexes and metadata. Did you try "optimize all" to compact db files ? But documents' number should be updated. How do you get metadata information ? Best regards, Fabrice Etanchaud Questel/Orbit http://docs.basex.org/wiki/Database_Module#db:optimize De : basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de> [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 14:29 À : basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Objet : [basex-talk] Pruned, optimized DB shows same document count, db size Hello all, I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize. The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same. I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious? Thanks in advance, Constantine Hondros Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
HE
Hondros, Constantine (ELS-AMS)12 Jan 2015 17:09 #
Hi Fabrice, I used the Xquery delete node command, rather than the (document-oriented) db:delete. I guess that explains the difference. C.
Quoted text · Fabrice Etanchaud · 43 lines
From: Fabrice Etanchaud [mailto:fetanchaud at questel.com] Sent: 12 January 2015 17:01 To: Hondros, Constantine (ELS-AMS); basex-talk at mailman.uni-konstanz.de Subject: RE: [basex-talk] Pruned, optimized DB shows same document count, db size Constantine, Did you mean db :delete or delete nodes ? I'm using 8.0 769b53b and after db:delete, even without optimize, I get correct documents' count in the GUI window. Best, Fabrice De : basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de> [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 16:32 À : basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Objet : Re: [basex-talk] Pruned, optimized DB shows same document count, db size Aha, of course, the all boolean. I did not try that, and will. DB metadata I get via the GUI menu : Database -> Open and Manage Cheers, C. From: Fabrice Etanchaud [mailto:fetanchaud at questel.com] Sent: 12 January 2015 16:14 To: Hondros, Constantine (ELS-AMS); basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Subject: RE: Pruned, optimized DB shows same document count, db size Dear Constantine, From what I know, optimize only rebuilt indexes and metadata. Did you try "optimize all" to compact db files ? But documents' number should be updated. How do you get metadata information ? Best regards, Fabrice Etanchaud Questel/Orbit http://docs.basex.org/wiki/Database_Module#db:optimize De : basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de> [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 14:29 À : basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Objet : [basex-talk] Pruned, optimized DB shows same document count, db size Hello all, I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize. The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same. I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious? Thanks in advance, Constantine Hondros Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
FE
Fabrice Etanchaud12 Jan 2015 17:23 #
Constantine, It seems Questel and Elsevier are competitors... But using the same great piece of software ! Best regards, Fabrice
Quoted text · Hondros, Constantine (ELS-AMS) · 51 lines
De : basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 17:10 À : basex-talk at mailman.uni-konstanz.de Objet : Re: [basex-talk] Pruned, optimized DB shows same document count, db size Hi Fabrice, I used the Xquery delete node command, rather than the (document-oriented) db:delete. I guess that explains the difference. C. From: Fabrice Etanchaud [mailto:fetanchaud at questel.com] Sent: 12 January 2015 17:01 To: Hondros, Constantine (ELS-AMS); basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Subject: RE: [basex-talk] Pruned, optimized DB shows same document count, db size Constantine, Did you mean db :delete or delete nodes ? I'm using 8.0 769b53b and after db:delete, even without optimize, I get correct documents' count in the GUI window. Best, Fabrice De : basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de> [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 16:32 À : basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Objet : Re: [basex-talk] Pruned, optimized DB shows same document count, db size Aha, of course, the all boolean. I did not try that, and will. DB metadata I get via the GUI menu : Database -> Open and Manage Cheers, C. From: Fabrice Etanchaud [mailto:fetanchaud at questel.com] Sent: 12 January 2015 16:14 To: Hondros, Constantine (ELS-AMS); basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Subject: RE: Pruned, optimized DB shows same document count, db size Dear Constantine, From what I know, optimize only rebuilt indexes and metadata. Did you try "optimize all" to compact db files ? But documents' number should be updated. How do you get metadata information ? Best regards, Fabrice Etanchaud Questel/Orbit http://docs.basex.org/wiki/Database_Module#db:optimize De : basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de> [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 14:29 À : basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Objet : [basex-talk] Pruned, optimized DB shows same document count, db size Hello all, I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize. The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same. I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious? Thanks in advance, Constantine Hondros Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
HE
Hondros, Constantine (ELS-AMS)16 Jan 2015 09:45 #
Yup - and for the record, BaseX simply rocks. Great work, guys!
Quoted text · Fabrice Etanchaud · 62 lines
From: Fabrice Etanchaud [mailto:fetanchaud at questel.com] Sent: 12 January 2015 17:24 To: Hondros, Constantine (ELS-AMS) Cc: basex-talk at mailman.uni-konstanz.de Subject: RE: [basex-talk] Pruned, optimized DB shows same document count, db size Constantine, It seems Questel and Elsevier are competitors... But using the same great piece of software ! Best regards, Fabrice De : basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de> [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 17:10 À : basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Objet : Re: [basex-talk] Pruned, optimized DB shows same document count, db size Hi Fabrice, I used the Xquery delete node command, rather than the (document-oriented) db:delete. I guess that explains the difference. C. From: Fabrice Etanchaud [mailto:fetanchaud at questel.com] Sent: 12 January 2015 17:01 To: Hondros, Constantine (ELS-AMS); basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Subject: RE: [basex-talk] Pruned, optimized DB shows same document count, db size Constantine, Did you mean db :delete or delete nodes ? I'm using 8.0 769b53b and after db:delete, even without optimize, I get correct documents' count in the GUI window. Best, Fabrice De : basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de> [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 16:32 À : basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Objet : Re: [basex-talk] Pruned, optimized DB shows same document count, db size Aha, of course, the all boolean. I did not try that, and will. DB metadata I get via the GUI menu : Database -> Open and Manage Cheers, C. From: Fabrice Etanchaud [mailto:fetanchaud at questel.com] Sent: 12 January 2015 16:14 To: Hondros, Constantine (ELS-AMS); basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Subject: RE: Pruned, optimized DB shows same document count, db size Dear Constantine, From what I know, optimize only rebuilt indexes and metadata. Did you try "optimize all" to compact db files ? But documents' number should be updated. How do you get metadata information ? Best regards, Fabrice Etanchaud Questel/Orbit http://docs.basex.org/wiki/Database_Module#db:optimize De : basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de> [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 14:29 À : basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Objet : [basex-talk] Pruned, optimized DB shows same document count, db size Hello all, I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize. The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same. I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious? Thanks in advance, Constantine Hondros Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
DK
Dirk Kirsten16 Jan 2015 10:09 #
I vote that we have this statement ("BaseX simply rocks. Great work, guys!") eternalized in broad, golden letters in our office. Thanks for the nice feedback, it is appreciated :-)
Quoted text · Constantine (ELS-AMS) · 113 lines
On 01/16/2015 09:45 AM, Hondros, Constantine (ELS-AMS) wrote: > Yup - and for the record, BaseX simply rocks. Great work, guys! > > From: Fabrice Etanchaud [mailto:fetanchaud at questel.com] > Sent: 12 January 2015 17:24 > To: Hondros, Constantine (ELS-AMS) > Cc: basex-talk at mailman.uni-konstanz.de > Subject: RE: [basex-talk] Pruned, optimized DB shows same document count, db size > > Constantine, > > It seems Questel and Elsevier are competitors... > But using the same great piece of software ! > > Best regards, > Fabrice > > > De : basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de> [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) > Envoyé : lundi 12 janvier 2015 17:10 > À : basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> > Objet : Re: [basex-talk] Pruned, optimized DB shows same document count, db size > > Hi Fabrice, > > I used the Xquery delete node command, rather than the (document-oriented) db:delete. > > I guess that explains the difference. > > C. > > From: Fabrice Etanchaud [mailto:fetanchaud at questel.com] > Sent: 12 January 2015 17:01 > To: Hondros, Constantine (ELS-AMS); basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> > Subject: RE: [basex-talk] Pruned, optimized DB shows same document count, db size > > Constantine, > > Did you mean db :delete or delete nodes ? > > I'm using 8.0 769b53b and after db:delete, even without optimize, I get correct documents' count in the GUI window. > > Best, > Fabrice > > De : basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de> [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) > Envoyé : lundi 12 janvier 2015 16:32 > À : basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> > Objet : Re: [basex-talk] Pruned, optimized DB shows same document count, db size > > Aha, of course, the all boolean. I did not try that, and will. > > DB metadata I get via the GUI menu : Database -> Open and Manage > > Cheers, > C. > > From: Fabrice Etanchaud [mailto:fetanchaud at questel.com] > Sent: 12 January 2015 16:14 > To: Hondros, Constantine (ELS-AMS); basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> > Subject: RE: Pruned, optimized DB shows same document count, db size > > Dear Constantine, > > From what I know, optimize only rebuilt indexes and metadata. > Did you try "optimize all" to compact db files ? > > But documents' number should be updated. > How do you get metadata information ? > > Best regards, > > Fabrice Etanchaud > Questel/Orbit > > http://docs.basex.org/wiki/Database_Module#db:optimize > > > De : basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de> [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) > Envoyé : lundi 12 janvier 2015 14:29 > À : basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> > Objet : [basex-talk] Pruned, optimized DB shows same document count, db size > > Hello all, > > I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize. > > The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same. > > I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious? > > Thanks in advance, > > Constantine Hondros > > > > ________________________________ > > Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. > > ________________________________ > > Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. > > ________________________________ > > Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. > > ________________________________ > > Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. >
-- Dirk Kirsten, BaseX GmbH, http://basexgmbh.de |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22

Written for BaseX 7.9. BaseX may have changed since: please check the current documentation.

« Update: BaseX Server DownUser login »
⚡Generated with XQuery