One document per database or multiple?
Tuesday, 6 February 2018 · 4 messages · last reply 7 Feb 2018
CG
Hi Jonathan,
In the Create Database dialog of the GUI, you can also specify a local directory as input; it will be recursively parsed.
In our own set of use cases, we have one database instance with appr. 8 million documents, but in most cases, the number is much smaller. There are some sample statistics in our documentation [1], they show you what is possible.
Hope this helps,
Christian
[1] http://docs.basex.org/wiki/Statistics
Am 06.02.2018 11:31 nachm. schrieb "Jonathan Robie" <
jonathan.robie at gmail.com>:
If I have a set of related documents (a text, a lexicon, frequency counts, a discourse analysis, etc), how should I decide when to put more than one document in a single database at different paths, as opposed to putting one document in each database?
When I create a database from the GUI, it seems to prefer one document per database. Should I take that as a hint?
Jonathan
FE
Jonathan, in my humble opinion, here are the main reasons you may need several collections :
- FullText indexing in several languages (because language is collection wide) : a per language partition of your data
- Size (usually in number of nodes) limitation
- Huge updates : a read only backlog collection + a read/write front collection of fresh data + queries tailored to read both collections.
Best regards,
And maybe good night ?
Fabrice Etanchaud
Quoted text · Jonathan Robie · 7 lines
De : basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Jonathan Robie Envoyé : mardi 6 février 2018 23:31 À : BaseX Objet : [basex-talk] One document per database or multiple? If I have a set of related documents (a text, a lexicon, frequency counts, a discourse analysis, etc), how should I decide when to put more than one document in a single database at different paths, as opposed to putting one document in each database? When I create a database from the GUI, it seems to prefer one document per database. Should I take that as a hint? Jonathan
JR
Jonathan Robie7 Feb 2018 00:30 #
If I have a set of related documents (a text, a lexicon, frequency counts, a discourse analysis, etc), how should I decide when to put more than one document in a single database at different paths, as opposed to putting one document in each database?
When I create a database from the GUI, it seems to prefer one document per database. Should I take that as a hint?
Jonathan
JR
Thanks to both of you, Fabrice and Christian. I should have asked this question a long time ago - it will help a lot with organizing my work better.
Jonathan
Quoted text · Fabrice ETANCHAUD · 49 lines
On Tue, Feb 6, 2018 at 5:41 PM, Fabrice ETANCHAUD < fetanchaud at pch.cerfrance.fr> wrote: > Jonathan, in my humble opinion, here are the main reasons you may need > several collections : > > > > - FullText indexing in several languages (because language is > collection wide) : a per language partition of your data > > - Size (usually in number of nodes) limitation > > - Huge updates : a read only backlog collection + a read/write > front collection of fresh data + queries tailored to read both collections. > > > > Best regards, > > And maybe good night ? > > > > Fabrice Etanchaud > > > > *De :* basex-talk-bounces at mailman.uni-konstanz.de [mailto: > basex-talk-bounces at mailman.uni-konstanz.de] *De la part de* Jonathan Robie > *Envoyé :* mardi 6 février 2018 23:31 > *À :* BaseX > *Objet :* [basex-talk] One document per database or multiple? > > > > If I have a set of related documents (a text, a lexicon, frequency counts, > a discourse analysis, etc), how should I decide when to put more than one > document in a single database at different paths, as opposed to putting one > document in each database? > > > > When I create a database from the GUI, it seems to prefer one document per > database. Should I take that as a hint? > > > > Jonathan >
Written for BaseX 8.6. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery