Node Maximum 2^31
Wednesday, 16 December 2015 · 2 messages
BG
Benjamin Gathmann16 Dec 2015 13:36 #
I read here that the current maximum number of nodes in a BaseX DB is 2^31 (2'147'483'648):
http://docs.basex.org/wiki/Statistics
Is this related to 32bit CPU architecture? And are there plans for a 64bit version of BaseX that would allow larger databases?
I am asking because I am testing BaseX with large XML files (usually 5 to 15MB each) with a very deep nested structure and many small elements. Right now, I have created a DB with 3979 such documents.
DB Size: 29,697 MB
Nodes: 1,172,761,158
This means that I will reach the max. #nodes already with around 5500 reports and a DB size of 55 GB.
This in turn will mean that I will have to split my documents on rather many database instances if I import e.g. 50,000 documents, which worries me a bit. Or do I worry with no reason?
CG
Christian Grün16 Dec 2015 15:04 #
Hi again,
Quoted text · Benjamin Gathmann · 3 lines
> I read here that the current maximum number of nodes in a BaseX DB is > 2^31 (2'147'483'648): > http://docs.basex.org/wiki/Statistics
This limit still exists. There is a GitHub issue that refers to this limitation [1]. As you can address multiple databases via a single XQuery expression in BaseX, it is currently a non-issue for our own applications. Besides that, it would probably take quite some efforts to make it happen.
Cheers,
Christian
[1] https://github.com/BaseXdb/basex/issues/902
Quoted text · Benjamin Gathmann · 14 lines
> Is this related to 32bit CPU architecture? And are there plans for a > 64bit version of BaseX that would allow larger databases? > > I am asking because I am testing BaseX with large XML files (usually 5 > to 15MB each) with a very deep nested structure and many small elements. > Right now, I have created a DB with 3979 such documents. > DB Size: 29,697 MB > Nodes: 1,172,761,158 > > This means that I will reach the max. #nodes already with around 5500 > reports and a DB size of 55 GB. > This in turn will mean that I will have to split my documents on rather > many database instances if I import e.g. 50,000 documents, which worries > me a bit. Or do I worry with no reason?
Written for BaseX 8.3. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery