upper limits on storage; database admin
Wednesday, 2 January 2019 · 2 messages · last reply 3 Jan 2019
T
thufir2 Jan 2019 23:39 #
What are the upper bounds to basex in size? Assuming it's just text xml, gigabytes is quite a bit to my thinking. At a certain, it's "big data" -- but how do you know when you're approaching that point?
Or, is the bottleneck more read/write and consistency problems? What little I know of RDBMS is that master/slave can alleviate some bottlenecks.
To put this another way: I'm so enthusiastic about basex that I'm having trouble finding a place it doesn't fit. As you approach terabytes and beyond what dbadmin approaches are employed?
-Thufir
FE
Fabrice ETANCHAUD3 Jan 2019 10:12 #
Hi Thufir,
You will find here [1] the BaseX current limitations.
The only limit I found was the maximum number of XML nodes in a collection. But I never handled more than 100 Gb of content.
You can safely add documents by enabling the
ADDCACHE option [2]; May the collection overflow, the operation will fail gracefully without corrupting it. As indexes are mono-collection structures, it is all about dispatching your data wisely in collections.
And you are right, IMHO BaseX can fit in many places, and shines in prototyping !
Best regards,
[1] http://docs.basex.org/wiki/Statistics
[2] http://docs.basex.org/wiki/Options#ADDCACHE
-----Message d'origine-----Quoted text · thufir · 13 lines
De : BaseX-Talk [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de thufir Envoyé : mercredi 2 janvier 2019 22:39 À : BaseX Objet : [basex-talk] upper limits on storage; database admin What are the upper bounds to basex in size? Assuming it's just text xml, gigabytes is quite a bit to my thinking. At a certain, it's "big data" -- but how do you know when you're approaching that point? Or, is the bottleneck more read/write and consistency problems? What little I know of RDBMS is that master/slave can alleviate some bottlenecks. To put this another way: I'm so enthusiastic about basex that I'm having trouble finding a place it doesn't fit. As you approach terabytes and beyond what dbadmin approaches are employed? -Thufir
Written for BaseX 9.1. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery