Clustered XML database
Tuesday, 12 January 2010 · 1 message
CG
Christian Grün12 Jan 2010 09:26 #
Hi Erol,
it's true that one database and one single XQuery call won't be enough to process data of that size. As BaseX does not natively support distributed storage and parallel queries, I would suggest the classical (manual) way: split the data to multiple databases, run the query on each instance and eventually merge the textual results. This could be done in a few lines of code, provided that the queries are not sophisticated, the database contents are disjunct, no concurrency issues have to be considered, etc.
Hope this helps,
Christian
Quoted text · Erol Akarsu · 46 lines
On Tue, Jan 12, 2010 at 3:22 AM, Erol Akarsu <eakarsu at gmail.com> wrote: > Hi Christian, > > I appreciate a new release of BaseX. > > I have one question on storing a huge data,say 10-50 petabytes data, with > Basex. I expect current version may not handle it properly. > Can we distribute XML data across clusters of nodes and have a parallel > xquery to be run across nodes and get result? > I would like to know how it can be implemented such a big clustered xml > database and have efficient xquery on this cluster. > > Regards > > Erol Akarsu > > On Sun, Jan 10, 2010 at 4:49 PM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> Dear all, >> >> we are excited to finally announce the release of BaseX 6.0! It offers >> a bunch of new features, such as: >> >> - a full implementation of XQuery Update, reaching 100% in the W3C Test >> Suite. >> - an enhanced client/server architecture with ACID-safe transactions, >> user and password management and transaction logging. >> - extended XQuery Full Text features, supporting TF/IDF scoring, stop >> word lists, fuzzy querying, etc. >> - the improved creation of index structures, using constant memory. >> - optimizations to increase the use of index structures and iterative >> query processing. >> >> As usual, please find more information at http://basex.org/ >> >> All the best, >> >> Christian Grün >> BaseX Team >> _______________________________________________ >> BaseX-Announce mailing list >> BaseX-Announce at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-announce > >
Written for BaseX 6. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery