multi-core performance of BaseX

> I was wondering if anyone has some information about BaseX multi-core > performance. > > Does BaseX utilize multi-core architecture to parallel-process the Xqueries?

-----Original Message----- From: basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of Christian Grün Sent: lundi 19 mai 2014 10:40 To: the joj Cc: BaseX Subject: Re: [basex-talk] multi-core performance of BaseX Hi Roy, > I was wondering if anyone has some information about BaseX multi-core > performance. > > Does BaseX utilize multi-core architecture to parallel-process the Xqueries? There has been some discussion on the list in the past; see e.g.: http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg03076.html http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg03159.html http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg02018.html A general note: as BaseX is a full programming language, it is a non-trivial problem to compute arbitrary queries in parallel, and I'm not aware of any other XQuery processor that provides such capabilities. Disk access is often slower when done in parallel (in particular on conventional hard-disks), as parallel, uncoordinagted queries may lead to random access pattern. Potential solutions could be to provide... * functions for explicitly creating and joining threads * optimize special XQuery expressions for parallel executions. Hope this helps, Christian

On Thu, May 22, 2014 at 10:14 AM, Gilles CARRY <gilles.carry at st.com> wrote: > Hi, > > Just my 2cts: > We have a server running BaseX and configured to hold all data in memory (RAM=64GB). > After collecting a few metrics, it turns out the bottleneck is the Cpu (100% and virtually no disk access) and unfortunately, BaseX only uses 1 cpu over the 8 the machine has. > > Best regards, > Gilles. > > -----Original Message----- > From: basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of Christian Grün > Sent: lundi 19 mai 2014 10:40 > To: the joj > Cc: BaseX > Subject: Re: [basex-talk] multi-core performance of BaseX > > Hi Roy, > >> I was wondering if anyone has some information about BaseX multi-core >> performance. >> >> Does BaseX utilize multi-core architecture to parallel-process the Xqueries? > > There has been some discussion on the list in the past; see e.g.: > > http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg03076.html > http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg03159.html > http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg02018.html > > A general note: as BaseX is a full programming language, it is a non-trivial problem to compute arbitrary queries in parallel, and I'm not aware of any other XQuery processor that provides such capabilities. Disk access is often slower when done in parallel (in particular on conventional hard-disks), as parallel, uncoordinagted queries may lead to random access pattern. > > Potential solutions could be to provide... > > * functions for explicitly creating and joining threads > * optimize special XQuery expressions for parallel executions. > > Hope this helps, > Christian

-----Original Message----- From: Christian Grün [mailto:christian.gruen at gmail.com] Sent: jeudi 22 mai 2014 10:20 To: Gilles CARRY Cc: the joj; BaseX Subject: Re: [basex-talk] multi-core performance of BaseX Hi Gilles, true, the CPU is often a bottleneck today. We are always interested in suggestions on how to parallelize XQuery in an effective way; suggestions are welcome. Christian On Thu, May 22, 2014 at 10:14 AM, Gilles CARRY <gilles.carry at st.com> wrote: > Hi, > > Just my 2cts: > We have a server running BaseX and configured to hold all data in memory (RAM=64GB). > After collecting a few metrics, it turns out the bottleneck is the Cpu (100% and virtually no disk access) and unfortunately, BaseX only uses 1 cpu over the 8 the machine has. > > Best regards, > Gilles. > > -----Original Message----- > From: basex-talk-bounces at mailman.uni-konstanz.de > [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of > Christian Grün > Sent: lundi 19 mai 2014 10:40 > To: the joj > Cc: BaseX > Subject: Re: [basex-talk] multi-core performance of BaseX > > Hi Roy, > >> I was wondering if anyone has some information about BaseX multi-core >> performance. >> >> Does BaseX utilize multi-core architecture to parallel-process the Xqueries? > > There has been some discussion on the list in the past; see e.g.: > > http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg03076.html > http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg03159.html > > http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg02 > 018.html > > A general note: as BaseX is a full programming language, it is a non-trivial problem to compute arbitrary queries in parallel, and I'm not aware of any other XQuery processor that provides such capabilities. Disk access is often slower when done in parallel (in particular on conventional hard-disks), as parallel, uncoordinagted queries may lead to random access pattern. > > Potential solutions could be to provide... > > * functions for explicitly creating and joining threads > * optimize special XQuery expressions for parallel executions. > > Hope this helps, > Christian

> However I'm interested in having a look at the documentation of the XQuery engine structure. I might then be able to do some suggestions. > Where should I start reading from?