Module parsing performance
Friday, 19 June 2015 · 6 messages · last reply 22 Jun 2015
AS
Alexander Shpack19 Jun 2015 17:18 #
Hi BaseX team!
How to increase performance of parsing and executing on custom xqm module? I have 10K lines of sxq code and it makes huge CPU load on the parsing process.
Any ideas?
--
s0rr0w
CG
Christian Grün22 Jun 2015 08:29 #
Hi Alex,
10k of XQuery code doesn't sound too much. Did you already test how much time goes into parsing, compiling and evaluating the query? You can e.g. see this by looking at the blue bars in the Info panel in the GUI, or by using -V on command line.
Best,
Christian
Quoted text · Alexander Shpack · 11 lines
On Fri, Jun 19, 2015 at 5:18 PM, Alexander Shpack <shadowkin at gmail.com> wrote: > Hi BaseX team! > > How to increase performance of parsing and executing on custom xqm module? > I have 10K lines of sxq code and it makes huge CPU load on the parsing > process. > > Any ideas? > > -- > s0rr0w
AS
Alexander Shpack22 Jun 2015 11:42 #
Quoted text · Christian Grün · 9 lines
On Mon, Jun 22, 2015 at 9:29 AM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Alex, > > 10k of XQuery code doesn't sound too much. Did you already test how > much time goes into parsing, compiling and evaluating the query? You > can e.g. see this by looking at the blue bars in the Info panel in the > GUI, or by using -V on command line. >
2-3 seconds on slow CPU (Athlon).
If I run one function from the module directly, it works fine. The trouble in module parsing.
Quoted text · Christian Grün · 19 lines
> > Best, > Christian > > > On Fri, Jun 19, 2015 at 5:18 PM, Alexander Shpack <shadowkin at gmail.com> > wrote: > > Hi BaseX team! > > > > How to increase performance of parsing and executing on custom xqm > module? > > I have 10K lines of sxq code and it makes huge CPU load on the parsing > > process. > > > > Any ideas? > > > > -- > > s0rr0w >
--
s0rr0w
CG
Christian Grün22 Jun 2015 12:39 #
Quoted text · Alexander Shpack · 3 lines
> 2-3 seconds on slow CPU (Athlon). > If I run one function from the module directly, it works fine. The trouble > in module parsing.
The largest XQuery module I have processed so far was about 1mb, and execution time for the parsing step was less than a second, so I would be interested to see the numbers that are output by BaseX (...maybe it's not really the parsing or compilation time that causes the bottleneck). Do you think that's possible?
Thanks in advance,
Christian
Quoted text · Alexander Shpack · 19 lines
>> On Fri, Jun 19, 2015 at 5:18 PM, Alexander Shpack <shadowkin at gmail.com> >> wrote: >> > Hi BaseX team! >> > >> > How to increase performance of parsing and executing on custom xqm >> > module? >> > I have 10K lines of sxq code and it makes huge CPU load on the parsing >> > process. >> > >> > Any ideas? >> > >> > -- >> > s0rr0w > > > > > -- > s0rr0w
AS
Parsing: 1539.6 ms
Compiling: 46.47 ms
Evaluating: 115.02 ms
Printing: 5.84 ms
Total Time: 1706.94 ms
Hit(s): 1 Item
Updated: 0 Items
Printed: 1927 Bytes
Read Locking: local [***, ***]
Write Locking: none
Query executed in 1706.94 ms.
Quoted text · Christian Grün · 27 lines
On Mon, Jun 22, 2015 at 9:29 AM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Alex, > > 10k of XQuery code doesn't sound too much. Did you already test how > much time goes into parsing, compiling and evaluating the query? You > can e.g. see this by looking at the blue bars in the Info panel in the > GUI, or by using -V on command line. > > Best, > Christian > > > On Fri, Jun 19, 2015 at 5:18 PM, Alexander Shpack <shadowkin at gmail.com> > wrote: > > Hi BaseX team! > > > > How to increase performance of parsing and executing on custom xqm > module? > > I have 10K lines of sxq code and it makes huge CPU load on the parsing > > process. > > > > Any ideas? > > > > -- > > s0rr0w >
--
s0rr0w
CG
Christian Grün22 Jun 2015 15:36 #
That's indeed unusual. Maybe some things are happening in the parsing step that are supposed to happen at evaluation time? Could you possibly run the query with the following call and send us the result?
java -Xrunhprof:cpu=samples,depth=25 -cp BaseX.jar
org.basex.Basex -r10 your-query.xq
If you manage to create an SSCCE of the code and data, that would even be better.
Back to your original question: Currently, there is no way to precompile XQuery expressions. However, repeatedly executed queries should be evaluated faster when using the client/server architecture.
Christian
Quoted text · Alexander Shpack · 14 lines
On Mon, Jun 22, 2015 at 3:31 PM, Alexander Shpack <shadowkin at gmail.com> wrote: > Parsing: 1539.6 ms > Compiling: 46.47 ms > Evaluating: 115.02 ms > Printing: 5.84 ms > Total Time: 1706.94 ms > > Hit(s): 1 Item > Updated: 0 Items > Printed: 1927 Bytes > Read Locking: local [***, ***] > Write Locking: none > > Query executed in 1706.94 ms.
Written for BaseX 8.2. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery