Cache compiled expressions for use with QueryProcessor

> Hi all, > We have lots of non dynamic (i.e. do not change at run-time) > queries/expressions that are always parsed/compiled using a new > QueryProcessor instance. > > How to cache compiled expressions for use with QueryProcessor? > Or can we just cache QueryProcessor instances? If yes, are QueryProcessor > instances thread-safe? > > Thanks! > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

> Hi Erdal, > > parsing and compilation phase is pretty fast in BaseX. Before you > think about precompiling your queries, it would be interesting to know > if parsing and compilation is a real bottleneck.. Have you already > done some profiling? > > The following example of running a (very simple) query 100'000 times > demonstrates that there may be no need to cache queries that are to be > executed multiple times: > > basex -V -r100000 "1+2" > > Parsing: 0.04 ms (avg) > Compiling: 0.01 ms (avg) > Evaluating: 0.0 ms (avg) > Printing: 0.03 ms (avg) > Total Time: 0.08 ms (avg) > > In a complex query with lots of modules being imported, however, the > time for parsing and compiling a query may increase. For those cases, > we indeed have some plans to precompile queries in the background. I > can’t give you a timeframe yet. > > Christian > ___________________________ > > 2013/10/22 Erdal Karaca <erdal.karaca.de at gmail.com>: > > Hi all, > > We have lots of non dynamic (i.e. do not change at run-time) > > queries/expressions that are always parsed/compiled using a new > > QueryProcessor instance. > > > > How to cache compiled expressions for use with QueryProcessor? > > Or can we just cache QueryProcessor instances? If yes, are QueryProcessor > > instances thread-safe? > > > > Thanks! > > > > _______________________________________________ > > BaseX-Talk mailing list > > BaseX-Talk at mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > >

> Hi Christian, > In this case, BaseX is embedded in a Java application (Eclipse RCP) and > provides "live editing" of parts of big XML files (managed by BaseX). Just > imagine we have an O(n^2) algorithm which visits 100x100 "data items" while > executing 10 different (and "simple") expressions (in the inner loop). That > would be 100x100x10 = 100'000 parsing/compiling operations taking in sum > 5000 ms if 1 parsing/compiling operation takes 0.05 ms. If the expressions > would be cacheable, then, maybe it would just take 10 x (0.05 ms + (lookup > of compiled expression)) > Of course, O(n^2) is bad, but that is another story. > > The idea is to be able to cache compiled expressions (like parsing/compiling > regex in java). > > Thanks! > > > 2013/10/22 Christian Grün <christian.gruen at gmail.com> >> >> Hi Erdal, >> >> parsing and compilation phase is pretty fast in BaseX. Before you >> think about precompiling your queries, it would be interesting to know >> if parsing and compilation is a real bottleneck.. Have you already >> done some profiling? >> >> The following example of running a (very simple) query 100'000 times >> demonstrates that there may be no need to cache queries that are to be >> executed multiple times: >> >> basex -V -r100000 "1+2" >> >> Parsing: 0.04 ms (avg) >> Compiling: 0.01 ms (avg) >> Evaluating: 0.0 ms (avg) >> Printing: 0.03 ms (avg) >> Total Time: 0.08 ms (avg) >> >> In a complex query with lots of modules being imported, however, the >> time for parsing and compiling a query may increase. For those cases, >> we indeed have some plans to precompile queries in the background. I >> can’t give you a timeframe yet. >> >> Christian >> ___________________________ >> >> 2013/10/22 Erdal Karaca <erdal.karaca.de at gmail.com>: >> > Hi all, >> > We have lots of non dynamic (i.e. do not change at run-time) >> > queries/expressions that are always parsed/compiled using a new >> > QueryProcessor instance. >> > >> > How to cache compiled expressions for use with QueryProcessor? >> > Or can we just cache QueryProcessor instances? If yes, are >> > QueryProcessor >> > instances thread-safe? >> > >> > Thanks! >> > >> > _______________________________________________ >> > BaseX-Talk mailing list >> > BaseX-Talk at mailman.uni-konstanz.de >> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > > >

> Hi Erdal, > > in this case, you’ll either need to wait, or think about sponsoring > the feature to speed up its implementation… ;) > > Best, > Christian > ___________________________ > > > Hi Christian, > > In this case, BaseX is embedded in a Java application (Eclipse RCP) and > > provides "live editing" of parts of big XML files (managed by BaseX). > Just > > imagine we have an O(n^2) algorithm which visits 100x100 "data items" > while > > executing 10 different (and "simple") expressions (in the inner loop). > That > > would be 100x100x10 = 100'000 parsing/compiling operations taking in sum > > 5000 ms if 1 parsing/compiling operation takes 0.05 ms. If the > expressions > > would be cacheable, then, maybe it would just take 10 x (0.05 ms + > (lookup > > of compiled expression)) > > Of course, O(n^2) is bad, but that is another story. > > > > The idea is to be able to cache compiled expressions (like > parsing/compiling > > regex in java). > > > > Thanks! > > > > > > 2013/10/22 Christian Grün <christian.gruen at gmail.com> > >> > >> Hi Erdal, > >> > >> parsing and compilation phase is pretty fast in BaseX. Before you > >> think about precompiling your queries, it would be interesting to know > >> if parsing and compilation is a real bottleneck.. Have you already > >> done some profiling? > >> > >> The following example of running a (very simple) query 100'000 times > >> demonstrates that there may be no need to cache queries that are to be > >> executed multiple times: > >> > >> basex -V -r100000 "1+2" > >> > >> Parsing: 0.04 ms (avg) > >> Compiling: 0.01 ms (avg) > >> Evaluating: 0.0 ms (avg) > >> Printing: 0.03 ms (avg) > >> Total Time: 0.08 ms (avg) > >> > >> In a complex query with lots of modules being imported, however, the > >> time for parsing and compiling a query may increase. For those cases, > >> we indeed have some plans to precompile queries in the background. I > >> can’t give you a timeframe yet. > >> > >> Christian > >> ___________________________ > >> > >> 2013/10/22 Erdal Karaca <erdal.karaca.de at gmail.com>: > >> > Hi all, > >> > We have lots of non dynamic (i.e. do not change at run-time) > >> > queries/expressions that are always parsed/compiled using a new > >> > QueryProcessor instance. > >> > > >> > How to cache compiled expressions for use with QueryProcessor? > >> > Or can we just cache QueryProcessor instances? If yes, are > >> > QueryProcessor > >> > instances thread-safe? > >> > > >> > Thanks! > >> > > >> > _______________________________________________ > >> > BaseX-Talk mailing list > >> > BaseX-Talk at mailman.uni-konstanz.de > >> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >> > > > > > >