Main Page » Mailing List » February 2011

BaseX question

Thursday, 24 February 2011 · 11 messages

Christian Grün24 Feb 2011 02:20 #

Hi Phil (cc'ed to the list), as far as I can judge, XQuery expressions are too heterogeneous to provide realistic cost estimations. While our query compiler has some notion of "costs", those values are all relative, and used to select the (presumably) cheapest query plan or index access. The same applies to the estimation of query results: the costs and size of single XQuery results can vary a lot, as a single result node can span several MB, while a single atomic item may just take a few bytes. Hope this helps (at least a little), Christian

Quoted text · 1 line

> Hi Christian - is there a way to perform a low-cost (very fast) estimation of the amount of time a particular query would take without actually executing the query? In a server environment this is extremely useful. I would like to be able to rapidly check that queries will not be issued which exceed a certain execution time (roughly or even very roughly - e.g. on the order of a second vs. on the order of 100 or 1000 seconds.) Since my queries are going to be dynamically generated based on user supplied parameters this is an issue. A related issue is estimating the number of nodes retrieved as a result of a given query (again without actually executing it.) The estimation would not have to be very exact.

NewIntellectual24 Feb 2011 03:47 #

On Wed, Feb 23, 2011 at 8:20 PM, Christian Grün <christian.gruen at gmail.com>wrote:

Quoted text · Christian Grün · 5 lines

> as far as I can judge, XQuery expressions are too heterogeneous to > provide realistic cost estimations. While our query compiler has some > notion of "costs", those values are all relative, and used to select > the (presumably) cheapest query plan or index access. >

I see. Can you remark on how hard it might be to specify a parameter in the appropriate query methods which would designate a maximum execution time before the query is cleanly aborted (in "wall clock time")? This would be in a client/server context.

Jan Vlčinský (CAD)24 Feb 2011 07:32 #

Phil At http://docs.basex.org/wiki/Options#Server_options is mentioned server wide option TIMEOUT, which can limit time for query execution. I wonder, if there is any way to specify this per query. Jan 2011/2/24 NewIntellectual <newintellectual at gmail.com>

Quoted text · Christian Grün · 21 lines

> On Wed, Feb 23, 2011 at 8:20 PM, Christian Grün <christian.gruen at gmail.com > > wrote: > >> as far as I can judge, XQuery expressions are too heterogeneous to >> provide realistic cost estimations. While our query compiler has some >> notion of "costs", those values are all relative, and used to select >> the (presumably) cheapest query plan or index access. >> > > I see. Can you remark on how hard it might be to specify a parameter in the > appropriate query methods which would designate a maximum execution time > before the query is cleanly aborted (in "wall clock time")? This would be > in a client/server context. > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >

-- *Ing. Jan Vlčinský* CAD programy Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic tel: +420-597 602 024; mob: +420-608 979 040 skype: janvlcinsky; GoogleTalk: jan.vlcinsky at gmail.com http://cz.linkedin.com/in/vlcinsky

Andreas Weiler24 Feb 2011 09:01 #

Hi, this timeout is not server specific, so you can set it for each client separately. The easiest way is set timeout, run query, set timeout back. -- Andreas

Quoted text · Jan Vlčinský (CAD) · 37 lines

Am 24.02.2011 um 07:32 schrieb Jan Vlčinský (CAD): > Phil > At http://docs.basex.org/wiki/Options#Server_options is mentioned server wide option TIMEOUT, which can limit time for query execution. > I wonder, if there is any way to specify this per query. > > Jan > > 2011/2/24 NewIntellectual <newintellectual at gmail.com> > On Wed, Feb 23, 2011 at 8:20 PM, Christian Grün <christian.gruen at gmail.com> wrote: > as far as I can judge, XQuery expressions are too heterogeneous to > provide realistic cost estimations. While our query compiler has some > notion of "costs", those values are all relative, and used to select > the (presumably) cheapest query plan or index access. > > I see. Can you remark on how hard it might be to specify a parameter in the appropriate query methods which would designate a maximum execution time before the query is cleanly aborted (in "wall clock time")? This would be in a client/server context. > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > > > -- > Ing. Jan Vlčinský > CAD programy > Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic > tel: +420-597 602 024; mob: +420-608 979 040 > skype: janvlcinsky; GoogleTalk: jan.vlcinsky at gmail.com > http://cz.linkedin.com/in/vlcinsky > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Christian Grünin reply to Jan Vlčinský (CAD), 24 Feb 2011, 07:3224 Feb 2011 13:53 #

Quoted text · Jan Vlčinský (CAD) · 10 lines

>> I see. Can you remark on how hard it might be to specify a parameter in >> the appropriate query methods which would designate a maximum execution time >> before the query is cleanly aborted (in "wall clock time")? This would be >> in a client/server context. > > Phil > At http://docs.basex.org/wiki/Options#Server_options is mentioned server > wide option TIMEOUT, which can limit time for query execution. > I wonder, if there is any way to specify this per query. > Jan

....exactly; this option will do exactly what you need. Thanks, Christian

Quoted text · Jan Vlčinský (CAD) · 28 lines

> > 2011/2/24 NewIntellectual <newintellectual at gmail.com> >> >> On Wed, Feb 23, 2011 at 8:20 PM, Christian Grün >> <christian.gruen at gmail.com> wrote: >>> >>> as far as I can judge, XQuery expressions are too heterogeneous to >>> provide realistic cost estimations. While our query compiler has some >>> notion of "costs", those values are all relative, and used to select >>> the (presumably) cheapest query plan or index access. >> >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > > > > -- > Ing. Jan Vlčinský > CAD programy > Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic > tel: +420-597 602 024; mob: +420-608 979 040 > skype: janvlcinsky; GoogleTalk: jan.vlcinsky at gmail.com > http://cz.linkedin.com/in/vlcinsky >

Jan Vlčinský (CAD)24 Feb 2011 14:16 #

Another question is - is there any way to set timeout when using REST interface? Jan 2011/2/24 Christian Grün <christian.gruen at gmail.com>

Quoted text · Christian Grün · 49 lines

> >> I see. Can you remark on how hard it might be to specify a parameter in > >> the appropriate query methods which would designate a maximum execution > time > >> before the query is cleanly aborted (in "wall clock time")? This would > be > >> in a client/server context. > > > > Phil > > At http://docs.basex.org/wiki/Options#Server_options is mentioned server > > wide option TIMEOUT, which can limit time for query execution. > > I wonder, if there is any way to specify this per query. > > Jan > > ....exactly; this option will do exactly what you need. > > Thanks, > Christian > > > > > > > 2011/2/24 NewIntellectual <newintellectual at gmail.com> > >> > >> On Wed, Feb 23, 2011 at 8:20 PM, Christian Grün > >> <christian.gruen at gmail.com> wrote: > >>> > >>> as far as I can judge, XQuery expressions are too heterogeneous to > >>> provide realistic cost estimations. While our query compiler has some > >>> notion of "costs", those values are all relative, and used to select > >>> the (presumably) cheapest query plan or index access. > >> > >> > >> _______________________________________________ > >> BaseX-Talk mailing list > >> BaseX-Talk at mailman.uni-konstanz.de > >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >> > > > > > > > > -- > > Ing. Jan Vlčinský > > CAD programy > > Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic > > tel: +420-597 602 024; mob: +420-608 979 040 > > skype: janvlcinsky; GoogleTalk: jan.vlcinsky at gmail.com > > http://cz.linkedin.com/in/vlcinsky > > >

Christian Grün24 Feb 2011 14:20 #

Quoted text · Jan Vlčinský (CAD) · 3 lines

> Another question is - is there any way to set timeout when using REST > interface? > Jan

As our JAX-RX (REST) interface communicates with a BaseX server instance in the background, you can as well set the TIMEOUT option to limit your query execution time. Hope this helps, Christian

Quoted text · Jan Vlčinský (CAD) · 62 lines

> > 2011/2/24 Christian Grün <christian.gruen at gmail.com> >> >> >> I see. Can you remark on how hard it might be to specify a parameter in >> >> the appropriate query methods which would designate a maximum execution >> >> time >> >> before the query is cleanly aborted (in "wall clock time")? This would >> >> be >> >> in a client/server context. >> > >> > Phil >> > At http://docs.basex.org/wiki/Options#Server_options is mentioned server >> > wide option TIMEOUT, which can limit time for query execution. >> > I wonder, if there is any way to specify this per query. >> > Jan >> >> ....exactly; this option will do exactly what you need. >> >> Thanks, >> Christian >> >> >> >> > >> > 2011/2/24 NewIntellectual <newintellectual at gmail.com> >> >> >> >> On Wed, Feb 23, 2011 at 8:20 PM, Christian Grün >> >> <christian.gruen at gmail.com> wrote: >> >>> >> >>> as far as I can judge, XQuery expressions are too heterogeneous to >> >>> provide realistic cost estimations. While our query compiler has some >> >>> notion of "costs", those values are all relative, and used to select >> >>> the (presumably) cheapest query plan or index access. >> >> >> >> >> >> _______________________________________________ >> >> BaseX-Talk mailing list >> >> BaseX-Talk at mailman.uni-konstanz.de >> >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> >> >> > >> > >> > >> > -- >> > Ing. Jan Vlčinský >> > CAD programy >> > Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic >> > tel: +420-597 602 024; mob: +420-608 979 040 >> > skype: janvlcinsky; GoogleTalk: jan.vlcinsky at gmail.com >> > http://cz.linkedin.com/in/vlcinsky >> > > > > > -- > Ing. Jan Vlčinský > CAD programy > Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic > tel: +420-597 602 024; mob: +420-608 979 040 > skype: janvlcinsky; GoogleTalk: jan.vlcinsky at gmail.com > http://cz.linkedin.com/in/vlcinsky >

Jan Vlčinský (CAD)24 Feb 2011 14:28 #

OK, I understand, that REST allows to perform the (SET) command, but to use it with a query, I have to send the query in another, following, request. Will this second request keep enough context at the server side to know, it shall use that timeout? As REST is (or shall be) stateless, no context shall be assumed, so the timeout shall be forgotten. Or is there an option to send query AND Jan 2011/2/24 Christian Grün <christian.gruen at gmail.com>

Quoted text · Christian Grün · 80 lines

> > Another question is - is there any way to set timeout when using REST > > interface? > > Jan > > As our JAX-RX (REST) interface communicates with a BaseX server > instance in the background, you can as well set the TIMEOUT option to > limit your query execution time. > > Hope this helps, > Christian > > > > > > 2011/2/24 Christian Grün <christian.gruen at gmail.com> > >> > >> >> I see. Can you remark on how hard it might be to specify a parameter > in > >> >> the appropriate query methods which would designate a maximum > execution > >> >> time > >> >> before the query is cleanly aborted (in "wall clock time")? This > would > >> >> be > >> >> in a client/server context. > >> > > >> > Phil > >> > At http://docs.basex.org/wiki/Options#Server_options is mentioned > server > >> > wide option TIMEOUT, which can limit time for query execution. > >> > I wonder, if there is any way to specify this per query. > >> > Jan > >> > >> ....exactly; this option will do exactly what you need. > >> > >> Thanks, > >> Christian > >> > >> > >> > >> > > >> > 2011/2/24 NewIntellectual <newintellectual at gmail.com> > >> >> > >> >> On Wed, Feb 23, 2011 at 8:20 PM, Christian Grün > >> >> <christian.gruen at gmail.com> wrote: > >> >>> > >> >>> as far as I can judge, XQuery expressions are too heterogeneous to > >> >>> provide realistic cost estimations. While our query compiler has > some > >> >>> notion of "costs", those values are all relative, and used to select > >> >>> the (presumably) cheapest query plan or index access. > >> >> > >> >> > >> >> _______________________________________________ > >> >> BaseX-Talk mailing list > >> >> BaseX-Talk at mailman.uni-konstanz.de > >> >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >> >> > >> > > >> > > >> > > >> > -- > >> > Ing. Jan Vlčinský > >> > CAD programy > >> > Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic > >> > tel: +420-597 602 024; mob: +420-608 979 040 > >> > skype: janvlcinsky; GoogleTalk: jan.vlcinsky at gmail.com > >> > http://cz.linkedin.com/in/vlcinsky > >> > > > > > > > > > -- > > Ing. Jan Vlčinský > > CAD programy > > Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic > > tel: +420-597 602 024; mob: +420-608 979 040 > > skype: janvlcinsky; GoogleTalk: jan.vlcinsky at gmail.com > > http://cz.linkedin.com/in/vlcinsky > > >

Jan Vlčinský (CAD)24 Feb 2011 14:29 #

OK, I understand, that REST allows to perform the (SET) command, but to use it with a query, I have to send the query in another, following, request. Will this second request keep enough context at the server side to know, it shall use that timeout? As REST is (or shall be) stateless, no context shall be assumed, so the timeout shall be forgotten. Or is there an option to send query AND the set command in one request? Jan PS: Sorry for previous incomplete e-mail 2011/2/24 Jan Vlčinský (CAD) <jan.vlcinsky at cad-programs.com>

Quoted text · Jan Vlčinský (CAD) · 99 lines

> > > Jan > > 2011/2/24 Christian Grün <christian.gruen at gmail.com> > >> > Another question is - is there any way to set timeout when using REST >> > interface? >> > Jan >> >> As our JAX-RX (REST) interface communicates with a BaseX server >> instance in the background, you can as well set the TIMEOUT option to >> limit your query execution time. >> >> Hope this helps, >> Christian >> >> >> > >> > 2011/2/24 Christian Grün <christian.gruen at gmail.com> >> >> >> >> >> I see. Can you remark on how hard it might be to specify a parameter >> in >> >> >> the appropriate query methods which would designate a maximum >> execution >> >> >> time >> >> >> before the query is cleanly aborted (in "wall clock time")? This >> would >> >> >> be >> >> >> in a client/server context. >> >> > >> >> > Phil >> >> > At http://docs.basex.org/wiki/Options#Server_options is mentioned >> server >> >> > wide option TIMEOUT, which can limit time for query execution. >> >> > I wonder, if there is any way to specify this per query. >> >> > Jan >> >> >> >> ....exactly; this option will do exactly what you need. >> >> >> >> Thanks, >> >> Christian >> >> >> >> >> >> >> >> > >> >> > 2011/2/24 NewIntellectual <newintellectual at gmail.com> >> >> >> >> >> >> On Wed, Feb 23, 2011 at 8:20 PM, Christian Grün >> >> >> <christian.gruen at gmail.com> wrote: >> >> >>> >> >> >>> as far as I can judge, XQuery expressions are too heterogeneous to >> >> >>> provide realistic cost estimations. While our query compiler has >> some >> >> >>> notion of "costs", those values are all relative, and used to >> select >> >> >>> the (presumably) cheapest query plan or index access. >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> >> BaseX-Talk mailing list >> >> >> BaseX-Talk at mailman.uni-konstanz.de >> >> >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> >> >> >> >> > >> >> > >> >> > >> >> > -- >> >> > Ing. Jan Vlčinský >> >> > CAD programy >> >> > Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic >> >> > tel: +420-597 602 024; mob: +420-608 979 040 >> >> > skype: janvlcinsky; GoogleTalk: jan.vlcinsky at gmail.com >> >> > http://cz.linkedin.com/in/vlcinsky >> >> > >> > >> > >> > >> > -- >> > Ing. Jan Vlčinský >> > CAD programy >> > Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic >> > tel: +420-597 602 024; mob: +420-608 979 040 >> > skype: janvlcinsky; GoogleTalk: jan.vlcinsky at gmail.com >> > http://cz.linkedin.com/in/vlcinsky >> > >> > > > > -- > *Ing. Jan Vlčinský* > CAD programy > Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic > tel: +420-597 602 024; mob: +420-608 979 040 > skype: janvlcinsky; GoogleTalk: jan.vlcinsky at gmail.com > http://cz.linkedin.com/in/vlcinsky > >

Christian Grün24 Feb 2011 15:27 #

Quoted text · Jan Vlčinský (CAD) · 4 lines

> OK, I understand, that REST allows to perform the (SET) command, but to use > it with a query, I have to send the query in another, following, request. > Will this second request keep enough context at the server side to know, it > shall use that timeout?

Just recently, I've thought about extending the JAX-RX Interface to allow more than a single query or command, or to allow both commands and queries; regarding the syntax, this could be easily realized for the GET method: http://localhost:8984/basex/jax-rx/factbook?query=1&query=2 http://localhost:8984/basex/jax-rx/factbook?command=set+timeout+1&query=1&command=info On the other hand, I haven't decided yet how to best realize it for the POST method without giving up compatibility. One solution could be to introduce alternative "request" root elements:

  <request xmlns="http://jax-rx.sourceforge.net">
    <command>
      <text>set timeout 1</text>
    </command>
    <query>
      <text>//name</text>
      <parameter name="count" value="5"/>
    </query>
  </request>

What does everyone think? Christian

NewIntellectual24 Feb 2011 23:57 #

Thanks, all, for your answers.

Written for BaseX 6. BaseX may have changed since: please check the current documentation.

⚡Generated with XQuery

`J`	Select the next message
`K`	Select the previous message
`N`	Next thread
`P`	Previous thread
`U`	Go up one level
`/`	Jump to the search field