count(//elem) not optimized, even though `elem` is in the index

> I wonder if the presence of the namespace somehow confuses the optimizer.

> I was stressing the BaseX 8.6 planner/optimizer when I noticed that > expressions like `count(//elem)` are not optimized at all, even though they > are correctly indexed, as demonstrated by `index:element-names()`. > > The current database is a 300 MB TEI document. All the elements are in the > `http://www.tei-c.org/ns/1.0` namespace. > > The following test case will report the correct number, but it will take a > couple of seconds to run, instead of a few milliseconds. > > ``` > declare namespace tei="http://www.tei-c.org/ns/1.0"; > > let $n := index:element-names("monier")[. = 're']/@count > > let $c := count(//tei:re) > > return <res><in-index>{$n}</in-index><in-doc>{$c}</in-doc></res> > ``` > > I wonder if the presence of the namespace somehow confuses the optimizer. > The same problem can be observed running the same test case with > > ``` > declare default element namespace "http://www.tei-c.org/ns/1.0"; > [...] > let $c := count(//re) > ``` > > Regards, > > -- > Gioele Barabucci <gioele.barabucci at uni-koeln.de> >

Am 22.02.2017 um 15:53 schrieb Christian Grün: >> I wonder if the presence of the namespace somehow confuses the optimizer. > > For the same reason, various optimizations that are based on the > database statistics will only get into effect if a document contains > no, or at most one global, namespace declaration.

From: basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of Christian Grün Sent: Wednesday, February 22, 2017 9:54 AM To: Gioele Barabucci <gioele at svario.it> Cc: BaseX <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] count(//elem) not optimized, even though `elem` is in the index Hi Gioele, > I wonder if the presence of the namespace somehow confuses the optimizer. Exactly, that’s the reason. For some historical reason (but not such a wise one, as most quoted “historical reasons” are), we decided to index the node names without considering the namespace URI. As a result, the index:element-names function will yield… <entry count="2">xml</entry> …for the following document: <xml> <xml xmlns='uri'/> </xml> For the same reason, various optimizations that are based on the database statistics will only get into effect if a document contains no, or at most one global, namespace declaration. In various cases, optimizations could still be made possible (e.g. if we know that the element/attribute names with and without namespace URIs are distinct), but that hasn’t been implemented so far. Cheers, Christian > I was stressing the BaseX 8.6 planner/optimizer when I noticed that > expressions like `count(//elem)` are not optimized at all, even though they > are correctly indexed, as demonstrated by `index:element-names()`. > > The current database is a 300 MB TEI document. All the elements are in the > `http://www.tei-c.org/ns/1.0<http://www.tei-c.org/ns/1.0>` namespace. > > The following test case will report the correct number, but it will take a > couple of seconds to run, instead of a few milliseconds. > > ``` > declare namespace tei="http://www.tei-c.org/ns/1.0<http://www.tei-c.org/ns/1.0>"; > > let $n := index:element-names("monier")[. = 're']/@count > > let $c := count(//tei:re) > > return <res><in-index>{$n}</in-index><in-doc>{$c}</in-doc></res> > ``` > > I wonder if the presence of the namespace somehow confuses the optimizer. > The same problem can be observed running the same test case with > > ``` > declare default element namespace "http://www.tei-c.org/ns/1.0<http://www.tei-c.org/ns/1.0>"; > [...] > let $c := count(//re) > ``` > > Regards, > > -- > Gioele Barabucci <gioele.barabucci at uni-koeln.de<mailto:gioele.barabucci at uni-koeln.de>> >

> The first question: in my case all the elements are in one namespace, > declared in the root element. Shouldn't the optimizations kick in?

> Any workaround aside reimporting the database stripping the namespace > information?

> And now the request. Would it be possible to document in the wiki which > optimizations are not possible when namespaces (or more than one namespace) > are used?

> Does this mean, for a given set of XML files that have several namespace > declarations attached to the root element including a default namespace, if > namespaces are removed when these XML files are loaded into BaseX, for > example by using the “Strip namespaces” option in the GUI, BaseX may be able > to use additional query optimizations?