Using an XSD with BaseX (to provide extra type information)
Wednesday, 22 May 2013 · 5 messages · last reply 5 Jun 2013
LK
Ludovic Kuty22 May 2013 16:33 #
Hello,
I would like to know if BaseX will use the XSD referenced in an XML document to get extra informations about types (he PSVI) and if it will validate the document in the process. If it does so, where do I have to store the XSD ? I would like to store it inside the database itself (or another one). I could also store it in the Web application provided by BaseX and make my URL point to it.
In my opinion, the use of the validation module should not be necessary as it should happen automatically.
Thank in advance for your insights and informations,
Ludovic Kuty
CG
Christian Grün26 May 2013 11:45 #
Hi Ludovic,
sorry for the delay. The current version of BaseX provides no schema support; instead, some type information for optimizing queries will be derived from the data itself. I agree that a schema-driven approach may be beneficial (in particular if you are luck to *have* good schemas for your data, and not some of those which have been extended and adapted over years to fit all kinds of new requirements), but it would mean some extra effort to include this in our implementation.
Best,
Christian
Quoted text · Ludovic Kuty · 15 lines
On Wed, May 22, 2013 at 4:33 PM, Ludovic Kuty <mailing at kuty.be> wrote: > Hello, > > I would like to know if BaseX will use the XSD referenced in an XML document to get extra informations about types (he PSVI) and if it will validate the document in the process. If it does so, where do I have to store the XSD ? I would like to store it inside the database itself (or another one). I could also store it in the Web application provided by BaseX and make my URL point to it. > > In my opinion, the use of the validation module should not be necessary as it should happen automatically. > > Thank in advance for your insights and informations, > > Ludovic Kuty > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
LK
Ludovic Kuty28 May 2013 14:30 #
Thanks for your answer. No problem for the delay. Schema support is not necessary in our application even if our schemas are small, good and built from scratch.
Ludovic
Quoted text · Christian Grün · 30 lines
On 26 May 2013, at 11:45, Christian Grün wrote: > Hi Ludovic, > > sorry for the delay. The current version of BaseX provides no schema > support; instead, some type information for optimizing queries will be > derived from the data itself. I agree that a schema-driven approach > may be beneficial (in particular if you are luck to *have* good > schemas for your data, and not some of those which have been extended > and adapted over years to fit all kinds of new requirements), but it > would mean some extra effort to include this in our implementation. > > Best, > Christian > ___________________________ > > On Wed, May 22, 2013 at 4:33 PM, Ludovic Kuty <mailing at kuty.be> wrote: >> Hello, >> >> I would like to know if BaseX will use the XSD referenced in an XML document to get extra informations about types (he PSVI) and if it will validate the document in the process. If it does so, where do I have to store the XSD ? I would like to store it inside the database itself (or another one). I could also store it in the Web application provided by BaseX and make my URL point to it. >> >> In my opinion, the use of the validation module should not be necessary as it should happen automatically. >> >> Thank in advance for your insights and informations, >> >> Ludovic Kuty >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
WP
Dear Christian,
Could we please have more detail regarding BaseX's inferences of types in the data?
This is my scenario:
I have XML documents generated automatically, for which I have an XSD which you can consider "good" in the sense you imply in your note to Ludovic. The data is tight and it's valid, but the documents are not small (MB not KB).
I need to perform numeric comparison operations of the sort "($a/@x ge $b/@x) and ($a/@y le $b/@y)". @x and @y are attributes whose values are always integers. There are a lot of these comparisons and they have to happen fast.
In general, my assumption would be that a schema declaring my attributes as integers would enable a processor to optimize both their storage and processing (value comparisons).
But maybe in BaseX I don't have to do this?
If not, is there a way to pre-process my data to provide the type information for BaseX, without the XSD?
Thanks, Wendell
Quoted text · Christian Grün · 35 lines
On Sun, May 26, 2013 at 5:45 AM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Ludovic, > > sorry for the delay. The current version of BaseX provides no schema > support; instead, some type information for optimizing queries will be > derived from the data itself. I agree that a schema-driven approach > may be beneficial (in particular if you are luck to *have* good > schemas for your data, and not some of those which have been extended > and adapted over years to fit all kinds of new requirements), but it > would mean some extra effort to include this in our implementation. > > Best, > Christian > ___________________________ > > On Wed, May 22, 2013 at 4:33 PM, Ludovic Kuty <mailing at kuty.be> wrote: >> Hello, >> >> I would like to know if BaseX will use the XSD referenced in an XML document to get extra informations about types (he PSVI) and if it will validate the document in the process. If it does so, where do I have to store the XSD ? I would like to store it inside the database itself (or another one). I could also store it in the Web application provided by BaseX and make my URL point to it. >> >> In my opinion, the use of the validation module should not be necessary as it should happen automatically. >> >> Thank in advance for your insights and informations, >> >> Ludovic Kuty >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
--
Wendell Piez | http://www.wendellpiez.com
XML | XSLT | electronic publishing
Eat Your Vegetables
_____oo_________o_o___ooooo____ooooooo_^
CG
Christian Grün5 Jun 2013 12:03 #
Hi Wendell,
sorry for letting you wait.
Quoted text · Wendell Piez · 2 lines
> Could we please have more detail regarding BaseX's inferences of types > in the data?
Sure. If the string value of an attribute or element only includes numbers, this will be remembered in the database statistics of names and paths (those can be viewed in the GUI via Database → Properties → Names/Path Summary, by playing around with the Explorer View visualization, or by analyzing the input of the "show index" command and the XQuerz index functions [1]).
In terms of performance, the query is rewritten to an index-supported range query. If we have an attribute "income", which only contains numeric values, the following query..
//*[@income >= 10000 and @income <= 40000]
..will be rewritten for index access and usually sped up. We currently have no pure range index implementation for numbers and dates, which would be faster, but if you have integer values of the same length (which is e.g. the case when looking for year values), the string range index can be helpful [2].
Hope this helps,
Christian
[1] http://docs.basex.org/wiki/Index_Module#index:element-names [2] http://docs.basex.org/wiki/Indexes#Range_Queries
Quoted text · Wendell Piez · 59 lines
> In general, my assumption would be that a schema declaring my > attributes as integers would enable a processor to optimize both their > storage and processing (value comparisons). > > But maybe in BaseX I don't have to do this? > > If not, is there a way to pre-process my data to provide the type > information for BaseX, without the XSD? > > Thanks, Wendell > > > On Sun, May 26, 2013 at 5:45 AM, Christian Grün > <christian.gruen at gmail.com> wrote: >> Hi Ludovic, >> >> sorry for the delay. The current version of BaseX provides no schema >> support; instead, some type information for optimizing queries will be >> derived from the data itself. I agree that a schema-driven approach >> may be beneficial (in particular if you are luck to *have* good >> schemas for your data, and not some of those which have been extended >> and adapted over years to fit all kinds of new requirements), but it >> would mean some extra effort to include this in our implementation. >> >> Best, >> Christian >> ___________________________ >> >> On Wed, May 22, 2013 at 4:33 PM, Ludovic Kuty <mailing at kuty.be> wrote: >>> Hello, >>> >>> I would like to know if BaseX will use the XSD referenced in an XML document to get extra informations about types (he PSVI) and if it will validate the document in the process. If it does so, where do I have to store the XSD ? I would like to store it inside the database itself (or another one). I could also store it in the Web application provided by BaseX and make my URL point to it. >>> >>> In my opinion, the use of the validation module should not be necessary as it should happen automatically. >>> >>> Thank in advance for your insights and informations, >>> >>> Ludovic Kuty >>> >>> _______________________________________________ >>> BaseX-Talk mailing list >>> BaseX-Talk at mailman.uni-konstanz.de >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > > -- > Wendell Piez | http://www.wendellpiez.com > XML | XSLT | electronic publishing > Eat Your Vegetables > _____oo_________o_o___ooooo____ooooooo_^ > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Written for BaseX 7.6. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery