Main Page » Mailing List » July 2019

Enforce index with dynamic values

Wednesday, 10 July 2019 · 4 messages · last reply 12 Jul 2019
BJ
Beck, Julia10 Jul 2019 15:17 #
Hello BaseX team, I am struggeling with the enforce index pragma and get a "Improper use? Potential bug? Your feedback is welcome". So here is my feedback/question: I have XML data describing books in several databases that also contain (potentially outdated) control numbers for authority data and I have an XML doc with a list of old numbers and their replacement (== new number). So I want to go through this list and replace the old numbers in my data with the new numbers. My XQuery works for each database on its own, but as soon as I provide a list of databases, I get a NullPointerException. The xml doc with the authority numbers looks like this:
<gnds>
  <gnd>
    <old>gnd_4452199-6</old>;
    <new>gnd_2163369-1</new>;
  </gnd>
  <gnd>
    <old>gnd_16288049-2</old>;
    <old>gnd_4683522-2</old>;
    <new>gnd_1901229-9</new>;
  </gnd>
 ....
</gnds>
The data in the databases looks like this:
<rdf:RDF>
  <edm:ProvidedCHO @rdf:about="...">
    <edm:wasPresentAt @rdf:resource="gnd_4452199-6"/>
    <dc:contributor @rdf:resource="gnd_4683522-2"/>
    ...
 </edm:ProvidedCHO>
 ...
</rdf:RDF>
In the XQuery, I left out the replacement part (which I thought to be the problem first, but the script already throws the error when I just want to have the new numbers printed). It works when I run it for "tmd" and "tes" individually. When I leave out the pragma it takes really long time (obviously not using the index). I more or less followed the 3rd example in http://docs.basex.org/wiki/Indexes#Enforce_Rewritings
let $old_gnds := doc("oldIds.xml")/gnds/gnd,
    $all_db := ("tmd","tes")
for $gnd in $old_gnds/old/text()
  let $new := $gnd/../../new/text()
  for $db in $all_db
    for $attr in db:open($db)/rdf:RDF/*/*/@rdf:resource
    where (# db:enforceindex #) {$attr = $gnd}
    return $new
Best wishes, Julia
-- JULIA BECK Elektronische Dienste / FIDDK Universitätsbibliothek Johann Christian Senckenberg Bockenheimer Landstraße 134-138 60325 Frankfurt am Main 069 798 39387
BJ
Beck, Julia10 Jul 2019 15:57 #
I forgot to mention: I am using BaseX 9.2.3 and the databases both have a text and an attribute index that are "UPTODATE: true".
Quoted text · +0000 · 68 lines
Am Mittwoch, den 10.07.2019, 13:17 +0000 schrieb Beck, Julia: > Hello BaseX team, > > I am struggeling with the enforce index pragma and get a "Improper > use? > Potential bug? Your feedback is welcome". So here is my > feedback/question: > > I have XML data describing books in several databases that also > contain > (potentially outdated) control numbers for authority data and I have > an > XML doc with a list of old numbers and their replacement (== new > number). So I want to go through this list and replace the old > numbers > in my data with the new numbers. My XQuery works for each database on > its own, but as soon as I provide a list of databases, I get a > NullPointerException. > > The xml doc with the authority numbers looks like this: > > <gnds> > <gnd> > <old>gnd_4452199-6</old>; > <new>gnd_2163369-1</new>; > </gnd> > <gnd> > <old>gnd_16288049-2</old>; > <old>gnd_4683522-2</old>; > <new>gnd_1901229-9</new>; > </gnd> > .... > </gnds> > > The data in the databases looks like this: > > <rdf:RDF> > <edm:ProvidedCHO @rdf:about="..."> > <edm:wasPresentAt @rdf:resource="gnd_4452199-6"/> > <dc:contributor @rdf:resource="gnd_4683522-2"/> > ... > </edm:ProvidedCHO> > ... > </rdf:RDF> > > In the XQuery, I left out the replacement part (which I thought to be > the problem first, but the script already throws the error when I > just > want to have the new numbers printed). It works when I run it for > "tmd" > and "tes" individually. When I leave out the pragma it takes really > long time (obviously not using the index). > I more or less followed the 3rd example in > http://docs.basex.org/wiki/Indexes#Enforce_Rewritings > > let $old_gnds := doc("oldIds.xml")/gnds/gnd, > $all_db := ("tmd","tes") > for $gnd in $old_gnds/old/text() > let $new := $gnd/../../new/text() > for $db in $all_db > for $attr in db:open($db)/rdf:RDF/*/*/@rdf:resource > where (# db:enforceindex #) {$attr = $gnd} > return $new > > Best wishes, > Julia > >
CG
Christian Grünin reply to Beck, Julia, 10 Jul 2019, 15:1710 Jul 2019 17:59 #
Hi Julia, Thanks for the report. I’ve opened an issue to track this down [1], and I’ll give you feedback once the bug has been fixed. All the best from Konstanz, Christian [1] https://github.com/BaseXdb/basex/issues/1696
Quoted text · Julia · 71 lines
On Wed, Jul 10, 2019 at 3:20 PM Beck, Julia <J.Beck at ub.uni-frankfurt.de> wrote: > > Hello BaseX team, > > I am struggeling with the enforce index pragma and get a "Improper use? > Potential bug? Your feedback is welcome". So here is my > feedback/question: > > I have XML data describing books in several databases that also contain > (potentially outdated) control numbers for authority data and I have an > XML doc with a list of old numbers and their replacement (== new > number). So I want to go through this list and replace the old numbers > in my data with the new numbers. My XQuery works for each database on > its own, but as soon as I provide a list of databases, I get a > NullPointerException. > > The xml doc with the authority numbers looks like this: > > <gnds> > <gnd> > <old>gnd_4452199-6</old>; > <new>gnd_2163369-1</new>; > </gnd> > <gnd> > <old>gnd_16288049-2</old>; > <old>gnd_4683522-2</old>; > <new>gnd_1901229-9</new>; > </gnd> > .... > </gnds> > > The data in the databases looks like this: > > <rdf:RDF> > <edm:ProvidedCHO @rdf:about="..."> > <edm:wasPresentAt @rdf:resource="gnd_4452199-6"/> > <dc:contributor @rdf:resource="gnd_4683522-2"/> > ... > </edm:ProvidedCHO> > ... > </rdf:RDF> > > In the XQuery, I left out the replacement part (which I thought to be > the problem first, but the script already throws the error when I just > want to have the new numbers printed). It works when I run it for "tmd" > and "tes" individually. When I leave out the pragma it takes really > long time (obviously not using the index). > I more or less followed the 3rd example in > http://docs.basex.org/wiki/Indexes#Enforce_Rewritings > > let $old_gnds := doc("oldIds.xml")/gnds/gnd, > $all_db := ("tmd","tes") > for $gnd in $old_gnds/old/text() > let $new := $gnd/../../new/text() > for $db in $all_db > for $attr in db:open($db)/rdf:RDF/*/*/@rdf:resource > where (# db:enforceindex #) {$attr = $gnd} > return $new > > Best wishes, > Julia > > > -- > JULIA BECK > Elektronische Dienste / FIDDK > > Universitätsbibliothek Johann Christian Senckenberg > Bockenheimer Landstraße 134-138 > 60325 Frankfurt am Main > 069 798 39387
CG
Christian Grünin reply to Beck, Julia, 10 Jul 2019, 15:1712 Jul 2019 13:50 #
Hi Julia, The index rewriting code has been fixed (the nifty details: in the cost estimation step, the number of results was computed although the info is not statically available). A new snapshot is available [1]. In order to enforce the rewriting, you may possibly need to wrap the pragma around around your db:open function and the path expression:
  for $attr in (# db:enforceindex #) {
    db:open($db)/rdf:RDF/*/*/@rdf:resource[. = $gnd]
  }
Hope this helps, have fun, Christian [1] http://files.basex.org/releases/latest/
Quoted text · Julia · 71 lines
On Wed, Jul 10, 2019 at 3:20 PM Beck, Julia <J.Beck at ub.uni-frankfurt.de> wrote: > > Hello BaseX team, > > I am struggeling with the enforce index pragma and get a "Improper use? > Potential bug? Your feedback is welcome". So here is my > feedback/question: > > I have XML data describing books in several databases that also contain > (potentially outdated) control numbers for authority data and I have an > XML doc with a list of old numbers and their replacement (== new > number). So I want to go through this list and replace the old numbers > in my data with the new numbers. My XQuery works for each database on > its own, but as soon as I provide a list of databases, I get a > NullPointerException. > > The xml doc with the authority numbers looks like this: > > <gnds> > <gnd> > <old>gnd_4452199-6</old>; > <new>gnd_2163369-1</new>; > </gnd> > <gnd> > <old>gnd_16288049-2</old>; > <old>gnd_4683522-2</old>; > <new>gnd_1901229-9</new>; > </gnd> > .... > </gnds> > > The data in the databases looks like this: > > <rdf:RDF> > <edm:ProvidedCHO @rdf:about="..."> > <edm:wasPresentAt @rdf:resource="gnd_4452199-6"/> > <dc:contributor @rdf:resource="gnd_4683522-2"/> > ... > </edm:ProvidedCHO> > ... > </rdf:RDF> > > In the XQuery, I left out the replacement part (which I thought to be > the problem first, but the script already throws the error when I just > want to have the new numbers printed). It works when I run it for "tmd" > and "tes" individually. When I leave out the pragma it takes really > long time (obviously not using the index). > I more or less followed the 3rd example in > http://docs.basex.org/wiki/Indexes#Enforce_Rewritings > > let $old_gnds := doc("oldIds.xml")/gnds/gnd, > $all_db := ("tmd","tes") > for $gnd in $old_gnds/old/text() > let $new := $gnd/../../new/text() > for $db in $all_db > for $attr in db:open($db)/rdf:RDF/*/*/@rdf:resource > where (# db:enforceindex #) {$attr = $gnd} > return $new > > Best wishes, > Julia > > > -- > JULIA BECK > Elektronische Dienste / FIDDK > > Universitätsbibliothek Johann Christian Senckenberg > Bockenheimer Landstraße 134-138 > 60325 Frankfurt am Main > 069 798 39387

Written for BaseX 9.2. BaseX may have changed since: please check the current documentation.

« Time complexityFile-io:error. File lock by another process (windows-only) »
⚡Generated with XQuery