UPDINDEX on custom index structures not working... where is my missing link?

> Hi, > > I'm trying to build something that is similar to custom indices but that > actually returns resolved nodes (nodes created by manipulating and merging > nodes from the original DB). > > 1- I created a script that creates new index DBs. UPDINDEX and > AUTOOPTIMIZED are set on both the 'index' DB and the DB from which the new > DBs were created > 2- I can create a query on any of the new 'index' DBs to extract the nodes > I want. > 3- After I change the content in the source DB, the response shows that > the nodes in the 'index' DBs are not updated. I get the old result. > > I feel that I am missing a link between the source and the index DBs to > make the update mechanism work. Since I do not want to get the original > node, but one that I create, I did not use db:node-id or db:open-id shown > in the documentation example. Is that where the link is created? > > CODE THAT CREATES THE INDEX DBs > > let $langs := > for $lang in ('en-us', 'es-es') (:db:list()[string-length(.) = > 5][substring(., 3, 1) = '-']:) > return $lang > return > for $lang in $langs > let $index := <index>{ > for $string-group in > db:open('global-content')//*[name()='string-group'] > let $prompt-this-lang := > $string-group/child::*[@xml:lang=$lang][normalize-space(.)!=''] > let $prompt-base := > if (exists($prompt-this-lang)) > then $prompt-this-lang > else $string-group/child::*[@xml:lang='en-us'] > let $prompts := > for $prompt-match in > db:open('global-content')/*[name()='prompt-refs']/*[name()='prompt'][@package=$string-group/data(@package)][@pkg-version=$string-group/data(@pkg-version)][@key=$string-group/data(@key)] > return > copy $copy := $prompt-base > modify ( > for $attr in $prompt-match/@*[name()!='package' > and name()!='pkg-version' and name()!='key'] > return insert node $attr into $copy > ) > return $copy > > return > $prompts > > > }</index> > > return db:create('index-prompt-' || $lang, $index, 'prompts-' || $lang > || '.xml', map { 'updindex': true(), 'autooptimize': true() }) > > CODE THAT QUERIES THE 'INDEX' DBS > > (# db:enforceindex #) { > for $prompt in db:attribute('index-prompt-en-us', 'setup_as_new', > 'name')/.. > return $prompt => prof:time() > } > > -- > France Baril > Architecte documentaire / Documentation architect > france.baril at architextus.com >

On Sat, Dec 21, 2019 at 9:19 AM Christian Grün <christian.gruen at gmail.com> wrote: > Hi France, > > 1. If you use autooptimize, updindex won’t have any effect. What’s the > background for the combination you have chosen? > > 2. How does your the section of your query looks like in which you are > updating your index databases? > > Best, > Christian > > > > > > France Baril <france.baril at architextus.com> schrieb am Fr., 20. Dez. > 2019, 20:40: > >> Hi, >> >> I'm trying to build something that is similar to custom indices but that >> actually returns resolved nodes (nodes created by manipulating and merging >> nodes from the original DB). >> >> 1- I created a script that creates new index DBs. UPDINDEX and >> AUTOOPTIMIZED are set on both the 'index' DB and the DB from which the new >> DBs were created >> 2- I can create a query on any of the new 'index' DBs to extract the >> nodes I want. >> 3- After I change the content in the source DB, the response shows that >> the nodes in the 'index' DBs are not updated. I get the old result. >> >> I feel that I am missing a link between the source and the index DBs to >> make the update mechanism work. Since I do not want to get the original >> node, but one that I create, I did not use db:node-id or db:open-id shown >> in the documentation example. Is that where the link is created? >> >> CODE THAT CREATES THE INDEX DBs >> >> let $langs := >> for $lang in ('en-us', 'es-es') (:db:list()[string-length(.) = >> 5][substring(., 3, 1) = '-']:) >> return $lang >> return >> for $lang in $langs >> let $index := <index>{ >> for $string-group in >> db:open('global-content')//*[name()='string-group'] >> let $prompt-this-lang := >> $string-group/child::*[@xml:lang=$lang][normalize-space(.)!=''] >> let $prompt-base := >> if (exists($prompt-this-lang)) >> then $prompt-this-lang >> else $string-group/child::*[@xml:lang='en-us'] >> let $prompts := >> for $prompt-match in >> db:open('global-content')/*[name()='prompt-refs']/*[name()='prompt'][@package=$string-group/data(@package)][@pkg-version=$string-group/data(@pkg-version)][@key=$string-group/data(@key)] >> return >> copy $copy := $prompt-base >> modify ( >> for $attr in $prompt-match/@*[name()!='package' >> and name()!='pkg-version' and name()!='key'] >> return insert node $attr into $copy >> ) >> return $copy >> >> return >> $prompts >> >> >> }</index> >> >> return db:create('index-prompt-' || $lang, $index, 'prompts-' || $lang >> || '.xml', map { 'updindex': true(), 'autooptimize': true() }) >> >> CODE THAT QUERIES THE 'INDEX' DBS >> >> (# db:enforceindex #) { >> for $prompt in db:attribute('index-prompt-en-us', 'setup_as_new', >> 'name')/.. >> return $prompt => prof:time() >> } >> >> -- >> France Baril >> Architecte documentaire / Documentation architect >> france.baril at architextus.com >> >

On Sat, Dec 21, 2019 at 11:17 AM France Baril <france.baril at architextus.com> wrote: > Hi > > 1. I tried UPDINDEX and when it didn't work I figured I would need > to AUTOOPTIMIZE, I just removed it, > 2. I only have the create .xq, If I need to add code to update, I don't > know where/how to connect it, which brings me back to my question, how do I > connect my query so it will update my indices. That's where the missing > link is. > > > > On Sat, Dec 21, 2019 at 9:19 AM Christian Grün <christian.gruen at gmail.com> > wrote: > >> Hi France, >> >> 1. If you use autooptimize, updindex won’t have any effect. What’s the >> background for the combination you have chosen? >> >> 2. How does your the section of your query looks like in which you are >> updating your index databases? >> >> Best, >> Christian >> >> >> >> >> >> France Baril <france.baril at architextus.com> schrieb am Fr., 20. Dez. >> 2019, 20:40: >> >>> Hi, >>> >>> I'm trying to build something that is similar to custom indices but that >>> actually returns resolved nodes (nodes created by manipulating and merging >>> nodes from the original DB). >>> >>> 1- I created a script that creates new index DBs. UPDINDEX and >>> AUTOOPTIMIZED are set on both the 'index' DB and the DB from which the new >>> DBs were created >>> 2- I can create a query on any of the new 'index' DBs to extract the >>> nodes I want. >>> 3- After I change the content in the source DB, the response shows that >>> the nodes in the 'index' DBs are not updated. I get the old result. >>> >>> I feel that I am missing a link between the source and the index DBs to >>> make the update mechanism work. Since I do not want to get the original >>> node, but one that I create, I did not use db:node-id or db:open-id shown >>> in the documentation example. Is that where the link is created? >>> >>> CODE THAT CREATES THE INDEX DBs >>> >>> let $langs := >>> for $lang in ('en-us', 'es-es') (:db:list()[string-length(.) = >>> 5][substring(., 3, 1) = '-']:) >>> return $lang >>> return >>> for $lang in $langs >>> let $index := <index>{ >>> for $string-group in >>> db:open('global-content')//*[name()='string-group'] >>> let $prompt-this-lang := >>> $string-group/child::*[@xml:lang=$lang][normalize-space(.)!=''] >>> let $prompt-base := >>> if (exists($prompt-this-lang)) >>> then $prompt-this-lang >>> else $string-group/child::*[@xml:lang='en-us'] >>> let $prompts := >>> for $prompt-match in >>> db:open('global-content')/*[name()='prompt-refs']/*[name()='prompt'][@package=$string-group/data(@package)][@pkg-version=$string-group/data(@pkg-version)][@key=$string-group/data(@key)] >>> return >>> copy $copy := $prompt-base >>> modify ( >>> for $attr in $prompt-match/@*[name()!='package' >>> and name()!='pkg-version' and name()!='key'] >>> return insert node $attr into $copy >>> ) >>> return $copy >>> >>> return >>> $prompts >>> >>> >>> }</index> >>> >>> return db:create('index-prompt-' || $lang, $index, 'prompts-' || >>> $lang || '.xml', map { 'updindex': true(), 'autooptimize': true() }) >>> >>> CODE THAT QUERIES THE 'INDEX' DBS >>> >>> (# db:enforceindex #) { >>> for $prompt in db:attribute('index-prompt-en-us', 'setup_as_new', >>> 'name')/.. >>> return $prompt => prof:time() >>> } >>> >>> -- >>> France Baril >>> Architecte documentaire / Documentation architect >>> france.baril at architextus.com >>> >> > > -- > France Baril > Architecte documentaire / Documentation architect > france.baril at architextus.com >

> > 2. I only have the create .xq, If I need to add code to update, I don't > know where/how to connect it, which brings me back to my question, how do I > connect my query so it will update my indices. That's where the missing > link is. >

On Sat, Dec 21, 2019 at 11:35 AM Christian Grün <christian.gruen at gmail.com> wrote: > 2. I only have the create .xq, If I need to add code to update, I don't >> know where/how to connect it, which brings me back to my question, how do I >> connect my query so it will update my indices. That's where the missing >> link is. >> > > It’s not sufficient to connect your query, as it (as far as I can see) > recreates the full index. Instead you’ll need to write additional code that > updates the relevant parts of your custom index structure. > > Hope this helps > Christian > > >

> I'm good with writing the code, but how do I make it run automatically on > content update of the original source DB? What I am missing is where/how to > connect that code so it runs on its own. > > On Sat, Dec 21, 2019 at 11:35 AM Christian Grün <christian.gruen at gmail.com> > wrote: > >> 2. I only have the create .xq, If I need to add code to update, I don't >>> know where/how to connect it, which brings me back to my question, how do I >>> connect my query so it will update my indices. That's where the missing >>> link is. >>> >> >> It’s not sufficient to connect your query, as it (as far as I can see) >> recreates the full index. Instead you’ll need to write additional code that >> updates the relevant parts of your custom index structure. >> >> Hope this helps >> Christian >> >> >> > > -- > France Baril > Architecte documentaire / Documentation architect > france.baril at architextus.com >

On Sat, Dec 21, 2019 at 12:07 PM Christian Grün <christian.gruen at gmail.com> wrote: > Your code needs to take care of both the actual update and the index > update. As indicated in another thread on this list this or last week, the > integration of triggers (with pre/post commit hooks) may allow you to > separate these two tasks in a future version of BaseX. > > > > France Baril <france.baril at architextus.com> schrieb am Sa., 21. Dez. > 2019, 11:53: > >> I'm good with writing the code, but how do I make it run automatically on >> content update of the original source DB? What I am missing is where/how to >> connect that code so it runs on its own. >> >> On Sat, Dec 21, 2019 at 11:35 AM Christian Grün < >> christian.gruen at gmail.com> wrote: >> >>> 2. I only have the create .xq, If I need to add code to update, I don't >>>> know where/how to connect it, which brings me back to my question, how do I >>>> connect my query so it will update my indices. That's where the missing >>>> link is. >>>> >>> >>> It’s not sufficient to connect your query, as it (as far as I can see) >>> recreates the full index. Instead you’ll need to write additional code that >>> updates the relevant parts of your custom index structure. >>> >>> Hope this helps >>> Christian >>> >>> >>> >> >> -- >> France Baril >> Architecte documentaire / Documentation architect >> france.baril at architextus.com >> >