Paging results of an XQuery-Search
Hi all, is it possible to present the results of an XQuery search in parts of 50? In order to speed up the search, only the first 50 results should be displayed and only if the user is interested in further results, 50 new results each should be displayed. Is this even possible? I did not find anything about it in the documentation.
Example:
let $collection := collection("data")
let $hits := $collection//*:s[.//text() contains text {$query_string}]
let $count_hits := count($hits)
return
<div>
{
if $count_hits <= 50 then
<ol>
{for $hit in $hits
return
<li>{$hit}</li>
</ol>
else
???
}
</div>Thanks for any advice. Best regards Guenter
Perhaps you want to use fn:subsequence($hits, $start, $num)? But of course if you have millions of hits that is not efficient enough.
Quoted text · Günter Dunz-Wolff · 27 lines
Am 25.09.2020 um 13:23 schrieb Günter Dunz-Wolff: > Hi all, > is it possible to present the results of an XQuery search in parts of 50? In order to speed up the search, only the first 50 results should be displayed and only if the user is interested in further results, 50 new results each should be displayed. Is this even possible? I did not find anything about it in the documentation. > > Example: > let $collection := collection("data") > let $hits := $collection//*:s[.//text() contains text {$query_string}] > let $count_hits := count($hits) > return > <div> > { > if $count_hits <= 50 then > <ol> > {for $hit in $hits > return > <li>{$hit}</li> > </ol> > else > ??? > } > </div> > > Thanks for any advice. > Best regards > Guenter > >
Xquery native windowing instructions despite a slightly complex syntax are extremely powerfull and you can rely on the optimizer being able to optimize where ever possible. M.
Quoted text · Omar Siam · 36 lines
On 25/09/20 15:55, Omar Siam wrote: > Perhaps you want to use fn:subsequence($hits, $start, $num)? > But of course if you have millions of hits that is not efficient enough. > > Am 25.09.2020 um 13:23 schrieb Günter Dunz-Wolff: >> Hi all, >> is it possible to present the results of an XQuery search in parts of >> 50? In order to speed up the search, only the first 50 results should >> be displayed and only if the user is interested in further results, >> 50 new results each should be displayed. Is this even possible? I did >> not find anything about it in the documentation. >> >> Example: >> let $collection := collection("data") >> let $hits := $collection//*:s[.//text() contains text {$query_string}] >> let $count_hits := count($hits) >> return >> <div> >> { >> if $count_hits <= 50 then >> <ol> >> {for $hit in $hits >> return >> <li>{$hit}</li> >> </ol> >> else >> ??? >> } >> </div> >> >> Thanks for any advice. >> Best regards >> Guenter >> >> >
Hi Günter,
Here’s one way to do it:
declare variable $QUERY external := 'mailto';
declare variable $START external := 1;
declare variable $MAX external := 50;
let $hits := db:open('data')//text()
[. contains text { $QUERY }]
let $count := count($hits)
return <div>{
<p>{ concat($count, ' results, showing ', $START, ' – ',
min(($count, $START + $MAX - 1)), ':') }</p>,
<ol>{
for $hit in subsequence($hits, $START, $MAX)
return <li>{ $hit }</li>
}</ol>
}</div>As Omar has already indicated, you can e.g. use fn:subsequence to limit the size of a result sequence. I have defined some external variables, which you can assign before running your query. If you use RESTXQ, you can supply them with query parameters.
Hope this helps, Christian
Quoted text · Günter Dunz-Wolff · 29 lines
On Fri, Sep 25, 2020 at 1:23 PM Günter Dunz-Wolff <guenter.dunzwolff at gmail.com> wrote: > > Hi all, > is it possible to present the results of an XQuery search in parts of 50? In order to speed up the search, only the first 50 results should be displayed and only if the user is interested in further results, 50 new results each should be displayed. Is this even possible? I did not find anything about it in the documentation. > > Example: > let $collection := collection("data") > let $hits := $collection//*:s[.//text() contains text {$query_string}] > let $count_hits := count($hits) > return > <div> > { > if $count_hits <= 50 then > <ol> > {for $hit in $hits > return > <li>{$hit}</li> > </ol> > else > ??? > } > </div> > > Thanks for any advice. > Best regards > Guenter > >
Hi Christian, hi all
thank you for your advice. I took the approach from Christian and it’s working. I’m using it with query-params (form-params) with RESTXQ. But it only works for the very first fifty hits. I have no idea, how to implement the functionality to serve the next 50 hits and so on. Is it possible to run the same query with new $start and $max params without using the searchform again? Any further help appreciated. Thanks in advance.
Best regards, Günter
Quoted text · Christian Grün · 60 lines
> Am 25.09.2020 um 20:13 schrieb Christian Grün <christian.gruen at gmail.com>: > > Hi Günter, > > Here’s one way to do it: > > declare variable $QUERY external := 'mailto'; > declare variable $START external := 1; > declare variable $MAX external := 50; > > let $hits := db:open('data')//text() > [. contains text { $QUERY }] > let $count := count($hits) > return <div>{ > <p>{ concat($count, ' results, showing ', $START, ' – ', > min(($count, $START + $MAX - 1)), ':') }</p>, > <ol>{ > for $hit in subsequence($hits, $START, $MAX) > return <li>{ $hit }</li> > }</ol> > }</div> > > As Omar has already indicated, you can e.g. use fn:subsequence to > limit the size of a result sequence. I have defined some external > variables, which you can assign before running your query. If you use > RESTXQ, you can supply them with query parameters. > > Hope this helps, > Christian > > > On Fri, Sep 25, 2020 at 1:23 PM Günter Dunz-Wolff > <guenter.dunzwolff at gmail.com> wrote: >> >> Hi all, >> is it possible to present the results of an XQuery search in parts of 50? In order to speed up the search, only the first 50 results should be displayed and only if the user is interested in further results, 50 new results each should be displayed. Is this even possible? I did not find anything about it in the documentation. >> >> Example: >> let $collection := collection("data") >> let $hits := $collection//*:s[.//text() contains text {$query_string}] >> let $count_hits := count($hits) >> return >> <div> >> { >> if $count_hits <= 50 then >> <ol> >> {for $hit in $hits >> return >> <li>{$hit}</li> >> </ol> >> else >> ??? >> } >> </div> >> >> Thanks for any advice. >> Best regards >> Guenter >> >>
If you provide the search RESTXQ endpoint also (or only) for GET requests, you can easily create links for results 51–100, etc.
Invoking search and passing all parameters per GET is better anyway, since people can bookmark the search query.
I was hesitant initially to run the same query over and over again, only to pick a different slice of the results. But if there is a full text index and if it is used, this has been ultra fast for my use cases so far.
Gerrit
Quoted text · Günter Dunz-Wolff · 69 lines
On 26.09.2020 16:55, Günter Dunz-Wolff wrote: > Hi Christian, hi all > > thank you for your advice. I took the approach from Christian and it’s working. I’m using it with query-params (form-params) with RESTXQ. But it only works for the very first fifty hits. I have no idea, how to implement the functionality to serve the next 50 hits and so on. Is it possible to run the same query with new $start and $max params without using the searchform again? Any further help appreciated. Thanks in advance. > > Best regards, > Günter > >> Am 25.09.2020 um 20:13 schrieb Christian Grün <christian.gruen at gmail.com>: >> >> Hi Günter, >> >> Here’s one way to do it: >> >> declare variable $QUERY external := 'mailto'; >> declare variable $START external := 1; >> declare variable $MAX external := 50; >> >> let $hits := db:open('data')//text() >> [. contains text { $QUERY }] >> let $count := count($hits) >> return <div>{ >> <p>{ concat($count, ' results, showing ', $START, ' – ', >> min(($count, $START + $MAX - 1)), ':') }</p>, >> <ol>{ >> for $hit in subsequence($hits, $START, $MAX) >> return <li>{ $hit }</li> >> }</ol> >> }</div> >> >> As Omar has already indicated, you can e.g. use fn:subsequence to >> limit the size of a result sequence. I have defined some external >> variables, which you can assign before running your query. If you use >> RESTXQ, you can supply them with query parameters. >> >> Hope this helps, >> Christian >> >> >> On Fri, Sep 25, 2020 at 1:23 PM Günter Dunz-Wolff >> <guenter.dunzwolff at gmail.com> wrote: >>> >>> Hi all, >>> is it possible to present the results of an XQuery search in parts of 50? In order to speed up the search, only the first 50 results should be displayed and only if the user is interested in further results, 50 new results each should be displayed. Is this even possible? I did not find anything about it in the documentation. >>> >>> Example: >>> let $collection := collection("data") >>> let $hits := $collection//*:s[.//text() contains text {$query_string}] >>> let $count_hits := count($hits) >>> return >>> <div> >>> { >>> if $count_hits <= 50 then >>> <ol> >>> {for $hit in $hits >>> return >>> <li>{$hit}</li> >>> </ol> >>> else >>> ??? >>> } >>> </div> >>> >>> Thanks for any advice. >>> Best regards >>> Guenter >>> >>> >
Thanks Gerrit, invoking search and passing all params per GET is the way to go.
Quoted text · Günter Dunz-Wolff · 3 lines
On Sun, 2020-09-27 at 14:43 +0200, Günter Dunz-Wolff wrote: > Thanks Gerrit, invoking search and passing all params per GET is the > way to go.
For a public-facing example, see the search box in e.g. https://words.fromoldbooks.org/Chalmers-Biography/n/newton-sir-isaac.html
works this way, doing a fresh search for each page. But there's only some six million words (almost all English) in the book.
It's very much a work in progress, but it got useful enough to make public. I'll add jump links (page: 1 2 3...) and more features later.
On the biography pages the results of the "similar images" query at the foot of the page are cached, although that's mostly because it was easier to reuse the framework i already had for the rest of the site than not to cache them. The site is not using RESTXQ.
Liam
Written for BaseX 9.4. BaseX may have changed since: please check the current documentation.