[XQUERY] Filter and paging with window clause

On 28.08.2018 11:04, Marco Lettere wrote: > here's a question related to XQuery, sorry for being slightly off-topic. > > I'm struggling to find a way to combine the windowing clause and FLOWR > in order to get a paged result of a subset of items which respect a > given filter. > > Of course I'm able to get this by first applying the filter to the whole > input and then a second FLOWR for the windowing clause.

> The closer I get is [1] which is not what I'd need because I get > 2,4,6,8,10 as result for page 1 but I'd really want 10 results per page > thus 2,4,6,8,10,12,14,16,18,20. > > Thanks for any help on this in the meanwhile I'll stick to my naive > solution. > > Marco. > > [1] > > let $page := 1 > let $count := 10 > return > for tumbling window $w in (1 to 100) > start $first at $s when ($s = 1 + ($page - 1) * $count) > end $last at $e when ($e - $s = $count - 1) > return > $w ! (if (. mod 2 = 0 ) then . else ())

On 28/08/2018 14:19, Martin Honnen wrote: > On 28.08.2018 11:04, Marco Lettere wrote: > >> here's a question related to XQuery, sorry for being slightly off-topic. >> >> I'm struggling to find a way to combine the windowing clause and >> FLOWR in order to get a paged result of a subset of items which >> respect a given filter. >> >> Of course I'm able to get this by first applying the filter to the >> whole input and then a second FLOWR for the windowing clause. > > So what is the filter condition? What is the input sequence? > >> The closer I get is [1] which is not what I'd need because I get >> 2,4,6,8,10 as result for page 1 but I'd really want 10 results per >> page thus 2,4,6,8,10,12,14,16,18,20. >> >> Thanks for any help on this in the meanwhile I'll stick to my naive >> solution. >> >> Marco. >> >> [1] >> >> let $page := 1 >> let $count := 10 >> return >> for tumbling window $w in (1 to 100) >> start $first at $s when ($s = 1 + ($page - 1) * $count) >> end $last at $e when ($e - $s = $count - 1) >> return >> $w ! (if (. mod 2 = 0 ) then . else ()) > > > Can't you simply use the filter as a predicate on the expression you > use in the windowing for clause? > > Does > > let $page-size := 10 > for tumbling window $page in (1 to 100)[. mod 2 = 0] > start at $sp when $sp mod $page-size = 1 > end at $np when ($np + 1) mod $page-size = 1 > return <page>{$page}</page> > > help? > > It returns > > <page>2 4 6 8 10 12 14 16 18 20</page> > <page>22 24 26 28 30 32 34 36 38 40</page> > <page>42 44 46 48 50 52 54 56 58 60</page> > <page>62 64 66 68 70 72 74 76 78 80</page> > <page>82 84 86 88 90 92 94 96 98 100</page> > > >

On Tue, Aug 28, 2018 at 3:27 PM Marco Lettere <m.lettere at gmail.com> wrote: > > Oh, yes. Thank you Martin. > That is obviously a good point. > The input is a database with several thousands of documents stored in it. > The filters are a dynamic sequence of xpath expressions to be applied on the documents expressed as function references. > I need to check whether I can compress all this into a simple predicate as you've shown in your example. > where clause would have been much more easier in conjunction with the map operator (!) but if it works I'm going to coope with the predicate. > Thanks for your suggestion in the meanwhile. > > Here a simplified example of my current approach where you could think of $filters as a sequence of boolean functions to be applied to the documents in the db. > > let $seq := > for $doc in db:open("adatabase") > where not(false() = ($filters ! (.($doc))) > return $doc > let $out := > for tumbling window $w in $seq > start $first at $s when ($s = 1 + ($page - 1) * $count) > end $last at $e when $e - $s = $count - 1 > return $w > > Regards, > Marco. > > On 28/08/2018 14:19, Martin Honnen wrote: > > On 28.08.2018 11:04, Marco Lettere wrote: > > here's a question related to XQuery, sorry for being slightly off-topic. > > I'm struggling to find a way to combine the windowing clause and FLOWR in order to get a paged result of a subset of items which respect a given filter. > > Of course I'm able to get this by first applying the filter to the whole input and then a second FLOWR for the windowing clause. > > > So what is the filter condition? What is the input sequence? > > The closer I get is [1] which is not what I'd need because I get 2,4,6,8,10 as result for page 1 but I'd really want 10 results per page thus 2,4,6,8,10,12,14,16,18,20. > > Thanks for any help on this in the meanwhile I'll stick to my naive solution. > > Marco. > > [1] > > let $page := 1 > let $count := 10 > return > for tumbling window $w in (1 to 100) > start $first at $s when ($s = 1 + ($page - 1) * $count) > end $last at $e when ($e - $s = $count - 1) > return > $w ! (if (. mod 2 = 0 ) then . else ()) > > > > Can't you simply use the filter as a predicate on the expression you use in the windowing for clause? > > Does > > let $page-size := 10 > for tumbling window $page in (1 to 100)[. mod 2 = 0] > start at $sp when $sp mod $page-size = 1 > end at $np when ($np + 1) mod $page-size = 1 > return <page>{$page}</page> > > help? > > It returns > > <page>2 4 6 8 10 12 14 16 18 20</page> > <page>22 24 26 28 30 32 34 36 38 40</page> > <page>42 44 46 48 50 52 54 56 58 60</page> > <page>62 64 66 68 70 72 74 76 78 80</page> > <page>82 84 86 88 90 92 94 96 98 100</page> > > > >

On 28/08/2018 15:41, Christian Grün wrote: > Hi Marco, > > Here is a minimized representation of your last query: > > for tumbling window $w in > db:open("adatabase")[every $f in $filters satisfies $f(.)] > start $first at $s when ($s = 1 + ($page - 1) * $count) > end $last at $e when $e - $s = $count - 1 > return $w > > Cheers, > Christian > > > On Tue, Aug 28, 2018 at 3:27 PM Marco Lettere <m.lettere at gmail.com> wrote: >> Oh, yes. Thank you Martin. >> That is obviously a good point. >> The input is a database with several thousands of documents stored in it. >> The filters are a dynamic sequence of xpath expressions to be applied on the documents expressed as function references. >> I need to check whether I can compress all this into a simple predicate as you've shown in your example. >> where clause would have been much more easier in conjunction with the map operator (!) but if it works I'm going to coope with the predicate. >> Thanks for your suggestion in the meanwhile. >> >> Here a simplified example of my current approach where you could think of $filters as a sequence of boolean functions to be applied to the documents in the db. >> >> let $seq := >> for $doc in db:open("adatabase") >> where not(false() = ($filters ! (.($doc))) >> return $doc >> let $out := >> for tumbling window $w in $seq >> start $first at $s when ($s = 1 + ($page - 1) * $count) >> end $last at $e when $e - $s = $count - 1 >> return $w >> >> Regards, >> Marco. >> >> On 28/08/2018 14:19, Martin Honnen wrote: >> >> On 28.08.2018 11:04, Marco Lettere wrote: >> >> here's a question related to XQuery, sorry for being slightly off-topic. >> >> I'm struggling to find a way to combine the windowing clause and FLOWR in order to get a paged result of a subset of items which respect a given filter. >> >> Of course I'm able to get this by first applying the filter to the whole input and then a second FLOWR for the windowing clause. >> >> >> So what is the filter condition? What is the input sequence? >> >> The closer I get is [1] which is not what I'd need because I get 2,4,6,8,10 as result for page 1 but I'd really want 10 results per page thus 2,4,6,8,10,12,14,16,18,20. >> >> Thanks for any help on this in the meanwhile I'll stick to my naive solution. >> >> Marco. >> >> [1] >> >> let $page := 1 >> let $count := 10 >> return >> for tumbling window $w in (1 to 100) >> start $first at $s when ($s = 1 + ($page - 1) * $count) >> end $last at $e when ($e - $s = $count - 1) >> return >> $w ! (if (. mod 2 = 0 ) then . else ()) >> >> >> >> Can't you simply use the filter as a predicate on the expression you use in the windowing for clause? >> >> Does >> >> let $page-size := 10 >> for tumbling window $page in (1 to 100)[. mod 2 = 0] >> start at $sp when $sp mod $page-size = 1 >> end at $np when ($np + 1) mod $page-size = 1 >> return <page>{$page}</page> >> >> help? >> >> It returns >> >> <page>2 4 6 8 10 12 14 16 18 20</page> >> <page>22 24 26 28 30 32 34 36 38 40</page> >> <page>42 44 46 48 50 52 54 56 58 60</page> >> <page>62 64 66 68 70 72 74 76 78 80</page> >> <page>82 84 86 88 90 92 94 96 98 100</page> >> >> >> >>

On 28/08/2018 15:41, Christian Grün wrote: > Hi Marco, > > Here is a minimized representation of your last query: > > for tumbling window $w in > db:open("adatabase")[every $f in $filters satisfies $f(.)] > start $first at $s when ($s = 1 + ($page - 1) * $count) > end $last at $e when $e - $s = $count - 1 > return $w > > Cheers, > Christian > > > On Tue, Aug 28, 2018 at 3:27 PM Marco Lettere <m.lettere at gmail.com> wrote: >> Oh, yes. Thank you Martin. >> That is obviously a good point. >> The input is a database with several thousands of documents stored in it. >> The filters are a dynamic sequence of xpath expressions to be applied on the documents expressed as function references. >> I need to check whether I can compress all this into a simple predicate as you've shown in your example. >> where clause would have been much more easier in conjunction with the map operator (!) but if it works I'm going to coope with the predicate. >> Thanks for your suggestion in the meanwhile. >> >> Here a simplified example of my current approach where you could think of $filters as a sequence of boolean functions to be applied to the documents in the db. >> >> let $seq := >> for $doc in db:open("adatabase") >> where not(false() = ($filters ! (.($doc))) >> return $doc >> let $out := >> for tumbling window $w in $seq >> start $first at $s when ($s = 1 + ($page - 1) * $count) >> end $last at $e when $e - $s = $count - 1 >> return $w >> >> Regards, >> Marco. >> >> On 28/08/2018 14:19, Martin Honnen wrote: >> >> On 28.08.2018 11:04, Marco Lettere wrote: >> >> here's a question related to XQuery, sorry for being slightly off-topic. >> >> I'm struggling to find a way to combine the windowing clause and FLOWR in order to get a paged result of a subset of items which respect a given filter. >> >> Of course I'm able to get this by first applying the filter to the whole input and then a second FLOWR for the windowing clause. >> >> >> So what is the filter condition? What is the input sequence? >> >> The closer I get is [1] which is not what I'd need because I get 2,4,6,8,10 as result for page 1 but I'd really want 10 results per page thus 2,4,6,8,10,12,14,16,18,20. >> >> Thanks for any help on this in the meanwhile I'll stick to my naive solution. >> >> Marco. >> >> [1] >> >> let $page := 1 >> let $count := 10 >> return >> for tumbling window $w in (1 to 100) >> start $first at $s when ($s = 1 + ($page - 1) * $count) >> end $last at $e when ($e - $s = $count - 1) >> return >> $w ! (if (. mod 2 = 0 ) then . else ()) >> >> >> >> Can't you simply use the filter as a predicate on the expression you use in the windowing for clause? >> >> Does >> >> let $page-size := 10 >> for tumbling window $page in (1 to 100)[. mod 2 = 0] >> start at $sp when $sp mod $page-size = 1 >> end at $np when ($np + 1) mod $page-size = 1 >> return <page>{$page}</page> >> >> help? >> >> It returns >> >> <page>2 4 6 8 10 12 14 16 18 20</page> >> <page>22 24 26 28 30 32 34 36 38 40</page> >> <page>42 44 46 48 50 52 54 56 58 60</page> >> <page>62 64 66 68 70 72 74 76 78 80</page> >> <page>82 84 86 88 90 92 94 96 98 100</page> >> >> >> >>

On Tue, Aug 28, 2018 at 4:01 PM Marco Lettere <m.lettere at gmail.com> wrote: > > .... and what if, a pervert I know, would like to have the docs also > globally ordered by any sort criteria? > Any chance of squeezing the order by into this elegant FLOWR? > Thanks again. > M. > > On 28/08/2018 15:41, Christian Grün wrote: > > Hi Marco, > > > > Here is a minimized representation of your last query: > > > > for tumbling window $w in > > db:open("adatabase")[every $f in $filters satisfies $f(.)] > > start $first at $s when ($s = 1 + ($page - 1) * $count) > > end $last at $e when $e - $s = $count - 1 > > return $w > > > > Cheers, > > Christian > > > > > > On Tue, Aug 28, 2018 at 3:27 PM Marco Lettere <m.lettere at gmail.com> wrote: > >> Oh, yes. Thank you Martin. > >> That is obviously a good point. > >> The input is a database with several thousands of documents stored in it. > >> The filters are a dynamic sequence of xpath expressions to be applied on the documents expressed as function references. > >> I need to check whether I can compress all this into a simple predicate as you've shown in your example. > >> where clause would have been much more easier in conjunction with the map operator (!) but if it works I'm going to coope with the predicate. > >> Thanks for your suggestion in the meanwhile. > >> > >> Here a simplified example of my current approach where you could think of $filters as a sequence of boolean functions to be applied to the documents in the db. > >> > >> let $seq := > >> for $doc in db:open("adatabase") > >> where not(false() = ($filters ! (.($doc))) > >> return $doc > >> let $out := > >> for tumbling window $w in $seq > >> start $first at $s when ($s = 1 + ($page - 1) * $count) > >> end $last at $e when $e - $s = $count - 1 > >> return $w > >> > >> Regards, > >> Marco. > >> > >> On 28/08/2018 14:19, Martin Honnen wrote: > >> > >> On 28.08.2018 11:04, Marco Lettere wrote: > >> > >> here's a question related to XQuery, sorry for being slightly off-topic. > >> > >> I'm struggling to find a way to combine the windowing clause and FLOWR in order to get a paged result of a subset of items which respect a given filter. > >> > >> Of course I'm able to get this by first applying the filter to the whole input and then a second FLOWR for the windowing clause. > >> > >> > >> So what is the filter condition? What is the input sequence? > >> > >> The closer I get is [1] which is not what I'd need because I get 2,4,6,8,10 as result for page 1 but I'd really want 10 results per page thus 2,4,6,8,10,12,14,16,18,20. > >> > >> Thanks for any help on this in the meanwhile I'll stick to my naive solution. > >> > >> Marco. > >> > >> [1] > >> > >> let $page := 1 > >> let $count := 10 > >> return > >> for tumbling window $w in (1 to 100) > >> start $first at $s when ($s = 1 + ($page - 1) * $count) > >> end $last at $e when ($e - $s = $count - 1) > >> return > >> $w ! (if (. mod 2 = 0 ) then . else ()) > >> > >> > >> > >> Can't you simply use the filter as a predicate on the expression you use in the windowing for clause? > >> > >> Does > >> > >> let $page-size := 10 > >> for tumbling window $page in (1 to 100)[. mod 2 = 0] > >> start at $sp when $sp mod $page-size = 1 > >> end at $np when ($np + 1) mod $page-size = 1 > >> return <page>{$page}</page> > >> > >> help? > >> > >> It returns > >> > >> <page>2 4 6 8 10 12 14 16 18 20</page> > >> <page>22 24 26 28 30 32 34 36 38 40</page> > >> <page>42 44 46 48 50 52 54 56 58 60</page> > >> <page>62 64 66 68 70 72 74 76 78 80</page> > >> <page>82 84 86 88 90 92 94 96 98 100</page> > >> > >> > >> > >> >

On 28/08/2018 16:05, Christian Grün wrote: > You can use the fn:sort function for that: > > for tumbling window $w in sort( > db:open("adatabase")[every $f in $filters satisfies $f(.)], > (), > function($doc) { db:path($doc) } > ) > ... > > Sorting is a blocking operator, though: All documents will need to be > checked first in order to tell which one will be the smallest or > largest hit. > > Best, > Christian > > > > On Tue, Aug 28, 2018 at 4:01 PM Marco Lettere <m.lettere at gmail.com> wrote: >> .... and what if, a pervert I know, would like to have the docs also >> globally ordered by any sort criteria? >> Any chance of squeezing the order by into this elegant FLOWR? >> Thanks again. >> M. >> >> On 28/08/2018 15:41, Christian Grün wrote: >>> Hi Marco, >>> >>> Here is a minimized representation of your last query: >>> >>> for tumbling window $w in >>> db:open("adatabase")[every $f in $filters satisfies $f(.)] >>> start $first at $s when ($s = 1 + ($page - 1) * $count) >>> end $last at $e when $e - $s = $count - 1 >>> return $w >>> >>> Cheers, >>> Christian >>> >>> >>> On Tue, Aug 28, 2018 at 3:27 PM Marco Lettere <m.lettere at gmail.com> wrote: >>>> Oh, yes. Thank you Martin. >>>> That is obviously a good point. >>>> The input is a database with several thousands of documents stored in it. >>>> The filters are a dynamic sequence of xpath expressions to be applied on the documents expressed as function references. >>>> I need to check whether I can compress all this into a simple predicate as you've shown in your example. >>>> where clause would have been much more easier in conjunction with the map operator (!) but if it works I'm going to coope with the predicate. >>>> Thanks for your suggestion in the meanwhile. >>>> >>>> Here a simplified example of my current approach where you could think of $filters as a sequence of boolean functions to be applied to the documents in the db. >>>> >>>> let $seq := >>>> for $doc in db:open("adatabase") >>>> where not(false() = ($filters ! (.($doc))) >>>> return $doc >>>> let $out := >>>> for tumbling window $w in $seq >>>> start $first at $s when ($s = 1 + ($page - 1) * $count) >>>> end $last at $e when $e - $s = $count - 1 >>>> return $w >>>> >>>> Regards, >>>> Marco. >>>> >>>> On 28/08/2018 14:19, Martin Honnen wrote: >>>> >>>> On 28.08.2018 11:04, Marco Lettere wrote: >>>> >>>> here's a question related to XQuery, sorry for being slightly off-topic. >>>> >>>> I'm struggling to find a way to combine the windowing clause and FLOWR in order to get a paged result of a subset of items which respect a given filter. >>>> >>>> Of course I'm able to get this by first applying the filter to the whole input and then a second FLOWR for the windowing clause. >>>> >>>> >>>> So what is the filter condition? What is the input sequence? >>>> >>>> The closer I get is [1] which is not what I'd need because I get 2,4,6,8,10 as result for page 1 but I'd really want 10 results per page thus 2,4,6,8,10,12,14,16,18,20. >>>> >>>> Thanks for any help on this in the meanwhile I'll stick to my naive solution. >>>> >>>> Marco. >>>> >>>> [1] >>>> >>>> let $page := 1 >>>> let $count := 10 >>>> return >>>> for tumbling window $w in (1 to 100) >>>> start $first at $s when ($s = 1 + ($page - 1) * $count) >>>> end $last at $e when ($e - $s = $count - 1) >>>> return >>>> $w ! (if (. mod 2 = 0 ) then . else ()) >>>> >>>> >>>> >>>> Can't you simply use the filter as a predicate on the expression you use in the windowing for clause? >>>> >>>> Does >>>> >>>> let $page-size := 10 >>>> for tumbling window $page in (1 to 100)[. mod 2 = 0] >>>> start at $sp when $sp mod $page-size = 1 >>>> end at $np when ($np + 1) mod $page-size = 1 >>>> return <page>{$page}</page> >>>> >>>> help? >>>> >>>> It returns >>>> >>>> <page>2 4 6 8 10 12 14 16 18 20</page> >>>> <page>22 24 26 28 30 32 34 36 38 40</page> >>>> <page>42 44 46 48 50 52 54 56 58 60</page> >>>> <page>62 64 66 68 70 72 74 76 78 80</page> >>>> <page>82 84 86 88 90 92 94 96 98 100</page> >>>> >>>> >>>> >>>>