tf/idf

Am 30.03.2011 um 22:02 schrieb Wiard Vasen: > Dear sirs of Basex, > > I am doing my Master thesis on the letters of Vincent van Gogh at the University of Amsterdam. > For that purpose I use BaseX to analyze the letters. > > I wonder, is there the possibility to generate a tf/idf score automatically? > In your faq I noticed there needs to be a special term like 'SET SCORING 0' to be able to get a tf/idf score. > > This information I get from the following page: http://docs.basex.org/wiki/Full-Text > > Could you help me with this? > > I would be very grateful. > > Kind regards, > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

On Thu, Mar 31, 2011 at 8:33 AM, Andreas Weiler <andreas.weiler at uni-konstanz.de> wrote: > Dear Wiard Vasen, > you just need to set the scoring property once. > If you work in the GUI: > Go to the top input bar, choose command and type: > set scoring * > > as * set the scoring algorithm you like. > In the console just type: set scoring * > After setting this you can use the score function, like in the 8th query of > our online demo (basex.org/products/live-demo): > let $names := ('Jack London', 'Jack', 'Jim Beam', 'Jack Daniels') > for $name at $pos score $score in $names[. contains text 'Jack'] > order by $score descending > return <name pos='{ $pos }'>{ $name }</name> > Don't hesitate to ask for more, > Andreas > Am 30.03.2011 um 22:02 schrieb Wiard Vasen: > > Dear sirs of Basex, > I am doing my Master thesis on the letters of Vincent van Gogh at the > University of Amsterdam. > For that purpose I use BaseX to analyze the letters. > I wonder, is there the possibility to generate a tf/idf score automatically? > In your faq I noticed there needs to be a special term like 'SET SCORING 0' > to be able to get a tf/idf score. > This information I get from the following > page: http://docs.basex.org/wiki/Full-Text > Could you help me with this? > I would be very grateful. > Kind regards, > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >

Am 02.04.2011 um 18:01 schrieb Wiard Vasen: > Thanks a lot Andreas for your answer on my question. > I will look at your solution this week. > > I was in Berlin this weekend. > > Is it possible to use set scoring * and then enter full text queries in the upper text-bar? > > Thank you for the live-demo as well. > > If I succeed I come back to you. > > Regards, > > Wiard Vasen > > Vrije Universiteit van Amsterdam > Masterthesis: Deep diving into Van Gogh > > 2011/3/31 Andreas Weiler <andreas.weiler at uni-konstanz.de> > Dear Wiard Vasen, > > you just need to set the scoring property once. > If you work in the GUI: > > Go to the top input bar, choose command and type: > > set scoring * > > as * set the scoring algorithm you like. > > In the console just type: set scoring * > > After setting this you can use the score function, like in the 8th query of our online demo (basex.org/products/live-demo): > > let $names := ('Jack London', 'Jack', 'Jim Beam', 'Jack Daniels') > for $name at $pos score $score in $names[. contains text 'Jack'] > order by $score descending > return <name pos='{ $pos }'>{ $name }</name> > > Don't hesitate to ask for more, > Andreas > > Am 30.03.2011 um 22:02 schrieb Wiard Vasen: > >> Dear sirs of Basex, >> >> I am doing my Master thesis on the letters of Vincent van Gogh at the University of Amsterdam. >> For that purpose I use BaseX to analyze the letters. >> >> I wonder, is there the possibility to generate a tf/idf score automatically? >> In your faq I noticed there needs to be a special term like 'SET SCORING 0' to be able to get a tf/idf score. >> >> This information I get from the following page: http://docs.basex.org/wiki/Full-Text >> >> Could you help me with this? >> >> I would be very grateful. >> >> Kind regards, >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >

> Hi Wiard, > > the tf/idf scoring is only available if you are working with full-text > index structures. If you have built a full-text index for your > database "DB", the following query will yield different scoring > results, depending on the chosen scoring model: > > ft:score(db:open("DB")//*[text() contains text 'xml']) > > As Andreas indicated, however, you may either set the SCORING property > with a db command or explicitly choose the type of scoring in the > GUI's database creation dialog (Database -> New -> Fulltext -> TF/IDF > Scoring). > > Christian > > > > On Thu, Mar 31, 2011 at 8:33 AM, Andreas Weiler > <andreas.weiler at uni-konstanz.de> wrote: > > Dear Wiard Vasen, > > you just need to set the scoring property once. > > If you work in the GUI: > > Go to the top input bar, choose command and type: > > set scoring * > > > > as * set the scoring algorithm you like. > > In the console just type: set scoring * > > After setting this you can use the score function, like in the 8th query > of > > our online demo (basex.org/products/live-demo): > > let $names := ('Jack London', 'Jack', 'Jim Beam', 'Jack Daniels') > > for $name at $pos score $score in $names[. contains text 'Jack'] > > order by $score descending > > return <name pos='{ $pos }'>{ $name }</name> > > Don't hesitate to ask for more, > > Andreas > > Am 30.03.2011 um 22:02 schrieb Wiard Vasen: > > > > Dear sirs of Basex, > > I am doing my Master thesis on the letters of Vincent van Gogh at the > > University of Amsterdam. > > For that purpose I use BaseX to analyze the letters. > > I wonder, is there the possibility to generate a tf/idf score > automatically? > > In your faq I noticed there needs to be a special term like 'SET SCORING > 0' > > to be able to get a tf/idf score. > > This information I get from the following > > page: http://docs.basex.org/wiki/Full-Text > > Could you help me with this? > > I would be very grateful. > > Kind regards, > > _______________________________________________ > > BaseX-Talk mailing list > > BaseX-Talk at mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > > > > _______________________________________________ > > BaseX-Talk mailing list > > BaseX-Talk at mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > > >

Am 03.04.2011 um 12:42 schrieb Wiard Vasen: > Dear Christian and Andreas, > > Thanks for your great help! > I used Christians solution: ft:score(db:open("DB")//*[text() contains text 'xml']) > And it works fine. > > The next step is that I want to get the associated documents with these scores. > > Could you help me with this step? > > The results I get now is a list with frequencies, without the references to the particular documents. > I think what is needed is the tf/idf score. > > Regards, > > Wiard > > > 2011/3/31 Christian Grün <christian.gruen at gmail.com> > Hi Wiard, > > the tf/idf scoring is only available if you are working with full-text > index structures. If you have built a full-text index for your > database "DB", the following query will yield different scoring > results, depending on the chosen scoring model: > > ft:score(db:open("DB")//*[text() contains text 'xml']) > > As Andreas indicated, however, you may either set the SCORING property > with a db command or explicitly choose the type of scoring in the > GUI's database creation dialog (Database -> New -> Fulltext -> TF/IDF > Scoring). > > Christian > > > > On Thu, Mar 31, 2011 at 8:33 AM, Andreas Weiler > <andreas.weiler at uni-konstanz.de> wrote: > > Dear Wiard Vasen, > > you just need to set the scoring property once. > > If you work in the GUI: > > Go to the top input bar, choose command and type: > > set scoring * > > > > as * set the scoring algorithm you like. > > In the console just type: set scoring * > > After setting this you can use the score function, like in the 8th query of > > our online demo (basex.org/products/live-demo): > > let $names := ('Jack London', 'Jack', 'Jim Beam', 'Jack Daniels') > > for $name at $pos score $score in $names[. contains text 'Jack'] > > order by $score descending > > return <name pos='{ $pos }'>{ $name }</name> > > Don't hesitate to ask for more, > > Andreas > > Am 30.03.2011 um 22:02 schrieb Wiard Vasen: > > > > Dear sirs of Basex, > > I am doing my Master thesis on the letters of Vincent van Gogh at the > > University of Amsterdam. > > For that purpose I use BaseX to analyze the letters. > > I wonder, is there the possibility to generate a tf/idf score automatically? > > In your faq I noticed there needs to be a special term like 'SET SCORING 0' > > to be able to get a tf/idf score. > > This information I get from the following > > page: http://docs.basex.org/wiki/Full-Text > > Could you help me with this? > > I would be very grateful. > > Kind regards, > > _______________________________________________ > > BaseX-Talk mailing list > > BaseX-Talk at mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > > > > _______________________________________________ > > BaseX-Talk mailing list > > BaseX-Talk at mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > > >

> Hi Wiard, > > you could use the base-uri function of XQuery, like (probably can be done > easier): > > for $i at $pos in db:open("DB")//* > where $i[text() contains text 'xml'] > return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text > 'xml'])}</score></hit> > > -- Andreas > > Am 03.04.2011 um 12:42 schrieb Wiard Vasen: > > Dear Christian and Andreas, > > Thanks for your great help! > I used Christians solution: ft:score(db:open("DB")//*[text() contains text > 'xml']) > And it works fine. > > The next step is that I want to get the associated documents with these > scores. > > Could you help me with this step? > > The results I get now is a list with frequencies, without the references to > the particular documents. > I think what is needed is the tf/idf score. > > Regards, > > Wiard > > > 2011/3/31 Christian Grün <christian.gruen at gmail.com> > >> Hi Wiard, >> >> the tf/idf scoring is only available if you are working with full-text >> index structures. If you have built a full-text index for your >> database "DB", the following query will yield different scoring >> results, depending on the chosen scoring model: >> >> ft:score(db:open("DB")//*[text() contains text 'xml']) >> >> As Andreas indicated, however, you may either set the SCORING property >> with a db command or explicitly choose the type of scoring in the >> GUI's database creation dialog (Database -> New -> Fulltext -> TF/IDF >> Scoring). >> >> Christian >> >> >> >> On Thu, Mar 31, 2011 at 8:33 AM, Andreas Weiler >> <andreas.weiler at uni-konstanz.de> wrote: >> > Dear Wiard Vasen, >> > you just need to set the scoring property once. >> > If you work in the GUI: >> > Go to the top input bar, choose command and type: >> > set scoring * >> > >> > as * set the scoring algorithm you like. >> > In the console just type: set scoring * >> > After setting this you can use the score function, like in the 8th query >> of >> > our online demo (basex.org/products/live-demo): >> > let $names := ('Jack London', 'Jack', 'Jim Beam', 'Jack Daniels') >> > for $name at $pos score $score in $names[. contains text 'Jack'] >> > order by $score descending >> > return <name pos='{ $pos }'>{ $name }</name> >> > Don't hesitate to ask for more, >> > Andreas >> > Am 30.03.2011 um 22:02 schrieb Wiard Vasen: >> > >> > Dear sirs of Basex, >> > I am doing my Master thesis on the letters of Vincent van Gogh at the >> > University of Amsterdam. >> > For that purpose I use BaseX to analyze the letters. >> > I wonder, is there the possibility to generate a tf/idf score >> automatically? >> > In your faq I noticed there needs to be a special term like 'SET SCORING >> 0' >> > to be able to get a tf/idf score. >> > This information I get from the following >> > page: http://docs.basex.org/wiki/Full-Text >> > Could you help me with this? >> > I would be very grateful. >> > Kind regards, >> > _______________________________________________ >> > BaseX-Talk mailing list >> > BaseX-Talk at mailman.uni-konstanz.de >> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > >> > >> > _______________________________________________ >> > BaseX-Talk mailing list >> > BaseX-Talk at mailman.uni-konstanz.de >> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > >> > >> > > >

Am 03.04.2011 um 14:24 schrieb Wiard Vasen: > Hi Andreas, > > Wow! This is the complete answer to my question! > > I hope you can help me with the next question. > Because I am analyzing changes in the artistic life of Van Gogh, I am partitioning the relatively large repository annotated xml files on the basis of residence. > > For that reason I need to put a query like: > > for $i at $pos in db:open("tfidfbrievenvangogh")//* > where $i[text() contains text 'kleur'] > return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text 'kleur'])}</score></hit> > > with the extension: given the interval, all xml-files betweenlet567.xml and let689.xml. > What means that I know that in this partition xml-files Van Gogh was in Arles. > And I want to know what is the tf-idf score of the dutch word 'kleur'. > > To give a resume of my question: How do I partition the repository in subsets, so that I can produce information on these subsets. > And how do I do this in BasX with xquery. > > Thanks a lot beforehand! > > Kind regards, > > Wiard > > > 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> > Hi Wiard, > > you could use the base-uri function of XQuery, like (probably can be done easier): > > for $i at $pos in db:open("DB")//* > where $i[text() contains text 'xml'] > return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text 'xml'])}</score></hit> > > -- Andreas > > Am 03.04.2011 um 12:42 schrieb Wiard Vasen: > >> Dear Christian and Andreas, >> >> Thanks for your great help! >> I used Christians solution: ft:score(db:open("DB")//*[text() contains text 'xml']) >> And it works fine. >> >> The next step is that I want to get the associated documents with these scores. >> >> Could you help me with this step? >> >> The results I get now is a list with frequencies, without the references to the particular documents. >> I think what is needed is the tf/idf score. >> >> Regards, >> >> Wiard >> >> >> 2011/3/31 Christian Grün <christian.gruen at gmail.com> >> Hi Wiard, >> >> the tf/idf scoring is only available if you are working with full-text >> index structures. If you have built a full-text index for your >> database "DB", the following query will yield different scoring >> results, depending on the chosen scoring model: >> >> ft:score(db:open("DB")//*[text() contains text 'xml']) >> >> As Andreas indicated, however, you may either set the SCORING property >> with a db command or explicitly choose the type of scoring in the >> GUI's database creation dialog (Database -> New -> Fulltext -> TF/IDF >> Scoring). >> >> Christian >> >> >> >> On Thu, Mar 31, 2011 at 8:33 AM, Andreas Weiler >> <andreas.weiler at uni-konstanz.de> wrote: >> > Dear Wiard Vasen, >> > you just need to set the scoring property once. >> > If you work in the GUI: >> > Go to the top input bar, choose command and type: >> > set scoring * >> > >> > as * set the scoring algorithm you like. >> > In the console just type: set scoring * >> > After setting this you can use the score function, like in the 8th query of >> > our online demo (basex.org/products/live-demo): >> > let $names := ('Jack London', 'Jack', 'Jim Beam', 'Jack Daniels') >> > for $name at $pos score $score in $names[. contains text 'Jack'] >> > order by $score descending >> > return <name pos='{ $pos }'>{ $name }</name> >> > Don't hesitate to ask for more, >> > Andreas >> > Am 30.03.2011 um 22:02 schrieb Wiard Vasen: >> > >> > Dear sirs of Basex, >> > I am doing my Master thesis on the letters of Vincent van Gogh at the >> > University of Amsterdam. >> > For that purpose I use BaseX to analyze the letters. >> > I wonder, is there the possibility to generate a tf/idf score automatically? >> > In your faq I noticed there needs to be a special term like 'SET SCORING 0' >> > to be able to get a tf/idf score. >> > This information I get from the following >> > page: http://docs.basex.org/wiki/Full-Text >> > Could you help me with this? >> > I would be very grateful. >> > Kind regards, >> > _______________________________________________ >> > BaseX-Talk mailing list >> > BaseX-Talk at mailman.uni-konstanz.de >> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > >> > >> > _______________________________________________ >> > BaseX-Talk mailing list >> > BaseX-Talk at mailman.uni-konstanz.de >> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > >> > >> > >

> Hi Wiard, > > i hope i understand your plans, here is what i would do: > > for $n in ("betweenlet567.xml", "let689.xml") > return > for $i at $pos in db:open("tfidfbrievenvangogh")//* > where ends-with(base-uri($i), $n) > and $i[text() contains text 'kleur'] > return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text > 'kleur'])}</score></hit> > > Now you can extend the variable $n with all filenames you like to have. > > I hope this helps, > Andreas > > Am 03.04.2011 um 14:24 schrieb Wiard Vasen: > > Hi Andreas, > > Wow! This is the complete answer to my question! > > I hope you can help me with the next question. > Because I am analyzing changes in the artistic life of Van Gogh, I am > partitioning the relatively large repository annotated xml files on the > basis of residence. > > For that reason I need to put a query like: > > for $i at $pos in db:open("tfidfbrievenvangogh")//* > where $i[text() contains text 'kleur'] > return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text > 'kleur'])}</score></hit> > > with the extension: given the interval, all xml-files betweenlet567.xml > and let689.xml. > What means that I know that in this partition xml-files Van Gogh was in > Arles. > And I want to know what is the tf-idf score of the dutch word 'kleur'. > > To give a resume of my question: How do I partition the repository in > subsets, so that I can produce information on these subsets. > And how do I do this in BasX with xquery. > > Thanks a lot beforehand! > > Kind regards, > > Wiard > > > 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> > >> Hi Wiard, >> >> you could use the base-uri function of XQuery, like (probably can be done >> easier): >> >> for $i at $pos in db:open("DB")//* >> where $i[text() contains text 'xml'] >> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >> 'xml'])}</score></hit> >> >> -- Andreas >> >> Am 03.04.2011 um 12:42 schrieb Wiard Vasen: >> >> Dear Christian and Andreas, >> >> Thanks for your great help! >> I used Christians solution: ft:score(db:open("DB")//*[text() contains >> text 'xml']) >> And it works fine. >> >> The next step is that I want to get the associated documents with these >> scores. >> >> Could you help me with this step? >> >> The results I get now is a list with frequencies, without the references >> to the particular documents. >> I think what is needed is the tf/idf score. >> >> Regards, >> >> Wiard >> >> >> 2011/3/31 Christian Grün <christian.gruen at gmail.com> >> >>> Hi Wiard, >>> >>> the tf/idf scoring is only available if you are working with full-text >>> index structures. If you have built a full-text index for your >>> database "DB", the following query will yield different scoring >>> results, depending on the chosen scoring model: >>> >>> ft:score(db:open("DB")//*[text() contains text 'xml']) >>> >>> As Andreas indicated, however, you may either set the SCORING property >>> with a db command or explicitly choose the type of scoring in the >>> GUI's database creation dialog (Database -> New -> Fulltext -> TF/IDF >>> Scoring). >>> >>> Christian >>> >>> >>> >>> On Thu, Mar 31, 2011 at 8:33 AM, Andreas Weiler >>> <andreas.weiler at uni-konstanz.de> wrote: >>> > Dear Wiard Vasen, >>> > you just need to set the scoring property once. >>> > If you work in the GUI: >>> > Go to the top input bar, choose command and type: >>> > set scoring * >>> > >>> > as * set the scoring algorithm you like. >>> > In the console just type: set scoring * >>> > After setting this you can use the score function, like in the 8th >>> query of >>> > our online demo (basex.org/products/live-demo): >>> > let $names := ('Jack London', 'Jack', 'Jim Beam', 'Jack Daniels') >>> > for $name at $pos score $score in $names[. contains text 'Jack'] >>> > order by $score descending >>> > return <name pos='{ $pos }'>{ $name }</name> >>> > Don't hesitate to ask for more, >>> > Andreas >>> > Am 30.03.2011 um 22:02 schrieb Wiard Vasen: >>> > >>> > Dear sirs of Basex, >>> > I am doing my Master thesis on the letters of Vincent van Gogh at the >>> > University of Amsterdam. >>> > For that purpose I use BaseX to analyze the letters. >>> > I wonder, is there the possibility to generate a tf/idf score >>> automatically? >>> > In your faq I noticed there needs to be a special term like 'SET >>> SCORING 0' >>> > to be able to get a tf/idf score. >>> > This information I get from the following >>> > page: http://docs.basex.org/wiki/Full-Text >>> > Could you help me with this? >>> > I would be very grateful. >>> > Kind regards, >>> > _______________________________________________ >>> > BaseX-Talk mailing list >>> > BaseX-Talk at mailman.uni-konstanz.de >>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>> > >>> > >>> > _______________________________________________ >>> > BaseX-Talk mailing list >>> > BaseX-Talk at mailman.uni-konstanz.de >>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>> > >>> > >>> >> >> >> > >

Am 03.04.2011 um 18:14 schrieb Wiard Vasen: > Hi Andreas, > > Thanks a lot! It works fine. > > I was wondering if instead of putting in the next query in BaseX: > > for $n in ("let680.xml", "let681.xml", "let682.xml", "let683.xml", "let684.xml", "let685.xml", "let686.xml", "let687.xml", "let688.xml", "let689.xml") > return > for $i at $pos in db:open("tfidfbrievenvangogh")//* > where ends-with(base-uri($i), $n) > and $i[text() contains text 'above'] > return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text 'above'])}</score></hit> > > It is also possible to do something like: > > for ("let680.xml" )<= $n <= ("let689") > return > for $i at $pos in db:open("tfidfbrievenvangogh")//* > where ends-with(base-uri($i), $n) > and $i[text() contains text 'above'] > return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text 'above'])}</score></hit> > > That way I hope to define the outer documents of a subset and get all the documents in between, with the outer documents included. > > Do you think this is possible in a query like shown above? > > Regards, > > Wiard > > > > > 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> > Hi Wiard, > > i hope i understand your plans, here is what i would do: > > for $n in ("betweenlet567.xml", "let689.xml") > return > for $i at $pos in db:open("tfidfbrievenvangogh")//* > where ends-with(base-uri($i), $n) > and $i[text() contains text 'kleur'] > return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text 'kleur'])}</score></hit> > > Now you can extend the variable $n with all filenames you like to have. > > I hope this helps, > Andreas > > Am 03.04.2011 um 14:24 schrieb Wiard Vasen: > >> Hi Andreas, >> >> Wow! This is the complete answer to my question! >> >> I hope you can help me with the next question. >> Because I am analyzing changes in the artistic life of Van Gogh, I am partitioning the relatively large repository annotated xml files on the basis of residence. >> >> For that reason I need to put a query like: >> >> for $i at $pos in db:open("tfidfbrievenvangogh")//* >> where $i[text() contains text 'kleur'] >> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text 'kleur'])}</score></hit> >> >> with the extension: given the interval, all xml-files betweenlet567.xml and let689.xml. >> What means that I know that in this partition xml-files Van Gogh was in Arles. >> And I want to know what is the tf-idf score of the dutch word 'kleur'. >> >> To give a resume of my question: How do I partition the repository in subsets, so that I can produce information on these subsets. >> And how do I do this in BasX with xquery. >> >> Thanks a lot beforehand! >> >> Kind regards, >> >> Wiard >> >> >> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >> Hi Wiard, >> >> you could use the base-uri function of XQuery, like (probably can be done easier): >> >> for $i at $pos in db:open("DB")//* >> where $i[text() contains text 'xml'] >> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text 'xml'])}</score></hit> >> >> -- Andreas >> >> Am 03.04.2011 um 12:42 schrieb Wiard Vasen: >> >>> Dear Christian and Andreas, >>> >>> Thanks for your great help! >>> I used Christians solution: ft:score(db:open("DB")//*[text() contains text 'xml']) >>> And it works fine. >>> >>> The next step is that I want to get the associated documents with these scores. >>> >>> Could you help me with this step? >>> >>> The results I get now is a list with frequencies, without the references to the particular documents. >>> I think what is needed is the tf/idf score. >>> >>> Regards, >>> >>> Wiard >>> >>> >>> 2011/3/31 Christian Grün <christian.gruen at gmail.com> >>> Hi Wiard, >>> >>> the tf/idf scoring is only available if you are working with full-text >>> index structures. If you have built a full-text index for your >>> database "DB", the following query will yield different scoring >>> results, depending on the chosen scoring model: >>> >>> ft:score(db:open("DB")//*[text() contains text 'xml']) >>> >>> As Andreas indicated, however, you may either set the SCORING property >>> with a db command or explicitly choose the type of scoring in the >>> GUI's database creation dialog (Database -> New -> Fulltext -> TF/IDF >>> Scoring). >>> >>> Christian >>> >>> >>> >>> On Thu, Mar 31, 2011 at 8:33 AM, Andreas Weiler >>> <andreas.weiler at uni-konstanz.de> wrote: >>> > Dear Wiard Vasen, >>> > you just need to set the scoring property once. >>> > If you work in the GUI: >>> > Go to the top input bar, choose command and type: >>> > set scoring * >>> > >>> > as * set the scoring algorithm you like. >>> > In the console just type: set scoring * >>> > After setting this you can use the score function, like in the 8th query of >>> > our online demo (basex.org/products/live-demo): >>> > let $names := ('Jack London', 'Jack', 'Jim Beam', 'Jack Daniels') >>> > for $name at $pos score $score in $names[. contains text 'Jack'] >>> > order by $score descending >>> > return <name pos='{ $pos }'>{ $name }</name> >>> > Don't hesitate to ask for more, >>> > Andreas >>> > Am 30.03.2011 um 22:02 schrieb Wiard Vasen: >>> > >>> > Dear sirs of Basex, >>> > I am doing my Master thesis on the letters of Vincent van Gogh at the >>> > University of Amsterdam. >>> > For that purpose I use BaseX to analyze the letters. >>> > I wonder, is there the possibility to generate a tf/idf score automatically? >>> > In your faq I noticed there needs to be a special term like 'SET SCORING 0' >>> > to be able to get a tf/idf score. >>> > This information I get from the following >>> > page: http://docs.basex.org/wiki/Full-Text >>> > Could you help me with this? >>> > I would be very grateful. >>> > Kind regards, >>> > _______________________________________________ >>> > BaseX-Talk mailing list >>> > BaseX-Talk at mailman.uni-konstanz.de >>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>> > >>> > >>> > _______________________________________________ >>> > BaseX-Talk mailing list >>> > BaseX-Talk at mailman.uni-konstanz.de >>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>> > >>> > >>> >> >> > >

> Hi Wiard, > > you can get the position of your wished first and last document with: (add > a where clause to get the right documents, like where > ends-with(base-uri($n), "test.doc")) > > for $n at $pos in db:open("tfidfbrievenvangogh") > return <hit><doc>{base-uri($i)</doc><pos>{$pos}</pos></hit> > > then set these position for x and y in the below query. > > for $n at $pos in db:open("tfidfbrievenvangogh") > where $pos > x and $pos < y > return > for $i in $n//* > where $i[text() contains text 'above'] > return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text > 'above'])}</score></hit> > > I hope this works. > > -- Andreas > > Am 03.04.2011 um 18:14 schrieb Wiard Vasen: > > Hi Andreas, > > Thanks a lot! It works fine. > > I was wondering if instead of putting in the next query in BaseX: > > for $n in ("let680.xml", "let681.xml", "let682.xml", "let683.xml", > "let684.xml", "let685.xml", "let686.xml", "let687.xml", "let688.xml", > "let689.xml") > return > for $i at $pos in db:open("tfidfbrievenvangogh")//* > where ends-with(base-uri($i), $n) > and $i[text() contains text 'above'] > return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text > 'above'])}</score></hit> > > It is also possible to do something like: > > for ("let680.xml" )<= $n <= ("let689") > return > for $i at $pos in db:open("tfidfbrievenvangogh")//* > where ends-with(base-uri($i), $n) > and $i[text() contains text 'above'] > return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text > 'above'])}</score></hit> > > That way I hope to define the outer documents of a subset and get all the > documents in between, with the outer documents included. > > Do you think this is possible in a query like shown above? > > Regards, > > Wiard > > > > > 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> > >> Hi Wiard, >> >> i hope i understand your plans, here is what i would do: >> >> for $n in ("betweenlet567.xml", "let689.xml") >> return >> for $i at $pos in db:open("tfidfbrievenvangogh")//* >> where ends-with(base-uri($i), $n) >> and $i[text() contains text 'kleur'] >> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >> 'kleur'])}</score></hit> >> >> Now you can extend the variable $n with all filenames you like to have. >> >> I hope this helps, >> Andreas >> >> Am 03.04.2011 um 14:24 schrieb Wiard Vasen: >> >> Hi Andreas, >> >> Wow! This is the complete answer to my question! >> >> I hope you can help me with the next question. >> Because I am analyzing changes in the artistic life of Van Gogh, I am >> partitioning the relatively large repository annotated xml files on the >> basis of residence. >> >> For that reason I need to put a query like: >> >> for $i at $pos in db:open("tfidfbrievenvangogh")//* >> where $i[text() contains text 'kleur'] >> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >> 'kleur'])}</score></hit> >> >> with the extension: given the interval, all xml-files betweenlet567.xml >> and let689.xml. >> What means that I know that in this partition xml-files Van Gogh was in >> Arles. >> And I want to know what is the tf-idf score of the dutch word 'kleur'. >> >> To give a resume of my question: How do I partition the repository in >> subsets, so that I can produce information on these subsets. >> And how do I do this in BasX with xquery. >> >> Thanks a lot beforehand! >> >> Kind regards, >> >> Wiard >> >> >> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >> >>> Hi Wiard, >>> >>> you could use the base-uri function of XQuery, like (probably can be done >>> easier): >>> >>> for $i at $pos in db:open("DB")//* >>> where $i[text() contains text 'xml'] >>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>> 'xml'])}</score></hit> >>> >>> -- Andreas >>> >>> Am 03.04.2011 um 12:42 schrieb Wiard Vasen: >>> >>> Dear Christian and Andreas, >>> >>> Thanks for your great help! >>> I used Christians solution: ft:score(db:open("DB")//*[text() contains >>> text 'xml']) >>> And it works fine. >>> >>> The next step is that I want to get the associated documents with these >>> scores. >>> >>> Could you help me with this step? >>> >>> The results I get now is a list with frequencies, without the references >>> to the particular documents. >>> I think what is needed is the tf/idf score. >>> >>> Regards, >>> >>> Wiard >>> >>> >>> 2011/3/31 Christian Grün <christian.gruen at gmail.com> >>> >>>> Hi Wiard, >>>> >>>> the tf/idf scoring is only available if you are working with full-text >>>> index structures. If you have built a full-text index for your >>>> database "DB", the following query will yield different scoring >>>> results, depending on the chosen scoring model: >>>> >>>> ft:score(db:open("DB")//*[text() contains text 'xml']) >>>> >>>> As Andreas indicated, however, you may either set the SCORING property >>>> with a db command or explicitly choose the type of scoring in the >>>> GUI's database creation dialog (Database -> New -> Fulltext -> TF/IDF >>>> Scoring). >>>> >>>> Christian >>>> >>>> >>>> >>>> On Thu, Mar 31, 2011 at 8:33 AM, Andreas Weiler >>>> <andreas.weiler at uni-konstanz.de> wrote: >>>> > Dear Wiard Vasen, >>>> > you just need to set the scoring property once. >>>> > If you work in the GUI: >>>> > Go to the top input bar, choose command and type: >>>> > set scoring * >>>> > >>>> > as * set the scoring algorithm you like. >>>> > In the console just type: set scoring * >>>> > After setting this you can use the score function, like in the 8th >>>> query of >>>> > our online demo (basex.org/products/live-demo): >>>> > let $names := ('Jack London', 'Jack', 'Jim Beam', 'Jack Daniels') >>>> > for $name at $pos score $score in $names[. contains text 'Jack'] >>>> > order by $score descending >>>> > return <name pos='{ $pos }'>{ $name }</name> >>>> > Don't hesitate to ask for more, >>>> > Andreas >>>> > Am 30.03.2011 um 22:02 schrieb Wiard Vasen: >>>> > >>>> > Dear sirs of Basex, >>>> > I am doing my Master thesis on the letters of Vincent van Gogh at the >>>> > University of Amsterdam. >>>> > For that purpose I use BaseX to analyze the letters. >>>> > I wonder, is there the possibility to generate a tf/idf score >>>> automatically? >>>> > In your faq I noticed there needs to be a special term like 'SET >>>> SCORING 0' >>>> > to be able to get a tf/idf score. >>>> > This information I get from the following >>>> > page: http://docs.basex.org/wiki/Full-Text >>>> > Could you help me with this? >>>> > I would be very grateful. >>>> > Kind regards, >>>> > _______________________________________________ >>>> > BaseX-Talk mailing list >>>> > BaseX-Talk at mailman.uni-konstanz.de >>>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>> > >>>> > >>>> > _______________________________________________ >>>> > BaseX-Talk mailing list >>>> > BaseX-Talk at mailman.uni-konstanz.de >>>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>> > >>>> > >>>> >>> >>> >>> >> >> > >

Am 03.04.2011 um 18:44 schrieb Wiard Vasen: > Hi Andreas, > > Maybe I don't understand the query you suggested. I worked it out this way: > > for $n at $pos in db:open("tfidfbrievenvangogh") > where $pos > ( "let001.xml") and $pos < ( "let201.xml") > return > for $i in $n//* > where $i[text() contains text 'above'] > return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text 'above'])}</score></hit> > > I do understand though the error: xs:integer and xs:string can't be compared > > How do I improve this query, so that it works? > > Regards, > > Wiard > > 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> > Hi Wiard, > > you can get the position of your wished first and last document with: (add a where clause to get the right documents, like where ends-with(base-uri($n), "test.doc")) > > for $n at $pos in db:open("tfidfbrievenvangogh") > return <hit><doc>{base-uri($i)</doc><pos>{$pos}</pos></hit> > > then set these position for x and y in the below query. > > for $n at $pos in db:open("tfidfbrievenvangogh") > where $pos > x and $pos < y > return > for $i in $n//* > where $i[text() contains text 'above'] > return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text 'above'])}</score></hit> > > I hope this works. > > -- Andreas > > Am 03.04.2011 um 18:14 schrieb Wiard Vasen: > >> Hi Andreas, >> >> Thanks a lot! It works fine. >> >> I was wondering if instead of putting in the next query in BaseX: >> >> for $n in ("let680.xml", "let681.xml", "let682.xml", "let683.xml", "let684.xml", "let685.xml", "let686.xml", "let687.xml", "let688.xml", "let689.xml") >> return >> for $i at $pos in db:open("tfidfbrievenvangogh")//* >> where ends-with(base-uri($i), $n) >> and $i[text() contains text 'above'] >> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text 'above'])}</score></hit> >> >> It is also possible to do something like: >> >> for ("let680.xml" )<= $n <= ("let689") >> return >> for $i at $pos in db:open("tfidfbrievenvangogh")//* >> where ends-with(base-uri($i), $n) >> and $i[text() contains text 'above'] >> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text 'above'])}</score></hit> >> >> That way I hope to define the outer documents of a subset and get all the documents in between, with the outer documents included. >> >> Do you think this is possible in a query like shown above? >> >> Regards, >> >> Wiard >> >> >> >> >> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >> Hi Wiard, >> >> i hope i understand your plans, here is what i would do: >> >> for $n in ("betweenlet567.xml", "let689.xml") >> return >> for $i at $pos in db:open("tfidfbrievenvangogh")//* >> where ends-with(base-uri($i), $n) >> and $i[text() contains text 'kleur'] >> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text 'kleur'])}</score></hit> >> >> Now you can extend the variable $n with all filenames you like to have. >> >> I hope this helps, >> Andreas >> >> Am 03.04.2011 um 14:24 schrieb Wiard Vasen: >> >>> Hi Andreas, >>> >>> Wow! This is the complete answer to my question! >>> >>> I hope you can help me with the next question. >>> Because I am analyzing changes in the artistic life of Van Gogh, I am partitioning the relatively large repository annotated xml files on the basis of residence. >>> >>> For that reason I need to put a query like: >>> >>> for $i at $pos in db:open("tfidfbrievenvangogh")//* >>> where $i[text() contains text 'kleur'] >>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text 'kleur'])}</score></hit> >>> >>> with the extension: given the interval, all xml-files betweenlet567.xml and let689.xml. >>> What means that I know that in this partition xml-files Van Gogh was in Arles. >>> And I want to know what is the tf-idf score of the dutch word 'kleur'. >>> >>> To give a resume of my question: How do I partition the repository in subsets, so that I can produce information on these subsets. >>> And how do I do this in BasX with xquery. >>> >>> Thanks a lot beforehand! >>> >>> Kind regards, >>> >>> Wiard >>> >>> >>> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >>> Hi Wiard, >>> >>> you could use the base-uri function of XQuery, like (probably can be done easier): >>> >>> for $i at $pos in db:open("DB")//* >>> where $i[text() contains text 'xml'] >>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text 'xml'])}</score></hit> >>> >>> -- Andreas >>> >>> Am 03.04.2011 um 12:42 schrieb Wiard Vasen: >>> >>>> Dear Christian and Andreas, >>>> >>>> Thanks for your great help! >>>> I used Christians solution: ft:score(db:open("DB")//*[text() contains text 'xml']) >>>> And it works fine. >>>> >>>> The next step is that I want to get the associated documents with these scores. >>>> >>>> Could you help me with this step? >>>> >>>> The results I get now is a list with frequencies, without the references to the particular documents. >>>> I think what is needed is the tf/idf score. >>>> >>>> Regards, >>>> >>>> Wiard >>>> >>>> >>>> 2011/3/31 Christian Grün <christian.gruen at gmail.com> >>>> Hi Wiard, >>>> >>>> the tf/idf scoring is only available if you are working with full-text >>>> index structures. If you have built a full-text index for your >>>> database "DB", the following query will yield different scoring >>>> results, depending on the chosen scoring model: >>>> >>>> ft:score(db:open("DB")//*[text() contains text 'xml']) >>>> >>>> As Andreas indicated, however, you may either set the SCORING property >>>> with a db command or explicitly choose the type of scoring in the >>>> GUI's database creation dialog (Database -> New -> Fulltext -> TF/IDF >>>> Scoring). >>>> >>>> Christian >>>> >>>> >>>> >>>> On Thu, Mar 31, 2011 at 8:33 AM, Andreas Weiler >>>> <andreas.weiler at uni-konstanz.de> wrote: >>>> > Dear Wiard Vasen, >>>> > you just need to set the scoring property once. >>>> > If you work in the GUI: >>>> > Go to the top input bar, choose command and type: >>>> > set scoring * >>>> > >>>> > as * set the scoring algorithm you like. >>>> > In the console just type: set scoring * >>>> > After setting this you can use the score function, like in the 8th query of >>>> > our online demo (basex.org/products/live-demo): >>>> > let $names := ('Jack London', 'Jack', 'Jim Beam', 'Jack Daniels') >>>> > for $name at $pos score $score in $names[. contains text 'Jack'] >>>> > order by $score descending >>>> > return <name pos='{ $pos }'>{ $name }</name> >>>> > Don't hesitate to ask for more, >>>> > Andreas >>>> > Am 30.03.2011 um 22:02 schrieb Wiard Vasen: >>>> > >>>> > Dear sirs of Basex, >>>> > I am doing my Master thesis on the letters of Vincent van Gogh at the >>>> > University of Amsterdam. >>>> > For that purpose I use BaseX to analyze the letters. >>>> > I wonder, is there the possibility to generate a tf/idf score automatically? >>>> > In your faq I noticed there needs to be a special term like 'SET SCORING 0' >>>> > to be able to get a tf/idf score. >>>> > This information I get from the following >>>> > page: http://docs.basex.org/wiki/Full-Text >>>> > Could you help me with this? >>>> > I would be very grateful. >>>> > Kind regards, >>>> > _______________________________________________ >>>> > BaseX-Talk mailing list >>>> > BaseX-Talk at mailman.uni-konstanz.de >>>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>> > >>>> > >>>> > _______________________________________________ >>>> > BaseX-Talk mailing list >>>> > BaseX-Talk at mailman.uni-konstanz.de >>>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>> > >>>> > >>>> >>> >>> >> >> > >

Am 03.04.2011 18:44, schrieb Wiard Vasen: > for $n at $pos in db:open("tfidfbrievenvangogh") > where $pos > ( "let001.xml") and $pos < ( "let201.xml") > return > for $i in $n//* > where $i[text() contains text 'above'] > return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text > 'above'])}</score></hit>

> Dear Wiard, > > Am 03.04.2011 18:44, schrieb Wiard Vasen: > > for $n at $pos in db:open("tfidfbrievenvangogh") > > where $pos > ( "let001.xml") and $pos < ( "let201.xml") > > return > > for $i in $n//* > > where $i[text() contains text 'above'] > > return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text > > 'above'])}</score></hit> > > does this (rather drastical) rewriting approximately do what you want? > > let $range := 1 to 201 > for $doc in collection('tfidfbrievenvangogh') > let $uri := base-uri($doc), > $num := substring($uri, string-length($uri) - 6, 3) > where $num castable as xs:integer > and xs:integer($num) = $range > return <document uri='{$uri}'>{ > for $n score $s in $doc//*[text() contains text 'above'] > return <hit score='{$s}'>{ $n }</hit> > }</document > > I added some structure to the output (which you can easily remove), used > only standard functions and optimized the query for performance where I > saw fit. > > Hope that helps, > cheers Leo >

> I did think that you first get the positions of your first and last > document in your range (first query). > Note them and put them into the second query for x and y. > > -- Andreas > > Am 03.04.2011 um 18:44 schrieb Wiard Vasen: > > Hi Andreas, > > Maybe I don't understand the query you suggested. I worked it out this way: > > for $n at $pos in db:open("tfidfbrievenvangogh") > where $pos > ( "let001.xml") and $pos < ( "let201.xml") > return > for $i in $n//* > where $i[text() contains text 'above'] > return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text > 'above'])}</score></hit> > > I do understand though the error: xs:integer and xs:string can't be > compared > > How do I improve this query, so that it works? > > Regards, > > Wiard > > 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> > >> Hi Wiard, >> >> you can get the position of your wished first and last document with: (add >> a where clause to get the right documents, like where >> ends-with(base-uri($n), "test.doc")) >> >> for $n at $pos in db:open("tfidfbrievenvangogh") >> return <hit><doc>{base-uri($i)</doc><pos>{$pos}</pos></hit> >> >> then set these position for x and y in the below query. >> >> for $n at $pos in db:open("tfidfbrievenvangogh") >> where $pos > x and $pos < y >> return >> for $i in $n//* >> where $i[text() contains text 'above'] >> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >> 'above'])}</score></hit> >> >> I hope this works. >> >> -- Andreas >> >> Am 03.04.2011 um 18:14 schrieb Wiard Vasen: >> >> Hi Andreas, >> >> Thanks a lot! It works fine. >> >> I was wondering if instead of putting in the next query in BaseX: >> >> for $n in ("let680.xml", "let681.xml", "let682.xml", "let683.xml", >> "let684.xml", "let685.xml", "let686.xml", "let687.xml", "let688.xml", >> "let689.xml") >> return >> for $i at $pos in db:open("tfidfbrievenvangogh")//* >> where ends-with(base-uri($i), $n) >> and $i[text() contains text 'above'] >> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >> 'above'])}</score></hit> >> >> It is also possible to do something like: >> >> for ("let680.xml" )<= $n <= ("let689") >> return >> for $i at $pos in db:open("tfidfbrievenvangogh")//* >> where ends-with(base-uri($i), $n) >> and $i[text() contains text 'above'] >> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >> 'above'])}</score></hit> >> >> That way I hope to define the outer documents of a subset and get all the >> documents in between, with the outer documents included. >> >> Do you think this is possible in a query like shown above? >> >> Regards, >> >> Wiard >> >> >> >> >> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >> >>> Hi Wiard, >>> >>> i hope i understand your plans, here is what i would do: >>> >>> for $n in ("betweenlet567.xml", "let689.xml") >>> return >>> for $i at $pos in db:open("tfidfbrievenvangogh")//* >>> where ends-with(base-uri($i), $n) >>> and $i[text() contains text 'kleur'] >>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>> 'kleur'])}</score></hit> >>> >>> Now you can extend the variable $n with all filenames you like to have. >>> >>> I hope this helps, >>> Andreas >>> >>> Am 03.04.2011 um 14:24 schrieb Wiard Vasen: >>> >>> Hi Andreas, >>> >>> Wow! This is the complete answer to my question! >>> >>> I hope you can help me with the next question. >>> Because I am analyzing changes in the artistic life of Van Gogh, I am >>> partitioning the relatively large repository annotated xml files on the >>> basis of residence. >>> >>> For that reason I need to put a query like: >>> >>> for $i at $pos in db:open("tfidfbrievenvangogh")//* >>> where $i[text() contains text 'kleur'] >>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>> 'kleur'])}</score></hit> >>> >>> with the extension: given the interval, all xml-files betweenlet567.xml >>> and let689.xml. >>> What means that I know that in this partition xml-files Van Gogh was in >>> Arles. >>> And I want to know what is the tf-idf score of the dutch word 'kleur'. >>> >>> To give a resume of my question: How do I partition the repository in >>> subsets, so that I can produce information on these subsets. >>> And how do I do this in BasX with xquery. >>> >>> Thanks a lot beforehand! >>> >>> Kind regards, >>> >>> Wiard >>> >>> >>> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >>> >>>> Hi Wiard, >>>> >>>> you could use the base-uri function of XQuery, like (probably can be >>>> done easier): >>>> >>>> for $i at $pos in db:open("DB")//* >>>> where $i[text() contains text 'xml'] >>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>> 'xml'])}</score></hit> >>>> >>>> -- Andreas >>>> >>>> Am 03.04.2011 um 12:42 schrieb Wiard Vasen: >>>> >>>> Dear Christian and Andreas, >>>> >>>> Thanks for your great help! >>>> I used Christians solution: ft:score(db:open("DB")//*[text() contains >>>> text 'xml']) >>>> And it works fine. >>>> >>>> The next step is that I want to get the associated documents with these >>>> scores. >>>> >>>> Could you help me with this step? >>>> >>>> The results I get now is a list with frequencies, without the references >>>> to the particular documents. >>>> I think what is needed is the tf/idf score. >>>> >>>> Regards, >>>> >>>> Wiard >>>> >>>> >>>> 2011/3/31 Christian Grün <christian.gruen at gmail.com> >>>> >>>>> Hi Wiard, >>>>> >>>>> the tf/idf scoring is only available if you are working with full-text >>>>> index structures. If you have built a full-text index for your >>>>> database "DB", the following query will yield different scoring >>>>> results, depending on the chosen scoring model: >>>>> >>>>> ft:score(db:open("DB")//*[text() contains text 'xml']) >>>>> >>>>> As Andreas indicated, however, you may either set the SCORING property >>>>> with a db command or explicitly choose the type of scoring in the >>>>> GUI's database creation dialog (Database -> New -> Fulltext -> TF/IDF >>>>> Scoring). >>>>> >>>>> Christian >>>>> >>>>> >>>>> >>>>> On Thu, Mar 31, 2011 at 8:33 AM, Andreas Weiler >>>>> <andreas.weiler at uni-konstanz.de> wrote: >>>>> > Dear Wiard Vasen, >>>>> > you just need to set the scoring property once. >>>>> > If you work in the GUI: >>>>> > Go to the top input bar, choose command and type: >>>>> > set scoring * >>>>> > >>>>> > as * set the scoring algorithm you like. >>>>> > In the console just type: set scoring * >>>>> > After setting this you can use the score function, like in the 8th >>>>> query of >>>>> > our online demo (basex.org/products/live-demo): >>>>> > let $names := ('Jack London', 'Jack', 'Jim Beam', 'Jack Daniels') >>>>> > for $name at $pos score $score in $names[. contains text 'Jack'] >>>>> > order by $score descending >>>>> > return <name pos='{ $pos }'>{ $name }</name> >>>>> > Don't hesitate to ask for more, >>>>> > Andreas >>>>> > Am 30.03.2011 um 22:02 schrieb Wiard Vasen: >>>>> > >>>>> > Dear sirs of Basex, >>>>> > I am doing my Master thesis on the letters of Vincent van Gogh at the >>>>> > University of Amsterdam. >>>>> > For that purpose I use BaseX to analyze the letters. >>>>> > I wonder, is there the possibility to generate a tf/idf score >>>>> automatically? >>>>> > In your faq I noticed there needs to be a special term like 'SET >>>>> SCORING 0' >>>>> > to be able to get a tf/idf score. >>>>> > This information I get from the following >>>>> > page: http://docs.basex.org/wiki/Full-Text >>>>> > Could you help me with this? >>>>> > I would be very grateful. >>>>> > Kind regards, >>>>> > _______________________________________________ >>>>> > BaseX-Talk mailing list >>>>> > BaseX-Talk at mailman.uni-konstanz.de >>>>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>>> > >>>>> > >>>>> > _______________________________________________ >>>>> > BaseX-Talk mailing list >>>>> > BaseX-Talk at mailman.uni-konstanz.de >>>>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>>> > >>>>> > >>>>> >>>> >>>> >>>> >>> >>> >> >> > >

> Thank you very much Andreas, > > You, Christian and Leonard have helped me a lot! > > Have a nice evening! > > Regards, > > Wiard > > > have helped me a lot! > > > 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> > >> I did think that you first get the positions of your first and last >> document in your range (first query). >> Note them and put them into the second query for x and y. >> >> -- Andreas >> >> Am 03.04.2011 um 18:44 schrieb Wiard Vasen: >> >> Hi Andreas, >> >> Maybe I don't understand the query you suggested. I worked it out this >> way: >> >> for $n at $pos in db:open("tfidfbrievenvangogh") >> where $pos > ( "let001.xml") and $pos < ( "let201.xml") >> return >> for $i in $n//* >> where $i[text() contains text 'above'] >> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >> 'above'])}</score></hit> >> >> I do understand though the error: xs:integer and xs:string can't be >> compared >> >> How do I improve this query, so that it works? >> >> Regards, >> >> Wiard >> >> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >> >>> Hi Wiard, >>> >>> you can get the position of your wished first and last document with: >>> (add a where clause to get the right documents, like where >>> ends-with(base-uri($n), "test.doc")) >>> >>> for $n at $pos in db:open("tfidfbrievenvangogh") >>> return <hit><doc>{base-uri($i)</doc><pos>{$pos}</pos></hit> >>> >>> then set these position for x and y in the below query. >>> >>> for $n at $pos in db:open("tfidfbrievenvangogh") >>> where $pos > x and $pos < y >>> return >>> for $i in $n//* >>> where $i[text() contains text 'above'] >>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>> 'above'])}</score></hit> >>> >>> I hope this works. >>> >>> -- Andreas >>> >>> Am 03.04.2011 um 18:14 schrieb Wiard Vasen: >>> >>> Hi Andreas, >>> >>> Thanks a lot! It works fine. >>> >>> I was wondering if instead of putting in the next query in BaseX: >>> >>> for $n in ("let680.xml", "let681.xml", "let682.xml", "let683.xml", >>> "let684.xml", "let685.xml", "let686.xml", "let687.xml", "let688.xml", >>> "let689.xml") >>> return >>> for $i at $pos in db:open("tfidfbrievenvangogh")//* >>> where ends-with(base-uri($i), $n) >>> and $i[text() contains text 'above'] >>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>> 'above'])}</score></hit> >>> >>> It is also possible to do something like: >>> >>> for ("let680.xml" )<= $n <= ("let689") >>> return >>> for $i at $pos in db:open("tfidfbrievenvangogh")//* >>> where ends-with(base-uri($i), $n) >>> and $i[text() contains text 'above'] >>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>> 'above'])}</score></hit> >>> >>> That way I hope to define the outer documents of a subset and get all the >>> documents in between, with the outer documents included. >>> >>> Do you think this is possible in a query like shown above? >>> >>> Regards, >>> >>> Wiard >>> >>> >>> >>> >>> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >>> >>>> Hi Wiard, >>>> >>>> i hope i understand your plans, here is what i would do: >>>> >>>> for $n in ("betweenlet567.xml", "let689.xml") >>>> return >>>> for $i at $pos in db:open("tfidfbrievenvangogh")//* >>>> where ends-with(base-uri($i), $n) >>>> and $i[text() contains text 'kleur'] >>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>> 'kleur'])}</score></hit> >>>> >>>> Now you can extend the variable $n with all filenames you like to have. >>>> >>>> I hope this helps, >>>> Andreas >>>> >>>> Am 03.04.2011 um 14:24 schrieb Wiard Vasen: >>>> >>>> Hi Andreas, >>>> >>>> Wow! This is the complete answer to my question! >>>> >>>> I hope you can help me with the next question. >>>> Because I am analyzing changes in the artistic life of Van Gogh, I am >>>> partitioning the relatively large repository annotated xml files on the >>>> basis of residence. >>>> >>>> For that reason I need to put a query like: >>>> >>>> for $i at $pos in db:open("tfidfbrievenvangogh")//* >>>> where $i[text() contains text 'kleur'] >>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>> 'kleur'])}</score></hit> >>>> >>>> with the extension: given the interval, all xml-files betweenlet567.xml >>>> and let689.xml. >>>> What means that I know that in this partition xml-files Van Gogh was in >>>> Arles. >>>> And I want to know what is the tf-idf score of the dutch word 'kleur'. >>>> >>>> To give a resume of my question: How do I partition the repository in >>>> subsets, so that I can produce information on these subsets. >>>> And how do I do this in BasX with xquery. >>>> >>>> Thanks a lot beforehand! >>>> >>>> Kind regards, >>>> >>>> Wiard >>>> >>>> >>>> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >>>> >>>>> Hi Wiard, >>>>> >>>>> you could use the base-uri function of XQuery, like (probably can be >>>>> done easier): >>>>> >>>>> for $i at $pos in db:open("DB")//* >>>>> where $i[text() contains text 'xml'] >>>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>>> 'xml'])}</score></hit> >>>>> >>>>> -- Andreas >>>>> >>>>> Am 03.04.2011 um 12:42 schrieb Wiard Vasen: >>>>> >>>>> Dear Christian and Andreas, >>>>> >>>>> Thanks for your great help! >>>>> I used Christians solution: ft:score(db:open("DB")//*[text() contains >>>>> text 'xml']) >>>>> And it works fine. >>>>> >>>>> The next step is that I want to get the associated documents with these >>>>> scores. >>>>> >>>>> Could you help me with this step? >>>>> >>>>> The results I get now is a list with frequencies, without the >>>>> references to the particular documents. >>>>> I think what is needed is the tf/idf score. >>>>> >>>>> Regards, >>>>> >>>>> Wiard >>>>> >>>>> >>>>> 2011/3/31 Christian Grün <christian.gruen at gmail.com> >>>>> >>>>>> Hi Wiard, >>>>>> >>>>>> the tf/idf scoring is only available if you are working with full-text >>>>>> index structures. If you have built a full-text index for your >>>>>> database "DB", the following query will yield different scoring >>>>>> results, depending on the chosen scoring model: >>>>>> >>>>>> ft:score(db:open("DB")//*[text() contains text 'xml']) >>>>>> >>>>>> As Andreas indicated, however, you may either set the SCORING property >>>>>> with a db command or explicitly choose the type of scoring in the >>>>>> GUI's database creation dialog (Database -> New -> Fulltext -> TF/IDF >>>>>> Scoring). >>>>>> >>>>>> Christian >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Mar 31, 2011 at 8:33 AM, Andreas Weiler >>>>>> <andreas.weiler at uni-konstanz.de> wrote: >>>>>> > Dear Wiard Vasen, >>>>>> > you just need to set the scoring property once. >>>>>> > If you work in the GUI: >>>>>> > Go to the top input bar, choose command and type: >>>>>> > set scoring * >>>>>> > >>>>>> > as * set the scoring algorithm you like. >>>>>> > In the console just type: set scoring * >>>>>> > After setting this you can use the score function, like in the 8th >>>>>> query of >>>>>> > our online demo (basex.org/products/live-demo): >>>>>> > let $names := ('Jack London', 'Jack', 'Jim Beam', 'Jack Daniels') >>>>>> > for $name at $pos score $score in $names[. contains text 'Jack'] >>>>>> > order by $score descending >>>>>> > return <name pos='{ $pos }'>{ $name }</name> >>>>>> > Don't hesitate to ask for more, >>>>>> > Andreas >>>>>> > Am 30.03.2011 um 22:02 schrieb Wiard Vasen: >>>>>> > >>>>>> > Dear sirs of Basex, >>>>>> > I am doing my Master thesis on the letters of Vincent van Gogh at >>>>>> the >>>>>> > University of Amsterdam. >>>>>> > For that purpose I use BaseX to analyze the letters. >>>>>> > I wonder, is there the possibility to generate a tf/idf score >>>>>> automatically? >>>>>> > In your faq I noticed there needs to be a special term like 'SET >>>>>> SCORING 0' >>>>>> > to be able to get a tf/idf score. >>>>>> > This information I get from the following >>>>>> > page: http://docs.basex.org/wiki/Full-Text >>>>>> > Could you help me with this? >>>>>> > I would be very grateful. >>>>>> > Kind regards, >>>>>> > _______________________________________________ >>>>>> > BaseX-Talk mailing list >>>>>> > BaseX-Talk at mailman.uni-konstanz.de >>>>>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>>>> > >>>>>> > >>>>>> > _______________________________________________ >>>>>> > BaseX-Talk mailing list >>>>>> > BaseX-Talk at mailman.uni-konstanz.de >>>>>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>>>> > >>>>>> > >>>>>> >>>>> >>>>> >>>>> >>>> >>>> >>> >>> >> >> >

>

> > > Kind regards, > > Wiard > > > > 2011/4/3 Wiard Vasen <wiard.vasen at gmail.com> > >> Thank you very much Andreas, >> >> You, Christian and Leonard have helped me a lot! >> >> Have a nice evening! >> >> Regards, >> >> Wiard >> >> >> have helped me a lot! >> >> >> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >> >>> I did think that you first get the positions of your first and last >>> document in your range (first query). >>> Note them and put them into the second query for x and y. >>> >>> -- Andreas >>> >>> Am 03.04.2011 um 18:44 schrieb Wiard Vasen: >>> >>> Hi Andreas, >>> >>> Maybe I don't understand the query you suggested. I worked it out this >>> way: >>> >>> for $n at $pos in db:open("tfidfbrievenvangogh") >>> where $pos > ( "let001.xml") and $pos < ( "let201.xml") >>> return >>> for $i in $n//* >>> where $i[text() contains text 'above'] >>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>> 'above'])}</score></hit> >>> >>> I do understand though the error: xs:integer and xs:string can't be >>> compared >>> >>> How do I improve this query, so that it works? >>> >>> Regards, >>> >>> Wiard >>> >>> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >>> >>>> Hi Wiard, >>>> >>>> you can get the position of your wished first and last document with: >>>> (add a where clause to get the right documents, like where >>>> ends-with(base-uri($n), "test.doc")) >>>> >>>> for $n at $pos in db:open("tfidfbrievenvangogh") >>>> return <hit><doc>{base-uri($i)</doc><pos>{$pos}</pos></hit> >>>> >>>> then set these position for x and y in the below query. >>>> >>>> for $n at $pos in db:open("tfidfbrievenvangogh") >>>> where $pos > x and $pos < y >>>> return >>>> for $i in $n//* >>>> where $i[text() contains text 'above'] >>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>> 'above'])}</score></hit> >>>> >>>> I hope this works. >>>> >>>> -- Andreas >>>> >>>> Am 03.04.2011 um 18:14 schrieb Wiard Vasen: >>>> >>>> Hi Andreas, >>>> >>>> Thanks a lot! It works fine. >>>> >>>> I was wondering if instead of putting in the next query in BaseX: >>>> >>>> for $n in ("let680.xml", "let681.xml", "let682.xml", "let683.xml", >>>> "let684.xml", "let685.xml", "let686.xml", "let687.xml", "let688.xml", >>>> "let689.xml") >>>> return >>>> for $i at $pos in db:open("tfidfbrievenvangogh")//* >>>> where ends-with(base-uri($i), $n) >>>> and $i[text() contains text 'above'] >>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>> 'above'])}</score></hit> >>>> >>>> It is also possible to do something like: >>>> >>>> for ("let680.xml" )<= $n <= ("let689") >>>> return >>>> for $i at $pos in db:open("tfidfbrievenvangogh")//* >>>> where ends-with(base-uri($i), $n) >>>> and $i[text() contains text 'above'] >>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>> 'above'])}</score></hit> >>>> >>>> That way I hope to define the outer documents of a subset and get all >>>> the documents in between, with the outer documents included. >>>> >>>> Do you think this is possible in a query like shown above? >>>> >>>> Regards, >>>> >>>> Wiard >>>> >>>> >>>> >>>> >>>> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >>>> >>>>> Hi Wiard, >>>>> >>>>> i hope i understand your plans, here is what i would do: >>>>> >>>>> for $n in ("betweenlet567.xml", "let689.xml") >>>>> return >>>>> for $i at $pos in db:open("tfidfbrievenvangogh")//* >>>>> where ends-with(base-uri($i), $n) >>>>> and $i[text() contains text 'kleur'] >>>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>>> 'kleur'])}</score></hit> >>>>> >>>>> Now you can extend the variable $n with all filenames you like to have. >>>>> >>>>> I hope this helps, >>>>> Andreas >>>>> >>>>> Am 03.04.2011 um 14:24 schrieb Wiard Vasen: >>>>> >>>>> Hi Andreas, >>>>> >>>>> Wow! This is the complete answer to my question! >>>>> >>>>> I hope you can help me with the next question. >>>>> Because I am analyzing changes in the artistic life of Van Gogh, I am >>>>> partitioning the relatively large repository annotated xml files on the >>>>> basis of residence. >>>>> >>>>> For that reason I need to put a query like: >>>>> >>>>> for $i at $pos in db:open("tfidfbrievenvangogh")//* >>>>> where $i[text() contains text 'kleur'] >>>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>>> 'kleur'])}</score></hit> >>>>> >>>>> with the extension: given the interval, all xml-files betweenlet567.xml >>>>> and let689.xml. >>>>> What means that I know that in this partition xml-files Van Gogh was in >>>>> Arles. >>>>> And I want to know what is the tf-idf score of the dutch word 'kleur'. >>>>> >>>>> To give a resume of my question: How do I partition the repository in >>>>> subsets, so that I can produce information on these subsets. >>>>> And how do I do this in BasX with xquery. >>>>> >>>>> Thanks a lot beforehand! >>>>> >>>>> Kind regards, >>>>> >>>>> Wiard >>>>> >>>>> >>>>> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >>>>> >>>>>> Hi Wiard, >>>>>> >>>>>> you could use the base-uri function of XQuery, like (probably can be >>>>>> done easier): >>>>>> >>>>>> for $i at $pos in db:open("DB")//* >>>>>> where $i[text() contains text 'xml'] >>>>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>>>> 'xml'])}</score></hit> >>>>>> >>>>>> -- Andreas >>>>>> >>>>>> Am 03.04.2011 um 12:42 schrieb Wiard Vasen: >>>>>> >>>>>> Dear Christian and Andreas, >>>>>> >>>>>> Thanks for your great help! >>>>>> I used Christians solution: ft:score(db:open("DB")//*[text() contains >>>>>> text 'xml']) >>>>>> And it works fine. >>>>>> >>>>>> The next step is that I want to get the associated documents with >>>>>> these scores. >>>>>> >>>>>> Could you help me with this step? >>>>>> >>>>>> The results I get now is a list with frequencies, without the >>>>>> references to the particular documents. >>>>>> I think what is needed is the tf/idf score. >>>>>> >>>>>> Regards, >>>>>> >>>>>> Wiard >>>>>> >>>>>> >>>>>> 2011/3/31 Christian Grün <christian.gruen at gmail.com> >>>>>> >>>>>>> Hi Wiard, >>>>>>> >>>>>>> the tf/idf scoring is only available if you are working with >>>>>>> full-text >>>>>>> index structures. If you have built a full-text index for your >>>>>>> database "DB", the following query will yield different scoring >>>>>>> results, depending on the chosen scoring model: >>>>>>> >>>>>>> ft:score(db:open("DB")//*[text() contains text 'xml']) >>>>>>> >>>>>>> As Andreas indicated, however, you may either set the SCORING >>>>>>> property >>>>>>> with a db command or explicitly choose the type of scoring in the >>>>>>> GUI's database creation dialog (Database -> New -> Fulltext -> TF/IDF >>>>>>> Scoring). >>>>>>> >>>>>>> Christian >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Thu, Mar 31, 2011 at 8:33 AM, Andreas Weiler >>>>>>> <andreas.weiler at uni-konstanz.de> wrote: >>>>>>> > Dear Wiard Vasen, >>>>>>> > you just need to set the scoring property once. >>>>>>> > If you work in the GUI: >>>>>>> > Go to the top input bar, choose command and type: >>>>>>> > set scoring * >>>>>>> > >>>>>>> > as * set the scoring algorithm you like. >>>>>>> > In the console just type: set scoring * >>>>>>> > After setting this you can use the score function, like in the 8th >>>>>>> query of >>>>>>> > our online demo (basex.org/products/live-demo): >>>>>>> > let $names := ('Jack London', 'Jack', 'Jim Beam', 'Jack Daniels') >>>>>>> > for $name at $pos score $score in $names[. contains text 'Jack'] >>>>>>> > order by $score descending >>>>>>> > return <name pos='{ $pos }'>{ $name }</name> >>>>>>> > Don't hesitate to ask for more, >>>>>>> > Andreas >>>>>>> > Am 30.03.2011 um 22:02 schrieb Wiard Vasen: >>>>>>> > >>>>>>> > Dear sirs of Basex, >>>>>>> > I am doing my Master thesis on the letters of Vincent van Gogh at >>>>>>> the >>>>>>> > University of Amsterdam. >>>>>>> > For that purpose I use BaseX to analyze the letters. >>>>>>> > I wonder, is there the possibility to generate a tf/idf score >>>>>>> automatically? >>>>>>> > In your faq I noticed there needs to be a special term like 'SET >>>>>>> SCORING 0' >>>>>>> > to be able to get a tf/idf score. >>>>>>> > This information I get from the following >>>>>>> > page: http://docs.basex.org/wiki/Full-Text >>>>>>> > Could you help me with this? >>>>>>> > I would be very grateful. >>>>>>> > Kind regards, >>>>>>> > _______________________________________________ >>>>>>> > BaseX-Talk mailing list >>>>>>> > BaseX-Talk at mailman.uni-konstanz.de >>>>>>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>>>>> > >>>>>>> > >>>>>>> > _______________________________________________ >>>>>>> > BaseX-Talk mailing list >>>>>>> > BaseX-Talk at mailman.uni-konstanz.de >>>>>>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>>>>> > >>>>>>> > >>>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>> >>> >> >

> Dear Wiard, > > My question is: Does the scores given back represent tf/idf? >> > > It depends on your full text index; which scoring mode have you > chosen? Next, please have a look at the Query Info (Query -> Query Info) to > get some more insight into the internals. > > Best, > Christian > > >> >> >> Kind regards, >> >> Wiard >> >> >> >> 2011/4/3 Wiard Vasen <wiard.vasen at gmail.com> >> >>> Thank you very much Andreas, >>> >>> You, Christian and Leonard have helped me a lot! >>> >>> Have a nice evening! >>> >>> Regards, >>> >>> Wiard >>> >>> >>> have helped me a lot! >>> >>> >>> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >>> >>>> I did think that you first get the positions of your first and last >>>> document in your range (first query). >>>> Note them and put them into the second query for x and y. >>>> >>>> -- Andreas >>>> >>>> Am 03.04.2011 um 18:44 schrieb Wiard Vasen: >>>> >>>> Hi Andreas, >>>> >>>> Maybe I don't understand the query you suggested. I worked it out this >>>> way: >>>> >>>> for $n at $pos in db:open("tfidfbrievenvangogh") >>>> where $pos > ( "let001.xml") and $pos < ( "let201.xml") >>>> return >>>> for $i in $n//* >>>> where $i[text() contains text 'above'] >>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>> 'above'])}</score></hit> >>>> >>>> I do understand though the error: xs:integer and xs:string can't be >>>> compared >>>> >>>> How do I improve this query, so that it works? >>>> >>>> Regards, >>>> >>>> Wiard >>>> >>>> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >>>> >>>>> Hi Wiard, >>>>> >>>>> you can get the position of your wished first and last document with: >>>>> (add a where clause to get the right documents, like where >>>>> ends-with(base-uri($n), "test.doc")) >>>>> >>>>> for $n at $pos in db:open("tfidfbrievenvangogh") >>>>> return <hit><doc>{base-uri($i)</doc><pos>{$pos}</pos></hit> >>>>> >>>>> then set these position for x and y in the below query. >>>>> >>>>> for $n at $pos in db:open("tfidfbrievenvangogh") >>>>> where $pos > x and $pos < y >>>>> return >>>>> for $i in $n//* >>>>> where $i[text() contains text 'above'] >>>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>>> 'above'])}</score></hit> >>>>> >>>>> I hope this works. >>>>> >>>>> -- Andreas >>>>> >>>>> Am 03.04.2011 um 18:14 schrieb Wiard Vasen: >>>>> >>>>> Hi Andreas, >>>>> >>>>> Thanks a lot! It works fine. >>>>> >>>>> I was wondering if instead of putting in the next query in BaseX: >>>>> >>>>> for $n in ("let680.xml", "let681.xml", "let682.xml", "let683.xml", >>>>> "let684.xml", "let685.xml", "let686.xml", "let687.xml", "let688.xml", >>>>> "let689.xml") >>>>> return >>>>> for $i at $pos in db:open("tfidfbrievenvangogh")//* >>>>> where ends-with(base-uri($i), $n) >>>>> and $i[text() contains text 'above'] >>>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>>> 'above'])}</score></hit> >>>>> >>>>> It is also possible to do something like: >>>>> >>>>> for ("let680.xml" )<= $n <= ("let689") >>>>> return >>>>> for $i at $pos in db:open("tfidfbrievenvangogh")//* >>>>> where ends-with(base-uri($i), $n) >>>>> and $i[text() contains text 'above'] >>>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>>> 'above'])}</score></hit> >>>>> >>>>> That way I hope to define the outer documents of a subset and get all >>>>> the documents in between, with the outer documents included. >>>>> >>>>> Do you think this is possible in a query like shown above? >>>>> >>>>> Regards, >>>>> >>>>> Wiard >>>>> >>>>> >>>>> >>>>> >>>>> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >>>>> >>>>>> Hi Wiard, >>>>>> >>>>>> i hope i understand your plans, here is what i would do: >>>>>> >>>>>> for $n in ("betweenlet567.xml", "let689.xml") >>>>>> return >>>>>> for $i at $pos in db:open("tfidfbrievenvangogh")//* >>>>>> where ends-with(base-uri($i), $n) >>>>>> and $i[text() contains text 'kleur'] >>>>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>>>> 'kleur'])}</score></hit> >>>>>> >>>>>> Now you can extend the variable $n with all filenames you like to >>>>>> have. >>>>>> >>>>>> I hope this helps, >>>>>> Andreas >>>>>> >>>>>> Am 03.04.2011 um 14:24 schrieb Wiard Vasen: >>>>>> >>>>>> Hi Andreas, >>>>>> >>>>>> Wow! This is the complete answer to my question! >>>>>> >>>>>> I hope you can help me with the next question. >>>>>> Because I am analyzing changes in the artistic life of Van Gogh, I am >>>>>> partitioning the relatively large repository annotated xml files on the >>>>>> basis of residence. >>>>>> >>>>>> For that reason I need to put a query like: >>>>>> >>>>>> for $i at $pos in db:open("tfidfbrievenvangogh")//* >>>>>> where $i[text() contains text 'kleur'] >>>>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>>>> 'kleur'])}</score></hit> >>>>>> >>>>>> with the extension: given the interval, all xml-files >>>>>> betweenlet567.xml and let689.xml. >>>>>> What means that I know that in this partition xml-files Van Gogh was >>>>>> in Arles. >>>>>> And I want to know what is the tf-idf score of the dutch word 'kleur'. >>>>>> >>>>>> To give a resume of my question: How do I partition the repository in >>>>>> subsets, so that I can produce information on these subsets. >>>>>> And how do I do this in BasX with xquery. >>>>>> >>>>>> Thanks a lot beforehand! >>>>>> >>>>>> Kind regards, >>>>>> >>>>>> Wiard >>>>>> >>>>>> >>>>>> 2011/4/3 Andreas Weiler <andreas.weiler at uni-konstanz.de> >>>>>> >>>>>>> Hi Wiard, >>>>>>> >>>>>>> you could use the base-uri function of XQuery, like (probably can be >>>>>>> done easier): >>>>>>> >>>>>>> for $i at $pos in db:open("DB")//* >>>>>>> where $i[text() contains text 'xml'] >>>>>>> return <hit>{base-uri($i)}<score>{ft:score($i[text() contains text >>>>>>> 'xml'])}</score></hit> >>>>>>> >>>>>>> -- Andreas >>>>>>> >>>>>>> Am 03.04.2011 um 12:42 schrieb Wiard Vasen: >>>>>>> >>>>>>> Dear Christian and Andreas, >>>>>>> >>>>>>> Thanks for your great help! >>>>>>> I used Christians solution: ft:score(db:open("DB")//*[text() >>>>>>> contains text 'xml']) >>>>>>> And it works fine. >>>>>>> >>>>>>> The next step is that I want to get the associated documents with >>>>>>> these scores. >>>>>>> >>>>>>> Could you help me with this step? >>>>>>> >>>>>>> The results I get now is a list with frequencies, without the >>>>>>> references to the particular documents. >>>>>>> I think what is needed is the tf/idf score. >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Wiard >>>>>>> >>>>>>> >>>>>>> 2011/3/31 Christian Grün <christian.gruen at gmail.com> >>>>>>> >>>>>>>> Hi Wiard, >>>>>>>> >>>>>>>> the tf/idf scoring is only available if you are working with >>>>>>>> full-text >>>>>>>> index structures. If you have built a full-text index for your >>>>>>>> database "DB", the following query will yield different scoring >>>>>>>> results, depending on the chosen scoring model: >>>>>>>> >>>>>>>> ft:score(db:open("DB")//*[text() contains text 'xml']) >>>>>>>> >>>>>>>> As Andreas indicated, however, you may either set the SCORING >>>>>>>> property >>>>>>>> with a db command or explicitly choose the type of scoring in the >>>>>>>> GUI's database creation dialog (Database -> New -> Fulltext -> >>>>>>>> TF/IDF >>>>>>>> Scoring). >>>>>>>> >>>>>>>> Christian >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Mar 31, 2011 at 8:33 AM, Andreas Weiler >>>>>>>> <andreas.weiler at uni-konstanz.de> wrote: >>>>>>>> > Dear Wiard Vasen, >>>>>>>> > you just need to set the scoring property once. >>>>>>>> > If you work in the GUI: >>>>>>>> > Go to the top input bar, choose command and type: >>>>>>>> > set scoring * >>>>>>>> > >>>>>>>> > as * set the scoring algorithm you like. >>>>>>>> > In the console just type: set scoring * >>>>>>>> > After setting this you can use the score function, like in the 8th >>>>>>>> query of >>>>>>>> > our online demo (basex.org/products/live-demo): >>>>>>>> > let $names := ('Jack London', 'Jack', 'Jim Beam', 'Jack Daniels') >>>>>>>> > for $name at $pos score $score in $names[. contains text 'Jack'] >>>>>>>> > order by $score descending >>>>>>>> > return <name pos='{ $pos }'>{ $name }</name> >>>>>>>> > Don't hesitate to ask for more, >>>>>>>> > Andreas >>>>>>>> > Am 30.03.2011 um 22:02 schrieb Wiard Vasen: >>>>>>>> > >>>>>>>> > Dear sirs of Basex, >>>>>>>> > I am doing my Master thesis on the letters of Vincent van Gogh at >>>>>>>> the >>>>>>>> > University of Amsterdam. >>>>>>>> > For that purpose I use BaseX to analyze the letters. >>>>>>>> > I wonder, is there the possibility to generate a tf/idf score >>>>>>>> automatically? >>>>>>>> > In your faq I noticed there needs to be a special term like 'SET >>>>>>>> SCORING 0' >>>>>>>> > to be able to get a tf/idf score. >>>>>>>> > This information I get from the following >>>>>>>> > page: http://docs.basex.org/wiki/Full-Text >>>>>>>> > Could you help me with this? >>>>>>>> > I would be very grateful. >>>>>>>> > Kind regards, >>>>>>>> > _______________________________________________ >>>>>>>> > BaseX-Talk mailing list >>>>>>>> > BaseX-Talk at mailman.uni-konstanz.de >>>>>>>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>>>>>> > >>>>>>>> > >>>>>>>> > _______________________________________________ >>>>>>>> > BaseX-Talk mailing list >>>>>>>> > BaseX-Talk at mailman.uni-konstanz.de >>>>>>>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>>>>>> > >>>>>>>> > >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>> >> >

On Tue, Apr 5, 2011 at 3:04 PM, Wiard Vasen <wiard.vasen at gmail.com> wrote: > Dear Christian, > > When I initialized the database I marked in 'Full Text' properties the TF / > IDF checkbox. > So, I think that 'score' in the query gives this score back. > > Do you think I am right? > > Thanks in advance for your answer. > > Kind regards, > > Wiard > > >

> Dear Wiard, > > what does the query info tell you? Just copy&paste the info to this list. > > Thanks > Christian > ___________________________ > > > On Tue, Apr 5, 2011 at 3:04 PM, Wiard Vasen <wiard.vasen at gmail.com> wrote: > >> Dear Christian, >> >> When I initialized the database I marked in 'Full Text' properties the TF >> / IDF checkbox. >> So, I think that 'score' in the query gives this score back. >> >> Do you think I am right? >> >> Thanks in advance for your answer. >> >> Kind regards, >> >> Wiard >> >> >>

On Tue, Apr 5, 2011 at 4:52 PM, Wiard Vasen <wiard.vasen at gmail.com> wrote: > Hi Christian, > This is the result of the query: > <document uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let001.xml"/> > <document uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let002.xml"/> > <document uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let003.xml"/> > <document uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let004.xml"/> > <document uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let005.xml"> > <hit score="0.03590675482297878"> > <ab xmlns="http://www.tei-c.org/ns/1.0" rend="indent">How is your > boarding-house? Is it still to your liking? That’s important. Above all, you > must write more about the kind of things you see. Sunday a fortnight ago I > was in Amsterdam to see an exhibition of the paintings going to Vienna from > here.<anchor n="6" xml:id="note-t-6"/>It was very interesting, and I’m > curious<pb f="1r" n="4" xml:id="pb-trans-1r-4" facs="#zone-pb-1r-4"/>as to > the impression the Dutch will make in Vienna.</ab> > </hit> > </document> > And this is the query: > let $range := 1 to 5 > for $doc in collection('christian') > let $uri := base-uri($doc), > $num := substring($uri, string-length($uri) - 6, 3) > where $num castable as xs:integer > and xs:integer($num) = $range > return <document uri='{$uri}'>{ > for $n score $s in $doc//*[text() contains text 'above'] > return <hit score='{$s}'>{ $n }</hit> > }</document> > Kind regards, > Wiard > 2011/4/5 Christian Grün <christian.gruen at gmail.com> >> >> Dear Wiard, >> what does the query info tell you? Just copy&paste the info to this list. >> Thanks >> Christian >> ___________________________ >> >> On Tue, Apr 5, 2011 at 3:04 PM, Wiard Vasen <wiard.vasen at gmail.com> wrote: >>> >>> Dear Christian, >>> When I initialized the database I marked in 'Full Text' properties the TF >>> / IDF checkbox. >>> So, I think that 'score' in the query gives this score back. >>> Do you think I am right? >>> Thanks in advance for your answer. >>> Kind regards, >>> Wiard >>> > >

> Hi Wiard, > > looks like you sent me the query result. To tell you if the index was > utilized, I need the output from the »Query Info« panel, or (if that > won't help) the original data instances. > > Christian > > > On Tue, Apr 5, 2011 at 4:52 PM, Wiard Vasen <wiard.vasen at gmail.com> wrote: > > Hi Christian, > > This is the result of the query: > > <document > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let001.xml"/> > > <document > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let002.xml"/> > > <document > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let003.xml"/> > > <document > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let004.xml"/> > > <document uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let005.xml"> > > <hit score="0.03590675482297878"> > > <ab xmlns="http://www.tei-c.org/ns/1.0" rend="indent">How is your > > boarding-house? Is it still to your liking? That’s important. Above all, > you > > must write more about the kind of things you see. Sunday a fortnight ago > I > > was in Amsterdam to see an exhibition of the paintings going to Vienna > from > > here.<anchor n="6" xml:id="note-t-6"/>It was very interesting, and I’m > > curious<pb f="1r" n="4" xml:id="pb-trans-1r-4" facs="#zone-pb-1r-4"/>as > to > > the impression the Dutch will make in Vienna.</ab> > > </hit> > > </document> > > And this is the query: > > let $range := 1 to 5 > > for $doc in collection('christian') > > let $uri := base-uri($doc), > > $num := substring($uri, string-length($uri) - 6, 3) > > where $num castable as xs:integer > > and xs:integer($num) = $range > > return <document uri='{$uri}'>{ > > for $n score $s in $doc//*[text() contains text 'above'] > > return <hit score='{$s}'>{ $n }</hit> > > }</document> > > Kind regards, > > Wiard > > 2011/4/5 Christian Grün <christian.gruen at gmail.com> > >> > >> Dear Wiard, > >> what does the query info tell you? Just copy&paste the info to this > list. > >> Thanks > >> Christian > >> ___________________________ > >> > >> On Tue, Apr 5, 2011 at 3:04 PM, Wiard Vasen <wiard.vasen at gmail.com> > wrote: > >>> > >>> Dear Christian, > >>> When I initialized the database I marked in 'Full Text' properties the > TF > >>> / IDF checkbox. > >>> So, I think that 'score' in the query gives this score back. > >>> Do you think I am right? > >>> Thanks in advance for your answer. > >>> Kind regards, > >>> Wiard > >>> > > > > >

> This is the information from the Query Info panel: > Compiling: > - binding static variable $range > - pre-evaluating collection("christian") > - optimizing descendant-or-self step(s) > - removing variable $range

> Result: for $doc in (document-node { "let001.xml" }, document-node { > "let002.xml" }, ...) let $uri := base-uri($doc) let $num := substring($uri, > string-length($uri) - 6, 3) where $num castable as xs:integer and $num cast > as xs:integer? = 1 to 5 return element { "document" } { attribute { "uri" } > { $uri }, for $n score $s as xs:double in $doc/descendant::*[text() contains > text "above"] return element { "hit" } { attribute { "score" } { $s }, $n } > } > Timing: > - Parsing: 0.88 ms > - Compiling: 5.71 ms > - Evaluating: 93.72 ms > - Printing: 0.27 ms > - Total Time: 100.6 ms > Query plan: > <FLWR> > <For var="$doc"> > <sequence size="927"> > <document-node() name="christian"/> > <document-node() name="christian" pre="480"/> > <document-node() name="christian" pre="913"/> > <document-node() name="christian" pre="1897"/> > <document-node() name="christian" pre="2928"/> > </sequence> > </For> > <Let var="$uri"> > <FNNode name="base-uri([node])"> > <VarRef name="$doc"/> > </FNNode> > </Let> > <Let var="$num"> > <FNStr name="substring(string,start[,len])"> > <VarRef name="$uri"/> > <Arith op="-"> > <FNAcc name="string-length([item])"> > <VarRef name="$uri"/> > </FNAcc> > <Item value="6" type="xs:integer"/> > </Arith> > <Item value="3" type="xs:integer"/> > </FNStr> > </Let> > <Where> > <And> > <Castable type="xs:integer"> > <VarRef name="$num"/> > </Castable> > <CmpG op="="> > <Cast type="xs:integer?"> > <VarRef name="$num"/> > </Cast> > <Range> > <Item value="1" type="xs:integer"/> > <Item value="5" type="xs:integer"/> > </Range> > </CmpG> > </And> > </Where> > <Return> > <CElem> > <Item value="document" type="xs:QName"/> > <CAttr> > <Item value="uri" type="xs:QName"/> > <VarRef name="$uri"/> > </CAttr> > <FLWR> > <For var="$n" score="$s as xs:double"> > <AxisPath> > <VarRef name="$doc"/> > <IterStep axis="descendant" test="*"> > <FTContains> > <AxisPath> > <IterStep axis="child" test="text()"/> > </AxisPath> > <FTWords> > <Item value="above" type="xs:string"/> > </FTWords> > </FTContains> > </IterStep> > </AxisPath> > </For> > <Return> > <CElem> > <Item value="hit" type="xs:QName"/> > <CAttr> > <Item value="score" type="xs:QName"/> > <VarRef name="$s as xs:double"/> > </CAttr> > <VarRef name="$n"/> > </CElem> > </Return> > </FLWR> > </CElem> > </Return> > </FLWR> > Thanks! > Regards, > Wiard > > 2011/4/5 Christian Grün <christian.gruen at gmail.com> >> >> Hi Wiard, >> >> looks like you sent me the query result. To tell you if the index was >> utilized, I need the output from the »Query Info« panel, or (if that >> won't help) the original data instances. >> >> Christian >> >> >> On Tue, Apr 5, 2011 at 4:52 PM, Wiard Vasen <wiard.vasen at gmail.com> wrote: >> > Hi Christian, >> > This is the result of the query: >> > <document >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let001.xml"/> >> > <document >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let002.xml"/> >> > <document >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let003.xml"/> >> > <document >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let004.xml"/> >> > <document >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let005.xml"> >> > <hit score="0.03590675482297878"> >> > <ab xmlns="http://www.tei-c.org/ns/1.0" rend="indent">How is your >> > boarding-house? Is it still to your liking? That’s important. Above all, >> > you >> > must write more about the kind of things you see. Sunday a fortnight ago >> > I >> > was in Amsterdam to see an exhibition of the paintings going to Vienna >> > from >> > here.<anchor n="6" xml:id="note-t-6"/>It was very interesting, and I’m >> > curious<pb f="1r" n="4" xml:id="pb-trans-1r-4" facs="#zone-pb-1r-4"/>as >> > to >> > the impression the Dutch will make in Vienna.</ab> >> > </hit> >> > </document> >> > And this is the query: >> > let $range := 1 to 5 >> > for $doc in collection('christian') >> > let $uri := base-uri($doc), >> > $num := substring($uri, string-length($uri) - 6, 3) >> > where $num castable as xs:integer >> > and xs:integer($num) = $range >> > return <document uri='{$uri}'>{ >> > for $n score $s in $doc//*[text() contains text 'above'] >> > return <hit score='{$s}'>{ $n }</hit> >> > }</document> >> > Kind regards, >> > Wiard >> > 2011/4/5 Christian Grün <christian.gruen at gmail.com> >> >> >> >> Dear Wiard, >> >> what does the query info tell you? Just copy&paste the info to this >> >> list. >> >> Thanks >> >> Christian >> >> ___________________________ >> >> >> >> On Tue, Apr 5, 2011 at 3:04 PM, Wiard Vasen <wiard.vasen at gmail.com> >> >> wrote: >> >>> >> >>> Dear Christian, >> >>> When I initialized the database I marked in 'Full Text' properties the >> >>> TF >> >>> / IDF checkbox. >> >>> So, I think that 'score' in the query gives this score back. >> >>> Do you think I am right? >> >>> Thanks in advance for your answer. >> >>> Kind regards, >> >>> Wiard >> >>> >> > >> > > >

> > This is the information from the Query Info panel: > > Compiling: > > - binding static variable $range > > - pre-evaluating collection("christian") > > - optimizing descendant-or-self step(s) > > - removing variable $range > > Your query seems to be too complex to be evaluated via the full-text > index (probably because the nested flwor expression); otherwise, the > query info would contain the following line at least once: > > - applying full-text index > > If you don't want to spend too much time into rewriting your query, > you might as well access the index directly, such as: > > for $d in collection('coll') > for $x in ft:search($d, 'text') > where $x/ancestor::node()[. = $d] > return ft:score($x) > > Hope this helps, > Christian > > > > > > > Result: for $doc in (document-node { "let001.xml" }, document-node { > > "let002.xml" }, ...) let $uri := base-uri($doc) let $num := > substring($uri, > > string-length($uri) - 6, 3) where $num castable as xs:integer and $num > cast > > as xs:integer? = 1 to 5 return element { "document" } { attribute { "uri" > } > > { $uri }, for $n score $s as xs:double in $doc/descendant::*[text() > contains > > text "above"] return element { "hit" } { attribute { "score" } { $s }, $n > } > > } > > Timing: > > - Parsing: 0.88 ms > > - Compiling: 5.71 ms > > - Evaluating: 93.72 ms > > - Printing: 0.27 ms > > - Total Time: 100.6 ms > > Query plan: > > <FLWR> > > <For var="$doc"> > > <sequence size="927"> > > <document-node() name="christian"/> > > <document-node() name="christian" pre="480"/> > > <document-node() name="christian" pre="913"/> > > <document-node() name="christian" pre="1897"/> > > <document-node() name="christian" pre="2928"/> > > </sequence> > > </For> > > <Let var="$uri"> > > <FNNode name="base-uri([node])"> > > <VarRef name="$doc"/> > > </FNNode> > > </Let> > > <Let var="$num"> > > <FNStr name="substring(string,start[,len])"> > > <VarRef name="$uri"/> > > <Arith op="-"> > > <FNAcc name="string-length([item])"> > > <VarRef name="$uri"/> > > </FNAcc> > > <Item value="6" type="xs:integer"/> > > </Arith> > > <Item value="3" type="xs:integer"/> > > </FNStr> > > </Let> > > <Where> > > <And> > > <Castable type="xs:integer"> > > <VarRef name="$num"/> > > </Castable> > > <CmpG op="="> > > <Cast type="xs:integer?"> > > <VarRef name="$num"/> > > </Cast> > > <Range> > > <Item value="1" type="xs:integer"/> > > <Item value="5" type="xs:integer"/> > > </Range> > > </CmpG> > > </And> > > </Where> > > <Return> > > <CElem> > > <Item value="document" type="xs:QName"/> > > <CAttr> > > <Item value="uri" type="xs:QName"/> > > <VarRef name="$uri"/> > > </CAttr> > > <FLWR> > > <For var="$n" score="$s as xs:double"> > > <AxisPath> > > <VarRef name="$doc"/> > > <IterStep axis="descendant" test="*"> > > <FTContains> > > <AxisPath> > > <IterStep axis="child" test="text()"/> > > </AxisPath> > > <FTWords> > > <Item value="above" type="xs:string"/> > > </FTWords> > > </FTContains> > > </IterStep> > > </AxisPath> > > </For> > > <Return> > > <CElem> > > <Item value="hit" type="xs:QName"/> > > <CAttr> > > <Item value="score" type="xs:QName"/> > > <VarRef name="$s as xs:double"/> > > </CAttr> > > <VarRef name="$n"/> > > </CElem> > > </Return> > > </FLWR> > > </CElem> > > </Return> > > </FLWR> > > Thanks! > > Regards, > > Wiard > > > > 2011/4/5 Christian Grün <christian.gruen at gmail.com> > >> > >> Hi Wiard, > >> > >> looks like you sent me the query result. To tell you if the index was > >> utilized, I need the output from the »Query Info« panel, or (if that > >> won't help) the original data instances. > >> > >> Christian > >> > >> > >> On Tue, Apr 5, 2011 at 4:52 PM, Wiard Vasen <wiard.vasen at gmail.com> > wrote: > >> > Hi Christian, > >> > This is the result of the query: > >> > <document > >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let001.xml"/> > >> > <document > >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let002.xml"/> > >> > <document > >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let003.xml"/> > >> > <document > >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let004.xml"/> > >> > <document > >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let005.xml"> > >> > <hit score="0.03590675482297878"> > >> > <ab xmlns="http://www.tei-c.org/ns/1.0" rend="indent">How is your > >> > boarding-house? Is it still to your liking? That’s important. Above > all, > >> > you > >> > must write more about the kind of things you see. Sunday a fortnight > ago > >> > I > >> > was in Amsterdam to see an exhibition of the paintings going to Vienna > >> > from > >> > here.<anchor n="6" xml:id="note-t-6"/>It was very interesting, and I’m > >> > curious<pb f="1r" n="4" xml:id="pb-trans-1r-4" > facs="#zone-pb-1r-4"/>as > >> > to > >> > the impression the Dutch will make in Vienna.</ab> > >> > </hit> > >> > </document> > >> > And this is the query: > >> > let $range := 1 to 5 > >> > for $doc in collection('christian') > >> > let $uri := base-uri($doc), > >> > $num := substring($uri, string-length($uri) - 6, 3) > >> > where $num castable as xs:integer > >> > and xs:integer($num) = $range > >> > return <document uri='{$uri}'>{ > >> > for $n score $s in $doc//*[text() contains text 'above'] > >> > return <hit score='{$s}'>{ $n }</hit> > >> > }</document> > >> > Kind regards, > >> > Wiard > >> > 2011/4/5 Christian Grün <christian.gruen at gmail.com> > >> >> > >> >> Dear Wiard, > >> >> what does the query info tell you? Just copy&paste the info to this > >> >> list. > >> >> Thanks > >> >> Christian > >> >> ___________________________ > >> >> > >> >> On Tue, Apr 5, 2011 at 3:04 PM, Wiard Vasen <wiard.vasen at gmail.com> > >> >> wrote: > >> >>> > >> >>> Dear Christian, > >> >>> When I initialized the database I marked in 'Full Text' properties > the > >> >>> TF > >> >>> / IDF checkbox. > >> >>> So, I think that 'score' in the query gives this score back. > >> >>> Do you think I am right? > >> >>> Thanks in advance for your answer. > >> >>> Kind regards, > >> >>> Wiard > >> >>> > >> > > >> > > > > > >

Am 05.04.2011 um 20:35 schrieb Wiard Vasen: > Hi Christian, > > This query gives good results on tf-scores: > > ft:score(db:open("tfidfbrievenvangogh")//*[text() contains text 'man']) > > But the problem is that I need the specific documents connected with the given scores. > > For that reason I thought that the following query: > > let $range := 1 to 640 > for $doc in collection('tfidfbrievenvangogh') > let $uri := base-uri($doc), > $num := substring($uri, string-length($uri) - 6, 3) > where $num castable as xs:integer > and xs:integer($num) = $range > return <document uri='{$uri}'>{ > for $n score $s in $doc//*[text() contains text 'above'] > return <hit score='{$s}'>{ $n }</hit> > }</document> > > would automatically give the tf/idf score because here 'score' is a reserved word and tf/idf where checked while initializing the full-text repository. > > I am not sure whether this thought is right. > > Maybe you know the answer. > > Kind regards, > > Wiard > > 2011/4/5 Christian Grün <christian.gruen at gmail.com> > > This is the information from the Query Info panel: > > Compiling: > > - binding static variable $range > > - pre-evaluating collection("christian") > > - optimizing descendant-or-self step(s) > > - removing variable $range > > Your query seems to be too complex to be evaluated via the full-text > index (probably because the nested flwor expression); otherwise, the > query info would contain the following line at least once: > > - applying full-text index > > If you don't want to spend too much time into rewriting your query, > you might as well access the index directly, such as: > > for $d in collection('coll') > for $x in ft:search($d, 'text') > where $x/ancestor::node()[. = $d] > return ft:score($x) > > Hope this helps, > Christian > > > > > > > Result: for $doc in (document-node { "let001.xml" }, document-node { > > "let002.xml" }, ...) let $uri := base-uri($doc) let $num := substring($uri, > > string-length($uri) - 6, 3) where $num castable as xs:integer and $num cast > > as xs:integer? = 1 to 5 return element { "document" } { attribute { "uri" } > > { $uri }, for $n score $s as xs:double in $doc/descendant::*[text() contains > > text "above"] return element { "hit" } { attribute { "score" } { $s }, $n } > > } > > Timing: > > - Parsing: 0.88 ms > > - Compiling: 5.71 ms > > - Evaluating: 93.72 ms > > - Printing: 0.27 ms > > - Total Time: 100.6 ms > > Query plan: > > <FLWR> > > <For var="$doc"> > > <sequence size="927"> > > <document-node() name="christian"/> > > <document-node() name="christian" pre="480"/> > > <document-node() name="christian" pre="913"/> > > <document-node() name="christian" pre="1897"/> > > <document-node() name="christian" pre="2928"/> > > </sequence> > > </For> > > <Let var="$uri"> > > <FNNode name="base-uri([node])"> > > <VarRef name="$doc"/> > > </FNNode> > > </Let> > > <Let var="$num"> > > <FNStr name="substring(string,start[,len])"> > > <VarRef name="$uri"/> > > <Arith op="-"> > > <FNAcc name="string-length([item])"> > > <VarRef name="$uri"/> > > </FNAcc> > > <Item value="6" type="xs:integer"/> > > </Arith> > > <Item value="3" type="xs:integer"/> > > </FNStr> > > </Let> > > <Where> > > <And> > > <Castable type="xs:integer"> > > <VarRef name="$num"/> > > </Castable> > > <CmpG op="="> > > <Cast type="xs:integer?"> > > <VarRef name="$num"/> > > </Cast> > > <Range> > > <Item value="1" type="xs:integer"/> > > <Item value="5" type="xs:integer"/> > > </Range> > > </CmpG> > > </And> > > </Where> > > <Return> > > <CElem> > > <Item value="document" type="xs:QName"/> > > <CAttr> > > <Item value="uri" type="xs:QName"/> > > <VarRef name="$uri"/> > > </CAttr> > > <FLWR> > > <For var="$n" score="$s as xs:double"> > > <AxisPath> > > <VarRef name="$doc"/> > > <IterStep axis="descendant" test="*"> > > <FTContains> > > <AxisPath> > > <IterStep axis="child" test="text()"/> > > </AxisPath> > > <FTWords> > > <Item value="above" type="xs:string"/> > > </FTWords> > > </FTContains> > > </IterStep> > > </AxisPath> > > </For> > > <Return> > > <CElem> > > <Item value="hit" type="xs:QName"/> > > <CAttr> > > <Item value="score" type="xs:QName"/> > > <VarRef name="$s as xs:double"/> > > </CAttr> > > <VarRef name="$n"/> > > </CElem> > > </Return> > > </FLWR> > > </CElem> > > </Return> > > </FLWR> > > Thanks! > > Regards, > > Wiard > > > > 2011/4/5 Christian Grün <christian.gruen at gmail.com> > >> > >> Hi Wiard, > >> > >> looks like you sent me the query result. To tell you if the index was > >> utilized, I need the output from the »Query Info« panel, or (if that > >> won't help) the original data instances. > >> > >> Christian > >> > >> > >> On Tue, Apr 5, 2011 at 4:52 PM, Wiard Vasen <wiard.vasen at gmail.com> wrote: > >> > Hi Christian, > >> > This is the result of the query: > >> > <document > >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let001.xml"/> > >> > <document > >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let002.xml"/> > >> > <document > >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let003.xml"/> > >> > <document > >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let004.xml"/> > >> > <document > >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let005.xml"> > >> > <hit score="0.03590675482297878"> > >> > <ab xmlns="http://www.tei-c.org/ns/1.0" rend="indent">How is your > >> > boarding-house? Is it still to your liking? That’s important. Above all, > >> > you > >> > must write more about the kind of things you see. Sunday a fortnight ago > >> > I > >> > was in Amsterdam to see an exhibition of the paintings going to Vienna > >> > from > >> > here.<anchor n="6" xml:id="note-t-6"/>It was very interesting, and I’m > >> > curious<pb f="1r" n="4" xml:id="pb-trans-1r-4" facs="#zone-pb-1r-4"/>as > >> > to > >> > the impression the Dutch will make in Vienna.</ab> > >> > </hit> > >> > </document> > >> > And this is the query: > >> > let $range := 1 to 5 > >> > for $doc in collection('christian') > >> > let $uri := base-uri($doc), > >> > $num := substring($uri, string-length($uri) - 6, 3) > >> > where $num castable as xs:integer > >> > and xs:integer($num) = $range > >> > return <document uri='{$uri}'>{ > >> > for $n score $s in $doc//*[text() contains text 'above'] > >> > return <hit score='{$s}'>{ $n }</hit> > >> > }</document> > >> > Kind regards, > >> > Wiard > >> > 2011/4/5 Christian Grün <christian.gruen at gmail.com> > >> >> > >> >> Dear Wiard, > >> >> what does the query info tell you? Just copy&paste the info to this > >> >> list. > >> >> Thanks > >> >> Christian > >> >> ___________________________ > >> >> > >> >> On Tue, Apr 5, 2011 at 3:04 PM, Wiard Vasen <wiard.vasen at gmail.com> > >> >> wrote: > >> >>> > >> >>> Dear Christian, > >> >>> When I initialized the database I marked in 'Full Text' properties the > >> >>> TF > >> >>> / IDF checkbox. > >> >>> So, I think that 'score' in the query gives this score back. > >> >>> Do you think I am right? > >> >>> Thanks in advance for your answer. > >> >>> Kind regards, > >> >>> Wiard > >> >>> > >> > > >> > > > > > >

> Hi Wiard, > > try the following: > > let $range := 1 to 640 > for $doc in collection('tfidfbrievenvangogh') > let $uri := base-uri($doc), > $num := substring($uri, string-length($uri) - 6, 3) > where $num castable as xs:integer > and xs:integer($num) = $range > return <document uri='{$uri}'>{ > for $n in $doc//* > where $n contains text 'above' > return <hit score='{ft:score($n[text() contains text 'above'])}'>{ $n > }</hit> > }</document> > > -- Andreas > > Am 05.04.2011 um 20:35 schrieb Wiard Vasen: > > Hi Christian, > > This query gives good results on tf-scores: > > ft:score(db:open("tfidfbrievenvangogh")//*[text() contains text 'man']) > > But the problem is that I need the specific documents connected with the > given scores. > > For that reason I thought that the following query: > > let $range := 1 to 640 > for $doc in collection('tfidfbrievenvangogh') > let $uri := base-uri($doc), > $num := substring($uri, string-length($uri) - 6, 3) > where $num castable as xs:integer > and xs:integer($num) = $range > return <document uri='{$uri}'>{ > for $n score $s in $doc//*[text() contains text 'above'] > return <hit score='{$s}'>{ $n }</hit> > }</document> > > would automatically give the tf/idf score because here 'score' is a > reserved word and tf/idf where checked while initializing the full-text > repository. > > I am not sure whether this thought is right. > > Maybe you know the answer. > > Kind regards, > > Wiard > > 2011/4/5 Christian Grün <christian.gruen at gmail.com> > >> > This is the information from the Query Info panel: >> > Compiling: >> > - binding static variable $range >> > - pre-evaluating collection("christian") >> > - optimizing descendant-or-self step(s) >> > - removing variable $range >> >> Your query seems to be too complex to be evaluated via the full-text >> index (probably because the nested flwor expression); otherwise, the >> query info would contain the following line at least once: >> >> - applying full-text index >> >> If you don't want to spend too much time into rewriting your query, >> you might as well access the index directly, such as: >> >> for $d in collection('coll') >> for $x in ft:search($d, 'text') >> where $x/ancestor::node()[. = $d] >> return ft:score($x) >> >> Hope this helps, >> Christian >> >> >> >> >> >> > Result: for $doc in (document-node { "let001.xml" }, document-node { >> > "let002.xml" }, ...) let $uri := base-uri($doc) let $num := >> substring($uri, >> > string-length($uri) - 6, 3) where $num castable as xs:integer and $num >> cast >> > as xs:integer? = 1 to 5 return element { "document" } { attribute { >> "uri" } >> > { $uri }, for $n score $s as xs:double in $doc/descendant::*[text() >> contains >> > text "above"] return element { "hit" } { attribute { "score" } { $s }, >> $n } >> > } >> > Timing: >> > - Parsing: 0.88 ms >> > - Compiling: 5.71 ms >> > - Evaluating: 93.72 ms >> > - Printing: 0.27 ms >> > - Total Time: 100.6 ms >> > Query plan: >> > <FLWR> >> > <For var="$doc"> >> > <sequence size="927"> >> > <document-node() name="christian"/> >> > <document-node() name="christian" pre="480"/> >> > <document-node() name="christian" pre="913"/> >> > <document-node() name="christian" pre="1897"/> >> > <document-node() name="christian" pre="2928"/> >> > </sequence> >> > </For> >> > <Let var="$uri"> >> > <FNNode name="base-uri([node])"> >> > <VarRef name="$doc"/> >> > </FNNode> >> > </Let> >> > <Let var="$num"> >> > <FNStr name="substring(string,start[,len])"> >> > <VarRef name="$uri"/> >> > <Arith op="-"> >> > <FNAcc name="string-length([item])"> >> > <VarRef name="$uri"/> >> > </FNAcc> >> > <Item value="6" type="xs:integer"/> >> > </Arith> >> > <Item value="3" type="xs:integer"/> >> > </FNStr> >> > </Let> >> > <Where> >> > <And> >> > <Castable type="xs:integer"> >> > <VarRef name="$num"/> >> > </Castable> >> > <CmpG op="="> >> > <Cast type="xs:integer?"> >> > <VarRef name="$num"/> >> > </Cast> >> > <Range> >> > <Item value="1" type="xs:integer"/> >> > <Item value="5" type="xs:integer"/> >> > </Range> >> > </CmpG> >> > </And> >> > </Where> >> > <Return> >> > <CElem> >> > <Item value="document" type="xs:QName"/> >> > <CAttr> >> > <Item value="uri" type="xs:QName"/> >> > <VarRef name="$uri"/> >> > </CAttr> >> > <FLWR> >> > <For var="$n" score="$s as xs:double"> >> > <AxisPath> >> > <VarRef name="$doc"/> >> > <IterStep axis="descendant" test="*"> >> > <FTContains> >> > <AxisPath> >> > <IterStep axis="child" test="text()"/> >> > </AxisPath> >> > <FTWords> >> > <Item value="above" type="xs:string"/> >> > </FTWords> >> > </FTContains> >> > </IterStep> >> > </AxisPath> >> > </For> >> > <Return> >> > <CElem> >> > <Item value="hit" type="xs:QName"/> >> > <CAttr> >> > <Item value="score" type="xs:QName"/> >> > <VarRef name="$s as xs:double"/> >> > </CAttr> >> > <VarRef name="$n"/> >> > </CElem> >> > </Return> >> > </FLWR> >> > </CElem> >> > </Return> >> > </FLWR> >> > Thanks! >> > Regards, >> > Wiard >> > >> > 2011/4/5 Christian Grün <christian.gruen at gmail.com> >> >> >> >> Hi Wiard, >> >> >> >> looks like you sent me the query result. To tell you if the index was >> >> utilized, I need the output from the »Query Info« panel, or (if that >> >> won't help) the original data instances. >> >> >> >> Christian >> >> >> >> >> >> On Tue, Apr 5, 2011 at 4:52 PM, Wiard Vasen <wiard.vasen at gmail.com> >> wrote: >> >> > Hi Christian, >> >> > This is the result of the query: >> >> > <document >> >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let001.xml"/> >> >> > <document >> >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let002.xml"/> >> >> > <document >> >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let003.xml"/> >> >> > <document >> >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let004.xml"/> >> >> > <document >> >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let005.xml"> >> >> > <hit score="0.03590675482297878"> >> >> > <ab xmlns="http://www.tei-c.org/ns/1.0" rend="indent">How is >> your >> >> > boarding-house? Is it still to your liking? That’s important. Above >> all, >> >> > you >> >> > must write more about the kind of things you see. Sunday a fortnight >> ago >> >> > I >> >> > was in Amsterdam to see an exhibition of the paintings going to >> Vienna >> >> > from >> >> > here.<anchor n="6" xml:id="note-t-6"/>It was very interesting, and >> I’m >> >> > curious<pb f="1r" n="4" xml:id="pb-trans-1r-4" >> facs="#zone-pb-1r-4"/>as >> >> > to >> >> > the impression the Dutch will make in Vienna.</ab> >> >> > </hit> >> >> > </document> >> >> > And this is the query: >> >> > let $range := 1 to 5 >> >> > for $doc in collection('christian') >> >> > let $uri := base-uri($doc), >> >> > $num := substring($uri, string-length($uri) - 6, 3) >> >> > where $num castable as xs:integer >> >> > and xs:integer($num) = $range >> >> > return <document uri='{$uri}'>{ >> >> > for $n score $s in $doc//*[text() contains text 'above'] >> >> > return <hit score='{$s}'>{ $n }</hit> >> >> > }</document> >> >> > Kind regards, >> >> > Wiard >> >> > 2011/4/5 Christian Grün <christian.gruen at gmail.com> >> >> >> >> >> >> Dear Wiard, >> >> >> what does the query info tell you? Just copy&paste the info to this >> >> >> list. >> >> >> Thanks >> >> >> Christian >> >> >> ___________________________ >> >> >> >> >> >> On Tue, Apr 5, 2011 at 3:04 PM, Wiard Vasen <wiard.vasen at gmail.com> >> >> >> wrote: >> >> >>> >> >> >>> Dear Christian, >> >> >>> When I initialized the database I marked in 'Full Text' properties >> the >> >> >>> TF >> >> >>> / IDF checkbox. >> >> >>> So, I think that 'score' in the query gives this score back. >> >> >>> Do you think I am right? >> >> >>> Thanks in advance for your answer. >> >> >>> Kind regards, >> >> >>> Wiard >> >> >>> >> >> > >> >> > >> > >> > >> > > >

> Hi Andreas, > > Thanks a lot for your solution! > I am going to look at the result to see if it gives the right scores and > come back on you. > > Kind regards, > > Wiard > > > > > > 2011/4/5 Andreas Weiler <andreas.weiler at uni-konstanz.de> > >> Hi Wiard, >> >> try the following: >> >> let $range := 1 to 640 >> for $doc in collection('tfidfbrievenvangogh') >> let $uri := base-uri($doc), >> $num := substring($uri, string-length($uri) - 6, 3) >> where $num castable as xs:integer >> and xs:integer($num) = $range >> return <document uri='{$uri}'>{ >> for $n in $doc//* >> where $n contains text 'above' >> return <hit score='{ft:score($n[text() contains text 'above'])}'>{ $n >> }</hit> >> }</document> >> >> -- Andreas >> >> Am 05.04.2011 um 20:35 schrieb Wiard Vasen: >> >> Hi Christian, >> >> This query gives good results on tf-scores: >> >> ft:score(db:open("tfidfbrievenvangogh")//*[text() contains text 'man']) >> >> But the problem is that I need the specific documents connected with the >> given scores. >> >> For that reason I thought that the following query: >> >> let $range := 1 to 640 >> for $doc in collection('tfidfbrievenvangogh') >> let $uri := base-uri($doc), >> $num := substring($uri, string-length($uri) - 6, 3) >> where $num castable as xs:integer >> and xs:integer($num) = $range >> return <document uri='{$uri}'>{ >> for $n score $s in $doc//*[text() contains text 'above'] >> return <hit score='{$s}'>{ $n }</hit> >> }</document> >> >> would automatically give the tf/idf score because here 'score' is a >> reserved word and tf/idf where checked while initializing the full-text >> repository. >> >> I am not sure whether this thought is right. >> >> Maybe you know the answer. >> >> Kind regards, >> >> Wiard >> >> 2011/4/5 Christian Grün <christian.gruen at gmail.com> >> >>> > This is the information from the Query Info panel: >>> > Compiling: >>> > - binding static variable $range >>> > - pre-evaluating collection("christian") >>> > - optimizing descendant-or-self step(s) >>> > - removing variable $range >>> >>> Your query seems to be too complex to be evaluated via the full-text >>> index (probably because the nested flwor expression); otherwise, the >>> query info would contain the following line at least once: >>> >>> - applying full-text index >>> >>> If you don't want to spend too much time into rewriting your query, >>> you might as well access the index directly, such as: >>> >>> for $d in collection('coll') >>> for $x in ft:search($d, 'text') >>> where $x/ancestor::node()[. = $d] >>> return ft:score($x) >>> >>> Hope this helps, >>> Christian >>> >>> >>> >>> >>> >>> > Result: for $doc in (document-node { "let001.xml" }, document-node { >>> > "let002.xml" }, ...) let $uri := base-uri($doc) let $num := >>> substring($uri, >>> > string-length($uri) - 6, 3) where $num castable as xs:integer and $num >>> cast >>> > as xs:integer? = 1 to 5 return element { "document" } { attribute { >>> "uri" } >>> > { $uri }, for $n score $s as xs:double in $doc/descendant::*[text() >>> contains >>> > text "above"] return element { "hit" } { attribute { "score" } { $s }, >>> $n } >>> > } >>> > Timing: >>> > - Parsing: 0.88 ms >>> > - Compiling: 5.71 ms >>> > - Evaluating: 93.72 ms >>> > - Printing: 0.27 ms >>> > - Total Time: 100.6 ms >>> > Query plan: >>> > <FLWR> >>> > <For var="$doc"> >>> > <sequence size="927"> >>> > <document-node() name="christian"/> >>> > <document-node() name="christian" pre="480"/> >>> > <document-node() name="christian" pre="913"/> >>> > <document-node() name="christian" pre="1897"/> >>> > <document-node() name="christian" pre="2928"/> >>> > </sequence> >>> > </For> >>> > <Let var="$uri"> >>> > <FNNode name="base-uri([node])"> >>> > <VarRef name="$doc"/> >>> > </FNNode> >>> > </Let> >>> > <Let var="$num"> >>> > <FNStr name="substring(string,start[,len])"> >>> > <VarRef name="$uri"/> >>> > <Arith op="-"> >>> > <FNAcc name="string-length([item])"> >>> > <VarRef name="$uri"/> >>> > </FNAcc> >>> > <Item value="6" type="xs:integer"/> >>> > </Arith> >>> > <Item value="3" type="xs:integer"/> >>> > </FNStr> >>> > </Let> >>> > <Where> >>> > <And> >>> > <Castable type="xs:integer"> >>> > <VarRef name="$num"/> >>> > </Castable> >>> > <CmpG op="="> >>> > <Cast type="xs:integer?"> >>> > <VarRef name="$num"/> >>> > </Cast> >>> > <Range> >>> > <Item value="1" type="xs:integer"/> >>> > <Item value="5" type="xs:integer"/> >>> > </Range> >>> > </CmpG> >>> > </And> >>> > </Where> >>> > <Return> >>> > <CElem> >>> > <Item value="document" type="xs:QName"/> >>> > <CAttr> >>> > <Item value="uri" type="xs:QName"/> >>> > <VarRef name="$uri"/> >>> > </CAttr> >>> > <FLWR> >>> > <For var="$n" score="$s as xs:double"> >>> > <AxisPath> >>> > <VarRef name="$doc"/> >>> > <IterStep axis="descendant" test="*"> >>> > <FTContains> >>> > <AxisPath> >>> > <IterStep axis="child" test="text()"/> >>> > </AxisPath> >>> > <FTWords> >>> > <Item value="above" type="xs:string"/> >>> > </FTWords> >>> > </FTContains> >>> > </IterStep> >>> > </AxisPath> >>> > </For> >>> > <Return> >>> > <CElem> >>> > <Item value="hit" type="xs:QName"/> >>> > <CAttr> >>> > <Item value="score" type="xs:QName"/> >>> > <VarRef name="$s as xs:double"/> >>> > </CAttr> >>> > <VarRef name="$n"/> >>> > </CElem> >>> > </Return> >>> > </FLWR> >>> > </CElem> >>> > </Return> >>> > </FLWR> >>> > Thanks! >>> > Regards, >>> > Wiard >>> > >>> > 2011/4/5 Christian Grün <christian.gruen at gmail.com> >>> >> >>> >> Hi Wiard, >>> >> >>> >> looks like you sent me the query result. To tell you if the index was >>> >> utilized, I need the output from the »Query Info« panel, or (if that >>> >> won't help) the original data instances. >>> >> >>> >> Christian >>> >> >>> >> >>> >> On Tue, Apr 5, 2011 at 4:52 PM, Wiard Vasen <wiard.vasen at gmail.com> >>> wrote: >>> >> > Hi Christian, >>> >> > This is the result of the query: >>> >> > <document >>> >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let001.xml"/> >>> >> > <document >>> >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let002.xml"/> >>> >> > <document >>> >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let003.xml"/> >>> >> > <document >>> >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let004.xml"/> >>> >> > <document >>> >> > uri="file:/Users/wiardvasen/Desktop/brievenvangogh/let005.xml"> >>> >> > <hit score="0.03590675482297878"> >>> >> > <ab xmlns="http://www.tei-c.org/ns/1.0" rend="indent">How is >>> your >>> >> > boarding-house? Is it still to your liking? That’s important. Above >>> all, >>> >> > you >>> >> > must write more about the kind of things you see. Sunday a fortnight >>> ago >>> >> > I >>> >> > was in Amsterdam to see an exhibition of the paintings going to >>> Vienna >>> >> > from >>> >> > here.<anchor n="6" xml:id="note-t-6"/>It was very interesting, and >>> I’m >>> >> > curious<pb f="1r" n="4" xml:id="pb-trans-1r-4" >>> facs="#zone-pb-1r-4"/>as >>> >> > to >>> >> > the impression the Dutch will make in Vienna.</ab> >>> >> > </hit> >>> >> > </document> >>> >> > And this is the query: >>> >> > let $range := 1 to 5 >>> >> > for $doc in collection('christian') >>> >> > let $uri := base-uri($doc), >>> >> > $num := substring($uri, string-length($uri) - 6, 3) >>> >> > where $num castable as xs:integer >>> >> > and xs:integer($num) = $range >>> >> > return <document uri='{$uri}'>{ >>> >> > for $n score $s in $doc//*[text() contains text 'above'] >>> >> > return <hit score='{$s}'>{ $n }</hit> >>> >> > }</document> >>> >> > Kind regards, >>> >> > Wiard >>> >> > 2011/4/5 Christian Grün <christian.gruen at gmail.com> >>> >> >> >>> >> >> Dear Wiard, >>> >> >> what does the query info tell you? Just copy&paste the info to this >>> >> >> list. >>> >> >> Thanks >>> >> >> Christian >>> >> >> ___________________________ >>> >> >> >>> >> >> On Tue, Apr 5, 2011 at 3:04 PM, Wiard Vasen <wiard.vasen at gmail.com >>> > >>> >> >> wrote: >>> >> >>> >>> >> >>> Dear Christian, >>> >> >>> When I initialized the database I marked in 'Full Text' properties >>> the >>> >> >>> TF >>> >> >>> / IDF checkbox. >>> >> >>> So, I think that 'score' in the query gives this score back. >>> >> >>> Do you think I am right? >>> >> >>> Thanks in advance for your answer. >>> >> >>> Kind regards, >>> >> >>> Wiard >>> >> >>> >>> >> > >>> >> > >>> > >>> > >>> >> >> >> >