Initial xquery for full text and geospatial search of ISO 19115 docs
Monday, 31 December 2012 · 2 messages
DS
David Stuebe31 Dec 2012 20:31 #
Hi Basex Folks
I have written a simple minded xquery script which can be used in a post to search ISO 19115 metadata documents. As I am a newbie to xquery and basex I expect that is much that I could do to improve performance, but currently searches take up to 15 seconds. The server hardware is not blazing fast and we are running inside tomcat… but that is certainly not acceptable for a database which is still quiet small. Initially it was fine, responding in less than a second, but when we expanded from less than 100 docs to about 35000 the time grew at least linearly. I am hoping this is due to my poor xquery programming or a setting on the server.
Here is the query we are running using a post request with the declared variables at the bottom filled in by the UI: https://github.com/asascience-open/glos_catalog/blob/master/queries/full_search.xml
Here are some of the ISO xml documents that we have in our basex DB: https://github.com/asascience-open/glos_catalog/tree/master/ISOs
It is used by this site: http://explorer.glos.us/
To provide geospatial metadata search. Fill in a text value like "water" or "temperature" in the search box in the top right…
The server is here: http://64.9.200.113:8080/BaseX73/
User Name: user
Password: glos
ACL: read only
I am hoping it is as simple as a setting on the basex server. Here is the current server info:
info database
Database Properties
Name: glos
Size: 422 MB
Nodes: 17045388
Documents: 34922
Binaries: 0
Timestamp: 27.12.2012 00:58:03
Resource Properties
Timestamp: 22.12.2012 01:43:05
Encoding: UTF-8
Whitespace Chopping: ON
Indexes
Up-to-date: true
Text Index: ON
Attribute Index: ON
Full-Text Index: ON
The index info is available here:
https://github.com/asascience-open/glos_catalog/blob/master/queries/glos_index_info.txt
David Stuebe
Scientist & Software Engineer – RPS ASA
55 Village Square Drive
South Kingstown, RI 02879-8248
Tel: +1 (401) 789-6224
Email: David.Stuebe at rpsgroup.com<mailto:David.Stuebe at rpsgroup.com> www: asascience.com<http://www.asascience.com/> | rpsgroup.com<http://www.rpsgroup.com/>
A member of the RPS Group plc
CG
Christian Grün31 Dec 2012 20:56 #
Hi David,
thanks for the insight into your project. It may be that the full-text index is not utilized by your XQuery expression. Did you have a look at the query info (e.g. via GUI, InfoView, or -V on command line)? Did you manage to write simpler queries that are processed faster?
Best,
Christian
Quoted text · David Stuebe · 82 lines
On Mon, Dec 31, 2012 at 7:31 PM, David Stuebe <DStuebe at asascience.com> wrote: > > > > Hi Basex Folks > > I have written a simple minded xquery script which can be used in a post to > search ISO 19115 metadata documents. > As I am a newbie to xquery and basex I expect that is much that I could do > to improve performance, but currently searches take up to 15 seconds. The > server hardware is not blazing fast and we are running inside tomcat… but > that is certainly not acceptable for a database which is still quiet small. > Initially it was fine, responding in less than a second, but when we > expanded from less than 100 docs to about 35000 the time grew at least > linearly. I am hoping this is due to my poor xquery programming or a setting > on the server. > > Here is the query we are running using a post request with the declared > variables at the bottom filled in by the UI: > https://github.com/asascience-open/glos_catalog/blob/master/queries/full_search.xml > > Here are some of the ISO xml documents that we have in our basex DB: > https://github.com/asascience-open/glos_catalog/tree/master/ISOs > > It is used by this site: http://explorer.glos.us/ > To provide geospatial metadata search. Fill in a text value like "water" or > "temperature" in the search box in the top right… > > The server is here: http://64.9.200.113:8080/BaseX73/ > User Name: user > Password: glos > ACL: read only > > > I am hoping it is as simple as a setting on the basex server. Here is the > current server info: > > info database > Database Properties > Name: glos > Size: 422 MB > Nodes: 17045388 > Documents: 34922 > Binaries: 0 > Timestamp: 27.12.2012 00:58:03 > > Resource Properties > Timestamp: 22.12.2012 01:43:05 > Encoding: UTF-8 > Whitespace Chopping: ON > > Indexes > Up-to-date: true > Text Index: ON > Attribute Index: ON > Full-Text Index: ON > > The index info is available here: > https://github.com/asascience-open/glos_catalog/blob/master/queries/glos_index_info.txt > > > > David Stuebe > > Scientist & Software Engineer – RPS ASA > > 55 Village Square Drive > South Kingstown, RI 02879-8248 > > Tel: +1 (401) 789-6224 > > Email: David.Stuebe at rpsgroup.com > www: asascience.com | rpsgroup.com > > A member of the RPS Group plc > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >
Written for BaseX 7.5. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery