Working with indices
Friday, 29 October 2010 · 2 messages
JV
Jan Vlčinský29 Oct 2010 08:27 #
Hi
I have a bigger collection
DDR-All
Database Properties
Name: DDR-All
Size: 743 MB
Nodes: 33340760
Height: 8
Database Creation
Source Path: D:/var/TICEReports/JSDI data/bezne
Time Stamp: 26.10.2010 02:08:30
Input Size: 542 MB
Encoding: utf-8
Documents: 79313
Whitespace Chopping: ON
Entity Parsing: OFF
Indexes
Path Summary: ON
Text Index: ON
Attribute Index: ON
Full-Text Index: OFF
When I run xquery count(//MSG)
it shows result 511990, which is correct.
However, it takes a lot of time to produce that result. I thought, that with existing indices it shall bring the result in seconds, and not in minutes (like 10 minutes)
Note, that I have created the collection before 6.3 was released, so this might be the case and I would have to import it all again.
I was trying to run xquery, using indices
xquery basex:index('MSG', 'path')
xquery basex:index('//MSG', 'path')
but I got a feeling, I do not got it properly, as it responds "Unknown index path"
What I would like to know is
- names of indices available
- what is basex:index function usable for and for what it is not
- can I create my own indices? Like calculating date from dateTime fileds in XML. I guess it is not supported.
- any further reading? Most of the description is related to full text, but this is not my case.
Jan
PS: Despite of some performance problems (query lasting 25 minutes) I was able to dig out some very valuable information, which would be inefficient and probably much more error prone in any other way.
http://janvlcinsky.s3.amazonaws.com/projects/Monitor/websamples/timeplot/tgen-count.html
--
*Jan Vlčinský*
Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic
tel: +420-597 602 024; mob: +420-608 979 040
skype: janvlcinsky; GoogleTalk: jan.vlcinsky at gmail.com
CG
Christian Grün29 Oct 2010 09:08 #
Hi Jan,
Quoted text · Jan Vlčinský · 5 lines
> When I run xquery count(//MSG) > it shows result 511990, which is correct. > However, it takes a lot of time to produce that result. I thought, that with > existing indices it shall bring the result in seconds, and not in minutes > (like 10 minutes)
Wow; this query should be optimized and evaluated in constant time. You should try to..
- call the "optimize" database command on the database
- use count( //*:MSG ) if the database included namespace
If this doesn't help, feel free to give us the output of the optimizer (which you can trigger via -V on command line, or by checking out the InfoView in the GUI)?
Quoted text · Jan Vlčinský · 2 lines
> xquery basex:index('MSG', 'path') > xquery basex:index('//MSG', 'path')
The basex:index function is only applicable to the text, attribute and full-text index (see http://basex.org/xqfunc). The path index is only utilized internally, and at quite diffferent places (such as for the count() function, as you correctly guessed...).
Hope this helps, feel free to ask for more,
Christian
Quoted text · Jan Vlčinský · 25 lines
> What I would like to know is > names of indices available > what is basex:index function usable for and for what it is not > can I create my own indices? Like calculating date from dateTime fileds in > XML. I guess it is not supported. > any further reading? Most of the description is related to full text, but > this is not my case. > > Jan > PS: Despite of some performance problems (query lasting 25 minutes) I was > able to dig out some very valuable information, which would be inefficient > and probably much more error prone in any other way. > http://janvlcinsky.s3.amazonaws.com/projects/Monitor/websamples/timeplot/tgen-count.html > -- > Jan Vlčinský > Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic > tel: +420-597 602 024; mob: +420-608 979 040 > skype: janvlcinsky; GoogleTalk: jan.vlcinsky at gmail.com > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >
Written for BaseX 6. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery