Changes

The query compiler tries to optimize and speed up queries by applying the index whenever it is possible and seems promising. To see how a query is rewritten, and if an index is used, you can turn on the [[GUI#Visualizations|Info View]] in the GUI or use the [[Command-Line Options#BaseX_Standalone|-V flag]] on the command line:

=Structural Indexes=

The name index is e.g. applied to discard location steps that will never yield results:

(: will be rewritten to an empty sequence :)

/non-existing-name

The contents of the name indexes can be directly accessed with the XQuery functions [[Index Module#index:element-names|index:element-names]] and [[Index Module#index:attribute-names|index:attribute-names]].

* Descendant steps will be rewritten to multiple child steps. Child steps are evaluated faster, as fewer nodes have to be traversed:

doc('factbook.xml')//province,

(: ...will be rewritten to... :)

doc('factbook.xml')/mondial/country/province

* The {{Code|fn:count}} function will be pre-evaluated by looking up the number in the index:

count(doc('factbook')//country)

* The distinct values of elements or attributes can be looked up in the index as well:

distinct-values(db:open('factbook')//religions)

The contents of the path index can be directly accessed with the XQuery function [[Index Module#index:facets|index:facets]].

With XQuery, index structures can be created and dropped via {{Function|Database|db:optimize}}:

(: Optimize specified database, create full-text index for texts of the specified elements :)

db:optimize(

map { 'ftindex': true(), 'ftinclude': 'p div' }

)

==Text Index==

===Exact Queries===

//*[text() = 'Germany'],

doc('factbook.xml')//name[. = 'Germany'],

for $c in db:open('factbook')//country

where $c//city/name = 'Hanoi'

return $c/name

db:optimize(

'mydb',

map { 'updindex':true(), 'textindex': true(), 'textinclude':'id' }

)

===Range Queries===

The text index also supports range queries based on string comparisons:

db:open('Library')//Medium[Year >= '2011' and Year <= '2016'],

let $min := '2014-04-16T00:00:00'

let $max := '2014-04-19T23:59:59'

return db:open('news')//entry[date-time > $min and date-time < $max]

==Attribute Index==

(: 1st example :)

//country[@car_code = 'J'],

(: 4th example :)

fn:id('f0_119', db:open('factbook'))

The {{Option|UPDINDEX}} option can be activated to keep this index up-to-date.

In many XML dialects, such as HTML or DITA, multiple tokens are stored in attribute values. The token index can be created to speed up the retrieval of these tokens. The XQuery functions {{Code|fn:contains-token}}, {{Code|fn:tokenize}} and {{Code|fn:idref}} are rewritten for index access whenever possible. If a token index exists, it will e.g. be utilized for the following queries:

(: 1st example :)

//div[contains-token(@class, 'row')],

(: 3rd example :)

doc('graph.xml')/idref('edge8')

==Full-Text Index==

If the full-text index exists, the following queries will all be rewritten for index access:

(: 1st example :)

//country[name/text() contains text 'and'],

//religions[.//text() contains text { 'Catholic', 'Roman' }

using case insensitive distance at most 2 words]

The index provides support for the following full-text features (the values can be changed in the GUI or via the {{Command|SET}} command):

The options that have been used for creating the full-text index will also be applied to the optimized full-text queries. However, the defaults can be overwritten if you supply options in your query. For example, if words were stemmed in the index, and if the query can be rewritten for index access, the query terms will be stemmed as well, unless stemming is not explicitly disabled. This is demonstrated in the following [[Commands#Command_Scripts|Command Script]]:

<commands>

<!-- Create database with stemmed full-text index -->

<xquery> /text[. contains text { 'houses' } using no stemming] </xquery>

</commands>

Text nodes can be directly requested from the index via the XQuery function {{Function|Full-Text|ft:search}}. The index contents can be accessed with {{Function|Full-Text|ft:tokens}}.

; Commands

SET ATTRINCLUDE id,name

CREATE DB factbook http://files.basex.org/xml/factbook.xml'

# Restore default

SET ATTRINCLUDE

; XQuery

db:create('factbook', 'http://files.basex.org/xml/factbook.xml', '',

map { 'attrinclude': 'id,name' })

With {{Command|CREATE INDEX}} and {{Function|Database|db:optimize}}, new selective indexing options will ba applied to an existing database.

* In the query below, 10 databases will be addressed. If it is known in advance that these databases contain an up-to-date text index, the index rewriting can be enforced as follows:

(# db:enforceindex #) {

for $n in 1 to 10

return db:open($db)//person[name/text() = 'John']

}

* The following query contains two predicates that may both be rewritten for index access. If the automatically chosen rewriting is known not to be optimal, another index rewriting can enforced by surrounding the specific expression with the pragma:

db:open('factbook')//country

[(# db:enforceindex #) {

}]

[religions/text() = 'Protestant']

for $name in ('Germany', 'Italy')

for $country in db:open('factbook')//country

where $country/religions/text() = 'Protestant'

return $country

Please note that:

* The option should only be enabled if the addressed databases exist, have all required index structures and are up-to-date (otherwise, you will be given an error message).

* If you have more than one enforce pragma in a single path expression, only the first will be considered.

=Custom Index Structures=

With XQuery, it is comparatively easy to create your own, custom index structures. The following query demonstrate how you can create a {{Code|factbook-index}} database, which contains all texts of the original database in lower case:

let $db := 'factbook'

return db:create($db || '-index', $index, $db || '-index.xml')

In the following query, a text string is searched, and the text nodes of the original database are retrieved:

let $db := 'factbook'

let $text := 'italian'

for $id in db:open($db || '-index')//*[@string = $text]/id

return db:open-id($db, $id)/..

With some extra effort, and if {{Option|UPDINDEX}} is enabled for both your original and your index database (see below), your index database will support updates as well (try it, it’s fun!).

If {{Option|DEBUG}} is enabled, the command-line output might help you to find a good split size. The following example shows the output for creating a database for an XMark document with 1 GB, and with 128 MB assigned to the JVM:

> basex -d -c"SET FTINDEX ON; SET TOKENINDEX ON; CREATE DB xmark 1gb.xml"

Creating Database...

Indexing Full-Text...

..|.|.|.|...|...|..|.|..| 116.33 M operations, 138740.94 ms (106 MB). Recommended SPLITSIZE: 12.

The output can be interpreted as follows: