XQuery Extensions

From BaseX Documentation

(Difference between revisions)
Jump to: navigation, search
(Namespaces)
 
(67 intermediate revisions not shown)
Line 1: Line 1:
-
This article is part of the [[XQuery|XQuery Portal]]. It lists specific extensions and optimizations of the BaseX XQuery processor.
+
This article is part of the [[XQuery|XQuery Portal]]. It lists extensions and optimizations that are specific to the BaseX XQuery processor.
 +
 
 +
=Expressions=
 +
 
 +
Some of the extensions that have been added to BaseX may also be made available in other XQuery processors in the near future.
 +
 
 +
==Ternary If==
 +
 
 +
The [https://en.wikipedia.org/wiki/%3F: ternary if] operator provides a short syntax for conditions. It is also called '''conditional operator''' or '''ternary operator'''. In most languages, the syntax is <code>a ? b : c</code>. As <code>?</code> and <code>:</code> have already been taken in XQuery, the syntax of Perl 6 is used:
 +
 
 +
<pre class="brush:xquery">
 +
$test ?? 'ok' !! 'fails'
 +
</pre>
 +
 
 +
The expression returns <code>ok</code> if the effective boolean value of <code>$test</code> is true, and it returns <code>fails</code> otherwise.
 +
 
 +
==Elvis Operator==
 +
 
 +
The Elvis operator is also available in other languages. It is sometimes called [https://en.wikipedia.org/wiki/Null_coalescing_operator null-coalescing operator]. In XQuery, the value of the first operand will be returned if it is a non-empty sequence. Otherwise, the value of the second operand will be returned.
 +
 
 +
<pre class="brush:xquery">
 +
let $number := 123
 +
return (
 +
  (: if/then/else :)
 +
  if (exists($number)) then $number else 0,
 +
  (: elvis operator :)
 +
  $number ?: 0
 +
)
 +
</pre>
 +
 
 +
The behavior of the operator is equivalent to the {{Function|Utility|util:or}} function.
 +
 
 +
==If Without Else==
 +
 
 +
In XQuery 3.1, both branches of the <code>if</code> expression need to be specified. In many cases, only one branch is required, so the <code>else</code> branch was made optional in BaseX. If the second branch is omitted, an empty sequence will be returned if the effective boolean value of the test expression is false. Some examples:
 +
 
 +
<pre class="brush:xquery">
 +
if (doc-available($doc)) then doc($doc),
 +
if (file:exists($file)) then file:delete($file),
 +
if (permissions:valid($user)) then <html>Welcome!</html>
 +
</pre>
 +
 
 +
If conditions are nested, a trailing else branch will be associated with the innermost <code>if</code>:
 +
 
 +
<pre class="brush:xquery">
 +
if ($a) then if($b) then '$a and $b is true' else 'only $b is true'
 +
</pre>
 +
 
 +
In general, if you have multiple or nested if expressions, additional parentheses can improve the readibility of your code:
 +
 
 +
<pre class="brush:xquery">
 +
if ($a) then (
 +
  if($b) then '$a and $b is true' else 'only $b is true'
 +
)
 +
</pre>
 +
 
 +
The behavior of the if expression is equivalent to the {{Function|Utility|util:if}} function.
 +
 
 +
=Functions=
 +
 
 +
==Regular Expressions==
 +
 
 +
In analogy with Saxon, you can specify the flag {{Code|j}} to revert to Java’s default regex parser. For example, this allows you to use the word boundary option {{Code|\b}}, which has not been included in the XQuery grammar for regular expressions:
 +
 
 +
'''Example:'''
 +
<pre class="brush:xquery">
 +
(: yields "!Hi! !there!" :)
 +
replace('Hi there', '\b', '!', 'j')
 +
</pre>
 +
 
 +
=Serialization=
 +
 
 +
* <code>basex</code>is used as the default serialization method: nodes are serialized as XML, atomic values are serialized as string, and items of binary type are output in their native byte representation. Function items (including maps and arrays) are output just like with the [[XQuery 3.1#Adaptive Serialization|adaptive]] method.
 +
* With {{Code|csv}}, you can output XML nodes as CSV data (see the [[CSV Module]] for more details).
 +
* With {{Code|json}}, items are output as JSON as described in the [https://www.w3.org/TR/xslt-xquery-serialization-31/#json-output official specification]. If the root node is of type {{Code|element(json)}}, items are serialized as described for the {{Code|direct}} format in the [[JSON Module]].
 +
 
 +
For more information and some additional BaseX-specific parameters, see the article on [[Serialization]].
=Option Declarations=
=Option Declarations=
-
[[Options|Local database options]] can be set in the prolog of an XQuery expression. In the option declaration, options need to be bound to the [[Database Module]] namespace. All values will be reset after the evaluation of a query:
+
==Database Options==
 +
 
 +
[[Options|Local database options]] can be set in the prolog of an XQuery main module. In the option declaration, options need to be bound to the [[Database Module]] namespace. All values will be reset after the evaluation of a query:
<pre class="brush:xquery">
<pre class="brush:xquery">
declare option db:chop 'false';
declare option db:chop 'false';
doc('doc.xml')
doc('doc.xml')
 +
</pre>
 +
 +
==XQuery Locks==
 +
 +
If [[Transactions#XQuery_Locks|XQuery Locks]] are defined in the query prolog of a module, access to functions of this module locks will be controlled by the central transaction management.
 +
 +
If the following XQuery code is called by two clients in parallel, the queries will be evaluated one after another:
 +
 +
<pre class="brush:xquery">
 +
declare option basex:write-lock 'CONFIGLOCK';
 +
file:write('config.xml', <config/>)
</pre>
</pre>
=Pragmas=
=Pragmas=
-
[[Options|Local database options]] can be assigned locally via pragmas:
+
==BaseX Pragmas==
 +
 
 +
Many optimizations in BaseX will only be performed if an expression is ''deterministic'' (i. e., if it always yields the same output and does not have side effects). By flagging an expression as non-deterministic, optimizations and query rewritings can be suppressed:
 +
 
 +
<pre class="brush:xquery">
 +
sum( (# basex:non-deterministic #) {
 +
  1 to 100000000
 +
})
 +
</pre>
 +
 
 +
This pragma can be helpful when debugging your code.
 +
 
 +
In analogy with option declarations and function annotations, [[Transactions#XQuery_Locks|XQuery Locks]] can also set via pragmas:
 +
 
 +
<pre class="brush:xquery">
 +
(# basex:write-lock CONFIGLOCK #) {
 +
  file:write('config.xml', <config/>)
 +
}
 +
</pre>
 +
 
 +
==Database Pragmas==
 +
 
 +
All [[Options|local options]] can be assigned via pragmas. Some examples:
 +
 
 +
* Enforce query to be rewritten for index access. This can e. g. be helpful if the name of a database is not static (see [[Indexes#Enforce Rewritings|Enforce Rewritings]] for more examples):
 +
 
 +
<pre class="brush:xquery">
 +
(# db:enforceindex #) {
 +
  for $db in ('persons1', 'persons2', 'persons3')
 +
  return db:open($db)//name[text() = 'John']
 +
}
 +
</pre>
 +
 
 +
* Temporarily disable node copying in node constructors (see {{Option|COPYNODE}} for more details). The following query will be evaluated faster, and take much less memory, than without pragma, because the database nodes will not be fully copied, but only attached to the new {{Code|xml}} parent element:
<pre class="brush:xquery">
<pre class="brush:xquery">
-
(# db:chop false #) { doc('doc.xml') }
+
file:write(
 +
  'wrapped-db-nodes.xml',
 +
  (# db:copynode false #) {
 +
    <xml>{ db:open('huge') }</xml>
 +
  }
 +
)
</pre>
</pre>
=Annotations=
=Annotations=
-
The following implementation-defined annotations are available:
+
==Function Inlining==
-
* {{Code|%basex:inline([limit])}} enforces the inlining of a function. Example:
+
{{Code|%basex:inline([limit])}} controls if functions will be inlined.
 +
 
 +
If XQuery functions are ''inlined'', the function call will be replaced by a FLWOR expression, in which the function variables are bound to let clauses, and in which the function body is returned. This optimization triggers further query rewritings that will speed up your query. An example:
 +
 
 +
'''Query:'''
-
'''Example:'''
 
<pre class="brush:xquery">
<pre class="brush:xquery">
-
declare option db:inlinelimit '0';
+
declare function local:square($a) { $a * $a };
-
declare %basex:inline function local:id($x) { $x };
+
for $i in 1 to 3
-
local:id(123)
+
return local:square($i)
</pre>
</pre>
-
In this query, function inlining has been deactivated by setting [[Options#INLINELIMIT|inlinelimit]] to {{Code|0}}. The annotation enforces inlining for the given function, though, resulting in the optimized query expression {{Code|123}}.
+
'''Query after function inlining:'''
-
If an integer is specified as annotation argument, it will be interpreted a local inline limit.
+
<pre class="brush:xquery">
 +
for $i in 1 to 3
 +
return
 +
  let $a := $i
 +
  return $a * $a
 +
</pre>
 +
 
 +
'''Query after further optimizations:'''
 +
 
 +
<pre class="brush:xquery">
 +
for $i in 1 to 3
 +
return $i * $i
 +
</pre>
 +
 
 +
By default, XQuery functions will be ''inlined'' if the query body is not too large and does not exceed a fixed number of expressions, which can be adjusted via the {{Option|INLINELIMIT}} option.
 +
 
 +
The annotation can be used to overwrite this global limit: Function inlining can be enforced if no argument is specified. Inlining will be disabled if {{Code|0}} is specified.
 +
 
 +
'''Example:'''
 +
 
 +
<pre class="brush:xquery">
 +
(: disable function inlining; the full stack trace will be shown... :)
 +
declare %basex:inline(0) function local:e() { error() };
 +
local:e()
 +
</pre>
 +
 
 +
'''Result:'''
 +
 
 +
<pre class="brush:xml">
 +
Stopped at query.xq, 1/53:
 +
[FOER0000] Halted on error().
 +
 
 +
Stack Trace:
 +
- query.xq, 2/9
 +
</pre>
 +
 
 +
==Lazy Evaluation==
-
* {{Code|%basex:lazy}} enforces the lazy evaluation of a global variable. Example:
+
{{Code|%basex:lazy}} enforces lazy evaluation of a global variable. An example:
'''Example:'''  
'''Example:'''  
<pre class="brush:xquery">
<pre class="brush:xquery">
declare %basex:lazy variable $january := doc('does-not-exist');
declare %basex:lazy variable $january := doc('does-not-exist');
-
if(month-from-date(current-date()) == 1) then $january else ()
+
if(month-from-date(current-date()) = 1) then $january else ()
</pre>
</pre>
-
The annotation ensures that an error will only be thrown if the condition yields true. Without the annotation, the error will always be thrown, because the referenced document is not found.
+
The annotation ensures that an error will only be raised if the condition yields true. Without the annotation, the error will always be raised, because the referenced document is not found.
-
=Optimizations=
+
==XQuery Locks==
-
==Non-determinism==
+
In analogy with option declarations and pragmas, [[Transactions#XQuery_Locks|XQuery Locks]] can also set via annotations:
 +
 
 +
<pre class="brush:xquery">
 +
declare %basex:write-lock('CONFIGLOCK') function local:write() {
 +
  file:write('config.xml', <config/>)
 +
};
 +
</pre>
 +
 
 +
=Non-Determinism=
In [http://www.w3.org/TR/xpath-functions-31/#dt-deterministic XQuery], ''deterministic'' functions are “guaranteed to produce ·identical· results from repeated calls within a single ·execution scope· if the explicit and implicit arguments are identical”. In BaseX, many extension functions are non-deterministic or side-effecting. If an expression is internally flagged as non-deterministic, various optimizations that might change their execution order will not be applied.
In [http://www.w3.org/TR/xpath-functions-31/#dt-deterministic XQuery], ''deterministic'' functions are “guaranteed to produce ·identical· results from repeated calls within a single ·execution scope· if the explicit and implicit arguments are identical”. In BaseX, many extension functions are non-deterministic or side-effecting. If an expression is internally flagged as non-deterministic, various optimizations that might change their execution order will not be applied.
Line 77: Line 251:
Two non-deterministic functions will be bound to <code>$read</code>, and the result of the function call will be bound to <code>$ignored</code>. As the variable is not referenced in the subsequent code, the let clause would usually be discarded by the compiler. In the given query, however, execution will be enforced because of the BaseX-specific {{Code|non-deterministic}} keyword.
Two non-deterministic functions will be bound to <code>$read</code>, and the result of the function call will be bound to <code>$ignored</code>. As the variable is not referenced in the subsequent code, the let clause would usually be discarded by the compiler. In the given query, however, execution will be enforced because of the BaseX-specific {{Code|non-deterministic}} keyword.
-
=Other Extensions=
+
=Namespaces=
 +
 
 +
In XQuery, some namespaces are statically bound to prefixes. The following query requires no additional namespaces declarations in the query prolog:
 +
 
 +
<pre class="brush:xquery">
 +
<xml:abc xmlns:prefix='uri' local:fn='x'/>,
 +
fn:exists(1)
 +
</pre>
 +
 
 +
In BaseX, various other namespaces are predefined. Apart from the namespaces that are listed on the [[Module Library]] page, the following namespaces are statically bound:
 +
 
 +
{| class="wikitable sortable"
 +
|-
 +
! Description
 +
! Prefix
 +
! Namespace URI
 +
|-
 +
| [[#Annotations|BaseX Annotations]], [[#Pragmas|Pragmas]], …
 +
| <code>basex</code>
 +
| <code><nowiki>http://basex.org</nowiki></code>
 +
|-
 +
| [[RESTXQ#Input Options|RESTXQ: Input Options]]
 +
| <code>input</code>
 +
| <code><nowiki>http://basex.org/modules/input</nowiki></code>
 +
|-
 +
| [[Repository#EXPath_Packaging|EXPath Packages]]
 +
| <code>pkg</code>
 +
| <code><nowiki>http://expath.org/ns/pkg</nowiki></code>
 +
|-
 +
| [[XQuery Errors]]
 +
| <code>err</code>
 +
| <code><nowiki>http://www.w3.org/2005/xqt-errors</nowiki></code>
 +
|-
 +
| [[Serialization]]
 +
| <code>output</code>
 +
| <code><nowiki>http://www.w3.org/2010/xslt-xquery-serialization</nowiki></code>
 +
|}
 +
 
 +
=Suffixes=
 +
 
 +
In BaseX, files with the suffixes {{Code|.xq}}, {{Code|.xqm}}, {{Code|.xqy}}, {{Code|.xql}}, {{Code|.xqu}} and {{Code|.xquery}} are treated as XQuery files. In XQuery, there are main and library modules:
 +
 
 +
* Main modules have an expression as query body. Here is a minimum example:
 +
 
 +
<pre class="brush:xquery">
 +
'Hello World!'
 +
</pre>
 +
 
 +
* Library modules start with a module namespace declaration and have no query body:
 +
 
 +
<pre class="brush:xquery">
 +
module namespace hello = 'http://basex.org/examples/hello';
 +
 
 +
declare function hello:world() {
 +
  'Hello World!'
 +
};
 +
</pre>
 +
 
 +
We recommend {{Code|.xq}} as suffix for for main modules, and {{Code|.xqm}} for library modules. However, the actual module type will dynamically be detected when a file is opened and parsed.
 +
 
 +
=Miscellaneous=
 +
 
 +
Various other extensions are described in the articles on [[Full-Text#BaseX Features|XQuery Full Text]] and [[Updates|XQuery Update]].
 +
 
 +
=Changelog=
 +
 
 +
;Version 9.1:
-
Separate articles exist on [[Full-Text#BaseX Features|Full Text]] and [[Updates#Concepts|Update]] extensions.
+
* Added: New [[#Expressions|Expressions]]: Ternary if, elvis Operator, if without else
 +
* Added: XQuery Locks via pragmas and function annotations.
 +
* Added: [[#Regular expressions|Regular Expressions]], {{Code|j}} flag for using Java’s default regex parser.

Latest revision as of 11:03, 6 May 2019

This article is part of the XQuery Portal. It lists extensions and optimizations that are specific to the BaseX XQuery processor.

Contents

[edit] Expressions

Some of the extensions that have been added to BaseX may also be made available in other XQuery processors in the near future.

[edit] Ternary If

The ternary if operator provides a short syntax for conditions. It is also called conditional operator or ternary operator. In most languages, the syntax is a ? b : c. As ? and : have already been taken in XQuery, the syntax of Perl 6 is used:

$test ?? 'ok' !! 'fails'

The expression returns ok if the effective boolean value of $test is true, and it returns fails otherwise.

[edit] Elvis Operator

The Elvis operator is also available in other languages. It is sometimes called null-coalescing operator. In XQuery, the value of the first operand will be returned if it is a non-empty sequence. Otherwise, the value of the second operand will be returned.

let $number := 123
return (
  (: if/then/else :)
  if (exists($number)) then $number else 0,
  (: elvis operator :)
  $number ?: 0
)

The behavior of the operator is equivalent to the util:or function.

[edit] If Without Else

In XQuery 3.1, both branches of the if expression need to be specified. In many cases, only one branch is required, so the else branch was made optional in BaseX. If the second branch is omitted, an empty sequence will be returned if the effective boolean value of the test expression is false. Some examples:

if (doc-available($doc)) then doc($doc),
if (file:exists($file)) then file:delete($file),
if (permissions:valid($user)) then <html>Welcome!</html>

If conditions are nested, a trailing else branch will be associated with the innermost if:

if ($a) then if($b) then '$a and $b is true' else 'only $b is true'

In general, if you have multiple or nested if expressions, additional parentheses can improve the readibility of your code:

if ($a) then (
  if($b) then '$a and $b is true' else 'only $b is true'
)

The behavior of the if expression is equivalent to the util:if function.

[edit] Functions

[edit] Regular Expressions

In analogy with Saxon, you can specify the flag j to revert to Java’s default regex parser. For example, this allows you to use the word boundary option \b, which has not been included in the XQuery grammar for regular expressions:

Example:

(: yields "!Hi! !there!" :)
replace('Hi there', '\b', '!', 'j')

[edit] Serialization

For more information and some additional BaseX-specific parameters, see the article on Serialization.

[edit] Option Declarations

[edit] Database Options

Local database options can be set in the prolog of an XQuery main module. In the option declaration, options need to be bound to the Database Module namespace. All values will be reset after the evaluation of a query:

declare option db:chop 'false';
doc('doc.xml')

[edit] XQuery Locks

If XQuery Locks are defined in the query prolog of a module, access to functions of this module locks will be controlled by the central transaction management.

If the following XQuery code is called by two clients in parallel, the queries will be evaluated one after another:

declare option basex:write-lock 'CONFIGLOCK';
file:write('config.xml', <config/>)

[edit] Pragmas

[edit] BaseX Pragmas

Many optimizations in BaseX will only be performed if an expression is deterministic (i. e., if it always yields the same output and does not have side effects). By flagging an expression as non-deterministic, optimizations and query rewritings can be suppressed:

sum( (# basex:non-deterministic #) {
  1 to 100000000
})

This pragma can be helpful when debugging your code.

In analogy with option declarations and function annotations, XQuery Locks can also set via pragmas:

(# basex:write-lock CONFIGLOCK #) {
  file:write('config.xml', <config/>)
}

[edit] Database Pragmas

All local options can be assigned via pragmas. Some examples:

(# db:enforceindex #) {
  for $db in ('persons1', 'persons2', 'persons3')
  return db:open($db)//name[text() = 'John']
}
file:write(
  'wrapped-db-nodes.xml',
  (# db:copynode false #) {
    <xml>{ db:open('huge') }</xml>
  }
)

[edit] Annotations

[edit] Function Inlining

%basex:inline([limit]) controls if functions will be inlined.

If XQuery functions are inlined, the function call will be replaced by a FLWOR expression, in which the function variables are bound to let clauses, and in which the function body is returned. This optimization triggers further query rewritings that will speed up your query. An example:

Query:

declare function local:square($a) { $a * $a };
for $i in 1 to 3
return local:square($i)

Query after function inlining:

for $i in 1 to 3
return
  let $a := $i
  return $a * $a

Query after further optimizations:

for $i in 1 to 3
return $i * $i

By default, XQuery functions will be inlined if the query body is not too large and does not exceed a fixed number of expressions, which can be adjusted via the INLINELIMIT option.

The annotation can be used to overwrite this global limit: Function inlining can be enforced if no argument is specified. Inlining will be disabled if 0 is specified.

Example:

(: disable function inlining; the full stack trace will be shown... :)
declare %basex:inline(0) function local:e() { error() };
local:e()

Result:

Stopped at query.xq, 1/53:
[FOER0000] Halted on error().

Stack Trace:
- query.xq, 2/9

[edit] Lazy Evaluation

%basex:lazy enforces lazy evaluation of a global variable. An example:

Example:

declare %basex:lazy variable $january := doc('does-not-exist');
if(month-from-date(current-date()) = 1) then $january else ()

The annotation ensures that an error will only be raised if the condition yields true. Without the annotation, the error will always be raised, because the referenced document is not found.

[edit] XQuery Locks

In analogy with option declarations and pragmas, XQuery Locks can also set via annotations:

declare %basex:write-lock('CONFIGLOCK') function local:write() {
  file:write('config.xml', <config/>)
};

[edit] Non-Determinism

In XQuery, deterministic functions are “guaranteed to produce ·identical· results from repeated calls within a single ·execution scope· if the explicit and implicit arguments are identical”. In BaseX, many extension functions are non-deterministic or side-effecting. If an expression is internally flagged as non-deterministic, various optimizations that might change their execution order will not be applied.

(: QUERY A... :)
let $n := 456
for $i in 1 to 2
return $n

(: ...will be optimized to :)
for $i in 1 to 2
return 456

(: QUERY B will not be rewritten :)
let $n := random:integer()
for $i in 1 to 2
return $n

In some cases, functions may contain non-deterministic code, but the query compiler may not be able to detect this statically. See the following example:

for $read in (file:read-text#1, file:read-binary#1)
let $ignored := non-deterministic $read('input.file')
return ()

Two non-deterministic functions will be bound to $read, and the result of the function call will be bound to $ignored. As the variable is not referenced in the subsequent code, the let clause would usually be discarded by the compiler. In the given query, however, execution will be enforced because of the BaseX-specific non-deterministic keyword.

[edit] Namespaces

In XQuery, some namespaces are statically bound to prefixes. The following query requires no additional namespaces declarations in the query prolog:

<xml:abc xmlns:prefix='uri' local:fn='x'/>,
fn:exists(1)

In BaseX, various other namespaces are predefined. Apart from the namespaces that are listed on the Module Library page, the following namespaces are statically bound:

Description Prefix Namespace URI
BaseX Annotations, Pragmas, … basex http://basex.org
RESTXQ: Input Options input http://basex.org/modules/input
EXPath Packages pkg http://expath.org/ns/pkg
XQuery Errors err http://www.w3.org/2005/xqt-errors
Serialization output http://www.w3.org/2010/xslt-xquery-serialization

[edit] Suffixes

In BaseX, files with the suffixes .xq, .xqm, .xqy, .xql, .xqu and .xquery are treated as XQuery files. In XQuery, there are main and library modules:

'Hello World!'
module namespace hello = 'http://basex.org/examples/hello';

declare function hello:world() {
  'Hello World!'
};

We recommend .xq as suffix for for main modules, and .xqm for library modules. However, the actual module type will dynamically be detected when a file is opened and parsed.

[edit] Miscellaneous

Various other extensions are described in the articles on XQuery Full Text and XQuery Update.

[edit] Changelog

Version 9.1
Personal tools
Namespaces
Variants
Actions
Navigation
Print/export