Difference between revisions of "XQuery Extensions"

From BaseX Documentation
Jump to navigation Jump to search
m (Text replacement - "syntaxhighlight" to "pre")
 
(9 intermediate revisions by the same user not shown)
Line 3: Line 3:
 
=Expressions=
 
=Expressions=
  
−
Some of the extensions that have been added to BaseX may also be made available in other XQuery processors in the near future.  
+
{{Announce|Removed with Version 11}}: Elvis operator <code>?:</code>, in favor of the new <code>[https://qt4cg.org/specifications/xquery-40/xquery-40.html#id-otherwise otherwise]</code> expression.
  
 
==Ternary If==
 
==Ternary If==
Line 9: Line 9:
 
The [https://en.wikipedia.org/wiki/%3F: ternary if] operator provides a short syntax for conditions. It is also called '''conditional operator''' or '''ternary operator'''. In most languages, the syntax is <code>a ? b : c</code>. As <code>?</code> and <code>:</code> have already been taken in XQuery, the syntax of Perl 6 is used:
 
The [https://en.wikipedia.org/wiki/%3F: ternary if] operator provides a short syntax for conditions. It is also called '''conditional operator''' or '''ternary operator'''. In most languages, the syntax is <code>a ? b : c</code>. As <code>?</code> and <code>:</code> have already been taken in XQuery, the syntax of Perl 6 is used:
  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
−
$test ?? 'ok' !! 'fails'
+
(: if/then/else :)
−
</syntaxhighlight>
+
if ($ok) then 1 else 0,
 +
(: ternary if :)
 +
$ok ?? 1 !! 0
 +
</pre>
  
 
The expression returns <code>ok</code> if the effective boolean value of <code>$test</code> is true, and it returns <code>fails</code> otherwise.
 
The expression returns <code>ok</code> if the effective boolean value of <code>$test</code> is true, and it returns <code>fails</code> otherwise.
−
 
−
==Elvis Operator==
 
−
 
−
The Elvis operator is also available in other languages. It is sometimes called [https://en.wikipedia.org/wiki/Null_coalescing_operator null-coalescing operator]. In XQuery, the value of the first operand will be returned if it is a non-empty sequence. Otherwise, the value of the second operand will be returned.
 
−
 
−
<syntaxhighlight lang="xquery">
 
−
let $number := 123
 
−
return (
 
−
  (: if/then/else :)
 
−
  if (exists($number)) then $number else 0,
 
−
  (: elvis operator :)
 
−
  $number ?: 0
 
−
)
 
−
</syntaxhighlight>
 
−
 
−
The behavior of the operator is equivalent to the {{Function|Utility|util:or}} function.
 
  
 
==If Without Else==
 
==If Without Else==
Line 35: Line 22:
 
In XQuery 3.1, both branches of the <code>if</code> expression need to be specified. In many cases, only one branch is required, so the <code>else</code> branch was made optional in BaseX. If the second branch is omitted, an empty sequence will be returned if the effective boolean value of the test expression is false. Some examples:
 
In XQuery 3.1, both branches of the <code>if</code> expression need to be specified. In many cases, only one branch is required, so the <code>else</code> branch was made optional in BaseX. If the second branch is omitted, an empty sequence will be returned if the effective boolean value of the test expression is false. Some examples:
  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
 
if (doc-available($doc)) then doc($doc),
 
if (doc-available($doc)) then doc($doc),
 
if (file:exists($file)) then file:delete($file),
 
if (file:exists($file)) then file:delete($file),
 
if (permissions:valid($user)) then <html>Welcome!</html>
 
if (permissions:valid($user)) then <html>Welcome!</html>
−
</syntaxhighlight>
+
</pre>
  
 
If conditions are nested, a trailing else branch will be associated with the innermost <code>if</code>:
 
If conditions are nested, a trailing else branch will be associated with the innermost <code>if</code>:
  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
 
if ($a) then if($b) then '$a and $b is true' else 'only $a is true'
 
if ($a) then if($b) then '$a and $b is true' else 'only $a is true'
−
</syntaxhighlight>
+
</pre>
  
 
In general, if you have multiple or nested if expressions, additional parentheses can improve the readibility of your code:
 
In general, if you have multiple or nested if expressions, additional parentheses can improve the readibility of your code:
  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
 
if ($a) then (
 
if ($a) then (
 
   if($b) then '$a and $b is true' else 'only $a is true'
 
   if($b) then '$a and $b is true' else 'only $a is true'
 
)
 
)
−
</syntaxhighlight>
+
</pre>
−
 
 
−
The behavior of the if expression is equivalent to the {{Function|Utility|util:if}} function.
 
  
 
=Functions=
 
=Functions=
Line 64: Line 49:
  
 
'''Example:'''  
 
'''Example:'''  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
 
(: yields "!Hi! !there!" :)
 
(: yields "!Hi! !there!" :)
 
replace('Hi there', '\b', '!', 'j')
 
replace('Hi there', '\b', '!', 'j')
−
</syntaxhighlight>
+
</pre>
  
 
=Serialization=
 
=Serialization=
Line 83: Line 68:
 
[[Options|Local database options]] can be set in the prolog of an XQuery main module. In the option declaration, options need to be bound to the [[Database Module]] namespace. All values will be reset after the evaluation of a query:
 
[[Options|Local database options]] can be set in the prolog of an XQuery main module. In the option declaration, options need to be bound to the [[Database Module]] namespace. All values will be reset after the evaluation of a query:
  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
 
declare option db:catalog 'etc/w3-catalog.xml';
 
declare option db:catalog 'etc/w3-catalog.xml';
 
doc('doc.xml')
 
doc('doc.xml')
−
</syntaxhighlight>
+
</pre>
  
 
==XQuery Locks==
 
==XQuery Locks==
Line 96: Line 81:
 
==BaseX Pragmas==
 
==BaseX Pragmas==
  
−
Many optimizations in BaseX will only be performed if an expression is ''deterministic'' (i. e., if it always yields the same output and does not have side effects). By flagging an expression as non-deterministic, optimizations and query rewritings can be suppressed:
+
{{Announce|Updated with Version 11}}: Renamed from {{Code|non-deterministic}} to {{Code|nondeterministic}}.
 +
 
 +
Many optimizations in BaseX will only be performed if an expression is ''deterministic'' (i. e., if it always yields the same output and does not have side effects). By flagging an expression as nondeterministic, optimizations and query rewritings can be suppressed:
  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
−
sum( (# basex:non-deterministic #) {
+
sum( (# basex:nondeterministic #) {
 
   1 to 100000000
 
   1 to 100000000
 
})
 
})
−
</syntaxhighlight>
+
</pre>
  
 
This pragma can be helpful when debugging your code.
 
This pragma can be helpful when debugging your code.
Line 108: Line 95:
 
In analogy with option declarations and function annotations, XQuery locks can also set via pragmas. See [[Transaction Management#Options|Transaction Management]] for details and examples.
 
In analogy with option declarations and function annotations, XQuery locks can also set via pragmas. See [[Transaction Management#Options|Transaction Management]] for details and examples.
  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
 
(# basex:write-lock CONFIGLOCK #) {
 
(# basex:write-lock CONFIGLOCK #) {
 
   file:write('config.xml', <config/>)
 
   file:write('config.xml', <config/>)
 
}
 
}
−
</syntaxhighlight>
+
</pre>
  
 
==Database Pragmas==
 
==Database Pragmas==
Line 120: Line 107:
 
* [[Indexes|Index access rewritings]] can be enforced. This is helpful if the name of a database is not static (see [[Indexes#Enforce Rewritings|Enforce Rewritings]] for more details):
 
* [[Indexes|Index access rewritings]] can be enforced. This is helpful if the name of a database is not static (see [[Indexes#Enforce Rewritings|Enforce Rewritings]] for more details):
  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
 
(# db:enforceindex #) {
 
(# db:enforceindex #) {
 
   for $db in ('persons1', 'persons2', 'persons3')
 
   for $db in ('persons1', 'persons2', 'persons3')
−
   return db:pre($db)//name[text() = 'John']
+
   return db:get($db)//name[text() = 'John']
 
}
 
}
−
</syntaxhighlight>
+
</pre>
  
 
* Node copying in node constructors can be disabled (see {{Option|COPYNODE}} for more details). The following query will consume much less memory than without pragma as the database nodes will not be fully duplicated, but only attached to the {{Code|xml}} parent element:
 
* Node copying in node constructors can be disabled (see {{Option|COPYNODE}} for more details). The following query will consume much less memory than without pragma as the database nodes will not be fully duplicated, but only attached to the {{Code|xml}} parent element:
  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
 
file:write(
 
file:write(
 
   'wrapped-db-nodes.xml',
 
   'wrapped-db-nodes.xml',
 
   (# db:copynode false #) {
 
   (# db:copynode false #) {
−
     <xml>{ db:pre('huge') }</xml>
+
     <xml>{ db:get('huge') }</xml>
 
   }
 
   }
 
)
 
)
−
</syntaxhighlight>
+
</pre>
  
 
* An XML catalog can be specified for URI rewritings. See the [[Catalog Resolver]] section for an example.
 
* An XML catalog can be specified for URI rewritings. See the [[Catalog Resolver]] section for an example.
Line 150: Line 137:
 
'''Query:'''
 
'''Query:'''
  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
 
declare function local:square($a) { $a * $a };
 
declare function local:square($a) { $a * $a };
 
for $i in 1 to 3
 
for $i in 1 to 3
 
return local:square($i)
 
return local:square($i)
−
</syntaxhighlight>
+
</pre>
  
 
'''Query after function inlining:'''
 
'''Query after function inlining:'''
  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
 
for $i in 1 to 3
 
for $i in 1 to 3
 
return
 
return
 
   let $a := $i
 
   let $a := $i
 
   return $a * $a
 
   return $a * $a
−
</syntaxhighlight>
+
</pre>
  
 
'''Query after further optimizations:'''
 
'''Query after further optimizations:'''
  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
 
for $i in 1 to 3
 
for $i in 1 to 3
 
return $i * $i
 
return $i * $i
−
</syntaxhighlight>
+
</pre>
  
 
By default, XQuery functions will be ''inlined'' if the query body is not too large and does not exceed a fixed number of expressions, which can be adjusted via the {{Option|INLINELIMIT}} option.
 
By default, XQuery functions will be ''inlined'' if the query body is not too large and does not exceed a fixed number of expressions, which can be adjusted via the {{Option|INLINELIMIT}} option.
Line 178: Line 165:
 
'''Example:'''
 
'''Example:'''
  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
 
(: disable function inlining; the full stack trace will be shown... :)
 
(: disable function inlining; the full stack trace will be shown... :)
 
declare %basex:inline(0) function local:e() { error() };
 
declare %basex:inline(0) function local:e() { error() };
 
local:e()
 
local:e()
−
</syntaxhighlight>
+
</pre>
  
 
'''Result:'''
 
'''Result:'''
  
−
<syntaxhighlight lang="xml">
+
<pre lang="xml">
 
Stopped at query.xq, 1/53:
 
Stopped at query.xq, 1/53:
 
[FOER0000] Halted on error().
 
[FOER0000] Halted on error().
Line 192: Line 179:
 
Stack Trace:
 
Stack Trace:
 
- query.xq, 2/9
 
- query.xq, 2/9
−
</syntaxhighlight>
+
</pre>
  
 
==Lazy Evaluation==
 
==Lazy Evaluation==
Line 199: Line 186:
  
 
'''Example:'''  
 
'''Example:'''  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
 
declare %basex:lazy variable $january := doc('does-not-exist.xml');
 
declare %basex:lazy variable $january := doc('does-not-exist.xml');
 
if(month-from-date(current-date()) = 1) then $january else ()
 
if(month-from-date(current-date()) = 1) then $january else ()
−
</syntaxhighlight>
+
</pre>
  
 
The annotation ensures that an error is only raised if the condition yields true. Without the annotation, the error is always raised if the referenced document is not found.
 
The annotation ensures that an error is only raised if the condition yields true. Without the annotation, the error is always raised if the referenced document is not found.
Line 209: Line 196:
  
 
In analogy with option declarations and pragmas, locks can also set via annotations. See [[Transaction Management#Annotations|Transaction Management]] for details and examples.
 
In analogy with option declarations and pragmas, locks can also set via annotations. See [[Transaction Management#Annotations|Transaction Management]] for details and examples.
−
 
−
=Non-Determinism=
 
−
 
−
In [https://www.w3.org/TR/xpath-functions-31/#dt-deterministic XQuery], ''deterministic'' functions are “guaranteed to produce ·identical· results from repeated calls within a single ·execution scope· if the explicit and implicit arguments are identical”. In BaseX, many extension functions are non-deterministic or side-effecting. If an expression is internally flagged as non-deterministic, various optimizations that might change their execution order will not be applied.
 
−
 
−
<syntaxhighlight lang="xquery">
 
−
(: QUERY A... :)
 
−
let $n := 456
 
−
for $i in 1 to 2
 
−
return $n
 
−
 
−
(: ...will be optimized to :)
 
−
for $i in 1 to 2
 
−
return 456
 
−
 
−
(: QUERY B will not be rewritten :)
 
−
let $n := random:integer()
 
−
for $i in 1 to 2
 
−
return $n
 
−
</syntaxhighlight>
 
−
 
−
In some cases, functions may contain non-deterministic code, but the query compiler may not be able to detect this statically. See the following example:
 
−
 
−
<syntaxhighlight lang="xquery">
 
−
for $read in (file:read-text#1, file:read-binary#1)
 
−
let $ignored := non-deterministic $read('input.file')
 
−
return ()
 
−
</syntaxhighlight>
 
−
 
−
Two non-deterministic functions will be bound to <code>$read</code>, and the result of the function call will be bound to <code>$ignored</code>. As the variable is not referenced in the subsequent code, the let clause would usually be discarded by the compiler. In the given query, however, execution will be enforced because of the BaseX-specific {{Code|non-deterministic}} keyword.
 
  
 
=Namespaces=
 
=Namespaces=
Line 244: Line 201:
 
In XQuery, some namespaces are statically bound to prefixes. The following query requires no additional namespaces declarations in the query prolog:
 
In XQuery, some namespaces are statically bound to prefixes. The following query requires no additional namespaces declarations in the query prolog:
  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
 
<xml:abc xmlns:prefix='uri' local:fn='x'/>,
 
<xml:abc xmlns:prefix='uri' local:fn='x'/>,
 
fn:exists(1)
 
fn:exists(1)
−
</syntaxhighlight>
+
</pre>
  
 
In BaseX, various other namespaces are predefined. Apart from the namespaces that are listed on the [[Module Library]] page, the following namespaces are statically bound:
 
In BaseX, various other namespaces are predefined. Apart from the namespaces that are listed on the [[Module Library]] page, the following namespaces are statically bound:
Line 284: Line 241:
 
* Main modules have an expression as query body. Here is a minimum example:
 
* Main modules have an expression as query body. Here is a minimum example:
  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
 
'Hello World!'
 
'Hello World!'
−
</syntaxhighlight>
+
</pre>
  
 
* Library modules start with a module namespace declaration and have no query body:
 
* Library modules start with a module namespace declaration and have no query body:
  
−
<syntaxhighlight lang="xquery">
+
<pre lang='xquery'>
 
module namespace hello = 'http://basex.org/examples/hello';
 
module namespace hello = 'http://basex.org/examples/hello';
  
Line 296: Line 253:
 
   'Hello World!'
 
   'Hello World!'
 
};
 
};
−
</syntaxhighlight>
+
</pre>
  
 
We recommend {{Code|.xq}} as suffix for for main modules, and {{Code|.xqm}} for library modules. However, the actual module type will dynamically be detected when a file is opened and parsed.
 
We recommend {{Code|.xq}} as suffix for for main modules, and {{Code|.xqm}} for library modules. However, the actual module type will dynamically be detected when a file is opened and parsed.
Line 305: Line 262:
  
 
=Changelog=
 
=Changelog=
 +
 +
;Version 11:
 +
 +
* Removed: Elvis operator <code>?:</code>, in favor of the new <code>[https://qt4cg.org/specifications/xquery-40/xquery-40.html#id-otherwise otherwise]</code> expression.
 +
* Updated: Renamed from {{Code|non-deterministic}} to {{Code|nondeterministic}}.
  
 
;Version 9.1:
 
;Version 9.1:

Latest revision as of 18:39, 1 December 2023

This article is part of the XQuery Portal. It lists extensions and optimizations that are specific to the BaseX XQuery processor.

Expressions[edit]

Removed with Version 11: Elvis operator ?:, in favor of the new otherwise expression.

Ternary If[edit]

The ternary if operator provides a short syntax for conditions. It is also called conditional operator or ternary operator. In most languages, the syntax is a ? b : c. As ? and : have already been taken in XQuery, the syntax of Perl 6 is used: 

(: if/then/else :)
if ($ok) then 1 else 0,
(: ternary if :)
$ok ?? 1 !! 0

The expression returns ok if the effective boolean value of $test is true, and it returns fails otherwise.

If Without Else[edit]

In XQuery 3.1, both branches of the if expression need to be specified. In many cases, only one branch is required, so the else branch was made optional in BaseX. If the second branch is omitted, an empty sequence will be returned if the effective boolean value of the test expression is false. Some examples: 

if (doc-available($doc)) then doc($doc),
if (file:exists($file)) then file:delete($file),
if (permissions:valid($user)) then <html>Welcome!</html>

If conditions are nested, a trailing else branch will be associated with the innermost if: 

if ($a) then if($b) then '$a and $b is true' else 'only $a is true'

In general, if you have multiple or nested if expressions, additional parentheses can improve the readibility of your code: 

if ($a) then (
  if($b) then '$a and $b is true' else 'only $a is true'
)

Functions[edit]

Regular Expressions[edit]

In analogy with Saxon, you can specify the flag j to revert to Java’s default regex parser. For example, this allows you to use the word boundary option \b, which has not been included in the XQuery grammar for regular expressions:

Example: 

(: yields "!Hi! !there!" :)
replace('Hi there', '\b', '!', 'j')

Serialization[edit]

  • basexis used as the default serialization method: nodes are serialized as XML, atomic values are serialized as string, and items of binary type are output in their native byte representation. Function items (including maps and arrays) are output just like with the adaptive method.
  • With csv, you can output XML nodes as CSV data (see the CSV Module for more details).
  • With json, items are output as JSON as described in the official specification. If the root node is of type element(json), items are serialized as described for the direct format in the JSON Module.

For more information and some additional BaseX-specific parameters, see the article on Serialization.

Option Declarations[edit]

Database Options[edit]

Local database options can be set in the prolog of an XQuery main module. In the option declaration, options need to be bound to the Database Module namespace. All values will be reset after the evaluation of a query: 

declare option db:catalog 'etc/w3-catalog.xml';
doc('doc.xml')

XQuery Locks[edit]

If locks are declared in the query prolog of a module via the basex:lock option, access to functions of this module locks will be controlled by the central transaction management. See Transaction Management for further details.

Pragmas[edit]

BaseX Pragmas[edit]

Updated with Version 11: Renamed from non-deterministic to nondeterministic.

Many optimizations in BaseX will only be performed if an expression is deterministic (i. e., if it always yields the same output and does not have side effects). By flagging an expression as nondeterministic, optimizations and query rewritings can be suppressed: 

sum( (# basex:nondeterministic #) {
  1 to 100000000
})

This pragma can be helpful when debugging your code.

In analogy with option declarations and function annotations, XQuery locks can also set via pragmas. See Transaction Management for details and examples. 

(# basex:write-lock CONFIGLOCK #) {
  file:write('config.xml', <config/>)
}

Database Pragmas[edit]

Local database options can also be assigned via pragmas: 

(# db:enforceindex #) {
  for $db in ('persons1', 'persons2', 'persons3')
  return db:get($db)//name[text() = 'John']
}
  • Node copying in node constructors can be disabled (see COPYNODE for more details). The following query will consume much less memory than without pragma as the database nodes will not be fully duplicated, but only attached to the xml parent element:
file:write(
  'wrapped-db-nodes.xml',
  (# db:copynode false #) {
    <xml>{ db:get('huge') }</xml>
  }
)
  • An XML catalog can be specified for URI rewritings. See the Catalog Resolver section for an example.

Annotations[edit]

Function Inlining[edit]

%basex:inline([limit]) controls if functions will be inlined.

If XQuery functions are inlined, the function call will be replaced by a FLWOR expression, in which the function variables are bound to let clauses, and in which the function body is returned. This optimization triggers further query rewritings that will speed up your query. An example:

Query: 

declare function local:square($a) { $a * $a };
for $i in 1 to 3
return local:square($i)

Query after function inlining: 

for $i in 1 to 3
return
  let $a := $i
  return $a * $a

Query after further optimizations: 

for $i in 1 to 3
return $i * $i

By default, XQuery functions will be inlined if the query body is not too large and does not exceed a fixed number of expressions, which can be adjusted via the INLINELIMIT option.

The annotation can be used to overwrite this global limit: Function inlining can be enforced if no argument is specified. Inlining will be disabled if 0 is specified.

Example: 

(: disable function inlining; the full stack trace will be shown... :)
declare %basex:inline(0) function local:e() { error() };
local:e()

Result: 

Stopped at query.xq, 1/53:
[FOER0000] Halted on error().

Stack Trace:
- query.xq, 2/9

Lazy Evaluation[edit]

%basex:lazy enforces lazy evaluation of a global variable. An example:

Example: 

declare %basex:lazy variable $january := doc('does-not-exist.xml');
if(month-from-date(current-date()) = 1) then $january else ()

The annotation ensures that an error is only raised if the condition yields true. Without the annotation, the error is always raised if the referenced document is not found.

XQuery Locks[edit]

In analogy with option declarations and pragmas, locks can also set via annotations. See Transaction Management for details and examples.

Namespaces[edit]

In XQuery, some namespaces are statically bound to prefixes. The following query requires no additional namespaces declarations in the query prolog: 

<xml:abc xmlns:prefix='uri' local:fn='x'/>,
fn:exists(1)

In BaseX, various other namespaces are predefined. Apart from the namespaces that are listed on the Module Library page, the following namespaces are statically bound:

Description Prefix Namespace URI
BaseX Annotations, Pragmas, … basex http://basex.org
RESTXQ: Input Options input http://basex.org/modules/input
EXPath Packages pkg http://expath.org/ns/pkg
XQuery Errors err http://www.w3.org/2005/xqt-errors
Serialization output http://www.w3.org/2010/xslt-xquery-serialization

Suffixes[edit]

In BaseX, files with the suffixes .xq, .xqm, .xqy, .xql, .xqu and .xquery are treated as XQuery files. In XQuery, there are main and library modules:

  • Main modules have an expression as query body. Here is a minimum example:
'Hello World!'
  • Library modules start with a module namespace declaration and have no query body:
module namespace hello = 'http://basex.org/examples/hello';

declare function hello:world() {
  'Hello World!'
};

We recommend .xq as suffix for for main modules, and .xqm for library modules. However, the actual module type will dynamically be detected when a file is opened and parsed.

Miscellaneous[edit]

Various other extensions are described in the articles on XQuery Full Text and XQuery Update.

Changelog[edit]

Version 11
  • Removed: Elvis operator ?:, in favor of the new otherwise expression.
  • Updated: Renamed from non-deterministic to nondeterministic.
Version 9.1
  • Added: New Expressions: Ternary if, elvis Operator, if without else
  • Added: XQuery Locks via pragmas and function annotations.
  • Added: Regular Expressions, j flag for using Java’s default regex parser.
Retrieved from "http://docs.basex.org/index.php?title=XQuery_Extensions&oldid=16885"