Main Page » XQuery » XQuery Extensions

XQuery Extensions

This article lists extensions and optimizations that are specific to the BaseX XQuery processor.

Expressions

If Without Else

In XQuery 4.0, two variants of the if expression exist:

  • If the condition is followed by then, the else branch is mandatory.
  • If the condition is followed by curly braces, the else branch must be omitted.

With BaseX, the else branch can be specified or omitted with both constructs. Examples:

if (doc-available($doc)) then doc($doc),
if (file:exists($file)) then file:delete($file),
if (permissions:valid($user)) { <html>Welcome!</html> } else { <html>Please log in</html> }

If conditions are nested, a trailing else branch will be associated with the innermost if:

if ($a) then if ($b) then '$a and $b is true' else 'only $a is true'

Serialization

  • basex is used as the default serialization method: Nodes are serialized as XML, atomic values are serialized as strings, and binary items are output in their native byte representation. Function items (including maps and arrays) are output as with the Adaptive Serialization method. JNodes are serialized so that their keys are preserved.
  • With json, if the root node is of type element(json), items are serialized as described for JSON Functions and the direct format.

For more information and some additional BaseX-specific parameters, see the article on Serialization.

Debugging

Various functions generate diagnostic output: fn:trace, fn:message, prof:time, prof:memory, prof:type and prof:variables, and fn:deep-equal if its debug option is enabled.

The destination of this output depends on the context in which a query is evaluated:

Context Destination Limit
Command Line Standard error
Graphical User Interface Info View 50000 messages, 10 MB
Client/Server Query information, returned to the client 10000 messages
Web Application Database logs 100 messages

In a web application, the output can be redirected to standard error by disabling the LOGTRACE option.

As the destinations are reserved for administrators, output is only generated for users with CREATE permission. For all other users, it is discarded. This is particularly relevant if queries of untrusted users are evaluated with xquery:eval and reduced permissions.

As the generated output is a side effect, all of these functions are nondeterministic: their calls will not be pre-evaluated at compile time, and they will not be discarded if their result is not used.

Option Declarations

Local database options can be set in the prolog of an XQuery main module. In the option declaration, options need to be bound to the Database Functions namespace. All values will be reset after the evaluation of a query: 
declare option db:catalog 'etc/w3-catalog.xml';
doc('doc.xml')

Pragmas

BaseX Pragmas

Two pragmas are bound to the basex prefix: basex:nondeterministic and basex:lock.

Many optimizations in BaseX will only be performed if an expression is deterministic (i.e., if it always yields the same output and does not have side effects). With basex:nondeterministic, an expression is flagged as nondeterministic, and optimizations and query rewritings are suppressed:

sum(
  (# basex:nondeterministic #) { 1 to 100000000 }
)

This pragma can be helpful when debugging your code.

The basex:lock pragma is described in XQuery Locks.

Database Pragmas

Local database options can also be assigned via pragmas:
  • Index access rewritings can be enforced. This is helpful if the name of a database is not static (see Enforce Rewritings for more details): 
    (# db:enforceindex #) {
  for $db in ('persons1', 'persons2', 'persons3')
  return db:get($db)//name[text() = 'John']
}
  • Node copying in node constructors can be disabled (see COPYNODE for more details). Constructors whose results are only serialized do not copy nodes anyway (see Node Construction). In the following query, the constructed element is bound to a variable that is used twice. With the pragma, the database nodes will not be fully duplicated, but only attached to the xml parent element: 
    let $xml := (# db:copynode false #) {
  <xml>{ db:get('huge') }</xml>
}
return (
  file:write('wrapped-db-nodes.xml', $xml),
  count($xml//*)
)
  • An XML catalog can be specified for URI rewritings. See the Catalog Resolver section for an example.

Annotations

The following sections describe the annotations that are bound to the basex prefix; %basex:lock is described in XQuery Locks. Various other annotations are supplied by BaseX:

  • %rest:…: RESTXQ services.
  • %input:csv, %input:html, %input:json: parsers for request bodies (see Content Types).
  • %output:…: serialization parameters for responses (see Output).
  • %perm:allow, %perm:check: Permissions of RESTXQ functions.
  • %ws:…: WebSockets services.
  • %unit:…: Unit Functions for unit tests.

Function Inlining

%basex:inline([limit]) specifies whether functions will be inlined.

If XQuery functions are inlined, the function call is replaced by a FLWOR expression; see Function Inlining for the rewriting and its effects.

By default, XQuery functions will be inlined if the query body is not too large and does not exceed a fixed number of expressions, which can be adjusted via the INLINELIMIT option.

The annotation can be used to override this global limit: Function inlining can be enforced if no argument is specified. Inlining will be disabled if 0 is specified.

Example: 
(: disable function inlining; the full stack trace will be shown... :)
declare %basex:inline(0) function local:e() { error() };
local:e()
Result: 
Stopped at local:e#0 (query.xq, 2/52):
[FOER0000] Halted on error().

Stack Trace:
- query.xq, 3/8

Lazy Evaluation

%basex:lazy enforces lazy evaluation of a global variable.

Example: 
declare %basex:lazy variable $january := doc('does-not-exist.xml');
if (month-from-date(current-date()) = 1) then $january else ()

The annotation ensures that an error is only raised if the condition yields true. Without the annotation, the error is always raised if the referenced document is not found.

Errors that are raised while a global variable is evaluated cannot be caught with try/catch: a variable without the annotation is evaluated before the query body is run, and the error of a lazy variable can surface at any point of the evaluation.

Example: 
declare variable $error := 1 idiv 0;
try { $error } catch * { 'not reached' }
Result: 
Stopped at $error (query.xq, 1/35):
[FOAR0001] 1 cannot be divided by zero.

Memoization

Added: New annotation.

%basex:memo caches the results of a function: if the function is called again with the same arguments, the cached result is returned and the function body is not evaluated again. Results are cached until the query has been evaluated.

Example: 
declare %basex:memo function local:fib($n as xs:integer) as xs:integer {
  if ($n < 2) then $n else local:fib($n - 1) + local:fib($n - 2)
};
local:fib(90)
Result: 
2880067194370816120

Without the annotation, the number of function calls grows exponentially, and the query would not terminate in reasonable time. With the annotation, local:fib is evaluated once for each number from 0 to 90.

Memoization also pays off if an expensive function is called many times with a small number of distinct arguments:

Example: 
declare %basex:memo function local:key($name as xs:string) as xs:string {
  $name => normalize-unicode('NFKD') => replace('\p{Mn}', '') => lower-case() => normalize-space()
};
distinct-values(db:get('customers')//name ! local:key(.))

The function is evaluated once for each distinct name, no matter how often the name occurs in the database.

Arguments are compared after they have been converted to the parameter types. Atomic values must have the same type and value: 1 and 1.0 are cached separately, as are dates with different timezones. Nodes are compared by identity.

Memoization never changes the result of a query. A basex:annotation error is raised if the annotated function is nondeterministic, or if it constructs nodes and does not declare an atomic return type (cached nodes would have the same identity). Memoized functions are not inlined.

Evaluation in Other Queries

Added: New annotation.

%basex:eval indicates that a function will be evaluated by another query. Its body is compiled without accessing the dynamic context of the calling query: databases are not opened, and functions like fn:current-dateTime are evaluated by the other query.

The annotation is needed if a function item is supplied to xquery:eval, job:eval or another function that invokes it in a separate query context (see Function Items). Inline functions that are directly supplied as function or bindings argument are annotated automatically. Functions that are referenced via variables or function calls must be annotated explicitly. Otherwise, the body of the function may be optimized with the databases of the calling query, and a basex:eval error is raised when it is passed on.

Example: 
declare %basex:inline(0) function local:run($f) { xquery:eval($f) };
let $x := db:get('db')//a[1]/string()
return local:run(%basex:eval fn() { db:get('db')//a[text() = $x] })

Without the annotation, the function body would be rewritten for index access, and the function could not be evaluated by the other query.

If the annotation is attached to a function declaration, it applies to all references of the function. As a consequence, calls of the function that are not inlined are not rewritten for index access either.

XQuery Locks

Locks can be declared with the basex:lock option in the prolog of a module, with the %basex:lock function annotation, and with the basex:lock pragma for a single expression. Access is then controlled by the central transaction management. See Transaction Management for details and examples.

Namespaces

In XQuery, some namespaces are statically bound to prefixes. The following query requires no additional namespaces declarations in the query prolog:

<xml:abc xmlns:prefix='uri' local:fn='x'/>,
fn:exists(1)

In BaseX, various other namespaces are predefined. Apart from the namespaces that are listed on the Functions page, the following namespaces are statically bound:

Description Prefix Namespace URI
BaseX Annotations, Pragmas, … basex http://basex.org
RESTXQ: Input Options input http://basex.org/modules/input
EXPath Packages pkg http://expath.org/ns/pkg
Java Bindings java http://basex.org/modules/java
Permissions perm http://basex.org/modules/perm
Errors of EXPath modules (File, HTTP Client, …) experr http://expath.org/ns/error

Suffixes

In BaseX, files with the suffixes .xq, .xqm, .xqy, .xql, .xqu, .xquery and .xpath are treated as XQuery files. We recommend .xq as suffix for main modules, and .xqm for library modules. However, the actual module type will dynamically be detected when a file is opened and parsed.

Grammar Extensions

Here is a summary of our current extensions to the XQuery grammar:
  • An FTFuzzyOption alternative is added to FtMatchOption, defined as FTFuzzyOption ::= "fuzzy" (IntegerLiteral "errors")?.
  • In UpdatingFunctionCall, the keyword invoke is optional.
  • In UnbracedActions, the else branch for if is optional.
  • BracedAction is replaced by BracedActions, with optional else if and else branches.
  • A CoerceExpr rule is inserted between TreatExpr and CastableExpr, defined as CoerceExpr ::= CastableExpr ("coerce" "to" SequenceType)?.
  • For TransformWithExpr, the keyword update is added as an alternative.
  • UnreservedName is removed: all names are allowed in CompNodeName.
  • Both node and nodes can be used in RenameExpr and ReplaceExpr.

Miscellaneous

Various other extensions are described in the articles on Java Bindings, XQuery Full Text and XQuery Update.

Changelog

Version 13.0
  • Added: %basex:memo annotation for the Memoization of function results.
  • Added: %basex:eval annotation for the Evaluation in Other Queries.
  • Removed: The regular expression flags j, ! and ;.
  • Removed: The nondeterministic prefix for dynamic function calls: calls of functions that are statically unknown are now treated as nondeterministic.
Version 12.0
  • Removed: Stack trace output with $err:additional; replaced with $err:stack-trace.
Version 11.0
  • Updated: Renamed from non-deterministic to nondeterministic.
  • Removed: Elvis operator ?:, in favor of the new otherwise expression.
  • Removed: Ternary if A ?? B !! C.
Version 9.1
  • Added: New Expressions: Ternary if, Elvis operator, if without else
  • Added: XQuery Locks via pragmas and function annotations.
  • Added: Regular expressions: j flag for using Java’s default regex parser.
⚡Generated with XQuery
 
XQuery Extensions