https://docs.basex.org/api.php?action=feedcontributions&user=CG&feedformat=atomBaseX Documentation - User contributions [en]2024-03-29T09:23:00ZUser contributionsMediaWiki 1.34.0https://docs.basex.org/index.php?title=Main_Page&diff=16924Main Page2024-03-01T09:16:27Z<p>CG: </p>
<hr />
<div><div style="float:left;"><br />
[[Image:BaseX-Screenshot.png|thumb|240px|BaseX, Graphical User Interface]]<br />
<br />
'''NOTE:''' This is the documentation for '''[[BaseX 10]]'''. The documentation for BaseX 11 beta is currently located at https://help.basex.org.<br />
<br />
[https://basex.org/ BaseX] is a light-weight, high-performance and scalable XML Database and an XQuery 3.1 Processor with full support for the W3C Update and Full-Text extensions. It allows you to store, query and process large corpora of textual (XML, HTML, JSON, CSV, others) and binary data. The GUI provides an XQuery editor for writing complex applications and provides various visualizations to interactively explore data. RESTXQ enables Web Application development in XQuery. BaseX is platform-independent BSD-licensed (find more on [https://en.wikipedia.org/wiki/BaseX Wikipedia]).<br />
<br />
If you have questions, or if you want to get into direct contact with the developer team and users of BaseX, please write to our [https://basex.org/open-source/ mailing lists]. Many questions are being discussed at [https://stackoverflow.com/questions/tagged/basex StackOverflow]; confirmed bugs and feature requests are discussed on [https://github.com/basexdb/basex/issues GitHub].<br />
</div><br />
<br />
<div style="float:left; width:48%;"><br />
=[[Getting Started]]=<br />
<br />
The getting started section gives you a quick introduction to BaseX. We suggest that you start with the [[Graphical User Interface]] as this is the easiest way to access your XML data, and to get an idea of how XQuery and BaseX works.<br />
<br />
=[[XQuery|XQuery Portal]]=<br />
<br />
More information on using the wide range of XQuery functions and performing XPath and XQuery requests with BaseX can be found in our XQuery Portal.<br />
</div><div style="float:left; width:4%;">&nbsp;<br />
</div><div style="float:left; width:48%;"><br />
<br />
=[[Developing|Developer Section]]=<br />
<br />
The developer section provides useful information for developers. Here you can find information on our supported client APIs and HTTP services, and we present different ways how you can integrate BaseX into your project.<br />
<br />
=[[Advanced User's Guide]]=<br />
<br />
Information for advanced users can be found in our advanced user's guide, which contains details on the BaseX storage, the Client/Server architecture, and some querying features.<br />
</div><br />
__NOTOC__</div>CGhttps://docs.basex.org/index.php?title=Main_Page&diff=16923Main Page2024-03-01T09:14:23Z<p>CG: </p>
<hr />
<div><div style="float:left;"><br />
[[Image:BaseX-Screenshot.png|thumb|240px|BaseX, Graphical User Interface]]<br />
<br />
'''NOTE:''' This is the documentation for '''[[BaseX 10]]'''. The documentation for BaseX 11 is currently located at https://help.basex.org.<br />
<br />
[https://basex.org/ BaseX] is a light-weight, high-performance and scalable XML Database and an XQuery 3.1 Processor with full support for the W3C Update and Full-Text extensions. It allows you to store, query and process large corpora of textual (XML, HTML, JSON, CSV, others) and binary data. The GUI provides an XQuery editor for writing complex applications and provides various visualizations to interactively explore data. RESTXQ enables Web Application development in XQuery. BaseX is platform-independent BSD-licensed (find more on [https://en.wikipedia.org/wiki/BaseX Wikipedia]).<br />
<br />
If you have questions, or if you want to get into direct contact with the developer team and users of BaseX, please write to our [https://basex.org/open-source/ mailing lists]. Many questions are being discussed at [https://stackoverflow.com/questions/tagged/basex StackOverflow]; confirmed bugs and feature requests are discussed on [https://github.com/basexdb/basex/issues GitHub].<br />
</div><br />
<br />
<div style="float:left; width:48%;"><br />
=[[Getting Started]]=<br />
<br />
The getting started section gives you a quick introduction to BaseX. We suggest that you start with the [[Graphical User Interface]] as this is the easiest way to access your XML data, and to get an idea of how XQuery and BaseX works.<br />
<br />
=[[XQuery|XQuery Portal]]=<br />
<br />
More information on using the wide range of XQuery functions and performing XPath and XQuery requests with BaseX can be found in our XQuery Portal.<br />
</div><div style="float:left; width:4%;">&nbsp;<br />
</div><div style="float:left; width:48%;"><br />
<br />
=[[Developing|Developer Section]]=<br />
<br />
The developer section provides useful information for developers. Here you can find information on our supported client APIs and HTTP services, and we present different ways how you can integrate BaseX into your project.<br />
<br />
=[[Advanced User's Guide]]=<br />
<br />
Information for advanced users can be found in our advanced user's guide, which contains details on the BaseX storage, the Client/Server architecture, and some querying features.<br />
</div><br />
__NOTOC__</div>CGhttps://docs.basex.org/index.php?title=Java_Bindings&diff=16922Java Bindings2024-01-22T12:23:27Z<p>CG: /* Functions and Variables */</p>
<hr />
<div>This article is part of the [[XQuery|XQuery Portal]]. It demonstrates different ways to invoke Java code from XQuery, and it presents extensions to access the current query context from Java.<br />
<br />
The Java Binding feature is an extensibility mechanism which enables developers<br />
to directly access Java variables and execute code from XQuery. Addressed Java code must either be contained in the Java classpath, or it must be located in the [[Repository]].<br />
<br />
Please bear in mind that the execution of Java code may cause side effects that conflict with the functional nature of XQuery, or may introduce new security risks to your project.<br />
<br />
Some more notes:<br />
* With the middle dot notation, three adjacent dots can be used to specify array types.<br />
* The path to the standard package {{Code|java.lang.}} can be omitted.<br />
* Java objects are wrapped into function items.<br />
* Results of constructor calls are always returned as function item.<br />
* With {{Option|WRAPJAVA}}, it can be controlled how Java values are converted to XQuery.<br />
<br />
=Identification=<br />
<br />
==Classes==<br />
<br />
A Java class is identified by a namespace URI. The original URI is rewritten as follows:<br />
<br />
# The [[#URI Rewriting|URI Rewriting]] steps are applied to the URI.<br />
# Slashes in the resulting URI are replaced with dots.<br />
# The last path segment of the URI is capitalized and rewritten to [https://en.wikipedia.org/wiki/CamelCase CamelCase].<br />
<br />
The normalization steps are skipped if the URI is prefixed with {{Code|java:}}. The path to the standard package {{Code|java.lang.}} can be omitted:<br />
<br />
* <code><nowiki>http://basex.org/modules/meta-data</nowiki></code> → <code>org.basex.modules.MetaData</code><br />
* <code>java:java.lang.String</code> → <code>java.lang.String</code><br />
* <code>StringBuilder</code> → <code>java.lang.StringBuilder</code><br />
<br />
==Functions and Variables==<br />
<br />
Java constructors, functions and variables can be referenced and evaluated by the existing XQuery function syntax:<br />
<br />
* The namespace of the function name identifies the Java class.<br />
* The local part of the name, which is rewritten to camel case, identifies a variable or function of that class.<br />
* The middle dot character <code>[https://www.fileformat.info/info/unicode/char/b7/index.htm ·]</code> (<code>&amp;#xB7;</code>), a valid character in XQuery names, but not in Java) can be used to append exact Java parameter types to the function name. Class types must be referenced by their full path. Three adjacent dots can be used to address an array argument.<br />
<br />
{| class="wikitable"<br />
|- valign="top"<br />
! Addressed code<br />
! XQuery<br />
! Java<br />
|- valign="top"<br />
| Variable<br />
| <code>Q{Integer}MIN_VALUE()</code><br />
| <code>Integer.MIN_VALUE</code><br />
|- valign="top"<br />
| Function<br />
| <code>Q{Object}hash-code($object)</code><br />
| <code>object.hashCode()</code><br />
|- valign="top"<br />
| Function with argument<br />
| <code>Q{String}split·String·int($string, ';', xs:int(3))</code><br />
| <code>string.split(";", 3)</code><br />
|- valign="top"<br />
| Constructor with array argument<br />
| <code>Q{String}new·byte...(xs:hexBinary('414243'))</code><br />
| <code>new String(new byte[] { 41, 42, 43 })</code><br />
|}<br />
<br />
As XQuery and Java have different type systems, XQuery arguments must be converted to equivalent Java values, and the result of a Java function is converted back to an XQuery value (see [[#Data Types|Data Types]]).<br />
<br />
If the Java function you want to address is not detected, you may need to cast your values to the target type. For example, if a Java function expects a primitive {{Code|int}} value, you will need to convert your XQuery integers to {{Code|xs:int}}.<br />
<br />
=Namespace Declarations=<br />
<br />
In the following example, the Java {{Code|Math}} class is referenced. When executed, the query returns the cosine of an angle by calling the static method {{Code|cos()}}, and the value of π by addressing the static variable via {{Code|PI()}}:<br />
<br />
<pre lang='xquery'><br />
declare namespace math = "java:java.lang.Math";<br />
math:cos(xs:double(0)), math:PI()<br />
</pre><br />
<br />
With the [[XQuery 3.0#Expanded QNames|Expanded QName]] notation of XQuery 3.0, the namespace can directly be embedded in the function call:<br />
<br />
<pre lang='xquery'><br />
Q{java:java.lang.Math}cos(xs:double(0))<br />
</pre><br />
<br />
The constructor of a class can be invoked by calling the virtual function {{Code|new()}}. Instance methods can then called by passing on the resulting Java object as first argument. In the following example, 256 bytes are written to the file {{Code|output.txt}}. First, a new {{Code|FileWriter}} instance is created, and its {{Code|write()}} function is called in the next step:<br />
<br />
<pre lang='xquery'><br />
declare namespace fw = 'java:java.io.FileWriter';<br />
let $file := fw:new('output.txt')<br />
return (<br />
for $i in 0 to 255<br />
return fw:write($file, xs:int($i)),<br />
fw:close($file)<br />
)<br />
</pre><br />
<br />
If the result of a Java call contains invalid XML characters, it will be rejected. The validity check can be disabled by setting {{Option|CHECKSTRINGS}} to false. In the example below, a file with a single {{Code|00}} byte is written, and this file will then be accessed by via Java functions:<br />
<br />
<pre lang='xquery'><br />
declare namespace br = 'java:java.io.BufferedReader';<br />
declare namespace fr = 'java:java.io.FileReader';<br />
<br />
declare option db:checkstrings 'false';<br />
<br />
(: write file :)<br />
file:write-binary('00.bin', xs:hexBinary('00')),<br />
(: read file :)<br />
let $br := br:new(fr:new('00.bin'))<br />
return (<br />
br:readLine($br), <br />
br:close($br)<br />
)<br />
</pre><br />
<br />
The option can also be specified via a pragma:<br />
<br />
<pre lang='xquery'><br />
(# db:checkstrings #) {<br />
br:new(fr:new('00.bin')) ! (br:readLine(.), br:close(.))<br />
}<br />
</pre><br />
<br />
=Module Imports=<br />
<br />
A Java class can be instantiated by ''importing'' them as a module: A new instance of the addressed class will be constructed, which can then be referenced in the query body.<br />
<br />
In the (side-effecting) example below, a HashSet instance is created, values are added, and the size of the set is returned. As {{Code|set:add()}} returns boolean values, {{Function|Profiling|prof:void}} is used to swallow the values:<br />
<br />
<pre lang='xquery'><br />
import module namespace set = "java:java.util.HashSet";<br />
prof:void(<br />
for $s in ("one", "two", "one")<br />
return set:add($s)<br />
),<br />
set:size()<br />
</pre><br />
<br />
The execution of imported classes is more efficient than the execution of instances that have been created via {{Code|new()}}. In turn, no arguments can be supplied in the import statement, and the construction will only be successful if the class can be instantiated without arguments.<br />
<br />
=Integration=<br />
<br />
Java classes can be coupled more closely to BaseX. If a class inherits the abstract [https://github.com/BaseXdb/basex/blob/master/basex-core/src/main/java/org/basex/query/QueryModule.java QueryModule] class, the two variables [https://github.com/BaseXdb/basex/blob/master/basex-core/src/main/java/org/basex/query/QueryContext.java queryContext] and [https://github.com/BaseXdb/basex/blob/master/basex-core/src/main/java/org/basex/query/StaticContext.java staticContext] get available, which provide access to the global and static context of a query.<br />
<br />
The [https://github.com/BaseXdb/basex/blob/master/basex-core/src/main/java/org/basex/query/QueryResource.java QueryResource] interface can be implemented to enforce finalizing operations, such as the closing of opened connections or resources in a module. Its {{Code|close()}} method will be called after the XQuery expression has been fully evaluated.<br />
<br />
==Annotations==<br />
<br />
The internal properties of functions can be assigned via annotations:<br />
<br />
* Java functions can only be executed by users with [[User_Management|Admin permissions]]. You can annotate a function with {{Code|@Requires(<Permission>)}} to also make it accessible to users with fewer privileges.<br />
* Java code is treated as ''nondeterministic'', as its behavior cannot be predicted by the XQuery processor. You may annotate a function as {{Code|@Deterministic}} if you know that it will have no side effects and will always yield the same result.<br />
* Java code is treated as ''context-independent''. If a function accesses the query context, it should be annotated as {{Code|@ContextDependent}}<br />
* Java code is treated as ''focus-independent''. If a function accesses the current context item, position or size, it should be annotated as {{Code|@FocusDependent}}<br />
<br />
In the following code, information from the static query context is returned by the first function, and a query exception is raised by the second function:<br />
<br />
<pre lang='xquery'><br />
import module namespace context = 'org.basex.examples.query.ContextModule';<br />
<br />
element user {<br />
context:user()<br />
},<br />
try {<br />
element to-int { context:to-int('abc') }<br />
} catch basex:error {<br />
element error { $err:description }<br />
}<br />
</pre><br />
<br />
The imported Java class is shown below:<br />
<br />
<pre lang="java"><br />
package org.basex.examples.query;<br />
<br />
import org.basex.query.*;<br />
import org.basex.query.value.item.*;<br />
import org.basex.util.*;<br />
<br />
/**<br />
* This example inherits the {@link QueryModule} class and<br />
* implements the QueryResource interface.<br />
*/<br />
public class ContextModule extends QueryModule implements QueryResource {<br />
/**<br />
* Returns the name of the logged-in user.<br />
* @return user string<br />
*/<br />
@Requires(Permission.NONE)<br />
@Deterministic<br />
@ContextDependent<br />
public String user() {<br />
return queryContext.context.user.name;<br />
}<br />
<br />
/**<br />
* Converts the specified string to an integer.<br />
* @param value string to be converted<br />
* @return resulting integer<br />
* @throws QueryException query exception<br />
*/<br />
@Requires(Permission.NONE)<br />
@Deterministic<br />
public int toInt(final String value) throws QueryException {<br />
try {<br />
return Integer.parseInt(value);<br />
} catch(NumberFormatException ex) {<br />
throw new QueryException("Integer conversion failed: " + value);<br />
}<br />
}<br />
<br />
@Override<br />
public void close() {<br />
// defined in QueryResource interface, will be called after query evaluation<br />
}<br />
}<br />
</pre><br />
<br />
The result will look as follows:<br />
<br />
<pre lang="xml"><br />
<user>admin</admin><br />
<error>Integer conversion failed: abc</error><br />
</pre><br />
<br />
Please visit the XQuery 3.0 specification if you want to get more insight into<br />
[https://www.w3.org/TR/xpath-functions-31/#properties-of-functions function properties].<br />
<br />
==Updates==<br />
<br />
The {{Code|@Updating}} annotation can be applied to mark Java functions that perform write or update operations:<br />
<br />
<pre lang="java"><br />
@Updating<br />
public void backup() {<br />
// ...<br />
}<br />
</pre><br />
<br />
An XQuery expression will be handled as an [[XQuery Update#Updating Expressions|updating expression]] if it calls an updating Java function. In contrast to XQuery update operations, the Java code will immediately be executed, but the result will be cached as if {{Function|Update|update:output}} was called.<br />
<br />
The annotation is particularly helpful if combined with a lock annotation.<br />
<br />
==Locking==<br />
<br />
By default, a Java function will be executed in parallel with other code. If a Java function performs sensitive operations, it is advisable to explicitly lock the code.<br />
<br />
===Java Locks===<br />
<br />
Java provides a handful of mechanism to control the execution of code. The concurrent execution of functions can be avoided with the {{Code|synchronized}} keyword. For more complex scenarios, the Lock, Semaphore and Atomic classes can be brought into play.<br />
<br />
===XQuery Locks===<br />
<br />
If you want to synchronize the execution of your code with BaseX locks, you can take advantage of the {{Code|@Lock}} annotation:<br />
<br />
<pre lang="java"><br />
@Lock("HEAVYIO")<br />
public void read() {<br />
// ...<br />
}<br />
<br />
@Updating<br />
@Lock("HEAVYIO")<br />
public void write() {<br />
// ...<br />
}<br />
</pre><br />
<br />
If an XQuery expression invokes {{Code|write()}}, any other query that calls {{Code|write()}} or {{Code|read()}} needs to wait for the query to be finished. The {{Code|read()}} function can be run in parallel; whereas queries will be queued if {{Code|write()}} is called.<br />
<br />
More details on concurrent querying can be found in the article on [[Transaction Management]].<br />
<br />
==Data Types==<br />
<br />
===Conversion to Java===<br />
<br />
Before Java code is executed, the arguments are converted to Java values, depending on the addressed function or constructor parameters. The accepted Java types and the original XQuery types are depicted in the second and first column of the table below.<br />
<br />
If a numeric value is supplied for which no exact matching is defined, it is cast to the appropriate type unless it exceeds its limits. The following two function calls are equivalent:<br />
<br />
<pre lang='xquery'><br />
(: exact match :)<br />
Q{String}codePointAt('ABC', xs:int(1)),<br />
(: xs:byte and xs:integer casts :)<br />
Q{String}codePointAt('ABC', xs:byte(1)),<br />
Q{String}codePointAt('ABC', 1)<br />
</pre><br />
<br />
===Conversion to XQuery===<br />
<br />
By default, Java values with the most common types (as shown in the second and third column of the table) are converted to XQuery values. All other values are returned as ''Java items'', which are function items with a wrapped Java value. The results of constructor calls are always returned as Java items.<br />
<br />
The conversion of the wrapped Java value to XQuery is enforced by invoking the function item: Values in {{Code|Iterator}} and {{Code|Iterable}} instances (Lists, Sets and Collections) are converted to items, and maps are converted to XQuery maps:<br />
<br />
<pre lang='xquery'><br />
declare namespace Scanner = 'java:java.util.Scanner';<br />
let $scanner := Scanner:new("A B C") => Scanner:useDelimiter(" ")<br />
return $scanner()<br />
</pre><br />
<br />
If no conversion is defined, a string is returned, resulting from the {{Code|toString()}} method of the object. This method is also called if the string representation of a Java item is requested:<br />
<br />
<pre lang='xquery'><br />
(: returns the string representations of a HashMap and an ArrayList instance :)<br />
'Map: ' || Q{java.util.HashMap}new(),<br />
string(Q{java:java.util.ArrayList}new())<br />
</pre><br />
<br />
The conversion can be further controlled with the {{Option|WRAPJAVA}} option. The following values exist:<br />
<br />
{| class="wikitable"<br />
|- valign="top"<br />
! Value<br />
! Description<br />
|- valign="top"<br />
| {{Code|some}}<br />
| The default: Java values of the most common types are converted, others are wrapped into Java items.<br />
|- valign="top"<br />
| {{Code|none}}<br />
| All Java values are converted. If no conversion is defined, a string is returned, resulting from the {{Code|toString()}} method.<br />
|- valign="top"<br />
| {{Code|all}}<br />
| Java values are wrapped into Java items (excluding those inheriting the internal type {{Code|org.basex.query.value.Value}}).<br />
|- valign="top"<br />
| {{Code|instance}}<br />
| If the method of a class instance was called, the Java value is ignored and the instance is wrapped into a Java item. Otherwise, the Java value is returned.<br />
|- valign="top"<br />
| {{Code|void}}<br />
| Java values are ignored, and an empty sequence is returned instead.<br />
|}<br />
<br />
In the following example, the result of the first function – a char array – is wrapped and passed on to a {{Code|CharBuffer}} function. Without the option, the single-value array would be converted to an {{Code|xs:unsignedShort}} item and the second function call would fail:<br />
<br />
<pre lang='xquery'><br />
(: Without the pragma, the result of toChars would be converted to an xs:unsignedShort item, and the second function call would fail :)<br />
<br />
(# db:wrapjava all #) {<br />
Q{Character}toChars(xs:int(33))<br />
=> Q{java.nio.CharBuffer}wrap()<br />
}<br />
</pre><br />
<br />
The next example demonstrates a use case for the {{Code|instance}} option:<br />
<br />
<pre lang='xquery'><br />
(: Thanks to the pragma, the function calls can be chained :)<br />
<br />
declare namespace set = 'java:java.util.HashSet';<br />
let $set := (# db:wrapjava instance #) {<br />
set:new()<br />
=> set:add('1')<br />
=> set:add('2')<br />
}<br />
return $set()<br />
</pre><br />
<br />
The {{Code|void}} option is helpful if side-effecting methods return values that do not contribute to the final result:<br />
<br />
<pre lang='xquery'><br />
(: Without the pragma, 100 booleans would be returned by the FLWOR expression :)<br />
<br />
declare namespace set = 'java:java.util.HashSet';<br />
let $set := set:new()<br />
return (<br />
(# db:wrapjava void #) {<br />
for $i in 1 to 100<br />
return set:add($set, $i)<br />
},<br />
$set()<br />
)<br />
</pre><br />
<br />
The irrelevant results could also be swallowed with {{Function|Profiling|prof:void}}.<br />
<br />
{| class="wikitable"<br />
|- valign="top"<br />
! XQuery input<br />
! Expected or returned Java type<br />
! XQuery output<br />
|- valign="top"<br />
| <code>item()*</code> (no conversion)<br />
| <code>org.basex.query.value.Value</code><br />
| <code>item()*</code> (no conversion)<br />
|- valign="top"<br />
| <code>empty-sequence()</code><br />
| <code>null</code><br />
| <code>empty-sequence()</code><br />
|- valign="top"<br />
| <code>xs:string</code>, <code>xs:untypedAtomic</code><br />
| <code>String</code><br />
| <code>xs:string</code><br />
|- valign="top"<br />
| <code>xs:unsignedShort</code><br />
| <code>char</code>, <code>Character</code><br />
| <code>xs:unsignedShort</code><br />
|- valign="top"<br />
| <code>xs:boolean</code><br />
| <code>boolean</code>, <code>Boolean</code><br />
| <code>xs:boolean</code><br />
|- valign="top"<br />
| <code>xs:byte</code><br />
| <code>byte</code>, <code>Byte</code><br />
| <code>xs:byte</code><br />
|- valign="top"<br />
| <code>xs:short</code><br />
| <code>short</code>, <code>Short</code><br />
| <code>xs:short</code><br />
|- valign="top"<br />
| <code>xs:int</code><br />
| <code>int</code>, <code>Integer</code><br />
| <code>xs:int</code><br />
|- valign="top"<br />
| <code>xs:integer</code>, <code>xs:long</code><br />
| <code>long</code>, <code>Long</code><br />
| <code>xs:integer</code><br />
|- valign="top"<br />
| <code>xs:unsignedLong</code><br />
| <code>java.math.BigInteger</code><br />
| <code>xs:unsignedLong</code> (lossy)<br />
|- valign="top"<br />
| <code>xs:decimal</code><br />
| <code>java.math.BigDecimal</code><br />
| <code>xs:decimal</code><br />
|- valign="top"<br />
| <code>xs:float</code><br />
| <code>float</code>, <code>Float</code><br />
| <code>xs:float</code><br />
|- valign="top"<br />
| <code>xs:double</code><br />
| <code>double</code>, <code>Double</code><br />
| <code>xs:double</code><br />
|- valign="top"<br />
| <code>xs:QName</code><br />
| <code>javax.xml.namespace.QName</code><br />
| <code>xs:QName</code><br />
|- valign="top"<br />
| <code>xs:anyURI</code><br />
| <code>java.net.URI</code>, <code>java.net.URL</code><br />
| <code>xs:anyURI</code><br />
|- valign="top"<br />
| <code>xs:date</code><br />
| <code>javax.xml.datatype.XMLGregorianCalendar</code><br />
| <code>xs:date</code><br />
|- valign="top"<br />
| <code>xs:duration</code><br />
| <code>javax.xml.datatype.Duration</code><br />
| <code>xs:duration</code><br />
|- valign="top"<br />
| <code>node()</code><br />
| <code>org.w3c.dom.Node</code><br />
| <code>node()</code><br />
|- valign="top"<br />
| <code>array(xs:boolean)</code><br />
| <code>boolean[]</code><br />
| <code>xs:boolean*</code><br />
|- valign="top"<br />
| <code>array(xs:string)</code><br />
| <code>String[]</code><br />
| <code>xs:string*</code><br />
|- valign="top"<br />
| <code>array(xs:unsignedShort)</code><br />
| <code>char[]</code><br />
| <code>xs:unsignedShort*</code><br />
|- valign="top"<br />
| <code>array(xs:short)</code><br />
| <code>short[]</code><br />
| <code>xs:short*</code><br />
|- valign="top"<br />
| <code>array(xs:int)</code><br />
| <code>int[]</code><br />
| <code>xs:int*</code><br />
|- valign="top"<br />
| <code>array(xs:integer)</code>, <code>array(xs:long)</code><br />
| <code>long[]</code><br />
| <code>xs:integer*</code><br />
|- valign="top"<br />
| <code>array(xs:float)</code><br />
| <code>float[]</code><br />
| <code>xs:float*</code><br />
|- valign="top"<br />
| <code>array(xs:double)</code><br />
| <code>double[]</code><br />
| <code>xs:double*</code><br />
|- valign="top"<br />
| <code>Object[]</code> (others)<br />
| <code>item()*</code><br />
| <code>array(*)</code> (others)<br />
|- valign="top"<br />
| <code>map(*)</code><br />
| java.util.HashMap<br />
| <code>Wrapped Java object</code><br />
|}<br />
<br />
==URI Rewriting==<br />
<br />
Before a Java class or module is accessed, its namespace URI will be normalized:<br />
<br />
# If the URI is a URL:<br />
## colons will be replaced with slashes,<br />
## in the URI authority, the order of all substrings separated by dots is reversed, and<br />
## dots in the authority and the path are replaced by slashes. If no path exists, a single slash is appended.<br />
# Otherwise, if the URI is a URN, colons will be replaced with slashes.<br />
# Characters other than letters, dots and slashes will be replaced with dashes.<br />
# If the resulting string ends with a slash, the {{Code|index}} string is appended.<br />
<br />
If the resulting path has no file suffix, it may point to either an XQuery module or a Java archive:<br />
<br />
* {{Code|<nowiki>http://basex.org/modules/hello/World</nowiki>}} → {{Code|org/basex/modules/hello/World}}<br />
* {{Code|<nowiki>http://www.example.com</nowiki>}} → {{Code|com/example/www/index}}<br />
* {{Code|a/little/example}} → {{Code|a/little/example}}<br />
* {{Code|a:b:c}} → {{Code|a/b/c}}<br />
<br />
Note that the mapping is not unique: Different URIs may result in the same path.<br />
<br />
=Changelog=<br />
<br />
; Version 9.6<br />
* Updated: Java Bindings revised (new mappings, Java functiom items, {{Option|WRAPJAVA}} option).<br />
<br />
; Version 9.4<br />
* Added: Annotation for [[#Updates|updating functions]].<br />
* Updated: Single annotation for read and write locks.<br />
<br />
; Version 8.4<br />
* Updated: Rewriting rules<br />
<br />
;Version 8.2<br />
* Added: [[#URI Rewriting|URI Rewriting]]: support for URNs<br />
<br />
; Version 8.0<br />
* Added: {{Code|QueryResource}} interface, called after a query has been fully evaluated.<br />
<br />
; Version 7.8<br />
* Added: Java locking annotations<br />
* Updated: {{Code|context}} variable has been split into {{Code|queryContext}} and {{Code|staticContext}}.<br />
<br />
; Version 7.2.1<br />
* Added: import of Java modules, context awareness<br />
* Added: [[#Packaging|Packaging]], [[#URI Rewriting|URI Rewriting]]</div>CGhttps://docs.basex.org/index.php?title=Unit_Module&diff=16919Unit Module2023-12-07T14:19:56Z<p>CG: /* Annotations */</p>
<hr />
<div>This [[Module Library|XQuery Module]] contains annotations and functions for performing XQUnit tests.<br />
<br />
=Introduction=<br />
<br />
The more complex a software application grows, the more error-prone it gets. This is why testing frameworks have been developed, which provide a standardized, automated way of testing software. The [https://en.wikipedia.org/wiki/XUnit XUnit] frameworks (such as SUnit or JUnit) allow testing of atomic units of a program, such as single functions and algorithms.<br />
<br />
This module borrows heavily from the existing frameworks: it provides various annotations for testing XQuery functions. Unit functions are provided to assert the validity of arbitrary conditions expressed in XQuery and to raise errors whenever a condition is not satisfied. Some additional functions exist to run all unit tests of the current module or a set of specified library modules.<br />
<br />
=Usage=<br />
<br />
Tests are started via the {{Command|TEST}} command. It compiles all XQuery modules in a given file<br />
or directory and runs all functions that are annotated with {{Code|%unit:test}}. A test report is<br />
generated and returned, which resembles the format returned by other xUnit testing frameworks,<br />
such as the Maven Surefire Plugin ([[#Result|see below]]).<br />
<br />
=Conventions=<br />
<br />
All annotations, functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/unit</nowiki></code> namespace, which is statically bound to the {{Code|unit}} prefix.<br/><br />
<br />
=Annotations=<br />
<br />
==unit:test==<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Syntax'''<br />
|{{Code|%unit:test}}<br/>{{Code|%unit:test("expected", CODE)}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|With this annotation, a function can be marked as unit test. It will be evaluated if a test report is created for the module in which this function is located.<br/><code>error</code> can be supplied as additional string argument. It is followed by <code>CODE</code>, which must be a valid [[XQuery 3.0#Expanded QNames|EQName]] string. If the function expression does not raise that error, the test will fail.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* The following test will be successful, as it does nothing (and, hence, nothing wrong):<br />
<pre lang='xquery'><br />
declare %unit:test function local:void() { () };<br />
</pre><br />
* The following test will be successful, as the function body will raise <code>err:XPTY0004</code>:<br />
<pre lang='xquery'><br />
declare %unit:test('expected', "err:XPTY0004") function local:add() {<br />
123 + 'strings and integers cannot be added'<br />
};<br />
</pre><br />
|}<br />
<br />
==unit:before==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Syntax'''<br />
|{{Code|%unit:before}}<br/>{{Code|%unit:before(FUNCTION)}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|A function decorated with this annotation will be evaluated '''before each''' unit test as a separate transaction.<br/><code>FUNCTION</code> can be supplied as additional argument. It must be a valid [[XQuery 3.0#Expanded QNames|EQName]] string. If specified, the function will only be evaluated before a function with the given name is tested. This extension is e. g. helpful if the results of updates need to be tested.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* The first function will be evaluated before the actual test:<br />
<pre lang='xquery'><br />
declare %updating %unit:before("local:check") function local:before-check() {<br />
db:create('test-db')<br />
};<br />
declare %updating %unit:test function local:check() {<br />
unit:assert(db:exists('test-db'))<br />
};<br />
</pre><br />
|}<br />
<br />
==unit:after==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Syntax'''<br />
|{{Code|%unit:after}}<br/>{{Code|%unit:after(FUNCTION)}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|A function decorated with this annotation will be evaluated '''after each''' unit test as a separate transaction.<br/><code>FUNCTION</code> can be supplied as additional argument. It must be a valid [[XQuery 3.0#Expanded QNames|EQName]] string. If specified, the function will only be evaluated after a function with the given name is tested.<br />
|}<br />
<br />
==unit:before-module==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Syntax'''<br />
|{{Code|%unit:before-module}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|If a function is decorated with this annotation, it will be evaluated '''before all''' unit tests in the current module as a separate transaction.<br />
|}<br />
<br />
==unit:after-module==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Syntax'''<br />
|{{Code|%unit:after-module}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|If a function is decorated with this annotation, it will be evaluated '''after all''' unit tests in the current module as a separate transaction.<br />
|}<br />
<br />
==unit:ignore==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Syntax'''<br />
|{{Code|%unit:ignore}}<br/>{{Code|%unit:ignore(MESSAGE)}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|If a function is decorated with this annotation, it will temporarily be ignored by the test suite runner.<br />
|}<br />
<br />
=Functions=<br />
<br />
==unit:assert==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>unit:assert(<br />
$test as item()*,<br />
$info as item() := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Asserts that the effective boolean value of the specified {{Code|$test}} is true and returns an empty sequence. Otherwise, raises an error. The ''effective boolean value'' of an expression can be explicitly computed by using the {{Code|fn:boolean}} function.<br/>The default failure message can be overridden with the {{Code|$info}} argument.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|fail|#Errors}} the assertion failed, or an error was raised.<br />
|}<br />
<br />
==unit:assert-equals==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>unit:assert-equals(<br />
$returned as item()*,<br />
$expected as item()*,<br />
$info as item() := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Asserts that the specified arguments are equal according to the rules of the [https://www.w3.org/TR/xpath-functions-31/#func-deep-equal {{Code|fn:deep-equal}} function]. Otherwise, raises an error.<br/>The default failure message can be overridden with the {{Code|$info}} argument.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|fail|#Errors}} the assertion failed, or an error was raised.<br />
|}<br />
<br />
==unit:fail==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>unit:fail(<br />
$info as item() := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Raises a unit error. The default failure message can be overridden with the {{Code|$info}} argument.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|fail|#Errors}} default error raised by this function.<br />
|}<br />
<br />
=Example=<br />
<br />
The following XQUnit module {{Code|tests.xqm}} contains all available unit annotations:<br />
<br />
==Query==<br />
<br />
<pre lang='xquery'><br />
module namespace test = 'http://basex.org/modules/xqunit-tests';<br />
<br />
(:~ Initializing function, which is called once before all tests. :)<br />
declare %unit:before-module function test:before-all-tests() {<br />
()<br />
};<br />
<br />
(:~ Initializing function, which is called once after all tests. :)<br />
declare %unit:after-module function test:after-all-tests() {<br />
()<br />
};<br />
<br />
(:~ Initializing function, which is called before each test. :)<br />
declare %unit:before function test:before() {<br />
()<br />
};<br />
<br />
(:~ Initializing function, which is called after each test. :)<br />
declare %unit:after function test:after() {<br />
()<br />
};<br />
<br />
(:~ Function demonstrating a successful test. :)<br />
declare %unit:test function test:assert-success() {<br />
unit:assert(<a/>)<br />
};<br />
<br />
(:~ Function demonstrating a failure using unit:assert. :)<br />
declare %unit:test function test:assert-failure() {<br />
unit:assert((), 'Empty sequence.')<br />
};<br />
<br />
(:~ Function demonstrating a failure using unit:assert-equals. :)<br />
declare %unit:test function test:assert-equals-failure() {<br />
unit:assert-equals(4 + 5, 6)<br />
};<br />
<br />
(:~ Function demonstrating an unexpected success. :)<br />
declare %unit:test("expected", "err:FORG0001") function test:unexpected-success() {<br />
()<br />
};<br />
<br />
(:~ Function demonstrating an expected failure. :)<br />
declare %unit:test("expected", "err:FORG0001") function test:expected-failure() {<br />
1 + <a/><br />
};<br />
<br />
(:~ Function demonstrating the creation of a failure. :)<br />
declare %unit:test function test:failure() {<br />
unit:fail("Failure!")<br />
};<br />
<br />
(:~ Function demonstrating an error. :)<br />
declare %unit:test function test:error() {<br />
1 + <a/><br />
};<br />
<br />
(:~ Skipping a test. :)<br />
declare %unit:test %unit:ignore("Skipped!") function test:skipped() {<br />
()<br />
};<br />
</pre><br />
<br />
By running {{Code|TEST tests.xqm}}, the following report will be generated (timings may differ):<br />
<br />
==Result==<br />
<br />
<pre lang="xml"><br />
<testsuites time="PT0.256S"><br />
<testsuite name="file:///C:/Users/user/Desktop/test.xqm" time="PT0.212S" tests="8" failures="4" errors="1" skipped="1"><br />
<testcase name="assert-success" time="PT0.016S"/><br />
<testcase name="assert-failure" time="PT0.005S"><br />
<failure line="30" column="15"><br />
<info>Empty sequence.</info><br />
</failure><br />
</testcase><br />
<testcase name="assert-equals-failure" time="PT0.006S"><br />
<failure line="35" column="22"><br />
<returned item="1" type="xs:integer">9</returned><br />
<expected item="1" type="xs:integer">6</expected><br />
<info>Item 1: 6 expected, 9 returned.</info><br />
</failure><br />
</testcase><br />
<testcase name="unexpected-success" time="PT0.006S"><br />
<failure><br />
<expected>FORG0001</expected><br />
</failure><br />
</testcase><br />
<testcase name="expected-failure" time="PT0.004S"/><br />
<testcase name="failure" time="PT0.004S"><br />
<failure line="50" column="13"><br />
<info>Failure!</info><br />
</failure><br />
</testcase><br />
<testcase name="error" time="PT0.004S"><br />
<error line="55" column="6" type="FORG0001"><br />
<info>Cannot cast to xs:double: "".</info><br />
</error><br />
</testcase><br />
<testcase name="skipped" skipped="Skipped!" time="PT0S"/><br />
</testsuite><br />
</testsuites><br />
</pre><br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
|Description<br />
|- valign="top"<br />
|{{Code|fail}}<br />
|An assertion failed, or an error was raised.<br />
|- valign="top"<br />
|{{Code|no-args}}<br />
|A test function must have no arguments.<br />
|- valign="top"<br />
|{{Code|private}}<br />
|A test function must not be private.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 9.0<br />
<br />
* Updated: error codes updated; errors now use the module namespace<br />
<br />
;Version 8.0.2<br />
<br />
* Updated: (expected) errors are compared by QNames instead of local names (including namespaces).<br />
<br />
;Version 8.0<br />
<br />
* Deleted: {{Code|UNIT0006}} (ignore results returned by functions).<br />
* Added: {{Function||unit:fail}}, 0-argument signature.<br />
* Updated: the info argument of functions can now be an arbitrary item.<br />
* Updated: infos are now represented in an <code>info</code> child element.<br />
* Updated: {{Function||unit:before}} and {{Function||unit:after}} can be extended by a filter argument.<br />
<br />
;Version 7.9<br />
<br />
* Added: TEST command<br />
* Removed: {{Function||unit:test}}, {{Function||unit:test-uris}}<br />
<br />
;Version 7.8<br />
<br />
* Added: {{Function||unit:assert-equals}}<br />
* Updated: enhanced test report output<br />
<br />
This module was introduced with Version 7.7.</div>CGhttps://docs.basex.org/index.php?title=Utility_Module&diff=16918Utility Module2023-12-05T12:36:07Z<p>CG: </p>
<hr />
<div>This [[Module Library|XQuery Module]] contains some utility and helper functions.<br />
<br />
With {{Announce|Version 11}}, many functions have been removed in favor of new features of XQuery 4:<br />
<br />
{|<br />
|- valign="top"<br />
| '''BaseX 10'''<br />
| '''XQuery 4'''<br />
|- valign="top"<br />
| {{Code|util:array-members}}<br />
| [https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-array-members <code>array:members</code>]<br />
|- valign="top"<br />
| {{Code|util:array-values}}<br />
| [https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-array-values <code>array:values</code>]<br />
|- valign="top"<br />
| {{Code|util:chars}}<br />
| [https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-chars <code>fn:characters</code>]<br />
|- valign="top"<br />
| {{Code|util:duplicates}}<br />
| [https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-duplicate-values <code>fn:duplicate-values</code>]<br />
|- valign="top"<br />
| {{Code|util:init}}<br />
| [https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-trunk <code>fn:trunk</code>]<br />
|- valign="top"<br />
| {{Code|util:intersperse}}<br />
| [https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-intersperse <code>fn:intersperse</code>]<br />
|- valign="top"<br />
| {{Code|util:item}}<br />
| [https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-items-at <code>fn:items-at</code>]<br />
|- valign="top"<br />
| {{Code|util:last}}<br />
| [https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-foot <code>fn:foot</code>]<br />
|- valign="top"<br />
| {{Code|util:map-entries}}<br />
| [https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-map-entries <code>map:entries</code>]<br />
|- valign="top"<br />
| {{Code|util:map-values}}<br />
| [https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-map-values <code>map:values</code>]<br />
|- valign="top"<br />
| {{Code|util:or}}<br />
| <code>$expr1 otherwise $expr2<br />
|- valign="top"<br />
| {{Code|util:replicate}}<br />
| [https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-replicate <code>fn:replicate</code>]<br />
|}<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module and errors are assigned to the <code><nowiki>http://basex.org/modules/util</nowiki></code> namespace, which is statically bound to the {{Code|util}} prefix.<br/><br />
<br />
=Conditions and Ranges=<br />
<br />
==util:if==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>util:if(<br />
$condition as item()*,<br />
$then as item()*,<br />
$else as item()* := ()<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Alternative writing for the if/then/else expression:<br />
* If the ''effective boolean value'' of {{Code|$condition}} is true, the {{Code|$then}} branch will be evaluated.<br />
* Otherwise, {{Code|$else}} will be evaluated. If the third argument is omitted, an empty sequence will be returned.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>util:if(true(), 123, 456)</code> returns {{Code|123}}.<br />
* <code>util:if(0, 'wrong!')</code> returns an empty sequence.<br />
|}<br />
<br />
==util:count-within==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>util:count-within(<br />
$sequence as item()*,<br />
$min as xs:integer,<br />
$max as xs:integer := ()<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Checks if the specified {{Code|$sequence}} has at least {{Code|$min}} and, optionally, at most {{Code|$max}} items. Equivalent to:<br />
<pre lang='xquery'><br />
let $count := count($sequence)<br />
return $count >= $min and $count <= $max<br />
</pre><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>util:count-within(('a', 'b', 'c'), 2)</code> returns {{Code|true}}.<br />
* <code>util:count-within((1 to 1000000000)[. < 10], 3, 6)</code> returns {{Code|true}}.<br />
|}<br />
<br />
==util:range==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>util:range(<br />
$sequence as item()*,<br />
$first as xs:double,<br />
$last as xs:double<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns items from {{Code|$sequence}}, starting at position {{Code|$first}} and ending at {{Code|$last}}. Equivalent to:<br />
<pre lang='xquery'><br />
subsequence($sequence, $first, $last - $first + 1)<br />
</pre><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>util:range(//item, 11, 20)</code> returns all path results from (if available) position 11 to 20.<br />
|}<br />
<br />
=Node Functions=<br />
<br />
==util:ddo==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>util:ddo(<br />
$nodes as node()*<br />
) as node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns nodes in ''distinct document order'': duplicate nodes will be removed, and the remaining nodes will be returned in [https://www.w3.org/TR/xquery-31/#dt-document-order document order]. As results of path expressions are brought distinct document order before they are returned, the function is equivalent to:<br />
<pre lang='xquery'><br />
$nodes/self::node()<br />
</pre><br />
|}<br />
<br />
==util:root==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>util:root(<br />
$nodes as node()*<br />
) as document-node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the document nodes of the specified {{Code|$nodes}}. The path expression <code>/abc</code> is internally represented as <code>util:root(.)/abc</code>. Equivalent to:<br />
<pre lang='xquery'><br />
$nodes ! /<br />
</pre><br />
|}<br />
<br />
==util:strip-namespaces==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>util:strip-namespaces(<br />
$node as node(),<br />
$prefixes as xs:string* := ()<br />
) as node()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Removes namespaces with the specified <code>$prefixes</code> from the supplied <code>$node</code>. An empty string can be supplied to remove the default namespace. If no prefixes are specified, all namespaces will be removed.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* Remove all namespaces from an element and its descendants:<br />
<pre lang='xquery'><br />
util:strip-namespaces(<xml xmlns='uri' xmlns:prefix='uri2' prefix:name='value'><prefix:child/></xml>)<br />
<br />
(: yields :)<br />
<xml name='value'><child/></xml><br />
</pre><br />
* Remove all default namespaces:<br />
<pre lang='xquery'><br />
<xml xmlns='uri1'><child xmlns='uri2'/></xml><br />
=> util:strip-namespaces('')<br />
</pre><br />
|}<br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
|Description<br />
|- valign="top"<br />
|{{Code|negative}}<br />
|The specified number is negative.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 11.0<br />
* Removed: {{Code|util:array-members}}, {{Code|util:array-values}}, {{Code|util:chars}}, {{Code|util:duplicates}}, {{Code|util:init}}, {{Code|util:intersperse}}, {{Code|util:item}}, {{Code|util:last}}, {{Code|util:map-entries}}, {{Code|util:map-values}}, {{Code|util:replicate}}<br />
<br />
;Version 9.7<br />
* Added: {{Function||util:strip-namespaces}}<br />
* Updated: {{Function||util:count-within}}: Renamed from {{Code|util:within}}.<br />
<br />
;Version 9.5<br />
* Added: {{Code|util:intersperse}}, {{Code|util:within}}, {{Code|util:duplicates}}, {{Code|util:array-members}}, {{Code|util:array-values}}, {{Code|util:map-entries}}, {{Code|util:map-values}}<br />
* Updated: {{Code|util:replicate}}: Third argument added.<br />
<br />
;Version 9.4<br />
* Added: {{Function||util:root}}<br />
<br />
;Version 9.3<br />
* Added: {{Function||util:ddo}}<br />
<br />
;Version 9.2<br />
* Added: {{Code|util:chars}}, {{Code|util:init}}<br />
* Updated: {{Code|util:item}}, {{Code|util:last}}, {{Function||util:range}} renamed (before: {{Code|util:item-at}}, {{Code|util:item-range}}, {{Code|util:last-from}})<br />
<br />
;Version 9.1<br />
* Added: {{Function||util:if}}, {{Code|util:or}}<br />
<br />
;Version 9.0<br />
* Added: {{Code|util:replicate}}<br />
<br />
The Module was introduced with Version 8.5.</div>CGhttps://docs.basex.org/index.php?title=Database_Module&diff=16917Database Module2023-12-01T21:07:51Z<p>CG: /* db:export */</p>
<hr />
<div>This [[Module Library|XQuery Module]] contains functions for processing databases from within XQuery. Existing databases can be opened and listed, its contents can be directly accessed, documents can be added to and removed, etc.<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/db</nowiki></code> namespace, which is statically bound to the {{Code|db}} prefix.<br/><br />
<br />
==Database Nodes==<br />
<br />
In BaseX, two internal representations exist for XML nodes:<br />
<br />
1. Database nodes are:<br />
* stored in a persistent database on disk;<br />
* nodes of a document that has been generated temporarily with {{Code|fn:doc}}, {{Code|fn:parse-xml}} and other functions; or<br />
* the result of a [[XQuery Update#Main-Memory Updates|main-memory update]].<br />
2. Fragments are similar to DOM structures. They are created when XQuery node constructors are used (<code><a/></code>, <code>element a { }</code>, etc.).<br />
<br />
Fragments require less memory for small XML structures, but database nodes are more efficient for larger amounts of data. Fragments nodes can be converted to database nodes by applying a [[XQuery_Update#Main-Memory Updates|main-memory update]] with an empty body to a node:<br />
<br />
<pre lang='xquery'><br />
<xml>hello world</xml> update { }<br />
</pre><br />
<br />
==Updating Functions==<br />
<br />
Various functions in this module are ''updating''. Updating functions will not be immediately executed, but queued on the [[XQuery Update#Pending Update List|Pending Update List]], and processed after the remaining query has been evaluated. This means that the order in which the functions are specified in the query often does not reflect the order in which they will eventually be executed.<br />
<br />
=General Functions=<br />
<br />
==db:system==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:system() as element(system)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns general information on the database system the current values of all global and local [[Options]]. The {{Command|INFO}} command returns similar output.<br />
|}<br />
<br />
==db:option==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:option(<br />
$name as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the current value (string, integer, boolean, map) of a global or local [[Options|Option]] with the specified {{Code|$name}}. The {{Command|SHOW OPTIONS}} command returns similar output.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|option|#Errors}} the specified option is unknown.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>db:option('dbpath')</code> returns the database path string.<br />
* <code>db:option('serializer')</code> returns a map with the current serialization parameters.<br />
* <code>declare option db:stripws 'true'; db:option('stripws')</code> returns the locally assigned value.<br />
|}<br />
<br />
==db:info==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:info(<br />
$database as xs:string<br />
) as element(database)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns meta information on the specified {{Code|$database}}. The output is similar to the {{Command|INFO DB}} command.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|}<br />
<br />
==db:property==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:property(<br />
$database as xs:string,<br />
$name as xs:string<br />
) as xs:anyAtomicType</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the value (string, boolean, integer) of a property with the specified {{Code|$name}} in the specified {{Code|$database}}. The available properties are the ones returned by {{Function||db:info}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|property|#Errors}} the specified property is unknown.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>db:property('db', 'size')</code> returns the number of bytes occupied by the database <code>db</code>.<br />
* <code>db:property('xmark', 'textindex')</code> indicates if the <code>xmark</code> database has a text index.<br />
* <code>db:property('discogs', 'uptodate')</code> indicates if the database statistics and index structures of the <code>discogs</code> database are up-to-date.<br />
|}<br />
<br />
==db:list==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:list(<br />
$database as xs:string := (),<br />
$path as xs:string := ()<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Without arguments, the names of all databases are returned that are accessible to the current user. If {{Code|$database}} is specified, paths to all resources of this database are returned. The results can be restricted to resources starting with the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:list("docs")}} returns the names of all documents of a database named {{Code|docs}}.<br />
|}<br />
<br />
==db:list-details==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:list-details(<br />
$database as xs:string := (),<br />
$path as xs:string := ()<br />
) as element()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Without arguments, an element is returned for each database that is accessible to the current user:<br />
* An element has a value, which is the name of the database, and several attributes, which contain the number of stored resources, the modification date, the database size on disk (measured in bytes), and a path to the original database input.<br />
If {{Code|$database}} is specified, an element for each resource in this database is returned:<br />
* An element has a value, which is the name of the resource, and several attributes, which contain the content type, the modification date, the raw flag (which indicates if the resource is binary or XML), and the size of a resource.<br />
* The value of the size attribute depends on the resource type: for documents, it represents the number of nodes; for binary data, it represents the file size (measured in bytes).<br />
* The results can be restricted to resources starting with the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:list-details("shop")}} returns the names plus additional info on all resources of a database named {{Code|shop}}.<br />
|}<br />
<br />
==db:dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:dir(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as element()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns metadata on all directories and resources of a {{Code|$database}} in the specified {{Code|$path}}. Two types of elements are returned:<br />
* {{Code|resource}} represents a resource. The element value is the directory path; content type, modification date, raw flag (which indicates if the resource is binary or XML), and size of the resource are returned as attributes.<br />
* {{Code|dir}} represents a directory. The element value is the directory path; the modification date is returned in an attribute.<br />
The directories are not stored in the internal database layout. Instead, they result implicitly from the paths of stored resources.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|path|#Errors}} the specified path is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:dir('shop', 'books')}} returns all entries of the {{Code|books}} directory of a {{Code|shop}} database.<br />
|}<br />
<br />
=Read Operations=<br />
<br />
==db:get==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get(<br />
$database as xs:string,<br />
$path as xs:string := ()<br />
) as document-node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all documents from the specified {{Code|$database}}, or only documents matching the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get('docs')}} returns all documents from the database named {{Code|docs}}.<br />
* {{Code|db:get('db', 'one')}} returns all documents from the database named {{Code|db}} located in the path {{Code|one}}.<br />
* <code>for $i in 1 to 3 return db:get('db' || $i)//item</code> returns all item elements from the databases {{Code|db1}}, {{Code|db2}} and {{Code|db3}}.<br />
|}<br />
<br />
==db:get-pre==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-pre(<br />
$database as xs:string,<br />
$values as xs:integer*<br />
) as node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all nodes from a {{Code|$database}} with the specified PRE {{Code|values}} in [[Utility Module#util:ddo|distinct document order]].<br/>The [[Node Storage#PRE Value|PRE value]] provides very fast access to an existing database node, but it will change whenever a node with a smaller ''pre'' values is added to or deleted from a database.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|range|#Errors}} the specified PRE value does not exist in the database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get-pre("docs", 0)}} returns the first database node from the database named {{Code|docs}}.<br />
|}<br />
<br />
==db:get-id==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-id(<br />
$database as xs:string,<br />
$values as xs:integer*<br />
) as node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all nodes from a {{Code|$database}} with the specified ID {{Code|$values}} in [[Utility Module#util:ddo|distinct document order]].<br/>Each database node has a ''persistent'' [[Node Storage#ID Value|ID value]]. Access to the node ID can be sped up by turning on the {{Option|UPDINDEX}} option.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|range|#Errors}} the specified ID value does not exist in the database.<br />
|}<br />
<br />
==db:get-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-binary(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as item()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns a map with the paths and binary items of all resources in the specified {{Code|$database}}. A single {{Code|xs:base64Binary}} item is returned if a {{Code|$path}} is specified. All items are [[Lazy Module|lazy]], i.e., the actual data will only be retrieved if it is processed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get-binary('DB', 'music/01.mp3')}} returns the specified audio file as raw data.<br />
* <code><nowiki>stream:materialize(db:get-binary('DB', 'music/01.mp3'))</nowiki></code> materializes the streamable result in main-memory before returning it.<br />
* <code><nowiki>convert:binary-to-string(db:get-binary('DB', 'info.txt'), 'UTF-8')</nowiki></code> converts a binary database resource as UTF-8 text and returns a string.<br />
|}<br />
<br />
==db:get-value==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-value(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns a map with the paths and values of all resources in the specified {{Code|$database}}. A single value is returned if a {{Code|$path}} is specified.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get-value('DB', 'sequence')}} returns the specified sequence.<br />
|}<br />
<br />
==db:node-pre==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:node-pre(<br />
$nodes as node()*<br />
) as xs:integer*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the PRE values of the specified {{Code|$nodes}}, which must all be [[#Database Nodes|database nodes]].<br/>The [[Node Storage#PRE Value|PRE value]] provides very fast access to an existing database node, but it will change whenever a node with a smaller ''pre'' values is added to or deleted from a database.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:node-pre(doc("input"))}} returns {{Code|0}} if the database {{Code|input}} contains a single document.<br />
|}<br />
<br />
==db:node-id==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:node-id(<br />
$nodes as node()*<br />
) as xs:integer*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the ID values of the specified {{Code|$nodes}}, which must all be [[#Database Nodes|database nodes]].<br/>Each database node has a ''persistent'' [[Node Storage#ID Value|ID value]]. Access to the node id can be sped up by turning on the {{Option|UPDINDEX}} option.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|}<br />
<br />
==db:export==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:export(<br />
$database as xs:string,<br />
$path as xs:string,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Exports the specified {{Code|$database}} to the specified file {{Code|$path}}. Existing files will be overwritten.<br/>The {{Code|$options}} argument contains [[Serialization|serialization parameters]] (see [https://www.w3.org/TR/xpath-functions-31/#func-serialize fn:serialize]).<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
| Export all files as text:<br/><br />
<pre lang='xquery'><br />
db:export("DB", "/home/john/xml/texts", map { 'method': 'text' })<br />
</pre><br />
The following code can be used to export parts of the database:<br />
<pre lang='xquery'><br />
let $target := '/home/john/xml/target'<br />
for $doc in db:get('DB', 'collection')<br />
let $path := $target || db:path($doc)<br />
return (<br />
file:create-dir(file:parent($path)),<br />
file:write($path, $doc)<br />
)<br />
</pre><br />
|}<br />
<br />
=Value Indexes=<br />
<br />
==db:text==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:text(<br />
$database as xs:string,<br />
$values as xs:string*<br />
) as text()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all text nodes of a {{Code|$database}} that match one of the specified {{Code|$values}} and that are stored in the text index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:text("DB", "QUERY")/..}} returns the parents of all text nodes of the database {{Code|DB}} that match the string {{Code|QUERY}}.<br />
|}<br />
<br />
==db:text-range==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:text-range(<br />
$database as xs:string,<br />
$min as xs:string,<br />
$max as xs:string<br />
) as text()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all text nodes of a {{Code|$database}} whose values are larger than or equal to {{Code|$min}} and smaller than or equal to {{Code|$max}} and that are stored in the text index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:text-range("DB", "2000", "2001")}} returns all text nodes of the database {{Code|DB}} that are found in between {{Code|2000}} and {{Code|2001}}.<br />
|}<br />
<br />
==db:attribute==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:attribute(<br />
$database as xs:string,<br />
$values as xs:string*,<br />
$name as xs:string := ()<br />
) as attribute()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all attribute nodes of a {{Code|$database}} that match one of the specified {{Code|$values}} and that are stored in the attribute index.<br/>If {{Code|$name}} is specified, the resulting attribute nodes are filtered by their attribute name.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:attribute("DB", "QUERY", "id")/..}} returns the parents of all {{Code|id}} attribute nodes of the database {{Code|DB}} that have {{Code|QUERY}} as string value.<br />
|}<br />
<br />
==db:attribute-range==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:attribute-range(<br />
$database as xs:string,<br />
$min as xs:string,<br />
$max as xs:string,<br />
$name as xs:string := ()<br />
) as attribute()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all attributes of a {{Code|$database}} whose values are larger than or equal to {{Code|$min}} and smaller than or equal to {{Code|$max}} and that are stored in the attribute index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:attribute-range("DB", "id456", "id473", 'id')}} returns all {{Code|@id}} attributes of the database {{Code|DB}} that have a string value in between {{Code|id456}} and {{Code|id473}}.<br />
|}<br />
<br />
==db:token==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:token(<br />
$database as xs:string,<br />
$tokens as xs:string*,<br />
$name as xs:string := ()<br />
) as attribute()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all attribute nodes of a {{Code|$database}} the values of which contain one of the specified {{Code|$tokens}}.<br/>If {{Code|$name}} is specified, the resulting attribute nodes are filtered by their attribute name.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:token("DB", "row", "class")/parent::div}} returns all {{Code|div}} nodes of database {{Code|DB}} with a {{Code|class}} attribute that contains the token {{Code|row}}.<br />
|}<br />
<br />
=Updates=<br />
<br />
All functions in this section are [[#Updating Functions|Updating Functions]].<br />
<br />
==db:create==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:create(<br />
$database as xs:string,<br />
$inputs as item()* := (),<br />
$paths as xs:string* := (),<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a new {{Code|$database}} and adds the supplied {{Code|$inputs}} to the specified {{Code|$paths}}:<br />
* The inputs may be strings or nodes:<br />
** nodes may be of any type except for attributes<br />
** strings can be a URI pointing to a file/directory or an XML string (which is detected by the leading <code>&lt;</code> character)<br />
** a path must be specified if the input is not a file or directory reference<br />
* The parsing and indexing behavior can be controlled via {{Code|$options}}:<br />
** allowed options are {{Option|ADDCACHE}} and the [[Options#Indexing|indexing]], [[Options#Full-Text Indexing|full-text indexing]], [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options, all in lower case<br />
** parsing options will only impact string input (URIs, XML strings) because nodes have already been parsed.<br />
* An existing database will be overwritten.<br />
* Database creation takes place after most other update operations (see [[XQuery Update#Pending Update List|Pending Update List]]). As a consequence, a newly created database cannot be addressed in the same query.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} the specified name is not a [[Commands#Valid_Names|valid database name]].<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br/>{{Error|args|#Errors}} the number of specified inputs and paths differs.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:create("DB")}} creates the empty database {{Code|DB}}.<br />
* {{Code|db:create("DB", "/home/dir/doc.xml")}} creates the database {{Code|DB}} and adds the document {{Code|/home/dir/doc.xml}} as initial content.<br />
* {{Code|db:create("DB", <a/>, "doc.xml")}} creates the database {{Code|DB}} and adds the document with content {{Code|&lt;a/&gt;}} under the name {{Code|doc.xml}}.<br />
* {{Code|db:create("DB", "/home/dir/", "docs/dir")}} creates the database {{Code|DB}} and adds the documents in {{Code|/home/dir}} to the database under the path {{Code|docs/dir}}.<br />
* <code>db:create("DB", file:list('.'), (), map { 'ftindex': true() })</code> adds all files of the current working directory to a new database, preserving relative filesystem paths and creating a full-text index.<br />
|}<br />
<br />
==db:add==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:add(<br />
$database as xs:string,<br />
$input as item(),<br />
$path as xs:string? := (),<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Adds the specified {{Code|$input}} to a {{Code|$database}} with the specified {{Code|$path}}:<br />
* A document with the same path may occur more than once in a database. If you want to enforce single instances, use {{Function||db:put}} instead.<br />
* See {{Function||db:create}} for more details on the input and path arguments.<br />
* The parsing behavior can be controlled via {{Code|$options}}:<br />
** allowed options are {{Option|ADDCACHE}} and the [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options, all in lower case<br />
** parsing options will only impact string input (URIs, XML strings) because nodes have already been parsed<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>db:add("DB", "/home/dir/doc.xml")</code> adds the file {{Code|/home/dir/doc.xml}} to the database {{Code|DB}}.<br />
* <code>db:add("DB", <a/>, "doc.xml")</code> adds a document node to the database {{Code|DB}} under the name {{Code|doc.xml}}.<br />
* <code>db:add("DB", "/home/dir", "docs/dir", map { 'addcache': true() })</code> adds all documents in {{Code|/home/dir}} to the database {{Code|DB}} under the path {{Code|docs/dir}}. To reduce memory consumption, the files will be cached before being added to the database.<br />
|}<br />
<br />
==db:put==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:put(<br />
$database as xs:string,<br />
$input as item(),<br />
$path as xs:string,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Replaces a resource, specified by {{Code|$path}}, in a {{Code|$database}} with the contents of {{Code|$input}}, or adds it as a new resource:<br />
* The parsing behavior can be controlled via {{Code|$options}}:<br />
** Allowed options are {{Option|ADDCACHE}} and the [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options, all in lower case.<br />
** Parsing options will only impact string input (URIs, XML strings), because nodes have already been parsed.<br />
* See {{Function||db:create}} for more details on the input argument.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|target|#Errors}} the path points to a directory.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:put("DB", "/home/dir/doc.xml", "docs/dir/doc.xml")}} replaces the content of the document {{Code|docs/dir/doc.xml}} in the database {{Code|DB}} with the content of the file {{Code|/home/dir/doc.xml}}.<br />
* {{Code|db:put("DB", "<a/>", "docs/dir/doc.xml")}} replaces the content of the document {{Code|docs/dir/doc.xml}} in the database {{Code|DB}} with {{Code|&lt;a/&gt;}}.<br />
* {{Code|db:put("DB", document { <a/> }, "docs/dir/doc.xml")}} replaces the content of the document {{Code|docs/dir/doc.xml}} in the database {{Code|DB}} with the specified document node.<br />
The following query can be used to import files from a directory to a database:<br />
<pre lang='xquery'><br />
let $source := '/home/john/xml/source'<br />
for $file in file:list($source, true())<br />
let $path := $source || $file<br />
where not(file:is-dir($path))<br />
return db:put('db', doc($path), $file)<br />
</pre><br />
|}<br />
<br />
==db:put-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:put-binary(<br />
$database as xs:string,<br />
$input as item(),<br />
$path as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Stores a binary resource specified by {{Code|$input}} in a {{Code|$database}} at the specified {{Code|$path}}. Existing resources are overwritten.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:put-binary('DB', file:read-binary('video.mov'), 'video/sample.mov')}} stores the addressed video file at the specified location.<br />
* With the following query, you can copy the binary resources of one database into another:<br />
<pre lang='xquery'><br />
let $db := 'db'<br />
let $src-path := 'src/'<br />
let $trg-path := 'trg/'<br />
for $src in db:list($db, $src-path)<br />
where db:type($db, $src) = 'binary'<br />
let $trg := $trg-path || substring-after($src, $src-path)<br />
return db:put-binary($db, db:get-binary($db, $src), $trg)<br />
</pre><br />
|}<br />
<br />
==db:put-value==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:put-value(<br />
$database as xs:string,<br />
$input as item()*,<br />
$path as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Stores an {{Code|$input}} value in a {{Code|$database}} at the specified {{Code|$path}}. Existing resources are overwritten. The value can be an arbitrary sequence of atomic items, nodes, maps, and arrays.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:put-value('DB', 1 to 10000, 'sequence')}} stores a numeric range in the database.<br />
* With the following query, a map with countries and associated cities is stored in a database. The value resource can e.g. be used as index in future queries:<br />
<pre lang='xquery'><br />
db:put-value(<br />
'factbook',<br />
map:merge(<br />
for $country in db:get('factbook')//country<br />
return map:entry($country/@name, $country//city/name ! string())<br />
),<br />
'cities'<br />
)</pre><br />
|}<br />
<br />
==db:delete==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:delete(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Deletes resource(s), specified by {{Code|$path}}, from the specified {{Code|$database}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|path|#Errors}} the specified path is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:delete("DB", "docs/dir/doc.xml")}} deletes the resource {{Code|docs/dir/doc.xml}} from {{Code|DB}}.<br />
* {{Code|db:delete("DB", "docs/dir")}} deletes all resources from {{Code|DB}} in the specified path {{Code|docs/dir}}.<br />
|}<br />
<br />
==db:copy==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:copy(<br />
$database as xs:string,<br />
$new-name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a copy of {{Code|$database}}, which will be called {{Code|$new-name}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|}<br />
<br />
==db:alter==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:alter(<br />
$database as xs:string,<br />
$new-name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Renames a {{Code|$database}} to {{Code|$new-name}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|}<br />
<br />
==db:optimize==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:optimize(<br />
$database as xs:string,<br />
$all as xs:boolean? := false(),<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Optimizes the metadata and indexes of a {{Code|$database}}.<br/>If {{Code|$all}} is {{Code|true}}, the complete database will be rebuilt.<br/>The {{Code|$options}} argument can be used to control indexing. The syntax is identical to the {{Function||db:create}} function: Allowed options are all [[Options#Indexing|indexing]] and [[Options#Full-Text|full-text]] options. {{Option|UPDINDEX}} is only supported if {{Code|$all}} is {{Code|true}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:optimize("DB")}} optimizes the database structures of the database {{Code|DB}}.<br />
* <code>db:optimize("DB", true(), map { 'ftindex': true() })</code> optimizes all database structures of the database {{Code|DB}} and creates a full-text index.<br />
|}<br />
<br />
==db:rename==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:rename(<br />
$database as xs:string,<br />
$source as xs:string,<br />
$target as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Moves all resources(s) of a {{Code|$database}}, which are found in the supplied {{Code|$source}} path, to the supplied {{Code|$target}} path. The paths may point to single resources or directories. No updates will take place if a non-existing source path is supplied.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|path|#Errors}} the specified source or target path, or one of its descendants, is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:rename("DB", "docs/dir/doc.xml", "docs/dir/newdoc.xml")}} renames the resource {{Code|docs/dir/doc.xml}} to {{Code|docs/dir/newdoc.xml}} in the database {{Code|DB}}.<br />
* {{Code|db:rename("DB", "docs/dir", "docs/newdir")}} moves all resources in the database {{Code|DB}} from {{Code|docs/dir}} to {{Code|docs/newdir}}.<br />
|}<br />
<br />
==db:flush==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:flush(<br />
$database as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Explicitly flushes the buffers of a {{Code|$database}}. This command is only useful if {{Option|AUTOFLUSH}} has been set to {{Code|false}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|}<br />
<br />
==db:drop==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:drop(<br />
$database as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Drops a {{Code|$database}} and all connected resources.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:drop("DB")}} drops the database {{Code|DB}}.<br />
|}<br />
<br />
=Backups=<br />
<br />
All functions in this section except for {{Function||db:backups}} are {{Function|Database|Updating Functions}}.<br />
<br />
==db:create-backup==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:create-backup(<br />
$database as xs:string,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a backup of a {{Code|$database}}. If no name is supplied, general data will be backed up. The following {{Code|$options}} are available:<br />
* With {{Code|comment}}, a comment string can be attached to the backup.<br />
* By setting {{Code|compress}} to false, the backup will be created faster, but it will take more space on disk.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:create-backup('DB', map { 'compress': false() })}} creates a backup of the database {{Code|DB}} without compressing its entries.<br />
|}<br />
<br />
==db:drop-backup==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:drop-backup(<br />
$name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Drops all backups of the database with the specified {{Code|$name}}. If the name ends with a timestamp, only the specified backup file will be deleted. If no name is supplied, backups with general data are addressed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|backup|#Errors}} No backup file found.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:drop-backup("DB")}} drops all backups of the database {{Code|DB}}.<br />
* {{Code|db:drop-backup("DB-2014-03-13-17-36-44")}} drops the specific backup file {{Code|DB-2014-03-13-17-36-44.zip}} of the database {{Code|DB}}.<br />
|}<br />
<br />
==db:alter-backup==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:alter-backup(<br />
$name as xs:string,<br />
$new-name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Renames all backups of the database with the specified {{Code|$name}} to {{Code|$new-name}}. If the name ends with a date, only the specified backup file will be renamed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|backup|#Errors}} No backup file found.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:alter-backup("DB", "DB2)}} renames all backups of the database {{Code|DB}} to {{Code|DB2}}.<br />
|}<br />
<br />
==db:restore==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:restore(<br />
$name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Restores the database with the specified {{Code|$name}}. The {{Code|$name}} may include the timestamp of the backup file. If no name is supplied, general data will be restored. If general data is restored, it will only be available after BaseX has been restarted.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|no-backup|#Errors}} No backup found.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:restore("DB")}} restores the database {{Code|DB}}.<br />
* {{Code|db:restore("DB-2014-03-13-18-05-45")}} restores the database {{Code|DB}} from the backup file with the given timestamp.<br />
|}<br />
<br />
==db:backups==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:backups(<br />
$database as xs:string := ()<br />
) as element(backup)*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns an element sequence containing all available database backups with timestamp, file size and comment.<br/>If a {{Code|$database}} is specified, the sequence will be restricted to the backups matching this database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:backups("factbook")}} returns all backups that have been made from the {{Code|factbook}} database.<br />
|}<br />
<br />
=Helper Functions=<br />
<br />
==db:name==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:name(<br />
$node as node()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the name of the database in which the specified [[#Database Nodes|database node]] {{Code|$node}} is stored.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|}<br />
<br />
==db:path==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:path(<br />
$node as node()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the path of the database document in which the specified [[#Database Nodes|database node]] {{Code|$node}} is stored.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|}<br />
<br />
==db:exists==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:exists(<br />
$database as xs:string,<br />
$path as xs:string := ()<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Checks if a {{Code|$database}} exists, or a resource located at {{Code|$path}} in this database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:exists("DB")}} returns {{Code|true}} if the database {{Code|DB}} exists.<br />
* {{Code|db:exists("DB", "resource")}} returns {{Code|true}} if {{Code|resource}} exists in this database.<br />
|}<br />
<br />
==db:type==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:type(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the type ({{Code|xml}}, {{Code|binary}}, {{Code|value}}) of a resource in a {{Code|$database}} at the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:type("DB", "factbook.xml")}} returns {{Code|true}} if the specified resource is an XML document.<br />
|}<br />
<br />
==db:content-type==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:content-type(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Retrieves the content-type of a resource in a {{Code|$database}} at the specified {{Code|$path}}.<br/>The file extension is used to recognize the content-type of a resource stored in the database. {{Code|application/xml}} will be returned for any XML document stored in the database, regardless of its file name extension.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:content-type("DB", "docs/doc01.pdf")}} returns {{Code|application/pdf}}.<br />
* {{Code|db:content-type("DB", "docs/doc01.xml")}} returns {{Code|application/xml}}.<br />
* {{Code|db:content-type("DB", "docs/doc01")}} returns {{Code|application/xml}}, if {{Code|db:is-xml("DB", "docs/doc01")}} returns {{Code|true}}.<br />
|}<br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
| Description<br />
|- valign="top"<br />
|{{Code|args}}<br />
|The number of specified inputs and paths differs.<br />
|- valign="top"<br />
|{{Code|conflict}}<br />
|Multiple update operations point to the same target.<br />
|- valign="top"<br />
|{{Code|lock}}<br />
|A database cannot be updated because it is opened by another process.<br />
|- valign="top"<br />
|{{Code|mainmem}}<br />
|The addressed database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
|{{Code|name}}<br />
|The name of the specified database is invalid.<br />
|- valign="top"<br />
|{{Code|no-backup}}<br />
|No backup exists for a database.<br />
|- valign="top"<br />
|{{Code|node}}<br />
|The referenced XML node is no [[#Database Nodes|database node]], i.e. it is neither stored in a database nor represented as database fragment.<br />
|- valign="top"<br />
|{{Code|no-index}}<br />
|The database lacks an index structure required by the called function.<br />
|- valign="top"<br />
|{{Code|open}}<br />
|The addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
|{{Code|option}}<br />
|The specified option is unknown.<br />
|- valign="top"<br />
|{{Code|path}}<br />
|The specified database path is invalid.<br />
|- valign="top"<br />
|{{Code|property}}<br />
|The specified database property is unknown.<br />
|- valign="top"<br />
|{{Code|range}}<br />
|The addressed database ID or PRE value is out of range.<br />
|- valign="top"<br />
|{{Code|target}}<br />
|Path points to an invalid target.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 10<br />
* Added: {{Function||db:get}}, {{Function||db:put}}, {{Function||db:type}}.<br />
* Added: [[#Backups|Backups]]: Support for general data ([[User Management|registered users]], [[Job Module#Services|scheduled services]] and [[Store Module|key-value stores]]).<br />
* Updated: {{Function||db:get}}, {{Function||db:get-id}}, {{Function||db:get-pre}} renamed (before: {{Code|db:open}}, {{Code|db:open-id}}, {{Code|db:open-pre}})<br />
* Updated: {{Function||db:put}} renamed (before: {{Code|db:replace}}); function signature aligned with {{Function||db:add}} (second and third argument swapped).<br />
* Updated: {{Function||db:put-binary}} renamed (before: {{Code|db:store}}); function signature aligned with {{Function||db:add}} (second and third argument swapped).<br />
* Updated: {{Function||db:get-binary}} renamed (before: {{Code|db:retrieve}}).<br />
* Updated: {{Function||db:backups}}, {{Function||db:create-backup}}: Options added.<br />
* Removed: {{Code|db:is-raw}}, {{Code|db:is-raw}} (new: {{Function||db:type}}).<br />
<br />
;Version 9.3<br />
* Added: {{Function||db:alter-backup}}<br />
* Updated: {{Code|db:open-id}}, {{Code|db:open-pre}}: support for multiple integers<br />
<br />
;Version 9.2<br />
* Added: {{Function||db:dir}}<br />
* Updated: {{Function||db:add}}: {{Code|$path}} allow empty path argument<br />
<br />
;Version 9.0<br />
* Added: {{Function||db:option}}<br />
* Updated: db:output renamed to {{Function|Update|update:output}}, db:output-cache renamed to {{Function|Update|update:cache}}<br />
* Updated: error codes updated; errors now use the module namespace<br />
<br />
;Version 8.6<br />
* Added: {{Function||db:property}}<br />
<br />
;Version 8.4<br />
* Updated: {{Function||db:create}}, {{Function||db:add}}, {{Code|db:replace}}: support for {{Code|ADDCACHE}} option.<br />
* Added: {{Function||db:token}}<br />
<br />
;Version 8.3<br />
* Updated: {{Function||db:list-details}}: attributes with name of database and date of backup added to results.<br />
* Updated: {{Function||db:backups}} now include attributes with name of database and date of backup.<br />
* Updated: {{Function|Database|Value Indexes}}: raise error if no index exists.<br />
<br />
;Version 8.2<br />
* Added: {{Function||db:output-cache}}<br />
* Removed: db:event<br />
<br />
;Version 7.9<br />
* Updated: parsing options added to {{Function||db:create}}, {{Function||db:add}} and {{Code|db:replace}}.<br />
* Updated: allow {{Option|UPDINDEX}} if {{Code|$all}} is {{Code|true}}.<br />
<br />
;Version 7.8.2<br />
* Added: {{Function||db:alter}}, {{Function||db:copy}}, {{Function||db:create-backup}}, {{Function||db:drop-backup}}, {{Function||db:restore}}<br />
<br />
;Version 7.8<br />
* Removed: db:fulltext (use {{Function|Full-Text|ft:search}} instead)<br />
<br />
;Version 7.7<br />
* Added: {{Function||db:export}}, {{Function||db:name}}, {{Function||db:path}}<br />
* Updated: {{Code|$options}} argument added to {{Function||db:create}} and {{Function||db:optimize}}.<br />
* Updated: the functions no longer accept [[#Database Nodes|database nodes]] as reference. Instead, the name of a database must now be specified.<br />
<br />
;Version 7.6<br />
* Updated: {{Function||db:create}}: allow more than one input and path.<br />
<br />
;Version 7.5<br />
* Updated: {{Function||db:add}}: input nodes will be automatically converted to document nodes<br />
* Added: {{Function||db:backups}}<br />
* Added: {{Function||db:create}}<br />
* Added: {{Function||db:drop}}<br />
<br />
;Version 7.3<br />
* Added: {{Function||db:flush}}<br />
<br />
;Version 7.2.1<br />
* Added: {{Function||db:text-range}}, {{Function||db:attribute-range}}, {{Function||db:output}}<br />
<br />
;Version 7.1<br />
* Added: {{Function||db:list-details}}, {{Function||db:content-type}}<br />
* Updated: {{Function||db:info}}, {{Function||db:system}}, {{Code|db:retrieve}}<br />
<br />
;Version 7.0<br />
* Added: {{Function||db:exists}}, {{Code|db:retrieve}}, {{Code|db:store}}, {{Code|db:is-raw}}, {{Code|db:is-xml}}<br />
* Updated: {{Function||db:list}}, {{Code|db:open}}, {{Function||db:add}}</div>CGhttps://docs.basex.org/index.php?title=Database_Module&diff=16916Database Module2023-12-01T21:07:32Z<p>CG: </p>
<hr />
<div>This [[Module Library|XQuery Module]] contains functions for processing databases from within XQuery. Existing databases can be opened and listed, its contents can be directly accessed, documents can be added to and removed, etc.<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/db</nowiki></code> namespace, which is statically bound to the {{Code|db}} prefix.<br/><br />
<br />
==Database Nodes==<br />
<br />
In BaseX, two internal representations exist for XML nodes:<br />
<br />
1. Database nodes are:<br />
* stored in a persistent database on disk;<br />
* nodes of a document that has been generated temporarily with {{Code|fn:doc}}, {{Code|fn:parse-xml}} and other functions; or<br />
* the result of a [[XQuery Update#Main-Memory Updates|main-memory update]].<br />
2. Fragments are similar to DOM structures. They are created when XQuery node constructors are used (<code><a/></code>, <code>element a { }</code>, etc.).<br />
<br />
Fragments require less memory for small XML structures, but database nodes are more efficient for larger amounts of data. Fragments nodes can be converted to database nodes by applying a [[XQuery_Update#Main-Memory Updates|main-memory update]] with an empty body to a node:<br />
<br />
<pre lang='xquery'><br />
<xml>hello world</xml> update { }<br />
</pre><br />
<br />
==Updating Functions==<br />
<br />
Various functions in this module are ''updating''. Updating functions will not be immediately executed, but queued on the [[XQuery Update#Pending Update List|Pending Update List]], and processed after the remaining query has been evaluated. This means that the order in which the functions are specified in the query often does not reflect the order in which they will eventually be executed.<br />
<br />
=General Functions=<br />
<br />
==db:system==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:system() as element(system)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns general information on the database system the current values of all global and local [[Options]]. The {{Command|INFO}} command returns similar output.<br />
|}<br />
<br />
==db:option==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:option(<br />
$name as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the current value (string, integer, boolean, map) of a global or local [[Options|Option]] with the specified {{Code|$name}}. The {{Command|SHOW OPTIONS}} command returns similar output.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|option|#Errors}} the specified option is unknown.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>db:option('dbpath')</code> returns the database path string.<br />
* <code>db:option('serializer')</code> returns a map with the current serialization parameters.<br />
* <code>declare option db:stripws 'true'; db:option('stripws')</code> returns the locally assigned value.<br />
|}<br />
<br />
==db:info==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:info(<br />
$database as xs:string<br />
) as element(database)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns meta information on the specified {{Code|$database}}. The output is similar to the {{Command|INFO DB}} command.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|}<br />
<br />
==db:property==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:property(<br />
$database as xs:string,<br />
$name as xs:string<br />
) as xs:anyAtomicType</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the value (string, boolean, integer) of a property with the specified {{Code|$name}} in the specified {{Code|$database}}. The available properties are the ones returned by {{Function||db:info}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|property|#Errors}} the specified property is unknown.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>db:property('db', 'size')</code> returns the number of bytes occupied by the database <code>db</code>.<br />
* <code>db:property('xmark', 'textindex')</code> indicates if the <code>xmark</code> database has a text index.<br />
* <code>db:property('discogs', 'uptodate')</code> indicates if the database statistics and index structures of the <code>discogs</code> database are up-to-date.<br />
|}<br />
<br />
==db:list==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:list(<br />
$database as xs:string := (),<br />
$path as xs:string := ()<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Without arguments, the names of all databases are returned that are accessible to the current user. If {{Code|$database}} is specified, paths to all resources of this database are returned. The results can be restricted to resources starting with the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:list("docs")}} returns the names of all documents of a database named {{Code|docs}}.<br />
|}<br />
<br />
==db:list-details==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:list-details(<br />
$database as xs:string := (),<br />
$path as xs:string := ()<br />
) as element()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Without arguments, an element is returned for each database that is accessible to the current user:<br />
* An element has a value, which is the name of the database, and several attributes, which contain the number of stored resources, the modification date, the database size on disk (measured in bytes), and a path to the original database input.<br />
If {{Code|$database}} is specified, an element for each resource in this database is returned:<br />
* An element has a value, which is the name of the resource, and several attributes, which contain the content type, the modification date, the raw flag (which indicates if the resource is binary or XML), and the size of a resource.<br />
* The value of the size attribute depends on the resource type: for documents, it represents the number of nodes; for binary data, it represents the file size (measured in bytes).<br />
* The results can be restricted to resources starting with the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:list-details("shop")}} returns the names plus additional info on all resources of a database named {{Code|shop}}.<br />
|}<br />
<br />
==db:dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:dir(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as element()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns metadata on all directories and resources of a {{Code|$database}} in the specified {{Code|$path}}. Two types of elements are returned:<br />
* {{Code|resource}} represents a resource. The element value is the directory path; content type, modification date, raw flag (which indicates if the resource is binary or XML), and size of the resource are returned as attributes.<br />
* {{Code|dir}} represents a directory. The element value is the directory path; the modification date is returned in an attribute.<br />
The directories are not stored in the internal database layout. Instead, they result implicitly from the paths of stored resources.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|path|#Errors}} the specified path is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:dir('shop', 'books')}} returns all entries of the {{Code|books}} directory of a {{Code|shop}} database.<br />
|}<br />
<br />
=Read Operations=<br />
<br />
==db:get==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get(<br />
$database as xs:string,<br />
$path as xs:string := ()<br />
) as document-node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all documents from the specified {{Code|$database}}, or only documents matching the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get('docs')}} returns all documents from the database named {{Code|docs}}.<br />
* {{Code|db:get('db', 'one')}} returns all documents from the database named {{Code|db}} located in the path {{Code|one}}.<br />
* <code>for $i in 1 to 3 return db:get('db' || $i)//item</code> returns all item elements from the databases {{Code|db1}}, {{Code|db2}} and {{Code|db3}}.<br />
|}<br />
<br />
==db:get-pre==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-pre(<br />
$database as xs:string,<br />
$values as xs:integer*<br />
) as node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all nodes from a {{Code|$database}} with the specified PRE {{Code|values}} in [[Utility Module#util:ddo|distinct document order]].<br/>The [[Node Storage#PRE Value|PRE value]] provides very fast access to an existing database node, but it will change whenever a node with a smaller ''pre'' values is added to or deleted from a database.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|range|#Errors}} the specified PRE value does not exist in the database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get-pre("docs", 0)}} returns the first database node from the database named {{Code|docs}}.<br />
|}<br />
<br />
==db:get-id==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-id(<br />
$database as xs:string,<br />
$values as xs:integer*<br />
) as node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all nodes from a {{Code|$database}} with the specified ID {{Code|$values}} in [[Utility Module#util:ddo|distinct document order]].<br/>Each database node has a ''persistent'' [[Node Storage#ID Value|ID value]]. Access to the node ID can be sped up by turning on the {{Option|UPDINDEX}} option.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|range|#Errors}} the specified ID value does not exist in the database.<br />
|}<br />
<br />
==db:get-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-binary(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as item()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns a map with the paths and binary items of all resources in the specified {{Code|$database}}. A single {{Code|xs:base64Binary}} item is returned if a {{Code|$path}} is specified. All items are [[Lazy Module|lazy]], i.e., the actual data will only be retrieved if it is processed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get-binary('DB', 'music/01.mp3')}} returns the specified audio file as raw data.<br />
* <code><nowiki>stream:materialize(db:get-binary('DB', 'music/01.mp3'))</nowiki></code> materializes the streamable result in main-memory before returning it.<br />
* <code><nowiki>convert:binary-to-string(db:get-binary('DB', 'info.txt'), 'UTF-8')</nowiki></code> converts a binary database resource as UTF-8 text and returns a string.<br />
|}<br />
<br />
==db:get-value==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-value(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns a map with the paths and values of all resources in the specified {{Code|$database}}. A single value is returned if a {{Code|$path}} is specified.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get-value('DB', 'sequence')}} returns the specified sequence.<br />
|}<br />
<br />
==db:node-pre==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:node-pre(<br />
$nodes as node()*<br />
) as xs:integer*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the PRE values of the specified {{Code|$nodes}}, which must all be [[#Database Nodes|database nodes]].<br/>The [[Node Storage#PRE Value|PRE value]] provides very fast access to an existing database node, but it will change whenever a node with a smaller ''pre'' values is added to or deleted from a database.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:node-pre(doc("input"))}} returns {{Code|0}} if the database {{Code|input}} contains a single document.<br />
|}<br />
<br />
==db:node-id==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:node-id(<br />
$nodes as node()*<br />
) as xs:integer*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the ID values of the specified {{Code|$nodes}}, which must all be [[#Database Nodes|database nodes]].<br/>Each database node has a ''persistent'' [[Node Storage#ID Value|ID value]]. Access to the node id can be sped up by turning on the {{Option|UPDINDEX}} option.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|}<br />
<br />
==db:export==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:export(<br />
$database as xs:string,<br />
$path as xs:string,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Exports the specified {{Code|$database}} to the specified file {{Code|$path}}. Existing files will be overwritten.<br/>The {{Code|$options}} argument contains [[Serialization|serialization parameters]] (see [https://www.w3.org/TR/xpath-functions-31/#func-serialize fn:serialize()]).<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
| Export all files as text:<br/><br />
<pre lang='xquery'><br />
db:export("DB", "/home/john/xml/texts", map { 'method': 'text' })<br />
</pre><br />
The following code can be used to export parts of the database:<br />
<pre lang='xquery'><br />
let $target := '/home/john/xml/target'<br />
for $doc in db:get('DB', 'collection')<br />
let $path := $target || db:path($doc)<br />
return (<br />
file:create-dir(file:parent($path)),<br />
file:write($path, $doc)<br />
)<br />
</pre><br />
|}<br />
<br />
=Value Indexes=<br />
<br />
==db:text==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:text(<br />
$database as xs:string,<br />
$values as xs:string*<br />
) as text()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all text nodes of a {{Code|$database}} that match one of the specified {{Code|$values}} and that are stored in the text index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:text("DB", "QUERY")/..}} returns the parents of all text nodes of the database {{Code|DB}} that match the string {{Code|QUERY}}.<br />
|}<br />
<br />
==db:text-range==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:text-range(<br />
$database as xs:string,<br />
$min as xs:string,<br />
$max as xs:string<br />
) as text()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all text nodes of a {{Code|$database}} whose values are larger than or equal to {{Code|$min}} and smaller than or equal to {{Code|$max}} and that are stored in the text index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:text-range("DB", "2000", "2001")}} returns all text nodes of the database {{Code|DB}} that are found in between {{Code|2000}} and {{Code|2001}}.<br />
|}<br />
<br />
==db:attribute==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:attribute(<br />
$database as xs:string,<br />
$values as xs:string*,<br />
$name as xs:string := ()<br />
) as attribute()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all attribute nodes of a {{Code|$database}} that match one of the specified {{Code|$values}} and that are stored in the attribute index.<br/>If {{Code|$name}} is specified, the resulting attribute nodes are filtered by their attribute name.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:attribute("DB", "QUERY", "id")/..}} returns the parents of all {{Code|id}} attribute nodes of the database {{Code|DB}} that have {{Code|QUERY}} as string value.<br />
|}<br />
<br />
==db:attribute-range==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:attribute-range(<br />
$database as xs:string,<br />
$min as xs:string,<br />
$max as xs:string,<br />
$name as xs:string := ()<br />
) as attribute()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all attributes of a {{Code|$database}} whose values are larger than or equal to {{Code|$min}} and smaller than or equal to {{Code|$max}} and that are stored in the attribute index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:attribute-range("DB", "id456", "id473", 'id')}} returns all {{Code|@id}} attributes of the database {{Code|DB}} that have a string value in between {{Code|id456}} and {{Code|id473}}.<br />
|}<br />
<br />
==db:token==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:token(<br />
$database as xs:string,<br />
$tokens as xs:string*,<br />
$name as xs:string := ()<br />
) as attribute()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all attribute nodes of a {{Code|$database}} the values of which contain one of the specified {{Code|$tokens}}.<br/>If {{Code|$name}} is specified, the resulting attribute nodes are filtered by their attribute name.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:token("DB", "row", "class")/parent::div}} returns all {{Code|div}} nodes of database {{Code|DB}} with a {{Code|class}} attribute that contains the token {{Code|row}}.<br />
|}<br />
<br />
=Updates=<br />
<br />
All functions in this section are [[#Updating Functions|Updating Functions]].<br />
<br />
==db:create==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:create(<br />
$database as xs:string,<br />
$inputs as item()* := (),<br />
$paths as xs:string* := (),<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a new {{Code|$database}} and adds the supplied {{Code|$inputs}} to the specified {{Code|$paths}}:<br />
* The inputs may be strings or nodes:<br />
** nodes may be of any type except for attributes<br />
** strings can be a URI pointing to a file/directory or an XML string (which is detected by the leading <code>&lt;</code> character)<br />
** a path must be specified if the input is not a file or directory reference<br />
* The parsing and indexing behavior can be controlled via {{Code|$options}}:<br />
** allowed options are {{Option|ADDCACHE}} and the [[Options#Indexing|indexing]], [[Options#Full-Text Indexing|full-text indexing]], [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options, all in lower case<br />
** parsing options will only impact string input (URIs, XML strings) because nodes have already been parsed.<br />
* An existing database will be overwritten.<br />
* Database creation takes place after most other update operations (see [[XQuery Update#Pending Update List|Pending Update List]]). As a consequence, a newly created database cannot be addressed in the same query.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} the specified name is not a [[Commands#Valid_Names|valid database name]].<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br/>{{Error|args|#Errors}} the number of specified inputs and paths differs.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:create("DB")}} creates the empty database {{Code|DB}}.<br />
* {{Code|db:create("DB", "/home/dir/doc.xml")}} creates the database {{Code|DB}} and adds the document {{Code|/home/dir/doc.xml}} as initial content.<br />
* {{Code|db:create("DB", <a/>, "doc.xml")}} creates the database {{Code|DB}} and adds the document with content {{Code|&lt;a/&gt;}} under the name {{Code|doc.xml}}.<br />
* {{Code|db:create("DB", "/home/dir/", "docs/dir")}} creates the database {{Code|DB}} and adds the documents in {{Code|/home/dir}} to the database under the path {{Code|docs/dir}}.<br />
* <code>db:create("DB", file:list('.'), (), map { 'ftindex': true() })</code> adds all files of the current working directory to a new database, preserving relative filesystem paths and creating a full-text index.<br />
|}<br />
<br />
==db:add==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:add(<br />
$database as xs:string,<br />
$input as item(),<br />
$path as xs:string? := (),<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Adds the specified {{Code|$input}} to a {{Code|$database}} with the specified {{Code|$path}}:<br />
* A document with the same path may occur more than once in a database. If you want to enforce single instances, use {{Function||db:put}} instead.<br />
* See {{Function||db:create}} for more details on the input and path arguments.<br />
* The parsing behavior can be controlled via {{Code|$options}}:<br />
** allowed options are {{Option|ADDCACHE}} and the [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options, all in lower case<br />
** parsing options will only impact string input (URIs, XML strings) because nodes have already been parsed<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>db:add("DB", "/home/dir/doc.xml")</code> adds the file {{Code|/home/dir/doc.xml}} to the database {{Code|DB}}.<br />
* <code>db:add("DB", <a/>, "doc.xml")</code> adds a document node to the database {{Code|DB}} under the name {{Code|doc.xml}}.<br />
* <code>db:add("DB", "/home/dir", "docs/dir", map { 'addcache': true() })</code> adds all documents in {{Code|/home/dir}} to the database {{Code|DB}} under the path {{Code|docs/dir}}. To reduce memory consumption, the files will be cached before being added to the database.<br />
|}<br />
<br />
==db:put==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:put(<br />
$database as xs:string,<br />
$input as item(),<br />
$path as xs:string,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Replaces a resource, specified by {{Code|$path}}, in a {{Code|$database}} with the contents of {{Code|$input}}, or adds it as a new resource:<br />
* The parsing behavior can be controlled via {{Code|$options}}:<br />
** Allowed options are {{Option|ADDCACHE}} and the [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options, all in lower case.<br />
** Parsing options will only impact string input (URIs, XML strings), because nodes have already been parsed.<br />
* See {{Function||db:create}} for more details on the input argument.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|target|#Errors}} the path points to a directory.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:put("DB", "/home/dir/doc.xml", "docs/dir/doc.xml")}} replaces the content of the document {{Code|docs/dir/doc.xml}} in the database {{Code|DB}} with the content of the file {{Code|/home/dir/doc.xml}}.<br />
* {{Code|db:put("DB", "<a/>", "docs/dir/doc.xml")}} replaces the content of the document {{Code|docs/dir/doc.xml}} in the database {{Code|DB}} with {{Code|&lt;a/&gt;}}.<br />
* {{Code|db:put("DB", document { <a/> }, "docs/dir/doc.xml")}} replaces the content of the document {{Code|docs/dir/doc.xml}} in the database {{Code|DB}} with the specified document node.<br />
The following query can be used to import files from a directory to a database:<br />
<pre lang='xquery'><br />
let $source := '/home/john/xml/source'<br />
for $file in file:list($source, true())<br />
let $path := $source || $file<br />
where not(file:is-dir($path))<br />
return db:put('db', doc($path), $file)<br />
</pre><br />
|}<br />
<br />
==db:put-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:put-binary(<br />
$database as xs:string,<br />
$input as item(),<br />
$path as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Stores a binary resource specified by {{Code|$input}} in a {{Code|$database}} at the specified {{Code|$path}}. Existing resources are overwritten.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:put-binary('DB', file:read-binary('video.mov'), 'video/sample.mov')}} stores the addressed video file at the specified location.<br />
* With the following query, you can copy the binary resources of one database into another:<br />
<pre lang='xquery'><br />
let $db := 'db'<br />
let $src-path := 'src/'<br />
let $trg-path := 'trg/'<br />
for $src in db:list($db, $src-path)<br />
where db:type($db, $src) = 'binary'<br />
let $trg := $trg-path || substring-after($src, $src-path)<br />
return db:put-binary($db, db:get-binary($db, $src), $trg)<br />
</pre><br />
|}<br />
<br />
==db:put-value==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:put-value(<br />
$database as xs:string,<br />
$input as item()*,<br />
$path as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Stores an {{Code|$input}} value in a {{Code|$database}} at the specified {{Code|$path}}. Existing resources are overwritten. The value can be an arbitrary sequence of atomic items, nodes, maps, and arrays.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:put-value('DB', 1 to 10000, 'sequence')}} stores a numeric range in the database.<br />
* With the following query, a map with countries and associated cities is stored in a database. The value resource can e.g. be used as index in future queries:<br />
<pre lang='xquery'><br />
db:put-value(<br />
'factbook',<br />
map:merge(<br />
for $country in db:get('factbook')//country<br />
return map:entry($country/@name, $country//city/name ! string())<br />
),<br />
'cities'<br />
)</pre><br />
|}<br />
<br />
==db:delete==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:delete(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Deletes resource(s), specified by {{Code|$path}}, from the specified {{Code|$database}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|path|#Errors}} the specified path is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:delete("DB", "docs/dir/doc.xml")}} deletes the resource {{Code|docs/dir/doc.xml}} from {{Code|DB}}.<br />
* {{Code|db:delete("DB", "docs/dir")}} deletes all resources from {{Code|DB}} in the specified path {{Code|docs/dir}}.<br />
|}<br />
<br />
==db:copy==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:copy(<br />
$database as xs:string,<br />
$new-name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a copy of {{Code|$database}}, which will be called {{Code|$new-name}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|}<br />
<br />
==db:alter==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:alter(<br />
$database as xs:string,<br />
$new-name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Renames a {{Code|$database}} to {{Code|$new-name}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|}<br />
<br />
==db:optimize==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:optimize(<br />
$database as xs:string,<br />
$all as xs:boolean? := false(),<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Optimizes the metadata and indexes of a {{Code|$database}}.<br/>If {{Code|$all}} is {{Code|true}}, the complete database will be rebuilt.<br/>The {{Code|$options}} argument can be used to control indexing. The syntax is identical to the {{Function||db:create}} function: Allowed options are all [[Options#Indexing|indexing]] and [[Options#Full-Text|full-text]] options. {{Option|UPDINDEX}} is only supported if {{Code|$all}} is {{Code|true}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:optimize("DB")}} optimizes the database structures of the database {{Code|DB}}.<br />
* <code>db:optimize("DB", true(), map { 'ftindex': true() })</code> optimizes all database structures of the database {{Code|DB}} and creates a full-text index.<br />
|}<br />
<br />
==db:rename==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:rename(<br />
$database as xs:string,<br />
$source as xs:string,<br />
$target as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Moves all resources(s) of a {{Code|$database}}, which are found in the supplied {{Code|$source}} path, to the supplied {{Code|$target}} path. The paths may point to single resources or directories. No updates will take place if a non-existing source path is supplied.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|path|#Errors}} the specified source or target path, or one of its descendants, is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:rename("DB", "docs/dir/doc.xml", "docs/dir/newdoc.xml")}} renames the resource {{Code|docs/dir/doc.xml}} to {{Code|docs/dir/newdoc.xml}} in the database {{Code|DB}}.<br />
* {{Code|db:rename("DB", "docs/dir", "docs/newdir")}} moves all resources in the database {{Code|DB}} from {{Code|docs/dir}} to {{Code|docs/newdir}}.<br />
|}<br />
<br />
==db:flush==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:flush(<br />
$database as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Explicitly flushes the buffers of a {{Code|$database}}. This command is only useful if {{Option|AUTOFLUSH}} has been set to {{Code|false}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|}<br />
<br />
==db:drop==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:drop(<br />
$database as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Drops a {{Code|$database}} and all connected resources.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:drop("DB")}} drops the database {{Code|DB}}.<br />
|}<br />
<br />
=Backups=<br />
<br />
All functions in this section except for {{Function||db:backups}} are {{Function|Database|Updating Functions}}.<br />
<br />
==db:create-backup==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:create-backup(<br />
$database as xs:string,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a backup of a {{Code|$database}}. If no name is supplied, general data will be backed up. The following {{Code|$options}} are available:<br />
* With {{Code|comment}}, a comment string can be attached to the backup.<br />
* By setting {{Code|compress}} to false, the backup will be created faster, but it will take more space on disk.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:create-backup('DB', map { 'compress': false() })}} creates a backup of the database {{Code|DB}} without compressing its entries.<br />
|}<br />
<br />
==db:drop-backup==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:drop-backup(<br />
$name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Drops all backups of the database with the specified {{Code|$name}}. If the name ends with a timestamp, only the specified backup file will be deleted. If no name is supplied, backups with general data are addressed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|backup|#Errors}} No backup file found.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:drop-backup("DB")}} drops all backups of the database {{Code|DB}}.<br />
* {{Code|db:drop-backup("DB-2014-03-13-17-36-44")}} drops the specific backup file {{Code|DB-2014-03-13-17-36-44.zip}} of the database {{Code|DB}}.<br />
|}<br />
<br />
==db:alter-backup==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:alter-backup(<br />
$name as xs:string,<br />
$new-name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Renames all backups of the database with the specified {{Code|$name}} to {{Code|$new-name}}. If the name ends with a date, only the specified backup file will be renamed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|backup|#Errors}} No backup file found.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:alter-backup("DB", "DB2)}} renames all backups of the database {{Code|DB}} to {{Code|DB2}}.<br />
|}<br />
<br />
==db:restore==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:restore(<br />
$name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Restores the database with the specified {{Code|$name}}. The {{Code|$name}} may include the timestamp of the backup file. If no name is supplied, general data will be restored. If general data is restored, it will only be available after BaseX has been restarted.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|no-backup|#Errors}} No backup found.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:restore("DB")}} restores the database {{Code|DB}}.<br />
* {{Code|db:restore("DB-2014-03-13-18-05-45")}} restores the database {{Code|DB}} from the backup file with the given timestamp.<br />
|}<br />
<br />
==db:backups==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:backups(<br />
$database as xs:string := ()<br />
) as element(backup)*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns an element sequence containing all available database backups with timestamp, file size and comment.<br/>If a {{Code|$database}} is specified, the sequence will be restricted to the backups matching this database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:backups("factbook")}} returns all backups that have been made from the {{Code|factbook}} database.<br />
|}<br />
<br />
=Helper Functions=<br />
<br />
==db:name==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:name(<br />
$node as node()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the name of the database in which the specified [[#Database Nodes|database node]] {{Code|$node}} is stored.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|}<br />
<br />
==db:path==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:path(<br />
$node as node()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the path of the database document in which the specified [[#Database Nodes|database node]] {{Code|$node}} is stored.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|}<br />
<br />
==db:exists==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:exists(<br />
$database as xs:string,<br />
$path as xs:string := ()<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Checks if a {{Code|$database}} exists, or a resource located at {{Code|$path}} in this database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:exists("DB")}} returns {{Code|true}} if the database {{Code|DB}} exists.<br />
* {{Code|db:exists("DB", "resource")}} returns {{Code|true}} if {{Code|resource}} exists in this database.<br />
|}<br />
<br />
==db:type==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:type(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the type ({{Code|xml}}, {{Code|binary}}, {{Code|value}}) of a resource in a {{Code|$database}} at the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:type("DB", "factbook.xml")}} returns {{Code|true}} if the specified resource is an XML document.<br />
|}<br />
<br />
==db:content-type==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:content-type(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Retrieves the content-type of a resource in a {{Code|$database}} at the specified {{Code|$path}}.<br/>The file extension is used to recognize the content-type of a resource stored in the database. {{Code|application/xml}} will be returned for any XML document stored in the database, regardless of its file name extension.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:content-type("DB", "docs/doc01.pdf")}} returns {{Code|application/pdf}}.<br />
* {{Code|db:content-type("DB", "docs/doc01.xml")}} returns {{Code|application/xml}}.<br />
* {{Code|db:content-type("DB", "docs/doc01")}} returns {{Code|application/xml}}, if {{Code|db:is-xml("DB", "docs/doc01")}} returns {{Code|true}}.<br />
|}<br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
| Description<br />
|- valign="top"<br />
|{{Code|args}}<br />
|The number of specified inputs and paths differs.<br />
|- valign="top"<br />
|{{Code|conflict}}<br />
|Multiple update operations point to the same target.<br />
|- valign="top"<br />
|{{Code|lock}}<br />
|A database cannot be updated because it is opened by another process.<br />
|- valign="top"<br />
|{{Code|mainmem}}<br />
|The addressed database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
|{{Code|name}}<br />
|The name of the specified database is invalid.<br />
|- valign="top"<br />
|{{Code|no-backup}}<br />
|No backup exists for a database.<br />
|- valign="top"<br />
|{{Code|node}}<br />
|The referenced XML node is no [[#Database Nodes|database node]], i.e. it is neither stored in a database nor represented as database fragment.<br />
|- valign="top"<br />
|{{Code|no-index}}<br />
|The database lacks an index structure required by the called function.<br />
|- valign="top"<br />
|{{Code|open}}<br />
|The addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
|{{Code|option}}<br />
|The specified option is unknown.<br />
|- valign="top"<br />
|{{Code|path}}<br />
|The specified database path is invalid.<br />
|- valign="top"<br />
|{{Code|property}}<br />
|The specified database property is unknown.<br />
|- valign="top"<br />
|{{Code|range}}<br />
|The addressed database ID or PRE value is out of range.<br />
|- valign="top"<br />
|{{Code|target}}<br />
|Path points to an invalid target.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 10<br />
* Added: {{Function||db:get}}, {{Function||db:put}}, {{Function||db:type}}.<br />
* Added: [[#Backups|Backups]]: Support for general data ([[User Management|registered users]], [[Job Module#Services|scheduled services]] and [[Store Module|key-value stores]]).<br />
* Updated: {{Function||db:get}}, {{Function||db:get-id}}, {{Function||db:get-pre}} renamed (before: {{Code|db:open}}, {{Code|db:open-id}}, {{Code|db:open-pre}})<br />
* Updated: {{Function||db:put}} renamed (before: {{Code|db:replace}}); function signature aligned with {{Function||db:add}} (second and third argument swapped).<br />
* Updated: {{Function||db:put-binary}} renamed (before: {{Code|db:store}}); function signature aligned with {{Function||db:add}} (second and third argument swapped).<br />
* Updated: {{Function||db:get-binary}} renamed (before: {{Code|db:retrieve}}).<br />
* Updated: {{Function||db:backups}}, {{Function||db:create-backup}}: Options added.<br />
* Removed: {{Code|db:is-raw}}, {{Code|db:is-raw}} (new: {{Function||db:type}}).<br />
<br />
;Version 9.3<br />
* Added: {{Function||db:alter-backup}}<br />
* Updated: {{Code|db:open-id}}, {{Code|db:open-pre}}: support for multiple integers<br />
<br />
;Version 9.2<br />
* Added: {{Function||db:dir}}<br />
* Updated: {{Function||db:add}}: {{Code|$path}} allow empty path argument<br />
<br />
;Version 9.0<br />
* Added: {{Function||db:option}}<br />
* Updated: db:output renamed to {{Function|Update|update:output}}, db:output-cache renamed to {{Function|Update|update:cache}}<br />
* Updated: error codes updated; errors now use the module namespace<br />
<br />
;Version 8.6<br />
* Added: {{Function||db:property}}<br />
<br />
;Version 8.4<br />
* Updated: {{Function||db:create}}, {{Function||db:add}}, {{Code|db:replace}}: support for {{Code|ADDCACHE}} option.<br />
* Added: {{Function||db:token}}<br />
<br />
;Version 8.3<br />
* Updated: {{Function||db:list-details}}: attributes with name of database and date of backup added to results.<br />
* Updated: {{Function||db:backups}} now include attributes with name of database and date of backup.<br />
* Updated: {{Function|Database|Value Indexes}}: raise error if no index exists.<br />
<br />
;Version 8.2<br />
* Added: {{Function||db:output-cache}}<br />
* Removed: db:event<br />
<br />
;Version 7.9<br />
* Updated: parsing options added to {{Function||db:create}}, {{Function||db:add}} and {{Code|db:replace}}.<br />
* Updated: allow {{Option|UPDINDEX}} if {{Code|$all}} is {{Code|true}}.<br />
<br />
;Version 7.8.2<br />
* Added: {{Function||db:alter}}, {{Function||db:copy}}, {{Function||db:create-backup}}, {{Function||db:drop-backup}}, {{Function||db:restore}}<br />
<br />
;Version 7.8<br />
* Removed: db:fulltext (use {{Function|Full-Text|ft:search}} instead)<br />
<br />
;Version 7.7<br />
* Added: {{Function||db:export}}, {{Function||db:name}}, {{Function||db:path}}<br />
* Updated: {{Code|$options}} argument added to {{Function||db:create}} and {{Function||db:optimize}}.<br />
* Updated: the functions no longer accept [[#Database Nodes|database nodes]] as reference. Instead, the name of a database must now be specified.<br />
<br />
;Version 7.6<br />
* Updated: {{Function||db:create}}: allow more than one input and path.<br />
<br />
;Version 7.5<br />
* Updated: {{Function||db:add}}: input nodes will be automatically converted to document nodes<br />
* Added: {{Function||db:backups}}<br />
* Added: {{Function||db:create}}<br />
* Added: {{Function||db:drop}}<br />
<br />
;Version 7.3<br />
* Added: {{Function||db:flush}}<br />
<br />
;Version 7.2.1<br />
* Added: {{Function||db:text-range}}, {{Function||db:attribute-range}}, {{Function||db:output}}<br />
<br />
;Version 7.1<br />
* Added: {{Function||db:list-details}}, {{Function||db:content-type}}<br />
* Updated: {{Function||db:info}}, {{Function||db:system}}, {{Code|db:retrieve}}<br />
<br />
;Version 7.0<br />
* Added: {{Function||db:exists}}, {{Code|db:retrieve}}, {{Code|db:store}}, {{Code|db:is-raw}}, {{Code|db:is-xml}}<br />
* Updated: {{Function||db:list}}, {{Code|db:open}}, {{Function||db:add}}</div>CGhttps://docs.basex.org/index.php?title=File_Module&diff=16915File Module2023-12-01T21:06:53Z<p>CG: /* file:list */</p>
<hr />
<div>This [[Module Library|XQuery Module]] contains functions related to file system operations, such as listing, reading, or writing files.<br />
<br />
This module is based on the [http://expath.org/spec/file EXPath File Module]. The following enhancements have not been added to the specification yet:<br />
<br />
{| class="wikitable"<br />
|- valign="top"<br />
! Function<br />
! Description<br />
|- valign="top"<br />
| {{Function||file:descendants}}<br />
| new function<br />
|- valign="top"<br />
| {{Function||file:is-absolute}}<br />
| new function<br />
|- valign="top"<br />
| {{Function||file:read-text}}, {{Function||file:read-text-lines}}<br />
| <code>$fallback</code> argument added<br />
|- valign="top"<br />
| {{Function||file:read-text-lines}}<br />
| <code>$offset</code> and <code>$length</code> arguments added<br />
|- valign="top"<br />
| {{Function||file:resolve-path}}<br />
| <code>$base</code> argument added<br />
|}<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module are assigned to the <code><nowiki>http://expath.org/ns/file</nowiki></code> namespace, which is statically bound to the {{Code|file}} prefix.<br/><br />
<br />
For serialization parameters, the <code><nowiki>http://www.w3.org/2010/xslt-xquery-serialization</nowiki></code> namespace is used, which is statically bound to the {{Code|output}} prefix.<br/><br />
<br />
The error <code>[[#Errors|invalid-path]]</code> is raised if a path is invalid.<br />
<br />
==File Paths==<br />
<br />
* All file paths are resolved against the ''current working directory'' (the directory from which BaseX or, more generally, the Java Virtual Machine, was started). This directory can be retrieved via {{Function||file:base-dir}}.<br />
<br />
* A path can be specified as local filesystem path or as file URI.<br />
<br />
* Returned strings that refer to existing directories are suffixed with a directory separator.<br />
<br />
=Read Operations=<br />
<br />
==file:list==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:list(<br />
$dir as xs:string,<br />
$recursive as xs:boolean? := false(),<br />
$pattern as xs:string := ()<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Lists all files and directories found in the specified {{Code|$dir}}. The returned paths are relative to the provided path.<br/>If {{Code|$recursive}} is set to true, subdirectories will be traversed.<br/>The optional parameter {{Code|$pattern}} defines a file name pattern in the [[Commands#Glob_Syntax|Glob Syntax]]. If present, only those files and directories are returned that correspond to the pattern. Several patterns can be separated with a comma ({{Code|,}}).<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified file does not exist.<br/>{{Error|no-dir|#Errors}} the specified path does not point to a directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
| Return the contents of large {{Code|.txt}} files located in a specific directory and its subdirectories:<br />
<pre lang='xquery'><br />
let $root := 'path/to/files/'<br />
for $file in file:list($root, true(), '*.txt')<br />
let $path := $root || $file<br />
where file:size($path) > 1000000<br />
return file:read-text($path)<br />
</pre><br />
|}<br />
<br />
==file:children==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:children(<br />
$dir as xs:string<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the full paths to all files and directories found in the specified {{Code|$dir}}.<br/>The inverse function is {{Function||file:parent}}. The returned paths start with the specified directory. The related function {{Function||file:list}} returns relative file paths.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified file does not exist.<br/>{{Error|no-dir|#Errors}} the specified path does not point to a directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
| Return the contents of large {{Code|.txt}} files located in a specific directory:<br />
<pre lang='xquery'><br />
for $file in file:children('path/to/files/')<br />
where matches($file, '.txt', 'i') and file:size($file) > 1000000<br />
return file:read-text($$file)<br />
</pre><br />
|}<br />
<br />
==file:descendants==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:descendants(<br />
$dir as xs:string<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the full paths to all files and directories found in the specified {{Code|$dir}} and its subdirectories.<br/>. The returned paths start with the specified directory. The related function {{Function||file:list}} creates relative file paths.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified file does not exist.<br/>{{Error|no-dir|#Errors}} the specified path does not point to a directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
| Return the contents of all {{Code|.txt}} files located in a specific directory and its subdirectories:<br />
<pre lang='xquery'><br />
for $file in file:descendants('path/to/files/')<br />
where matches($file, '.txt', 'i') and file:size($file) > 1000000<br />
return file:read-text($$file)<br />
</pre><br />
|}<br />
<br />
==file:read-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:read-binary(<br />
$path as xs:string,<br />
$offset as xs:integer := (),<br />
$length as xs:integer := ()<br />
) as xs:base64Binary</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Reads the binary content of the file specified by {{Code|$path}} and returns it as [[Lazy Module|lazy]] {{Code|xs:base64Binary}} item.<br/>The optional parameters {{Code|$offset}} and {{Code|$length}} can be used to read chunks of a file.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified file does not exist.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|out-of-range|#Errors}} the offset or length is negative, or the chosen values would exceed the file bounds.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code><nowiki>lazy:cache(file:read-binary("config.data"))</nowiki></code> enforces the file access (otherwise, it will be delayed until requested first).<br />
|}<br />
<br />
==file:read-text==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:read-text(<br />
$path as xs:string,<br />
$encoding as xs:string := (),<br />
$fallback as xs:boolean? := false()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Reads the textual contents of the file specified by {{Code|$path}} and returns it as [[Lazy Module|lazy]] {{Code|xs:string}} item:<br />
* The UTF-8 default encoding can be overwritten with the optional {{Code|$encoding}} argument.<br />
* By default, invalid XML characters will be rejected. If {{Code|$fallback}} is enabled, the characters will be replaced with the Unicode replacement character <code>FFFD</code> (&#xFFFD;).<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified file does not exist.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|unknown-encoding|#Errors}} the specified encoding is not supported, or unknown.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code><nowiki>lazy:cache(file:read-text("ids.txt"))</nowiki></code> enforces the file access (otherwise, it will be delayed until requested first).<br />
|}<br />
<br />
==file:read-text-lines==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:read-text-lines(<br />
$path as xs:string,<br />
$encoding as xs:string := (),<br />
$fallback as xs:boolean? := false(),<br />
$offset as xs:integer? := (),<br />
$length as xs:integer? := ()<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Reads the textual contents of the file specified by {{Code|$path}} and returns it as a sequence of {{Code|xs:string}} items:<br />
* The UTF-8 default encoding can be overwritten with the optional {{Code|$encoding}} argument.<br />
* By default, invalid characters will be rejected. If {{Code|$fallback}} is set to true, these characters will be replaced with the Unicode replacement character <code>FFFD</code> (&#xFFFD;).<br />
The lines to be read can be restricted with the optional parameters {{Code|$offset}} and {{Code|$length}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified file does not exist.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|unknown-encoding|#Errors}} the specified encoding is not supported, or unknown.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
=Write Operations=<br />
<br />
==file:create-dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:create-dir(<br />
$dir as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates the directory specified by {{Code|$dir}} if it does not already exist. Non-existing parent directories will be created as well.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|exists|#Errors}} the specified target exists, but is no directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:create-temp-dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:create-temp-dir(<br />
$prefix as xs:string,<br />
$suffix as xs:string,<br />
$dir as xs:string := ()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a new temporary directory that did not exist before this function was called, and returns its full file path. The directory name begins and ends with the specified {{Code|$prefix}} and {{Code|$suffix}}. If no directory is specified via {{Code|$dir}}, the directory will be placed in the system’s default temporary directory. The operation will create all non-existing parent directories.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the specified directory points to a file.<br/>{{Error|io-error|#Errors}} the directory could not be created.<br />
|}<br />
<br />
==file:create-temp-file==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:create-temp-file(<br />
$prefix as xs:string,<br />
$suffix as xs:string,<br />
$dir as xs:string := ()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a new temporary file that did not exist before this function was called, and returns its full file path. The file name begins and ends with the specified {{Code|$prefix}} and {{Code|$suffix}}. If no directory is specified via {{Code|$dir}}, the file will be placed in the system’s default temporary directory. The operation will create all non-existing parent directories.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the specified directory points to a file.<br/>{{Error|io-error|#Errors}} the directory could not be created.<br />
|}<br />
<br />
==file:delete==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:delete(<br />
$path as xs:string,<br />
$recursive as xs:boolean? := false()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Recursively deletes a file or directory specified by {{Code|$path}}.<br/>The optional parameter {{Code|$recursive}} specifies whether subdirectories will be deleted, too.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified path does not exist.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:write==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:write(<br />
$path as xs:string,<br />
$input as item()*,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Writes serialized {{$input}} to the specified file. If the file already exists, it will be overwritten.<br/>The {{Code|$options}} argument contains [[Serialization|serialization parameters]]. As with [https://www.w3.org/TR/xpath-functions-31/#func-serialize fn:serialize()], the parameters can be specified<br/><br />
* either as children of an {{Code|&lt;output:serialization-parameters/&gt;}} element:<br />
<pre lang="xml"><br />
<output:serialization-parameters><br />
<output:method value='xml'/><br />
<output:cdata-section-elements value="div"/><br />
...<br />
</output:serialization-parameters><br />
</pre><br />
* or as map, which contains all key/value pairs:<br />
<pre lang='xquery'><br />
map { "method": "xml", "cdata-section-elements": "div", ... }<br />
</pre><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the parent of specified path is no directory.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code><nowiki>file:write('data.bin', xs:hexBinary('414243'))</nowiki></code> writes a hex representation to the specified file.<br />
* <code><nowiki>file:write('data.bin', xs:hexBinary('414243'), map { 'method': 'basex')</nowiki></code> writes binary data to the specified file (see [[XQuery_Extensions#Serialization|Serialization]] for more details).<br />
|}<br />
<br />
==file:write-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:write-binary(<br />
$path as xs:string,<br />
$value as xs:anyAtomicType,<br />
$offset as xs:integer := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Writes a binary item (xs:base64Binary, xs:hexBinary) to the specified file. If the file already exists, it will be overwritten.<br/>If {{Code|$offset}} is specified, data will be written at this file position. An existing file may be resized by that operation.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the parent of specified path is no directory.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|out-of-range|#Errors}} the offset is negative, or it exceeds the current file size.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:write-text==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:write-text(<br />
$path as xs:string,<br />
$value as xs:string,<br />
$encoding as xs:string := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Writes a string to the specified file. If the file already exists, it will be overwritten.<br/>The optional parameter {{Code|$encoding}} defines the output encoding (default: UTF-8).<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the parent of specified path is no directory.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|unknown-encoding|#Errors}} the specified encoding is not supported, or unknown.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:write-text-lines==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:write-text-lines(<br />
$path as xs:string,<br />
$values as xs:string*,<br />
$encoding as xs:string := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Writes a sequence of strings to the specified file, each followed by the system specific newline character. If the file already exists, it will be overwritten.<br/>The optional parameter {{Code|$encoding}} defines the output encoding (default: UTF-8).<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the parent of specified path is no directory.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|unknown-encoding|#Errors}} the specified encoding is not supported, or unknown.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:append==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:append(<br />
$path as xs:string,<br />
$input as item()*,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Appends a serialized sequence of {{Code|$input}} to the specified file, with the supplied {{Code|$options}} as serialization parameters. If the file does not exist, a new file is created.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the parent of specified path is no directory.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:append-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:append-binary(<br />
$path as xs:string,<br />
$value as xs:anyAtomicType<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Appends a binary item (xs:base64Binary, xs:hexBinary) to the specified file. If the file does not exists, a new one is created.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the parent of specified path is no directory.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:append-text==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:append-text(<br />
$path as xs:string,<br />
$value as xs:string,<br />
$encoding as xs:string := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Appends a string to a file specified by {{Code|$path}}. If the specified file does not exists, a new file is created.<br/>The optional parameter {{Code|$encoding}} defines the output encoding (default: UTF-8).<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the parent of specified path is no directory.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|unknown-encoding|#Errors}} the specified encoding is not supported, or unknown.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:append-text-lines==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:append-text-lines(<br />
$path as xs:string,<br />
$values as xs:string*,<br />
$encoding as xs:string := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Appends a sequence of strings to the specified file, each followed by the system specific newline character. If the specified file does not exists, a new file is created.<br/>The optional parameter {{Code|$encoding}} defines the output encoding (default: UTF-8).<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the parent of specified path is no directory.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|unknown-encoding|#Errors}} the specified encoding is not supported, or unknown.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:copy==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:copy(<br />
$source as xs:string,<br />
$target as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Copies a file or directory specified by {{Code|$source}} to the file or directory specified by {{Code|$target}}. If the target file already exists, it will be overwritten. No operation will be performed if the source and target path are equal.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified source does not exist.<br/>{{Error|exists|#Errors}} the specified source is a directory and the target is a file.<br/>{{Error|no-dir|#Errors}} the parent of the specified target is no directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:move==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:move(<br />
$source as xs:string,<br />
$target as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Moves or renames the file or directory specified by {{Code|$source}} to the path specified by {{Code|$target}}. If the target file already exists, it will be overwritten. No operation will be performed if the source and target path are equal.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified source does not exist.<br/>{{Error|exists|#Errors}} the specified source is a directory and the target is a file.<br/>{{Error|no-dir|#Errors}} the parent of the specified target is no directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|When renaming a file, remember that relative file paths are resolved against the current working directory. Some ways to rename:<br />
<pre lang='xquery'><br />
file:move('/projects/new/octopus', '/projects/new/septimus')<br />
</pre><br />
<pre lang='xquery'><br />
$path ! file:move(., file:parent(.)||$newName)<br />
</pre><br />
<br />
|}<br />
<br />
=File Properties=<br />
<br />
==file:exists==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:exists(<br />
$path as xs:string<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns an {{Code|xs:boolean}} indicating whether a file or directory specified by {{Code|$path}} exists in the file system.<br/><br />
|}<br />
<br />
==file:is-dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:is-dir(<br />
$path as xs:string<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns an {{Code|xs:boolean}} indicating whether the argument {{Code|$path}} points to an existing directory.<br/><br />
|}<br />
<br />
==file:is-absolute==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:is-absolute(<br />
$path as xs:string<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns an {{Code|xs:boolean}} indicating whether the argument {{Code|$path}} is absolute.<br/>The behavior of this function depends on the operating system: On Windows, an absolute path starts with the drive letter and a colon, whereas on Linux it starts with a slash.<br />
|}<br />
<br />
==file:is-file==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:is-file(<br />
$path as xs:string<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns an {{Code|xs:boolean}} indicating whether the argument {{Code|$path}} points to an existing file.<br/><br />
|}<br />
<br />
==file:last-modified==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:last-modified(<br />
$path as xs:string<br />
) as xs:dateTime</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Retrieves the timestamp of the last modification of the file or directory specified by {{Code|$path}}.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified path does not exist.<br/><br />
|}<br />
<br />
==file:size==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:size(<br />
$path as xs:string<br />
) as xs:integer</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the size, in bytes, of the file specified by {{Code|$path}}, or {{Code|0}} for directories.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified file does not exist.<br/><br />
|}<br />
<br />
=Path Functions=<br />
<br />
==file:name==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:name(<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the name of a file or directory specified by {{Code|$path}}. An empty string is returned if the path points to the root directory.<br />
|}<br />
<br />
==file:parent==<br />
<br />
{| width='100%'<br />
<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:parent(<br />
$path as xs:string<br />
) as xs:string?</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the absolute path to the parent directory of a file or directory specified by {{Code|$path}}. An empty sequence is returned if the path points to a root directory.<br/>The inverse function is {{Function||file:children}}.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code><nowiki>file:parent(static-base-uri())</nowiki></code> returns the directory of the current XQuery module.<br />
|}<br />
<br />
==file:path-to-native==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:path-to-native(<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Transforms the {{Code|$path}} argument to its native representation on the operating system.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified file does not exist.<br/>{{Error|io-error|#Errors}} the specified path cannot be transformed to its native representation.<br/><br />
|}<br />
<br />
==file:resolve-path==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:resolve-path(<br />
$path as xs:string,<br />
$base as xs:string := ()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Transforms the {{Code|$path}} argument to an absolute operating system path.<br/>If the path is relative, and if an absolute {{Code|$base}} path is specified, it will be resolved against this path.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|is-relative|#Errors}} the specified base path is relative.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|The following examples apply to Windows:<br />
* {{Code|file:resolve-path('file.txt', 'C:/Temp/')}} returns {{Code|C:/Temp/file.txt}}.<br />
* {{Code|file:resolve-path('file.txt', 'C:/Temp')}} returns {{Code|C:/file.txt}}.<br />
* {{Code|file:resolve-path('file.txt', 'Temp')}} raises an error.<br />
|}<br />
<br />
==file:path-to-uri==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:path-to-uri(<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Transforms the path specified by {{Code|$path}} into a URI with the {{Code|file://}} scheme.<br/><br />
|}<br />
<br />
=System Properties=<br />
<br />
==file:dir-separator==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|'''file:dir-separator'''() as xs:string}}<br/><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the directory separator used by the operating system, such as {{Code|/}} or {{Code|\}}.<br/><br />
|}<br />
<br />
==file:path-separator==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|'''file:path-separator'''() as xs:string}}<br/><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the path separator used by the operating system, such as {{Code|;}} or {{Code|:}}.<br/><br />
|}<br />
<br />
==file:line-separator==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:line-separator() as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the line separator used by the operating system, such as {{Code|&amp;#10;}}, {{Code|&amp;#13;&amp;#10;}} or {{Code|&amp;#13;}}.<br/><br />
|}<br />
<br />
==file:temp-dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:temp-dir() as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the system’s default temporary-file directory.<br/><br />
|}<br />
<br />
==file:current-dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:current-dir() as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the ''current working directory'', i.e., the directory from which the query processor was started. This function returns the same result as the function call <code>file:resolve-path("")</code>.<br />
|}<br />
<br />
==file:base-dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:base-dir() as xs:string?</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the parent directory of the static base URI.<br/>If the static base URI is undefined or does not point to a local resource, it returns the empty sequence. Otherwise, it returns the same result as {{Code|file:parent(static-base-uri())}}.<br />
|}<br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="160"|Code<br />
|Description<br />
|- valign="top"<br />
|{{Code|exists}}<br />
|A file with the same path already exists.<br />
|- valign="top"<br />
|{{Code|invalid-path}}<br />
|A specified path is invalid.<br />
|- valign="top"<br />
|{{Code|io-error}}<br />
|The operation fails for some other reason specific to the operating system.<br />
|- valign="top"<br />
|{{Code|is-dir}}<br />
|The specified path is a directory.<br />
|- valign="top"<br />
|{{Code|is-relative}}<br />
|The specified path is relative (and must be absolute).<br />
|- valign="top"<br />
|{{Code|no-dir}}<br />
|The specified path does not point to a directory.<br />
|- valign="top"<br />
|{{Code|not-found}}<br />
|A specified path does not exist.<br />
|- valign="top"<br />
|{{Code|out-of-range}}<br />
|The specified offset or length is negative, or the chosen values would exceed the file bounds.<br />
|- valign="top"<br />
|{{Code|unknown-encoding}}<br />
|The specified encoding is not supported, or unknown.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 9.3<br />
* Added: {{Function||file:descendants}}<br />
<br />
;Version 9.0<br />
* Updated: {{Function||file:read-text-lines}}: <code>$offset</code> and <code>$length</code> arguments added.<br />
<br />
;Version 8.5<br />
* Updated: {{Function||file:read-text}}, {{Function||file:read-text-lines}}: <code>$fallback</code> argument added.<br />
<br />
;Version 8.2<br />
* Added: {{Function||file:is-absolute}}<br />
* Updated: {{Function||file:resolve-path}}: base argument added<br />
<br />
;Version 8.0<br />
* Added: {{Function||file:current-dir}}, {{Function||file:base-dir}}, {{Function||file:children}}<br />
<br />
;Version 7.8<br />
* Added: {{Function||file:parent}}, {{Function||file:name}}<br />
* Updated: error codes; {{Function||file:read-binary}}, {{Function||file:write-binary}}: {{Code|$offset}} and {{Code|$length}} arguments added.<br />
* Deleted: file:base-name, file:dir-name<br />
<br />
;Version 7.7<br />
* Added: {{Function||file:create-temp-dir}}, {{Function||file:create-temp-file}}, {{Function||file:temp-dir}}<br />
* Updated: all returned strings that refer to existing directories will be suffixed with a directory separator.<br />
<br />
;Version 7.3<br />
* Added: {{Function||file:append-text}}, {{Function||file:write-text}}, {{Function||file:append-text-lines}}, {{Function||file:write-text-lines}}, {{Function||file:line-separator}}<br />
* Aligned with latest specification: $file:directory-separator → {{Function||file:dir-separator}}, $file:path-separator → {{Function||file:path-separator}}, file:is-directory → {{Function||file:is-dir}}, file:create-directory → {{Function||file:create-dir}}<br />
* Updated: {{Function||file:write-binary}}, {{Function||file:append-binary}}: output limited to a single value<br />
<br />
;Version 7.2.1<br />
* Updated: {{Function||file:delete}}: {{Code|$recursive}} parameter added to prevent subdirectories from being accidentally deleted.<br />
* Fixed: {{Function||file:list}} now returns relative instead of absolute paths.</div>CGhttps://docs.basex.org/index.php?title=Database_Module&diff=16914Database Module2023-12-01T16:58:48Z<p>CG: /* Database Nodes */</p>
<hr />
<div>This [[Module Library|XQuery Module]] contains functions for processing databases from within XQuery. Existing databases can be opened and listed, its contents can be directly accessed, documents can be added to and removed, etc.<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/db</nowiki></code> namespace, which is statically bound to the {{Code|db}} prefix.<br/><br />
<br />
==Database Nodes==<br />
<br />
In BaseX, two internal representations exist for XML nodes:<br />
<br />
1. Database nodes are:<br />
* stored in a persistent database on disk;<br />
* nodes of a document that has been generated temporarily with {{Code|fn:doc}}, {{Code|fn:parse-xml}} and other functions; or<br />
* the result of a [[XQuery Update#Main-Memory Updates|main-memory update]].<br />
2. Fragments are similar to DOM structures. They are created when XQuery node constructors are used (<code><a/></code>, <code>element a { }</code>, etc.).<br />
<br />
Fragments require less memory for small XML structures, but database nodes are more efficient for larger amounts of data. Fragments nodes can be converted to database nodes by applying a [[XQuery_Update#Main-Memory Updates|main-memory update]] with an empty body to a node:<br />
<br />
<pre lang='xquery'><br />
<xml>hello world</xml> update { }<br />
</pre><br />
<br />
==Updating Functions==<br />
<br />
Various functions in this module are ''updating''. Updating functions will not be immediately executed, but queued on the [[XQuery Update#Pending Update List|Pending Update List]], and processed after the remaining query has been evaluated. This means that the order in which the functions are specified in the query often does not reflect the order in which they will eventually be executed.<br />
<br />
=General Functions=<br />
<br />
==db:system==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:system() as element(system)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns general information on the database system the current values of all global and local [[Options]]. The {{Command|INFO}} command returns similar output.<br />
|}<br />
<br />
==db:option==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:option(<br />
$name as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the current value (string, integer, boolean, map) of a global or local [[Options|Option]] with the specified {{Code|$name}}. The {{Command|SHOW OPTIONS}} command returns similar output.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|option|#Errors}} the specified option is unknown.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>db:option('dbpath')</code> returns the database path string.<br />
* <code>db:option('serializer')</code> returns a map with the current serialization parameters.<br />
* <code>declare option db:stripws 'true'; db:option('stripws')</code> returns the locally assigned value.<br />
|}<br />
<br />
==db:info==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:info(<br />
$database as xs:string<br />
) as element(database)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns meta information on the specified {{Code|$database}}. The output is similar to the {{Command|INFO DB}} command.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|}<br />
<br />
==db:property==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:property(<br />
$database as xs:string,<br />
$name as xs:string<br />
) as xs:anyAtomicType</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the value (string, boolean, integer) of a property with the specified {{Code|$name}} in the specified {{Code|$database}}. The available properties are the ones returned by {{Function||db:info}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|property|#Errors}} the specified property is unknown.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>db:property('db', 'size')</code> returns the number of bytes occupied by the database <code>db</code>.<br />
* <code>db:property('xmark', 'textindex')</code> indicates if the <code>xmark</code> database has a text index.<br />
* <code>db:property('discogs', 'uptodate')</code> indicates if the database statistics and index structures of the <code>discogs</code> database are up-to-date.<br />
|}<br />
<br />
==db:list==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:list(<br />
$database as xs:string := (),<br />
$path as xs:string := ()<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Without arguments, the names of all databases are returned that are accessible to the current user. If {{Code|$database}} is specified, paths to all resources of this database are returned. The results can be restricted to resources starting with the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:list("docs")}} returns the names of all documents of a database named {{Code|docs}}.<br />
|}<br />
<br />
==db:list-details==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:list-details(<br />
$database as xs:string := (),<br />
$path as xs:string := ()<br />
) as element()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Without arguments, an element is returned for each database that is accessible to the current user:<br />
* An element has a value, which is the name of the database, and several attributes, which contain the number of stored resources, the modification date, the database size on disk (measured in bytes), and a path to the original database input.<br />
If {{Code|$database}} is specified, an element for each resource in this database is returned:<br />
* An element has a value, which is the name of the resource, and several attributes, which contain the content type, the modification date, the raw flag (which indicates if the resource is binary or XML), and the size of a resource.<br />
* The value of the size attribute depends on the resource type: for documents, it represents the number of nodes; for binary data, it represents the file size (measured in bytes).<br />
* The results can be restricted to resources starting with the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:list-details("shop")}} returns the names plus additional info on all resources of a database named {{Code|shop}}.<br />
|}<br />
<br />
==db:dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:dir(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as element()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns metadata on all directories and resources of a {{Code|$database}} in the specified {{Code|$path}}. Two types of elements are returned:<br />
* {{Code|resource}} represents a resource. The element value is the directory path; content type, modification date, raw flag (which indicates if the resource is binary or XML), and size of the resource are returned as attributes.<br />
* {{Code|dir}} represents a directory. The element value is the directory path; the modification date is returned in an attribute.<br />
The directories are not stored in the internal database layout. Instead, they result implicitly from the paths of stored resources.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|path|#Errors}} the specified path is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:dir('shop', 'books')}} returns all entries of the {{Code|books}} directory of a {{Code|shop}} database.<br />
|}<br />
<br />
=Read Operations=<br />
<br />
==db:get==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get(<br />
$database as xs:string,<br />
$path as xs:string := ()<br />
) as document-node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all documents from the specified {{Code|$database}}, or only documents matching the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get('docs')}} returns all documents from the database named {{Code|docs}}.<br />
* {{Code|db:get('db', 'one')}} returns all documents from the database named {{Code|db}} located in the path {{Code|one}}.<br />
* <code>for $i in 1 to 3 return db:get('db' || $i)//item</code> returns all item elements from the databases {{Code|db1}}, {{Code|db2}} and {{Code|db3}}.<br />
|}<br />
<br />
==db:get-pre==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-pre(<br />
$database as xs:string,<br />
$values as xs:integer*<br />
) as node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all nodes from a {{Code|$database}} with the specified PRE {{Code|values}} in [[Utility Module#util:ddo|distinct document order]].<br/>The [[Node Storage#PRE Value|PRE value]] provides very fast access to an existing database node, but it will change whenever a node with a smaller ''pre'' values is added to or deleted from a database.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|range|#Errors}} the specified PRE value does not exist in the database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get-pre("docs", 0)}} returns the first database node from the database named {{Code|docs}}.<br />
|}<br />
<br />
==db:get-id==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-id(<br />
$database as xs:string,<br />
$values as xs:integer*<br />
) as node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all nodes from a {{Code|$database}} with the specified ID {{Code|$values}} in [[Utility Module#util:ddo|distinct document order]].<br/>Each database node has a ''persistent'' [[Node Storage#ID Value|ID value]]. Access to the node ID can be sped up by turning on the {{Option|UPDINDEX}} option.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|range|#Errors}} the specified ID value does not exist in the database.<br />
|}<br />
<br />
==db:get-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-binary(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as item()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns a map with the paths and binary items of all resources in the specified {{Code|$database}}. A single {{Code|xs:base64Binary}} item is returned if a {{Code|$path}} is specified. All items are [[Lazy Module|lazy]], i.e., the actual data will only be retrieved if it is processed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get-binary('DB', 'music/01.mp3')}} returns the specified audio file as raw data.<br />
* <code><nowiki>stream:materialize(db:get-binary('DB', 'music/01.mp3'))</nowiki></code> materializes the streamable result in main-memory before returning it.<br />
* <code><nowiki>convert:binary-to-string(db:get-binary('DB', 'info.txt'), 'UTF-8')</nowiki></code> converts a binary database resource as UTF-8 text and returns a string.<br />
|}<br />
<br />
==db:get-value==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-value(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns a map with the paths and values of all resources in the specified {{Code|$database}}. A single value is returned if a {{Code|$path}} is specified.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get-value('DB', 'sequence')}} returns the specified sequence.<br />
|}<br />
<br />
==db:node-pre==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:node-pre(<br />
$nodes as node()*<br />
) as xs:integer*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the PRE values of the specified {{Code|$nodes}}, which must all be [[#Database Nodes|database nodes]].<br/>The [[Node Storage#PRE Value|PRE value]] provides very fast access to an existing database node, but it will change whenever a node with a smaller ''pre'' values is added to or deleted from a database.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:node-pre(doc("input"))}} returns {{Code|0}} if the database {{Code|input}} contains a single document.<br />
|}<br />
<br />
==db:node-id==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:node-id(<br />
$nodes as node()*<br />
) as xs:integer*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the ID values of the specified {{Code|$nodes}}, which must all be [[#Database Nodes|database nodes]].<br/>Each database node has a ''persistent'' [[Node Storage#ID Value|ID value]]. Access to the node id can be sped up by turning on the {{Option|UPDINDEX}} option.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|}<br />
<br />
==db:export==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:export(<br />
$database as xs:string,<br />
$path as xs:string,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Exports the specified {{Code|$database}} to the specified file {{Code|$path}}. Existing files will be overwritten.<br/>The {{Code|$options}} argument contains [[Serialization|serialization parameters]] (see [https://www.w3.org/TR/xpath-functions-31/#func-serialize fn:serialize()]).<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
| Export all files as text:<br/><br />
<pre lang='xquery'><br />
db:export("DB", "/home/john/xml/texts", map { 'method': 'text' })<br />
</pre><br />
The following code can be used to export parts of the database:<br />
<pre lang='xquery'><br />
let $target := '/home/john/xml/target'<br />
for $doc in db:get('DB', 'collection')<br />
let $path := $target || db:path($doc)<br />
return (<br />
file:create-dir(file:parent($path)),<br />
file:write($path, $doc)<br />
)<br />
</pre><br />
|}<br />
<br />
=Value Indexes=<br />
<br />
==db:text==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:text(<br />
$database as xs:string,<br />
$values as xs:string*<br />
) as text()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all text nodes of a {{Code|$database}} that match one of the specified {{Code|$values}} and that are stored in the text index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:text("DB", "QUERY")/..}} returns the parents of all text nodes of the database {{Code|DB}} that match the string {{Code|QUERY}}.<br />
|}<br />
<br />
==db:text-range==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:text-range(<br />
$database as xs:string,<br />
$min as xs:string,<br />
$max as xs:string<br />
) as text()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all text nodes of a {{Code|$database}} whose values are larger than or equal to {{Code|$min}} and smaller than or equal to {{Code|$max}} and that are stored in the text index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:text-range("DB", "2000", "2001")}} returns all text nodes of the database {{Code|DB}} that are found in between {{Code|2000}} and {{Code|2001}}.<br />
|}<br />
<br />
==db:attribute==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:attribute(<br />
$database as xs:string,<br />
$values as xs:string*,<br />
$name as xs:string := ()<br />
) as attribute()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all attribute nodes of a {{Code|$database}} that match one of the specified {{Code|$values}} and that are stored in the attribute index.<br/>If {{Code|$name}} is specified, the resulting attribute nodes are filtered by their attribute name.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:attribute("DB", "QUERY", "id")/..}} returns the parents of all {{Code|id}} attribute nodes of the database {{Code|DB}} that have {{Code|QUERY}} as string value.<br />
|}<br />
<br />
==db:attribute-range==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:attribute-range(<br />
$database as xs:string,<br />
$min as xs:string,<br />
$max as xs:string,<br />
$name as xs:string := ()<br />
) as attribute()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all attributes of a {{Code|$database}} whose values are larger than or equal to {{Code|$min}} and smaller than or equal to {{Code|$max}} and that are stored in the attribute index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:attribute-range("DB", "id456", "id473", 'id')}} returns all {{Code|@id}} attributes of the database {{Code|DB}} that have a string value in between {{Code|id456}} and {{Code|id473}}.<br />
|}<br />
<br />
==db:token==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:token(<br />
$database as xs:string,<br />
$tokens as xs:string*,<br />
$name as xs:string := ()<br />
) as attribute()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all attribute nodes of a {{Code|$database}} the values of which contain one of the specified {{Code|$tokens}}.<br/>If {{Code|$name}} is specified, the resulting attribute nodes are filtered by their attribute name.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:token("DB", "row", "class")/parent::div}} returns all {{Code|div}} nodes of database {{Code|DB}} with a {{Code|class}} attribute that contains the token {{Code|row}}.<br />
|}<br />
<br />
=Updates=<br />
<br />
All functions in this section are [[#Updating Functions|Updating Functions]].<br />
<br />
==db:create==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:create(<br />
$database as xs:string,<br />
$inputs as item()* := (),<br />
$paths as xs:string* := (),<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a new {{Code|$database}} and adds the supplied {{Code|$inputs}} to the specified {{Code|$paths}}:<br />
* The inputs may be strings or nodes:<br />
** nodes may be of any type except for attributes<br />
** strings can be a URI pointing to a file/directory or an XML string (which is detected by the leading <code>&lt;</code> character)<br />
** a path must be specified if the input is not a file or directory reference<br />
* The parsing and indexing behavior can be controlled via {{Code|$options}}:<br />
** allowed options are {{Option|ADDCACHE}} and the [[Options#Indexing|indexing]], [[Options#Full-Text Indexing|full-text indexing]], [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options, all in lower case<br />
** parsing options will only impact string input (URIs, XML strings) because nodes have already been parsed.<br />
* An existing database will be overwritten.<br />
* Database creation takes place after most other update operations (see [[XQuery Update#Pending Update List|Pending Update List]]). As a consequence, a newly created database cannot be addressed in the same query.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} the specified name is not a [[Commands#Valid_Names|valid database name]].<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br/>{{Error|args|#Errors}} the number of specified inputs and paths differs.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:create("DB")}} creates the empty database {{Code|DB}}.<br />
* {{Code|db:create("DB", "/home/dir/doc.xml")}} creates the database {{Code|DB}} and adds the document {{Code|/home/dir/doc.xml}} as initial content.<br />
* {{Code|db:create("DB", <a/>, "doc.xml")}} creates the database {{Code|DB}} and adds the document with content {{Code|&lt;a/&gt;}} under the name {{Code|doc.xml}}.<br />
* {{Code|db:create("DB", "/home/dir/", "docs/dir")}} creates the database {{Code|DB}} and adds the documents in {{Code|/home/dir}} to the database under the path {{Code|docs/dir}}.<br />
* <code>db:create("DB", file:list('.'), (), map { 'ftindex': true() })</code> adds all files of the current working directory to a new database, preserving relative filesystem paths and creating a full-text index.<br />
|}<br />
<br />
==db:add==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:add(<br />
$database as xs:string,<br />
$input as item(),<br />
$path as xs:string? := (),<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Adds the specified {{Code|$input}} to a {{Code|$database}} with the specified {{Code|$path}}:<br />
* A document with the same path may occur more than once in a database. If you want to enforce single instances, use {{Function||db:put}} instead.<br />
* See {{Function||db:create}} for more details on the input and path arguments.<br />
* The parsing behavior can be controlled via {{Code|$options}}:<br />
** allowed options are {{Option|ADDCACHE}} and the [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options, all in lower case<br />
** parsing options will only impact string input (URIs, XML strings) because nodes have already been parsed<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>db:add("DB", "/home/dir/doc.xml")</code> adds the file {{Code|/home/dir/doc.xml}} to the database {{Code|DB}}.<br />
* <code>db:add("DB", <a/>, "doc.xml")</code> adds a document node to the database {{Code|DB}} under the name {{Code|doc.xml}}.<br />
* <code>db:add("DB", "/home/dir", "docs/dir", map { 'addcache': true() })</code> adds all documents in {{Code|/home/dir}} to the database {{Code|DB}} under the path {{Code|docs/dir}}. To reduce memory consumption, the files will be cached before being added to the database.<br />
|}<br />
<br />
==db:put==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:put(<br />
$database as xs:string,<br />
$input as item(),<br />
$path as xs:string,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Replaces a resource, specified by {{Code|$path}}, in a {{Code|$database}} with the contents of {{Code|$input}}, or adds it as a new resource:<br />
* The parsing behavior can be controlled via {{Code|$options}}:<br />
** Allowed options are {{Option|ADDCACHE}} and the [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options, all in lower case.<br />
** Parsing options will only impact string input (URIs, XML strings), because nodes have already been parsed.<br />
* See {{Function||db:create}} for more details on the input argument.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|target|#Errors}} the path points to a directory.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:put("DB", "/home/dir/doc.xml", "docs/dir/doc.xml")}} replaces the content of the document {{Code|docs/dir/doc.xml}} in the database {{Code|DB}} with the content of the file {{Code|/home/dir/doc.xml}}.<br />
* {{Code|db:put("DB", "<a/>", "docs/dir/doc.xml")}} replaces the content of the document {{Code|docs/dir/doc.xml}} in the database {{Code|DB}} with {{Code|&lt;a/&gt;}}.<br />
* {{Code|db:put("DB", document { <a/> }, "docs/dir/doc.xml")}} replaces the content of the document {{Code|docs/dir/doc.xml}} in the database {{Code|DB}} with the specified document node.<br />
The following query can be used to import files from a directory to a database:<br />
<pre lang='xquery'><br />
let $source := '/home/john/xml/source'<br />
for $file in file:list($source, true())<br />
let $path := $source || $file<br />
where not(file:is-dir($path))<br />
return db:put('db', doc($path), $file)<br />
</pre><br />
|}<br />
<br />
==db:put-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:put-binary(<br />
$database as xs:string,<br />
$input as item(),<br />
$path as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Stores a binary resource specified by {{Code|$input}} in a {{Code|$database}} at the specified {{Code|$path}}. Existing resources are overwritten.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:put-binary('DB', file:read-binary('video.mov'), 'video/sample.mov')}} stores the addressed video file at the specified location.<br />
* With the following query, you can copy the binary resources of one database into another:<br />
<pre lang='xquery'><br />
let $db := 'db'<br />
let $src-path := 'src/'<br />
let $trg-path := 'trg/'<br />
for $src in db:list($db, $src-path)<br />
where db:type($db, $src) = 'binary'<br />
let $trg := $trg-path || substring-after($src, $src-path)<br />
return db:put-binary($db, db:get-binary($db, $src), $trg)<br />
</pre><br />
|}<br />
<br />
==db:put-value==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:put-value(<br />
$database as xs:string,<br />
$input as item()*,<br />
$path as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Stores an {{Code|$input}} value in a {{Code|$database}} at the specified {{Code|$path}}. Existing resources are overwritten. The value can be an arbitrary sequence of atomic items, nodes, maps, and arrays.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:put-value('DB', 1 to 10000, 'sequence')}} stores a numeric range in the database.<br />
* With the following query, a map with countries and associated cities is stored in a database. The value resource can e.g. be used as index in future queries:<br />
<pre lang='xquery'><br />
db:put-value(<br />
'factbook',<br />
map:merge(<br />
for $country in db:get('factbook')//country<br />
return map:entry($country/@name, $country//city/name ! string())<br />
),<br />
'cities'<br />
)</pre><br />
|}<br />
<br />
==db:delete==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:delete(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Deletes resource(s), specified by {{Code|$path}}, from the specified {{Code|$database}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|path|#Errors}} the specified path is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:delete("DB", "docs/dir/doc.xml")}} deletes the resource {{Code|docs/dir/doc.xml}} from {{Code|DB}}.<br />
* {{Code|db:delete("DB", "docs/dir")}} deletes all resources from {{Code|DB}} in the specified path {{Code|docs/dir}}.<br />
|}<br />
<br />
==db:copy==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:copy(<br />
$database as xs:string,<br />
$new-name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a copy of {{Code|$database}}, which will be called {{Code|$new-name}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|}<br />
<br />
==db:alter==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:alter(<br />
$database as xs:string,<br />
$new-name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Renames a {{Code|$database}} to {{Code|$new-name}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|}<br />
<br />
==db:optimize==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:optimize(<br />
$database as xs:string,<br />
$all as xs:boolean? := false(),<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Optimizes the metadata and indexes of a {{Code|$database}}.<br/>If {{Code|$all}} is {{Code|true}}, the complete database will be rebuilt.<br/>The {{Code|$options}} argument can be used to control indexing. The syntax is identical to the {{Function||db:create}} function: Allowed options are all [[Options#Indexing|indexing]] and [[Options#Full-Text|full-text]] options. {{Option|UPDINDEX}} is only supported if {{Code|$all}} is {{Code|true}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:optimize("DB")}} optimizes the database structures of the database {{Code|DB}}.<br />
* <code>db:optimize("DB", true(), map { 'ftindex': true() })</code> optimizes all database structures of the database {{Code|DB}} and creates a full-text index.<br />
|}<br />
<br />
==db:rename==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:rename(<br />
$database as xs:string,<br />
$source as xs:string,<br />
$target as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Moves all resources(s) of a {{Code|$database}}, which are found in the supplied {{Code|$source}} path, to the supplied {{Code|$target}} path. The paths may point to single resources or directories. No updates will take place if a non-existing source path is supplied.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|path|#Errors}} the specified source or target path, or one of its descendants, is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:rename("DB", "docs/dir/doc.xml", "docs/dir/newdoc.xml")}} renames the resource {{Code|docs/dir/doc.xml}} to {{Code|docs/dir/newdoc.xml}} in the database {{Code|DB}}.<br />
* {{Code|db:rename("DB", "docs/dir", "docs/newdir")}} moves all resources in the database {{Code|DB}} from {{Code|docs/dir}} to {Code|docs/newdir}}.<br />
|}<br />
<br />
==db:flush==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:flush(<br />
$database as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Explicitly flushes the buffers of a {{Code|$database}}. This command is only useful if {{Option|AUTOFLUSH}} has been set to {{Code|false}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|}<br />
<br />
==db:drop==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:drop(<br />
$database as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Drops a {{Code|$database}} and all connected resources.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:drop("DB")}} drops the database {{Code|DB}}.<br />
|}<br />
<br />
=Backups=<br />
<br />
All functions in this section except for {{Function||db:backups}} are {{Function|Database|Updating Functions}}.<br />
<br />
==db:create-backup==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:create-backup(<br />
$database as xs:string,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a backup of a {{Code|$database}}. If no name is supplied, general data will be backed up. The following {{Code|$options}} are available:<br />
* With {{Code|comment}}, a comment string can be attached to the backup.<br />
* By setting {{Code|compress}} to false, the backup will be created faster, but it will take more space on disk.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:create-backup('DB', map { 'compress': false() })}} creates a backup of the database {{Code|DB}} without compressing its entries.<br />
|}<br />
<br />
==db:drop-backup==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:drop-backup(<br />
$name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Drops all backups of the database with the specified {{Code|$name}}. If the name ends with a timestamp, only the specified backup file will be deleted. If no name is supplied, backups with general data are addressed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|backup|#Errors}} No backup file found.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:drop-backup("DB")}} drops all backups of the database {{Code|DB}}.<br />
* {{Code|db:drop-backup("DB-2014-03-13-17-36-44")}} drops the specific backup file {{Code|DB-2014-03-13-17-36-44.zip}} of the database {{Code|DB}}.<br />
|}<br />
<br />
==db:alter-backup==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:alter-backup(<br />
$name as xs:string,<br />
$new-name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Renames all backups of the database with the specified {{Code|$name}} to {{Code|$new-name}}. If the name ends with a date, only the specified backup file will be renamed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|backup|#Errors}} No backup file found.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:alter-backup("DB", "DB2)}} renames all backups of the database {{Code|DB}} to {{Code|DB2}}.<br />
|}<br />
<br />
==db:restore==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:restore(<br />
$name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Restores the database with the specified {{Code|$name}}. The {{Code|$name}} may include the timestamp of the backup file. If no name is supplied, general data will be restored. If general data is restored, it will only be available after BaseX has been restarted.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|no-backup|#Errors}} No backup found.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:restore("DB")}} restores the database {{Code|DB}}.<br />
* {{Code|db:restore("DB-2014-03-13-18-05-45")}} restores the database {{Code|DB}} from the backup file with the given timestamp.<br />
|}<br />
<br />
==db:backups==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:backups(<br />
$database as xs:string := ()<br />
) as element(backup)*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns an element sequence containing all available database backups with timestamp, file size and comment.<br/>If a {{Code|$database}} is specified, the sequence will be restricted to the backups matching this database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:backups("factbook")}} returns all backups that have been made from the {{Code|factbook}} database.<br />
|}<br />
<br />
=Helper Functions=<br />
<br />
==db:name==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:name(<br />
$node as node()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the name of the database in which the specified [[#Database Nodes|database node]] {{Code|$node}} is stored.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|}<br />
<br />
==db:path==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:path(<br />
$node as node()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the path of the database document in which the specified [[#Database Nodes|database node]] {{Code|$node}} is stored.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|}<br />
<br />
==db:exists==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:exists(<br />
$database as xs:string,<br />
$path as xs:string := ()<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Checks if a {{Code|$database}} exists, or a resource located at {{Code|$path}} in this database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:exists("DB")}} returns {{Code|true}} if the database {{Code|DB}} exists.<br />
* {{Code|db:exists("DB", "resource")}} returns {{Code|true}} if {{Code|resource}} exists in this database.<br />
|}<br />
<br />
==db:type==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:type(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the type ({{Code|xml}}, {{Code|binary}}, {{Code|value}}) of a resource in a {{Code|$database}} at the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:type("DB", "factbook.xml")}} returns {{Code|true}} if the specified resource is an XML document.<br />
|}<br />
<br />
==db:content-type==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:content-type(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Retrieves the content-type of a resource in a {{Code|$database}} at the specified {{Code|$path}}.<br/>The file extension is used to recognize the content-type of a resource stored in the database. {{Code|application/xml}} will be returned for any XML document stored in the database, regardless of its file name extension.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:content-type("DB", "docs/doc01.pdf")}} returns {{Code|application/pdf}}.<br />
* {{Code|db:content-type("DB", "docs/doc01.xml")}} returns {{Code|application/xml}}.<br />
* {{Code|db:content-type("DB", "docs/doc01")}} returns {{Code|application/xml}}, if {{Code|db:is-xml("DB", "docs/doc01")}} returns {{Code|true}}.<br />
|}<br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
| Description<br />
|- valign="top"<br />
|{{Code|args}}<br />
|The number of specified inputs and paths differs.<br />
|- valign="top"<br />
|{{Code|conflict}}<br />
|Multiple update operations point to the same target.<br />
|- valign="top"<br />
|{{Code|lock}}<br />
|A database cannot be updated because it is opened by another process.<br />
|- valign="top"<br />
|{{Code|mainmem}}<br />
|The addressed database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
|{{Code|name}}<br />
|The name of the specified database is invalid.<br />
|- valign="top"<br />
|{{Code|no-backup}}<br />
|No backup exists for a database.<br />
|- valign="top"<br />
|{{Code|node}}<br />
|The referenced XML node is no [[#Database Nodes|database node]], i.e. it is neither stored in a database nor represented as database fragment.<br />
|- valign="top"<br />
|{{Code|no-index}}<br />
|The database lacks an index structure required by the called function.<br />
|- valign="top"<br />
|{{Code|open}}<br />
|The addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
|{{Code|option}}<br />
|The specified option is unknown.<br />
|- valign="top"<br />
|{{Code|path}}<br />
|The specified database path is invalid.<br />
|- valign="top"<br />
|{{Code|property}}<br />
|The specified database property is unknown.<br />
|- valign="top"<br />
|{{Code|range}}<br />
|The addressed database ID or PRE value is out of range.<br />
|- valign="top"<br />
|{{Code|target}}<br />
|Path points to an invalid target.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 10<br />
* Added: {{Function||db:get}}, {{Function||db:put}}, {{Function||db:type}}.<br />
* Added: [[#Backups|Backups]]: Support for general data ([[User Management|registered users]], [[Job Module#Services|scheduled services]] and [[Store Module|key-value stores]]).<br />
* Updated: {{Function||db:get}}, {{Function||db:get-id}}, {{Function||db:get-pre}} renamed (before: {{Code|db:open}}, {{Code|db:open-id}}, {{Code|db:open-pre}})<br />
* Updated: {{Function||db:put}} renamed (before: {{Code|db:replace}}); function signature aligned with {{Function||db:add}} (second and third argument swapped).<br />
* Updated: {{Function||db:put-binary}} renamed (before: {{Code|db:store}}); function signature aligned with {{Function||db:add}} (second and third argument swapped).<br />
* Updated: {{Function||db:get-binary}} renamed (before: {{Code|db:retrieve}}).<br />
* Updated: {{Function||db:backups}}, {{Function||db:create-backup}}: Options added.<br />
* Removed: {{Code|db:is-raw}}, {{Code|db:is-raw}} (new: {{Function||db:type}}).<br />
<br />
;Version 9.3<br />
* Added: {{Function||db:alter-backup}}<br />
* Updated: {{Code|db:open-id}}, {{Code|db:open-pre}}: support for multiple integers<br />
<br />
;Version 9.2<br />
* Added: {{Function||db:dir}}<br />
* Updated: {{Function||db:add}}: {{Code|$path}} allow empty path argument<br />
<br />
;Version 9.0<br />
* Added: {{Function||db:option}}<br />
* Updated: db:output renamed to {{Function|Update|update:output}}, db:output-cache renamed to {{Function|Update|update:cache}}<br />
* Updated: error codes updated; errors now use the module namespace<br />
<br />
;Version 8.6<br />
* Added: {{Function||db:property}}<br />
<br />
;Version 8.4<br />
* Updated: {{Function||db:create}}, {{Function||db:add}}, {{Code|db:replace}}: support for {{Code|ADDCACHE}} option.<br />
* Added: {{Function||db:token}}<br />
<br />
;Version 8.3<br />
* Updated: {{Function||db:list-details}}: attributes with name of database and date of backup added to results.<br />
* Updated: {{Function||db:backups}} now include attributes with name of database and date of backup.<br />
* Updated: {{Function|Database|Value Indexes}}: raise error if no index exists.<br />
<br />
;Version 8.2<br />
* Added: {{Function||db:output-cache}}<br />
* Removed: db:event<br />
<br />
;Version 7.9<br />
* Updated: parsing options added to {{Function||db:create}}, {{Function||db:add}} and {{Code|db:replace}}.<br />
* Updated: allow {{Option|UPDINDEX}} if {{Code|$all}} is {{Code|true}}.<br />
<br />
;Version 7.8.2<br />
* Added: {{Function||db:alter}}, {{Function||db:copy}}, {{Function||db:create-backup}}, {{Function||db:drop-backup}}, {{Function||db:restore}}<br />
<br />
;Version 7.8<br />
* Removed: db:fulltext (use {{Function|Full-Text|ft:search}} instead)<br />
<br />
;Version 7.7<br />
* Added: {{Function||db:export}}, {{Function||db:name}}, {{Function||db:path}}<br />
* Updated: {{Code|$options}} argument added to {{Function||db:create}} and {{Function||db:optimize}}.<br />
* Updated: the functions no longer accept [[#Database Nodes|database nodes]] as reference. Instead, the name of a database must now be specified.<br />
<br />
;Version 7.6<br />
* Updated: {{Function||db:create}}: allow more than one input and path.<br />
<br />
;Version 7.5<br />
* Updated: {{Function||db:add}}: input nodes will be automatically converted to document nodes<br />
* Added: {{Function||db:backups}}<br />
* Added: {{Function||db:create}}<br />
* Added: {{Function||db:drop}}<br />
<br />
;Version 7.3<br />
* Added: {{Function||db:flush}}<br />
<br />
;Version 7.2.1<br />
* Added: {{Function||db:text-range}}, {{Function||db:attribute-range}}, {{Function||db:output}}<br />
<br />
;Version 7.1<br />
* Added: {{Function||db:list-details}}, {{Function||db:content-type}}<br />
* Updated: {{Function||db:info}}, {{Function||db:system}}, {{Code|db:retrieve}}<br />
<br />
;Version 7.0<br />
* Added: {{Function||db:exists}}, {{Code|db:retrieve}}, {{Code|db:store}}, {{Code|db:is-raw}}, {{Code|db:is-xml}}<br />
* Updated: {{Function||db:list}}, {{Code|db:open}}, {{Function||db:add}}</div>CGhttps://docs.basex.org/index.php?title=Database_Module&diff=16913Database Module2023-12-01T16:41:16Z<p>CG: /* Errors */</p>
<hr />
<div>This [[Module Library|XQuery Module]] contains functions for processing databases from within XQuery. Existing databases can be opened and listed, its contents can be directly accessed, documents can be added to and removed, etc.<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/db</nowiki></code> namespace, which is statically bound to the {{Code|db}} prefix.<br/><br />
<br />
==Database Nodes==<br />
<br />
In BaseX, two internal representations exist for XML nodes:<br />
<br />
* Database nodes are:<br />
** stored in a persistent database on disk;<br />
** nodes of a document that has been generated temporarily with {{Code|fn:doc}}, {{Code|fn:parse-xml}} and other functions; or<br />
** the result of a [[XQuery Update#Main-Memory Updates|main-memory update]].<br />
* Fragments are similar to DOM structures. They are created when XQuery node constructors are used (<code><a/></code>, <code>element a { }</code>, etc.).<br />
<br />
Fragments require less memory for small XML structures, but database nodes are more efficient for larger amounts of data. Fragments nodes can be converted to database nodes by applying a [[XQuery_Update#Main-Memory Updates|main-memory update]] with an empty body to a node:<br />
<br />
<pre lang='xquery'><br />
<xml>hello world</xml> update { }<br />
</pre><br />
<br />
==Updating Functions==<br />
<br />
Various functions in this module are ''updating''. Updating functions will not be immediately executed, but queued on the [[XQuery Update#Pending Update List|Pending Update List]], and processed after the remaining query has been evaluated. This means that the order in which the functions are specified in the query often does not reflect the order in which they will eventually be executed.<br />
<br />
=General Functions=<br />
<br />
==db:system==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:system() as element(system)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns general information on the database system the current values of all global and local [[Options]]. The {{Command|INFO}} command returns similar output.<br />
|}<br />
<br />
==db:option==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:option(<br />
$name as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the current value (string, integer, boolean, map) of a global or local [[Options|Option]] with the specified {{Code|$name}}. The {{Command|SHOW OPTIONS}} command returns similar output.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|option|#Errors}} the specified option is unknown.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>db:option('dbpath')</code> returns the database path string.<br />
* <code>db:option('serializer')</code> returns a map with the current serialization parameters.<br />
* <code>declare option db:stripws 'true'; db:option('stripws')</code> returns the locally assigned value.<br />
|}<br />
<br />
==db:info==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:info(<br />
$database as xs:string<br />
) as element(database)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns meta information on the specified {{Code|$database}}. The output is similar to the {{Command|INFO DB}} command.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|}<br />
<br />
==db:property==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:property(<br />
$database as xs:string,<br />
$name as xs:string<br />
) as xs:anyAtomicType</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the value (string, boolean, integer) of a property with the specified {{Code|$name}} in the specified {{Code|$database}}. The available properties are the ones returned by {{Function||db:info}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|property|#Errors}} the specified property is unknown.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>db:property('db', 'size')</code> returns the number of bytes occupied by the database <code>db</code>.<br />
* <code>db:property('xmark', 'textindex')</code> indicates if the <code>xmark</code> database has a text index.<br />
* <code>db:property('discogs', 'uptodate')</code> indicates if the database statistics and index structures of the <code>discogs</code> database are up-to-date.<br />
|}<br />
<br />
==db:list==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:list(<br />
$database as xs:string := (),<br />
$path as xs:string := ()<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Without arguments, the names of all databases are returned that are accessible to the current user. If {{Code|$database}} is specified, paths to all resources of this database are returned. The results can be restricted to resources starting with the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:list("docs")}} returns the names of all documents of a database named {{Code|docs}}.<br />
|}<br />
<br />
==db:list-details==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:list-details(<br />
$database as xs:string := (),<br />
$path as xs:string := ()<br />
) as element()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Without arguments, an element is returned for each database that is accessible to the current user:<br />
* An element has a value, which is the name of the database, and several attributes, which contain the number of stored resources, the modification date, the database size on disk (measured in bytes), and a path to the original database input.<br />
If {{Code|$database}} is specified, an element for each resource in this database is returned:<br />
* An element has a value, which is the name of the resource, and several attributes, which contain the content type, the modification date, the raw flag (which indicates if the resource is binary or XML), and the size of a resource.<br />
* The value of the size attribute depends on the resource type: for documents, it represents the number of nodes; for binary data, it represents the file size (measured in bytes).<br />
* The results can be restricted to resources starting with the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:list-details("shop")}} returns the names plus additional info on all resources of a database named {{Code|shop}}.<br />
|}<br />
<br />
==db:dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:dir(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as element()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns metadata on all directories and resources of a {{Code|$database}} in the specified {{Code|$path}}. Two types of elements are returned:<br />
* {{Code|resource}} represents a resource. The element value is the directory path; content type, modification date, raw flag (which indicates if the resource is binary or XML), and size of the resource are returned as attributes.<br />
* {{Code|dir}} represents a directory. The element value is the directory path; the modification date is returned in an attribute.<br />
The directories are not stored in the internal database layout. Instead, they result implicitly from the paths of stored resources.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|path|#Errors}} the specified path is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:dir('shop', 'books')}} returns all entries of the {{Code|books}} directory of a {{Code|shop}} database.<br />
|}<br />
<br />
=Read Operations=<br />
<br />
==db:get==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get(<br />
$database as xs:string,<br />
$path as xs:string := ()<br />
) as document-node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all documents from the specified {{Code|$database}}, or only documents matching the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get('docs')}} returns all documents from the database named {{Code|docs}}.<br />
* {{Code|db:get('db', 'one')}} returns all documents from the database named {{Code|db}} located in the path {{Code|one}}.<br />
* <code>for $i in 1 to 3 return db:get('db' || $i)//item</code> returns all item elements from the databases {{Code|db1}}, {{Code|db2}} and {{Code|db3}}.<br />
|}<br />
<br />
==db:get-pre==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-pre(<br />
$database as xs:string,<br />
$values as xs:integer*<br />
) as node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all nodes from a {{Code|$database}} with the specified PRE {{Code|values}} in [[Utility Module#util:ddo|distinct document order]].<br/>The [[Node Storage#PRE Value|PRE value]] provides very fast access to an existing database node, but it will change whenever a node with a smaller ''pre'' values is added to or deleted from a database.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|range|#Errors}} the specified PRE value does not exist in the database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get-pre("docs", 0)}} returns the first database node from the database named {{Code|docs}}.<br />
|}<br />
<br />
==db:get-id==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-id(<br />
$database as xs:string,<br />
$values as xs:integer*<br />
) as node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all nodes from a {{Code|$database}} with the specified ID {{Code|$values}} in [[Utility Module#util:ddo|distinct document order]].<br/>Each database node has a ''persistent'' [[Node Storage#ID Value|ID value]]. Access to the node ID can be sped up by turning on the {{Option|UPDINDEX}} option.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|range|#Errors}} the specified ID value does not exist in the database.<br />
|}<br />
<br />
==db:get-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-binary(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as item()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns a map with the paths and binary items of all resources in the specified {{Code|$database}}. A single {{Code|xs:base64Binary}} item is returned if a {{Code|$path}} is specified. All items are [[Lazy Module|lazy]], i.e., the actual data will only be retrieved if it is processed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get-binary('DB', 'music/01.mp3')}} returns the specified audio file as raw data.<br />
* <code><nowiki>stream:materialize(db:get-binary('DB', 'music/01.mp3'))</nowiki></code> materializes the streamable result in main-memory before returning it.<br />
* <code><nowiki>convert:binary-to-string(db:get-binary('DB', 'info.txt'), 'UTF-8')</nowiki></code> converts a binary database resource as UTF-8 text and returns a string.<br />
|}<br />
<br />
==db:get-value==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-value(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns a map with the paths and values of all resources in the specified {{Code|$database}}. A single value is returned if a {{Code|$path}} is specified.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get-value('DB', 'sequence')}} returns the specified sequence.<br />
|}<br />
<br />
==db:node-pre==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:node-pre(<br />
$nodes as node()*<br />
) as xs:integer*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the PRE values of the specified {{Code|$nodes}}, which must all be [[#Database Nodes|database nodes]].<br/>The [[Node Storage#PRE Value|PRE value]] provides very fast access to an existing database node, but it will change whenever a node with a smaller ''pre'' values is added to or deleted from a database.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:node-pre(doc("input"))}} returns {{Code|0}} if the database {{Code|input}} contains a single document.<br />
|}<br />
<br />
==db:node-id==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:node-id(<br />
$nodes as node()*<br />
) as xs:integer*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the ID values of the specified {{Code|$nodes}}, which must all be [[#Database Nodes|database nodes]].<br/>Each database node has a ''persistent'' [[Node Storage#ID Value|ID value]]. Access to the node id can be sped up by turning on the {{Option|UPDINDEX}} option.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|}<br />
<br />
==db:export==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:export(<br />
$database as xs:string,<br />
$path as xs:string,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Exports the specified {{Code|$database}} to the specified file {{Code|$path}}. Existing files will be overwritten.<br/>The {{Code|$options}} argument contains [[Serialization|serialization parameters]] (see [https://www.w3.org/TR/xpath-functions-31/#func-serialize fn:serialize()]).<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
| Export all files as text:<br/><br />
<pre lang='xquery'><br />
db:export("DB", "/home/john/xml/texts", map { 'method': 'text' })<br />
</pre><br />
The following code can be used to export parts of the database:<br />
<pre lang='xquery'><br />
let $target := '/home/john/xml/target'<br />
for $doc in db:get('DB', 'collection')<br />
let $path := $target || db:path($doc)<br />
return (<br />
file:create-dir(file:parent($path)),<br />
file:write($path, $doc)<br />
)<br />
</pre><br />
|}<br />
<br />
=Value Indexes=<br />
<br />
==db:text==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:text(<br />
$database as xs:string,<br />
$values as xs:string*<br />
) as text()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all text nodes of a {{Code|$database}} that match one of the specified {{Code|$values}} and that are stored in the text index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:text("DB", "QUERY")/..}} returns the parents of all text nodes of the database {{Code|DB}} that match the string {{Code|QUERY}}.<br />
|}<br />
<br />
==db:text-range==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:text-range(<br />
$database as xs:string,<br />
$min as xs:string,<br />
$max as xs:string<br />
) as text()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all text nodes of a {{Code|$database}} whose values are larger than or equal to {{Code|$min}} and smaller than or equal to {{Code|$max}} and that are stored in the text index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:text-range("DB", "2000", "2001")}} returns all text nodes of the database {{Code|DB}} that are found in between {{Code|2000}} and {{Code|2001}}.<br />
|}<br />
<br />
==db:attribute==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:attribute(<br />
$database as xs:string,<br />
$values as xs:string*,<br />
$name as xs:string := ()<br />
) as attribute()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all attribute nodes of a {{Code|$database}} that match one of the specified {{Code|$values}} and that are stored in the attribute index.<br/>If {{Code|$name}} is specified, the resulting attribute nodes are filtered by their attribute name.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:attribute("DB", "QUERY", "id")/..}} returns the parents of all {{Code|id}} attribute nodes of the database {{Code|DB}} that have {{Code|QUERY}} as string value.<br />
|}<br />
<br />
==db:attribute-range==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:attribute-range(<br />
$database as xs:string,<br />
$min as xs:string,<br />
$max as xs:string,<br />
$name as xs:string := ()<br />
) as attribute()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all attributes of a {{Code|$database}} whose values are larger than or equal to {{Code|$min}} and smaller than or equal to {{Code|$max}} and that are stored in the attribute index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:attribute-range("DB", "id456", "id473", 'id')}} returns all {{Code|@id}} attributes of the database {{Code|DB}} that have a string value in between {{Code|id456}} and {{Code|id473}}.<br />
|}<br />
<br />
==db:token==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:token(<br />
$database as xs:string,<br />
$tokens as xs:string*,<br />
$name as xs:string := ()<br />
) as attribute()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all attribute nodes of a {{Code|$database}} the values of which contain one of the specified {{Code|$tokens}}.<br/>If {{Code|$name}} is specified, the resulting attribute nodes are filtered by their attribute name.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:token("DB", "row", "class")/parent::div}} returns all {{Code|div}} nodes of database {{Code|DB}} with a {{Code|class}} attribute that contains the token {{Code|row}}.<br />
|}<br />
<br />
=Updates=<br />
<br />
All functions in this section are [[#Updating Functions|Updating Functions]].<br />
<br />
==db:create==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:create(<br />
$database as xs:string,<br />
$inputs as item()* := (),<br />
$paths as xs:string* := (),<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a new {{Code|$database}} and adds the supplied {{Code|$inputs}} to the specified {{Code|$paths}}:<br />
* The inputs may be strings or nodes:<br />
** nodes may be of any type except for attributes<br />
** strings can be a URI pointing to a file/directory or an XML string (which is detected by the leading <code>&lt;</code> character)<br />
** a path must be specified if the input is not a file or directory reference<br />
* The parsing and indexing behavior can be controlled via {{Code|$options}}:<br />
** allowed options are {{Option|ADDCACHE}} and the [[Options#Indexing|indexing]], [[Options#Full-Text Indexing|full-text indexing]], [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options, all in lower case<br />
** parsing options will only impact string input (URIs, XML strings) because nodes have already been parsed.<br />
* An existing database will be overwritten.<br />
* Database creation takes place after most other update operations (see [[XQuery Update#Pending Update List|Pending Update List]]). As a consequence, a newly created database cannot be addressed in the same query.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} the specified name is not a [[Commands#Valid_Names|valid database name]].<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br/>{{Error|args|#Errors}} the number of specified inputs and paths differs.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:create("DB")}} creates the empty database {{Code|DB}}.<br />
* {{Code|db:create("DB", "/home/dir/doc.xml")}} creates the database {{Code|DB}} and adds the document {{Code|/home/dir/doc.xml}} as initial content.<br />
* {{Code|db:create("DB", <a/>, "doc.xml")}} creates the database {{Code|DB}} and adds the document with content {{Code|&lt;a/&gt;}} under the name {{Code|doc.xml}}.<br />
* {{Code|db:create("DB", "/home/dir/", "docs/dir")}} creates the database {{Code|DB}} and adds the documents in {{Code|/home/dir}} to the database under the path {{Code|docs/dir}}.<br />
* <code>db:create("DB", file:list('.'), (), map { 'ftindex': true() })</code> adds all files of the current working directory to a new database, preserving relative filesystem paths and creating a full-text index.<br />
|}<br />
<br />
==db:add==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:add(<br />
$database as xs:string,<br />
$input as item(),<br />
$path as xs:string? := (),<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Adds the specified {{Code|$input}} to a {{Code|$database}} with the specified {{Code|$path}}:<br />
* A document with the same path may occur more than once in a database. If you want to enforce single instances, use {{Function||db:put}} instead.<br />
* See {{Function||db:create}} for more details on the input and path arguments.<br />
* The parsing behavior can be controlled via {{Code|$options}}:<br />
** allowed options are {{Option|ADDCACHE}} and the [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options, all in lower case<br />
** parsing options will only impact string input (URIs, XML strings) because nodes have already been parsed<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>db:add("DB", "/home/dir/doc.xml")</code> adds the file {{Code|/home/dir/doc.xml}} to the database {{Code|DB}}.<br />
* <code>db:add("DB", <a/>, "doc.xml")</code> adds a document node to the database {{Code|DB}} under the name {{Code|doc.xml}}.<br />
* <code>db:add("DB", "/home/dir", "docs/dir", map { 'addcache': true() })</code> adds all documents in {{Code|/home/dir}} to the database {{Code|DB}} under the path {{Code|docs/dir}}. To reduce memory consumption, the files will be cached before being added to the database.<br />
|}<br />
<br />
==db:put==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:put(<br />
$database as xs:string,<br />
$input as item(),<br />
$path as xs:string,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Replaces a resource, specified by {{Code|$path}}, in a {{Code|$database}} with the contents of {{Code|$input}}, or adds it as a new resource:<br />
* The parsing behavior can be controlled via {{Code|$options}}:<br />
** Allowed options are {{Option|ADDCACHE}} and the [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options, all in lower case.<br />
** Parsing options will only impact string input (URIs, XML strings), because nodes have already been parsed.<br />
* See {{Function||db:create}} for more details on the input argument.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|target|#Errors}} the path points to a directory.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:put("DB", "/home/dir/doc.xml", "docs/dir/doc.xml")}} replaces the content of the document {{Code|docs/dir/doc.xml}} in the database {{Code|DB}} with the content of the file {{Code|/home/dir/doc.xml}}.<br />
* {{Code|db:put("DB", "<a/>", "docs/dir/doc.xml")}} replaces the content of the document {{Code|docs/dir/doc.xml}} in the database {{Code|DB}} with {{Code|&lt;a/&gt;}}.<br />
* {{Code|db:put("DB", document { <a/> }, "docs/dir/doc.xml")}} replaces the content of the document {{Code|docs/dir/doc.xml}} in the database {{Code|DB}} with the specified document node.<br />
The following query can be used to import files from a directory to a database:<br />
<pre lang='xquery'><br />
let $source := '/home/john/xml/source'<br />
for $file in file:list($source, true())<br />
let $path := $source || $file<br />
where not(file:is-dir($path))<br />
return db:put('db', doc($path), $file)<br />
</pre><br />
|}<br />
<br />
==db:put-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:put-binary(<br />
$database as xs:string,<br />
$input as item(),<br />
$path as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Stores a binary resource specified by {{Code|$input}} in a {{Code|$database}} at the specified {{Code|$path}}. Existing resources are overwritten.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:put-binary('DB', file:read-binary('video.mov'), 'video/sample.mov')}} stores the addressed video file at the specified location.<br />
* With the following query, you can copy the binary resources of one database into another:<br />
<pre lang='xquery'><br />
let $db := 'db'<br />
let $src-path := 'src/'<br />
let $trg-path := 'trg/'<br />
for $src in db:list($db, $src-path)<br />
where db:type($db, $src) = 'binary'<br />
let $trg := $trg-path || substring-after($src, $src-path)<br />
return db:put-binary($db, db:get-binary($db, $src), $trg)<br />
</pre><br />
|}<br />
<br />
==db:put-value==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:put-value(<br />
$database as xs:string,<br />
$input as item()*,<br />
$path as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Stores an {{Code|$input}} value in a {{Code|$database}} at the specified {{Code|$path}}. Existing resources are overwritten. The value can be an arbitrary sequence of atomic items, nodes, maps, and arrays.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:put-value('DB', 1 to 10000, 'sequence')}} stores a numeric range in the database.<br />
* With the following query, a map with countries and associated cities is stored in a database. The value resource can e.g. be used as index in future queries:<br />
<pre lang='xquery'><br />
db:put-value(<br />
'factbook',<br />
map:merge(<br />
for $country in db:get('factbook')//country<br />
return map:entry($country/@name, $country//city/name ! string())<br />
),<br />
'cities'<br />
)</pre><br />
|}<br />
<br />
==db:delete==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:delete(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Deletes resource(s), specified by {{Code|$path}}, from the specified {{Code|$database}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|path|#Errors}} the specified path is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:delete("DB", "docs/dir/doc.xml")}} deletes the resource {{Code|docs/dir/doc.xml}} from {{Code|DB}}.<br />
* {{Code|db:delete("DB", "docs/dir")}} deletes all resources from {{Code|DB}} in the specified path {{Code|docs/dir}}.<br />
|}<br />
<br />
==db:copy==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:copy(<br />
$database as xs:string,<br />
$new-name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a copy of {{Code|$database}}, which will be called {{Code|$new-name}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|}<br />
<br />
==db:alter==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:alter(<br />
$database as xs:string,<br />
$new-name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Renames a {{Code|$database}} to {{Code|$new-name}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|}<br />
<br />
==db:optimize==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:optimize(<br />
$database as xs:string,<br />
$all as xs:boolean? := false(),<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Optimizes the metadata and indexes of a {{Code|$database}}.<br/>If {{Code|$all}} is {{Code|true}}, the complete database will be rebuilt.<br/>The {{Code|$options}} argument can be used to control indexing. The syntax is identical to the {{Function||db:create}} function: Allowed options are all [[Options#Indexing|indexing]] and [[Options#Full-Text|full-text]] options. {{Option|UPDINDEX}} is only supported if {{Code|$all}} is {{Code|true}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:optimize("DB")}} optimizes the database structures of the database {{Code|DB}}.<br />
* <code>db:optimize("DB", true(), map { 'ftindex': true() })</code> optimizes all database structures of the database {{Code|DB}} and creates a full-text index.<br />
|}<br />
<br />
==db:rename==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:rename(<br />
$database as xs:string,<br />
$source as xs:string,<br />
$target as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Moves all resources(s) of a {{Code|$database}}, which are found in the supplied {{Code|$source}} path, to the supplied {{Code|$target}} path. The paths may point to single resources or directories. No updates will take place if a non-existing source path is supplied.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|path|#Errors}} the specified source or target path, or one of its descendants, is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:rename("DB", "docs/dir/doc.xml", "docs/dir/newdoc.xml")}} renames the resource {{Code|docs/dir/doc.xml}} to {{Code|docs/dir/newdoc.xml}} in the database {{Code|DB}}.<br />
* {{Code|db:rename("DB", "docs/dir", "docs/newdir")}} moves all resources in the database {{Code|DB}} from {{Code|docs/dir}} to {Code|docs/newdir}}.<br />
|}<br />
<br />
==db:flush==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:flush(<br />
$database as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Explicitly flushes the buffers of a {{Code|$database}}. This command is only useful if {{Option|AUTOFLUSH}} has been set to {{Code|false}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|}<br />
<br />
==db:drop==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:drop(<br />
$database as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Drops a {{Code|$database}} and all connected resources.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:drop("DB")}} drops the database {{Code|DB}}.<br />
|}<br />
<br />
=Backups=<br />
<br />
All functions in this section except for {{Function||db:backups}} are {{Function|Database|Updating Functions}}.<br />
<br />
==db:create-backup==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:create-backup(<br />
$database as xs:string,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a backup of a {{Code|$database}}. If no name is supplied, general data will be backed up. The following {{Code|$options}} are available:<br />
* With {{Code|comment}}, a comment string can be attached to the backup.<br />
* By setting {{Code|compress}} to false, the backup will be created faster, but it will take more space on disk.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:create-backup('DB', map { 'compress': false() })}} creates a backup of the database {{Code|DB}} without compressing its entries.<br />
|}<br />
<br />
==db:drop-backup==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:drop-backup(<br />
$name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Drops all backups of the database with the specified {{Code|$name}}. If the name ends with a timestamp, only the specified backup file will be deleted. If no name is supplied, backups with general data are addressed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|backup|#Errors}} No backup file found.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:drop-backup("DB")}} drops all backups of the database {{Code|DB}}.<br />
* {{Code|db:drop-backup("DB-2014-03-13-17-36-44")}} drops the specific backup file {{Code|DB-2014-03-13-17-36-44.zip}} of the database {{Code|DB}}.<br />
|}<br />
<br />
==db:alter-backup==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:alter-backup(<br />
$name as xs:string,<br />
$new-name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Renames all backups of the database with the specified {{Code|$name}} to {{Code|$new-name}}. If the name ends with a date, only the specified backup file will be renamed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|backup|#Errors}} No backup file found.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:alter-backup("DB", "DB2)}} renames all backups of the database {{Code|DB}} to {{Code|DB2}}.<br />
|}<br />
<br />
==db:restore==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:restore(<br />
$name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Restores the database with the specified {{Code|$name}}. The {{Code|$name}} may include the timestamp of the backup file. If no name is supplied, general data will be restored. If general data is restored, it will only be available after BaseX has been restarted.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|no-backup|#Errors}} No backup found.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:restore("DB")}} restores the database {{Code|DB}}.<br />
* {{Code|db:restore("DB-2014-03-13-18-05-45")}} restores the database {{Code|DB}} from the backup file with the given timestamp.<br />
|}<br />
<br />
==db:backups==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:backups(<br />
$database as xs:string := ()<br />
) as element(backup)*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns an element sequence containing all available database backups with timestamp, file size and comment.<br/>If a {{Code|$database}} is specified, the sequence will be restricted to the backups matching this database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:backups("factbook")}} returns all backups that have been made from the {{Code|factbook}} database.<br />
|}<br />
<br />
=Helper Functions=<br />
<br />
==db:name==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:name(<br />
$node as node()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the name of the database in which the specified [[#Database Nodes|database node]] {{Code|$node}} is stored.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|}<br />
<br />
==db:path==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:path(<br />
$node as node()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the path of the database document in which the specified [[#Database Nodes|database node]] {{Code|$node}} is stored.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|}<br />
<br />
==db:exists==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:exists(<br />
$database as xs:string,<br />
$path as xs:string := ()<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Checks if a {{Code|$database}} exists, or a resource located at {{Code|$path}} in this database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:exists("DB")}} returns {{Code|true}} if the database {{Code|DB}} exists.<br />
* {{Code|db:exists("DB", "resource")}} returns {{Code|true}} if {{Code|resource}} exists in this database.<br />
|}<br />
<br />
==db:type==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:type(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the type ({{Code|xml}}, {{Code|binary}}, {{Code|value}}) of a resource in a {{Code|$database}} at the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:type("DB", "factbook.xml")}} returns {{Code|true}} if the specified resource is an XML document.<br />
|}<br />
<br />
==db:content-type==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:content-type(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Retrieves the content-type of a resource in a {{Code|$database}} at the specified {{Code|$path}}.<br/>The file extension is used to recognize the content-type of a resource stored in the database. {{Code|application/xml}} will be returned for any XML document stored in the database, regardless of its file name extension.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:content-type("DB", "docs/doc01.pdf")}} returns {{Code|application/pdf}}.<br />
* {{Code|db:content-type("DB", "docs/doc01.xml")}} returns {{Code|application/xml}}.<br />
* {{Code|db:content-type("DB", "docs/doc01")}} returns {{Code|application/xml}}, if {{Code|db:is-xml("DB", "docs/doc01")}} returns {{Code|true}}.<br />
|}<br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
| Description<br />
|- valign="top"<br />
|{{Code|args}}<br />
|The number of specified inputs and paths differs.<br />
|- valign="top"<br />
|{{Code|conflict}}<br />
|Multiple update operations point to the same target.<br />
|- valign="top"<br />
|{{Code|lock}}<br />
|A database cannot be updated because it is opened by another process.<br />
|- valign="top"<br />
|{{Code|mainmem}}<br />
|The addressed database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
|{{Code|name}}<br />
|The name of the specified database is invalid.<br />
|- valign="top"<br />
|{{Code|no-backup}}<br />
|No backup exists for a database.<br />
|- valign="top"<br />
|{{Code|node}}<br />
|The referenced XML node is no [[#Database Nodes|database node]], i.e. it is neither stored in a database nor represented as database fragment.<br />
|- valign="top"<br />
|{{Code|no-index}}<br />
|The database lacks an index structure required by the called function.<br />
|- valign="top"<br />
|{{Code|open}}<br />
|The addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
|{{Code|option}}<br />
|The specified option is unknown.<br />
|- valign="top"<br />
|{{Code|path}}<br />
|The specified database path is invalid.<br />
|- valign="top"<br />
|{{Code|property}}<br />
|The specified database property is unknown.<br />
|- valign="top"<br />
|{{Code|range}}<br />
|The addressed database ID or PRE value is out of range.<br />
|- valign="top"<br />
|{{Code|target}}<br />
|Path points to an invalid target.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 10<br />
* Added: {{Function||db:get}}, {{Function||db:put}}, {{Function||db:type}}.<br />
* Added: [[#Backups|Backups]]: Support for general data ([[User Management|registered users]], [[Job Module#Services|scheduled services]] and [[Store Module|key-value stores]]).<br />
* Updated: {{Function||db:get}}, {{Function||db:get-id}}, {{Function||db:get-pre}} renamed (before: {{Code|db:open}}, {{Code|db:open-id}}, {{Code|db:open-pre}})<br />
* Updated: {{Function||db:put}} renamed (before: {{Code|db:replace}}); function signature aligned with {{Function||db:add}} (second and third argument swapped).<br />
* Updated: {{Function||db:put-binary}} renamed (before: {{Code|db:store}}); function signature aligned with {{Function||db:add}} (second and third argument swapped).<br />
* Updated: {{Function||db:get-binary}} renamed (before: {{Code|db:retrieve}}).<br />
* Updated: {{Function||db:backups}}, {{Function||db:create-backup}}: Options added.<br />
* Removed: {{Code|db:is-raw}}, {{Code|db:is-raw}} (new: {{Function||db:type}}).<br />
<br />
;Version 9.3<br />
* Added: {{Function||db:alter-backup}}<br />
* Updated: {{Code|db:open-id}}, {{Code|db:open-pre}}: support for multiple integers<br />
<br />
;Version 9.2<br />
* Added: {{Function||db:dir}}<br />
* Updated: {{Function||db:add}}: {{Code|$path}} allow empty path argument<br />
<br />
;Version 9.0<br />
* Added: {{Function||db:option}}<br />
* Updated: db:output renamed to {{Function|Update|update:output}}, db:output-cache renamed to {{Function|Update|update:cache}}<br />
* Updated: error codes updated; errors now use the module namespace<br />
<br />
;Version 8.6<br />
* Added: {{Function||db:property}}<br />
<br />
;Version 8.4<br />
* Updated: {{Function||db:create}}, {{Function||db:add}}, {{Code|db:replace}}: support for {{Code|ADDCACHE}} option.<br />
* Added: {{Function||db:token}}<br />
<br />
;Version 8.3<br />
* Updated: {{Function||db:list-details}}: attributes with name of database and date of backup added to results.<br />
* Updated: {{Function||db:backups}} now include attributes with name of database and date of backup.<br />
* Updated: {{Function|Database|Value Indexes}}: raise error if no index exists.<br />
<br />
;Version 8.2<br />
* Added: {{Function||db:output-cache}}<br />
* Removed: db:event<br />
<br />
;Version 7.9<br />
* Updated: parsing options added to {{Function||db:create}}, {{Function||db:add}} and {{Code|db:replace}}.<br />
* Updated: allow {{Option|UPDINDEX}} if {{Code|$all}} is {{Code|true}}.<br />
<br />
;Version 7.8.2<br />
* Added: {{Function||db:alter}}, {{Function||db:copy}}, {{Function||db:create-backup}}, {{Function||db:drop-backup}}, {{Function||db:restore}}<br />
<br />
;Version 7.8<br />
* Removed: db:fulltext (use {{Function|Full-Text|ft:search}} instead)<br />
<br />
;Version 7.7<br />
* Added: {{Function||db:export}}, {{Function||db:name}}, {{Function||db:path}}<br />
* Updated: {{Code|$options}} argument added to {{Function||db:create}} and {{Function||db:optimize}}.<br />
* Updated: the functions no longer accept [[#Database Nodes|database nodes]] as reference. Instead, the name of a database must now be specified.<br />
<br />
;Version 7.6<br />
* Updated: {{Function||db:create}}: allow more than one input and path.<br />
<br />
;Version 7.5<br />
* Updated: {{Function||db:add}}: input nodes will be automatically converted to document nodes<br />
* Added: {{Function||db:backups}}<br />
* Added: {{Function||db:create}}<br />
* Added: {{Function||db:drop}}<br />
<br />
;Version 7.3<br />
* Added: {{Function||db:flush}}<br />
<br />
;Version 7.2.1<br />
* Added: {{Function||db:text-range}}, {{Function||db:attribute-range}}, {{Function||db:output}}<br />
<br />
;Version 7.1<br />
* Added: {{Function||db:list-details}}, {{Function||db:content-type}}<br />
* Updated: {{Function||db:info}}, {{Function||db:system}}, {{Code|db:retrieve}}<br />
<br />
;Version 7.0<br />
* Added: {{Function||db:exists}}, {{Code|db:retrieve}}, {{Code|db:store}}, {{Code|db:is-raw}}, {{Code|db:is-xml}}<br />
* Updated: {{Function||db:list}}, {{Code|db:open}}, {{Function||db:add}}</div>CGhttps://docs.basex.org/index.php?title=Database_Module&diff=16912Database Module2023-12-01T16:40:57Z<p>CG: /* Errors */</p>
<hr />
<div>This [[Module Library|XQuery Module]] contains functions for processing databases from within XQuery. Existing databases can be opened and listed, its contents can be directly accessed, documents can be added to and removed, etc.<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/db</nowiki></code> namespace, which is statically bound to the {{Code|db}} prefix.<br/><br />
<br />
==Database Nodes==<br />
<br />
In BaseX, two internal representations exist for XML nodes:<br />
<br />
* Database nodes are:<br />
** stored in a persistent database on disk;<br />
** nodes of a document that has been generated temporarily with {{Code|fn:doc}}, {{Code|fn:parse-xml}} and other functions; or<br />
** the result of a [[XQuery Update#Main-Memory Updates|main-memory update]].<br />
* Fragments are similar to DOM structures. They are created when XQuery node constructors are used (<code><a/></code>, <code>element a { }</code>, etc.).<br />
<br />
Fragments require less memory for small XML structures, but database nodes are more efficient for larger amounts of data. Fragments nodes can be converted to database nodes by applying a [[XQuery_Update#Main-Memory Updates|main-memory update]] with an empty body to a node:<br />
<br />
<pre lang='xquery'><br />
<xml>hello world</xml> update { }<br />
</pre><br />
<br />
==Updating Functions==<br />
<br />
Various functions in this module are ''updating''. Updating functions will not be immediately executed, but queued on the [[XQuery Update#Pending Update List|Pending Update List]], and processed after the remaining query has been evaluated. This means that the order in which the functions are specified in the query often does not reflect the order in which they will eventually be executed.<br />
<br />
=General Functions=<br />
<br />
==db:system==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:system() as element(system)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns general information on the database system the current values of all global and local [[Options]]. The {{Command|INFO}} command returns similar output.<br />
|}<br />
<br />
==db:option==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:option(<br />
$name as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the current value (string, integer, boolean, map) of a global or local [[Options|Option]] with the specified {{Code|$name}}. The {{Command|SHOW OPTIONS}} command returns similar output.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|option|#Errors}} the specified option is unknown.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>db:option('dbpath')</code> returns the database path string.<br />
* <code>db:option('serializer')</code> returns a map with the current serialization parameters.<br />
* <code>declare option db:stripws 'true'; db:option('stripws')</code> returns the locally assigned value.<br />
|}<br />
<br />
==db:info==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:info(<br />
$database as xs:string<br />
) as element(database)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns meta information on the specified {{Code|$database}}. The output is similar to the {{Command|INFO DB}} command.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|}<br />
<br />
==db:property==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:property(<br />
$database as xs:string,<br />
$name as xs:string<br />
) as xs:anyAtomicType</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the value (string, boolean, integer) of a property with the specified {{Code|$name}} in the specified {{Code|$database}}. The available properties are the ones returned by {{Function||db:info}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|property|#Errors}} the specified property is unknown.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>db:property('db', 'size')</code> returns the number of bytes occupied by the database <code>db</code>.<br />
* <code>db:property('xmark', 'textindex')</code> indicates if the <code>xmark</code> database has a text index.<br />
* <code>db:property('discogs', 'uptodate')</code> indicates if the database statistics and index structures of the <code>discogs</code> database are up-to-date.<br />
|}<br />
<br />
==db:list==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:list(<br />
$database as xs:string := (),<br />
$path as xs:string := ()<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Without arguments, the names of all databases are returned that are accessible to the current user. If {{Code|$database}} is specified, paths to all resources of this database are returned. The results can be restricted to resources starting with the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:list("docs")}} returns the names of all documents of a database named {{Code|docs}}.<br />
|}<br />
<br />
==db:list-details==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:list-details(<br />
$database as xs:string := (),<br />
$path as xs:string := ()<br />
) as element()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Without arguments, an element is returned for each database that is accessible to the current user:<br />
* An element has a value, which is the name of the database, and several attributes, which contain the number of stored resources, the modification date, the database size on disk (measured in bytes), and a path to the original database input.<br />
If {{Code|$database}} is specified, an element for each resource in this database is returned:<br />
* An element has a value, which is the name of the resource, and several attributes, which contain the content type, the modification date, the raw flag (which indicates if the resource is binary or XML), and the size of a resource.<br />
* The value of the size attribute depends on the resource type: for documents, it represents the number of nodes; for binary data, it represents the file size (measured in bytes).<br />
* The results can be restricted to resources starting with the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:list-details("shop")}} returns the names plus additional info on all resources of a database named {{Code|shop}}.<br />
|}<br />
<br />
==db:dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:dir(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as element()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns metadata on all directories and resources of a {{Code|$database}} in the specified {{Code|$path}}. Two types of elements are returned:<br />
* {{Code|resource}} represents a resource. The element value is the directory path; content type, modification date, raw flag (which indicates if the resource is binary or XML), and size of the resource are returned as attributes.<br />
* {{Code|dir}} represents a directory. The element value is the directory path; the modification date is returned in an attribute.<br />
The directories are not stored in the internal database layout. Instead, they result implicitly from the paths of stored resources.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|path|#Errors}} the specified path is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:dir('shop', 'books')}} returns all entries of the {{Code|books}} directory of a {{Code|shop}} database.<br />
|}<br />
<br />
=Read Operations=<br />
<br />
==db:get==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get(<br />
$database as xs:string,<br />
$path as xs:string := ()<br />
) as document-node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all documents from the specified {{Code|$database}}, or only documents matching the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get('docs')}} returns all documents from the database named {{Code|docs}}.<br />
* {{Code|db:get('db', 'one')}} returns all documents from the database named {{Code|db}} located in the path {{Code|one}}.<br />
* <code>for $i in 1 to 3 return db:get('db' || $i)//item</code> returns all item elements from the databases {{Code|db1}}, {{Code|db2}} and {{Code|db3}}.<br />
|}<br />
<br />
==db:get-pre==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-pre(<br />
$database as xs:string,<br />
$values as xs:integer*<br />
) as node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all nodes from a {{Code|$database}} with the specified PRE {{Code|values}} in [[Utility Module#util:ddo|distinct document order]].<br/>The [[Node Storage#PRE Value|PRE value]] provides very fast access to an existing database node, but it will change whenever a node with a smaller ''pre'' values is added to or deleted from a database.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|range|#Errors}} the specified PRE value does not exist in the database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get-pre("docs", 0)}} returns the first database node from the database named {{Code|docs}}.<br />
|}<br />
<br />
==db:get-id==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-id(<br />
$database as xs:string,<br />
$values as xs:integer*<br />
) as node()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all nodes from a {{Code|$database}} with the specified ID {{Code|$values}} in [[Utility Module#util:ddo|distinct document order]].<br/>Each database node has a ''persistent'' [[Node Storage#ID Value|ID value]]. Access to the node ID can be sped up by turning on the {{Option|UPDINDEX}} option.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|range|#Errors}} the specified ID value does not exist in the database.<br />
|}<br />
<br />
==db:get-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-binary(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as item()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns a map with the paths and binary items of all resources in the specified {{Code|$database}}. A single {{Code|xs:base64Binary}} item is returned if a {{Code|$path}} is specified. All items are [[Lazy Module|lazy]], i.e., the actual data will only be retrieved if it is processed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get-binary('DB', 'music/01.mp3')}} returns the specified audio file as raw data.<br />
* <code><nowiki>stream:materialize(db:get-binary('DB', 'music/01.mp3'))</nowiki></code> materializes the streamable result in main-memory before returning it.<br />
* <code><nowiki>convert:binary-to-string(db:get-binary('DB', 'info.txt'), 'UTF-8')</nowiki></code> converts a binary database resource as UTF-8 text and returns a string.<br />
|}<br />
<br />
==db:get-value==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:get-value(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns a map with the paths and values of all resources in the specified {{Code|$database}}. A single value is returned if a {{Code|$path}} is specified.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:get-value('DB', 'sequence')}} returns the specified sequence.<br />
|}<br />
<br />
==db:node-pre==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:node-pre(<br />
$nodes as node()*<br />
) as xs:integer*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the PRE values of the specified {{Code|$nodes}}, which must all be [[#Database Nodes|database nodes]].<br/>The [[Node Storage#PRE Value|PRE value]] provides very fast access to an existing database node, but it will change whenever a node with a smaller ''pre'' values is added to or deleted from a database.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:node-pre(doc("input"))}} returns {{Code|0}} if the database {{Code|input}} contains a single document.<br />
|}<br />
<br />
==db:node-id==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:node-id(<br />
$nodes as node()*<br />
) as xs:integer*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the ID values of the specified {{Code|$nodes}}, which must all be [[#Database Nodes|database nodes]].<br/>Each database node has a ''persistent'' [[Node Storage#ID Value|ID value]]. Access to the node id can be sped up by turning on the {{Option|UPDINDEX}} option.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|}<br />
<br />
==db:export==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:export(<br />
$database as xs:string,<br />
$path as xs:string,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Exports the specified {{Code|$database}} to the specified file {{Code|$path}}. Existing files will be overwritten.<br/>The {{Code|$options}} argument contains [[Serialization|serialization parameters]] (see [https://www.w3.org/TR/xpath-functions-31/#func-serialize fn:serialize()]).<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
| Export all files as text:<br/><br />
<pre lang='xquery'><br />
db:export("DB", "/home/john/xml/texts", map { 'method': 'text' })<br />
</pre><br />
The following code can be used to export parts of the database:<br />
<pre lang='xquery'><br />
let $target := '/home/john/xml/target'<br />
for $doc in db:get('DB', 'collection')<br />
let $path := $target || db:path($doc)<br />
return (<br />
file:create-dir(file:parent($path)),<br />
file:write($path, $doc)<br />
)<br />
</pre><br />
|}<br />
<br />
=Value Indexes=<br />
<br />
==db:text==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:text(<br />
$database as xs:string,<br />
$values as xs:string*<br />
) as text()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all text nodes of a {{Code|$database}} that match one of the specified {{Code|$values}} and that are stored in the text index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:text("DB", "QUERY")/..}} returns the parents of all text nodes of the database {{Code|DB}} that match the string {{Code|QUERY}}.<br />
|}<br />
<br />
==db:text-range==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:text-range(<br />
$database as xs:string,<br />
$min as xs:string,<br />
$max as xs:string<br />
) as text()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all text nodes of a {{Code|$database}} whose values are larger than or equal to {{Code|$min}} and smaller than or equal to {{Code|$max}} and that are stored in the text index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:text-range("DB", "2000", "2001")}} returns all text nodes of the database {{Code|DB}} that are found in between {{Code|2000}} and {{Code|2001}}.<br />
|}<br />
<br />
==db:attribute==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:attribute(<br />
$database as xs:string,<br />
$values as xs:string*,<br />
$name as xs:string := ()<br />
) as attribute()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all attribute nodes of a {{Code|$database}} that match one of the specified {{Code|$values}} and that are stored in the attribute index.<br/>If {{Code|$name}} is specified, the resulting attribute nodes are filtered by their attribute name.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:attribute("DB", "QUERY", "id")/..}} returns the parents of all {{Code|id}} attribute nodes of the database {{Code|DB}} that have {{Code|QUERY}} as string value.<br />
|}<br />
<br />
==db:attribute-range==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:attribute-range(<br />
$database as xs:string,<br />
$min as xs:string,<br />
$max as xs:string,<br />
$name as xs:string := ()<br />
) as attribute()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all attributes of a {{Code|$database}} whose values are larger than or equal to {{Code|$min}} and smaller than or equal to {{Code|$max}} and that are stored in the attribute index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:attribute-range("DB", "id456", "id473", 'id')}} returns all {{Code|@id}} attributes of the database {{Code|DB}} that have a string value in between {{Code|id456}} and {{Code|id473}}.<br />
|}<br />
<br />
==db:token==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:token(<br />
$database as xs:string,<br />
$tokens as xs:string*,<br />
$name as xs:string := ()<br />
) as attribute()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns all attribute nodes of a {{Code|$database}} the values of which contain one of the specified {{Code|$tokens}}.<br/>If {{Code|$name}} is specified, the resulting attribute nodes are filtered by their attribute name.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|no-index|#Errors}} the index is not available.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:token("DB", "row", "class")/parent::div}} returns all {{Code|div}} nodes of database {{Code|DB}} with a {{Code|class}} attribute that contains the token {{Code|row}}.<br />
|}<br />
<br />
=Updates=<br />
<br />
All functions in this section are [[#Updating Functions|Updating Functions]].<br />
<br />
==db:create==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:create(<br />
$database as xs:string,<br />
$inputs as item()* := (),<br />
$paths as xs:string* := (),<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a new {{Code|$database}} and adds the supplied {{Code|$inputs}} to the specified {{Code|$paths}}:<br />
* The inputs may be strings or nodes:<br />
** nodes may be of any type except for attributes<br />
** strings can be a URI pointing to a file/directory or an XML string (which is detected by the leading <code>&lt;</code> character)<br />
** a path must be specified if the input is not a file or directory reference<br />
* The parsing and indexing behavior can be controlled via {{Code|$options}}:<br />
** allowed options are {{Option|ADDCACHE}} and the [[Options#Indexing|indexing]], [[Options#Full-Text Indexing|full-text indexing]], [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options, all in lower case<br />
** parsing options will only impact string input (URIs, XML strings) because nodes have already been parsed.<br />
* An existing database will be overwritten.<br />
* Database creation takes place after most other update operations (see [[XQuery Update#Pending Update List|Pending Update List]]). As a consequence, a newly created database cannot be addressed in the same query.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} the specified name is not a [[Commands#Valid_Names|valid database name]].<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br/>{{Error|args|#Errors}} the number of specified inputs and paths differs.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:create("DB")}} creates the empty database {{Code|DB}}.<br />
* {{Code|db:create("DB", "/home/dir/doc.xml")}} creates the database {{Code|DB}} and adds the document {{Code|/home/dir/doc.xml}} as initial content.<br />
* {{Code|db:create("DB", <a/>, "doc.xml")}} creates the database {{Code|DB}} and adds the document with content {{Code|&lt;a/&gt;}} under the name {{Code|doc.xml}}.<br />
* {{Code|db:create("DB", "/home/dir/", "docs/dir")}} creates the database {{Code|DB}} and adds the documents in {{Code|/home/dir}} to the database under the path {{Code|docs/dir}}.<br />
* <code>db:create("DB", file:list('.'), (), map { 'ftindex': true() })</code> adds all files of the current working directory to a new database, preserving relative filesystem paths and creating a full-text index.<br />
|}<br />
<br />
==db:add==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:add(<br />
$database as xs:string,<br />
$input as item(),<br />
$path as xs:string? := (),<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Adds the specified {{Code|$input}} to a {{Code|$database}} with the specified {{Code|$path}}:<br />
* A document with the same path may occur more than once in a database. If you want to enforce single instances, use {{Function||db:put}} instead.<br />
* See {{Function||db:create}} for more details on the input and path arguments.<br />
* The parsing behavior can be controlled via {{Code|$options}}:<br />
** allowed options are {{Option|ADDCACHE}} and the [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options, all in lower case<br />
** parsing options will only impact string input (URIs, XML strings) because nodes have already been parsed<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>db:add("DB", "/home/dir/doc.xml")</code> adds the file {{Code|/home/dir/doc.xml}} to the database {{Code|DB}}.<br />
* <code>db:add("DB", <a/>, "doc.xml")</code> adds a document node to the database {{Code|DB}} under the name {{Code|doc.xml}}.<br />
* <code>db:add("DB", "/home/dir", "docs/dir", map { 'addcache': true() })</code> adds all documents in {{Code|/home/dir}} to the database {{Code|DB}} under the path {{Code|docs/dir}}. To reduce memory consumption, the files will be cached before being added to the database.<br />
|}<br />
<br />
==db:put==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:put(<br />
$database as xs:string,<br />
$input as item(),<br />
$path as xs:string,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Replaces a resource, specified by {{Code|$path}}, in a {{Code|$database}} with the contents of {{Code|$input}}, or adds it as a new resource:<br />
* The parsing behavior can be controlled via {{Code|$options}}:<br />
** Allowed options are {{Option|ADDCACHE}} and the [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options, all in lower case.<br />
** Parsing options will only impact string input (URIs, XML strings), because nodes have already been parsed.<br />
* See {{Function||db:create}} for more details on the input argument.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|target|#Errors}} the path points to a directory.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:put("DB", "/home/dir/doc.xml", "docs/dir/doc.xml")}} replaces the content of the document {{Code|docs/dir/doc.xml}} in the database {{Code|DB}} with the content of the file {{Code|/home/dir/doc.xml}}.<br />
* {{Code|db:put("DB", "<a/>", "docs/dir/doc.xml")}} replaces the content of the document {{Code|docs/dir/doc.xml}} in the database {{Code|DB}} with {{Code|&lt;a/&gt;}}.<br />
* {{Code|db:put("DB", document { <a/> }, "docs/dir/doc.xml")}} replaces the content of the document {{Code|docs/dir/doc.xml}} in the database {{Code|DB}} with the specified document node.<br />
The following query can be used to import files from a directory to a database:<br />
<pre lang='xquery'><br />
let $source := '/home/john/xml/source'<br />
for $file in file:list($source, true())<br />
let $path := $source || $file<br />
where not(file:is-dir($path))<br />
return db:put('db', doc($path), $file)<br />
</pre><br />
|}<br />
<br />
==db:put-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:put-binary(<br />
$database as xs:string,<br />
$input as item(),<br />
$path as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Stores a binary resource specified by {{Code|$input}} in a {{Code|$database}} at the specified {{Code|$path}}. Existing resources are overwritten.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:put-binary('DB', file:read-binary('video.mov'), 'video/sample.mov')}} stores the addressed video file at the specified location.<br />
* With the following query, you can copy the binary resources of one database into another:<br />
<pre lang='xquery'><br />
let $db := 'db'<br />
let $src-path := 'src/'<br />
let $trg-path := 'trg/'<br />
for $src in db:list($db, $src-path)<br />
where db:type($db, $src) = 'binary'<br />
let $trg := $trg-path || substring-after($src, $src-path)<br />
return db:put-binary($db, db:get-binary($db, $src), $trg)<br />
</pre><br />
|}<br />
<br />
==db:put-value==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:put-value(<br />
$database as xs:string,<br />
$input as item()*,<br />
$path as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Stores an {{Code|$input}} value in a {{Code|$database}} at the specified {{Code|$path}}. Existing resources are overwritten. The value can be an arbitrary sequence of atomic items, nodes, maps, and arrays.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|mainmem|#Errors}} the database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:put-value('DB', 1 to 10000, 'sequence')}} stores a numeric range in the database.<br />
* With the following query, a map with countries and associated cities is stored in a database. The value resource can e.g. be used as index in future queries:<br />
<pre lang='xquery'><br />
db:put-value(<br />
'factbook',<br />
map:merge(<br />
for $country in db:get('factbook')//country<br />
return map:entry($country/@name, $country//city/name ! string())<br />
),<br />
'cities'<br />
)</pre><br />
|}<br />
<br />
==db:delete==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:delete(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Deletes resource(s), specified by {{Code|$path}}, from the specified {{Code|$database}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|path|#Errors}} the specified path is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:delete("DB", "docs/dir/doc.xml")}} deletes the resource {{Code|docs/dir/doc.xml}} from {{Code|DB}}.<br />
* {{Code|db:delete("DB", "docs/dir")}} deletes all resources from {{Code|DB}} in the specified path {{Code|docs/dir}}.<br />
|}<br />
<br />
==db:copy==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:copy(<br />
$database as xs:string,<br />
$new-name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a copy of {{Code|$database}}, which will be called {{Code|$new-name}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|}<br />
<br />
==db:alter==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:alter(<br />
$database as xs:string,<br />
$new-name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Renames a {{Code|$database}} to {{Code|$new-name}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|}<br />
<br />
==db:optimize==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:optimize(<br />
$database as xs:string,<br />
$all as xs:boolean? := false(),<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Optimizes the metadata and indexes of a {{Code|$database}}.<br/>If {{Code|$all}} is {{Code|true}}, the complete database will be rebuilt.<br/>The {{Code|$options}} argument can be used to control indexing. The syntax is identical to the {{Function||db:create}} function: Allowed options are all [[Options#Indexing|indexing]] and [[Options#Full-Text|full-text]] options. {{Option|UPDINDEX}} is only supported if {{Code|$all}} is {{Code|true}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:optimize("DB")}} optimizes the database structures of the database {{Code|DB}}.<br />
* <code>db:optimize("DB", true(), map { 'ftindex': true() })</code> optimizes all database structures of the database {{Code|DB}} and creates a full-text index.<br />
|}<br />
<br />
==db:rename==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:rename(<br />
$database as xs:string,<br />
$source as xs:string,<br />
$target as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Moves all resources(s) of a {{Code|$database}}, which are found in the supplied {{Code|$source}} path, to the supplied {{Code|$target}} path. The paths may point to single resources or directories. No updates will take place if a non-existing source path is supplied.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|path|#Errors}} the specified source or target path, or one of its descendants, is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:rename("DB", "docs/dir/doc.xml", "docs/dir/newdoc.xml")}} renames the resource {{Code|docs/dir/doc.xml}} to {{Code|docs/dir/newdoc.xml}} in the database {{Code|DB}}.<br />
* {{Code|db:rename("DB", "docs/dir", "docs/newdir")}} moves all resources in the database {{Code|DB}} from {{Code|docs/dir}} to {Code|docs/newdir}}.<br />
|}<br />
<br />
==db:flush==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:flush(<br />
$database as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Explicitly flushes the buffers of a {{Code|$database}}. This command is only useful if {{Option|AUTOFLUSH}} has been set to {{Code|false}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|}<br />
<br />
==db:drop==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:drop(<br />
$database as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Drops a {{Code|$database}} and all connected resources.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:drop("DB")}} drops the database {{Code|DB}}.<br />
|}<br />
<br />
=Backups=<br />
<br />
All functions in this section except for {{Function||db:backups}} are {{Function|Database|Updating Functions}}.<br />
<br />
==db:create-backup==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:create-backup(<br />
$database as xs:string,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a backup of a {{Code|$database}}. If no name is supplied, general data will be backed up. The following {{Code|$options}} are available:<br />
* With {{Code|comment}}, a comment string can be attached to the backup.<br />
* By setting {{Code|compress}} to false, the backup will be created faster, but it will take more space on disk.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:create-backup('DB', map { 'compress': false() })}} creates a backup of the database {{Code|DB}} without compressing its entries.<br />
|}<br />
<br />
==db:drop-backup==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:drop-backup(<br />
$name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Drops all backups of the database with the specified {{Code|$name}}. If the name ends with a timestamp, only the specified backup file will be deleted. If no name is supplied, backups with general data are addressed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|backup|#Errors}} No backup file found.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:drop-backup("DB")}} drops all backups of the database {{Code|DB}}.<br />
* {{Code|db:drop-backup("DB-2014-03-13-17-36-44")}} drops the specific backup file {{Code|DB-2014-03-13-17-36-44.zip}} of the database {{Code|DB}}.<br />
|}<br />
<br />
==db:alter-backup==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:alter-backup(<br />
$name as xs:string,<br />
$new-name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Renames all backups of the database with the specified {{Code|$name}} to {{Code|$new-name}}. If the name ends with a date, only the specified backup file will be renamed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|backup|#Errors}} No backup file found.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:alter-backup("DB", "DB2)}} renames all backups of the database {{Code|DB}} to {{Code|DB2}}.<br />
|}<br />
<br />
==db:restore==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:restore(<br />
$name as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Restores the database with the specified {{Code|$name}}. The {{Code|$name}} may include the timestamp of the backup file. If no name is supplied, general data will be restored. If general data is restored, it will only be available after BaseX has been restarted.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|lock|#Errors}} a database is opened by another process.<br/>{{Error|name|#Errors}} invalid database name.<br/>{{Error|no-backup|#Errors}} No backup found.<br/>{{Error|conflict|#Errors}} the same database was addressed more than once.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:restore("DB")}} restores the database {{Code|DB}}.<br />
* {{Code|db:restore("DB-2014-03-13-18-05-45")}} restores the database {{Code|DB}} from the backup file with the given timestamp.<br />
|}<br />
<br />
==db:backups==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:backups(<br />
$database as xs:string := ()<br />
) as element(backup)*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns an element sequence containing all available database backups with timestamp, file size and comment.<br/>If a {{Code|$database}} is specified, the sequence will be restricted to the backups matching this database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:backups("factbook")}} returns all backups that have been made from the {{Code|factbook}} database.<br />
|}<br />
<br />
=Helper Functions=<br />
<br />
==db:name==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:name(<br />
$node as node()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the name of the database in which the specified [[#Database Nodes|database node]] {{Code|$node}} is stored.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|}<br />
<br />
==db:path==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:path(<br />
$node as node()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the path of the database document in which the specified [[#Database Nodes|database node]] {{Code|$node}} is stored.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|node|#Errors}} {{Code|$nodes}} contains a node which is not stored in a database.<br />
|}<br />
<br />
==db:exists==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:exists(<br />
$database as xs:string,<br />
$path as xs:string := ()<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Checks if a {{Code|$database}} exists, or a resource located at {{Code|$path}} in this database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:exists("DB")}} returns {{Code|true}} if the database {{Code|DB}} exists.<br />
* {{Code|db:exists("DB", "resource")}} returns {{Code|true}} if {{Code|resource}} exists in this database.<br />
|}<br />
<br />
==db:type==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:type(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the type ({{Code|xml}}, {{Code|binary}}, {{Code|value}}) of a resource in a {{Code|$database}} at the specified {{Code|$path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:type("DB", "factbook.xml")}} returns {{Code|true}} if the specified resource is an XML document.<br />
|}<br />
<br />
==db:content-type==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>db:content-type(<br />
$database as xs:string,<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Retrieves the content-type of a resource in a {{Code|$database}} at the specified {{Code|$path}}.<br/>The file extension is used to recognize the content-type of a resource stored in the database. {{Code|application/xml}} will be returned for any XML document stored in the database, regardless of its file name extension.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|open|#Errors}} the addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|db:content-type("DB", "docs/doc01.pdf")}} returns {{Code|application/pdf}}.<br />
* {{Code|db:content-type("DB", "docs/doc01.xml")}} returns {{Code|application/xml}}.<br />
* {{Code|db:content-type("DB", "docs/doc01")}} returns {{Code|application/xml}}, if {{Code|db:is-xml("DB", "docs/doc01")}} returns {{Code|true}}.<br />
|}<br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
! Description<br />
|- valign="top"<br />
|{{Code|args}}<br />
|The number of specified inputs and paths differs.<br />
|- valign="top"<br />
|{{Code|conflict}}<br />
|Multiple update operations point to the same target.<br />
|- valign="top"<br />
|{{Code|lock}}<br />
|A database cannot be updated because it is opened by another process.<br />
|- valign="top"<br />
|{{Code|mainmem}}<br />
|The addressed database is not ''persistent'' (stored on disk).<br />
|- valign="top"<br />
|{{Code|name}}<br />
|The name of the specified database is invalid.<br />
|- valign="top"<br />
|{{Code|no-backup}}<br />
|No backup exists for a database.<br />
|- valign="top"<br />
|{{Code|node}}<br />
|The referenced XML node is no [[#Database Nodes|database node]], i.e. it is neither stored in a database nor represented as database fragment.<br />
|- valign="top"<br />
|{{Code|no-index}}<br />
|The database lacks an index structure required by the called function.<br />
|- valign="top"<br />
|{{Code|open}}<br />
|The addressed database does not exist or could not be opened.<br />
|- valign="top"<br />
|{{Code|option}}<br />
|The specified option is unknown.<br />
|- valign="top"<br />
|{{Code|path}}<br />
|The specified database path is invalid.<br />
|- valign="top"<br />
|{{Code|property}}<br />
|The specified database property is unknown.<br />
|- valign="top"<br />
|{{Code|range}}<br />
|The addressed database ID or PRE value is out of range.<br />
|- valign="top"<br />
|{{Code|target}}<br />
|Path points to an invalid target.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 10<br />
* Added: {{Function||db:get}}, {{Function||db:put}}, {{Function||db:type}}.<br />
* Added: [[#Backups|Backups]]: Support for general data ([[User Management|registered users]], [[Job Module#Services|scheduled services]] and [[Store Module|key-value stores]]).<br />
* Updated: {{Function||db:get}}, {{Function||db:get-id}}, {{Function||db:get-pre}} renamed (before: {{Code|db:open}}, {{Code|db:open-id}}, {{Code|db:open-pre}})<br />
* Updated: {{Function||db:put}} renamed (before: {{Code|db:replace}}); function signature aligned with {{Function||db:add}} (second and third argument swapped).<br />
* Updated: {{Function||db:put-binary}} renamed (before: {{Code|db:store}}); function signature aligned with {{Function||db:add}} (second and third argument swapped).<br />
* Updated: {{Function||db:get-binary}} renamed (before: {{Code|db:retrieve}}).<br />
* Updated: {{Function||db:backups}}, {{Function||db:create-backup}}: Options added.<br />
* Removed: {{Code|db:is-raw}}, {{Code|db:is-raw}} (new: {{Function||db:type}}).<br />
<br />
;Version 9.3<br />
* Added: {{Function||db:alter-backup}}<br />
* Updated: {{Code|db:open-id}}, {{Code|db:open-pre}}: support for multiple integers<br />
<br />
;Version 9.2<br />
* Added: {{Function||db:dir}}<br />
* Updated: {{Function||db:add}}: {{Code|$path}} allow empty path argument<br />
<br />
;Version 9.0<br />
* Added: {{Function||db:option}}<br />
* Updated: db:output renamed to {{Function|Update|update:output}}, db:output-cache renamed to {{Function|Update|update:cache}}<br />
* Updated: error codes updated; errors now use the module namespace<br />
<br />
;Version 8.6<br />
* Added: {{Function||db:property}}<br />
<br />
;Version 8.4<br />
* Updated: {{Function||db:create}}, {{Function||db:add}}, {{Code|db:replace}}: support for {{Code|ADDCACHE}} option.<br />
* Added: {{Function||db:token}}<br />
<br />
;Version 8.3<br />
* Updated: {{Function||db:list-details}}: attributes with name of database and date of backup added to results.<br />
* Updated: {{Function||db:backups}} now include attributes with name of database and date of backup.<br />
* Updated: {{Function|Database|Value Indexes}}: raise error if no index exists.<br />
<br />
;Version 8.2<br />
* Added: {{Function||db:output-cache}}<br />
* Removed: db:event<br />
<br />
;Version 7.9<br />
* Updated: parsing options added to {{Function||db:create}}, {{Function||db:add}} and {{Code|db:replace}}.<br />
* Updated: allow {{Option|UPDINDEX}} if {{Code|$all}} is {{Code|true}}.<br />
<br />
;Version 7.8.2<br />
* Added: {{Function||db:alter}}, {{Function||db:copy}}, {{Function||db:create-backup}}, {{Function||db:drop-backup}}, {{Function||db:restore}}<br />
<br />
;Version 7.8<br />
* Removed: db:fulltext (use {{Function|Full-Text|ft:search}} instead)<br />
<br />
;Version 7.7<br />
* Added: {{Function||db:export}}, {{Function||db:name}}, {{Function||db:path}}<br />
* Updated: {{Code|$options}} argument added to {{Function||db:create}} and {{Function||db:optimize}}.<br />
* Updated: the functions no longer accept [[#Database Nodes|database nodes]] as reference. Instead, the name of a database must now be specified.<br />
<br />
;Version 7.6<br />
* Updated: {{Function||db:create}}: allow more than one input and path.<br />
<br />
;Version 7.5<br />
* Updated: {{Function||db:add}}: input nodes will be automatically converted to document nodes<br />
* Added: {{Function||db:backups}}<br />
* Added: {{Function||db:create}}<br />
* Added: {{Function||db:drop}}<br />
<br />
;Version 7.3<br />
* Added: {{Function||db:flush}}<br />
<br />
;Version 7.2.1<br />
* Added: {{Function||db:text-range}}, {{Function||db:attribute-range}}, {{Function||db:output}}<br />
<br />
;Version 7.1<br />
* Added: {{Function||db:list-details}}, {{Function||db:content-type}}<br />
* Updated: {{Function||db:info}}, {{Function||db:system}}, {{Code|db:retrieve}}<br />
<br />
;Version 7.0<br />
* Added: {{Function||db:exists}}, {{Code|db:retrieve}}, {{Code|db:store}}, {{Code|db:is-raw}}, {{Code|db:is-xml}}<br />
* Updated: {{Function||db:list}}, {{Code|db:open}}, {{Function||db:add}}</div>CGhttps://docs.basex.org/index.php?title=Repository&diff=16911Repository2023-12-01T16:39:55Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This article is part of the [[XQuery|XQuery Portal]].<br />
It describes how external XQuery modules and Java code can be installed<br />
in the XQuery module repository, and how new packages are built and deployed.<br />
<br />
=Introduction=<br />
<br />
One of the things that makes languages successful is the availability of external libraries. As XQuery comes with only 150 pre-defined functions, which cannot meet all requirements, additional library modules exist – such as [http://www.functx.com/ FunctX] – which extend the language with new features.<br />
<br />
BaseX offers the following mechanisms to make external modules accessible to the XQuery processor:<br />
<br />
# The internal [[#Packaging|Packaging]] mechanism will install single XQuery and JAR modules in the repository.<br />
# The [[#EXPath Packaging|EXPath Packaging]] system provides a generic mechanism for adding XQuery modules to query processors. A package is defined as a {{Code|.xar}} archive, which encapsulates one or more extension libraries.<br />
<br />
==Accessing Modules==<br />
<br />
Library modules can be imported with the {{Code|import module}} statement, followed by a freely choosable prefix and the namespace of the target module. The specified location may be absolute or relative; in the latter case, it is resolved against the location (i.e., ''static base URI'') of the calling module. Import module statements must be placed at the beginning of a module:<br />
<br />
'''Main Module''' <code>hello-universe.xq</code>:<br />
<br />
<pre lang='xquery'><br />
import module namespace m = 'http://basex.org/modules/hello' at 'hello-world.xqm';<br />
m:hello("Universe")<br />
</pre><br />
<br />
'''Library Module''' <code>hello-world.xqm</code> (in the same directory):<br />
<br />
<pre lang='xquery'><br />
module namespace m = 'http://basex.org/modules/Hello';<br />
declare function m:hello($world) {<br />
'Hello ' || $world<br />
};<br />
</pre><br />
<br />
If no location is supplied, modules will be looked up in the repository. Repository modules are stored in the {{Code|repo}} directory, which resides in your [[Configuration#Home Directory|home directory]]. XQuery modules can be manually copied to the repository directory or installed and deleted via [[#Commands|commands]].<br />
<br />
The following example calls a function from the FunctX module in the repository:<br />
<br />
<pre lang='xquery'><br />
import module namespace functx = 'http://www.functx.com';<br />
functx:capitalize-first('test')<br />
</pre><br />
<br />
=Commands=<br />
<br />
There are various ways to organize your packages:<br />
<br />
* Execute BaseX REPO commands (listed below)<br />
* Call XQuery functions of the [[Repository Module]]<br />
* Use the GUI (''Options'' → ''Packages'')<br />
<br />
You can even manually add and remove packages in the repository directory; all changes will automatically be detected by BaseX.<br />
<br />
==Installation==<br />
<br />
A module or package can be installed with {{Command|REPO INSTALL}}. The path to the file has to be given as a parameter:<br />
<br />
REPO INSTALL https://files.basex.org/modules/expath/functx-1.0.xar<br />
REPO INSTALL hello-world.xqm<br />
<br />
The installation will only succeed if the specified file conforms to the constraints described below. If you know that your input is valid, you may as well copy the files directly to the repository directory, or edit its contents in the repository without deleting and reinstalling them.<br />
<br />
==Listing==<br />
<br />
All currently installed packages can be listed with {{Command|REPO LIST}}. The names of all packages are listed, along with their version, their package type, and the repository path:<br />
<br />
Name Version Type Path<br />
-----------------------------------------------------------------<br />
<nowiki>http://www.functx.com</nowiki> 1.0 EXPath http-www.functx.com-1.0<br />
<br />
==Removal==<br />
<br />
A package can be deleted with {{Command|REPO DELETE}} and an additional argument, containing its name or the name suffixed with a hyphen and the package version:<br />
<br />
REPO DELETE <nowiki>http://www.functx.com</nowiki><br />
REPO DELETE <nowiki>http://www.functx.com-1.0</nowiki><br />
<br />
=Packaging=<br />
<br />
==XQuery==<br />
<br />
If an XQuery file is specified as input for the install command, it will be parsed as XQuery library module. If the file can successfully be parsed, the module URI will be [[Java Bindings#URI Rewriting|rewritten]] to a file path and attached with the {{Code|.xqm}} file suffix, and the original file will possibly be renamed and copied to that path into the repository.<br />
<br />
'''Example:'''<br />
<br />
Installation (the original file will be copied to the {{Code|org/basex/modules/Hello}} subdirectory of the repository):<br />
<br />
REPO INSTALL https://files.basex.org/modules/org/basex/modules/Hello/HelloWorld.xqm<br />
<br />
Importing the repository module:<br />
<br />
<pre lang='xquery'><br />
import module namespace m = 'http://basex.org/modules/Hello';<br />
m:hello("Universe")<br />
</pre><br />
<br />
==Java==<br />
<br />
For general notes on importing Java classes, please read the Java Bindings article on [[Java Bindings#Module_Imports|Module Imports]].<br />
<br />
Java archives (JARs) may contain one or more class files. One of them will be chosen as main class, which must be specified in a {{Code|Main-Class}} entry in the manifest file ({{Code|META-INF/MANIFEST.MF}}). This fully qualified Java class name will be rewritten to a file path by replacing the dots with slashes and attaching the {{Code|.jar}} file suffix, and the original file will be renamed and copied to that path into the repository.<br />
<br />
If the class will be imported in the prolog of the XQuery module, an instance of it will be created, and its public functions can then be addressed from XQuery. A class may extend the {{Code|QueryModule}} class to get access to the current query context and to be enriched by some helpful annotations (see [[Java_Bindings#Annotations|Annotations]]).<br />
<br />
'''Example:'''<br />
<br />
Structure of the <code>[https://files.basex.org/modules/org/basex/modules/Hello/HelloWorld.jar HelloWorld.jar]</code> archive:<br />
<br />
META-INF/<br />
MANIFEST.MF<br />
org/basex/modules/<br />
Hello.class<br />
<br />
Contents of the file {{Code|MANIFEST.mf}} (the whitespaces are obligatory):<br />
<br />
Manifest-Version: 1.0<br />
Main-Class: org.basex.modules.Hello<br />
<br />
Contents of the file {{Code|Hello.java}} (comments removed):<br />
<br />
<pre lang="java"><br />
package org.basex.modules;<br />
public class Hello {<br />
public String hello(final String world) {<br />
return "Hello " + world;<br />
}<br />
}<br />
</pre><br />
<br />
Installation (the file will be copied to {{Code|org/basex/modules/Hello.jar}}):<br />
<br />
REPO INSTALL HelloWorld.jar<br />
<br />
XQuery file <code>[https://files.basex.org/modules/org/basex/modules/Hello/HelloUniverse.xq HelloUniverse.xq]</code> (same as above):<br />
<br />
<pre lang='xquery'><br />
import module namespace m = 'http://basex.org/modules/Hello';<br />
m:hello("Universe")<br />
</pre><br />
<br />
After having installed the module, all of the following URIs can be used in XQuery to import this module or call its functions (see [[Java Bindings#URI Rewriting|URI Rewriting]] for more information):<br />
<br />
<nowiki>http://basex.org/modules/Hello</nowiki><br />
org/basex/modules/Hello<br />
org.basex.modules.Hello<br />
<br />
===Additional Libraries===<br />
<br />
A Java class may depend on additional libraries. The dependencies can be resolved by creating a fat JAR file, i.e., extracting all files of the library archives and producing a single, flat JAR package.<br />
<br />
Another solution is to copy the libraries into a {{Code|lib}} directory of the JAR package. When the package is installed, the additional library archives will be extracted and copied to a hidden subdirectory in the repository. If the package is deleted, the hidden subdirectory will be removed as well.<br />
<br />
; Examplary contents of {{Code|Image.jar}}<br />
<br />
lib/<br />
Images.jar<br />
META-INF/<br />
MANIFEST.MF<br />
org/basex/modules/<br />
Image.class<br />
<br />
; Directory structure of the repository directory after installing the package<br />
<br />
org/basex/modules/<br />
Image.class<br />
.Images/<br />
Images.jar<br />
<br />
==Combined==<br />
<br />
It makes sense to combine the advantages of XQuery and Java packages:<br />
<br />
* Instead of directly calling Java code, a wrapper module can be provided. This module contains functions that invoke the Java functions.<br />
* These functions can be strictly typed. This reduces the danger of erroneous or unexpected conversions between XQuery and Java code.<br />
* In addition, the entry functions can have properly maintained XQuery comments.<br />
<br />
XQuery and Java can be combined as follows:<br />
<br />
* First, a JAR package is created (as described above).<br />
* A new XQuery wrapper module is created, which is named identically to the Java main class.<br />
* The URL of the {{Code|import module}} statement in the wrapper module must start with the {{Code|java:}} prefix.<br />
* The finalized XQuery module must be copied into the JAR file, and placed in the same directory as the Java main class.<br />
<br />
If the resulting JAR file is installed, the embedded XQuery module will be extracted, and will be called first if the module will be imported.<br />
<br />
; Main Module {{Code|hello-universe.xq}}:<br />
<br />
<pre lang='xquery'><br />
import module namespace m = 'http://basex.org/modules/Hello';<br />
m:hello("Universe")<br />
</pre><br />
<br />
; Wrapper Module {{Code|Hello.xqm}}:<br />
<br />
<pre lang='xquery'><br />
module namespace hello = 'http://basex.org/modules/Hello';<br />
<br />
(: Import JAR file :)<br />
import module namespace java = 'java:org.basex.modules.Hello';<br />
<br />
(:~<br />
: Say hello to someone.<br />
: @param $world the one to be greeted<br />
: @return welcome string<br />
:)<br />
declare function hello:hello(<br />
$world as xs:string<br />
) as xs:string {<br />
java:hello($world)<br />
};<br />
</pre><br />
<br />
; Java class {{Code|Hello.java}}:<br />
<br />
<pre lang="java"><br />
package org.basex.modules;<br />
<br />
public class Hello {<br />
public String hello(final String world) {<br />
return "Hello " + world;<br />
}<br />
}<br />
</pre><br />
<br />
If the JAR file is installed, {{Code|Combined}} will be displayed as type:<br />
<br />
REPO INSTALL https://files.basex.org/modules/org/basex/modules/Hello.jar<br />
REPO LIST<br />
<br />
Name Version Type Path<br />
-----------------------------------------------------------------------<br />
org.basex.modules.Hello - Combined org/basex/modules/Hello.xqm<br />
<br />
=EXPath Packaging=<br />
<br />
The [http://expath.org/spec/pkg EXPath specification] defines the structure of a .xar archive. The package contains at its root a package descriptor named <code>expath-pkg.xml</code>. This descriptor presents some metadata about the package as well as the libraries which it contains and their dependencies on other libraries or processors.<br />
<br />
==XQuery==<br />
<br />
Apart from the package descriptor, a {{Code|.xar}} archive contains a directory which includes the actual XQuery modules. For example, the [https://files.basex.org/modules/expath/functx-1.0.xar FunctX XAR archive] is packaged as follows:<br />
<br />
<pre><br />
expath-pkg.xml<br />
functx/<br />
functx.xql<br />
functx.xsl<br />
</pre><br />
<br />
==Java==<br />
<br />
If you want to package an EXPath archive with Java code, some additional requirements have to be fulfilled:<br />
<br />
* Apart from the package descriptor <code>expath-pkg.xml</code>, the package has to contain a descriptor file at its root, defining the included jars and the binary names of their public classes. It must be named <code>basex.xml</code> and must conform to the following structure:<br />
<br />
<pre lang="xml"><br />
<package xmlns="http://expath.org/ns/pkg"><br />
<jar>...</jar><br />
....<br />
<class>...</class><br />
<class>...</class><br />
....<br />
</package><br />
</pre><br />
<br />
* The jar file itself along with an XQuery file defining wrapper functions around the java methods has to reside in the module directory. The following example illustrates how java methods are wrapped with XQuery functions:<br />
<br />
'''Example:'''<br />
<br />
Suppose we have a simple class <code>Printer</code> having just one public method <code>print()</code>:<br />
<br />
<pre lang="java"><br />
package test;<br />
<br />
public final class Printer {<br />
public String print(final String s) {<br />
return new Writer(s).write();<br />
}<br />
}<br />
</pre><br />
<br />
We want to extend BaseX with this class and use its method. In order to make this possible we have to define an XQuery function which wraps the <code>print</code> method of our class. This can be done in the following way:<br />
<br />
<pre lang='xquery'><br />
import module namespace j="http://basex.org/lib/testJar";<br />
<br />
declare namespace p="java:test.Printer";<br />
<br />
declare function j:print($str as xs:string) as xs:string {<br />
let $printer := p:new()<br />
return p:print($printer, $str)<br />
};<br />
</pre><br />
<br />
As it can be seen, the class {{Code|Printer}} is declared with its binary name as a namespace prefixed with "java" and the XQuery function is implemented using the [http://docs.basex.org/wiki/Java_Bindings Java Bindings] offered by BaseX.<br />
<br />
On our [https://files.basex.org/modules/ file server], you can find some example libraries packaged as XML archives (xar files). You can use them to try our packaging API or just as a reference for creating your own packages.<br />
<br />
=Performance=<br />
<br />
Importing XQuery modules that are located in the repository is just as fast as importing any other modules. Modules that are imported several times in a project will only be compiled once.<br />
<br />
Imported Java archives will be dynamically added to the classpath and unregistered after query execution. This requires some constant overhead and may lead to unexpected effects in scenarios with highly concurrent read operations. If you want to get optimal performance, it is recommendable to move your JAR files into the {{Code|lib/custom}} directory of BaseX. This way, the archive will be added to the classpath if BaseX is started. If you have installed a [[#Combined|Combined Package]], you can simply keep your XQuery module in the repository, and the Java classes will be automatically detected.<br />
<br />
=Changelog=<br />
<br />
;Version 9.0<br />
<br />
* Added: [[#Combined|Combined]] XQuery and Java packages<br />
* Added: [[#Additional Libraries|Additional Libraries]]<br />
<br />
;Version 7.2.1<br />
<br />
* Updated: [[#Installation|Installation]]: existing packages will be replaced without raising an error<br />
* Updated: [[#Removal|Removal]]: remove specific version of a package<br />
<br />
;Version 7.1<br />
<br />
* Added: [[Repository Module]]<br />
<br />
;Version 7.0<br />
<br />
* Added: [[#EXPath Packaging|EXPath Packaging]]</div>CGhttps://docs.basex.org/index.php?title=Git&diff=16910Git2023-12-01T16:39:54Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This page is part of the [[Developer Section]]. It describes how to use [https://git-scm.com/ git] to manage the BaseX sources.<br />
<br />
==Using Git to contribute to BaseX ==<br />
<br />
Our team uses git and [https://github.com GitHub] to manage the source code. All team members have read/write access to the repository, and external contributors are welcome to fork the project. <br />
<br />
Git makes it easy to retain a full copy of the repository for yourself. To get started and running, simply ''fork'' BaseX:<br />
<br />
# Head over to https://github.com and create an account<br />
# Fork https://github.com/BaseXdb/basex, so you have a version on your own<br />
# The forked project can then be cloned on your local machine, and changes can be pushed back to your remote repository<br />
<br />
==Using Git & Eclipse==<br />
<br />
===Clone===<br />
<br />
* In the '''Package Explorer''' to the left, use right-click and choose '''Import...'''<br />
* Select '''Projects from Git''' and click '''Next'''<br />
* Choose the '''Clone''' option to create a local copy of the remote repository. This copy will include the full project history<br />
* Copy & Paste the GitHub URI in the Location field. If you want to use SSH, make sure you provided GitHub with your public key to allow write-access. If in doubt, use the https URI and authenticate yourself with your GitHub credentials. The read-only URI of the repository is {{Code|https://github.com/BaseXdb/basex.git}}.<br />
* Select the master branch (or arbitrary branches you like) <br />
* Now choose a location where the local repository is stored: Create &lt;workspace&gt;'''/repos/BaseX''' and click "'''Finish'''". <br />
<br />
<gallery><br />
File:Git01.png|Package Explorer<br />
File:Git02.png|Projects from Git<br />
File:Git03.png|Clone<br />
File:Git04.png|GitHub URI<br />
File:Git05.png|Select Branch<br />
File:Git06.png|Location<br />
</gallery><br />
<br />
===Create the project ===<br />
<br />
* Select our newly cloned repository and click Next<br />
* Select "'''Import Existing Projects'''" and depending on your Eclipse version enable automatic sharing. More recent versions will not offer this feature as sharing is enabled by default.<br />
* Click next to select the Project to import<br />
* Check "basex" to check out and click finish<br />
* You are now ready to contribute. <br />
<br />
<gallery><br />
File:Git07.png|Select Repository<br />
File:Git08.png|Import existing Projects<br />
File:Git09.png|Select basex Project<br />
</gallery><br />
<br />
===EGit & SSH===<br />
<br />
The Eclipse git plugin uses the [http://www.jcraft.com/jsch/ JSch] library, which had [https://bugs.eclipse.org/bugs/show_bug.cgi?id=326526 problems with RSA SSH keys] in Linux and possibly other platforms. If the problem persists, the path to the native SSH executable can be assigned to the {{Code|GIT_SSH}} variable.<br />
<br />
==Using Git on Command-Line==<br />
<br />
'''Note''': this is not intended to be a complete git reference; its purpose is to quickly introduce BaseX developers to the most commonly used git commands in the context of the BaseX project.<br />
<br />
===Preparation===<br />
<br />
# Create a GitHub user account: [https://github.com/signup/free here] (your github username will be referenced as $username)<br />
# Set up SSH access to GitHub as described [https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh here]<br />
# Create a fork of one of the BaseXdb projects (it will be referenced as $project)<br />
# Choose a directory where the project will be created and make it your working directory (e. g. /home/user/myprojects)<br />
<br />
===Clone Repository===<br />
<br />
<pre lang="shell"><br />
$ git clone git@github.com:$username/$project.git<br />
Cloning into $project...<br />
Enter passphrase for key '/home/user/.ssh/id_rsa': <br />
...<br />
<br />
$ ls -d -1 $PWD/*<br />
/home/user/myprojects/$project<br />
</pre><br />
Note that git automatically creates a directory where the repository content will be checked out.<br />
<br />
===List Remote Repositories===<br />
<br />
<pre lang="shell"><br />
$ git remote -v<br />
origin git@github.com:$username/$project.git (fetch)<br />
origin git@github.com:$username/$project.git (push)<br />
</pre><br />
Currently, there is only one remote repository; it is automatically registered during the clone operation. Git remembers this repository as the default repository for push/pull operations.<br />
<br />
===List Local Changes===<br />
<br />
After some files have been changed locally, the changes can be seen as follows:<br />
<pre lang="shell"><br />
$ git diff<br />
diff --git a/readme.txt b/readme.txt<br />
index fabaeaa..cd09568 100644<br />
--- a/readme.txt<br />
+++ b/readme.txt<br />
@@ -49,6 +49,10 @@ ADDING CHECKSTYLE --------------------------------------------------------------<br />
- Enter the URL: http://eclipse-cs.sourceforge.net/update<br />
- Follow the installation procedure and restart Eclipse<br />
<br />
+USING GIT ----------------------------------------------------------------------<br />
<br />
Any kind of feedback is welcome; please check out the online documentation at<br />
</pre><br />
<br />
===Commit to Local Repository===<br />
<br />
'''Note''': this commit operation does '''not''' commit into the remote repository!<br />
<br />
First, it is needed to select the modified files which should be committed:<br />
<pre lang="shell"><br />
$ git add readme.txt<br />
</pre><br />
<br />
Then perform the actual commit:<br />
<pre lang="shell"><br />
$ git commit<br />
[master 0fde1fb] Added TODO in section "USING GIT"<br />
1 files changed, 4 insertions(+), 0 deletions(-)<br />
</pre><br />
Before executing the actual commit, git will open the default shell editor (determined using the $EDITOR variable, usually vi) to enter a message describing the commit changes.<br />
<br />
Alternative way is to commit all changed files, i. e. it is not needed to explicitly add the changed files:<br />
<pre lang="shell"><br />
$ git commit -a<br />
[master 0fde1fb] Added TODO in section "USING GIT"<br />
1 files changed, 4 insertions(+), 0 deletions(-)<br />
</pre><br />
<br />
===Pushing Changes to Remote Repository===<br />
<br />
<pre lang="shell"><br />
$ git push<br />
Enter passphrase for key '/home/user/.ssh/id_rsa': <br />
Everything up-to-date<br />
</pre><br />
<br />
===Pulling Changes from Remote Repository===<br />
<br />
<pre lang="shell"><br />
$ git pull<br />
Enter passphrase for key '/home/user/.ssh/id_rsa': <br />
Already up-to-date.<br />
</pre><br />
<br />
===Add Upstream Repository===<br />
<br />
The upstream repository is the one from which the BaseX releases are made and the one from which the personal repository was forked.<br />
<br />
<pre lang="shell"><br />
$ git remote add upstream git@github.com:BaseXdb/$project.git<br />
<br />
$ git remote -v<br />
origin git@github.com:$username/$project.git (fetch)<br />
origin git@github.com:$username/$project.git (push)<br />
upstream git@github.com:BaseXdb/$project.git (fetch)<br />
upstream git@github.com:BaseXdb/$project.git (push)<br />
</pre><br />
<br />
===Pulling Changes from Upstream to Local Repository===<br />
<br />
When some changes are made in the upstream repository, they can be pulled to the local repository as follows:<br />
<br />
<pre lang="shell"><br />
$ git pull upstream master<br />
Enter passphrase for key '/home/user/.ssh/id_rsa': <br />
From github.com:BaseXdb/$project<br />
* branch master -> FETCH_HEAD<br />
Already up-to-date.<br />
</pre><br />
<br />
The changes can then be pushed in the personal repository:<br />
<pre lang="shell"><br />
$ git push<br />
</pre><br />
<br />
Check out the links at the end of the page for more git options.<br />
</td></tr></table><br />
<br />
===Developing a new feature or bug fix===<br />
<br />
It is always a good idea to create a new branch for a new feature or a big fix you are working on. So first, let's make sure you have the most up-to-date source code. We assume, that you added BaseX as upstream repository as described above and you are currently in the ''master'' branch:<br />
<br />
<pre lang="shell"><br />
$ git pull upstream master<br />
</pre><br />
<br />
Now, we create a new branch, based on the master branch<br />
<br />
<pre lang="shell"><br />
$ git checkout -b new-feature<br />
Switched to a new branch 'new-feature'<br />
</pre><br />
<br />
Your are now automatically switched to the ''new-feature'' branch. Now you can make all your changes in one or several commits. You can commit all changes using<br />
<br />
<pre lang="shell"><br />
$ git commit -a<br />
</pre><br />
<br />
Now, you want to push these changes to the repository on GitHub. Remember, that up to now your changes just reside on your local drive, so now you want to push it to your remote fork of BaseX. Simply do:<br />
<br />
<pre lang="shell"><br />
$ git push origin new-feature<br />
Counting objects: 318, done.<br />
Delta compression using up to 4 threads.<br />
Compressing objects: 100% (107/107), done.<br />
Writing objects: 100% (154/154), 22.96 KiB | 0 bytes/s, done.<br />
Total 154 (delta 93), reused 81 (delta 26)<br />
To git@github.com:$username/basex.git<br />
* [new branch] new-feature -> new-feature<br />
</pre><br />
<br />
You can now use your web browser and go to your fork of BaseX. You will see the following message:<br />
<br />
[[File:Git11.png]]<br />
<br />
You can now click the "Compare & pull request" button. You can now review the changes you are going to push. <br />
<br />
'''Please review them carefully. Also, please give a meaningful comment so we can quickly determine what your changes are doing.''' After clicking the "Create Pull request" button you are done and we will review your changes and either merge the pull request or get back to you.<br />
<br />
==Links==<br />
<br />
* [https://docs.github.com/en/github/getting-started-with-github/set-up-git GitHub: git Installation Guide]<br />
* [https://help.github.com/ Comprehensive Getting Starting Guide on GitHub]<br />
* [https://book.git-scm.com/ The git book]</div>CGhttps://docs.basex.org/index.php?title=CSV_Module&diff=16909CSV Module2023-12-01T16:39:53Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This [[Module Library|XQuery Module]] contains a single function to parse CSV input. [https://en.wikipedia.org/wiki/Comma-separated_values CSV] (comma-separated values) is a popular representation for tabular data, exported e. g. from Excel.<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/csv</nowiki></code> namespace, which is statically bound to the {{Code|csv}} prefix.<br/><br />
<br />
==Conversion Formats==<br />
<br />
===XML: Direct, Attributes===<br />
<br />
If the {{Code|direct}} or {{Code|attributes}} format is chosen, a CSV string is converted to XML:<br />
<br />
* The resulting XML document has a {{Code|csv}} root element.<br />
* Rows are represented via {{Code|record}} elements.<br />
* Fields are represented via {{Code|entry}} elements. The value of a field is represented as text node.<br />
* If the {{Code|header}} option is set to {{Code|true}}, the first text line is parsed as table header:<br />
** If {{Code|format}} is set to {{Code|direct}}, the field names are encoded, as described in the [[Conversion Module#Keys|Conversion Module]], and used as element names.<br />
** Otherwise, if {{Code|format}} is {{Code|attributes}}, the field names will be stored in name attributes.<br />
<br />
'''A little advice''': in the Database Creation dialog of the GUI, if you select CSV Parsing and switch to the ''Parsing'' tab, you can see the effects of some of the conversion options.<br />
<br />
===XQuery===<br />
<br />
With the {{Code|xquery}} format, CSV records are converted to a sequence of arrays:<br />
<br />
* The resulting value will be a map with a {{Code|records}} entry and an optional {{Code|names}} entry.<br />
* Records are organized as a sequence of arrays. An array contains the entries of a single record.<br />
* The {{Code|names}} entry contains an array with the column names. It is generated if the {{Code|header}} option is set to {{Code|true}}.<br />
<br />
The CSV map can e.g. be accessed as follows:<br />
<br />
* <code>$csv?records[5]</code> returns all entries of the 5th record (row)<br />
* <code>$csv?records?(2)</code> returns all entries of the 2nd field (column)<br />
* <code>$csv?names?*</code> returns the names of all fields (if available)<br />
* Return enumerated strings for all records:<br />
<pre lang='xquery'><br />
for $record at $pos in $csv?records<br />
return $pos || ". " || string-join($record?*, ', ')<br />
</pre><br />
<br />
The resulting representation consumes less memory than XML-based formats, and values can be directly accessed without conversion. Thus, it is recommendable for very large inputs and for efficient ad-hoc processing.<br />
<br />
==Options==<br />
<br />
In the following table, all available options are listed. The Excel column lists recommended options for data that is processed with Excel or Open/Libre Office.<br />
<br />
{| class="wikitable sortable" width="100%"<br />
|- valign="top"<br />
! Option<br />
! Description<br />
! Allowed<br />
! Default<br />
! Direction<br />
! Excel<br />
|- valign="top"<br />
| {{Code|separator}}<br />
| Defines the character which separates the values of a single record.<br />
| {{Code|comma}}, {{Code|semicolon}}, {{Code|colon}}, {{Code|tab}}, {{Code|space}} or a ''single character''<br />
| {{Code|comma}}<br />
| ''parse'', ''serialize''<br />
| {{Code|semicolon}} or {{Code|comma}}, depending on the region<br />
|- valign="top"<br />
| {{Code|header}}<br />
| Indicates if the first line of the parsed or serialized CSV data is a table header.<br />
| {{Code|yes}}, {{Code|no}}<br />
| {{Code|no}}<br />
| ''parse'', ''serialize''<br />
|<br />
|- valign="top"<br />
| {{Code|format}}<br />
| Specifies the format for converting CSV data ([[#Conversion Formats|see above]]).<br />
| {{Code|direct}}, {{Code|attributes}}, {{Code|xquery}}<br />
| {{Code|direct}}<br />
| ''parse'', ''serialize''<br />
|<br />
|- valign="top"<br />
| {{Code|lax}}<br />
| Specifies if [[Conversion Module#Keys|lax conversion rules]] are used to convert QNames to JSON names.<br />
| {{Code|yes}}, {{Code|no}}<br />
| {{Code|yes}}<br />
| ''parse'', ''serialize''<br />
| {{Code|no}}<br />
|- valign="top"<br />
| {{Code|quotes}}<br />
| Specifies how quotes are parsed:<br />
* Parsing: If the option is enabled, quotes at the start and end of a value will be treated as control characters. Separators and newlines within the quotes will be adopted without change.<br />
* Serialization: If the option is enabled, the value will be wrapped with quotes if it contains characters that might be treated as control characters. A quote character in the value will be encoded according to the rules of the {{Code|backslashes}} option.<br />
| {{Code|yes}}, {{Code|no}}<br />
| {{Code|yes}}<br />
| ''parse'', ''serialize''<br />
| {{Code|yes}}<br />
|- valign="top"<br />
| {{Code|backslashes}}<br />
| Specifies how quotes and other characters are escaped:<br />
* Parsing: If the option is enabled, {{Code|\r}}, {{Code|n}} and {{Code|\t}} will be replaced with the corresponding control characters. All other escaped characters will be adopted as literals (e.g.: {{Code|\"}} → {{Code|"}}). If the option is disabled, two consecutive quotes will be replaced with a single quote (unless {{Code|quotes}} is enabled and the quote is the first or last character of a value).<br />
* Serialization: If the option is enabled, {{Code|\r}}, {{Code|n}}, {{Code|\t}}, {{Code|"}} and the separator character will be encoded with a backslash. If the option is disabled, quotes will be duplicated.<br />
| {{Code|yes}}, {{Code|no}}<br />
| {{Code|no}}<br />
| ''parse'', ''serialize''<br />
| {{Code|no}}<br />
|- valign="top"<br />
| {{Code|skip-empty}}<br />
| {{Announce|Version 11:}} Indicates if empty fields are included in the result. Only gets effective for the formats {{Code|direct}} or {{Code|attribute}}, and if the {{Code|header}} option is enabled. Please note that if this option is used and the data is serialized again, the resulting CSV header may be incomplete.<br />
| {{Code|yes}}, {{Code|no}}<br />
| {{Code|no}}<br />
| ''parse''<br />
| {{Code|no}}<br />
|- valign="top"<br />
| {{Code|allow}}<br />
| In Excel, a value will be evaluated if it starts with the character {{Code|-}}, {{Code|+}}, <code>=</code>, {{Code|@}}, {{Code|\t}} or {{Code|\r}}. A regular expression can be specified to reject data that will be handled differently than expected by an application, or that may be malicious (see https://owasp.org/www-community/attacks/CSV_Injection for more details).<br />
| ''string''<br />
|<br />
| ''serialize''<br />
| <code><nowiki>[^-+=\t\r].*|[-+]\d*([,.]\d+)?</nowiki></code><br />
|}<br />
<br />
=Functions=<br />
<br />
==csv:doc==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>csv:doc(<br />
$href as xs:string?,<br />
$options as map(*)? := map { }<br />
) as item()?</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Fetches the CSV document referred to by the given {{Code|$href}} and converts it to an XQuery value. The {{Code|$options}} argument can be used to control the way the input is converted.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|parse|#Errors}} the specified input cannot be parsed as CSV document.<br/>{{Error|options|#Errors}} the specified options are conflicting.<br />
|}<br />
<br />
==csv:parse==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>csv:parse(<br />
$value as xs:string?,<br />
$options as map(*)? := map { }<br />
) as item()?</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Converts the CSV {{Code|$value}} to an XQuery value. The {{Code|$options}} argument can be used to control the way the input is converted.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|parse|#Errors}} the specified input cannot be parsed as CSV document.<br />
|}<br />
<br />
==csv:serialize==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>csv:serialize(<br />
$input as item()?,<br />
$options as map(*)? := map { }<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Serializes the specified {{Code|$input}} as CSV, using the specified {{Code|$options}}, and returns the result as string.<br />
Values can also be serialized as CSV with the standard [[Serialization]] feature of XQuery:<br />
* The parameter {{Code|method}} needs to be set to {{Code|csv}}, and<br />
* the options presented in this article need to be assigned to the {{Code|csv}} parameter.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|serialize|#Errors}} the input cannot be serialized.<br />
|}<br />
<br />
=Examples=<br />
<br />
'''Example 1:''' Converts CSV data to XML, interpreting the first row as table header:<br />
<br />
'''Input''' {{Code|addressbook.csv}}:<br />
<pre lang="xml"><br />
Name,First Name,Address,City<br />
Huber,Sepp,Hauptstraße 13,93547 Hintertupfing<br />
</pre><br />
<br />
'''Query:'''<br />
<pre lang='xquery'><br />
let $text := file:read-text('addressbook.csv')<br />
return csv:parse($text, map { 'header': true() })<br />
</pre><br />
<br />
'''Result:'''<br />
<pre lang="xml"><br />
<csv><br />
<record><br />
<Name>Huber</Name><br />
<First_Name>Sepp</First_Name><br />
<Address>Hauptstraße 13</Address><br />
<City>93547 Hintertupfing</City><br />
</record><br />
</csv><br />
</pre><br />
'''<br />
<br />
'''Example 2:''' Converts some CSV data to XML and back, and checks if the input and output are equal. The expected result is {{Code|true}}:<br />
<br />
'''Query:'''<br />
<pre lang='xquery'><br />
let $options := map { 'lax': false() }<br />
let $input := file:read-text('some-data.csv')<br />
let $output := $input => csv:parse($options) => csv:serialize($options)<br />
return $input eq $output<br />
</pre><br />
<br />
'''Example 3:''' Converts CSV data to XQuery and returns distinct column values:<br />
<br />
'''Query:'''<br />
<pre lang='xquery'><br />
let $text := ``[Name,City<br />
Jack,Chicago<br />
Jack,Washington<br />
John,New York<br />
]``<br />
let $options := map { 'format': 'xquery', 'header': true() }<br />
let $csv := csv:parse($text, $options)<br />
return (<br />
'Distinct values:',<br />
let $records := $csv('records')<br />
for $name at $pos in $csv('names')?*<br />
let $values := $records?($pos)<br />
return (<br />
'* ' || $name || ': ' || string-join(distinct-values($values), ', ')<br />
)<br />
)<br />
</pre><br />
<br />
'''Result:'''<br />
<pre lang='xquery'><br />
Distinct values:<br />
* Name: Jack, John<br />
* City: Chicago, Washington, New York<br />
</pre><br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
|Description<br />
|- valign="top"<br />
|{{Code|parse}}<br />
| The input cannot be parsed.<br />
|- valign="top"<br />
|{{Code|serialize}}<br />
| The node cannot be serialized.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 11<br />
* Added: [[#Options|Options]]: <code>skip-empty</code> option.<br />
<br />
;Version 9.7<br />
* Added: [[#Options|Options]]: <code>allow</code> option.<br />
<br />
;Version 9.4<br />
* Added: {{Function||csv:doc}}<br />
<br />
; Version 9.1<br />
* Updated: {{Function||csv:parse}} can be called with empty sequence.<br />
<br />
;Version 9.0<br />
<br />
* Added: {{Code|xquery}} option<br />
* Removed: {{Code|map}} option<br />
* Updated: error codes updated; errors now use the module namespace<br />
<br />
;Version 8.6<br />
<br />
* Updated: [[#Options|Options]]: improved Excel compatibility<br />
<br />
;Version 8.0<br />
<br />
* Added: {{Code|backslashes}} option<br />
<br />
;Version 7.8<br />
<br />
* Updated: {{Function||csv:parse}} now returns a document node instead of an element, or an XQuery map if {{Code|format}} is set to {{Code|map}}.<br />
* Added: {{Code|format}} and {{Code|lax}} options<br />
<br />
The module was introduced with Version 7.7.2.</div>CGhttps://docs.basex.org/index.php?title=Inspection_Module&diff=16908Inspection Module2023-12-01T16:39:51Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This [[Module Library|XQuery Module]] contains functions for extracting internal information about modules and functions and generating documentation.<br />
<br />
With {{Announce|Version 11}}, {{Code|inspect:function-annotations}} has been removed as it is now available in the official specification as [https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-function-annotations <code>fn:function-annotations</code>].<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/inspect</nowiki></code> namespace, which is statically bound to the {{Code|inspect}} prefix.<br/><br />
xqDoc document instances are assigned to the <code><nowiki>http://www.xqdoc.org/1.0</nowiki></code> namespace, which is statically bound to the {{Code|xqdoc}} prefix.<br />
<br />
=Reflection=<br />
<br />
==inspect:functions==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>inspect:functions(<br />
$href as xs:string := ()<br />
) as function(*)*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns function items for all user-defined functions (both public and private) that are known in the current query context. If an {{Code|$href}} value is specified, the specified resource will be retrieved as a string and compiled, and its functions will be added to the query context and returned to the user. A relative URI will be resolved against the static base URI of the query.<br />
|- valign="top"<br />
|'''Examples'''<br />
|Invokes the declared functions and returns their values:<br/><br />
<pre lang='xquery'><br />
declare %private function local:one() { 12 };<br />
declare %private function local:two() { 34 };<br />
for $f in inspect:functions() return $f()<br />
</pre><br />
Compiles all functions in {{Code|code.xqm}} and invokes the function named {{Code|run}}:<br />
<pre lang='xquery'><br />
let $uri := 'code.xqm'<br />
let $name := 'run'<br />
for $f in inspect:functions($uri)<br />
where local-name-from-QName(function-name($f)) = $name<br />
return $f()<br />
</pre><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|parse|#Errors}} Error while parsing a module.<br />
|}<br />
<br />
==inspect:static-context==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>inspect:static-context(<br />
$function as function(*)?,<br />
$name as xs:string<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns a component of the [https://www.w3.org/TR/xquery-31/#dt-static-context static context] of a {{Code|$function}} with the specified {{Code|$name}}. If no function is supplied, the current static context is considered.<br/>The following components can be requested:<br />
* {{Code|base-uri}}: Static base URI.<br />
* {{Code|namespaces}}: Prefix/URI map with all statically known namespaces.<br />
* {{Code|element-namespace}}: Default element/type namespace URI, or an empty sequence if it is absent.<br />
* {{Code|function-namespace}}: Default function namespace URI, or an empty sequence if it is absent.<br />
* {{Code|collation}}: URI of the default collation.<br />
* {{Code|ordering}}: Ordering mode ({{Code|ordered}}/{{Code|unordered}})<br />
* {{Code|construction}}: Construction mode ({{Code|preserve}}/{{Code|strip}})<br />
* {{Code|default-order-empty}}: Default order for empty sequences ({{Code|greatest}}/{{Code|least}})<br />
* {{Code|boundary-space}}: Boundary-space policy ({{Code|preserve}}/{{Code|strip}})<br />
* {{Code|copy-namespaces}}: Copy-namespaces mode ({{Code|inherit}}/{{Code|no-inherit}}, {{Code|preserve}}/{{Code|no-preserve}})<br />
* {{Code|decimal-formats}}: Nested map with all statically known decimal formats<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* Returns the static base URI (same as {{Code|static-base-uri()}}):<br />
<pre lang='xquery'><br />
inspect:static-context((), 'base-uri')<br />
</pre><br />
* Returns a map with all namespaces that are statically known in the module of the specified function:<br />
<pre lang='xquery'><br />
import module namespace data = 'data.xqm';<br />
inspect:static-context(data:get#1, 'namespaces')<br />
</pre><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|unknown|#Errors}} The specified component does not exist.<br />
|}<br />
<br />
=Documentation=<br />
<br />
==inspect:type==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>inspect:type(<br />
$input as item()*,<br />
$options as map(*)? := map { }<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns a string representation of the type of a the given {{Code|$input}}:<br />
* The string includes the occurrence indicator.<br />
* The type of functions and nodes may be stricter than the returned type.<br />
* For type checking, the standard expressions {{Code|typeswitch}} and {{Code|instance of}} should be used instead.<br />
The following {{Code|$options}} are available:<br />
* {{Code|item}}: If enabled, only the item type is returned and the occurrence indicator is omitted. The default is {{Code|false()}}.<br />
* {{Code|mode}}: If {{Code|value}} is specified, the assigned type of the result value is returned. With {{Code|expression}} the type of the input expression is returned (please note that the original expression may already have been rewritten at compile-time). With {{Code|computed}}, the exact value is computed at runtime, based on the expression and the result value. The default is {{Code|computed}}.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>inspect:type((<a/>, <a/>))</code> yields <code>element(a)+</code><br />
* <code>inspect:type(map { 'a': (1, 2)[. = 1] })</code> yields <code>map(xs:string, xs:integer)</code><br />
* <code>inspect:type(1 to 100, map { 'item': true() })</code> yields <code>xs:integer</code><br />
|}<br />
<br />
==inspect:function==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>inspect:function(<br />
$function as function(*)<br />
) as element(function)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Inspects the specified {{Code|$function}} and returns an element that describes its structure. The output of this function is similar to eXist-db’s [https://exist-db.org/exist/apps/fundocs/view.html?uri=http://exist-db.org/xquery/inspection&location=java:org.exist.xquery.functions.inspect.InspectionModule inspect:inspect-function] function.<br />
|- valign="top"<br />
|'''Examples'''<br />
|The query {{Code|inspect:function(count#1)}} yields:<br />
<pre lang="xml"><br />
<function name="count" uri="http://www.w3.org/2005/xpath-functions" external="false"><br />
<argument type="item()" occurrence="*"/><br />
<return type="xs:integer"/><br />
</function><br />
</pre><br />
The function…<br />
<pre lang='xquery'><br />
(:~<br />
: This function simply returns the specified integer.<br />
: @param $number number to return<br />
: @return specified number<br />
:)<br />
declare %private function local:same($number as xs:integer) as xs:integer {<br />
$number<br />
};<br />
</pre><br />
…is represented by {{Code|inspect:function(local:same#1)}} as…<br />
<pre lang="xml"><br />
<function name="local:same" uri="http://www.w3.org/2005/xquery-local-functions" external="false"><br />
<argument type="xs:integer" name="number">number to return</argument><br />
<annotation name="private" uri="http://www.w3.org/2012/xquery"/><br />
<description>This function simply returns the specified integer.</description><br />
<return type="xs:integer">specified number</return><br />
</function><br />
</pre><br />
|}<br />
<br />
==inspect:context==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>inspect:context() as element(context)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Generates an element that describes all variables and functions in the current query context.<br />
|- valign="top"<br />
| '''Examples'''<br />
|Evaluate all user-defined functions with zero arguments in the query context:<br/><br />
<pre lang='xquery'><br />
inspect:context()/function ! function-lookup(QName(@uri, @name), 0) ! .()<br />
</pre><br />
Return the names of all private functions in the current context:<br />
<pre lang='xquery'><br />
for $f in inspect:context()/function<br />
where $f/annotation/@name = 'private'<br />
return $f/@name/string()<br />
</pre><br />
|}<br />
<br />
==inspect:module==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>inspect:module(<br />
$uri as xs:string<br />
) as element(module)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Retrieves the resource located at the specified {{Code|$uri}}, parses it as XQuery module, and generates an element that describes the module's structure. A relative URI will be resolved against the static base URI of the query.<br />
|- valign="top"<br />
|'''Examples'''<br />
|An example is [[#Examples|shown below]].<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|parse|#Errors}} Error while parsing a module.<br />
|}<br />
<br />
==inspect:xqdoc==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>inspect:xqdoc(<br />
$uri as xs:string<br />
) as element(xqdoc:xqdoc)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Retrieves the resource located at the specified {{Code|$uri}}, parses it as XQuery module, and generates an xqDoc element. A relative URI will be resolved against the static base URI of the query.<br/>[http://xqdoc.org/ xqDoc] provides a simple vendor-neutral solution for generating documentation from XQuery modules. The documentation conventions have been inspired by the JavaDoc standard. Documentation comments begin with {{Code|(:~}} and end with {{Code|:)}}, and tags start with {{Code|@}}. xqDoc comments can be specified for main and library modules and variable and function declarations.<br/><br />
We have slightly extended the xqDoc conventions to do justice to more recent versions of XQuery (Schema: [https://files.basex.org/etc/xqdoc-1.1.30052013.xsd xqdoc-1.1.30052013.xsd]):<br/><br />
* an {{Code|<xqdoc:annotations/>}} node is added to each variable or function that uses annotations. The xqdoc:annotation child nodes may have additional {{Code|xqdoc:literal}} elements with {{Code|type}} attributes (xs:string, xs:integer, xs:decimal, xs:double) and values.<br />
* a single {{Code|<xqdoc:namespaces/>}} node is added to the root element, which summarizes all prefixes and namespace URIs used or declared in the module.<br />
* name and type elements are added to variables.<br />
|- valign="top"<br />
|'''Examples'''<br />
|An example is [[#Examples|shown below]].<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|parse|#Errors}} Error while parsing a module.<br />
|}<br />
<br />
=Examples=<br />
<br />
This is the {{Code|sample.xqm}} library module:<br />
<br />
<pre lang='xquery'><br />
(:~ <br />
: This module provides some sample functions to demonstrate<br />
: the features of the Inspection Module.<br />
:<br />
: @author BaseX Team<br />
: @see http://docs.basex.org/wiki/XQDoc_Module<br />
: @version 1.0<br />
:)<br />
module namespace samples = 'http://basex.org/modules/samples';<br />
<br />
(:~ This is a sample string. :)<br />
declare variable $samples:test-string as xs:string := 'this is a string';<br />
<br />
(:~<br />
: This function simply returns the specified integer.<br />
: @param $number number to return<br />
: @return specified number<br />
:)<br />
declare %private function samples:same($number as xs:integer) as xs:integer {<br />
$number<br />
};<br />
</pre><br />
<br />
If {{Code|inspect:module('sample.xqm')}} is run, the following output will be generated:<br />
<br />
<pre lang="xml"><br />
<module prefix="samples" uri="http://basex.org/modules/samples"><br />
<description>This module provides some sample functions to demonstrate<br />
the features of the Inspection Module.</description><br />
<author>BaseX Team</author><br />
<see>http://docs.basex.org/wiki/XQDoc_Module</see><br />
<version>1.0</version><br />
<variable name="samples:test-string" uri="http://basex.org/modules/samples" type="xs:string" external="false"><br />
<description>This is a sample string.</description><br />
</variable><br />
<function name="samples:same" uri="http://basex.org/modules/samples" external="false"><br />
<argument name="number" type="xs:integer">number to return</argument><br />
<annotation name="private" uri="http://www.w3.org/2012/xquery"/><br />
<description>This function simply returns the specified integer.</description><br />
<return type="xs:integer">specified number</return><br />
</function><br />
</module><br />
</pre><br />
<br />
The output looks as follows if {{Code|inspect:xqdoc('sample.xqm')}} is called:<br />
<br />
<pre lang="xml"><br />
<xqdoc:xqdoc xmlns:xqdoc="http://www.xqdoc.org/1.0"><br />
<xqdoc:control><br />
<xqdoc:date>2013-06-01T16:59:33.654+02:00</xqdoc:date><br />
<xqdoc:version>1.1</xqdoc:version><br />
</xqdoc:control><br />
<xqdoc:module type="library"><br />
<xqdoc:uri>http://basex.org/modules/samples</xqdoc:uri><br />
<xqdoc:name>sample.xqm</xqdoc:name><br />
<xqdoc:comment><br />
<xqdoc:description>This module provides some sample functions to demonstrate<br />
the features of the Inspection Module.</xqdoc:description><br />
<xqdoc:author>BaseX Team</xqdoc:author><br />
<xqdoc:see>http://docs.basex.org/wiki/XQDoc_Module</xqdoc:see><br />
<xqdoc:version>1.0</xqdoc:version><br />
</xqdoc:comment><br />
</xqdoc:module><br />
<xqdoc:namespaces><br />
<xqdoc:namespace prefix="samples" uri="http://basex.org/modules/samples"/><br />
</xqdoc:namespaces><br />
<xqdoc:imports/><br />
<xqdoc:variables><br />
<xqdoc:variable><br />
<xqdoc:name>samples:test-string</xqdoc:name><br />
<xqdoc:comment><br />
<xqdoc:description>This is a sample string.</xqdoc:description><br />
</xqdoc:comment><br />
<xqdoc:type>xs:string</xqdoc:type><br />
</xqdoc:variable><br />
</xqdoc:variables><br />
<xqdoc:functions><br />
<xqdoc:function arity="1"><br />
<xqdoc:comment><br />
<xqdoc:description>This function simply returns the specified integer.</xqdoc:description><br />
<xqdoc:param>$number number to return</xqdoc:param><br />
<xqdoc:return>specified number</xqdoc:return><br />
</xqdoc:comment><br />
<xqdoc:name>samples:same</xqdoc:name><br />
<xqdoc:annotations><br />
<xqdoc:annotation name="private"/><br />
</xqdoc:annotations><br />
<xqdoc:signature>declare %private function samples:same($number as xs:integer) as xs:integer</xqdoc:signature><br />
<xqdoc:parameters><br />
<xqdoc:parameter><br />
<xqdoc:name>number</xqdoc:name><br />
<xqdoc:type>xs:integer</xqdoc:type><br />
</xqdoc:parameter><br />
</xqdoc:parameters><br />
<xqdoc:return><br />
<xqdoc:type>xs:integer</xqdoc:type><br />
</xqdoc:return><br />
</xqdoc:function><br />
</xqdoc:functions><br />
</xqdoc:xqdoc><br />
</pre><br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
|Description<br />
|- valign="top"<br />
|{{Code|parse}}<br />
|Error while parsing a module.<br />
|- valign="top"<br />
|{{Code|unknown}}<br />
|The specified component does not exist.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 11.0<br />
* Removed: {{Code|inspect:function-annotations}}<br />
<br />
;Version 9.6<br />
* Updated: {{Function||inspect:type}}: options added<br />
<br />
;Version 9.3<br />
* Added: {{Function||inspect:type}}<br />
<br />
;Version 8.5<br />
* Added: {{Function||inspect:function-annotations}}, {{Function||inspect:static-context}}<br />
* Updated: {{Code|external}} attribute added to variables and functions<br />
* Updated: Relative URIs will always be resolved against the static base URI of the query<br />
<br />
;Version 7.9<br />
* Updated: a query URI can now be specified with {{Function||inspect:functions}}.<br />
<br />
This module was introduced with Version 7.7.</div>CGhttps://docs.basex.org/index.php?title=Android&diff=16907Android2023-12-01T16:39:50Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>It is possible to create an Android port of BaseX. The present tutorial outlines the creation of a BaseX Android library, which can be used in any other application project.<br />
<br />
For the creation of the library the [https://developer.android.com/studio IDE Android Studio] is used, but the steps are more or less equal using the Eclipse IDE.<br />
<br />
----<br />
<br />
== Creating the Android Library Project ==<br />
<br />
The first step is to create an Android library project, which will be later modified to represent the BaseX Android library.<br />
<br />
In Android Studio the 'Create New Project' menu item needs to be chosen. In order to this the displayed window appears.<br />
<br />
[[File:android-studio-new-project-dialog.png]]<br />
<br />
It is important that the minimum Android version is Gingerbread 2.3.3, because of some String methods used in BaseX which are not supported by Android versions older than Gingerbread.<br />
<br />
To create an Android library project, the 'Mark this project as library' item need to be checked. An Android library is not executable and therefore does not need the creation of an Activity, which is the reason why this item is unchecked in the picture above.<br />
<br />
After finishing the dialog Android Studio creates an empty library project with all needed folders and files.<br />
<br />
The next step is to copy the BaseX code into the created project folder 'src/main/java'.<br />
<br />
Except the package 'gui' and the Java file 'BaseXGui.java' inside the 'src.main.java.org.basex'[https://github.com/BaseXdb/basex/tree/master/basex-core/src/main/java/org/basex] package can be copied into the project folder. Android does not support Java AWT and Swing, which is the reason for not copying the gui package.<br />
<br />
----<br />
<br />
== Adjusting the Code ==<br />
<br />
After successfully copying the corresponding BaseX packages and java files into the created Android library project a few adjustments have to be done in order to get a working Android library.<br />
<br />
At this moment the BaseX source code is presented in the Android library project as well as an empty android package, as it is shown in the following image.<br />
<br />
[[File:library-project-after-copying.png]]<br />
<br />
In the empty android package a new Java class needs to be created, this class is used to create the necessary BaseX files and communicate with BaseX. This class needs the data directory of the application for storing the corresponding BaseX files. This files should be stored in the apps /data/data/.. folder which is only accessible from the application.<br />
This information is only available inside the applications context and not inside a library project, therefore it is necessary to pass this information to this class at the constructor call.<br />
The following source code shows a minimal example for a BaseX class.<br />
<br />
<pre lang="java"><br />
public class BaseXDatabase {<br />
private Context basexContext = null;<br />
<br />
public BaseXDatabase(String data_dir) {<br />
basexContext = new Context(data_dir);<br />
}<br />
<br />
}<br />
</pre><br />
<br />
This class can be called in every Android application which uses the BaseX library with the following call, for example:<br />
<br />
<pre lang="java"><br />
BaseXDatabase baseXDatabase = new BaseXDatabase(getApplicationInfo().dataDir);<br />
</pre><br />
<br />
At the moment it is not possible to use the BaseX library, therefore more adjustments have to be done in the BaseX code.<br />
<br />
First it is necessary to add an additional constructor to the Context class to create the BaseX files in the right directory and adjust the default constructor of it.<br />
The following code shows the changes inside the Context.java file:<br />
<br />
<pre lang="java"><br />
public Context(String data_dir) {<br />
this(true, (Prop.HOME = data_dir + "/"), (Prop.USERHOME = data_dir + "/")); <br />
File dir = new File(Prop.HOME, "basex/data");<br />
if(!dir.exists()) {<br />
if(!dir.mkdir()) {<br />
android.util.Log.i("BASEX", "CREATING BASEX DIRECTORIES");<br />
} <br />
}<br />
}<br />
<br />
private Context(final boolean file, String home, String userhome) {<br />
this(new MainProp(file));<br />
}<br />
</pre><br />
<br />
As shown in the adjustment above, it is necessary to set the two variables 'Prop.HOME' and Prop.USERHOME' during the constructor call. In the BaseX code those variables are final, which need also be changed in order to set them during the call.<br />
<br />
The reason for this change is that the in BaseX used System.getProperty(user.dir) returns an [https://developer.android.com/reference/java/lang/System.html#getProperty(java.lang.String) empty string in Android].<br />
<br />
----<br />
<br />
The next adjustment, which needs to be done, is to remove not supported packages inside the BaseX code.<br />
Therefore the package 'org.basex.query.util.crypto' need to be removed, because it uses external packages which are not supported by Android. <br />
The class which uses these files can be found inside the FNCrypto.java file in the 'query.func' package. This file needs to be deleted as well as its usage inside the Function.java file, which can also be found inside the 'query.func' package.<br />
The following lines need to be removed:<br />
<br />
<pre lang="java"><br />
/** XQuery function. */<br />
_CRYPTO_HMAC(FNCrypto.class, "hmac(message,key,algorithm[,encoding])",<br />
arg(STR, STR, STR, STR_ZO), STR),<br />
/** XQuery function. */<br />
_CRYPTO_ENCRYPT(FNCrypto.class, "encrypt(input,encryption,key,algorithm)",<br />
arg(STR, STR, STR, STR), STR),<br />
/** XQuery function. */<br />
_CRYPTO_DECRYPT(FNCrypto.class, "decrypt(input,type,key,algorithm)",<br />
arg(STR, STR, STR, STR), STR),<br />
/** XQuery function. */<br />
_CRYPTO_GENERATE_SIGNATURE(FNCrypto.class, "generate-signature" +<br />
"(input,canonicalization,digest,signature,prefix,type[,item1][,item2])",<br />
arg(NOD, STR, STR, STR, STR, STR, ITEM_ZO, ITEM_ZO), NOD),<br />
/** XQuery function. */<br />
_CRYPTO_VALIDATE_SIGNATURE(FNCrypto.class, "validate-signature(node)", arg(NOD), BLN),<br />
<br />
URIS.put(FNCrypto.class, CRYPTOURI);<br />
</pre><br />
<br />
The result of this adjustment is, that it is now possible to use BaseX as an Android library, with the lack of support of the following XQuery functions:<br />
<br />
* hmac(string,string,string[,string])<br />
* encrypt(string,string,string,string)<br />
* decrypt(string,string,string,string)<br />
* generate-signature(node,string,string,string,string,string[,item][,item])<br />
* validate-signature(node)<br />
<br />
----<br />
<br />
== Using the BaseX Android Library ==<br />
<br />
To use the BaseX library the above created BaseXDatabase class can be extended with additional methods which are delegating requests to the BaseX database and return the results.<br />
<br />
An example of this can be seen in the following code:<br />
<br />
<pre lang="java"><br />
public String executeXQuery(String query) throws IOException {<br />
if(basexContext != null)<br />
return new XQuery(query).execute(basexContext);<br />
else<br />
Log.e("BaseXDatabase", "No context");<br />
return "";<br />
}<br />
</pre><br />
<br />
This methods of the BaseXDatabase class can now be used in every Android application which includes the created BaseX Android library.<br />
<br />
It is possible to create a .jar, or an [http://tools.android.com/tech-docs/new-build-system/aar-format .aar file] out of the BaseX library, by just building the source code.<br />
This file need to be copied inside the lib folder of the Android project which wants to use the library. Additionally the build file of the application needs to be adjusted to use the library.<br />
<br />
Using Gradle, the Android build system, it can be done by adding the following line to the gradle build file. This tells the build system that every library, inside the libs folder, is being compiled into the projects file.<br />
<br />
<pre lang="java"><br />
dependencies {<br />
compile fileTree(dir: 'libs', include: ['*.jar', '*.aar'])<br />
}<br />
</pre></div>CGhttps://docs.basex.org/index.php?title=Indexes&diff=16906Indexes2023-12-01T16:39:47Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This article is part of the [[XQuery|XQuery Portal]]. It contains information on the available index structures.<br />
<br />
The query compiler tries to optimize and speed up queries by applying the index whenever it is possible, and seems promising. To see how a query is rewritten, and if an index is used, you can turn on the [[GUI#Visualizations|Info View]] in the GUI or use the [[Command-Line Options#BaseX_Standalone|-V flag]] on the command line:<br />
<br />
* A message like <code>apply text index for "Japan"</code> indicates that the text index is applied to speed up the search of the shown string. The following message…<br />
* <code>no index results</code> indicates that a string in a path expression will never yield results. Hence, the path does not need to be evaluated at all.<br />
* If you cannot find any index optimization hints in the info output, it often helps if you rewrite and simplify your query.<br />
<br />
Additional examples for index rewritings are presented in our article on [[XQuery Optimizations]].<br />
<br />
=Structural Indexes=<br />
<br />
Structural indexes are automatically created and cannot be dropped by the user:<br />
<br />
==Name Index==<br />
<br />
The name index contains references to the names of all elements and attributes in a database. It contains some basic statistical information, such as the number of occurrence of a name.<br />
<br />
The name index is e.g. applied to discard location steps that will never yield results:<br />
<br />
<pre lang='xquery'><br />
(: will be rewritten to an empty sequence :)<br />
/non-existing-name<br />
</pre><br />
<br />
The contents of the name indexes can be directly accessed with the XQuery functions {{Function|Index|index:element-names}} and {{Function|Index|index:attribute-names}}.<br />
<br />
If a database is updated, new names will be added incrementally, but the statistical information will get out-dated.<br />
<br />
==Path Index==<br />
<br />
The path index (which is also called ''path summary'' or ''data guide'') stores all distinct paths of the documents in the database. It contains additional statistical information, such as the number of occurrence of a path, its distinct string values, and the minimum/maximum of numeric values. The maximum number of distinct values to store per name can be changed via {{Option|MAXCATS}}. Distinct values are also stored for elements and attributes of numeric type.<br />
<br />
Various queries will be evaluated much faster if an up-to-date path index is available (as can be observed when opening the [[GUI#Visualizations|Info View]]):<br />
<br />
* Descendant steps will be rewritten to multiple child steps. Child steps are evaluated faster, as fewer nodes have to be traversed:<br />
<br />
<pre lang='xquery'><br />
doc('factbook.xml')//province,<br />
(: ...will be rewritten to... :)<br />
doc('factbook.xml')/mondial/country/province<br />
</pre><br />
<br />
* The {{Code|fn:count}} function will be pre-evaluated by looking up the number in the index:<br />
<br />
<pre lang='xquery'><br />
count(doc('factbook')//country)<br />
</pre><br />
<br />
* The distinct values of elements or attributes can be looked up in the index as well:<br />
<br />
<pre lang='xquery'><br />
distinct-values(db:get('factbook')//religions)<br />
</pre><br />
<br />
The contents of the path index can be directly accessed with the XQuery function {{Function|Index|index:facets}}.<br />
<br />
If a database is updated, the statistics in the path index will be invalidated.<br />
<br />
==Document Index==<br />
<br />
The document index contains references to all document nodes in a database. Once documents with specific paths are requested, the index will be extended to also contain document paths.<br />
<br />
The index generally speeds up access to single documents and database paths. It will always be kept up-to-date.<br />
<br />
=Value Indexes=<br />
<br />
Value indexes can be created and dropped by the user. Four types of values indexes are available: a text and attribute index, and an optional token and full-text index. By default, the text, and attribute index will automatically be created.<br />
<br />
In the GUI, index structures can be managed in the dialog windows for creating new databases or displaying the database properties. On command-line, the commands {{Command|CREATE INDEX}} and {{Command|DROP INDEX}} are used to create and drop index structures. With {{Command|INFO INDEX}}, you get some insight into the contents of an index structure, and {{Command|SET}} allows you to change the index defaults for new databases:<br />
<br />
* <code>OPEN factbook; CREATE INDEX fulltext</code>: Open database; create full-text index<br />
* <code>OPEN factbook; INFO INDEX TOKEN</code>: Open database; show info on token index<br />
* <code>SET ATTRINDEX true; SET ATTRINCLUDE id name; CREATE DB factbook.xml</code>: Enable attribute index; only index 'id' and 'name' attributes; create database<br />
<br />
With XQuery, index structures can be created and dropped via {{Function|Database|db:optimize}}:<br />
<br />
<pre lang='xquery'><br />
(: Optimize specified database, create full-text index for texts of the specified elements :)<br />
db:optimize(<br />
'factbook',<br />
false(),<br />
map { 'ftindex': true(), 'ftinclude': 'p div' }<br />
)<br />
</pre><br />
<br />
==Text Index==<br />
<br />
===Exact Queries===<br />
<br />
This index references text nodes of documents. It will be utilized to accelerate string comparisons in path expressions. The following queries will all be rewritten for index access:<br />
<br />
<pre lang='xquery'><br />
(: example 1 :)<br />
//*[text() = 'Germany'],<br />
(: example 2 :)<br />
doc('factbook.xml')//name[. = 'Germany'],<br />
(: example 3 :)<br />
for $c in db:get('factbook')//country<br />
where $c//city/name = 'Hanoi'<br />
return $c/name<br />
</pre><br />
<br />
Before the actual index rewriting takes places, some preliminary optimizations are applied:<br />
* In example 2, the context item expression {{Code|.}} will be replaced with a {{Code|text()}} step.<br />
* In example 3, the {{Code|where}} clause will be rewritten to a predicate and attached to the first path expression.<br />
<br />
The indexed text nodes can be accessed directly with the XQuery function {{Function|Database|db:text}}. The indexed string values can be looked up via {{Function|Index|index:text}}.<br />
<br />
The {{Option|UPDINDEX}} option can be enabled to keep this index up-to-date:<br />
<br />
<pre lang='xquery'><br />
db:optimize(<br />
'mydb',<br />
true(),<br />
map { 'updindex':true(), 'textindex': true(), 'textinclude':'id' }<br />
)<br />
</pre><br />
<br />
===Range Queries===<br />
<br />
The text index also supports range queries based on string comparisons:<br />
<br />
<pre lang='xquery'><br />
(: example 1 :)<br />
db:get('Library')//Medium[Year >= '2011' and Year <= '2016'],<br />
(: example 2 :)<br />
let $min := '2014-04-16T00:00:00'<br />
let $max := '2014-04-19T23:59:59' <br />
return db:get('news')//entry[date-time > $min and date-time < $max]<br />
</pre><br />
<br />
With {{Function|Database|db:text-range}}, you can access all text nodes whose values are between a minimum and maximum value.<br />
<br />
Please note that the index structures do not support queries for numbers and dates.<br />
<br />
==Attribute Index==<br />
<br />
Similar to the text index, this index speeds up string and range comparisons on attribute values. Additionally, the XQuery function {{Code|fn:id}} takes advantage of the index whenever possible. The following queries will all be rewritten for index access:<br />
<br />
<pre lang='xquery'><br />
(: 1st example :)<br />
//country[@car_code = 'J'],<br />
(: 2nd example :)<br />
//province[@* = 'Hokkaido']//name,<br />
(: 3rd example :)<br />
//sea[@depth > '2100' and @depth < '4000']<br />
(: 4th example :)<br />
fn:id('f0_119', db:get('factbook'))<br />
</pre><br />
<br />
''Attribute nodes'' (which you can use as starting points of navigation) can directly be retrieved from the index with the XQuery functions {{Function|Database|db:attribute}} and {{Function|Database|db:attribute-range}}. The index contents (''strings'') can be accessed with {{Function|Index|index:attributes}}.<br />
<br />
The {{Option|UPDINDEX}} option can be activated to keep this index up-to-date.<br />
<br />
==Token Index==<br />
<br />
In many XML dialects, such as HTML or DITA, multiple tokens are stored in attribute values. The token index can be created to speed up the retrieval of these tokens. The XQuery functions {{Code|fn:contains-token}}, {{Code|fn:tokenize}} and {{Code|fn:idref}} are rewritten for index access whenever possible. If a token index exists, it will, e.g., be utilized for the following queries:<br />
<br />
<pre lang='xquery'><br />
(: 1st example :)<br />
//div[contains-token(@class, 'row')],<br />
(: 2nd example :)<br />
//p[tokenize(@class) = 'row'],<br />
(: 3rd example :)<br />
doc('graph.xml')/idref('edge8')<br />
</pre><br />
<br />
''Attribute nodes'' with a matching value (containing at least one from a set of given tokens) can be directly retrieved from the index with the XQuery function {{Function|Database|db:token}}. The index contents (''token strings'') can be accessed with {{Function|Index|index:tokens}}.<br />
<br />
==Full-Text Index==<br />
<br />
The [[Full-Text]] index contains the normalized tokens of text nodes of a document. It is utilized to speed up queries with the {{Code|contains text}} expression, and it is capable of processing wildcard and fuzzy search operations. Three evaluation strategies are available: the standard sequential database scan, a full-text index-based evaluation and a hybrid one, combining both strategies (see [https://files.basex.org/publications/Gruen%20et%20al.%20%5B2009%5D,%20XQuery%20Full%20Text%20Implementation%20in%20BaseX.pdf XQuery Full Text implementation in BaseX]).<br />
<br />
If the full-text index exists, the following queries will all be rewritten for index access:<br />
<br />
<pre lang='xquery'><br />
(: 1st example :)<br />
//country[name/text() contains text 'and'],<br />
(: 2nd example :)<br />
//religions[.//text() contains text { 'Catholic', 'Roman' }<br />
using case insensitive distance at most 2 words]<br />
</pre><br />
<br />
The index provides support for the following full-text features (the values can be changed in the GUI or via the {{Command|SET}} command):<br />
<br />
* '''Stemming''': tokens are stemmed before being indexed (option: {{Option|STEMMING}})<br />
* '''Case Sensitive''': tokens are indexed in case-sensitive mode (option: {{Option|CASESENS}})<br />
* '''Diacritics''': diacritics are indexed as well (option: {{Option|DIACRITICS}})<br />
* '''Stopword List''': a stop word list can be defined to reduce the number of indexed tokens (option: {{Option|STOPWORDS}})<br />
* '''Language''': see [[Full-Text#Languages|Languages]] for more details (option: {{Option|LANGUAGE}})<br />
<br />
The options that have been used for creating the full-text index will also be applied to the optimized full-text queries. However, the defaults can be overwritten if you supply options in your query. For example, if words were stemmed in the index, and if the query can be rewritten for index access, the query terms will be stemmed as well, unless stemming is not explicitly disabled. This is demonstrated in the following [[Commands#Command_Scripts|Command Script]]:<br />
<br />
<pre lang="xml"><br />
<commands><br />
<!-- Create database with stemmed full-text index --><br />
<set option='stemming'>true</set><br />
<set option='ftindex'>true</set><br />
<create-db name='test-db'> <text>house</text> </create-db><br />
<!-- Index access: Query term will be stemmed --><br />
<xquery> /text[. contains text { 'houses' }] </xquery><br />
<!-- Disable stemming (query will not be evaluated by the index) --><br />
<xquery> /text[. contains text { 'houses' } using no stemming] </xquery><br />
</commands><br />
</pre><br />
<br />
Text nodes can be directly requested from the index via the XQuery function {{Function|Full-Text|ft:search}}. The index contents can be accessed with {{Function|Full-Text|ft:tokens}}.<br />
<br />
==Selective Indexing==<br />
<br />
Value indexing can be restricted to specific elements and attributes. The nodes to be indexed can be restricted via the {{Option|TEXTINCLUDE}}, {{Option|ATTRINCLUDE}}, {{Option|TOKENINCLUDE}} and {{Option|FTINCLUDE}} options. The options take a list of name patterns, which are separated by commas. The following name patterns are supported:<br />
<br />
* <code>*</code>: all names<br />
* <code>name</code>: elements or attributes called <code>name</code>, which are in the empty default namespace<br />
* <code>*:name</code>: elements or attributes called <code>name</code>, no matter which namespace<br />
* <code>Q{uri}*</code>: all elements or attributes in the <code>uri</code> namespace<br />
* <code>Q{uri}name</code>: elements or attributes called <code>name</code> in the <code>uri</code> namespace<br />
<br />
The options can either be specified via the {{Command|SET}} command or via XQuery. With the following operations, an attribute index is created for all {{Code|id}} and {{Code|name}} attributes:<br />
<br />
; Commands<br />
<pre lang="xml"><br />
SET ATTRINCLUDE id,name<br />
CREATE DB factbook http://files.basex.org/xml/factbook.xml'<br />
# Restore default<br />
SET ATTRINCLUDE<br />
</pre><br />
<br />
; XQuery<br />
<pre lang='xquery'><br />
db:create('factbook', 'http://files.basex.org/xml/factbook.xml', '',<br />
map { 'attrinclude': 'id,name' })<br />
</pre><br />
<br />
With {{Command|CREATE INDEX}} and {{Function|Database|db:optimize}}, new selective indexing options will be applied to an existing database.<br />
<br />
==Enforce Rewritings==<br />
<br />
In various cases, existing index structures will not be utilized by the query optimizer. This is usually the case if the name of the database is not a static string (e.g. because it is bound to a variable or passed on as an argument of a function call). Furthermore, several candidates for index rewritings may exist, and the query optimizer may decide for a rewriting that turns out to be suboptimal.<br />
<br />
With the {{Option|ENFORCEINDEX}} option, certain index rewritings can be enforced. While the option can be globally enabled, it is usually better to supply it as [[XQuery Extensions#Pragmas|Pragma]]. Two examples:<br />
<br />
* In the query below, 10 databases will be addressed. If it is known in advance that these databases contain an up-to-date text index, the index rewriting can be enforced as follows:<br />
<br />
<pre lang='xquery'><br />
(# db:enforceindex #) {<br />
for $n in 1 to 10<br />
let $db := 'persons' || $n<br />
return db:get($db)//person[name/text() = 'John']<br />
}<br />
</pre><br />
<br />
* The following query contains two predicates that may both be rewritten for index access. If the automatically chosen rewriting is known not to be optimal, another index rewriting can enforced by surrounding the specific expression with the pragma:<br />
<br />
<pre lang='xquery'><br />
db:get('factbook')//country<br />
[(# db:enforceindex #) {<br />
@population > '10000000' and<br />
@population < '10999999'<br />
}]<br />
[religions/text() = 'Protestant']<br />
</pre><br />
<br />
The option can also be assigned to predicates with dynamic values. In the following example, the comparison of the first comparison will be rewritten for index access. Without the pragma expression, the second comparison is preferred and chosen for the rewriting because the statically known string allows for an exact cost estimation:<br />
<br />
<pre lang='xquery'><br />
for $name in ('Germany', 'Italy')<br />
for $country in db:get('factbook')//country<br />
where (# db:enforceindex #) { $country/name = $name }<br />
where $country/religions/text() = 'Protestant'<br />
return $country<br />
</pre><br />
<br />
Please note that:<br />
<br />
* The option should only be enabled if the addressed databases exist, have all required index structures and are up-to-date (otherwise, you will be given an error message).<br />
* If you address the full-text index, and if you use non-default indexing options, you will have to specify them in your query (via {{Code|using stemming}}, {{Code|using language 'de'}}, etc).<br />
* If you have more than one enforce pragma in a single path expression, only the first will be considered.<br />
* In general, there are always expressions that cannot be rewritten for index access. If you enforce rewritings, you will have no guarantee that an index will be used.<br />
<br />
=Custom Index Structures=<br />
<br />
With XQuery, it is comparatively easy to create your own, custom index structures. The following query demonstrates how you can create a {{Code|factbook-index}} database, which contains all texts of the original database in lower case:<br />
<br />
<pre lang='xquery'><br />
let $db := 'factbook'<br />
<br />
let $index := <index>{<br />
for $nodes in db:get($db)//text()<br />
group by $text := lower-case($nodes)<br />
return <text string='{ $text }'>{<br />
for $node in $nodes<br />
return <id>{ db:node-id($node ) }</id><br />
}</text><br />
}</index><br />
<br />
return db:create($db || '-index', $index, $db || '-index.xml')<br />
</pre><br />
<br />
In the following query, a text string is searched, and the text nodes of the original database are retrieved:<br />
<br />
<pre lang='xquery'><br />
let $db := 'factbook'<br />
let $text := 'italian'<br />
for $id in db:get($db || '-index')//*[@string = $text]/id<br />
return db:get-id($db, $id)/..<br />
</pre><br />
<br />
With some extra effort, and if {{Option|UPDINDEX}} is enabled for both your original and your index database (see below), your index database will support updates as well (try it, it’s fun!).<br />
<br />
=Performance=<br />
<br />
If main memory runs out while creating a value index, the current index structures will be partially written to disk and eventually merged. If the memory heuristics fail for some reason (i.e. because multiple index operations run at the same time, or because the applied JVM does not support explicit garbage collections), a fixed index split sizes may be chosen via the {{Option|SPLITSIZE}} option.<br />
<br />
If {{Option|DEBUG}} is enabled, the command-line output might help you find a good split size. The following example shows the output for creating a database for an XMark document with 1 GB, and with 128 MB assigned to the JVM:<br />
<br />
<pre><br />
> basex -d -c"SET FTINDEX ON; SET TOKENINDEX ON; CREATE DB xmark 1gb.xml"<br />
Creating Database...<br />
................................ 76559.99 ms (29001 KB)<br />
Indexing Text...<br />
....|...|...|.....|. 9.81 M operations, 18576.92 ms (13523 KB). Recommended SPLITSIZE: 20.<br />
Indexing Attribute Values...<br />
.........|....... 3.82 M operations, 7151.77 ms (6435 KB). Recommended SPLITSIZE: 20.<br />
Indexing Tokens...<br />
.......|..|.....|.. 3.82 M operations, 9636.73 ms (10809 KB). Recommended SPLITSIZE: 10.<br />
Indexing Full-Text...<br />
..|.|.|.|...|...|..|.|..| 116.33 M operations, 138740.94 ms (106 MB). Recommended SPLITSIZE: 12.<br />
</pre><br />
<br />
The output can be interpreted as follows:<br />
<br />
* The vertical bar <code>|</code> indicates that a partial index structure was written to disk.<br />
* The mean value of the recommendations can be assigned to the {{Option|SPLITSIZE}} option. Please note that the recommendation is only a vague proposal, so try different values if you get main-of-memory errors or indexing gets too slow. Greater values will require more main memory.<br />
* In the example, the full-text index was split 12 times. 116 million tokens were indexed, processing time was 2.5 minutes, and final main memory consumption (after writing the index to disk) was 76 MB. A good value for the split size option could be {{Code|15}}.<br />
<br />
=Updates=<br />
<br />
Generally, update operations are very fast in BaseX. By default, the index structures will be invalidated by updates; as a result, queries that benefit from index structures may slow down after updates. There are different alternatives to cope with this:<br />
<br />
* After the execution of one or more update operations, the {{Command|OPTIMIZE}} command or the {{Function|Database|db:optimize}} function can be called to rebuild the index structures.<br />
* The {{Option|UPDINDEX}} option can be activated before creating or optimizing the database. As a result, the text, attribute and token indexes will be incrementally updated after each database update. Please note that incremental updates are not available for the full-text index and database statistics. This also explains why the UPTODATE flag, which is e.g. displayed via {{Command|INFO DB}} or {{Function|Database|db:info}}, will be set to {{Code|false}} until the database will be optimized again (various optimizations won’t be triggered. For example, count(//item) can be extremely fast if all metadata is up-to-date.<br />
* The {{Option|AUTOOPTIMIZE}} option can be enabled before creating or optimizing the database. All outdated index structures and statistics will then be recreated after each database update. This option should only be done for small and medium-sized databases.<br />
* Both options can be used side by side: {{Option|UPDINDEX}} will take care that the value index structures will be updated as part of the actual update operation. {{Option|AUTOOPTIMIZE}} will update the remaining data structures (full-text index, database statistics).<br />
<br />
=Changelog=<br />
<br />
;Version 9.1<br />
* Updated: [[#Enforce Rewritings|Enforce Rewritings]], support for comparisons with dynamic values.<br />
<br />
;Version 9.0<br />
* Added: [[#Enforce Rewritings|Enforce Rewritings]]<br />
<br />
;Version 8.4<br />
* Updated: [[#Name Index|Name Index]], [[#Path Index|Path Index]]<br />
<br />
;Version 8.4<br />
* Added: [[#Token Index|Token Index]]<br />
<br />
;Version 8.3<br />
* Added: [[#Selective Indexing|Selective Indexing]]<br />
<br />
;Version 8.0<br />
* Added: AUTOOPTIMIZE option<br />
<br />
;Version 7.2.1<br />
* Added: string-based range queries</div>CGhttps://docs.basex.org/index.php?title=Community:Information&diff=16905Community:Information2023-12-01T16:39:44Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>Welcome to our community information page.<br />
This is the location to discuss all kinds of editor related information.<br />
<br />
==About the documentation project==<br />
We decided to open the documentation process to our community. To us this is a natural consequence from all the positive experience we made during the last years.<br />
<br />
We know,<br />
* writing good documentation is essential for both our users & developers<br />
* constant feedback helps improving so please feel free and discuss our articles or file requests for new ones<br />
* some of you have very profound knowledge of BaseX and its infrastructure. You are very welcome to contribute Articles & Edits.<br />
<br />
== Where to start ==<br />
If you have not signed up yet: this is the moment to [[Special:UserLogin|create your account]]. <br />
<br />
You may now start by either creating articles or choosing from a list of pages that might need some maintenance:<br />
*[[Special:WantedPages|Most wanted articles]] <br />
* [[Special:ShortPages|List of short articles]]<br />
*[[Special:DeadendPages| List of pages that have no outgoing links]]<br />
<br />
===Wanted Articles===<br />
You may also add links to Topics you would like to see in our documentation:<br />
<br />
Surround a word with double brackets:<br/><br />
<nowiki>[[Wanted Topic]]</nowiki> and that word will be highlighted in red.<br />
<br />
===Guidelines for new Articles ===<br />
<br />
For now we only have few guidelines we used when creating new articles. Feel free to extend or change this list.<br />
<br />
;Don't be afraid<br />
:We happily accept your contributions.<br />
<br />
;Article Title<br />
:Please make sure article titles are upper-case and singular nouns wherever possible.<br />
<br />
;Categories<br />
:Try to categorize your article via <nowiki>[[Category:Name]]</nowiki>.<br />
<br />
;Code Examples<br />
:Please use the <nowiki>[[Category:Code Examples]]</nowiki>.<br />
<br />
;Syntax Highlighting<br />
:You can highlight your code as follows: <nowiki><pre lang="{xml|xquery|java}>…</pre></nowiki><br />
<br />
==Writing Articles==<br />
<br />
Please see [[Help:Editing]] for some information on ''Wiki Syntax''.</div>CGhttps://docs.basex.org/index.php?title=Databases&diff=16904Databases2023-12-01T16:39:42Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This page is part of the [[Getting Started]] Section.<br />
<br />
In BaseX, a ''database'' is a pretty light-weight concept. It may contain one or more '''resources''', which are addressed by a unique database path. There is no explicit layer for collections: Instead, collections are implicitly created and deleted, and collections result from the existence of documents in specific paths. <br />
<br />
As a single database is restricted to 2 billion XML nodes (see [[Statistics]]), but resources can easily be distributed across multiple database instances. Multiple databases can be addressed (queried, updated) by a single XQuery expression. <br />
<br />
Three different resource types exist:<br />
<br />
{|class="wikitable"<br />
|- valign="top"<br />
| '''Resource Type'''<br />
| '''Description'''<br />
|- valign="top"<br />
| XML Documents<br />
| The default resource type. The storage and index features are optimized for XML contents, or any other contents stored in an XML representation.<br />
|- valign="top"<br />
| Binary Data<br />
| Binary data: Raw data of any type, stored in its binary representation. See [[Binary Data]] for more information.<br />
|- valign="top"<br />
| XQuery Values<br />
| Results of XQuery expressions, stored in a binary representation for fast retrieval. All value types are supported, including maps and arrays, but excluding any other [[Higher-Order Functions#Function Items|function items]].<br />
|}<br />
<br />
=Create Databases=<br />
<br />
Databases can be created via [[Commands]], via [[XQuery]], in the [[GUI]], and with various [[Developing|APIs]]. If an initial input is specified with a create operation, some time can be saved, as the specified resources will be added to the database in a bulk operation:<br />
<br />
* [[Startup#BaseX Standalone|Command-Line]]: <code>CREATE DB documents /path/to/resources</code>: Add resources in the specified path to a database named {{Code|documents}}.<br />
* [[Startup#BaseX GUI|GUI]]: Go to ''Database'' → ''New'', press ''Browse…'' to choose an initial file or directory, and press ''OK''.<br />
<br />
The database name is composed of a restricted set of characters (see [[Valid Names]]). Various [[Parsers]] can be selected to control the import process, or to convert data of different input type to XML.<br />
<br />
=Access Resources=<br />
<br />
Stored resources and external documents can be accessed in different ways:<br />
<br />
==XML Documents==<br />
<br />
Various XQuery functions exist to access XML documents in databases:<br />
<br />
{| class="wikitable"<br />
|- valign="top"<br />
!Function<br />
!Example<br />
!Description<br />
|- valign="top"<br />
|{{Function|Database|db:get}}<br />
|{{Code|db:get("db", "path/to/docs")}}<br />
|Returns all documents that are found in the database {{Code|db}} at the (optional) path {{Code|path/to/docs}}.<br />
|- valign="top"<br />
|<code>[http://www.xqueryfunctions.com/xq/fn_collection.html fn:collection]</code><br />
|{{Code|collection("db/path/to/docs")}}<br />
|Returns all documents at the location {{Code|path/to/docs}} in the database {{Code|db}}.<br/>If no path is specified after the database, all documents in the database will be returned.<br/>If no argument is specified, all documents of the database will be returned that has been opened in the global context.<br />
|- valign="top"<br />
|<code>[http://www.xqueryfunctions.com/xq/fn_doc.html fn:doc]</code><br />
|{{Code|doc("db/path/to/doc.xml")}}<br />
|Returns the document at the location {{Code|path/to/docs}} in the database {{Code|db}}.<br/>An error is raised if the specified yields zero or more than one document.<br />
|}<br />
<br />
You can access multiple databases in a single query:<br />
<br />
<pre lang='xquery'><br />
for $i in 1 to 100<br />
return db:get('books' || $i)//book/title<br />
</pre><br />
<br />
If the {{Option|DEFAULTDB}} option is turned on, the path argument of the {{Code|fn:doc}} or {{Code|fn:collection}} functions will first be resolved against the globally opened database.<br />
<br />
Two more functions are available for retrieving information on database nodes:<br />
<br />
{| class="wikitable"<br />
|-<br />
!Function<br />
!Example<br />
!Description<br />
|- valign="top"<br />
|{{Function|Database|db:name}}<br />
|{{Code|db:name($node)}}<br />
|Returns the name of the database in which the specified {{Code|$node}} is stored.<br />
|- valign="top"<br />
|{{Function|Database|db:path}}<br />
|{{Code|db:path($node)}}<br />
|Returns the path of the database document in which the specified {{Code|$node}} is stored.<br />
|}<br />
<br />
The {{Code|fn:document-uri}} and {{Code|fn:base-uri}} functions return URIs that can also be reused as arguments for the {{Code|fn:doc}} and {{Code|fn:collection}} functions. As a result, the following example query always returns {{Code|true}}:<br />
<br />
<pre lang='xquery'><br />
every $c in collection('anyDB')<br />
satisfies doc-available(document-uri($c))<br />
</pre><br />
<br />
If the argument of {{Code|fn:doc}} or {{Code|fn:collection}} does not start with a valid database name, or if the addressed database does not exist, the string is interpreted as URI reference, and the documents found at this location will be returned. Examples:<br />
<br />
* {{Code|doc("http://web.de")}}: retrieves the addressed URI and returns it as a main-memory document node.<br />
* {{Code|doc("myfile.xml")}}: retrieves the given file from the file system and returns it as a main-memory document node. Note that updates to main-memory nodes are not automatically written back to disk unless the {{Option|WRITEBACK}} option is set.<br />
* {{Code|collection("/path/to/docs")}}: returns a main-memory collection with all XML documents found at the addressed file path.<br />
<br />
==Binary Data==<br />
<br />
The {{Command|BINARY GET}} command and the {{Function|Database|db:get-binary}} function can be used to return files in their native byte representation.<br />
<br />
If the API you use does not support binary output (which is e.g. the case for various [[Clients|Client]] language bindings), you can convert your binary data to its string representation before returning it to the client:<br />
<br />
<pre lang='xquery'><br />
string(db:get-binary('multimedia', 'sample.avi'))<br />
</pre><br />
<br />
==XQuery Values==<br />
<br />
With {{Function|Database|db:get-value}}, XQuery values can be retrieved. In the following example, we assume that an XQuery map {{Code|cities}} was stored in an {{Code|indexes}} database:<br />
<br />
<pre lang='xquery'><br />
let $city-map := db:get-value('indexes', 'cities')<br />
return $city-map?Chile<br />
</pre><br />
<br />
=Update Resources=<br />
<br />
==Commands==<br />
<br />
Once you have created a database, additional commands exist to modify its contents:<br />
<br />
* XML documents can be added with the {{Command|PUT}} and {{Command|ADD}} commands.<br />
* Binary data is stored with {{Command|BINARY PUT}}.<br />
* Resources of all types can be deleted via {{Command|DELETE}}.<br />
<br />
{{Option|AUTOFLUSH}} can be turned off before ''bulk operations'' (i.e., before numerous new resources are added to the database).<br />
<br />
If {{Option|ADDCACHE}} is enabled, the input will be cached before it is added to the database. This is helpful when the input documents are expected to consume too much main-memory.<br />
<br />
With the following [[Commands#Command Scripts|command script]], an empty database is created, two resources are added (one directly, another one cached), and all data is exported to the file system:<br />
<br />
<pre><br />
CREATE DB example<br />
SET AUTOFLUSH false<br />
ADD example.xml<br />
SET ADDCACHE true<br />
ADD /path/to/xml/documents<br />
BINARY PUT TO images/ 123.jpg<br />
EXPORT /path/to/file-system/<br />
</pre><br />
<br />
==XQuery==<br />
<br />
You can also use functions from the [[Database Module]] to add, replace, or delete XML documents:<br />
<br />
<pre lang='xquery'><br />
db:add('documents', '/path/to/xml/resources/')<br />
</pre><br />
<br />
Function from other modules, such as the [[File Module]], can be utilized to filter the input. With the following code, all files that contain numbers in the filename are selected, and stored as XML. If an input file contains no well-formed XML, it is stored as binary resource, and the error message is stored as a string value:<br />
<br />
<pre lang='xquery'><br />
let $db := 'documents'<br />
let $root := '/path/to/resources/'<br />
for $path in file:list($root)<br />
where matches($path, '\d+')<br />
return try {<br />
db:put($db, fetch:doc($root || $path), $path)<br />
} catch * {<br />
db:put-binary($db, $root || $path, $path),<br />
db:put-value($db, $err:description, $path || '.error')<br />
}<br />
</pre><br />
<br />
The error messages can e.g. be analyzed in a second step:<br />
<br />
<pre lang='xquery'><br />
let $errors := db:get-value('documents')<br />
for $filename in map:keys($errors)<br />
where ends-with($filename, '.error')<br />
return $filename || ': ' || $errors?($filename)<br />
</pre><br />
<br />
=Export Database=<br />
<br />
All resources stored in a database can be ''exported'', i.e., written back to disk, e.g., as follows:<br />
<br />
* Commands: {{Command|EXPORT}} writes all resources to the specified target directory.<br />
* GUI: Go to ''Database'' → ''Export'', choose the target directory and press ''OK''.<br />
* XQuery: Use {{Function|Database|db:export}}.<br />
* WebDAV: Locate the database directory (or a subdirectory of it) and copy all contents to another location.<br />
<br />
=Main-Memory Databases=<br />
<br />
A database can be created in main-memory by enabling the {{Option|MAINMEM}} option. Next, in the standalone context, a main-memory database can be created, which can then be accessed by subsequent commands.<br />
<br />
If a BaseX server is started, and if a database is created in its context at startup time, e.g., with the [[Command-Line Options|command-line option -c]] and a {{Command|CREATE DB}} call, BaseX clients can then access and update this database:<br />
<br />
<pre lang="perl"><br />
# Server<br />
basexserver -c"SET mainmem on" -c"CREATE DB mainmem document.xml"<br />
BaseX [Server]<br />
Server was started (port: 1984).<br />
MAINMEM: true<br />
Database 'mainmem' created in 1782.80 ms.<br />
<br />
# Client<br />
basexclient<br />
Username: ...<br />
Password: ...<br />
BaseX [Client]<br />
Try 'help' to get more information.<br />
> XQUERY count(db:get('mainmem')//*)<br />
1876462<br />
Query executed in 0.97 ms.<br />
</pre><br />
<br />
Additional notes:<br />
* You can force an ordinary database, or parts of it, to being temporarily copied to memory by applying an empty [[XQuery_Update#Main-Memory_Updates|main-memory update]] on a database node: <code>db:get('some-db') update { }</code><br />
* If you open local or remote documents with <code>fn:doc</code> or <code>fn:collection</code>, the resulting internal representation is identical to those of main-memory database instances (regardless of which value is set for {{Option|MAINMEM}}).<br />
<br />
=Changelog=<br />
<br />
;Version 10.0<br />
* Added: New resource type for XQuery values.<br />
<br />
;Version 8.4<br />
* Updated: Items of binary type can be output without specifying the obsolete <code>raw</code> serialization method.<br />
<br />
;Version 7.2.1<br />
* Updated: {{Code|fn:document-uri}} and {{Code|fn:base-uri}} now return strings that can be reused with {{Code|fn:doc}} or {{Code|fn:collection}} to reopen the original document.</div>CGhttps://docs.basex.org/index.php?title=SQL_Module&diff=16903SQL Module2023-12-01T16:39:41Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This [[Module Library|XQuery Module]] contains functions to access relational databases from XQuery using SQL. With this module, you can execute query, update and prepared statements, and the result sets are returned as sequences of XML elements representing tuples. Each element has children representing the columns returned by the SQL statement.<br />
<br />
This module uses JDBC to connect to a SQL server. Hence, your JDBC driver will need to be added to the classpath, too. If you work with the full distributions of BaseX, you can copy the driver into the {{Code|lib}} directory. To connect to MySQL, for example, download the [https://dev.mysql.com/downloads/connector/j/ Connector/J Driver] and extract the archive into this directory.<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/sql</nowiki></code> namespace, which is statically bound to the {{Code|sql}} prefix.<br/><br />
<br />
=Functions=<br />
<br />
==sql:init==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>sql:init(<br />
$class as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|This function initializes a JDBC driver specified via {{Code|$class}}. This step might be superfluous if the SQL database is not embedded.<br/ ><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|init|#Errors}} the specified driver is not found.<br />
|}<br />
<br />
==sql:connect==<br />
<br />
{{Announce|Updated with Version 11:}} Optional credentials.<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>sql:connect(<br />
$url as xs:string,<br />
$options as map(*)? := map { }<br />
) as xs:anyURI</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|This function establishes a connection to a relational database and returns a connection id. The parameter {{Code|$url}} is the URL of the database and shall be of the form: {{Code|jdbc:<driver name>:[//<server>[/<database>]]}}. Values specified for {{Code|$username}} and {{Code|$password}} are used as credentials for connecting to the database. The {{Code|$options}} parameter can be used to set connection options.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|error|#Errors}} an SQL exception occurred when connecting to the database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|Connects to an SQL Server and sets autocommit to {{Code|true}}:<br />
<pre lang='xquery'><br />
sql:connect('dbc:sqlserver://DBServer', map { 'autocommit': true() })<br />
</pre><br />
|}<br />
<br />
==sql:execute==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>sql:execute(<br />
$id as xs:anyURI,<br />
$statement as xs:string,<br />
$options as map(*)? := map { }<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
| This function executes an SQL {{Code|$statement}}, using the connection with the specified {{Code|$id}}. The returned result depends on the kind of statement:<br />
* If an update statement was executed, the number of updated rows will be returned as integer.<br />
* Otherwise, an XML representation of all results will be returned.<br />
With {{Code|$options}}, the following parameter can be set:<br />
* {{Code|timeout}}: query execution will be interrupted after the specified number of seconds.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|error|#Errors}} an error occurred while executing SQL.<br/ >{{Error|id|#Errors}} the specified connection does not exist.<br/>{{Error|timeout|#Errors}} query execution exceeded timeout.<br/><br />
|}<br />
<br />
==sql:execute-prepared==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>sql:execute-prepared(<br />
$id as xs:anyURI,<br />
$params as element(sql:parameters),<br />
$options as map(*)? := map { }<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
| This function executes a prepared statement with the specified {{Code|$id}}. The output format is identical to {{Function||sql:execute}}. The optional parameter {{Code|$params}} is an element {{Code|<sql:parameters/>}} representing the parameters for a prepared statement along with their types and values. The following schema shall be used:<br/ ><br />
<pre lang='xquery'><br />
element sql:parameters {<br />
element sql:parameter {<br />
attribute type { "bigdecimal" | "boolean" | "byte" | "date" | "double" | "float" |<br />
"int" | "long" | "short" | "sqlxml" | "string" | "time" | "timestamp" },<br />
attribute null { "true" | "false" }?,<br />
text<br />
}+<br />
}?<br />
</pre><br />
With {{Code|$options}}, the following parameter can be set:<br />
* {{Code|timeout}}: query execution will be interrupted after the specified number of seconds.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|attribute|#Errors}} an attribute different from {{Code|type}} and {{Code|null}} is set for a {{Code|<sql:parameter/>}} element.<br/ >{{Error|error|#Errors}} an error occurred while executing SQL.<br/ >{{Error|id|#Errors}} the specified connection does not exist.<br/ >{{Error|parameters|#Errors}} no parameter type specified.<br/>{{Error|timeout|#Errors}} query execution exceeded timeout.<br/>{{Error|type|#Errors}} the value of a parameter cannot be converted to the specified format.<br />
|}<br />
<br />
==sql:prepare==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>sql:prepare(<br />
$id as xs:anyURI,<br />
$statement as xs:string<br />
) as xs:anyURI</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|This function prepares an SQL {{Code|$statement}}, using the specified connection {{Code|$id}}, and returns the id reference to this statement. The statement is a string with one or more '?' placeholders. If the value of a field has to be set to {{Code|NULL}}, then the attribute {{Code|null}} of the {{Code|<sql:parameter/>}} element must be {{Code|true}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|error|#Errors}} an error occurred while executing SQL.<br/ >{{Error|id|#Errors}} the specified connection does not exist.<br/ ><br />
|}<br />
<br />
==sql:commit==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>sql:commit(<br />
$id as xs:anyURI<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
| This function commits the changes made to a relational database, using the specified connection {{Code|$id}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|error|#Errors}} an error occurred while executing SQL.<br/ >{{Error|id|#Errors}} the specified connection does not exist.<br/ ><br />
|}<br />
<br />
==sql:rollback==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>sql:rollback(<br />
$id as xs:anyURI<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
| This function rolls back the changes made to a relational database, using the specified connection {{Code|$id}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|error|#Errors}} an error occurred while executing SQL.<br/ >{{Error|id|#Errors}} the specified connection does not exist.<br/ ><br />
|}<br />
<br />
==sql:close==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>sql:close(<br />
$id as xs:anyURI<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
| This function closes a database connection with the specified {{Code|$id}}.<br/>Opened connections will automatically be closed after the XQuery expression has been evaluated, but in order to save memory, it is always recommendable to close connections that are not used anymore.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|error|#Errors}} an error occurred while executing SQL.<br/ >{{Error|id|#Errors}} the specified connection does not exist.<br/ ><br />
|}<br />
<br />
=Examples=<br />
<br />
==Direct queries==<br />
<br />
A simple select statement can be executed as follows:<br />
<br />
<pre lang='xquery'><br />
let $id := sql:connect("jdbc:postgresql://localhost:5432/coffeehouse")<br />
return sql:execute($id, "SELECT * FROM coffees WHERE price < 10")<br />
</pre><br />
<br />
The result may look like:<br />
<br />
<pre lang="xml"><br />
<sql:row xmlns:sql="http://basex.org/modules/sql"><br />
<sql:column name="cof_name">French_Roast</sql:column><br />
<sql:column name="sup_id">49</sql:column><br />
<sql:column name="price">3.5</sql:column><br />
<sql:column name="sales">15</sql:column><br />
<sql:column name="total">30</sql:column><br />
</sql:row><br />
<sql:row xmlns:sql="http://basex.org/modules/sql"><br />
<sql:column name="cof_name">French_Roast_Decaf</sql:column><br />
<sql:column name="sup_id">49</sql:column><br />
<sql:column name="price">7.5</sql:column><br />
<sql:column name="sales">10</sql:column><br />
<sql:column name="total">14</sql:column><br />
</sql:row><br />
</pre><br />
<br />
==Prepared Statements==<br />
<br />
A prepared select statement can be executed in the following way:<br />
<br />
<pre lang='xquery'><br />
(: Establish a connection :)<br />
let $conn := sql:connect("jdbc:postgresql://localhost:5432/coffeehouse")<br />
(: Obtain a handle to a prepared statement :)<br />
let $prep := sql:prepare($conn, "SELECT * FROM coffees WHERE price < ? AND cof_name = ?")<br />
(: Values and types of prepared statement parameters :)<br />
let $params := <sql:parameters><br />
<sql:parameter type='double'>10</sql:parameter><br />
<sql:parameter type='string'>French_Roast</sql:parameter><br />
</sql:parameters><br />
(: Execute prepared statement :)<br />
return sql:execute-prepared($prep, $params)<br />
</pre><br />
<br />
==SQLite==<br />
<br />
The following expression demonstrates how SQLite can be addressed with the [https://bitbucket.org/xerial/sqlite-jdbc/ Xerial SQLite JDBC driver]:<br />
<br />
<pre lang='xquery'><br />
(: Initialize driver :)<br />
sql:init("org.sqlite.JDBC"),<br />
(: Establish a connection :)<br />
let $conn := sql:connect("jdbc:sqlite:database.db")<br />
return (<br />
(: Create a new table :)<br />
sql:execute($conn, "drop table if exists person"),<br />
sql:execute($conn, "create table person (id integer, name string)"),<br />
(: Run 10 updates :)<br />
for $i in 1 to 10<br />
let $q := "insert into person values(" || $i || ", '" || $i || "')"<br />
return sql:execute($conn, $q),<br />
(: Return table contents :)<br />
sql:execute($conn, "select * from person")<br />
)<br />
</pre><br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
|Description<br />
|- valign="top"<br />
|{{Code|attribute}}<br />
|An attribute different from {{Code|type}} and {{Code|null}} is set for a {{Code|&lt;sql:parameter/&gt;}} element.<br />
|- valign="top"<br />
|{{Code|error}}<br />
|An SQL exception occurred.<br />
|- valign="top"<br />
|{{Code|id}}<br />
|A connection does not exist.<br />
|- valign="top"<br />
|{{Code|init}}<br />
|A database driver is not found.<br />
|- valign="top"<br />
|{{Code|parameters}}<br />
|No parameter type specified.<br />
|- valign="top"<br />
|{{Code|timeout}}<br />
|Query execution exceeded timeout.<br />
|- valign="top"<br />
|{{Code|type}}<br />
|The value of a parameter cannot be converted to the specified format.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 11<br />
* Updated: {{Function||sql:connect}}: Optional credentials.<br />
<br />
;Version 9.6<br />
* Updated: {{Function||sql:execute-prepared}}, additional types added<br />
<br />
;Version 9.0<br />
* Updated: {{Function||sql:execute}}, {{Function||sql:execute-prepared}}: Return update count for updating statements. {{Code|$options}} argument added.<br />
* Updated: Connection ids are URIs now.<br />
* Updated: error codes updated; errors now use the module namespace<br />
<br />
;Version 7.5<br />
* Updated: prepared statements are now executed via {{Function||sql:execute-prepared}}<br />
<br />
The module was introduced with Version 7.0.</div>CGhttps://docs.basex.org/index.php?title=File_Module&diff=16902File Module2023-12-01T16:39:40Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This [[Module Library|XQuery Module]] contains functions related to file system operations, such as listing, reading, or writing files.<br />
<br />
This module is based on the [http://expath.org/spec/file EXPath File Module]. The following enhancements have not been added to the specification yet:<br />
<br />
{| class="wikitable"<br />
|- valign="top"<br />
! Function<br />
! Description<br />
|- valign="top"<br />
| {{Function||file:descendants}}<br />
| new function<br />
|- valign="top"<br />
| {{Function||file:is-absolute}}<br />
| new function<br />
|- valign="top"<br />
| {{Function||file:read-text}}, {{Function||file:read-text-lines}}<br />
| <code>$fallback</code> argument added<br />
|- valign="top"<br />
| {{Function||file:read-text-lines}}<br />
| <code>$offset</code> and <code>$length</code> arguments added<br />
|- valign="top"<br />
| {{Function||file:resolve-path}}<br />
| <code>$base</code> argument added<br />
|}<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module are assigned to the <code><nowiki>http://expath.org/ns/file</nowiki></code> namespace, which is statically bound to the {{Code|file}} prefix.<br/><br />
<br />
For serialization parameters, the <code><nowiki>http://www.w3.org/2010/xslt-xquery-serialization</nowiki></code> namespace is used, which is statically bound to the {{Code|output}} prefix.<br/><br />
<br />
The error <code>[[#Errors|invalid-path]]</code> is raised if a path is invalid.<br />
<br />
==File Paths==<br />
<br />
* All file paths are resolved against the ''current working directory'' (the directory from which BaseX or, more generally, the Java Virtual Machine, was started). This directory can be retrieved via {{Function||file:base-dir}}.<br />
<br />
* A path can be specified as local filesystem path or as file URI.<br />
<br />
* Returned strings that refer to existing directories are suffixed with a directory separator.<br />
<br />
=Read Operations=<br />
<br />
==file:list==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:list(<br />
$dir as xs:string,<br />
$recursive as xs:boolean? := false(),<br />
$pattern as xs:string := ()<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Lists all files and directories found in the specified {{Code|$dir}}. The returned paths are relative to the provided path.<br/>If {{Code|$recursive}} is set to true, subdirectories will be traversed.<br/>The optional parameter {{Code|$pattern}} defines a file name pattern in the [[Commands#Glob_Syntax|Glob Syntax]]. If present, only those files and directories are returned that correspond to the pattern. Several patterns can be separated with a comma ({{Code|,}}).<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified file does not exist.<br/>{{Error|no-dir|#Errors}} the specified path does not point to a directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
| Return the contents of large {Code|.txt} files located in a specific directory and its subdirectories:<br />
<pre lang='xquery'><br />
let $root := 'path/to/files/'<br />
for $file in file:list($root, true(), '*.txt')<br />
let $path := $root || $file<br />
where file:size($path) > 1000000<br />
return file:read-text($path)<br />
</pre><br />
|}<br />
<br />
==file:children==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:children(<br />
$dir as xs:string<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the full paths to all files and directories found in the specified {{Code|$dir}}.<br/>The inverse function is {{Function||file:parent}}. The returned paths start with the specified directory. The related function {{Function||file:list}} returns relative file paths.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified file does not exist.<br/>{{Error|no-dir|#Errors}} the specified path does not point to a directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
| Return the contents of large {{Code|.txt}} files located in a specific directory:<br />
<pre lang='xquery'><br />
for $file in file:children('path/to/files/')<br />
where matches($file, '.txt', 'i') and file:size($file) > 1000000<br />
return file:read-text($$file)<br />
</pre><br />
|}<br />
<br />
==file:descendants==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:descendants(<br />
$dir as xs:string<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the full paths to all files and directories found in the specified {{Code|$dir}} and its subdirectories.<br/>. The returned paths start with the specified directory. The related function {{Function||file:list}} creates relative file paths.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified file does not exist.<br/>{{Error|no-dir|#Errors}} the specified path does not point to a directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
| Return the contents of all {{Code|.txt}} files located in a specific directory and its subdirectories:<br />
<pre lang='xquery'><br />
for $file in file:descendants('path/to/files/')<br />
where matches($file, '.txt', 'i') and file:size($file) > 1000000<br />
return file:read-text($$file)<br />
</pre><br />
|}<br />
<br />
==file:read-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:read-binary(<br />
$path as xs:string,<br />
$offset as xs:integer := (),<br />
$length as xs:integer := ()<br />
) as xs:base64Binary</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Reads the binary content of the file specified by {{Code|$path}} and returns it as [[Lazy Module|lazy]] {{Code|xs:base64Binary}} item.<br/>The optional parameters {{Code|$offset}} and {{Code|$length}} can be used to read chunks of a file.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified file does not exist.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|out-of-range|#Errors}} the offset or length is negative, or the chosen values would exceed the file bounds.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code><nowiki>lazy:cache(file:read-binary("config.data"))</nowiki></code> enforces the file access (otherwise, it will be delayed until requested first).<br />
|}<br />
<br />
==file:read-text==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:read-text(<br />
$path as xs:string,<br />
$encoding as xs:string := (),<br />
$fallback as xs:boolean? := false()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Reads the textual contents of the file specified by {{Code|$path}} and returns it as [[Lazy Module|lazy]] {{Code|xs:string}} item:<br />
* The UTF-8 default encoding can be overwritten with the optional {{Code|$encoding}} argument.<br />
* By default, invalid XML characters will be rejected. If {{Code|$fallback}} is enabled, the characters will be replaced with the Unicode replacement character <code>FFFD</code> (&#xFFFD;).<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified file does not exist.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|unknown-encoding|#Errors}} the specified encoding is not supported, or unknown.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code><nowiki>lazy:cache(file:read-text("ids.txt"))</nowiki></code> enforces the file access (otherwise, it will be delayed until requested first).<br />
|}<br />
<br />
==file:read-text-lines==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:read-text-lines(<br />
$path as xs:string,<br />
$encoding as xs:string := (),<br />
$fallback as xs:boolean? := false(),<br />
$offset as xs:integer? := (),<br />
$length as xs:integer? := ()<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Reads the textual contents of the file specified by {{Code|$path}} and returns it as a sequence of {{Code|xs:string}} items:<br />
* The UTF-8 default encoding can be overwritten with the optional {{Code|$encoding}} argument.<br />
* By default, invalid characters will be rejected. If {{Code|$fallback}} is set to true, these characters will be replaced with the Unicode replacement character <code>FFFD</code> (&#xFFFD;).<br />
The lines to be read can be restricted with the optional parameters {{Code|$offset}} and {{Code|$length}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified file does not exist.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|unknown-encoding|#Errors}} the specified encoding is not supported, or unknown.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
=Write Operations=<br />
<br />
==file:create-dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:create-dir(<br />
$dir as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates the directory specified by {{Code|$dir}} if it does not already exist. Non-existing parent directories will be created as well.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|exists|#Errors}} the specified target exists, but is no directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:create-temp-dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:create-temp-dir(<br />
$prefix as xs:string,<br />
$suffix as xs:string,<br />
$dir as xs:string := ()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a new temporary directory that did not exist before this function was called, and returns its full file path. The directory name begins and ends with the specified {{Code|$prefix}} and {{Code|$suffix}}. If no directory is specified via {{Code|$dir}}, the directory will be placed in the system’s default temporary directory. The operation will create all non-existing parent directories.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the specified directory points to a file.<br/>{{Error|io-error|#Errors}} the directory could not be created.<br />
|}<br />
<br />
==file:create-temp-file==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:create-temp-file(<br />
$prefix as xs:string,<br />
$suffix as xs:string,<br />
$dir as xs:string := ()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a new temporary file that did not exist before this function was called, and returns its full file path. The file name begins and ends with the specified {{Code|$prefix}} and {{Code|$suffix}}. If no directory is specified via {{Code|$dir}}, the file will be placed in the system’s default temporary directory. The operation will create all non-existing parent directories.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the specified directory points to a file.<br/>{{Error|io-error|#Errors}} the directory could not be created.<br />
|}<br />
<br />
==file:delete==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:delete(<br />
$path as xs:string,<br />
$recursive as xs:boolean? := false()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Recursively deletes a file or directory specified by {{Code|$path}}.<br/>The optional parameter {{Code|$recursive}} specifies whether subdirectories will be deleted, too.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified path does not exist.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:write==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:write(<br />
$path as xs:string,<br />
$input as item()*,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Writes serialized {{$input}} to the specified file. If the file already exists, it will be overwritten.<br/>The {{Code|$options}} argument contains [[Serialization|serialization parameters]]. As with [https://www.w3.org/TR/xpath-functions-31/#func-serialize fn:serialize()], the parameters can be specified<br/><br />
* either as children of an {{Code|&lt;output:serialization-parameters/&gt;}} element:<br />
<pre lang="xml"><br />
<output:serialization-parameters><br />
<output:method value='xml'/><br />
<output:cdata-section-elements value="div"/><br />
...<br />
</output:serialization-parameters><br />
</pre><br />
* or as map, which contains all key/value pairs:<br />
<pre lang='xquery'><br />
map { "method": "xml", "cdata-section-elements": "div", ... }<br />
</pre><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the parent of specified path is no directory.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code><nowiki>file:write('data.bin', xs:hexBinary('414243'))</nowiki></code> writes a hex representation to the specified file.<br />
* <code><nowiki>file:write('data.bin', xs:hexBinary('414243'), map { 'method': 'basex')</nowiki></code> writes binary data to the specified file (see [[XQuery_Extensions#Serialization|Serialization]] for more details).<br />
|}<br />
<br />
==file:write-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:write-binary(<br />
$path as xs:string,<br />
$value as xs:anyAtomicType,<br />
$offset as xs:integer := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Writes a binary item (xs:base64Binary, xs:hexBinary) to the specified file. If the file already exists, it will be overwritten.<br/>If {{Code|$offset}} is specified, data will be written at this file position. An existing file may be resized by that operation.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the parent of specified path is no directory.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|out-of-range|#Errors}} the offset is negative, or it exceeds the current file size.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:write-text==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:write-text(<br />
$path as xs:string,<br />
$value as xs:string,<br />
$encoding as xs:string := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Writes a string to the specified file. If the file already exists, it will be overwritten.<br/>The optional parameter {{Code|$encoding}} defines the output encoding (default: UTF-8).<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the parent of specified path is no directory.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|unknown-encoding|#Errors}} the specified encoding is not supported, or unknown.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:write-text-lines==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:write-text-lines(<br />
$path as xs:string,<br />
$values as xs:string*,<br />
$encoding as xs:string := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Writes a sequence of strings to the specified file, each followed by the system specific newline character. If the file already exists, it will be overwritten.<br/>The optional parameter {{Code|$encoding}} defines the output encoding (default: UTF-8).<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the parent of specified path is no directory.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|unknown-encoding|#Errors}} the specified encoding is not supported, or unknown.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:append==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:append(<br />
$path as xs:string,<br />
$input as item()*,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Appends a serialized sequence of {{Code|$input}} to the specified file, with the supplied {{Code|$options}} as serialization parameters. If the file does not exist, a new file is created.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the parent of specified path is no directory.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:append-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:append-binary(<br />
$path as xs:string,<br />
$value as xs:anyAtomicType<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Appends a binary item (xs:base64Binary, xs:hexBinary) to the specified file. If the file does not exists, a new one is created.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the parent of specified path is no directory.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:append-text==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:append-text(<br />
$path as xs:string,<br />
$value as xs:string,<br />
$encoding as xs:string := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Appends a string to a file specified by {{Code|$path}}. If the specified file does not exists, a new file is created.<br/>The optional parameter {{Code|$encoding}} defines the output encoding (default: UTF-8).<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the parent of specified path is no directory.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|unknown-encoding|#Errors}} the specified encoding is not supported, or unknown.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:append-text-lines==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:append-text-lines(<br />
$path as xs:string,<br />
$values as xs:string*,<br />
$encoding as xs:string := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Appends a sequence of strings to the specified file, each followed by the system specific newline character. If the specified file does not exists, a new file is created.<br/>The optional parameter {{Code|$encoding}} defines the output encoding (default: UTF-8).<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|no-dir|#Errors}} the parent of specified path is no directory.<br/>{{Error|is-dir|#Errors}} the specified path is a directory.<br/>{{Error|unknown-encoding|#Errors}} the specified encoding is not supported, or unknown.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:copy==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:copy(<br />
$source as xs:string,<br />
$target as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Copies a file or directory specified by {{Code|$source}} to the file or directory specified by {{Code|$target}}. If the target file already exists, it will be overwritten. No operation will be performed if the source and target path are equal.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified source does not exist.<br/>{{Error|exists|#Errors}} the specified source is a directory and the target is a file.<br/>{{Error|no-dir|#Errors}} the parent of the specified target is no directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|}<br />
<br />
==file:move==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:move(<br />
$source as xs:string,<br />
$target as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Moves or renames the file or directory specified by {{Code|$source}} to the path specified by {{Code|$target}}. If the target file already exists, it will be overwritten. No operation will be performed if the source and target path are equal.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified source does not exist.<br/>{{Error|exists|#Errors}} the specified source is a directory and the target is a file.<br/>{{Error|no-dir|#Errors}} the parent of the specified target is no directory.<br/>{{Error|io-error|#Errors}} the operation fails for some other reason.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|When renaming a file, remember that relative file paths are resolved against the current working directory. Some ways to rename:<br />
<pre lang='xquery'><br />
file:move('/projects/new/octopus', '/projects/new/septimus')<br />
</pre><br />
<pre lang='xquery'><br />
$path ! file:move(., file:parent(.)||$newName)<br />
</pre><br />
<br />
|}<br />
<br />
=File Properties=<br />
<br />
==file:exists==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:exists(<br />
$path as xs:string<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns an {{Code|xs:boolean}} indicating whether a file or directory specified by {{Code|$path}} exists in the file system.<br/><br />
|}<br />
<br />
==file:is-dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:is-dir(<br />
$path as xs:string<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns an {{Code|xs:boolean}} indicating whether the argument {{Code|$path}} points to an existing directory.<br/><br />
|}<br />
<br />
==file:is-absolute==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:is-absolute(<br />
$path as xs:string<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns an {{Code|xs:boolean}} indicating whether the argument {{Code|$path}} is absolute.<br/>The behavior of this function depends on the operating system: On Windows, an absolute path starts with the drive letter and a colon, whereas on Linux it starts with a slash.<br />
|}<br />
<br />
==file:is-file==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:is-file(<br />
$path as xs:string<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns an {{Code|xs:boolean}} indicating whether the argument {{Code|$path}} points to an existing file.<br/><br />
|}<br />
<br />
==file:last-modified==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:last-modified(<br />
$path as xs:string<br />
) as xs:dateTime</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Retrieves the timestamp of the last modification of the file or directory specified by {{Code|$path}}.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified path does not exist.<br/><br />
|}<br />
<br />
==file:size==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:size(<br />
$path as xs:string<br />
) as xs:integer</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the size, in bytes, of the file specified by {{Code|$path}}, or {{Code|0}} for directories.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified file does not exist.<br/><br />
|}<br />
<br />
=Path Functions=<br />
<br />
==file:name==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:name(<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the name of a file or directory specified by {{Code|$path}}. An empty string is returned if the path points to the root directory.<br />
|}<br />
<br />
==file:parent==<br />
<br />
{| width='100%'<br />
<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:parent(<br />
$path as xs:string<br />
) as xs:string?</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the absolute path to the parent directory of a file or directory specified by {{Code|$path}}. An empty sequence is returned if the path points to a root directory.<br/>The inverse function is {{Function||file:children}}.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code><nowiki>file:parent(static-base-uri())</nowiki></code> returns the directory of the current XQuery module.<br />
|}<br />
<br />
==file:path-to-native==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:path-to-native(<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Transforms the {{Code|$path}} argument to its native representation on the operating system.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|not-found|#Errors}} the specified file does not exist.<br/>{{Error|io-error|#Errors}} the specified path cannot be transformed to its native representation.<br/><br />
|}<br />
<br />
==file:resolve-path==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:resolve-path(<br />
$path as xs:string,<br />
$base as xs:string := ()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Transforms the {{Code|$path}} argument to an absolute operating system path.<br/>If the path is relative, and if an absolute {{Code|$base}} path is specified, it will be resolved against this path.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|is-relative|#Errors}} the specified base path is relative.<br/><br />
|- valign="top"<br />
| '''Examples'''<br />
|The following examples apply to Windows:<br />
* {{Code|file:resolve-path('file.txt', 'C:/Temp/')}} returns {{Code|C:/Temp/file.txt}}.<br />
* {{Code|file:resolve-path('file.txt', 'C:/Temp')}} returns {{Code|C:/file.txt}}.<br />
* {{Code|file:resolve-path('file.txt', 'Temp')}} raises an error.<br />
|}<br />
<br />
==file:path-to-uri==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:path-to-uri(<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Transforms the path specified by {{Code|$path}} into a URI with the {{Code|file://}} scheme.<br/><br />
|}<br />
<br />
=System Properties=<br />
<br />
==file:dir-separator==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|'''file:dir-separator'''() as xs:string}}<br/><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the directory separator used by the operating system, such as {{Code|/}} or {{Code|\}}.<br/><br />
|}<br />
<br />
==file:path-separator==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|'''file:path-separator'''() as xs:string}}<br/><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the path separator used by the operating system, such as {{Code|;}} or {{Code|:}}.<br/><br />
|}<br />
<br />
==file:line-separator==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:line-separator() as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the line separator used by the operating system, such as {{Code|&amp;#10;}}, {{Code|&amp;#13;&amp;#10;}} or {{Code|&amp;#13;}}.<br/><br />
|}<br />
<br />
==file:temp-dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:temp-dir() as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the system’s default temporary-file directory.<br/><br />
|}<br />
<br />
==file:current-dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:current-dir() as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the ''current working directory'', i.e., the directory from which the query processor was started. This function returns the same result as the function call <code>file:resolve-path("")</code>.<br />
|}<br />
<br />
==file:base-dir==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>file:base-dir() as xs:string?</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the parent directory of the static base URI.<br/>If the static base URI is undefined or does not point to a local resource, it returns the empty sequence. Otherwise, it returns the same result as {{Code|file:parent(static-base-uri())}}.<br />
|}<br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="160"|Code<br />
|Description<br />
|- valign="top"<br />
|{{Code|exists}}<br />
|A file with the same path already exists.<br />
|- valign="top"<br />
|{{Code|invalid-path}}<br />
|A specified path is invalid.<br />
|- valign="top"<br />
|{{Code|io-error}}<br />
|The operation fails for some other reason specific to the operating system.<br />
|- valign="top"<br />
|{{Code|is-dir}}<br />
|The specified path is a directory.<br />
|- valign="top"<br />
|{{Code|is-relative}}<br />
|The specified path is relative (and must be absolute).<br />
|- valign="top"<br />
|{{Code|no-dir}}<br />
|The specified path does not point to a directory.<br />
|- valign="top"<br />
|{{Code|not-found}}<br />
|A specified path does not exist.<br />
|- valign="top"<br />
|{{Code|out-of-range}}<br />
|The specified offset or length is negative, or the chosen values would exceed the file bounds.<br />
|- valign="top"<br />
|{{Code|unknown-encoding}}<br />
|The specified encoding is not supported, or unknown.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 9.3<br />
* Added: {{Function||file:descendants}}<br />
<br />
;Version 9.0<br />
* Updated: {{Function||file:read-text-lines}}: <code>$offset</code> and <code>$length</code> arguments added.<br />
<br />
;Version 8.5<br />
* Updated: {{Function||file:read-text}}, {{Function||file:read-text-lines}}: <code>$fallback</code> argument added.<br />
<br />
;Version 8.2<br />
* Added: {{Function||file:is-absolute}}<br />
* Updated: {{Function||file:resolve-path}}: base argument added<br />
<br />
;Version 8.0<br />
* Added: {{Function||file:current-dir}}, {{Function||file:base-dir}}, {{Function||file:children}}<br />
<br />
;Version 7.8<br />
* Added: {{Function||file:parent}}, {{Function||file:name}}<br />
* Updated: error codes; {{Function||file:read-binary}}, {{Function||file:write-binary}}: {{Code|$offset}} and {{Code|$length}} arguments added.<br />
* Deleted: file:base-name, file:dir-name<br />
<br />
;Version 7.7<br />
* Added: {{Function||file:create-temp-dir}}, {{Function||file:create-temp-file}}, {{Function||file:temp-dir}}<br />
* Updated: all returned strings that refer to existing directories will be suffixed with a directory separator.<br />
<br />
;Version 7.3<br />
* Added: {{Function||file:append-text}}, {{Function||file:write-text}}, {{Function||file:append-text-lines}}, {{Function||file:write-text-lines}}, {{Function||file:line-separator}}<br />
* Aligned with latest specification: $file:directory-separator → {{Function||file:dir-separator}}, $file:path-separator → {{Function||file:path-separator}}, file:is-directory → {{Function||file:is-dir}}, file:create-directory → {{Function||file:create-dir}}<br />
* Updated: {{Function||file:write-binary}}, {{Function||file:append-binary}}: output limited to a single value<br />
<br />
;Version 7.2.1<br />
* Updated: {{Function||file:delete}}: {{Code|$recursive}} parameter added to prevent subdirectories from being accidentally deleted.<br />
* Fixed: {{Function||file:list}} now returns relative instead of absolute paths.</div>CGhttps://docs.basex.org/index.php?title=BaseX_10&diff=16901BaseX 102023-12-01T16:39:35Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>After 15 years of continuous development, the first double-digit version of BaseX sees the light of day.<br />
<br />
We have taken the version jump as an opportunity to perform some major refactorings of BaseX, both under the hood and on API and XQuery level. Before migrating your projects to the new version, some adjustments may be required, so please read this article carefully.<br />
<br />
=Prerequisites=<br />
<br />
BaseX 10 requires Java 11 or later to run. Databases created with the new version are backward compatible and can still be opened with BaseX 9.<br />
<br />
=Migrating Applications=<br />
<br />
The following modifications might be relevant when migrating existing applications:<br />
<br />
* The default ports for web applications have been changed from 8984/8985 to 8080/8081.<br />
* The default admin password has been removed. The {{Code|admin}} user can only be used if a password has been assigned, e.g., via the {{Command|PASSWORD}} command.<br />
* The conventions for functions in [[Clients]] in other programming languages were revised.<br />
* The <code>IGNOREHOSTNAME</code> option was dropped and merged with {{Option|IGNORECERT}}.<br />
<br />
=Storage=<br />
<br />
==Whitespaces==<br />
<br />
All whitespaces are now preserved when importing XML resources, unless whitespace stripping is enabled.<br />
<br />
The notorious {{Code|CHOP}} option was removed to prevent conflicting behavior caused by earlier installations. It was replaced by a new {{Option|STRIPWS}} option, which defaults to {{Code|false}}. In addition, the new default of the [[Serialization|serialization parameter]] {{Code|indent}} is {{Code|no}}.<br />
<br />
Please be warned that the new default can throw off existing applications. If you want to restore the old behavior, you should attach the following lines in your [[Configuration|.basex configuration file]]…<br />
<br />
<pre lang="php"><br />
# Local Options<br />
STRIPWS = true<br />
SERIALIZER = indent=yes<br />
</pre><br />
<br />
…or add context parameters in the {{Code|web.xml}} file of your [[Web Application]]:<br />
<br />
<pre lang='xquery'><br />
<context-param><br />
<param-name>org.basex.stripws</param-name><br />
<param-value>true</param-value><br />
</context-param><br />
<context-param><br />
<param-name>org.basex.serializer</param-name><br />
<param-value>indent=yes</param-value><br />
</context-param><br />
</pre><br />
<br />
In the GUI editor, a [[Shortcuts#Editor Shortcuts|shortcut]] and an icon were added to switch result indentation on and off.<br />
<br />
In addition, databases may considerably increase in size, as whitespaces used for indenting an XML document will be interpreted and stored as additional text nodes. If your XML resources are structured and have no [[Full-Text#Mixed Content|mixed content]], it is advisable to enable whitespaces stripping when importing them to a database.<br />
<br />
Note that {{Option|STRIPWS}} only applies to stored XML and not to XML constructed within XQuery source files. Within XQuery code, the standard {{Code|boundary-space}} setting applies. This defaults to {{Code|strip}}. To preserve XML whitespace in XQuery modules, add the following to the prolog:<br />
<pre lang='xquery'><br />
declare boundary-space preserve; <br />
<xml> </xml><br />
</pre><br />
<br />
==Value Resources==<br />
<br />
In addition to XML and binary resources, a third resource type has been added: XQuery values (atomic items and nodes, sequences, maps, arrays) can now be stored in databases as well. The {{Function|Database|db:put-value}} and {{Function|Database|db:get-value}} can be used to store to and retrieve values.<br />
<br />
The new feature can e.g. be used to store maps in a database:<br />
<br />
<pre lang='xquery'><br />
db:put-value(<br />
'factbook',<br />
map:merge(<br />
for $country in db:get('factbook')//country<br />
return map:entry($country/@name, $country//city/name ! string())<br />
),<br />
'cities'<br />
)<br />
</pre><br />
<br />
…and use them as index later on:<br />
<br />
<pre lang='xquery'><br />
let $cities := db:get-value('factbook', 'cities')<br />
for $country in ('Japan', 'Indonesia', 'Malaysia')<br />
return $country || ': ' || string-join($cities?($country), ', ')<br />
</pre><br />
<br />
==Backups==<br />
<br />
The [[Commands#Backups|Backup Commands]] and [[Database_Module#Backups|Backup Functions]] were enhanced to back up general data: [[User Management|registered users]], [[Job Module#Services|scheduled services]], [[Store Module|key-value stores]].<br />
<br />
=XQuery=<br />
<br />
==Compilation==<br />
<br />
The compilation has been split up into multiple steps to improve locking.<br />
<br />
So far, several internal steps were already performed when executing a query (see [[XQuery Optimizations#Introduction|XQuery Optimizations]] for more details):<br />
<br />
# The query is parsed, i.e., the original query string is transformed to an executable tree representation.<br />
# External values that are passed on by APIs are bound to variables and the query context. External values can be names of databases, or contribute to a name that will later on be constructed in the query.<br />
# The query is compiled and evaluated.<br />
<br />
The [[Transaction_Management|transaction manager]] gathers the names of the databases that will be accessed by a query. If it is not possible to uniquely identify all databases that may be opened by the query, global locking will be applied, and all databases will be locked. Detection can fail if the names of databases depend on external input. It can also fail if a query is too complex to associate character strings with database operations.<br />
<br />
The compilation phase now comprises two separate steps:<br />
<br />
# ''Compilation'' of logical, context-independent (static) operations. External values are bound to the query, and deterministic code is rewritten, simplified and pre-evaluated.<br />
# ''Optimization'' of physical, context-based (dynamic) operations. Databases are opened and checked for available indexes; current date/time is retrieved. The resulting code is further rewritten and optimized.<br />
<br />
Lock detection will be performed after the first step, and the code resulting from this step offers much more insight into which specific databases need to be locked. As a result, local locks can be applied to many more queries than before, and many queries can now run in parallel. An example:<br />
<br />
<pre lang='xquery'><br />
declare variable $n external;<br />
db:get('names-' || $n)<br />
</pre><br />
<br />
After the query has been parsed, a user-specific value (e.g., {{Code|123}}) will be bound to {{Code|$n}}. The variable will be inlined by the compiler, and the argument of {{Function||db:get}} will be pre-evaluated to {{Code|names123}}. It is then easy for the lock detector to collect the name of the database that needs to read-locked before the query is eventually executed.<br />
<br />
Another positive side effect of two-step compilation is that productive environments get faster in general: Queries can be compiled in parallel, and it’s only the optimization and evaluation of a query that may need to be delayed by locking.<br />
<br />
==Main-Memory Updates==<br />
<br />
XQuery Update provides constructs to [[XQuery Update#Main-Memory Updates|update XML nodes in main memory]]. The data structures for in-memory representations of XML resources have been revised, such that updates can be performed orders of magnitudes faster than before. With BaseX 9.x, the following query runs for several minutes, whereas it can now be computed in a few seconds:<br />
<br />
<pre lang='xquery'><br />
<xml>{<br />
(1 to 1000000) ! <child/><br />
}</xml> update {<br />
for $child at $pos in child<br />
return insert node text { $pos } into $child<br />
}<br />
</pre><br />
<br />
==Key-Value Store==<br />
<br />
A new [[Store Module]] provides functions to organize values in a persistent main-memory key-value store. The store allows you to speed up access to frequently accessed data.<br />
<br />
; Store data:<br />
<br />
<pre lang='xquery'><br />
let $email := map:merge(<br />
for $address in db:get('addressbook')//address<br />
return map:entry($address/name, $address/email)<br />
)<br />
return store:put('emails', $email)<br />
</pre><br />
<br />
; Retrieve data:<br />
<br />
<pre lang='xquery'><br />
let $name := 'Richard David James'<br />
return store:get('email')($name)<br />
</pre><br />
<br />
The store is persistent: Its contents are written to disk if BaseX is shut down, and retrieved again after a restart.<br />
<br />
==Modules==<br />
<br />
Functions of all modules, excluding the [[File Module]], now consistently resolve relative URI references against the static base URI, and not the current working directory.<br />
<br />
Various modules and functions have been revised, added, renamed or removed:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Description<br />
! BaseX 10<br />
! BaseX 9<br />
|-<br />
| Retrieve XML resources<br />
| {{Function|Database|db:get}}<br />
| {{Code|db:open}}<br />
|-<br />
| Retrieve nodes with specified pre values<br />
| {{Function|Database|db:get-pre}}<br />
| {{Code|db:open-pre}}<br />
|-<br />
| Retrieve nodes with specified IDs<br />
| {{Function|Database|db:get-id}}<br />
| {{Code|db:open-id}}<br />
|-<br />
| Retrieve binary resources<br />
| {{Function|Database|db:get-binary}}<br />
| {{Code|db:retrieve}}<br />
|-<br />
| Retrieve value resources<br />
| {{Function|Database|db:get-value}}<br />
| ''new''<br />
|-<br />
| Add or replace resource<br />
| {{Function|Database|db:put}}, arguments swapped!<br />
| {{Code|db:replace}}<br />
|-<br />
| Add or replace binary resource<br />
| {{Function|Database|db:put-binary}}, arguments swapped!<br />
| {{Code|db:store}}<br />
|-<br />
| Add or replace value resource<br />
| {{Function|Database|db:put-value}}<br />
| ''new''<br />
|-<br />
| Get resource type<br />
| {{Function|Database|db:type}}<br />
| {{Code|db:is-raw}}, {{Code|db:is-xml}}<br />
|-<br />
| Fetch XML document<br />
| {{Function|Fetch|fetch:doc}}<br />
| {{Code|fetch:xml}}<br />
|-<br />
| Convert binary data to XML<br />
| {{Function|Fetch|fetch:binary-doc}}<br />
| {{Code|fetch:xml-binary}}<br />
|-<br />
| Module: Process Geo data<br />
| ''removed''<br />
| Geo Module<br />
|-<br />
| XQuery jobs<br />
| [[Job Module]]<br />
| Jobs Module<br />
|-<br />
| Return variable bindings of a job<br />
| {{Function|Job|job:bindings}}<br />
| ''new''<br />
|-<br />
| Return variable bindings of a job<br />
| {{Function|Job|job:remove}}<br />
| {{Code|jobs:stop}}<br />
|-<br />
| Module: Main-memory key-value store<br />
| [[Store Module]]<br />
| ''new''<br />
|-<br />
| Module: String computations<br />
| [[String Module]]<br />
| Strings Module<br />
|-<br />
| Format string<br />
| {{Function|String|string:format}}<br />
| {{Code|out:format}}<br />
|-<br />
| Return control characters<br />
| {{Function|String|string:cr}}, {{Function|String|string:nl}}, {{Function|String|string:tab}}<br />
| {{Code|out:cr}}, {{Code|out:nl}}, {{Code|out:tab}}<br />
|-<br />
| Module: Process ZIP files<br />
| ''removed''<br />
| ZIP Module<br />
|}<br />
<br />
=Commands=<br />
<br />
The following commands have been revised:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Description<br />
! BaseX 10<br />
! BaseX 9<br />
|-<br />
| List directories and resources.<br />
| {{Command|DIR}}<br />
| ''new''<br />
|-<br />
| Retrieve single XML document<br />
| {{Command|GET}}<br />
| ''new''<br />
|-<br />
| Retrieve binary resource<br />
| {{Command|BINARY GET}}<br />
| {{Code|RETRIEVE}}<br />
|-<br />
| Add or replace resources<br />
| {{Command|PUT}}<br />
| {{Code|REPLACE}}<br />
|-<br />
| Store binary resource<br />
| {{Command|BINARY PUT}}<br />
| {{Code|STORE}}<br />
|-<br />
| Returns current option values<br />
| {{Command|SHOW OPTIONS}}<br />
| {{Code|GET}}<br />
|-<br />
| Lists jobs<br />
| ''removed''<br />
| {{Command|JOBS LIST}}<br />
|-<br />
| Returns a job result<br />
| ''removed''<br />
| {{Command|JOBS RESULT}}<br />
|-<br />
| Stops a job<br />
| ''removed''<br />
| {{Command|JOBS STOP}}<br />
|}<br />
<br />
=HTTP Requests=<br />
<br />
HTTP requests in BaseX are now based on the new [https://openjdk.org/groups/net/httpclient/intro.html Java HTTP Client]. This client provides a better overall performance, uses internal connection pools and follows redirects across different protocols (http, https).<br />
<br />
HTTP operations are, among others, performed by:<br />
<br />
* the [[HTTP Client Module]];<br />
* the [[Fetch Module]], [[Database Module]], [[Fetch Module]], [[Validation Module]], [[XSLT Module]] or [[Repository Module]];<br />
* <code>fn:doc</code> and <code>fn:collection</code>;<br />
* the {{Command|CREATE DB}} and {{Command|REPO INSTALL}} commands.<br />
<br />
=Catalogs=<br />
<br />
From early on, catalog resolvers had been neglected both in BaseX and Java. This has changed: The new [https://docs.oracle.com/en/java/javase/11/core/xml-catalog-api1.html XML Catalog API] from Java is universally used to resolve references to external resources. As an alternative, Norman Walsh’s [https://xmlresolver.org/ Enhanced XML Resolver] is utilized if it is found in the classpath. <br />
<br />
The option for supplying the XML catalog was renamed from <code>CATFILE</code> to {{Option|CATALOG}}. See [[Catalog Resolver]] for more details.<br />
<br />
=Graphical User Interface=<br />
<br />
The graphical user interface of BaseX has been revised and made more consistent.<br />
<br />
The icons were replaced by scalable ones, building upon the [https://openjdk.org/jeps/263 HiDPI graphics support for Windows and Linux].<br />
<br />
=REST=<br />
<br />
Results in the {{Code|rest}} namespace are now returned without prefix:<br />
<br />
<pre lang="xml"><br />
<!-- before --><br />
<rest:databases xmlns:rest="http://basex.org/rest"/><br />
<br />
<!-- now --><br />
<databases xmlns="http://basex.org/rest"/><br />
</pre><br />
<br />
When listing the resources of a database, {{Code|dir}} elements are returned for resources that are located in subdirectories. See [[REST]] for more details.</div>CGhttps://docs.basex.org/index.php?title=XQuery_Update&diff=16899XQuery Update2023-12-01T16:39:34Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This article is part of the [[XQuery|XQuery Portal]].<br />
It summarizes the update features of BaseX.<br />
<br />
BaseX offers a complete implementation of the [https://www.w3.org/TR/xquery-update-10/ XQuery Update Facility (XQUF)]. This article aims to provide a quick and basic introduction to XQUF. First, some examples for update expressions are given. Next, the challenges are addressed that arise due to the functional semantics of the language.<br />
<br />
=Features=<br />
<br />
==Updating Expressions==<br />
<br />
There are five new expressions to modify data. While {{Code|insert}}, {{Code|delete}}, {{Code|rename}} and {{Code|replace}} are basically self-explanatory, the {{Code|transform}} expression is different, as modified nodes are copied in advance and the original databases remain untouched.<br />
<br />
An expression consists of one or more target nodes (the nodes we want to alter) and (depending on the expression type) additional information like nodes to be inserted, a QName, etc., and optional modifiers. You can find a few examples and additional information below.<br />
<br />
===insert===<br />
<br />
<pre lang='xquery'><br />
insert node (attribute { 'a' } { 5 }, 'text', <e/>) into /n<br />
</pre><br />
<br />
Insert enables you to insert a sequence of nodes into a single target node. Several modifiers are available to specify the exact insert location: insert into '''as first'''/'''as last''', insert '''before'''/'''after''' and insert '''into'''.<br />
<br />
''Note'': in most cases, '''as last''' and '''after''' are evaluated faster than '''as first''' and '''before'''.<br />
<br />
===delete===<br />
<br />
<pre lang='xquery'><br />
delete node //n<br />
</pre><br />
<br />
The example query deletes all <code><n></code> elements in your database. In contrast to other updating expressions, multiple nodes can be supplied as a target.<br />
<br />
===replace===<br />
<br />
<pre lang='xquery'><br />
replace node /n with <a/><br />
</pre><br />
<br />
The target element is replaced by the DOM node <code><a/></code>. You can also replace the value of a node and its descendants by using the modifier '''value of''':<br />
<br />
<pre lang='xquery'><br />
replace value of node /n with 'newValue'<br />
</pre><br />
<br />
All descendants of /n are deleted, and the supplied text is inserted as the only child. The result of the insert sequence is either a single text node or an empty sequence. If the insert sequence is empty, all descendants of the target are deleted. Consequently, replacing the value of a node leaves the target with either a single text node or no descendants at all.<br />
<br />
===rename===<br />
<br />
<pre lang='xquery'><br />
for $n in //originalNode<br />
return rename node $n as 'renamedNode' <br />
</pre><br />
<br />
All <code>originalNode</code> elements are renamed. A loop can be used to modify multiple nodes within a single statement. Nodes on the {{Code|descendant}} or {{Code|attribute}} axis of the target are not affected.<br />
<br />
==Main-Memory Updates==<br />
<br />
With the following expressions, copies of nodes are created, which can then be modified with the already presented updating expressions. As the original node will not be changed, the expressions are called ''non-updating''.<br />
<br />
===copy/modify/return===<br />
<br />
<pre lang='xquery'><br />
copy $c := doc('example.xml')//originalNode<br />
modify rename node $c as 'copyOfNode'<br />
return $c<br />
</pre><br />
<br />
A copy of the {{Code|originalNode}} element is created, renamed and returned; the original document will not be updated. <br />
<br />
In the following example, multiple update operations are performed on the copied node:<br />
<br />
;Query:<br />
<br />
<pre lang='xquery'><br />
copy $c :=<br />
<entry><br />
<title>Transform expression example</title><br />
<author>BaseX Team</author><br />
</entry><br />
modify (<br />
replace value of node $c/author with 'BaseX',<br />
replace value of node $c/title with concat('Copy of: ', $c/title),<br />
insert node <author>Joey</author> into $c<br />
)<br />
return $c<br />
</pre><br />
<br />
;Result:<br />
<br />
<pre lang="xml"><br />
<entry><br />
<title>Copy of: Transform expression example</title><br />
<author>BaseX</author><br />
<author>Joey</author><br />
</entry><br />
</pre><br />
<br />
Instead of the main-memory {{Code|<entry>}} element, a database node can be supplied:<br />
<br />
<pre lang='xquery'><br />
copy $c := (db:get('example')//entry)[1]<br />
...<br />
</pre><br />
<br />
In this case, the database node remains untouched, as all updates are performed on the node copy.<br />
<br />
Entire documents can be copied and modified:<br />
<br />
<pre lang='xquery'><br />
copy $doc := doc("zaokeng.kml")<br />
modify (<br />
for $point in $doc//*:Point<br />
return insert node (<br />
<extrude>1</extrude>,<br />
<altitudeMode>relativeToGround</altitudeMode><br />
) before $point/*:coordinates<br />
)<br />
return $doc<br />
</pre><br />
<br />
===update===<br />
<br />
The {{Code|update}} expression is a BaseX-specific convenience operator for the bulky {{Code|copy/modify/return}} construct. Similar to the [[XQuery 3.0#Simple Map Operator|XQuery 3.0 map operator]], the nodes resulting from the first expression are bound as context items, and the bracketed expressions performs updates on the item. The updated nodes is returned as result:<br />
<br />
<pre lang='xquery'><br />
for $item in db:get('data')//item<br />
return $item update {<br />
delete node ./text()<br />
}<br />
</pre><br />
<br />
If multiple nodes are supplied as input, the updates will subsequently be performed on each node:<br />
<br />
<pre lang='xquery'><br />
db:get('data')//item update {<br />
delete node text()<br />
}<br />
</pre><br />
<br />
It is easy to chain subsequent update expressions:<br />
<br />
<pre lang='xquery'><br />
<root/> update {<br />
insert node <child/> into .<br />
} update {<br />
insert node "text" into child<br />
}<br />
</pre><br />
<br />
===transform with===<br />
<br />
The {{Code|transform with}} expression was added to the current [https://www.w3.org/TR/xquery-update-30/#id-transform-with XQuery Update 3.0] working draft. It is a simplified version of the [[#update|update]] expression (it is limited to single input nodes and cannot be chained):<br />
<br />
<pre lang='xquery'><br />
<xml>text</xml> transform with {<br />
replace value of node . with 'new-text'<br />
}<br />
</pre><br />
<br />
==Functions==<br />
<br />
===Built-in Functions===<br />
<br />
Numerous [[Database Module#Updates|Database Functions]] exist in BaseX for performing document- and database-wide updates.<br />
<br />
XQUF provides a single function {{Code|fn:put()}} for serializing nodes to secondary storage:<br />
<br />
* The function will be executed after all other updates.<br />
* Serialized documents therefore reflect all changes made effective during a query.<br />
* No files will be created if the addressed nodes have been deleted.<br />
* Serialization parameters can be specified as third argument (more details are found in the [https://www.w3.org/TR/xquery-update-30/#id-func-put XQUF 3.0 Specification]).<br />
<br />
If you want to write intermediate results to files, it is more flexible to use {{Function|File|file:write}}.<br />
<br />
===User-Defined Functions===<br />
<br />
Functions that performs updates need to be marked with an {{Code|%updating}} annotation:<br />
<br />
<pre lang='xquery'><br />
declare %updating function local:add($target, $node) {<br />
insert node $node into $target<br />
};<br />
<br />
<node/> update {<br />
local:add(., <sub/>)<br />
}<br />
</pre><br />
<br />
If update operations are defined in an anonymous function, it may be necessary to call the function with an additional {{Code|updating}} keyword:<br />
<br />
<pre lang='xquery'><br />
let $add := %updating function($target, $node) {<br />
insert node $node into $target<br />
}<br />
return <node/> update {<br />
updating $add(., <sub/>)<br />
}<br />
</pre><br />
<br />
=Concepts=<br />
<br />
In addition to the '''simple expression''', XQUF introduced '''updating expressions''':<br />
<br />
* All existing expressions are simple expressions. If such an expression is evaluated, the result is a sequence of items.<br />
* Updating expressions, which are presented in this article, result in a list of update primitives that are added to the '''Pending Update List'''.<br />
<br />
==Pending Update List==<br />
<br />
Updating statements are not executed immediately, but are first collected as update primitives within a set-like structure, the so-called Pending Update List (PUL). After the evaluation of the query, and after some consistency checks and optimizations, the update primitives will be applied in the following order:<br />
<br />
* '''Backups, Binary resources''': {{Function|Database|db:alter-backup}}, {{Function|Database|db:create-backup}}, {{Function|Database|db:put-value}}, {{Function|Database|db:put-binary}}<br />
* '''XQuery Update''': {{Code|insert before}}, {{Code|delete}}, {{Code|replace}}, {{Code|rename}}, {{Code|replace value}}, {{Code|insert attribute}}, {{Code|insert into first}}, {{Code|insert into}}, {{Code|insert into last}}, {{Code|insert}}, {{Code|insert after}}, {{Code|fn:put}}<br />
* '''Documents''': {{Function|Database|db:add}}, {{Function|Database|db:put}}, {{Function|Database|db:rename}}, {{Function|Database|db:delete}}, {{Function|Database|db:optimize}}, {{Function|Database|db:flush}},<br />
* '''Users''': {{Function|User|user:grant}}, {{Function|User|user:password}}, {{Function|User|user:drop}}, {{Function|User|user:alter}}, {{Function|User|user:create}}<br />
* '''Databases''': {{Function|Database|db:copy}}, {{Function|Database|db:drop}}, {{Function|Database|db:alter}}, {{Function|Database|db:create}}<br />
* '''Backups''': {{Function|Database|db:restore}}, {{Function|Database|db:drop-backup}}<br />
<br />
If an inconsistency is found, an error message is returned and all accessed databases remain untouched (ensuring atomicity). For the user, this means that updates are only visible '''after''' the end of a snapshot.<br />
<br />
It may be surprising to see <code>[[Database Module|db:create]]</code> in the lower part of this list. This means that a newly created database cannot be accessed by the same query, which can be explained by the semantics of updating queries: all expressions can only be evaluated on databases that already exist while the query is evaluated. As a consequence, {{Code|db:create}} is mainly useful in the context of [[Commands#Basics|Command Scripts]], or [[Web Application]]s, in which a redirect to another page can be triggered after having created a database.<br />
<br />
===Example===<br />
<br />
The query…<br />
<br />
<pre lang='xquery'><br />
insert node <b/> into /doc,<br />
/doc/* ! (rename node . as 'renamed')<br />
</pre><br />
<br />
…applied on the document…<br />
<br />
<pre lang="xml"><br />
<doc> <a/> </doc><br />
</pre><br />
<br />
…results in the following document:<br />
<br />
<pre lang="xml"><br />
<doc> <renamed/><b/> </doc><br />
</pre><br />
<br />
Despite explicitly renaming all child nodes of {{Code|<doc/>}}, the former {{Code|<a/>}} element is the only one to be renamed. The {{Code|<b/>}} element is inserted within the same snapshot and is therefore not yet visible to the user.<br />
<br />
==Returning Results==<br />
<br />
By default, it is not possible to mix different types of expressions in a query result. The root expression of a query must be a sequence of updating expressions. But there are two ways out:<br />
<br />
* The BaseX-specific {{Function|Update|update:output}} function bridges this gap: it caches the results of its arguments at runtime and returns them after all updates have been processed. The following example performs an update and returns a success message:<br />
<br />
<pre lang='xquery'><br />
update:output("Update successful."), insert node <c/> into doc('factbook')/mondial<br />
</pre><br />
<br />
* With {{Option|MIXUPDATES}}, all updating constraints will be turned off. Returned nodes will be copied before they are modified by updating expressions. An error is raised if items are returned within a transform expression.<br />
<br />
If you want to modify nodes in main memory, you can use the [[Update#transform|transform expression]].<br />
<br />
==Effects== <br />
<br />
==Original Files==<br />
<br />
In BaseX, all updates are performed on database nodes or in main memory. By default, update operations do not affect the original input file (the info string "Updates are not written back" appears in the query info to indicate this). The following solutions exist to write XML documents and binary resources to disk:<br />
<br />
* Updates on main-memory instances of files that have been retrieved via {{Code|fn:doc}} or {{Code|fn:collection}} will be propagated back to disk if {{Option|WRITEBACK}} is turned on. This option can also be activated on [[Command-Line Options#BaseX Standalone|command line]] via <code>-u</code>. Make sure you back up the original documents before running your queries.<br />
* Functions like {{Code|fn:put}} or {{Function|File|file:write}} can be used to write single XML documents to disk. With {{Function|File|file:write-binary}}, you can write binary resources.<br />
* The {{Command|EXPORT}} command can be used write all resources of a databases to disk.<br />
<br />
==Indexes==<br />
<br />
Index structures are discarded after update operations when {{Option|UPDINDEX}} is turned off (which is the default).<br />
More details are found in the article on [[Index#Updates|Indexing]].<br />
<br />
=Error Messages=<br />
<br />
Along with the Update Facility, a number of new error codes and messages have been added to the specification and BaseX. All errors are listed in the [[XQuery Errors#Update Errors|XQuery Errors]] overview.<br />
<br />
Please remember that the collected updates will be executed after the query evaluation. All logical errors will be raised before the updates are actually executed.<br />
<br />
=Changelog=<br />
<br />
;Version 10.0<br />
* Updated: {{Function|Database|db:put-binary}} is executed before XQuery Update expressions.<br />
* Updated: [[#update|update]]: Curly braces are now mandatory.<br />
<br />
;Version 9.0<br />
* Updated: [[#Built-in Functions|Built-in Functions]]: serialization parameters<br />
<br />
;Version 8.5<br />
* Added: [[#transform with|transform with]]<br />
* Updated: [[#update|update]] was extended.<br />
<br />
;Version 8.0<br />
* Added: <code>MIXUPDATES</code> option for [[#Returning Results|Returning Results]] in updating expressions<br />
* Added: information message if files are not written back<br />
<br />
;Version 7.8<br />
* Added: [[#update|update]] convenience operator</div>CGhttps://docs.basex.org/index.php?title=Serialization&diff=16900Serialization2023-12-01T16:39:34Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This page is part of the [[XQuery|XQuery Portal]].<br />
<br />
Serialization parameters define how XQuery items and XML nodes will be ''serialized'' (i.e., returned to the client or an API, usually in textual form). The official parameters are defined in the [https://www.w3.org/TR/xslt-xquery-serialization-31 W3C XQuery Serialization 3.1] document. In BaseX, they can be:<br />
<br />
* included in the [[XQuery_3.0#Serialization|prolog of the XQuery expression]];<br />
* specified in XQuery functions ({{Function|File|file:write}}, {{Function|Database|db:export}}, [https://www.w3.org/TR/xpath-functions-31/#func-serialize fn:serialize()]);<br />
* specified in [[REST#Parameters|REST query parameters]];<br />
* specified in [[RESTXQ#Output|RESTXQ output annotations]];<br />
* set via the {{Option|SERIALIZER}} option before running a query;<br />
* set via the {{Option|EXPORTER}} option before exporting a database; or<br />
* supplied with the {{Code|-s}} flag of the BaseX [[Command-Line Options#BaseX Standalone|command-line]] clients.<br />
<br />
The namespace for serialization parameters is statically bound to the {{Code|output}} prefix. This means that it need not (but may) be declared in the query prolog:<br />
<br />
<pre lang='xquery'><br />
declare namespace output = 'http://www.w3.org/2010/xslt-xquery-serialization';<br />
declare option output:method 'text';<br />
<xml>Hi there</xml><br />
</pre><br />
<br />
Due to the wide range of ways how parameters can be supplied, we deliberately ignored one rule of the specification, which requires non-official features to be defined in a non-null namespace URI. In the following, we will indicate which features are specific to our implementation.<br />
<br />
=Standard Parameters=<br />
<br />
The following serialization parameters are supported by BaseX (further details can be looked up in the official specification):<br />
<br />
{| class="wikitable sortable" width="100%"<br />
|- valign="top"<br />
! width="140" | Parameter<br />
! Description<br />
! Allowed<br />
! Default<br />
|- valign="top"<br />
| {{Code|method}}<br />
| Specifies the serialization method. {{Code|xml}}, {{Code|xhtml}}, {{Code|html}}, {{Code|text}} and {{Code|adaptive}} are part of the official specification. For more details on {{Code|basex}}, {{Code|csv}} and {{Code|json}}, see [[XQuery Extensions#Serialization|XQuery Extensions]].<br />
| {{Code|xml}}, {{Code|xhtml}}, {{Code|html}}, {{Code|text}}, {{Code|json}}, {{Code|adaptive}}, {{Code|csv}}, {{Code|basex}}<br />
| {{Code|basex}}<br />
|- valign="top"<br />
| {{Code|version}}<br />
| Specifies the version of the serialization method.<br />
| xml/xhtml: {{Code|1.0}}, {{Code|1.1}}<br/>html: {{Code|4.0}}, {{Code|4.01}}, {{Code|5.0}}<br/><br />
| {{Code|1.0}}<br />
|- valign="top"<br />
| {{Code|html-version}}<br />
| Specifies the version of the HTML serialization method.<br />
| {{Code|4.0}}, {{Code|4.01}}, {{Code|5.0}}<br />
| {{Code|4.0}}<br />
|- valign="top"<br />
| {{Code|item-separator}}<br />
| Determines a string to be used as item separator. If a separator is specified, the default separation of atomic values with single whitespaces will be skipped.<br />
| ''string''<br />
|<br />
|- valign="top"<br />
| {{Code|encoding}}<br />
| Encoding to be used for outputting the data.<br />
| ''[https://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html all encodings supported by Java]''<br />
| {{Code|UTF-8}}<br />
|- valign="top"<br />
| {{Code|indent}}<br />
| Adds leading whitespaces to make the output more readable.<br />
| {{Code|yes}}, {{Code|no}}<br />
| {{Code|no}}<br />
|- valign="top"<br />
| {{Code|cdata-section-elements}}<br />
| List of elements to be output as CDATA, separated by whitespaces.<br/>Example: {{Code|&lt;text&gt;&lt;![CDATA[ &lt;&gt; ]]&gt;&lt;/text&gt;}}<br />
| ''string''<br />
| <br />
|- valign="top"<br />
| {{Code|omit-xml-declaration}}<br />
| Omits the XML declaration, which is serialized before the actual query result<br/>Example: <code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;</code><br />
| {{Code|yes}}, {{Code|no}}<br />
| {{Code|yes}}<br />
|- valign="top"<br />
| {{Code|standalone}}<br />
| Prints or omits the {{Code|standalone}} attribute in the XML declaration.<br />
| {{Code|yes}}, {{Code|no}}, {{Code|omit}}<br />
| {{Code|omit}}<br />
|- valign="top"<br />
| {{Code|doctype-system}}<br />
| Introduces the output with a document type declaration and the given system identifier.<br/>Example: {{Code|&lt;!DOCTYPE x SYSTEM "entities.dtd"&gt;}}<br />
| ''string''<br />
|<br />
|- valign="top"<br />
| {{Code|doctype-public}}<br />
| If {{Code|doctype-system}} is specified, adds a public identifier.<br/>Example: {{Code|&lt;!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "<nowiki>http://www.w3.org/TR/html4/strict.dtd</nowiki>"&gt;}}<br />
| ''string''<br />
|<br />
|- valign="top"<br />
| {{Code|undeclare-prefixes}}<br />
| Undeclares prefixes in XML 1.1.<br />
| {{Code|yes}}, {{Code|no}}<br />
| {{Code|no}}<br />
|- valign="top"<br />
| {{Code|normalization-form}}<br />
| Specifies a normalization form. BaseX supports Form C ({{Code|NFC}}).<br />
| {{Code|NFC}}, {{Code|none}}<br />
| {{Code|NFC}}<br />
|- valign="top"<br />
| {{Code|media-type}}<br />
| Specifies the media type.<br />
| ''string''<br />
| {{Code|application/xml}}<br />
|- valign="top"<br />
| {{Code|use-character-maps}}<br />
| Defines character mappings. If mappings are supplied as a single string, keys and values are separated by the equal sign, and multiple pairs are separated by commas. Separators that are to be defined as keys or values can be encoded as entities.<br/>Example: <code>A=alpha,B=beta</code><br />
| ''string''<br />
| <br />
|- valign="top"<br />
| {{Code|byte-order-mark}}<br />
| Prints a byte-order-mark before starting serialization.<br />
| {{Code|yes}}, {{Code|no}}<br />
| {{Code|no}}<br />
|- valign="top"<br />
| {{Code|escape-uri-attributes}}<br />
| Escapes URI information in certain HTML attributes<br/>Example: <code>&lt;a&nbsp;href="%C3%A4%C3%B6%C3%BC"&gt;äöü&lt;a&gt;</code><br />
| {{Code|yes}}, {{Code|no}}<br />
| {{Code|no}}<br />
|- valign="top"<br />
| {{Code|include-content-type}}<br />
| Inserts a {{Code|meta}} content-type element into the head element if the result is output as HTML<br/>Example: <code>&lt;head&gt;&lt;meta http-equiv="Content-Type" content="text/html; charset=UTF-8"&gt;&lt;/head&gt;</code>. The head element must already exist or nothing will be added. Any existing {{Code|meta}} content-type elements will be removed.<br />
| {{Code|yes}}, {{Code|no}}<br />
| {{Code|yes}}<br />
|}<br />
<br />
=Query Prolog Parameters=<br />
<br />
The following parameters can only be specified in the query prolog (see [https://www.w3.org/TR/xquery-31/#id-serialization XQuery: Specification] for more details and examples):<br />
<br />
{| class="wikitable sortable" width="100%"<br />
|- valign="top"<br />
! width="140" | Parameter<br />
! Description<br />
! Allowed<br />
! Default<br />
|- valign="top"<br />
| {{Code|parameter-document}}<br />
| Parses the value as an XML document with additional serialization parameters.<br />
| ''string''<br />
| <br />
|}<br />
<br />
=Custom Parameters=<br />
<br />
BaseX provides some additional serialization parameters:<br />
<br />
{| class="wikitable sortable" width="100%"<br />
|- valign="top"<br />
! width="140" | Parameter<br />
! Description<br />
! Allowed<br />
! Default<br />
|- valign="top"<br />
| {{Code|csv}}<br />
| Defines the way how data is serialized as CSV.<br />
| see [[CSV Module]]<br />
|<br />
|- valign="top"<br />
| {{Code|json}}<br />
| Defines the way how data is serialized as JSON.<br />
| see [[JSON Module]]<br />
| <br />
|- valign="top"<br />
| {{Code|tabulator}}<br />
| Uses tab characters ({{Code|\t}}) instead of spaces for indenting elements.<br />
| {{Code|yes}}, {{Code|no}}<br />
| {{Code|no}}<br />
|- valign="top"<br />
| {{Code|indents}}<br />
| Specifies the number of characters to be indented.<br />
| ''positive number''<br />
| {{Code|2}}<br />
|- valign="top"<br />
| {{Code|indent-attributes}}<br />
| {{Announce|Version 11:}} Indent attributes after the element name, one below the other.<br />
| {{Code|yes}}, {{Code|no}}<br />
| {{Code|false}}<br />
|- valign="top"<br />
| {{Code|newline}}<br />
| Specifies the type of newline to be used as end-of-line marker. <br />
| {{Code|\n}}, {{Code|\r\n}}, {{Code|\r}}<br />
| ''system dependent''<br />
|- valign="top"<br />
| {{Code|limit}}<br />
| Stops serialization after the specified number of bytes has been serialized. If a negative number is specified, everything will be output.<br />
| ''positive number''<br />
| {{Code|-1}}<br />
|- valign="top"<br />
| {{Code|binary}}<br />
| Indicates if items of binary type are output in their native byte representation. Only applicable to the <code>base</code> serialization method.<br />
| {{Code|yes}}, {{Code|no}}<br />
| {{Code|yes}}<br />
|}<br />
<br />
The {{Code|csv}} and {{Code|json}} parameters are supplied with a list of options. Option names and values are combined with <code>=</code>, several options are separated by <code>,</code>:<br />
<br />
<pre lang='xquery'><br />
declare option output:method "csv";<br />
declare option output:csv "header=yes, separator=semicolon";<br />
<csv><br />
<record><br />
<Name>John</Name><br />
<City>Newton</City><br />
</record><br />
<record><br />
<Name>Jack</Name><br />
<City>Oldtown</City><br />
</record><br />
</csv><br />
</pre><br />
<br />
If {{Code|fn:serialize}} is called, output-specific parameters can be supplied via nested options:<br />
<br />
<pre lang='xquery'><br />
serialize(<br />
<csv><br />
<record><br />
<Name>John</Name><br />
<City>Newton</City><br />
</record><br />
<record><br />
<Name>Jack</Name><br />
<City>Oldtown</City><br />
</record><br />
</csv>,<br />
map {<br />
'method': 'csv',<br />
'csv': map { 'header': 'yes', 'separator': ';' }<br />
}<br />
)<br />
</pre><br />
<br />
'''Result''':<br />
<pre lang="xml"><br />
Name;City<br />
John;Newton<br />
Jack;Oldtown<br />
</pre><br />
<br />
=Character mappings=<br />
<br />
Character maps allow a specific character in the instance of the data model to be replaced with a specified string of characters during serialization. The string that is substituted is output “as is,” and the serializer performs no checks that the resulting document is well-formed. This may only occur in documents parsed with {{Code|parameter-document}}. If a character is mapped, then it is not subjected to XML or HTML escaping. For details, refer to section [https://www.w3.org/TR/2015/CR-xslt-xquery-serialization-31-20151217/#character-maps 11 Character maps] in the [https://www.w3.org/TR/xslt-xquery-serialization-31 W3C XQuery Serialization 3.1] document.<br />
<br />
This example maps the Unicode U+00A0 NO-BREAK SPACE as &amp;#160; (without the serialization parameter, the Unicode character would be output):<br />
<br />
'''Example query''':<br />
<pre lang='xquery'><br />
declare option output:parameter-document "map.xml";<br />
<x>&amp;#xA0;</x><br />
</pre><br />
<br />
'''Example parameter-document''':<br />
<pre lang="xml"><br />
<serialization-parameters<br />
xmlns="http://www.w3.org/2010/xslt-xquery-serialization"><br />
<use-character-maps><br />
<character-map character="&amp;#160;" map-string="&amp;amp;#160;"/><br />
</use-character-maps><br />
</serialization-parameters><br />
</pre><br />
<br />
=Changelog=<br />
<br />
;Version 11.0<br />
* Added: Serialization parameter {{Code|indent-attributes}}.<br />
<br />
;Version 10.0<br />
* Updated: {{Code|indent}}: Default changed from {{Code|yes}} to {{Code|no}}.<br />
<br />
;Version 9.2<br />
* Updated: New default value for {{Code|include-content-type}} is {{Code|yes}}.<br />
<br />
;Version 8.4<br />
* Added: Serialization parameter {{Code|binary}}.<br />
* Updated: New serialization method <code>basex</code>. By default, items of binary type are now output in their native byte representation. The method <code>raw</code> was removed.<br />
<br />
;Version 8.0<br />
* Added: Support for {{Code|use-character-maps}} and {{Code|parameter-document}}.<br />
* Added: Serialization method {{Code|adaptive}}.<br />
* Updated: {{Code|adaptive}} is new default method (before: {{Code|xml}}).<br />
* Removed: {{Code|format}}, {{Code|wrap-prefix}}, {{Code|wrap-uri}}.<br />
<br />
;Version 7.8.2<br />
* Added: {{Code|limit}}: Stops serialization after the specified number of bytes has been serialized.<br />
<br />
;Version 7.8<br />
* Added: {{Code|csv}} and {{Code|json}} serialization parameters.<br />
* Removed: {{Code|separator}} option (use {{Code|item-separator}} instead).<br />
<br />
;Version 7.7.2<br />
* Added: {{Code|csv}} serialization method.<br />
* Added: temporary serialization methods {{Code|csv-header}}, {{Code|csv-separator}}, {{Code|json-unescape}}, {{Code|json-spec}}, {{Code|json-format}}.<br />
<br />
;Version 7.5<br />
* Added: official {{Code|item-separator}} and {{Code|html-version}} parameter.<br />
* Updated: <code>method=html5</code> removed; serializers updated with the [https://www.w3.org/TR/2013/WD-xslt-xquery-serialization-30-20130108/ latest version of the specification], using <code>method=html</code> and <code>version=5.0</code>.<br />
<br />
;Version 7.2<br />
* Added: {{Code|separator}} parameter.<br />
<br />
;Version 7.1<br />
* Added: {{Code|newline}} parameter.<br />
<br />
;Version 7.0<br />
* Added: Serialization parameters added to [[REST API]]; JSON/JsonML/raw methods.</div>CGhttps://docs.basex.org/index.php?title=XQuery_Module&diff=16898XQuery Module2023-12-01T16:39:32Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This [[Module Library|XQuery Module]] contains functions for parsing and evaluating XQuery strings at runtime, and to run code in parallel.<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/xquery</nowiki></code> namespace, which is statically bound to the {{Code|xquery}} prefix.<br/><br />
<br />
=Evaluation=<br />
<br />
==xquery:eval==<br />
<br />
{{Announce|Updated with Version 11}}: The Clark notation was replaced with the [[XQuery 3.0#Expanded QNames|Expanded QNames]] notation.<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>xquery:eval(<br />
$query as xs:anyAtomicType,<br />
$bindings as map(*)? := map { },<br />
$options as map(*)? := map { }<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Evaluates the supplied {{Code|$query}} and returns the resulting items. If the query is of type {{Code|xs:anyURI}}, the module located at this URI will be retrieved (a relative URI will be resolved against the static base URI). Otherwise, the input is expected to be of type {{Code|xs:string}}.<br />
Variables and context items can be declared via {{Code|$bindings}}. The specified keys must be QNames or strings:<br />
* If a key is a QName, it will be directly adopted as variable name.<br />
* It a key is a string, it may be prefixed with a dollar sign. Namespace can be specified using the [[XQuery 3.0#Expanded QNames|Expanded QNames]] notation.<br />
* If the specified string is empty, the value will be bound to the context item.<br />
The {{Code|$options}} parameter contains evaluation options:<br />
* {{Code|permission}}: the query will be evaluated with the specified permissions (see [[User Management]]).<br />
* {{Code|timeout}}: query execution will be interrupted after the specified number of seconds.<br />
* {{Code|memory}}: query execution will be interrupted if the specified number of megabytes will be exceeded. This check works best if only one process is running at the same time. Moreover, please note that this option enforces garbage collection, so it will take some additional time, and it requires GC to be enabled in your JVM.<br />
* {{Code|base-uri}}: set [https://www.w3.org/TR/xquery-31/#dt-static-base-uri base-uri property] for the query. Overwrites the base URI of the query; will be used when resolving relative URIs by functions such as {{Code|fn:doc}}.<br />
* {{Code|pass}}: passes on the original error info (line and column number, optional file uri). By default, this option is {{Code|false}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|update|#Errors}} the query contains [[XQuery Update#Updating Expressions|updating expressions]].<br/>{{Error|permission|#Errors}} insufficient permissions for evaluating the query.<br/>{{Error|timeout|#Errors}} query execution exceeded timeout.<br/>{{Error|limit|#Errors}} query execution exceeded memory limit.<br/>{{Error|nested|#Errors}} nested query evaluation is not allowed.<br/>Any other error that may occur while evaluating the query.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|xquery:eval("1+3")}} returns {{Code|4}}.<br/><br />
* If a URI is supplied, the query in the specified file will be evaluated:<br />
<pre lang='xquery'><br />
xquery:eval(xs:anyURI('cleanup.xq'))<br />
</pre><br />
* You can bind the context and e.g. operate on a certain database only:<br/><br />
<pre lang='xquery'><br />
xquery:eval("//country", map { '': db:get('factbook') })<br />
</pre><br />
* The following expressions use strings as keys. All of them return 'XML':<br/><br />
<pre lang='xquery'><br />
xquery:eval(".", map { '': 'XML' }),<br />
<br />
xquery:eval("declare variable $xml external; $xml", map { 'xml': 'XML' }),<br />
<br />
xquery:eval(<br />
"declare namespace pref='URI';<br />
declare variable $pref:xml external;<br />
$pref:xml",<br />
map { '{URI}xml': 'XML' }<br />
)<br />
</pre><br />
* The following expressions use QNames as keys. All of them return 'XML':<br/><br />
<pre lang='xquery'><br />
declare namespace pref = 'URI';<br />
<br />
xquery:eval("declare variable $xml external; $xml", map { xs:QName('xml'): 'XML' }),<br />
<br />
let $query := "declare namespace pref='URI';<br />
declare variable $pref:xml external;<br />
$pref:xml"<br />
let $vars := map { xs:QName('pref:xml'): 'XML' }<br />
return xquery:eval($query, $vars)<br />
</pre><br />
|}<br />
<br />
==xquery:eval-update==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>xquery:eval-update(<br />
$query as xs:anyAtomicType,<br />
$bindings as map(*)? := (),<br />
$options as map(*)? := ()<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Evaluates a query as updating expression. All updates will be added to the [[XQuery Update#Pending Update List|Pending Update List]] of the main query and performed after the evaluation of the main query.<br/>The rules for all arguments are the same as for {{Function||xquery:eval}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|update|#Errors}} the query contains no [[XQuery Update#Updating Expressions|updating expressions]].<br/>{{Error|permission|#Errors}} insufficient permissions for evaluating the query.<br/>{{Error|timeout|#Errors}} query execution exceeded timeout.<br/>{{Error|limit|#Errors}} query execution exceeded memory limit.<br/>{{Error|nested|#Errors}} nested query evaluation is not allowed.<br/>Any other error that may occur while evaluating the query.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* Removes entries from a temporary databases and returns an info string:<br />
<pre lang='xquery'><br />
xquery:eval-update("<br />
delete node db:get('tmp')/*,<br />
update:output('TEMPORARY DATABASE WAS CLEANED UP')<br />
")<br />
</pre><br />
|}<br />
<br />
=Parsing=<br />
<br />
==xquery:parse==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>xquery:parse(<br />
$query as xs:anyAtomicType,<br />
$options as map(*)? := map { }<br />
) as item()?</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Parses the specified {{Code|$query}} as XQuery module and returns the resulting query plan. If the query is of type {{Code|xs:anyURI}}, the module located at this URI will be retrieved (a relative URI will be resolved against the static base URI). Otherwise, the input is expected to be of type {{Code|xs:string}}. The {{Code|$options}} parameter influences the output:<br />
* {{Code|compile}}: additionally compiles the query after parsing it. By default, this option is {{Code|false}}.<br />
* {{Code|plan}}: returns an XML representation of the internal query plan. By default, this option is {{Code|true}}. The naming of the expressions in the query plan may change over time<br />
* {{Code|pass}}: by default, the option is {{Code|false}}. If an error is raised, the line/column number and the optional file uri will refer to the location of the function call. If the option is enabled, the line/column and file uri will be adopted from the raised error.<br />
* {{Code|base-uri}}: set [https://www.w3.org/TR/xquery-31/#dt-static-base-uri base-uri property] for the query. This URI will be used when resolving relative URIs by functions such as {{Code|fn:doc}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|Any error that may occur while parsing the query.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|xquery:parse("1 + 3")}} returns:<br />
<pre lang="xml"><br />
<MainModule updating="false"><br />
<QueryPlan compiled="false"><br />
<Arith op="+"><br />
<Int value="1" type="xs:integer"/><br />
<Int value="3" type="xs:integer"/><br />
</Arith><br />
</QueryPlan><br />
</MainModule><br />
</pre><br />
|}<br />
<br />
=Parallelized Execution=<br />
<br />
Parallel query execution is recommendable if you have various calls that require a lot of time, but that cannot be sped up by rewriting the code. This is e. g. the case if external URLs are called. If you are parallelizing local data reads (such as the access to a database), single-threaded queries will usually be faster, because parallelized access to disk data often results in randomized access patterns, which will rarely be optimized by the caching strategies of HDDs, SSDs, or the operating system.<br />
<br />
==xquery:fork-join==<br />
<br />
{{Announce|Updated with Version 11:}} Options added.<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>xquery:fork-join(<br />
$functions as function(*)*,<br />
$options as map(*)? := map { }<br />
) as item()*</pre><br />
|- valign="top"<br />
|'''Summary'''<br />
|This function executes the supplied (non-updating) {{Code|$functions}} in parallel. The following {{Code|$options}} are available:<br />
* {{Code|parallel}}: Maximum number of parallel threads. If the value is smaller than {{Code|1}}, or if the option is omitted, the number of available processors is used.<br />
* {{Code|result}}: Suppress or return the function results (default: {{Code|true}}).<br />
* {{Code|errors}}: Ignore or raise errors (default: {{Code|true}}).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* Request 100 URLs, use at most 8 parallel threads:<br />
<pre lang='xquery'><br />
xquery:fork-join(<br />
for $segment in 1 to 100<br />
let $url := 'http://url.com/path/' || $segment<br />
return function() { http:send-request((), $url) },<br />
map { 'parallel': 8 }<br />
)<br />
</pre><br />
* Parallel sleep function calls. The function is expected to finish in 1 second if the system has at least 2 cores:<br />
<pre lang='xquery'><br />
let $f := function() { prof:sleep(1000) }<br />
return xquery:fork-join(($f, $f))<br />
</pre><br />
|- valign="top"<br />
|'''Errors'''<br />
|{{Error|error|#Errors}} an unexpected error occurred.<br />
|}<br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
|Description<br />
|- valign="top"<br />
|{{Code|permission}}<br />
|Insufficient permissions for evaluating the query.<br />
|- valign="top"<br />
|{{Code|update}}<br />
|[[XQuery Update#Updating Expressions|updating expression]] found or expected.<br />
|- valign="top"<br />
|{{Code|timeout}}<br />
|Query execution exceeded timeout.<br />
|- valign="top"<br />
|{{Code|memory}}<br />
|Query execution exceeded memory limit.<br />
|- valign="top"<br />
|{{Code|nested}}<br />
|Nested query evaluation is not allowed.<br />
|- valign="top"<br />
|{{Code|error}}<br />
|An unexpected error occurred.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 11<br />
<br />
* Updated: {{Function||xquery:fork-join}}: Options added.<br />
* Updated: The Clark notation was replaced with the [[XQuery 3.0#Expanded QNames|Expanded QNames]] notation.<br />
<br />
;Version 10<br />
<br />
* Deleted: xquery:parse-uri (merged with {{Function||xquery:parse}})<br />
* Updated: {{Function||xquery:parse}}: {{$query}} can additionally be of type {{Code|xs:anyURI}}.<br />
<br />
;Version 9.2<br />
<br />
* Deleted: xquery:invoke, xquery:invoke-update (merged with {{Function||xquery:eval}} and {{Function||xquery:eval-update}})<br />
<br />
;Version 9.0<br />
<br />
* Added: {{Function||xquery:invoke-update}}<br />
* Updated: {{Function||xquery:eval}}: {{Code|pass}} option added<br />
* Updated: {{Function||xquery:parse}}, {{Function||xquery:parse-uri}}: {{Code|base-uri}} option added<br />
* Updated: xquery:update renamed to {{Function||xquery:eval-update}}<br />
* Updated: error codes updated; errors now use the module namespace<br />
<br />
;Version 8.5<br />
<br />
* Added: {{Function||xquery:fork-join}}<br />
* Updated: {{Function||xquery:eval}}: {{Code|base-uri}} option added<br />
* Updated: Relative URIs will always be resolved against the static base URI of the query<br />
* Deleted: xquery:type (moved to [[Profiling Module]])<br />
<br />
;Version 8.4<br />
<br />
* Added: {{Function||xquery:parse-uri}}<br />
* Updated: {{Function||xquery:parse}}: {{Code|pass}} option added<br />
<br />
;Version 8.0<br />
<br />
* Added: xquery:update, {{Function||xquery:parse}}<br />
* Deleted: xquery:evaluate (opened databases will now be closed by main query)<br />
<br />
;Version 7.8.2<br />
<br />
* Added: {{Code|$options}} argument<br />
<br />
;Version 7.8<br />
<br />
* Added: {{Function||xquery:evaluate}}<br />
* Updated: used variables must be explicitly declared in the query string.<br />
<br />
This module was introduced with Version 7.3. Functions have been adopted from the obsolete Utility Module.</div>CGhttps://docs.basex.org/index.php?title=Start_Scripts&diff=16897Start Scripts2023-12-01T16:39:31Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>BaseX can be [[Startup|started in different ways]]. The [https://basex.org/download/ Windows and ZIP distributions] include various start scripts, which are presented in the following, and which can also be maintained separately.<br />
<br />
* We recommend you to manually add the {{Code|bin}} directory of your BaseX instance to the [https://en.wikipedia.org/wiki/PATH_(variable) PATH variable] of your environment.<br />
* If you use the Windows installer, that’s done automatically.<br />
* You can copy the start scripts to another location in your file system. After that, you should edit the scripts and assign the BaseX directory to the {{Code|MAIN}} variable.<br />
* If you are compiling the source code with [[Maven]], you can launch BaseX via the scripts in the [https://github.com/BaseXdb/basex/tree/master/basex-core/etc basex-core/etc] and [https://github.com/BaseXdb/basex/tree/master/basex-api/etc basex-api/etc] subdirectories of the project.<br />
<br />
If BaseX terminates with an {{Code|Out of Memory}} or {{Code|Java heap space}} error, you can assign more RAM via the {{Code|-Xmx}} flag (see below). A conservative value was chosen in our distributions to ensure that BaseX will also run on older JVMs.<br />
<br />
=Standalone=<br />
<br />
Use the following scripts to launch the standalone version of BaseX:<br />
<br />
==Windows: {{Code|basex.bat}}==<br />
<br />
<pre lang="batch"><br />
@echo off<br />
setLocal EnableDelayedExpansion<br />
<br />
REM Path to core and library classes<br />
set MAIN=%~dp0/..<br />
set CP=%MAIN%/BaseX.jar;%MAIN%/lib/*;%MAIN%/lib/custom/*<br />
<br />
REM Options for virtual machine<br />
set BASEX_JVM=-Xmx1200m %BASEX_JVM%<br />
<br />
REM Run code<br />
java -cp "%CP%" %BASEX_JVM% org.basex.BaseX %*<br />
</pre><br />
<br />
==Linux/Mac: {{Code|basex}}==<br />
<br />
<pre lang="bash"><br />
#!/usr/bin/env bash<br />
<br />
# Path to this script<br />
FILE="${BASH_SOURCE[0]}"<br />
while [ -h "$FILE" ] ; do<br />
SRC="$(readlink "$FILE")"<br />
FILE="$( cd -P "$(dirname "$FILE")" && \<br />
cd -P "$(dirname "$SRC")" && pwd )/$(basename "$SRC")"<br />
done<br />
MAIN="$( cd -P "$(dirname "$FILE")/.." && pwd )"<br />
<br />
# Core and library classes<br />
CP=$MAIN/BaseX.jar:$MAIN/lib/*:$MAIN/lib/custom/*:$CLASSPATH<br />
<br />
# Options for virtual machine (can be extended by global options)<br />
BASEX_JVM="-Xmx2g $BASEX_JVM"<br />
<br />
# Run code<br />
java -cp "$CP" $BASEX_JVM org.basex.BaseX "$@"<br />
</pre><br />
<br />
=GUI, Server, Client=<br />
<br />
If you would like to launch the GUI, Server or Client version of BaseX, please replace the class name in <code>org.basex.BaseX</code> with either <code>BaseXGUI</code>, <code>BaseXServer</code> or <code>BaseXClient</code>.<br />
<br />
=HTTP Server=<br />
<br />
The scripts for running [[Web Application|Web Applications]] can be found below:<br />
<br />
==Windows: {{Code|basexhttp.bat}}==<br />
<br />
<pre lang="batch"><br />
@echo off<br />
setLocal EnableDelayedExpansion<br />
<br />
REM Path to core and library classes<br />
set MAIN=%~dp0/..<br />
set CP=%MAIN%/BaseX.jar;%MAIN%/lib/*;%MAIN%/lib/custom/*<br />
<br />
REM Options for virtual machine<br />
set BASEX_JVM=-Xmx1200m %BASEX_JVM%<br />
<br />
REM Run code<br />
java -cp "%CP%" %BASEX_JVM% org.basex.BaseXHTTP %*<br />
</pre><br />
<br />
==Linux/Mac: {{Code|basexhttp}}==<br />
<br />
<pre lang="bash"><br />
#!/usr/bin/env bash<br />
<br />
# Path to this script<br />
FILE="${BASH_SOURCE[0]}"<br />
while [ -h "$FILE" ] ; do<br />
SRC="$(readlink "$FILE")"<br />
FILE="$( cd -P "$(dirname "$FILE")" && \<br />
cd -P "$(dirname "$SRC")" && pwd )/$(basename "$SRC")"<br />
done<br />
MAIN="$( cd -P "$(dirname "$FILE")/.." && pwd )"<br />
<br />
# API, core, and library classes<br />
CP=$MAIN/BaseX.jar:$MAIN/lib/*:$MAIN/lib/custom/*:$CLASSPATH<br />
<br />
# Options for virtual machine (can be extended by global options)<br />
BASEX_JVM="-Xmx2g $BASEX_JVM"<br />
<br />
# Run code<br />
java -cp "$CP" $BASEX_JVM org.basex.BaseXHTTP "$@"<br />
</pre><br />
<br />
=Included Start Scripts=<br />
<br />
The BaseX [https://basex.org/download/ Windows and ZIP distributions] include the following start scripts:<br />
<br />
{| class="wikitable"<br />
|- valign="top"<br />
! Windows<br />
! Linux/Mac<br />
! Description<br />
|- valign="top"<br />
| {{Code|basex.bat}}<br />
| {{Code|basex}}<br />
| Launches the BaseX standalone mode.<br />
|- valign="top"<br />
| {{Code|basexclient.bat}}<br />
| {{Code|basexclient}}<br />
| Starts a BaseX client.<br />
|- valign="top"<br />
| {{Code|basexgui.bat}}<br />
| {{Code|basexgui}}<br />
| Starts the BaseX GUI.<br />
|- valign="top"<br />
| {{Code|basexhttp.bat}}<br />
| {{Code|basexhttp}}<br />
| Starts the BaseX HTTP Server.<br />
|- valign="top"<br />
| {{Code|basexserver.bat}}<br />
| {{Code|basexserver}}<br />
| Starts the BaseX database server.<br />
|}<br />
<br />
For the BaseX HTTP and database server, additional stop scripts are available:<br />
<br />
{| class="wikitable"<br />
|- valign="top"<br />
! Windows<br />
! Linux/Mac<br />
! width="40%" | Description<br />
|- valign="top"<br />
|- valign="top"<br />
| {{Code|basexhttpstop.bat}}<br />
| {{Code|basexhttpstop}}<br />
| Stops the BaseX HTTP Server.<br />
|- valign="top"<br />
| {{Code|basexserverstop.bat}}<br />
| {{Code|basexserverstop}}<br />
| Stops the BaseX database server.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 7.5<br />
* Updated: Static dependencies removed from Windows batch scripts.<br />
<br />
;Version 7.2<br />
* Updated: The {{Code|BaseXHTTP}} start class moved from {{Code|org.basex.api}} to {{Code|org.basex}}.<br />
<br />
;Version 7.0<br />
* Updated: The {{Code|basexjaxrx}} scripts have been replaced with the {{Code|basexhttp}} scripts.</div>CGhttps://docs.basex.org/index.php?title=User_Module&diff=16896User Module2023-12-01T16:39:29Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This [[Module Library|XQuery Module]] contains functions for creating and administering database users. The [[User Management]] article provides more information on database users and permissions.<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/user</nowiki></code> namespace, which is statically bound to the {{Code|user}} prefix.<br/><br />
<br />
=Read Operations=<br />
<br />
==user:current==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>user:current() as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the name of the currently logged-in user.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* If the GUI or the standalone mode is used, {{Code|user:current()}} always returns {{Code|admin}}.<br />
|}<br />
<br />
==user:list==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>user:list() as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the names of all registered users who are visible to the current user.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* After a fresh installation, {{Code|user:list()}} will only return {{Code|admin}}.<br />
|}<br />
<br />
==user:list-details==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>user:list-details(<br />
$name as xs:string := ()<br />
) as element(user)*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns an element sequence, containing all registered users who are visible to the current user.<br/>In addition to the {{Command|SHOW USERS}} command, encoded password strings and database permissions will be output. A user {{Code|$name}} can be specified to filter the results in advance.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* After a fresh installation, {{Code|user:list-details()}} returns output similar to the following one:<br />
<pre lang="xml"><br />
<user name="admin" permission="admin"><br />
<password algorithm="digest"><br />
<hash>304bdfb0383c16f070a897fc1eb25cb4</hash><br />
</password><br />
<password algorithm="salted-sha256"><br />
<salt>871602799292195</salt><br />
<hash>a065ca66fa3d6da5762c227587f1c8258c6dc08ee867e44a605a72da115dcb41</hash><br />
</password><br />
</user><br />
</pre><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|unknown|#Errors}} The specified username is unknown.<br />
|}<br />
<br />
==user:exists==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>user:exists(<br />
$name as xs:string<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Checks if a user with the specified {{Code|$name}} exists.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|user:exists('admin')}} will always yield true.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|name|#Errors}} The specified username is invalid.<br />
|}<br />
<br />
==user:check==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>user:check(<br />
$name as xs:string,<br />
$password as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Checks if the specified user and password is correct. Raises errors otherwise.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|user:check('admin', '')}} will raise an error if the password of the admin user is a non-empty string.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|name|#Errors}} The specified username is invalid.<br/>{{Error|unknown|#Errors}} The specified user does not exist.<br/>{{Error|password|#Errors}} The specified password is wrong.<br/><br />
|}<br />
<br />
==user:info==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>user:info(<br />
$name as xs:string := ()<br />
) as element(info)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns an <code>info</code> element, which may contain application-specific data. If a user {{Code|$name}} is supplied, a user-specific element is returned. By default, the returned element has no contents. It can be modified via {{Function||user:update-info}}.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* After a fresh installation, {{Code|user:info()}} returns <code><info/></code>.<br />
|}<br />
<br />
=Updates=<br />
<br />
'''Important note:''' All functions in this section are ''updating functions'': they will not be immediately executed, but queued on the [[XQuery Update#Pending Update List|Pending Update List]], which will be processed after the actual query has been evaluated. This means that the order in which the functions are specified in the query usually does not reflect the order in which the code will be evaluated.<br />
<br />
==user:create==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>user:create(<br />
$name as xs:string,<br />
$password as xs:string,<br />
$permissions as xs:string* := (),<br />
$patterns as xs:string* := (),<br />
$info as element(info) := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a new user with the specified {{Code|$name}}, {{Code|$password}}, and {{Code|$permissions}}:<br />
* Local permissions are granted with non-empty glob {{Code|$patterns}}.<br />
* An {{Code|$info}} element with application-specific information can be supplied.<br />
* The default global permission (''none'') can be overwritten with an empty pattern or by omitting the last argument.<br />
* Existing users will be overwritten.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|user:create('John', '7e$j#!1', 'admin')}} creates a new user 'John' with admin permissions.<br />
* {{Code|user:create('Jack', 'top!secret', 'read', 'index*')}} creates a new user 'Jack' with read permissions for databases starting with the letters 'index'.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|name|#Errors}} The specified username is invalid.<br/>{{Error|permission|#Errors}} The specified permission is invalid.<br/>{{Error|admin|#Errors}} The "admin" user cannot be modified.<br/>{{Error|logged-in|#Errors}} The specified user is currently logged in.<br/>{{Error|update|#Errors}} The operation can only be performed once per user or database pattern.<br />
|}<br />
<br />
==user:grant==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>user:grant(<br />
$name as xs:string,<br />
$permissions as xs:string*,<br />
$patterns as xs:string* := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Grants global or local {{Code|$permissions}} to a user with the specified {{Code|$name}}. Local permissions are granted with non-empty glob {{Code|$patterns}}.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|user:grant('John', 'create')}} grants create permissions to the user 'John'.<br />
* {{Code|user:grant('John', ('read','write'), ('index*','unit*'))}} allows John to read all databases starting with the letters 'index', and to write to all databases starting with 'unit'.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|unknown|#Errors}} The specified username is unknown.<br/>{{Error|name|#Errors}} The specified username is invalid.<br/>{{Error|pattern|#Errors}} The specified database pattern is invalid.<br/>{{Error|permission|#Errors}} The specified permission is invalid.<br/>{{Error|admin|#Errors}} The "admin" user cannot be modified.<br/>{{Error|local|#Errors}} A local permission can only be 'none', 'read' or 'write'.<br/>{{Error|logged-in|#Errors}} The specified user is currently logged in.<br/>{{Error|update|#Errors}} The operation can only be performed once per user or database pattern.<br />
|}<br />
<br />
==user:drop==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>user:drop(<br />
$name as xs:string,<br />
$patterns as xs:string* := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Drops a user with the specified {{Code|$name}}. If non-empty glob {{Code|$patterns}} are specified, only the database patterns will be removed.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|user:drop('John')}} drops the user 'John'.<br />
* {{Code|user:grant('John', 'unit*')}} removes the 'unit*' database pattern. If John accesses any of these database, his global permission will be checked again.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|unknown|#Errors}} The specified username is unknown.<br/>{{Error|name|#Errors}} The specified username is invalid.<br/>{{Error|pattern|#Errors}} The specified database pattern is invalid.<br/>{{Error|admin|#Errors}} The "admin" user cannot be modified.<br/>{{Error|logged-in|#Errors}} The specified user is currently logged in.<br/>{{Error|update|#Errors}} The operation can only be performed once per user or database pattern.<br/>{{Error|conflict|#Errors}} A user cannot be both altered and dropped.<br />
|}<br />
<br />
==user:alter==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>user:alter(<br />
$name as xs:string,<br />
$newname as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Renames a user with the specified {{Code|$name}} to {{Code|$newname}}.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|user:alter('John', 'Jack')}} renames the user 'John' to 'Jack'.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|unknown|#Errors}} The specified username is unknown.<br/>{{Error|name|#Errors}} The specified username is invalid.<br/>{{Error|admin|#Errors}} The "admin" user cannot be modified.<br/>{{Error|logged-in|#Errors}} The specified user is currently logged in.<br/>{{Error|update|#Errors}} The operation can only be performed once per user or database pattern.<br/>{{Error|conflict|#Errors}} A user cannot be both altered and dropped.<br />
|}<br />
<br />
==user:password==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>user:password(<br />
$name as xs:string,<br />
$password as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Changes the {{Code|password}} of a user with the specified {{Code|$name}}.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|user:password('John', '')}} assigns user 'John' an empty password string.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|unknown|#Errors}} The specified username is unknown.<br/>{{Error|name|#Errors}} The specified username is invalid.<br/>{{Error|update|#Errors}} The operation can only be performed once per user or database pattern.<br />
|}<br />
<br />
==user:update-info==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>user:update-info(<br />
$info as element(info),<br />
$name as xs:string := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Assigns the specified {{Code|$info}} element to the user management or, if {{Code|$name}} is supplied, to a specific user. This function can be used to manage application-specific data (groups, enhanced user info, etc.).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* Store initial groups information:<br />
<pre lang='xquery'><br />
user:update-info(element info {<br />
for $group in ('editor', 'author', 'writer')<br />
return element group { $group }<br />
})<br />
</pre><br />
* Add a group to a specific user:<br />
<pre lang='xquery'><br />
user:update-info(<info group='editor'/>, 'john')<br />
</pre><br />
|}<br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
|Description<br />
|- valign="top"<br />
|{{Code|admin}}<br />
|The "admin" user cannot be modified.<br />
|- valign="top"<br />
|{{Code|conflict}}<br />
|A user cannot be both altered and dropped.<br />
|- valign="top"<br />
|{{Code|equal}}<br />
|Name of old and new user is equal.<br />
|- valign="top"<br />
|{{Code|local}}<br />
|A local permission can only be 'none', 'read' or 'write'.<br />
|- valign="top"<br />
|{{Code|logged-in}}<br />
|The specified user is currently logged in.<br />
|- valign="top"<br />
|{{Code|name}}<br />
|The specified username is invalid.<br />
|- valign="top"<br />
|{{Code|password}}<br />
|The specified password is wrong.<br />
|- valign="top"<br />
|{{Code|pattern}}<br />
|The specified database name is invalid.<br />
|- valign="top"<br />
|{{Code|permission}}<br />
|The specified permission is invalid.<br />
|- valign="top"<br />
|{{Code|unknown}}<br />
|The specified user does not exist.<br />
|- valign="top"<br />
|{{Code|update}}<br />
|The operation can only be performed once per user or database pattern.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 8.6<br />
* Updated: {{Function||user:create}}, {{Function||user:info}}, {{Function||user:update-info}}: {{Code|$name}} parameter added.<br />
<br />
;Version 8.6<br />
* Added: {{Function||user:check}}, {{Function||user:info}}, {{Function||user:update-info}}.<br />
* Updated: {{Function||user:list}}, {{Function||user:list-details}}: If called by non-admins, will only return the current user.<br />
<br />
;Version 8.4<br />
* Updated: {{Function||user:create}}, {{Function||user:grant}}, {{Function||user:drop}}: extended support for database patterns.<br />
<br />
;Version 8.1<br />
* Added: {{Function||user:current}}.<br />
<br />
The Module was introduced with Version 8.0.</div>CGhttps://docs.basex.org/index.php?title=Job_Module&diff=16894Job Module2023-12-01T16:39:27Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This [[Module Library|XQuery Module]] provides functions for registering new query jobs and orchestrating existing jobs. Jobs can be queries, commands, operations performed by a database client, and HTTP requests.<br />
<br />
=Conventions=<br />
<br />
All functions in this module are assigned to the <code><nowiki>http://basex.org/modules/job</nowiki></code> namespace, which is statically bound to the {{Code|job}} prefix. Errors will be bound to the same prefix.<br />
<br />
=Services=<br />
<br />
A job can be registered as ''service'' by supplying the {{Code|service}} option to {{Function||job:eval}}:<br />
<br />
<pre lang='xquery'><br />
(: register job as service; will be run every day at 1 am :)<br />
job:eval('db:drop("tmp")', (), map { 'id':'cleanup', 'start':'01:00:00', 'interval':'P1D', 'service': true() }),<br />
<br />
(: list registered services :)<br />
job:services(),<br />
(: result: <job base-uri="..." id="cleanup" interval="P1D" start="01:00:00">db:drop("tmp")</job> :)<br />
<br />
(: unregister job :)<br />
job:remove('cleanup', map { 'service': true() })<br />
</pre><br />
<br />
'''Some more notes:'''<br />
<br />
* All job services will be scheduled for evaluation when the BaseX server or BaseX HTTP server is started.<br />
* If a job service is outdated (e.g. because a supplied end time has been exceeded), it will be removed from the jobs file at startup time.<br />
* The job definitions are stored in a {{Code|jobs.xml}} file in the database directory. It can also be edited manually.<br />
<br />
=Executing Jobs=<br />
<br />
There are cases in which a client does not, or cannot, wait until a request is fully processed. The client may be a browser, which sends an HTTP request to the server to start another time-consuming query job. The functions in this section allow you to register new query jobs and access existing ones. Jobs can be executed immediately (i.e., as soon as a [[Transaction Management#Concurrency Control|free slot is available]]) or scheduled for repeated execution. Each registered job gets a job ID, and the ID can be used to retrieve a query result, stop a job, or wait for its termination.<br />
<br />
==job:eval==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>job:eval(<br />
$query as xs:anyAtomicType,<br />
$bindings as map(*)? := map { },<br />
$options as map(*)? := map { }<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Schedules the evaluation a new query job for the supplied {{Code|$query}} (of type {{Code|xs:string}}, or of type {{Code|xs:anyURI}} if points to a resource), and returns a job ID. The job will be queued until a free slot is available, and the query result can be cached. Queries can be updating, and variables and the context value can be declared via {{Code|$bindings}} (see {{Function|XQuery|xquery:eval}} for more details). The following {{Code|$options}} can be supplied:<br />
* {{Code|cache}}: indicates if the query result will be cached or ignored (default: {{Code|false}}):<br />
** The result will be cached in main-memory until it is fetched via {{Function||job:result}}, or until {{Option|CACHETIMEOUT}} is exceeded.<br />
** If the query raises an error, it will be cached and returned instead.<br />
* {{Code|start}}: a dayTimeDuration, time, dateTime or integer can be specified to delay the execution of the query:<br />
** If a dayTimeDuration is specified, the query will be queued after the specified duration has passed. Examples of valid values are: <code>P1D</code> (1 day), <code>PT5M</code> (5 minutes), <code>PT0.1S</code> (100 ms). An error will be raised if a negative value is specified.<br />
** If a dateTime is specified, the query will be executed at this date. Examples for valid values are: <code>2018-12-31T23:59:59</code> (New Year's Eve 2018, close to midnight). An error will be raised if the specified time lies in the past.<br />
** If a time is specified, the query will be executed at this time of the day. Examples of valid times are: <code>02:00:00</code> (2am local time), <code>12:00:00Z</code> (noon, UTC). If the time lies in the past, the query will be executed the next day.<br />
** An integer will be interpreted as minutes. If the specified number is greater than the elapsed minutes of the current hour, the query will be executed one hour later.<br />
* {{Code|interval}}: a dayTimeDuration string can be specified to execute the query periodically. An error is raised if the specified interval is less than one second (<code>PT1S</code>). If the next scheduled call is due, and if a query with the same ID is still running, it will be skipped.<br />
* {{Code|end}}: scheduling can be stopped after a given time or duration. The string format is the same as for {{Code|start}}. An error is raised if the resulting end time is smaller than the start time.<br />
* {{Code|base-uri}}: sets the [https://www.w3.org/TR/xquery-31/#dt-static-base-uri base-uri property] for the query. This URI will be used when resolving relative URIs, such as with {{Code|fn:doc}}.<br />
* {{Code|id}}: sets a custom job ID. The ID must not start with the standard <code>job</code> prefix, and it can only be assigned if no job with the same name exists.<br />
* {{Code|service}}: additionally registers the job as [[#Services|service]]. Registered services must have no variable bindings.<br />
* {{Code|log}}: writes the specified string to the [[Logging|database logs]]. Two log entries are stored, one at the beginning and another one after the execution of the job.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|overflow|#Errors}} Query execution is rejected because too many jobs are queued or being executed. {{Option|CACHETIMEOUT}} can be decreased if the default setting is too restrictive.<br/>{{Error|range|#Errors}} A specified time or duration is out of range.<br/>{{Error|id|#Errors}} The specified ID is invalid or has already been assigned.<br/>{{Error|options|#Errors}} The specified options are conflicting.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* Cache query result. The returned ID can be used to pick up the result with {{Function||job:result}}:<br />
<pre lang='xquery'><br />
job:eval("1+3", (), map { 'cache': true() })<br />
</pre><br />
* A happy birthday mail will be sent at the given date:<br />
<pre lang='xquery'><br />
job:eval("import module namespace mail='mail'; mail:send('Happy birthday!')",<br />
(), map { 'start': '2018-09-01T06:00:00' })}}<br />
</pre><br />
* The following [[RESTXQ]] functions can be called to execute a query at 2 am every day. An ID will be returned by the first function, which can be used to stop the scheduler via the second function:<br />
<pre lang='xquery'><br />
declare %rest:POST("{$query}") %rest:path('/start-scheduling') function local:start($query) {<br />
job:eval($query, (), map { 'start': '02:00:00', 'interval': 'P1D' })<br />
};<br />
declare %rest:path('/stop-scheduling/{$id}') function local:stop($id) {<br />
job:remove($id)<br />
};<br />
</pre><br />
* Query execution is scheduled for every second, and for 10 seconds in total. As the query itself will take 1.5 seconds, it will only be executed every second time:<br />
<pre lang='xquery'><br />
job:eval("prof:sleep(1500)", (), map { 'interval': 'PT1S', 'end': 'PT10S' })<br />
</pre><br />
* The query in the specified file will be evaluated once:<br />
<pre lang='xquery'><br />
job:eval(xs:anyURI('cleanup.xq'))<br />
</pre><br />
* The following expression, if stored in a file, will be evaluated every 5 seconds:<br />
<pre lang='xquery'><br />
job:eval(<br />
static-base-uri(),<br />
map { },<br />
map { 'start': 'PT5S' }<br />
)<br />
</pre><br />
|}<br />
<br />
==job:result==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>job:result(<br />
$id as xs:string,<br />
$options as map(*)? := map { }<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the cached result of a job with the specified job {{Code|$id}}:<br />
* If the original job has raised an error, the cached error will be raised instead.<br />
* The cached result or error will be dropped after it has been retrieved.<br />
* If the result has not been cached or if it has been dropped, an empty sequence is returned.<br />
The following {{Code|$options}} can be supplied:<br />
* {{Code|keep}}: Keep the cached result or error after retrieval.<br />
<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* The following [[RESTXQ]] function will either return the result of a previously started job or raise an error:<br />
<pre lang='xquery'><br />
declare %rest:path('/result/{$id}') function local:result($id) {<br />
job:result($id)<br />
};<br />
</pre><br />
* The following query demonstrates how the results of an executed query can be returned within the same query (see below why you should avoid this pattern in practice):<br />
<pre lang='xquery'><br />
let $query := job:eval('(1 to 10000000)[. = 1]', map { }, map { 'cache': true() })<br />
return (<br />
job:wait($query),<br />
job:result($query)<br />
)<br />
</pre><br />
Queries of this kind can cause deadlocks! If the original query and the new query perform updates on the same database, the second query will only be run after the first one has been executed, and the first query will wait for the second query forever. You should resort to {{Function|XQuery|xquery:fork-join}} if you want to have full control on parallel query execution.<br />
|}<br />
<br />
==job:remove==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>job:remove(<br />
$id as xs:string,<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Triggers the cancelation of a job with the specified {{Code|$id}}, cancels a scheduled job or removes a cached result. Unknown IDs are ignored. All jobs are gracefully stopped; it is up to the process to decide when it is safe to shut down. The following {{Code|$options}} can be supplied:<br />
* {{Code|service}}: additionally removes the job from the [[#Services|job services]] list.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>job:list()[. != job:current()] ! job:remove(.)</code> stops and discards all jobs except for the current one.<br />
* <code>job:remove(job:current())</code> interrupts the current job.<br />
|}<br />
<br />
==job:wait==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>job:wait(<br />
$id as xs:string<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Waits for the completion of a job with the specified {{Code|$id}}:<br />
* The function will terminate immediately if the job ID is unknown. This is the case if a future job has not been queued yet, or if the ID has already been discarded after job evaluation.<br />
* If the function is called with the ID of a queued job, or repeatedly executed job, it may stall and never terminate.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|self|#Errors}} The current job is addressed.<br/><br />
|}<br />
<br />
=Listing Jobs=<br />
<br />
==job:current==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>job:current() as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the ID of the current job.<br />
|}<br />
<br />
==job:list==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>job:list() as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the IDs of all jobs that are currently registered. The list includes scheduled, queued, running, stopped, and finished jobs with cached results.<br />
|- valign="top"<br />
| '''Examples'''<br />
| <code>job:list()</code> returns the same job ID as {{Function||job:current}} if no other job is registered.<br />
|}<br />
<br />
==job:list-details==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>job:list-details(<br />
$id as xs:string := ()<br />
) as element(job)*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns information on all jobs that are currently registered, or on a job with the specified {{Code|$id}} (or an empty sequence if this job is not found). The list includes scheduled, queued, running jobs, and cached jobs. A string representation of the job, or its URI, will be returned as a value. The returned elements have additional attributes:<br />
* <code>id</code>: job ID<br />
* <code>type</code>: type of the job (command, query, REST, RESTXQ, etc.)<br />
* <code>state</code>: current state of the job: <code>scheduled</code>, <code>queued</code>, <code>running</code>, <code>cached</code><br />
* <code>user</code>: user who started the job<br />
* <code>duration</code>: evaluation time (included if a job is running or if the result was cached)<br />
* <code>start</code>: next start of job (included if a job will be executed repeatedly)<br />
* <code>time</code>: time when job was registered<br />
|- valign="top"<br />
| '''Examples'''<br />
| <code>job:list-details()</code> returns information on the currently running job and possibly others:<br />
<pre lang="xml"><br />
<job id="job1" type="XQuery" state="running" user="admin" duration="PT0.001S"><br />
XQUERY job:list-details()<br />
</job><br />
</pre><br />
|}<br />
<br />
==job:bindings==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>job:bindings(<br />
$id as xs:string<br />
) as map(*)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the variable bindings of an existing job with the specified {{Code|$id}}. If no variables have been bound to this job, an empty map is returned.<br />
|}<br />
<br />
==job:finished==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>job:finished(<br />
$id as xs:string<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Indicates if the evaluation of an already running job with the specified {{Code|$id}} has finished. As the IDs of finished jobs will usually be discarded, unless caching is enabled, the function will also return <code>true</code> for unknown jobs.<br />
* <code>false</code> indicates that the job ID is scheduled, queued, or currently running.<br />
* <code>true</code> will be returned if the job has either finished, or if the ID is unknown (because the IDs of all finished jobs will not be cached).<br />
|}<br />
<br />
==job:services==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>job:services() as element(job)*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns a list of all jobs that have been persistently registered as [[#Services|Services]].<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|services|#Errors}} Registered services cannot be parsed.<br/><br />
|}<br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
|Description<br />
|- valign="top"<br />
|{{Code|options}}<br />
| The specified options are conflicting.<br />
|- valign="top"<br />
|{{Code|id}}<br />
| The specified ID is invalid or has already been assigned.<br />
|- valign="top"<br />
|{{Code|overflow}}<br />
| Too many queries or query results are queued.<br />
|- valign="top"<br />
|{{Code|range}}<br />
| A specified time or duration is out of range.<br />
|- valign="top"<br />
|{{Code|running}}<br />
| A query is still running.<br />
|- valign="top"<br />
|{{Code|self}}<br />
| The current job cannot be addressed.<br />
|- valign="top"<br />
|{{Code|service}}<br />
| Registered services cannot be parsed, added or removed.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 10.0<br />
* Updated: Renamed from ''Jobs Module'' to ''Job Module''. The namespace URI has been updated as well.<br />
* Updated: {{Function||job:remove}} renamed from {{Code|jobs:stop}}.<br />
* Updated: {{Function||job:result}}: options argument added.<br />
* Added: {{Function||job:bindings}}<br />
<br />
;Version 9.7<br />
* Updated: {{Function||job:result}}: return empty sequence if no result is cached.<br />
<br />
;Version 9.5<br />
* Updated: {{Function||job:eval}}: integers added as valid start and end times.<br />
<br />
;Version 9.4<br />
* Updated: {{Function||job:eval}}: option added for writing log entries.<br />
* Updated: {{Function||job:list-details}}: interval added.<br />
<br />
;Version 9.2<br />
* Deleted: job:invoke (merged with {{Function||job:eval}})<br />
<br />
;Version 9.1<br />
* Updated: {{Function||job:list-details}}: registration time added.<br />
<br />
;Version 9.0<br />
* Added: {{Function||job:invoke}}, [[#Services|Services]]<br />
<br />
;Version 8.6<br />
* Updated: {{Function||job:eval}}: <code>id</code> option added.<br />
<br />
The module was introduced with Version 8.5.</div>CGhttps://docs.basex.org/index.php?title=Cryptographic_Module&diff=16895Cryptographic Module2023-12-01T16:39:27Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This [[Module Library|XQuery Module]] contains functions to perform cryptographic operations in XQuery. <br />
The cryptographic module is based on an early draft of the [http://expath.org/spec/crypto/20110810 EXPath Cryptographic Module] and provides the following functionality: creation of message authentication codes (HMAC), encryption and decryption, and creation and validation of XML Digital Signatures.<br />
<br />
=Conventions=<br />
<br />
All functions in this module are assigned to the <code><nowiki>http://expath.org/ns/crypto</nowiki></code> namespace, which is statically bound to the {{Code|crypto}} prefix.<br/><br />
All errors are assigned to the <code><nowiki>http://expath.org/ns/error</nowiki></code> namespace, which is statically bound to the {{Code|experr}} prefix.<br />
<br />
=Message Authentication=<br />
<br />
==crypto:hmac==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>crypto:hmac(<br />
$data as xs:anyAtomicType,<br />
$key as xs:anyAtomicType,<br />
$algorithm as xs:string,<br />
$encoding as xs:string := ()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates an authentication code for the specified {{Code|$data}} via a cryptographic hash function:<br />
* {{Code|$key}} must not be empty.<br />
* {{Code|$algorithm}} describes the hash algorithm which is used for encryption. Currently supported are {{Code|md5}}, {{Code|sha1}}, {{Code|sha256}}, {{Code|sha384}}, {{Code|sha512}}. Default is {{Code|md5}}.<br />
* {{Code|$encoding}} must either be {{Code|hex}} or {{Code|base64}}; it specifies the encoding of the returned authentication code. Default is {{Code|base64}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|CX0013|#Errors}} the specified hashing algorithm is not supported.<br/>{{Error|CX0014|#Errors}} the specified encoding method is not supported.<br/>{{Error|CX0019|#Errors}} the specified secret key is invalid.<br/><br />
|- valign="top"<br />
| '''Example'''<br />
|Return message authentication code (MAC) for a given string:<br />
'''Query:'''<br />
<pre lang='xquery'><br />
crypto:hmac('message', 'secretkey', 'md5', 'hex')<br />
</pre><br />
<br />
'''Result:'''<br />
<pre lang="xml"><br />
34D1E3818B347252A75A4F6D747B21C2<br />
</pre><br />
|}<br />
<br />
=Encryption & Decryption=<br />
<br />
The encryption and decryption functions underlie several limitations:<br />
* Cryptographic algorithms are currently limited to {{Code|symmetric}} algorithms. This means that the same secret key is used for encryption and decryption.<br />
* Available algorithms are {{Code|DES}} and {{Code|AES}}.<br />
* Padding is fixed to {{Code|PKCS5Padding}}.<br />
* The result of an encryption using the same message, algorithm and key looks different each time it is executed. This is due to a random initialization vector (IV) which is appended to the message and simply increases security.<br />
* As the IV has to be passed along with the encrypted message somehow, data which has been encrypted by the {{Code|crypto:encrypt}} function in BaseX can only be decrypted by calling the {{Code|crypto:decrypt}} function.<br />
<br />
==crypto:encrypt==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>crypto:encrypt(<br />
$data as xs:anyAtomicType,<br />
$type as xs:string,<br />
$key as xs:anyAtomicType,<br />
$algorithm as xs:string<br />
) as xs:base64Binary</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Encrypts data with the specified key:<br />
* {{Code|$data}} must be a string or binary item.<br />
* {{Code|$type}} must be {{Code|symmetric}}.<br />
* {{Code|$key}} is the secret key which is used for both encryption and decryption of input data. It must be a string or binary item. Its length is fixed and depends on the chosen algorithm: 8 bytes for {{Code|DES}}, 16 bytes for {{Code|AES}}.<br />
* {{Code|$algorithm}} must either be {{Code|DES}} or {{Code|AES}}. Default is {{Code|DES}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|CX0016|#Errors}} padding problems arise.<br/>{{Error|CX0017|#Errors}} padding is incorrect.<br/>{{Error|CX0018|#Errors}} the encryption type is not supported.<br/>{{Error|CX0019|#Errors}} the secret key is invalid.<br/>{{Error|CX0020|#Errors}} the block size is incorrect.<br/>{{Error|CX0021|#Errors}} the specified encryption algorithm is not supported.<br/><br />
|- valign="top"<br />
| '''Example'''<br />
|Encrypt input data:<br />
<pre lang='xquery'><br />
crypto:encrypt('message', 'symmetric', 'keykeyke', 'DES')<br />
</pre><br />
|}<br />
<br />
==crypto:decrypt==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>crypto:decrypt(<br />
$data as xs:anyAtomicType,<br />
$type as xs:string,<br />
$key as xs:anyAtomicType,<br />
$algorithm as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Encrypts data with the specified key:<br />
* {{Code|$data}} must be a string or binary item.<br />
* {{Code|$type}} must be {{Code|symmetric}}.<br />
* {{Code|$key}} is the secret key which is used for both encryption and decryption of input data. It must be a string or binary item. Its length is fixed and depends on the chosen algorithm: 8 bytes for {{Code|DES}}, 16 bytes for {{Code|AES}}.<br />
* {{Code|$algorithm}} must either be {{Code|DES}} or {{Code|AES}}. Default is {{Code|DES}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|CX0016|#Errors}} padding problems arise.<br/>{{Error|CX0017|#Errors}} padding is incorrect.<br/>{{Error|CX0018|#Errors}} the encryption type is not supported.<br/>{{Error|CX0019|#Errors}} the secret key is invalid.<br/>{{Error|CX0020|#Errors}} the block size is incorrect.<br/>{{Error|CX0021|#Errors}} the specified encryption algorithm is not supported.<br/><br />
|- valign="top"<br />
| '''Example'''<br />
|Decrypt input data and return original string:<br />
'''Query:'''<br />
<pre lang='xquery'><br />
let $encrypted := crypto:encrypt('message', 'symmetric', 'keykeyke', 'DES')<br />
return crypto:decrypt($encrypted, 'symmetric', 'keykeyke', 'DES')<br />
</pre><br />
<br />
'''Result:'''<br />
<pre lang="xml"><br />
message<br />
</pre><br />
|}<br />
<br />
=XML Signatures=<br />
<br />
[https://www.w3.org/TR/xmldsig-core/ XML Signatures] are used to sign data. In our case, the data which is signed is an XQuery node. The following example shows the basic structure of an XML signature.<br />
<br />
'''XML Signature'''<br />
<pre lang="xml"><br />
<Signature><br />
<SignedInfo><br />
<CanonicalizationMethod/><br />
<SignatureMethod/><br />
<Reference><br />
<Transforms/><br />
<DigestMethod/><br />
<DigestValue/><br />
</Reference><br />
<Reference/><br />
</SignedInfo><br />
<SignatureValue/><br />
<KeyInfo/><br />
<Object/><br />
</Signature><br />
</pre><br />
<br />
* '''SignedInfo''' contains or references the signed data and lists algorithm information<br />
* '''Reference''' references the signed node<br />
* '''Transforms''' contains transformations (i.e. XPath expressions) that are applied to the input node in order to sign a subset <br />
* '''DigestValue''' holds digest value of the transformed references<br />
* '''SignatureValue''' contains the Base64 encoded value of the encrypted digest of the {{Code|SignedInfo}} element<br />
* '''KeyInfo''' provides information on the key that is used to validate the signature<br />
* '''Object''' contains the node which is signed if the signature is of type {{Code|enveloping}}<br />
<br />
'''Signature Types'''<br />
<br />
Depending on the signature type, the {{Code|signature}} element is either placed as a child of the signed node ({{Code|enveloped}} type), or directly contains the signed node ({{Code|enveloping}} type). {{Code|Detached}} signatures are so far not supported.<br />
<br />
'''Digital Certificate'''<br />
<br />
The {{Code|generate-signature}} function allows to pass a {{Code|digital certificate}}. This certificate holds parameters that allow to access key information stored in a Java key store which is then used to sign the input document. Passing a {{Code|digital certificate}} simply helps re-using the same key pair to sign and validate data. The {{Code|digital certificate}} is passed as a node and has the following form:<br />
<br />
<pre lang="xml"><br />
<digital-certificate><br />
<keystore-type>JKS</keystore-type><br />
<keystore-password>...</keystore-password><br />
<key-alias>...</key-alias><br />
<private-key-password>...</private-key-password><br />
<keystore-uri>...</keystore-uri><br />
</digital-certificate><br />
</pre><br />
<br />
==crypto:generate-signature==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>crypto:generate-signature(<br />
$input as node(),<br />
$canonicalization as xs:string,<br />
$digest as xs:string,<br />
$signature as xs:string,<br />
$prefix as xs:string,<br />
$type as xs:string,<br />
$ext1 as item(),<br />
$ext2 as node()<br />
) as node()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|{{Code|$canonicalization}} must either be {{Code|inclusive-with-comments}}, {{Code|inclusive}}, {{Code|exclusive-with-comments}} or {{Code|exclusive}}. '''Default is {{Code|inclusive-with-comments}}'''.<br/><br />
{{Code|$digest}} must be one of the following: {{Code|SHA1}}, {{Code|SHA256}} or {{Code|SHA512}}. '''Default is {{Code|SHA1}}'''.<br/><br />
{{Code|$signature}} must either be {{Code|RSA_SHA1}} or {{Code|DSA_SHA1}}. '''Default is {{Code|RSA_SHA1}}'''.<br/><br />
{{Code|$prefix}} may be empty and prefixes the {{Code|Signature}} element accordingly.<br/><br />
{{Code|$type}} is the signature type. It must either be {{Code|enveloped}} or {{Code|enveloping}} (detached signatures are not supported so far). '''Default is {{Code|enveloped}}'''.<br/><br />
{{Code|$ext1}} may either be an {{Code|$xpath}} expression or a {{Code|$certificate}}.<br/><br />
If {{Code|$ext2}} is specified as well, {{Code|$ext1}} is an arbitrary XPath expression which specifies a subset of the document that is to be signed, and {{Code|$ext2}} is the digitial certificate used to sign the input document.<br/><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|CX0001|#Errors}} the canonicalization algorithm is not supported.<br/>{{Error|CX0002|#Errors}} the digest algorithm is not supported.<br/>{{Error|CX0003|#Errors}} the signature algorithm is not supported.<br/>{{Error|CX0004|#Errors}} the {{Code|$xpath-expression}} is invalid.<br/>{{Error|CX0005|#Errors}} the root name of {{Code|$digital-certificate}} is not 'digital-certificate.<br/>{{Error|CX0007|#Errors}} the key store is null.<br/>{{Error|CX0012|#Errors}} the key cannot be found in the specified key store.<br/>{{Error|CX0023|#Errors}} the certificate alias is invalid.<br/>{{Error|CX0024|#Errors}} an invalid algorithm is specified.<br/>{{Error|CX0025|#Errors}} an exception occurs while the signing the document.<br/>{{Error|CX0026|#Errors}} an exception occurs during key store initialization.<br/>{{Error|CX0027|#Errors}} an IO exception occurs.<br/>{{Error|CX0028|#Errors}} the signature type is not supported.<br/><br />
|- valign="top"<br />
| '''Example'''<br />
|Generate [https://www.w3.org/TR/xmldsig-core/ XML Signature]:<br />
<br />
'''Query:'''<br />
<pre lang='xquery'><br />
crypto:generate-signature(<a/>, '', '', '', '', '')<br />
</pre><br />
<br />
'''Result:'''<br />
<pre lang="xml"><br />
<a><br />
<Signature xmlns="http://www.w3.org/2000/09/xmldsig#"><br />
<SignedInfo><br />
<CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315#WithComments"/><br />
<SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/><br />
<Reference URI=""><br />
<Transforms><br />
<Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/><br />
</Transforms><br />
<DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/><br />
<DigestValue>9hvH4qztnIYgYfJDRLnEMPJdoaY=</DigestValue><br />
</Reference><br />
</SignedInfo><br />
<SignatureValue>Pn/Jr44WBcdARff2UVYEiwYW1563XdqnU87nusAIaHgzd+U3SrjVJhPFLDe0DJfxVtYzLFaznTYE<br />
P3ddeoFmyA==</SignatureValue><br />
<KeyInfo><br />
<KeyValue><br />
<RSAKeyValue><br />
<Modulus>rtvpFSbCIE2BJePlVYLIRIjXl0R7ESr2+D+JOVKn7AM7VZbcbRDPeqRbjSkEz1HWC/N067tjB3qH<br />
4/4PPT9bGQ==</Modulus><br />
<Exponent>AQAB</Exponent><br />
</RSAKeyValue><br />
</KeyValue><br />
</KeyInfo><br />
</Signature><br />
</a><br />
</pre><br />
|}<br />
<br />
==crypto:validate-signature==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>crypto:validate-signature(<br />
$input-doc as node()<br />
) as xs:boolean</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Checks if the given node contains a {{Code|Signature}} element and whether the signature is valid. In this case {{Code|true}} is returned. If the signature is invalid the function returns {{Code|false}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|CX0015|#Errors}} the signature element cannot be found.<br/>{{Error|CX9994|#Errors}} an unspecified problem occurs during validation.<br/>{{Error|CX9996|#Errors}} an IO exception occurs during validation.<br/><br />
|- valign="top"<br />
| '''Example'''<br />
|Validate [https://www.w3.org/TR/xmldsig-core/ XML Signature]:<br />
<br />
'''Query:'''<br />
<pre lang='xquery'><br />
let $sig := crypto:generate-signature(<a/>, '', '', '', '', '')<br />
return crypto:validate-signature($sig)<br />
</pre><br />
<br />
'''Result:'''<br />
<pre lang="xml"><br />
true<br />
</pre><br />
|}<br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
|Description<br />
|- valign="top"<br />
|{{Code|CX0001}}<br />
|The canonicalization algorithm is not supported.<br />
|- valign="top"<br />
|{{Code|CX0002}}<br />
|The digest algorithm is not supported.<br />
|- valign="top"<br />
|{{Code|CX0003}}<br />
|The signature algorithm is not supported.<br />
|- valign="top"<br />
|{{Code|CX0004}}<br />
|The XPath expression is invalid.<br />
|- valign="top"<br />
|{{Code|CX0005}}<br />
|The root element of argument $digital-certificate must have the name 'digital-certificate'.<br />
|- valign="top"<br />
|{{Code|CX0006}}<br />
|The child element of argument $digital-certificate having position $position must have the name $child-element-name.<br />
|- valign="top"<br />
|{{Code|CX0007}}<br />
|The keystore is null.<br />
|- valign="top"<br />
|{{Code|CX0008}}<br />
|I/O error while reading keystore.<br />
|- valign="top"<br />
|{{Code|CX0009}}<br />
|Permission denied to read keystore.<br />
|- valign="top"<br />
|{{Code|CX0010}}<br />
|The keystore URL is invalid.<br />
|- valign="top"<br />
|{{Code|CX0011}}<br />
|The keystore type is not supported.<br />
|- valign="top"<br />
|{{Code|CX0012}}<br />
|Cannot find key for alias in given keystore.<br />
|- valign="top"<br />
|{{Code|CX0013}}<br />
|The hashing algorithm is not supported.<br />
|- valign="top"<br />
|{{Code|CX0014}}<br />
|The encoding method is not supported.<br />
|- valign="top"<br />
|{{Code|CX0015}}<br />
|Cannot find Signature element.<br />
|- valign="top"<br />
|{{Code|CX0016}}<br />
|No such padding.<br />
|- valign="top"<br />
|{{Code|CX0017}}<br />
|Incorrect padding.<br />
|- valign="top"<br />
|{{Code|CX0018}}<br />
|The encryption type is not supported.<br />
|- valign="top"<br />
|{{Code|CX0019}}<br />
|The secret key is invalid.<br />
|- valign="top"<br />
|{{Code|CX0020}}<br />
|Illegal block size.<br />
|- valign="top"<br />
|{{Code|CX0021}}<br />
|The algorithm is not supported.<br />
|- valign="top"<br />
|{{Code|CX0023}}<br />
|An invalid certificate alias is specified. Added to the official specification.<br />
|- valign="top"<br />
|{{Code|CX0024}}<br />
|The algorithm is invalid. Added to the official specification.<br />
|- valign="top"<br />
|{{Code|CX0025}}<br />
|Signature cannot be processed. Added to the official specification.<br />
|- valign="top"<br />
|{{Code|CX0026}}<br />
|Keystore cannot be processed. Added to the official specification.<br />
|- valign="top"<br />
|{{Code|CX0027}}<br />
|An I/O Exception occurred. Added to the official specification.<br />
|- valign="top"<br />
|{{Code|CX0028}}<br />
|The specified signature type is not supported. Added to the official specification.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 9.3<br />
* Updated: {{Function||crypto:hmac}}, {{Function||crypto:encrypt}}, {{Function||crypto:decrypt}}: Function types revised.<br />
<br />
;Version 8.6<br />
* Updated: {{Function||crypto:hmac}}: The key can now be a string or a binary item.<br />
<br />
The Module was introduced with Version 7.0.</div>CGhttps://docs.basex.org/index.php?title=Parsers&diff=16892Parsers2023-12-01T16:39:25Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This article is part of the [[Advanced User's Guide]]. It presents the available parsers that can be used to import various data sources in BaseX databases. Please visit the [[Serialization]] article if you want to know how to export data.<br />
<br />
=XML Parsers=<br />
<br />
BaseX provides two alternatives for parsing XML:<br />
<br />
* By default, Java’s [https://docs.oracle.com/en/java/javase/11/docs/api/java.xml/javax/xml/parsers/SAXParser.html SAXParser] is used to parse XML documents.<br />
* The internal, built-in XML parser is more fault-tolerant than Java’s XML parser. It supports standard HTML entities out of the box, and it is faster than the default parser, in particular if small documents are to be parsed. However, the internal parser does not support the full range of DTD features and cannot resolve [[Catalog_Resolver|catalogs]].<br />
<br />
==GUI==<br />
<br />
Go to Menu ''Database'' → ''New'', then choose the ''Parsing'' tab and (de)activate ''Use internal XML parser''. The parsing of DTDs can be turned on/off by selecting the checkbox below.<br />
<br />
==Command Line==<br />
<br />
To turn the internal XML parser and DTD parsing on/off, modify the <code>INTPARSE</code> and <code>DTD</code> options:<br />
<br />
SET {{Option|INTPARSE}} true<br />
SET {{Option|DTD}} true<br />
<br />
==XQuery==<br />
<br />
The {{Function|Database|db:add}} and {{Function|Database|db:put}} functions can also be used to add new XML documents to the database. The following example query uses the internal XML parser and adds all files to the database <code>DB</code> that are found in the directory <code>2Bimported</code>:<br />
<br />
<pre lang='xquery'><br />
for $file in file:list("2Bimported")<br />
return db:add('DB', $file, '', map { 'intparse': true() })<br />
</pre><br />
<br />
=HTML Parser=<br />
<br />
If [http://vrici.lojban.org/~cowan/tagsoup/ TagSoup] is found in the [[Startup#Distributions|classpath]], HTML input is converted to well-formed XML documents.<br />
<br />
If TagSoup is not available on a system, the default XML parser will be used, and the import will only succeed if the input is already well-formed XML (which is usually the case for XHTML documents).<br />
<br />
==Installation==<br />
<br />
===Downloads===<br />
<br />
TagSoup is already included in the full BaseX distributions ({{Code|BaseX.zip}}, {{Code|BaseX.exe}}, etc.). It can also be manually downloaded and embedded on the appropriate platforms.<br />
<br />
===Maven===<br />
<br />
An easy way to add TagSoup to your project is to as follows:<br />
<br />
1. Visit [https://mvnrepository.com/artifact/org.ccil.cowan.tagsoup/tagsoup/ MVN TagSoup Repository]<br />
<br />
2. Click on the version you want<br />
<br />
3. On the first tab, you can see an XML snippet like this:<br />
<br />
<pre lang="xml"><br />
<dependency><br />
<groupId>org.ccil.cowan.tagsoup</groupId><br />
<artifactId>tagsoup</artifactId><br />
<version>1.2.1</version><br />
</dependency><br />
</pre><br />
<br />
4. Insert the XML fragment into the <code><dependencies></code> element of your project’s <code>pom.xml</code> file.<br />
<br />
===Debian===<br />
<br />
With Debian, TagSoup will automatically be detected and included after it has been installed via:<br />
<br />
apt-get install libtagsoup-java<br />
<br />
==Options==<br />
<br />
TagSoup offers a variety of options to customize the HTML conversion. For the complete list, please visit the [http://vrici.lojban.org/~cowan/tagsoup/ TagSoup] website. BaseX supports most options, with a few exceptions:<br />
<br />
* '''encoding''': BaseX tries to guess the input encoding, but this can be overwritten by this option.<br />
* '''files''': not supported as the input documents are piped directly to the XML parser.<br />
* '''method''': set to 'xml' as default. If this is set to 'html' ending tags may be missing for instance.<br />
* '''version''': dismissed, as TagSoup always falls back to 'version 1.0', no matter what the input is.<br />
* '''standalone''': deactivated.<br />
* '''pyx''', '''pyxin''': not supported as the XML parser can't handle this kind of input.<br />
* '''output-encoding''': not supported, BaseX already takes care of that.<br />
* '''reuse''', '''help''': not supported.<br />
<br />
==GUI==<br />
<br />
Go to Menu ''Database'' → ''New'' and select "HTML" in the input format combo box. There's an info in the "Parsing" tab about whether TagSoup is available or not. The same applies to the "Resources" tab in the "Database Properties" dialog.<br />
<br />
These two dialogs come with an input field 'Parameters' where TagSoup options can be entered. Keys and values are separated by the equal sign, multiple options are separated by commas, e.g.: <code>method=xml,nons=true</code>.<br />
<br />
==Command Line==<br />
<br />
Turn on the HTML Parser before parsing documents, set a file filter and optional parsing options:<br />
<br />
SET {{Option|PARSER}} html<br />
SET {{Option|CREATEFILTER}} *.html<br />
SET {{Option|HTMLPARSER}} method=xml,nons=true<br />
<br />
==XQuery==<br />
<br />
The [[HTML Module]] provides XQuery functions for converting HTML to XML. In addition, all functions that accept [[Options#Parsing|Parsing Options]] can be used as well to convert HTML:<br />
<br />
<pre lang='xquery'><br />
fetch:doc("index.html", map {<br />
'parser': 'html',<br />
'htmlparser': map { 'method': 'xml', 'nons': true() }<br />
})<br />
</pre><br />
<br />
=JSON Parser=<br />
<br />
With BaseX, JSON documents can be imported. The JSON parsing options are described in the documentation of the [[JSON Module]].<br />
<br />
==GUI==<br />
<br />
Go to Menu ''Database'' → ''New'' and select "JSON" in the input format combo box. You can set the following options for parsing JSON documents in the "Parsing" tab.<br />
<br />
==Command Line==<br />
<br />
Turn on the JSON Parser before parsing documents, and set some optional, parser-specific options and a file filter:<br />
<br />
SET {{Option|PARSER}} json<br />
SET {{Option|JSONPARSER}} encoding=utf-8, jsonml=true<br />
SET {{Option|CREATEFILTER}} *.json<br />
<br />
==XQuery==<br />
<br />
The [[JSON Module]] provides functions for converting JSON to XML.<br />
<br />
In addition, all functions that accept [[Options#Parsing|Parsing Options]] can be used as well to convert JSON. The following query converts a JSON string to an XML document, using the conversion format {{Code|[[JSON Module#Basic|basic]]}}:<br />
<br />
<pre lang='xquery'><br />
db:create('json-db', '{ "key": "value" }', 'example.json', map {<br />
'parser': 'json',<br />
'jsonparser': map { 'format': 'basic' }<br />
})<br />
</pre><br />
<br />
=CSV Parser=<br />
<br />
BaseX can be used to import CSV documents. The CSV parsing options are described in the documentation of the [[CSV Module]].<br />
<br />
==GUI==<br />
<br />
Go to Menu ''Database'' → ''New'' and select "CSV" in the input format combo box.<br />
You can set the following options for parsing CSV documents in the "Parsing" tab.<br />
<br />
==Command Line==<br />
<br />
Enable CSV Parsing before parsing documents, and set optional, parser-specific options and a file filter:<br />
<br />
SET {{Option|PARSER}} csv<br />
SET {{Option|CSVPARSER}} encoding=utf-8,header=true,separator=semicolon<br />
SET {{Option|CREATEFILTER}} *.csv<br />
<br />
==XQuery==<br />
<br />
The [[CSV Module]] provides a function for converting CSV to XML.<br />
<br />
In addition, all functions that accept [[Options#Parsing|Parsing Options]] can be used as well to convert CSV. In the following, all CSV files from a directory are added to a database, with the first line of each file interpreted as column header:<br />
<br />
<pre lang='xquery'><br />
for $file in file:list("2Bimported", false(), "*.csv")<br />
return db:add("csv-db", $file, "", map {<br />
'parser': 'csv',<br />
'csvparser': map { 'header': true() }<br />
})<br />
</pre><br />
<br />
=Changelog=<br />
<br />
;Version 11.0<br />
* Removed: Text Parser<br />
<br />
;Version 7.8<br />
* Updated: parser options<br />
<br />
;Version 7.7.2<br />
* Removed: CSV option "format"<br />
<br />
;Version 7.3<br />
* Updated: the CSV {{Code|SEPARATOR}} option may now be assigned arbitrary single characters<br />
<br />
;Version 7.2<br />
* Updated: Enhanced support for TagSoup options</div>CGhttps://docs.basex.org/index.php?title=Higher-Order_Functions&diff=16893Higher-Order Functions2023-12-01T16:39:25Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This page present some ''higher-order functions'' of the XQuery specification. The BaseX-specific [[Higher-Order Functions Module]] contains some additional useful functions.<br />
<br />
=Function Items=<br />
<br />
Probably the most important new feature in XQuery 3.0 are ''function items'', i. e., items that act as functions, but can also be passed to and from other functions and expressions. This feature makes functions ''first-class citizens'' of the language. The [[XQuery_3.0#Function_Items|XQuery 3.0]] page goes into details on how function items can be obtained.<br />
<br />
== Function Types ==<br />
<br />
Like every XQuery item, function items have a ''sequence type''. It can be used to specify the ''arity'' (number of arguments the function takes) and the argument and result types.<br />
<br />
The most general function type is <code>function(*)</code>. It's the type of all function items. The following query for example goes through a list of XQuery items and, if it is a function item, prints its arity:<br />
<br />
<pre lang='xquery'><br />
for $item in (1, 'foo', fn:concat#3, function($a) { 42 * $a })<br />
where $item instance of function(*)<br />
return fn:function-arity($item)<br />
</pre><br />
''Result:'' <code>3 1</code><br />
<br />
The notation for specifying argument and return types is quite intuitive, as it closely resembles the function declaration. The XQuery function<br />
<br />
<pre lang='xquery'><br />
declare function local:char-at(<br />
$str as xs:string,<br />
$pos as xs:integer<br />
) as xs:string {<br />
fn:substring($str, $pos, 1)<br />
};<br />
</pre><br />
<br />
for example has the type <code>function(xs:string, xs:integer) as xs:string</code>. It isn't possible to specify only the argument and not the result type or the other way round. A good place-holder to use when no restriction is wanted is <code>item()*</code>, as it matches any XQuery value.<br />
<br />
Function types can also be nested. As an example we take <code>local:on-sequences</code>, which takes a function defined on single items and makes it work on sequences as well:<br />
<br />
<pre lang='xquery'><br />
declare function local:on-sequences(<br />
$fun as function(item()) as item()*<br />
) as function(item()*) as item()* {<br />
fn:for-each($fun, ?)<br />
};<br />
</pre><br />
<br />
We willl see later how <code>fn:for-each(...)</code> works. The type of <code>local:on-sequences(...)</code> on the other hand is easily constructed, if a bit long:<br />
<br />
<code>function(function(item()) as item()*) as function(item()*) as item()*</code>.<br />
<br />
=Higher-Order Functions=<br />
<br />
A ''higher-order function'' is a function that takes other functions as arguments and/or returns them as results. <code>fn:for-each</code> and <code>local:on-sequences</code> from the last chapter are nice examples.<br />
<br />
With the help of higher-order functions, one can extract common patterns of ''behavior'' and abstract them into a library function.<br />
<br />
== Sequences ==<br />
<br />
Some usage patterns on sequences are so common that the higher-order functions describing them are in the XQuery standard libraries. They are listed here, together with their possible XQuery implementation and some motivating examples.<br />
<br />
===fn:for-each===<br />
<br />
{|<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>fn:for-each(<br />
$input as item()*,<br />
$action as function(item()) as item()*<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Applies the specified <code>$action</code> to every item of <code>$input</code> and returns all results as a single sequence.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
<ul><li>Square all numbers from 1 to 10:<br />
<pre lang='xquery'><br />
fn:for-each(1 to 10, math:pow(?, 2))<br />
</pre><br />
''Result:'' <code>1 4 9 16 25 36 49 64 81 100</code><br />
</li><br />
<li>Apply a list of functions to a string:<br />
<pre lang='xquery'><br />
let $fs := (<br />
fn:upper-case#1,<br />
fn:substring(?, 4),<br />
fn:string-length#1<br />
)<br />
return fn:for-each($fs, function($f) { $f('foobar') })<br />
</pre><br />
''Result:'' <code>FOOBAR bar 6</code><br />
</li><br />
<li>Process each item of a sequence with the arrow operator:<br />
<pre lang='xquery'><br />
("one", "two", "three") => fn:for-each(fn:upper-case(?))<br />
</pre><br />
''Result:'' <code>ONE TWO THREE</code><br />
</li></ul><br />
|- valign="top"<br />
| '''XQuery 1.0'''<br />
|At the core, for-each is nothing else than a simple FLWOR expression:<br />
<pre lang='xquery'><br />
declare function local:for-each(<br />
$seq as item()*,<br />
$fun as function(item()) as item()*<br />
) as item()* {<br />
for $s in $seq<br />
return $fun($s)<br />
};<br />
</pre><br />
|}<br />
<br />
===fn:filter===<br />
<br />
{|<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>fn:filter(<br />
$input as item()*,<br />
$predicate as function(item()) as xs:boolean<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Applies the boolean <code>$predicate</code> to all elements of the sequence <code>$input</code>, returning those for which it returns <code>true()</code>.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<ul><li>All even integers until 10:<br />
<pre lang='xquery'><br />
fn:filter(1 to 10, function($x) { $x mod 2 eq 0 })<br />
</pre><br />
''Result:'' <code>2 4 6 8 10</code><br />
</li><br />
<li>Strings that start with an upper-case letter:<br />
<pre lang='xquery'><br />
let $first-upper := function($str) {<br />
let $first := fn:substring($str, 1, 1)<br />
return $first eq fn:upper-case($first)<br />
}<br />
return fn:filter(('FooBar', 'foo', 'BAR'), $first-upper)<br />
</pre><br />
''Result:'' <code>FooBar BAR</code><br />
</li><br />
<li>Inefficient prime number generator:<br />
<pre lang='xquery'><br />
let $is-prime := function($x) {<br />
$x gt 1 and (every $y in 2 to ($x - 1) satisfies $x mod $y ne 0)<br />
}<br />
return filter(1 to 20, $is-prime)<br />
</pre><br />
''Result:'' <code>2 3 5 7 11 13 17 19</code><br />
</li></ul><br />
|- valign="top"<br />
| '''Note'''<br />
|<code>fn:filter</code> can be easily implemented with <code>fn:for-each</code>:<br />
<pre lang='xquery'><br />
declare function local:filter($seq, $pred) {<br />
for-each(<br />
$seq,<br />
function($x) {<br />
if($pred($x)) then $x else ()<br />
}<br />
)<br />
};<br />
</pre><br />
|- valign="top"<br />
| '''XQuery 1.0'''<br />
|At the core, for-each is nothing else than a filter expression:<br />
<pre lang='xquery'><br />
declare function local:filter(<br />
$seq as item()*,<br />
$pred as function(item()) as xs:boolean<br />
) as item()* {<br />
$seq[$pred(.)]<br />
};<br />
</pre><br />
|}<br />
<br />
===fn:for-each-pair===<br />
<br />
{|<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>fn:for-each-pair(<br />
$input1 as item()*,<br />
$input2 as item()*,<br />
$action as function(item(), item()) as item()*<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Applies the specified {{Code|$action}} to the successive pairs of items of <code>$input1</code> and <code>$input2</code>. Evaluation is stopped if one sequence yields no more items.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<ul><li>Adding one to the numbers at odd positions:<br />
<pre lang='xquery'><br />
fn:for-each-pair(<br />
fn:for-each(1 to 10, function($x) { $x mod 2 }),<br />
(1, 1, 1, 1, 1),<br />
function($a, $b) { $a + $b }<br />
)<br />
</pre><br />
''Result:'' <code>2 1 2 1 2</code><br />
</li><br />
<li>Line numbering:<br />
<pre lang='xquery'><br />
let $number-words := function($str) {<br />
fn:string-join(<br />
fn:for-each-pair(<br />
1 to 1000000000,<br />
tokenize($str, ' +'),<br />
concat(?, ': ', ?)<br />
),<br />
'&#xa;'<br />
)<br />
}<br />
return $number-words('how are you?')<br />
</pre><br />
''Result:''<br />
<pre lang='xquery'><br />
1: how<br />
2: are<br />
3: you?<br />
</pre><br />
</li><br />
<li>Checking if a sequence is sorted:<br />
<pre lang='xquery'><br />
let $is-sorted := function($seq) {<br />
every $b in<br />
fn:for-each-pair(<br />
$seq,<br />
tail($seq),<br />
function($a, $b) { $a le $b }<br />
)<br />
satisfies $b<br />
}<br />
return (<br />
$is-sorted(1 to 10),<br />
$is-sorted((1, 2, 42, 4, 5))<br />
)<br />
</pre><br />
''Result:'' <code>true false</code></li></ul><br />
|- valign="top"<br />
| '''XQuery 1.0'''<br />
|<pre lang='xquery'><br />
declare function local:for-each-pair(<br />
$seq1 as item()*,<br />
$seq2 as item()*,<br />
$fun as function(item(), item()) as item()*<br />
) as item()* {<br />
for $pos in 1 to min((count($seq1), count($seq2)))<br />
return $fun($seq1[$pos], $seq2[$pos])<br />
};<br />
</pre><br />
|}<br />
<br />
== Folds ==<br />
<br />
A ''fold'', also called ''reduce'' or ''accumulate'' in other languages, is a very basic higher-order function on sequences. It starts from a seed value and incrementally builds up a result, consuming one element from the sequence at a time and combining it with the aggregate of a user-defined function.<br />
<br />
Folds are one solution to the problem of not having ''state'' in functional programs. Solving a problem in ''imperative'' programming languages often means repeatedly updating the value of variables, which isn't allowed in functional languages.<br />
<br />
Calculating the ''product'' of a sequence of integers for example is easy in <code>Java</code>:<br />
<br />
<pre lang="java"><br />
public int product(int[] seq) {<br />
int result = 1;<br />
for(int i : seq) {<br />
result = result * i;<br />
}<br />
return result;<br />
}<br />
</pre><br />
<br />
Nice and efficient implementations using folds will be given below.<br />
<br />
The ''linear'' folds on sequences come in two flavors. They differ in the direction in which they traverse the sequence:<br />
<br />
===fn:fold-left===<br />
<br />
{|<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>fn:fold-left(<br />
$input as item()*,<br />
$zero as item()*,<br />
$action as function(item()*, item()) as item()*<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|The ''left fold'' traverses the {{Code|$input}} from the left.<br />
The query <code>fn:fold-left(1 to 5, 0, $f)</code>, for example, would be evaluated as:<br />
<pre lang='xquery'><br />
$f($f($f($f($f(0, 1), 2), 3), 4), 5)<br />
</pre><br />
|- valign="top"<br />
| '''Examples'''<br />
|<ul><li>Product of a sequence of integers:<br />
<pre lang='xquery'><br />
fn:fold-left(1 to 5, 1,<br />
function($result, $curr) { $result * $curr }<br />
)<br />
</pre><br />
''Result:'' <code>120</code><br />
</li><br />
<li>Illustrating the evaluation order:<br />
<pre lang='xquery'><br />
fn:fold-left(1 to 5, '$seed',<br />
concat('$f(', ?, ', ', ?, ')')<br />
)<br />
</pre><br />
''Result:'' <code>$f($f($f($f($f($seed, 1), 2), 3), 4), 5)</code><br />
</li><br />
<li>Building a decimal number from digits:<br />
<pre lang='xquery'><br />
let $from-digits := fold-left(?, 0,<br />
function($n, $d) { 10 * $n + $d }<br />
)<br />
return (<br />
$from-digits(1 to 5),<br />
$from-digits((4, 2))<br />
)<br />
</pre><br />
''Result:'' <code>12345 42</code><br />
</li></ul><br />
|- valign="top"<br />
| '''XQuery 1.0'''<br />
|As folds are more general than ''FLWOR'' expressions, the implementation isn't as concise as the former ones:<br />
<pre lang='xquery'><br />
declare function local:fold-left(<br />
$input as item()*,<br />
$zero as item()*,<br />
$action as function(item()*, item()) as item()*<br />
) as item()* {<br />
if(empty($input)) then $zero<br />
else local:fold-left(<br />
tail($input),<br />
$action($zero, head($input)),<br />
$action<br />
) <br />
};<br />
</pre><br />
|}<br />
<br />
===fn:fold-right===<br />
<br />
{|<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>fn:fold-right(<br />
$input as item()*,<br />
$zero as item()*,<br />
$action as function(item(), item()*) as item()*<br />
) as item()*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|The ''right fold'' traverses the {{Code|$input}} from the right.<br />
The query <code>fn:fold-right(1 to 5, 0, $f)</code>, for example, would be evaluated as:<br />
<pre lang='xquery'><br />
$f(1, $f(2, $f(3, $f(4, $f(5, 0)))))<br />
</pre><br />
|- valign="top"<br />
| '''Examples'''<br />
|<ul><li>Product of a sequence of integers:<br />
<pre lang='xquery'><br />
fn:fold-right(1 to 5, 1,<br />
function($curr, $result) { $result * $curr }<br />
)<br />
</pre><br />
''Result:'' <code>120</code><br />
</li><br />
<li>Illustrating the evaluation order:<br />
<pre lang='xquery'><br />
fn:fold-right(1 to 5, '$seed',<br />
concat('$f(', ?, ', ', ?, ')')<br />
)<br />
</pre><br />
''Result:'' <code>$f(1, $f(2, $f(3, $f(4, $f(5, $seed)))))</code><br />
</li><br />
<li>Reversing a sequence of items:<br />
<pre lang='xquery'><br />
let $reverse := fn:fold-right(?, (),<br />
function($item, $rev) {<br />
$rev, $item<br />
}<br />
)<br />
return $reverse(1 to 10)<br />
</pre><br />
''Result:'' <code>10 9 8 7 6 5 4 3 2 1</code><br />
</li></ul><br />
|- valign="top"<br />
| '''XQuery 1.0'''<br />
|<pre lang='xquery'><br />
declare function local:fold-right(<br />
$input as item()*,<br />
$zero as item()*,<br />
$action as function(item(), item()*) as item()*<br />
) as item()* {<br />
if(empty($input)) then $zero<br />
else $action(<br />
head($input),<br />
local:fold-right(tail($input), $zero, $action)<br />
)<br />
};<br />
</pre><br />
Note that the order of the arguments of <code>$fun</code> are inverted compared to that in <code>fn:fold-left(...)</code>.<br />
|}</div>CGhttps://docs.basex.org/index.php?title=RESTXQ&diff=16891RESTXQ2023-12-01T16:39:21Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This page presents one of the [[Web Application]] services. It describes how to use the RESTXQ API of BaseX.<br />
<br />
RESTXQ, introduced by [http://www.adamretter.org.uk/ Adam Retter], is an API that facilitates the use of XQuery as a server-side processing language for the Web. It has been inspired by the Java [https://en.wikipedia.org/wiki/Java_API_for_RESTful_Web_Services JAX-RS API]: It provides a pre-defined set of XQuery 3.0 annotations for mapping HTTP requests to XQuery functions, which in turn generate and return HTTP responses.<br />
<br />
BaseX provides various extensions to the original draft of the specification:<br />
<br />
* Multipart types are supported, including {{Code|multipart/form-data}}<br />
* A {{Code|%rest:error}} annotation can be used to catch XQuery errors<br />
* Servlet errors can be redirected to other RESTXQ pages<br />
* A [[RESTXQ Module]] provides some helper functions<br />
* Parameters are implicitly cast to the type of the function argument<br />
* The [[#Paths|Path Annotation]] can contain regular expressions<br />
* <code>%input</code> annotations, support for input-specific content-type parameters<br />
* <code>%rest:single</code> annotation to cancel running RESTXQ functions<br />
* Quality factors in the [[#Content Negotiation|Accept header]] will be evaluated<br />
* Support for server-side quality factors in the [[#Content Negotiation|<code>%rest:produces</code>]] annotation<br />
* Better support for the OPTIONS and HEAD methods<br />
<br/><br />
<br />
=Introduction=<br />
<br />
==Preliminaries==<br />
<br />
The RESTXQ service is accessible via {{Code|http://localhost:8080/}}.<br />
<br />
All RESTXQ [[XQuery 3.0#Annotations|annotations]] are assigned to the <code><nowiki>http://exquery.org/ns/restxq</nowiki></code> namespace, which is statically bound to the {{Code|rest}} prefix. A ''Resource Function'' is an XQuery function that has been marked up with RESTXQ annotations. When an HTTP request comes in, a resource function will be invoked that matches the constraints indicated by its annotations.<br />
<br />
If a RESTXQ URL is requested, the {{Option|RESTXQPATH}} module directory and its subdirectories will be traversed, and all [[XQuery Extensions#Suffixes|XQuery files]] will be parsed for functions with RESTXQ annotations. Subdirectories that include an {{Code|.ignore}} file will be skipped.<br />
<br />
To speed up processing, the functions of the existing XQuery modules are automatically cached in main memory:<br />
* Functions will be invalidated and parsed again if the timestamp of their module changes.<br />
* File monitoring can be adjusted via the {{Option|PARSERESTXQ}} option. In productive environments with a high load, it may be recommendable to change the timeout, or completely disable monitoring.<br />
* If files are replaced while the web server is running, the RESTXQ module cache should be explicitly invalidated by calling the static root path {{Code|/.init}} or by calling the {{Function|RESTXQ|rest:init}} function.<br />
<br />
==Examples==<br />
<br />
A first RESTXQ function is shown below:<br />
<br />
<pre lang='xquery'><br />
module namespace page = 'http://basex.org/examples/web-page';<br />
<br />
declare %rest:path("hello/{$who}") %rest:GET function page:hello($who) {<br />
<response><br />
<title>Hello { $who }!</title><br />
</response><br />
};<br />
</pre><br />
<br />
If the URI http://localhost:8080/hello/World is accessed, the result will be:<br />
<br />
<pre lang="xml"><br />
<response><br />
<title>Hello World!</title><br />
</response><br />
</pre><br />
<br />
The next function demonstrates a POST request:<br />
<br />
<pre lang='xquery'><br />
declare<br />
%rest:path("/form")<br />
%rest:POST<br />
%rest:form-param("message","{$message}", "(no message)")<br />
%rest:header-param("User-Agent", "{$agent}")<br />
function page:hello-postman(<br />
$message as xs:string,<br />
$agent as xs:string*<br />
) as element(response) {<br />
<response type='form'><br />
<message>{ $message }</message><br />
<user-agent>{ $agent }</user-agent><br />
</response><br />
};<br />
</pre><br />
<br />
If you post something (e.g. using curl or the embedded form at http://localhost:8080/)...<br />
<br />
<pre lang="shell"><br />
curl -i -X POST --data "message='CONTENT'" http://localhost:8080/form<br />
</pre><br />
<br />
...you will receive something similar to the following result:<br />
<br />
<pre lang="shell"><br />
HTTP/1.1 200 OK<br />
Content-Type: application/xml; charset=UTF-8<br />
Content-Length: 107<br />
Server: Jetty(8.1.11.v20130520)<br />
</pre><br />
<br />
<pre lang="xml"><br />
<response type="form"><br />
<message>'CONTENT'</message><br />
<user-agent>curl/7.31.0</user-agent><br />
</response><br />
</pre><br />
<br />
=Request=<br />
<br />
This section shows how annotations are used to handle and process HTTP requests.<br />
<br />
==Constraints==<br />
<br />
Constraints restrict the HTTP requests that a resource function may process.<br />
<br />
===Paths===<br />
<br />
A resource function must have a single ''Path Annotation'' with a single string as argument. The function will be called if a URL matches the path segments and templates of the argument. ''Path templates'' contain variables in curly brackets, and map the corresponding segments of the request path to the arguments of the resource function. The first slash in the path is optional.<br />
<br />
The following example contains a path annotation with three segments and two templates. One of the function arguments is further specified with a data type, which means that the value for <code>$variable</code> will be cast to an <code>xs:integer</code> before being bound:<br />
<br />
<pre lang='xquery'><br />
declare %rest:path("/a/path/{$with}/some/{$variable}")<br />
function page:test($with, $variable as xs:integer) { ... };<br />
</pre><br />
<!-- TODO how matching works --><br />
<br />
Variables can be enhanced by regular expressions:<br />
<br />
<pre lang='xquery'><br />
(: Matches all paths with "app" as first, a number as second, and "order" as third segment :)<br />
declare %rest:path("app/{$code=[0-9]+}/order")<br />
function page:order($code) { ... };<br />
<br />
(: Matches all other all paths starting with "app/" :)<br />
declare %rest:path("app/{$path=.+}")<br />
function page:others($path) { ... };<br />
</pre><br />
<!-- TODO how matching works --><br />
<br />
If multiple path candidates are found for the request, the one with more segments will be preferred.<br />
<br />
===Content Negotiation===<br />
<br />
Functions can be restricted to specific Media Types. The default type is {{Code|*/*}}. Multiple types can either be specified by a single or by multiple annotations.<br />
<br />
====Consuming Data====<br />
<br />
A function will only be taken into consideration if the HTTP {{Code|Content-Type}} header of the request matches one of the given types:<br />
<br />
<pre lang='xquery'><br />
declare<br />
%rest:POST("{$body}")<br />
%rest:path("/xml")<br />
%rest:consumes("application/xml")<br />
%rest:consumes("text/xml")<br />
function page:xml($body) { $body };<br />
</pre><br />
<br />
====Producing Data====<br />
<br />
A function will only be chosen if the HTTP {{Code|Accept}} header of the request matches one of the given types:<br />
<br />
<pre lang='xquery'><br />
declare<br />
%rest:path("/xml")<br />
%rest:produces("application/xml", "text/xml")<br />
function page:xml() { <xml/> };<br />
</pre><br />
<br />
Note that the annotations will ''not'' affect the type of the actual response: You will need to supply an additional <code>[[#Output|%output:media-type]]</code> annotation or (if a single function may produce results of different types) generate an apt [[#Custom_Response|Custom Response]].<br />
<br />
====Quality Factors====<br />
<br />
A client can supply quality factors to influence the server-side function selection process. If a client sends the following HTTP header with quality factors…<br />
<br />
<pre><br />
Accept: */*;q=0.5,text/html;q=1.0<br />
</pre><br />
<br />
…and if two RESTXQ functions exist for the addressed path with two different annotations for producing data…<br />
<br />
<pre lang='xquery'><br />
declare function %rest:produces("text/html") ...<br />
...<br />
declare function %rest:produces("*/*") ...<br />
</pre><br />
<br />
…the first of these function will be chosen, as the quality factor for <code>text/html</code> documents is highest.<br />
<br />
As we cannot ensure that the client may supply quality factors, the selection process can also be controlled server-side. The <code>qs</code> parameter can be attached server-side to the Media Type. If multiple functions are left in the selection process, the one with the highest quality factor will be favored:<br />
<br />
<pre lang='xquery'><br />
declare function %rest:produces("application/json;qs=1") ...<br />
...<br />
declare function %rest:produces("*/*;qs=0.5") ...<br />
</pre><br />
<br />
===HTTP Methods===<br />
<br />
====Default Methods====<br />
<br />
The HTTP method annotations are equivalent to all [https://en.wikipedia.org/wiki/HTTP#Request_methods HTTP request methods] except TRACE and CONNECT. Zero or more methods may be used on a function; if none is specified, the function will be invoked for each method.<br />
<br />
The following function will be called if GET or POST is used as request method:<br />
<br />
<pre lang='xquery'><br />
declare %rest:GET %rest:POST %rest:path("/post")<br />
function page:post() { "This was a GET or POST request" };<br />
</pre><br />
<br />
The POST and PUT annotations may optionally take a string literal in order to map the HTTP request body to a [[#Parameters|function argument]]. Once again, the target variable must be embraced by curly brackets:<br />
<br />
<pre lang='xquery'><br />
declare %rest:PUT("{$body}") %rest:path("/put")<br />
function page:put($body) { "Request body: " || $body };<br />
</pre><br />
<br />
====Custom Methods====<br />
<br />
Custom HTTP methods can be specified with the {{Code|%rest:method}} annotation. An optional body variable can be supplied as second argument:<br />
<br />
<pre lang='xquery'><br />
declare<br />
%rest:path("binary-size")<br />
%rest:method("SIZE", "{$body}")<br />
function page:patch(<br />
$body as xs:base64Binary<br />
) {<br />
"Request method: " || request:method(),<br />
"Size of body: " || bin:length($body)<br />
};<br />
</pre><br />
<br />
If an OPTIONS request is received, and if no function is defined, an automatic response will be generated, which includes an <code>Allow</code> header with all supported methods.<br />
<br />
If a HEAD request is received, and if no function is defined, the corresponding GET function will be processed, but the response body will be discarded.<br />
<br />
==Content Types==<br />
<br />
The body of a POST or PUT request will be converted to an XQuery item. Conversion can be<br />
controlled by specifying a content type. It can be further influenced<br />
by specifying additional content-type parameters:<br />
<br />
{| class="wikitable"<br />
|- valign="top"<br />
! Content-Type<br />
! Parameters (<code>;name=value</code>)<br />
! Type of resulting XQuery item<br />
|- valign="top"<br />
| {{Code|text/xml}}, {{Code|application/xml}}<br />
|<br />
| {{Code|document-node()}}<br />
|- valign="top"<br />
| {{Code|text/*}}<br />
|<br />
| {{Code|xs:string}}<br />
|- valign="top"<br />
| {{Code|application/json}}<br />
| [[JSON Module#Options|JSON Options]]<br />
| {{Code|document-node()}} or {{Code|map(*)}}<br />
|- valign="top"<br />
| {{Code|text/html}}<br />
| [[HTML Module#Options|HTML Options]]<br />
| {{Code|document-node()}}<br />
|- valign="top"<br />
| {{Code|text/comma-separated-values}}<br />
| [[CSV Module#Options|CSV Options]]<br />
| {{Code|document-node()}} or {{Code|map(*)}}<br />
|- valign="top"<br />
| ''others''<br />
|<br />
| {{Code|xs:base64Binary}}<br />
|- valign="top"<br />
| {{Code|multipart/*}}<br />
|<br />
| sequence (see next paragraph)<br />
|}<br />
<br />
For example, if <code>application/json;lax=yes</code> is specified as content type, the input will be transformed to JSON, and the lax QName conversion rules will be applied, as described in the [[JSON Module]].<br />
<br />
===Input options===<br />
<br />
Conversion options for {{Option|JSON}}, {{Option|CSV}} and {{Option|HTML}} can also be specified via annotations with the <code>input</code> prefix. The following function interprets the input as text with the CP1252 encoding and treats the first line as header:<br />
<br />
<pre lang='xquery'><br />
declare<br />
%rest:path("/store.csv")<br />
%rest:POST("{$csv}")<br />
%input:csv("header=true,encoding=CP1252")<br />
function page:store-csv($csv as document-node()) {<br />
"Number of rows: " || count($csv/csv/record)<br />
};<br />
</pre><br />
<br />
===Multipart Types===<br />
<br />
The single parts of a multipart message are represented as a sequence,<br />
and each part is converted to an XQuery item as described in the last paragraph.<br />
<br />
A function that is capable of handling multipart types is identical to other RESTXQ functions:<br />
<br />
<pre lang='xquery'><br />
declare<br />
%rest:path("/multipart")<br />
%rest:POST("{$data}")<br />
%rest:consumes("multipart/mixed") (: optional :)<br />
function page:multipart($data as item()*) {<br />
"Number of items: " || count($data)<br />
};<br />
</pre><br />
<br />
==Parameters==<br />
<br />
The following annotations can be used to bind request values to function arguments. Values will implicitly be cast to the type of the argument.<br />
<br />
===Query Parameters===<br />
<br />
The value of the ''first parameter'', if found in the [[Request_Module#Conventions|query component]], will be assigned to the variable specified as ''second parameter''. If no value is specified in the HTTP request, all additional parameters will be bound to the variable (if no additional parameter is given, an empty sequence will be bound):<br />
<br />
<pre lang='xquery'><br />
declare<br />
%rest:path("/params")<br />
%rest:query-param("id", "{$id}")<br />
%rest:query-param("add", "{$add}", 42, 43, 44)<br />
function page:params($id as xs:string?, $add as xs:integer+) {<br />
<result id="{ $id }" sum="{ sum($add) }"/><br />
};<br />
</pre><br />
<br />
===HTML Form Fields===<br />
<br />
Form parameters are specified the same way as [[#Query Parameters|query parameters]]:<br />
<br />
<pre lang='xquery'><br />
%rest:form-param("city", "{$city}", "no-city-specified")<br />
</pre><br />
<br />
The values are the result of HTML forms submitted with the (default) content type <code>application/x-www-form-urlencoded</code>:<br />
<br />
<pre lang="xml"><br />
<form action="/process" method="POST" enctype="application/x-www-form-urlencoded"><br />
<input type="text" name="city"/><br />
<input type="submit"/><br />
</form><br />
</pre><br />
<br />
====File Uploads====<br />
<br />
Files can be uploaded to the server by using the content type {{Code|multipart/form-data}} (the HTML5 {{Code|multiple}} attribute enables the upload of multiple files):<br />
<br />
<pre lang="xml"><br />
<form action="/upload" method="POST" enctype="multipart/form-data"><br />
<input type="file" name="files" multiple="multiple"/><br />
<input type="submit"/><br />
</form><br />
</pre><br />
<br />
The file contents are placed in a [[Map Module|map]], with the filename serving as key. The following example shows how uploaded files can be stored in a temporary directory:<br />
<br />
<pre lang='xquery'><br />
declare<br />
%rest:POST<br />
%rest:path("/upload")<br />
%rest:form-param("files", "{$files}")<br />
function page:upload($files) {<br />
for $name in map:keys($files)<br />
let $content := $files($name)<br />
let $path := file:temp-dir() || $name<br />
return (<br />
file:write-binary($path, $content),<br />
<file name="{ $name }" size="{ file:size($path) }"/><br />
)<br />
};<br />
</pre><br />
<br />
===HTTP Headers===<br />
<br />
Header parameters are specified the same way as [[#Query Parameters|query parameters]]:<br />
<br />
<pre lang='xquery'><br />
%rest:header-param("User-Agent", "{$user-agent}")<br />
%rest:header-param("Referer", "{$referer}", "none")<br />
</pre><br />
<br />
===Cookies===<br />
<br />
Cookie parameters are specified the same way as [[#Query Parameters|query parameters]]:<br />
<br />
<pre lang='xquery'><br />
%rest:cookie-param("username", "{$user}")<br />
%rest:cookie-param("authentication", "{$auth}", "no_auth")<br />
</pre><br />
<br />
==Query Execution==<br />
<br />
In many web search scenarios, user input from browser forms is processed and search results are returned. Such operations can be made more interactive by sending a new search request to the server with each key click. However, this may lead to many parallel server-side requests, from which only the result of the last request will be relevant for the client.<br />
<br />
With the <code>%rest:single</code> annotation, it can be enforced that only one instance of a function will run at the same time and for the same client. If the same function will be called for the second time, a currently executed query will be stopped, and the HTTP error code {{Code|460}} will be returned instead:<br />
<br />
<pre lang='xquery'><br />
(: If fast enough, returns the result. Otherwise, if called again, raises 460 :)<br />
declare<br />
%rest:path("/search")<br />
%rest:query-param("term", "{$term}")<br />
%rest:single<br />
function page:search($term as xs:string) {<br />
<ul>{<br />
for $result in db:get('large-db')//*[text() = $term]<br />
return <li>{ $result }</li><br />
}</ul><br />
};<br />
</pre><br />
<br />
By adding a string value to with the annotation, functions can be bundled together, and a running query can be canceled by calling another one that has the same annotation value. This is shown by another example, in which the first function can be interrupted by the second one. If you call both functions in separate browser tabs, you will note that the first tab will return <code>460</code>, and the second one will return <xml>stopped</xml>.<br />
<br />
<pre lang='xquery'><br />
declare <br />
%rest:path("/compute")<br />
%rest:single("EXPENSIVE")<br />
function local:compute() {<br />
(1 to 100000000000000)[. = 0]<br />
};<br />
<br />
declare <br />
%rest:path("/stop")<br />
%rest:single("EXPENSIVE")<br />
function local:stop() {<br />
<xml>stopped</xml><br />
};<br />
</pre><br />
<br />
The following things should be noted:<br />
<br />
* If a query will be canceled, there will be no undesirable side effects. For example, it won’t be possible to abort a query if it is currently updating the database or performing any other I/O operations. As a result, the termination of a running query can take some more time as expected.<br />
* The currently executed function is bound to the current session. This way, a client will not be able to cancel requests from other clients. As a result, functions can only be stopped if there was at least one previous successful response, in which initial session data was returned to the client.<br />
<br />
=Response=<br />
<br />
By default, a successful request is answered with the HTTP status code {{Code|200}} (OK) and is followed by the given content. An erroneous request leads to an error code and an optional error message (e.g. {{Code|404}} for “resource not found”).<br />
<br />
A {{Code|Server-Timing}} HTTP header is attached to each response. It indicates how much time was spent for parsing, compiling, evaluating and serializing the query. The last value will not necessarily reflect the full time for serializing the result, as the header is generated before the result is sent to the client. Server-side serialization can be enforced by annotating a function with the <code>[[#Query Execution|%rest:single]]</code> annotation.<br />
<br />
==Custom Response==<br />
<br />
Custom responses can be generated in XQuery by returning an <code>rest:response</code> element, an <code>http:response</code> child node that matches the syntax of the [http://expath.org/spec/http-client EXPath HTTP Client Module] specification, and optional child nodes that will be serialized as usual. A function that yields a response on an unknown resource may look as follows:<br />
<br />
<pre lang='xquery'><br />
declare %output:method("text") %rest:path("") function page:error404() {<br />
<rest:response><br />
<http:response status="404"><br />
<http:header name="Content-Language" value="en"/><br />
<http:header name="Content-Type" value="text/plain; charset=utf-8"/><br />
</http:response><br />
</rest:response>,<br />
"The requested resource is not available."<br />
};<br />
</pre><br />
<br />
For the time being, it is not possible to create multipart responses.<br />
<br />
==Forwards and Redirects==<br />
<br />
===Redirects===<br />
<br />
The server can invite the client (e.g., the web browser) to make a second request to another URL by sending a 302 response:<br />
<br />
<pre lang="xml"><br />
<rest:response><br />
<http:response status="302"><br />
<http:header name="Location" value="new-location"/><br />
</http:response><br />
</rest:response><br />
</pre><br />
<br />
The convenience function {{Function|Web|web:redirect}} can be called to create such a response.<br />
<br />
In the XQuery context, redirects are particularly helpful if [[XQuery Update|Updates]] are performed. An updating request may send a redirect to a second function that generates a success message, or evaluates an updated database:<br />
<br />
<pre lang='xquery'><br />
declare %updating %rest:path('/app/init') function local:create() {<br />
db:create('app', <root/>, 'root.xml'),<br />
db:output(web:redirect('/app/ok'))<br />
};<br />
<br />
declare %rest:path('/app/ok') function local:ok() {<br />
'Stored documents: ' || count(db:get('app'))<br />
};<br />
</pre><br />
<br />
===Forwards===<br />
<br />
A server-side redirect is called forwarding. It reduces traffic among client and server, and the forwarding will not change the URL seen from the client’s perspective:<br />
<br />
<pre lang="xml"><br />
<rest:forward>new-location</rest:forward><br />
</pre><br />
<br />
The response can also be created with the convenience function {{Function|Web|web:forward}}.<br />
<br />
With {{Announce|Version 11}}, a log entry with the status code {{Code|204}} will be output before the forwarding takes place.<br />
<br />
==Output==<br />
<br />
The content-type of a response can be influenced by the user via [[Serialization|Serialization Parameters]]. The steps are described in the [[REST#Content Type|REST]] chapter. In RESTXQ, serialization parameters can be specified in the query prolog, via annotations, or within the REST response element:<br />
<br />
===Query Prolog===<br />
<br />
In main modules, serialization parameters may be specified in the query prolog. These parameters will then apply to all functions in a module. In the following example, the content type of the response is overwritten with the {{Code|media-type}} parameter:<br />
<br />
<pre lang='xquery'><br />
declare option output:media-type 'text/plain';<br />
<br />
declare %rest:path("version1") function page:version1() {<br />
'Keep it simple, stupid'<br />
};<br />
</pre><br />
<br />
===Annotations===<br />
<br />
Global serialization parameters can be overwritten via <code>%output</code> annotations. The following example serializes XML nodes as JSON, using the [[JSON Module|JsonML]] format:<br />
<br />
<pre lang='xquery'><br />
declare<br />
%rest:path("cities")<br />
%output:method("json")<br />
%output:json("format=jsonml")<br />
function page:cities() {<br />
element cities {<br />
db:get('factbook')//city/name<br />
}<br />
};<br />
</pre><br />
<br />
The next function, when called, generates XHTML headers, and {{Code|text/html}} will be set as content type:<br />
<br />
<pre lang='xquery'><br />
declare<br />
%rest:path("done")<br />
%output:method("xhtml")<br />
%output:omit-xml-declaration("no")<br />
%output:doctype-public("-//W3C//DTD XHTML 1.0 Transitional//EN") <br />
%output:doctype-system("http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd")<br />
function page:html() {<br />
<html xmlns="http://www.w3.org/1999/xhtml"><br />
<body>done</body><br />
</html><br />
};<br />
</pre><br />
<br />
===Response Element===<br />
<br />
Serialization parameters can also be specified in a REST reponse element in a query. Serialization parameters will be overwritten:<br />
<br />
<pre lang='xquery'><br />
declare %rest:path("version3") function page:version3() {<br />
<rest:response><br />
<output:serialization-parameters><br />
<output:media-type value='text/plain'/><br />
</output:serialization-parameters><br />
</rest:response>,<br />
'Not that simple anymore'<br />
};<br />
</pre><br />
<br />
=Error Handling=<br />
<br />
If an error is raised when RESTXQ code is parsed, compiled or evaluated, an HTTP response with the status code 500 is generated.<br />
<br />
By default, all server-side errors will be passed on to the client. This is particularly helpful during the development process. In a productive environment, however, it is advisable not to expose errors to the client. This can be realized via the {{Option|RESTXQERRORS}} option. If disabled,<br />
<br />
* XQuery modules that cannot be parsed will be ignored and<br />
* full error messages and stack traces will be suppressed and not included in the HTTP response.<br />
<br />
The full error information can still be looked up in the database logs.<br />
<br />
==Raise Errors==<br />
<br />
With {{Function|Web|web:error}}, you can abort query evaluation, enforce a premature HTTP response and report errors back to the client:<br />
<br />
<pre lang='xquery'><br />
declare<br />
%rest:path("/teapot")<br />
function page:teapot() {<br />
web:error(418, "I'm a pretty teapot")<br />
};<br />
</pre><br />
<br />
In contrast to the standard <code>fn:error</code> function, a status code can be supplied, and the response body will only contain the specified error message and no stack trace.<br />
<br />
==Catch XQuery Errors==<br />
<br />
XQuery runtime errors can be processed via ''error annotations''. Error annotations have one or more arguments, which represent the error codes to be caught. The codes equal the names of the [[XQuery 3.0#Try.2FCatch|try/catch]] construct:<br />
<br />
{| class="wikitable"<br />
|- valign="top"<br />
! Precedence<br />
! Syntax<br />
! Example<br />
|- valign="top"<br />
| 1<br />
| <code>prefix:name</code><br/><code>Q{uri}name</code><br />
| <code>err:FORG0001</code><br/><code><nowiki>Q{http://www.w3.org/2005/xqt-errors}FORG0001</nowiki></code><br />
|- valign="top"<br />
| 2<br />
| <code>prefix:*</code><br/><code>Q{uri}*</code><br />
| <code>err:*</code><br/><code><nowiki>Q{http://www.w3.org/2005/xqt-errors}*</nowiki></code><br />
|- valign="top"<br />
| 3<br />
| <code>*:name</code><br />
| <code>*:FORG0001</code><br />
|- valign="top"<br />
| 4<br />
| <code>*</code><br />
| <code>*</code><br />
|}<br />
<br />
All error codes that are specified for a function must have the same precedence.<br />
The following rules apply when catching errors:<br />
<br />
* Codes with a higher precedence (smaller number) will be given preference.<br />
* A global RESTXQ error will be raised if two functions with conflicting codes are found.<br />
<br />
Similar to try/catch, the pre-defined variables ({{Code|code}}, {{Code|description}}, {{Code|value}}, {{Code|module}}, {{Code|line-number}}, {{Code|column-number}}, {{Code|additional}}) can be bound to variables via ''error parameter annotations'', which are specified the same way as [[#Query Parameters|query parameters]].<br />
<br />
Errors may occur unexpectedly. However, they can also be triggered by a query, as demonstrated by the following example:<br />
<br />
<pre lang='xquery'><br />
declare<br />
%rest:path("/check/{$user}")<br />
function page:check($user) {<br />
if($user = ('jack', 'lisa'))<br />
then 'User exists'<br />
else fn:error(xs:QName('err:user'), $user)<br />
};<br />
<br />
declare <br />
%rest:error("err:user")<br />
%rest:error-param("description", "{$user}")<br />
function page:user-error($user) {<br />
'User "' || $user || '" is unknown'<br />
};<br />
</pre><br />
<br />
==Catch HTTP Errors==<br />
<br />
Errors that occur outside RESTXQ can be caught by adding {{Code|error-page}} elements with an error code and a target location to the {{Code|web.xml}} configuration file (find more details in the [http://www.eclipse.org/jetty/documentation/current/custom-error-pages.html Jetty Documentation]):<br />
<br />
<pre lang="xml"><br />
<error-page><br />
<error-code>404</error-code><br />
<location>/error404</location><br />
</error-page><br />
</pre><br />
<br />
The target location may be another RESTXQ function. The {{Function|Request|request:attribute}} function can be used to request details on the caught error:<br />
<br />
<pre lang='xquery'><br />
declare %rest:path("/error404") function page:error404() {<br />
"URL: " || request:attribute("javax.servlet.error.request_uri") || ", " || <br />
"Error message: " || request:attribute("javax.servlet.error.message")<br />
};<br />
</pre><br />
<br />
=User Authentication=<br />
<br />
If you want to provide restricted access to parts of a web applications, you will need to check permissions before returning a response to the client. The [[Permissions]] layer is a nice abstraction for defining permission checks.<br />
<br />
=Functions=<br />
<br />
The [[Request Module]] contains functions for accessing data related to the current HTTP request. Two modules exist for setting and retrieving server-side session data of the current user ([[Session Module]]) and all users known to the HTTP server ([[Sessions Module]]). The [[RESTXQ Module]] provides functions for requesting RESTXQ base URIs and generating a [https://www.w3.org/Submission/wadl/ WADL description] of all services. Please note that the namespaces of all of these modules must be explicitly specified via module imports in the query prolog.<br />
<br />
The following example returns the current host name:<br />
<br />
<pre lang='xquery'><br />
import module namespace request = "http://exquery.org/ns/request";<br />
<br />
declare %rest:path("/host-name") function page:host() {<br />
'Remote host name: ' || request:remote-hostname()<br />
};<br />
</pre><br />
<br />
=References=<br />
<br />
Documentation:<br />
<br />
* [http://exquery.org/spec/restxq RESTXQ Specification], First Draft<br />
* [http://www.adamretter.org.uk/papers/restful-xquery_january-2012.pdf RESTful XQuery, Standardised XQuery 3.0 Annotations for REST]. Paper, XMLPrague, 2012<br />
* [http://www.adamretter.org.uk/presentations/restxq_mugl_20120308.pdf RESTXQ]. Slides, MarkLogic User Group London, 2012<br />
* [https://files.basex.org/publications/xmlprague/2013/Develop-RESTXQ-WebApps-with-BaseX.pdf Web Application Development]. Slides from XMLPrague 2013<br />
<br />
Examples:<br />
<br />
* Sample code combining XQuery and JavaScript: [https://www.balisage.net/Proceedings/vol17/author-pkg/Galtman01/BalisageVol17-Galtman01.html Materials] and [https://www.balisage.net/Proceedings/vol17/html/Galtman01/BalisageVol17-Galtman01.html paper] from Amanda Galtman, Balisage 2016.<br />
* [[DBA]]: The Database Administration interface, bundled with the full distributions of BaseX.<br />
<br />
=Changelog=<br />
<br />
;Version 11.0<br />
* Updated: [[#Forwards|Forwards]]: A log entry with the status code {{Code|204}} will be output.<br />
<br />
;Version 9.6<br />
* Updated: [[#Response|Response]]: {{Code|Server-Timing}} HTTP header.<br />
<br />
;Version 9.5<br />
* Updated: [[#Raise Errors|Raise Errors]]: Status code {{Code|400}} changed to {{Code|500}}, omit stack trace.<br />
<br />
;Version 9.3<br />
* Updated: [[#Custom Methods|Custom Methods]]: Better support for the OPTIONS and HEAD methods.<br />
* Updated: [[#Catch XQuery Errors|XQuery Errors]]: Suppress stack trace and error code in the HTTP response.<br />
* Removed: {{Code|rest:redirect}} element ({{Function|Web|web:redirect}} can be used instead)<br />
<br />
;Version 9.2<br />
* Updated: Ignore XQuery modules that cannot be parsed<br />
<br />
;Version 9.0<br />
* Added: Support for server-side quality factors in the [[#Content Negotiation|<code>%rest:produces</code>]] annotation<br />
* Updated: Status code {{Code|410}} was replaced with {{Code|460}}<br />
* Removed: {{Code|restxq}} prefix<br />
<br />
;Version 8.4<br />
* Added: <code>%rest:single</code> annotation<br />
<br />
;Version 8.1<br />
* Added: support for input-specific content-type parameters<br />
* Added: <code>%input</code> annotations<br />
<br />
;Version 8.0<br />
* Added: Support for regular expresssions in the [[#Paths|Path Annotation]]<br />
* Added: Evaluation of quality factors that are supplied in the [[#Content Negotiation|Accept header]]<br />
<br />
;Version 7.9<br />
* Updated: [[#Catch XQuery Errors|XQuery Errors]], extended error annotations<br />
* Added: {{Code|%rest:method}}<br />
<br />
;Version 7.7<br />
* Added: [[#Error Handling|Error Handling]], [[#File Uploads|File Uploads]], [[#Multipart Types|Multipart Types]]<br />
* Updated: RESTXQ function may now also be specified in main modules (suffix: {{Code|*.xq}}).<br />
* Updated: the RESTXQ prefix has been changed from {{Code|restxq}} to {{Code|rest}}.<br />
* Updated: parameters are implicitly cast to the type of the function argument<br />
* Updated: the RESTXQ root url has been changed to {{Code|http://localhost:8080/}}<br />
<br />
;Version 7.5<br />
* Added: new XML elements {{Code|<rest:redirect/>}} and {{Code|<rest:forward/>}}</div>CGhttps://docs.basex.org/index.php?title=Standard_Mode&diff=16890Standard Mode2023-12-01T16:39:13Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div><p>In the standard mode of the [[Clients]], a database command can be<br />
sent to the server using the <code>execute()</code> function<br />
of the <code>Session</code>. This functions returns the whole<br />
result. With the <code>info()</code> function, you can request some<br />
information on your executed process. If an error occurs, an <br />
exception with the error message will be thrown.</p> <br />
<br />
==Usage== <br />
<br />
<p>The standard execution works as follows:</p> <br />
<br />
# Create a new session instance with hostname, port, username and password. <br />
# Call the <code>execute()</code> function of the session with the [[Commands|database commands]] as argument. <br />
# Receive the result of a successfully executed command. If an error occurs, an exception is thrown. <br />
# Optionally, call <code>info()</code> to get some process information<br />
# Continue using the client (back to 2.), or close the session.<br />
<br />
==Example in PHP==<br />
<br />
Taken from our [https://github.com/BaseXdb/basex/blob/master/basex-api/src/main/php/Example.php repository]:<br />
<br />
<pre lang="php"><br />
<?php<br />
/*<br />
* This example shows how database commands can be executed.<br />
* Documentation: http://basex.org/api<br />
*<br />
* (C) BaseX Team 2005-15, BSD License<br />
*/<br />
include("BaseXClient.php");<br />
<br />
try {<br />
// initialize timer<br />
$start = microtime(true);<br />
<br />
// create session<br />
$session = new Session("localhost", 1984, "admin", "...");<br />
<br />
// perform command and print returned string<br />
print $session->execute("xquery 1 to 10");<br />
<br />
// close session<br />
$session->close();<br />
<br />
// print time needed<br />
$time = (microtime(true) - $start) * 1000;<br />
print "\n$time ms\n";<br />
<br />
} catch (Exception $e) {<br />
// print exception<br />
print $e->getMessage();<br />
}<br />
?><br />
</pre></div>CGhttps://docs.basex.org/index.php?title=Commands&diff=16889Commands2023-12-01T16:39:11Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This article is part of the [[Getting Started]] Section.<br />
It lists all database commands supported by BaseX.<br />
<br />
Commands can be executed from the [[Command-Line Options#BaseX_Standalone|Command Line]], as part of [[#Command Scripts|Scripts]], via the [[Clients]], [[REST]], the input field in the [[GUI]], and other ways. If the GUI is used, all commands that are triggered by the GUI itself will show up in the [[GUI#Visualizations|Info View]]. The [[User_Management|Permission]] fields indicate which rights are required by a user to perform a command in the client/server architecture.<br />
<br />
=Basics=<br />
<br />
==Command Scripts==<br />
<br />
On command line, multiple commands can be written down in a single line (separated by semicolons). You can also put them into a command script: Database commands in both string and XML syntax can be placed in a text file and stored as file with the BaseX command script suffix {{Code|.bxs}}. If the path to a script file is passed on to BaseX on command-line, or if it is opened in the GUI editor, it will be recognized and evaluated as such.<br />
<br />
===String Syntax===<br />
<br />
Lines starting with <code>#</code> are interpreted as comments and are skipped. With the following script, a database is created, two documents are added to it, and a query is performed:<br />
<br />
<pre lang="xml"><br />
CREATE DB test<br />
ADD TO embedded.xml <root>embedded</root><br />
# run query<br />
XQUERY <hits>{ count(//text()) }</hits><br />
CLOSE<br />
</pre><br />
<br />
===XML Syntax===<br />
<br />
The string syntax is limited when XML snippets need to be embedded in a command, or when complex queries are to be specified.<br />
<br />
The XML syntax provides more flexibility here. Multiple commands can be enclosed by a {{Code|<commands/>}} root element. Some commands, such as {{Command|ADD}}, allow you to directly embed XML documents. If you want to embed XML in XQuery expressions, entities should be encoded, or the {{Code|CDATA}} syntax should be used:<br />
<br />
<pre lang="xml"><br />
<commands><br />
<create-db name='test'/><br />
<add path='embedded.xml'><root>embedded</root></add><br />
<!-- run query --><br />
<xquery><![CDATA[<br />
<hits>{ count(//text()) }</hits><br />
]]></xquery><br />
<close/><br />
</commands><br />
</pre><br />
<br />
==Glob Syntax==<br />
<br />
Some commands support the glob syntax to address more than one database or user. Question marks and asterisks can be used to match one or more characters, and commas can be used to separate multiple patterns. Some examples:<br />
<br />
* {{Code|AB?}} addresses all names with the characters {{Code|AB}} and one more character.<br />
* {{Code|*AB}} addresses all names ending with the characters {{Code|AB}}.<br />
* {{Code|X*,Y*,Z*}} addresses all names starting with the characters {{Code|X}}, {{Code|Y}}, or {{Code|Z}}.<br />
<br />
==Valid Names==<br />
<br />
Names of databases and users follow the same constraints: Names must at least have one printable character, including letters, numbers, and any of the special characters <code>!#$%&'()+-=@[]^_`{}~</code>. The following characters are disallowed:<br />
<br />
* <code>,?*</code>: [[#Glob Syntax|glob syntax]]<br />
* <code>;</code>: Separator for multiple database commands on the [[Command-Line Options|command line]]<br />
* <code>\/</code>: Directory path separators<br />
* <code>:"<>|</code>: invalid filename characters on Windows<br />
* Names starting or ending with <code>.</code>: hidden folders (e.g. the [[Logging|.logs directory]])<br />
<br />
==Aliases==<br />
<br />
In all commands, the {{Code|DB}} keyword can be replaced by {{Code|DATABASE}}.<br />
<br />
=Database Operations=<br />
<br />
==CREATE DB==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|CREATE DB [name] ([input])}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><create-db name='...'>([input])</create-db></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''CREATE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a new database with the specified {{Code|name}} and, optionally, an initial {{Code|input}}, and opens it. An existing database will be overwritten.<br/>The input can be a file or directory path to XML documents, a remote URL, or a string containing XML:<br />
* {{Code|name}} must be a [[#Valid Names|valid database name]]<br />
* database creation can be controlled by setting [[Options#Create Options|Create Options]]<br />
If you need to add initial resources, it is always faster to supply them at creation time than adding them in a subsequent step via {{Command|ADD}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if a database with the specified name is currently used by another process, if one of the documents to be added is not well-formed or if it cannot be parsed for some other reason.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|CREATE DB input}}<br/>creates an empty database {{Code|input}}.<br />
* {{Code|CREATE DB xmark http://files.basex.org/xml/xmark.xml}}<br/>creates the database {{Code|xmark}}, containing a single initial document called {{Code|xmark.xml}}.<br />
* {{Code|CREATE DATABASE coll /path/to/input}}<br/>creates the database {{Code|coll}} with all documents found in the {{Code|input}} directory.<br />
* {{Code|SET INTPARSE false}} and {{Code|CREATE DB input input.xml}}<br/>creates a database {{Code|input}} with {{Code|input.xml}} as initial document, which will be parsed with Java's [[Parsers#XML Parsers|default XML parser]].<br />
* <code><create-db name='simple'><hello>Universe</hello></create-db></code><br/>creates a database named {{Code|simple}} with an initial document {{Code|<hello>Universe</hello>}}.<br />
|}<br />
<br />
==OPEN==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|OPEN [name]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><open name='...'/></code><br />
|- valign="top"<br />
| '''Permission'''<br />
|''READ''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Opens the database specified by {{Code|name}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the specified database does not exist, is currently being updated by another process, or cannot be opened for some other reason.<br />
|}<br />
<br />
==CHECK==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|CHECK [input]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><check input='...'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''READ/CREATE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|This convenience command combines {{Command|OPEN}} and {{Command|CREATE DB}}: If a database with the name {{Code|input}} exists, and if there is no existing file or directory with the same name that has a newer timestamp, the database is opened. Otherwise, a new database is created; if the specified input points to an existing resource, it is stored as initial content.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the addressed database could neither be opened nor created.<br />
|}<br />
<br />
==CLOSE==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|CLOSE }}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><close/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''READ''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Closes the currently opened database.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the database files could not be closed for some reason.<br />
|}<br />
<br />
==LIST==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|LIST ([name] ([path]))}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><list (name='...' (path='...'))/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''NONE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Lists all available databases. If {{Code|name}} is specified, the resources of a database are listed. The output can be further restricted to the resources matching the specified {{Code|path}}.<br/>If database resources are listed, the size is either the number of nodes (for XML resources) or the number of bytes (for binary resources).<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the optional database cannot be opened, or if the existing databases cannot be listed for some other reason.<br />
|}<br />
<br />
==DIR==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|DIR ([path]))}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><dir (path='...')/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''READ''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Lists directories and resources of the currently opened database and the specified {{Code|path}}.<br/>If database resources are listed, the size is either the number of nodes (for XML resources) or the number of bytes (for binary resources).<br />
|}<br />
<br />
==EXPORT==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|EXPORT [path]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><export path='...'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''CREATE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Exports all documents in the database to the specified file {{Code|path}}, using the serializer options specified by the {{Option|EXPORTER}} option.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if no database is opened, if the target path points to a file or is invalid, if the serialization parameters are invalid, or if the documents cannot be serialized for some other reason.<br />
|}<br />
<br />
==CREATE INDEX==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|CREATE INDEX [TEXT&#x7c;ATTRIBUTE&#x7c;TOKEN&#x7c;FULLTEXT]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code>&lt;create-index type='text&#x7c;attribute&#x7c;token&#x7c;fulltext'/&gt;</code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''WRITE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates the specified [[Indexes#Value Indexes|Value Index]]. The current [[Options#Indexing|Index Options]] will be considered when creating the index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if no database is opened, if the specified index is unknown, or if indexing fails for some other reason.<br />
|}<br />
<br />
==DROP INDEX==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|DROP INDEX [TEXT&#x7c;ATTRIBUTE&#x7c;TOKEN&#x7c;FULLTEXT]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><drop-index type='text&#x7c;attribute&#x7c;token&#x7c;fulltext'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''WRITE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Drops the specified [[Indexes#Value Indexes|Value Index]].<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if no database is opened, if the specified index is unknown, or if it could not be deleted for some other reason.<br />
|}<br />
<br />
==ALTER DB==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|ALTER DB [name] [newname]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><alter-db name='...' newname='...'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''CREATE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Renames the database specified by {{Code|name}} to {{Code|newname}}. {{Code|newname}} must be a [[#Valid Names|valid database name]].<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the source database does not exist or is currently locked, or if it could not be renamed for some other reason.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|ALTER DB db tempdb}}<br/>renames the database {{Code|db}} into {{Code|tempdb}}.<br />
|}<br />
<br />
==DROP DB==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|DROP DB [name]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><drop-db name='...'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''CREATE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Drops the database with the specified {{Code|name}}. The [[#Glob Syntax|Glob Syntax]] can be used to address more than one database.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the specified database does not exist or is currently locked, or if the database could not be deleted for some other reason.<br />
|}<br />
<br />
==COPY==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|COPY [name] [newname]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><copy name='...' newname='...'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''CREATE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a copy of the database specified by {{Code|name}}. {{Code|newname}} must be a [[#Valid Names|valid database name]].<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the source database does not exist.<br />
|}<br />
<br />
==INSPECT==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|INSPECT}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><inspect/></code><br />
|- valign="top"<br />
| '''Permission'''<br />
|''READ''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Performs some integrity checks on the opened database and returns a brief summary.<br />
|}<br />
<br />
=Backups=<br />
<br />
==CREATE BACKUP==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|CREATE BACKUP ([name] ([comment]))}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><create-backup (name='...') (comment='...')/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''CREATE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a backup of the database specified by {{Code|name}}, with an optional {{Code|comment}}. If no name is supplied, general data will be backed up. The backup file will be suffixed with the current timestamp and stored in the database directory. The [[#Glob Syntax|Glob Syntax]] can be used to address more than one database.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the specified database does not exist, or if it could not be zipped for some other reason.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|BACKUP db}}<br/>creates a zip archive of the database {{Code|db}} (e.g. {{Code|db-2014-04-01-12-27-28.zip}}) in the [[Configuration#Database_Directory|database directory]].<br />
|}<br />
<br />
==DROP BACKUP==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|DROP BACKUP ([name])}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><drop-backup (name='...')/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''CREATE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Drops all backups of the database with the specified {{Code|name}}, or a specific backup file if the name ends with its timestamp. If no name is supplied, backups with general data are addressed. The [[#Glob Syntax|Glob Syntax]] can be used to address more than one database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|DROP BACKUP abc*}}<br/>deletes the backups of all databases starting with the characters {{Code|abc}}.<br />
* {{Code|DROP BACKUP factbook-2021-05-16-13-13-10}}<br/>deletes a specific backup file.<br />
|}<br />
<br />
==ALTER BACKUP==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|ALTER BACKUP [name] [newname]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><alter-backup name='...' newname='...'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''CREATE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Renames all backups of the database with the specified {{Code|name}} to {{Code|newname}}. If the name ends with a timestamp, only the specified backup file will be renamed. The [[#Glob Syntax|Glob Syntax]] can be used to address more than one database.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|ALTER BACKUP logs logs-backup}}<br/>renames the backups of the {{Code|logs}} database to {{Code|logs-backup}}.<br />
|}<br />
<br />
==RESTORE==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|RESTORE ([name])}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><restore (name='...')/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''CREATE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Restores a database with the specified {{Code|name}}. The name may include the timestamp of the backup file. If no name is supplied, general data will be restored. If general data is restored, it will only be available after BaseX has been restarted.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the specified backup does not exist, if the database to be restored is currently locked, or if it could not be restored for some other reason.<br />
|}<br />
<br />
==SHOW BACKUPS==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|SHOW BACKUPS}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><show-backups/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''CREATE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Shows all database backups.<br />
|}<br />
<br />
= Querying =<br />
<br />
==XQUERY==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|XQUERY [query]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><xquery>[query]</xquery></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''depends on query''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Runs the specified {{Code|query}} and prints the result.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the specified query is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>XQUERY 1 to 10</code><br/>returns the sequence {{Code|(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)}}.<br />
* <code>SET RUNS 10</code> and <code>XQUERY 1 to 10</code><br/>returns the results after having run the query 10 times.<br />
* <code>SET XMLPLAN true</code> and <code>XQUERY 1 to 10</code><br/>returns the result and prints the query plan as XML.<br />
|}<br />
<br />
==GET==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|GET [path]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><get path='...'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''READ''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Retrieves an XML document from the opened database at the specified {{Code|path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if no database is opened or if the source path is invalid.<br />
|}<br />
<br />
==BINARY GET==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|BINARY GET [path]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><binary-get path='...'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''READ''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Retrieves a [[Binary Data|binary resource]] from the opened database at the specified {{Code|path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if no database is opened or if the source path is invalid.<br />
|}<br />
<br />
==FIND==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|FIND [query]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><find>[query]</find></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''READ''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Builds and runs a query for the specified {{Code|query}} terms. Keywords can be enclosed in quotes to look for phrases. The following modifiers can be used to further limit search:<br />
<code>=</code> looks for exact text nodes<br/><code>~</code> looks for approximate hits<br/><code>@=</code> looks for exact attribute values<br/><code>@</code> looks for attributes<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if no database is opened.<br />
|}<br />
<br />
==TEST==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|TEST [path]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><test path='...'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''ADMIN''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Runs all [[Unit Module|XQUnit tests]] in the specified {{Code|path}}. The path can point to a single file or a directory.<br/>Unit testing can also be triggered via {{Code|-t}} on [[Command-Line Options#BaseX Standalone|command line]].<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if at least one test fails.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|TEST project/tests}}<br/>runs all tests in the directory {{Code|project/tests}}.<br />
|}<br />
<br />
==REPO INSTALL==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|REPO INSTALL [path]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><repo-install path='...'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''CREATE''<br />
|- valign="top"<br />
| '''Summary'''<br />
| Installs the package with path {{Code|path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
| The command fails in the following cases:<br />
* The package to be installed is not a xar file.<br />
* The package to be installed does not exist or is already installed.<br />
* The package descriptor is with invalid syntax.<br />
* The package to be installed depends on a package which is not installed.<br />
* The package is not supported by the current version of BaseX.<br />
* A component of the package is already installed as part of another package.<br />
|}<br />
<br />
==REPO LIST==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|REPO LIST}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><repo-list/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''READ''<br />
|- valign="top"<br />
| '''Summary'''<br />
| Lists all installed packages.<br />
|}<br />
<br />
==REPO DELETE==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|REPO DELETE [name]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><repo-delete name='...'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''CREATE''<br />
|- valign="top"<br />
| '''Summary'''<br />
| Deletes the specified package with the specified {{Code|name}}. What is called "name" can also be the id (which is the name followed by the version) or the directory of the package.<br />
|- valign="top"<br />
| '''Errors'''<br />
| The command fails if the package to be deleted is required by another package.<br />
|}<br />
<br />
=Updates=<br />
<br />
==ADD==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|ADD (TO [path]) [input]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><add (path='...')>[input]</add></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''WRITE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Adds a file, directory or XML string specified by {{Code|input}} to the currently opened database at the specified {{Code|path}}:<br />
* {{Code|input}} may either be a single XML document, a directory, a remote URL or a plain XML string.<br />
* A document with the same path may occur than once in a database. If this is unwanted, the {{Command|PUT}} command can be used.<br />
* If a file is too large to be added in one go, its data structures will be cached to disk first. Caching can be enforced by turning the {{Option|ADDCACHE}} option on.<br />
If files are to be added to an empty database, it is usually faster to use the {{Command|CREATE DB}} command and specify the initial input as argument.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if no database is opened, if one of the documents to be added is not well-formed, or if it could not be parsed for some other reason.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|ADD input.xml}}<br/>adds the file {{Code|input.xml}} to the database.<br />
* {{Code|ADD TO temp/one.xml input.xml}}<br/>adds {{Code|input.xml}} to the database and moves it to {{Code|temp/one.xml}}.<br />
* {{Code|ADD TO target/ xmldir}}<br/>adds all files from the {{Code|xmldir}} directory to the database in the {{Code|target}} path.<br />
|}<br />
<br />
==DELETE==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|DELETE [path]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><delete path='...'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''WRITE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Deletes all documents from the currently opened database that start with the specified {{Code|path}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if no database is opened.<br />
|}<br />
<br />
==RENAME==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|RENAME [path] [newpath]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><rename path='...' newpath='...'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''WRITE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Renames all document paths in the currently opened database that start with the specified {{Code|path}}. The command may be used to either rename single documents or directories.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if no database is opened, or if the target path is empty.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|RENAME one.xml two.xml}}<br/>renames the document {{Code|one.xml}} to {{Code|two.xml}}.<br />
* {{Code|RENAME / TOP}}<br/>moves all documents to a {{Code|TOP}} root directory.<br />
|}<br />
<br />
==PUT==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|PUT [path] [input]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><put path='...'>[input]</put></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''WRITE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Adds or replaces resources in the currently opened database, addressed by {{Code|path}}, with the file, directory or XML string specified by {{Code|input}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if no database is opened or if the specified path is invalid.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|PUT one.xml input.xml}}<br/>replaces {{Code|one.xml}} with the contents of the file {{Code|input.xml}}.<br />
* {{Code|PUT top.xml &lt;xml/&gt;}}<br/>replaces {{Code|top.xml}} with the XML document {{Code|&lt;xml/&gt;}}.<br />
|}<br />
<br />
==BINARY PUT==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|BINARY PUT (TO [path]) [input]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><binary-put (path='...')>[input]</store></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''WRITE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Stores a [[Binary Data|binary resource]] specified via {{Code|input}} in the opened database to the specified {{Code|path}}:<br />
* The input may either be a file reference, a remote URL, or a plain string.<br />
* If the path denotes a directory, it needs to be suffixed with a slash ({{Code|/}}).<br />
* An existing resource will be replaced.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if no database is opened, if the specified resource is not found, if the target path is invalid or if the data cannot not be written for some other reason.<br />
|}<br />
<br />
==OPTIMIZE==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|OPTIMIZE (ALL)}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><optimize/></code><br/><code><optimize-all/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''WRITE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Optimizes the index structures, metadata and statistics of the currently opened database:<br />
* If {{Code|ALL}} is specified, all database structures are completely reconstructed. The database size will be reduced, and all orphaned data will be deleted.<br />
* Without {{Code|ALL}}, only the outdated index structures and database statistics will be updated. If the database is completely up-to-date, nothing will be done.<br />
* Database options will be adopted from the original database. Only {{Option|AUTOOPTIMIZE}} and (if {{Code|ALL}} is specified) {{Option|UPDINDEX}} will be adopted from the current options.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if no database is opened, or if the currently opened database is a main-memory instance.<br />
|}<br />
<br />
==FLUSH==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|FLUSH}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><flush/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''WRITE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Explicitly flushes the buffers of the currently opened database to disk. This command is applied if {{Option|AUTOFLUSH}} has been set to {{Code|false}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if no database is opened.<br />
|}<br />
<br />
=User Management=<br />
<br />
==CREATE USER==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|CREATE USER [name] ([password])}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><create-user name='...'>([password])</create-user></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''ADMIN''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a user with the specified {{Code|name}} and {{Code|password}}. If no password is specified, it is requested via the chosen frontend (GUI or bash).<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the specified user already exists.<br />
|}<br />
<br />
==ALTER USER==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|ALTER USER [name] ([newname])}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><alter-user name='...' newname='...'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''ADMIN''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Renames the user with the specified {{Code|name}} to {{Code|newname}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the specified user does not exist, or if the new user already exists.<br />
|}<br />
<br />
==ALTER PASSWORD==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|ALTER PASSWORD [name] ([password])}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><alter-password name='...'>([password])</alter-password></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''ADMIN''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Alters the {{Code|password}} of the user with the specified {{Code|name}}. If no password is specified, it is requested via the chosen frontend (GUI or bash).<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the specified user does not exist.<br />
|}<br />
<br />
==DROP USER==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|DROP USER [name] (ON [pattern])}}:<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><drop-user name='...' (pattern='...')/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''ADMIN''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Drops the user with the specified {{Code|name}}. The [[#Glob Syntax|Glob Syntax]] can be used to address more than one database or user. If a glob {{Code|pattern}} is specified, only the assigned database pattern will be removed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if {{Code|admin}} is specified as username, or if the specified user does not exist or is currently logged in.<br />
|}<br />
<br />
==GRANT==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|GRANT [NONE&#x7c;READ&#x7c;WRITE&#x7c;CREATE&#x7c;ADMIN] (ON [pattern]) TO [user]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><grant name='...' permission='none&#x7c;read&#x7c;write&#x7c;create&#x7c;admin' (pattern='...')/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''ADMIN''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Grants the specified [[User_Management|permission]] to the specified {{Code|user}}. The [[#Glob Syntax|Glob Syntax]] can be used to address more than one user. If a glob {{Code|pattern}} is specified, the permission will be applied to all databases that match this pattern.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if {{Code|admin}} is specified as username or if the specified user does not exist.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|GRANT READ TO JoeWinson}}<br/>grants {{Code|READ}} permission to the user {{Code|JoeWinson}}.<br />
* {{Code|GRANT WRITE ON Wiki TO editor*}}<br/>grants {{Code|WRITE}} permissions on the {{Code|Wiki}} database to all users starting with the characters {{Code|editor*}}.<br />
|}<br />
<br />
==PASSWORD==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|PASSWORD ([password])}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><password>([password])</password></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''NONE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Changes the {{Code|password}} of the current user. If the command is run on command-line or in the GUI, the password can be omitted and entered interactively.<br />
|}<br />
<br />
=Administration=<br />
<br />
==SHOW OPTIONS==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|SHOW OPTIONS [name]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><show-options (name='...')/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''NONE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the current values of all [[Options]], or a single option with the specified {{Code|name}}. Global options can only be requested by users with ADMIN permissions.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the specified option is unknown.<br />
|}<br />
<br />
==SHOW SESSIONS==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|SHOW SESSIONS}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><show-sessions/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''ADMIN''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Shows all sessions that are connected to the current server instance.<br />
|}<br />
<br />
==SHOW USERS==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|SHOW USERS (ON [database])}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><show-users (database='...')/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''ADMIN''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Shows all users that are visible to the current user. If a {{Code|database}} is specified, only those users will be shown for which a pattern was specified that matches the database name.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the optional database could not be opened.<br />
|}<br />
<br />
==KILL==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|KILL [target]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><kill target='...'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''ADMIN''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Kills sessions of a user or an IP:port combination, specified by {{Code|target}}. The [[#Glob Syntax|Glob Syntax]] can be used to address more than one user.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if a user tried to kill his/her own session.<br />
|}<br />
<br />
==INFO DB==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|INFO DB}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><info-db/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''READ''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Shows general information and metadata on the currently opened database.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if no database is opened.<br />
|}<br />
<br />
==INFO INDEX==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|INFO INDEX ([ELEMNAME&#x7c;ATTRNAME&#x7c;PATH&#x7c;TEXT&#x7c;ATTRIBUTE&#x7c;TOKEN&#x7c;FULLTEXT])}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><info-index type='elemname&#x7c;attrname&#x7c;path&#x7c;text&#x7c;attribute&#x7c;token&#x7c;fulltext'/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''READ''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Shows information on the existing [[Indexes|index]] structures. The output can be optionally limited to the specified index.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if no database is opened, or if the specified index is unknown.<br />
|}<br />
<br />
==INFO STORAGE==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|INFO STORAGE [start end]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><info-storage (start='...') (end='...')/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''READ''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Shows the internal main table of the currently opened database. An integer range may be specified as argument.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if no database is opened, or if one of the specified arguments is invalid.<br />
|}<br />
<br />
=General Commands=<br />
<br />
==RUN==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|RUN [file]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><run file='...'/></code><br />
|- valign="top"<br />
| '''Permission'''<br />
|''depends on input''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Evaluates the contents of {{Code|file}} as XQuery expression. If the file ends with the suffix {{Code|.bxs}}, the file contents will be evaluated as [[#Basics|command script]]. This command can be used to run several commands in a row, with no other transaction intervening the execution.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the specified file does not exist, or if the retrieved input is invalid. It will be canceled as soon as one of the executed commands fails.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>RUN query.xq</code><br/>will evaluate the specified file as XQuery expression<br />
* <code>RUN commands.bxs</code><br/>will evaluate the specified file as command script<br />
|}<br />
<br />
==EXECUTE==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|EXECUTE [input]}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><execute>[input]</execute></code><br />
|- valign="top"<br />
| '''Permission'''<br />
|''depends on input''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Evaluates the specified {{Code|input}} as [[#Basics|command script]]. This command can be used to run several commands in a row, with no other transaction intervening the execution.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the syntax of the specified input is invalid. It will be canceled as soon as one of the executed commands fails.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>EXECUTE "<commands><create-db name='db1'/><create-db name='db2'/></commands>"</code><br/>Two databases will be created in a single transaction.<br />
|}<br />
<br />
==SET==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|SET [option] ([value])}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><set option='...'>([value])</set></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''NONE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Sets the [[Options|Option]] specified by {{Code|option}} to a new {{Code|value}}. Only local options can be modified. If no value is specified, and if the value is boolean, it will be inverted.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the specified option is unknown or if the specified value is invalid.<br />
|}<br />
<br />
==INFO==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|INFO}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><info/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''READ''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Shows global information.<br />
|}<br />
<br />
==HELP==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|HELP ([command])}}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><help>([command])</help></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''NONE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|If {{Code|command}} is specified, information on the specific command is printed; otherwise, all commands are listed.<br />
|- valign="top"<br />
| '''Errors'''<br />
|The command fails if the specified command is unknown.<br />
|}<br />
<br />
==EXIT==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|EXIT }}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><exit/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''NONE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Exits the console mode.<br />
|}<br />
<br />
==QUIT==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
|width='130'|'''Syntax'''<br />
|{{Code|QUIT }}<br />
|- valign="top"<br />
| '''XML&nbsp;Syntax'''&nbsp;&nbsp;&nbsp;<br />
|<code><quit/></code><br/><br />
|- valign="top"<br />
| '''Permission'''<br />
|''NONE''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Exits the console mode (alias of {{Command|EXIT}}).<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 10<br />
* Added: [[#Backups|Backups]]: Support for general data ([[User Management|registered users]], [[Job Module#Services|scheduled services]] and [[Store Module|key-value stores]]).<br />
* Added: {{Command|DIR}}, {{Command|GET}}<br />
* Updated: {{Command|SHOW OPTIONS}}: Renamed (before: {{Code|GET}}).<br />
* Updated: {{Command|BINARY GET}}: Renamed (before: {{Code|RETRIEVE}}).<br />
* Updated: {{Command|BINARY PUT}}: Renamed (before: {{Code|STORE}}).<br />
* Updated: {{Command|PUT}}: Renamed (before: {{Code|REPLACE}}).<br />
* Updated: {{Command|CREATE BACKUP}}: Support for comments.<br />
* Updated: {{Command|OPEN}}: {{Code|path}} argument dropped.<br />
* Removed: {{Code|JOBS LIST}}, {{Code|JOBS RESULT}}, {{Code|JOBS STOP}}<br />
<br />
;Version 9.7<br />
* Updated: {{Command|ALTER DB}}, {{Command|COPY}}: Overwrite existing databases.<br />
<br />
;Version 9.3<br />
* Added: {{Command|ALTER BACKUP}}<br />
<br />
;Version 8.6<br />
* Updated: {{Command|SHOW USERS}}: If called by non-admins, will only return the current user<br />
<br />
;Version 8.5<br />
* Added: {{Command|JOBS LIST}}, {{Command|JOBS RESULT}}, {{Command|JOBS STOP}}<br />
* Updated: [[#Valid Names|Valid Names]]: allow dots (except as first and last character)<br />
<br />
;Version 8.4<br />
* Updated: {{Command|CREATE INDEX}}, {{Command|DROP INDEX}}, {{Command|INFO INDEX}}: token index added<br />
* Updated: {{Command|INFO STORAGE}}: Query argument removed, start/end added to XML syntax.<br />
* Updated: {{Command|INFO INDEX}}: Token index added; index {{Code|TAG}} renamed to {{Code|ELEMNAME}}; index {{Code|ATTNAME}} renamed to {{Code|ATTRNAME}}<br />
* Updated: {{Command|OPTIMIZE}}: adopt original index options<br />
<br />
;Version 8.2<br />
* Removed: {{Code|CREATE EVENT}}, {{Code|DROP EVENT}} and {{Code|SHOW EVENTS}} command<br />
<br />
;Version 8.0<br />
* Updated: commands for [[#User Management|User Management]]<br />
* Updated: {{Command|OPEN}}: path argument added<br />
* Removed: {{Code|CS}} command<br />
* Added: {{Command|QUIT}}<br />
<br />
;Version 7.9<br />
* Added: {{Command|TEST}} runs XQUnit tests.<br />
<br />
;Version 7.7<br />
* Updated: syntax of [[#Valid Names|valid names]].<br />
<br />
;Version 7.5<br />
* Added: {{Command|EXECUTE}} executes a command script.<br />
* Added: {{Command|INSPECT}} performs integrity checks.<br />
* Added: automatic detection of [[#Basics|Command Scripts]].<br />
* Removed: {{Code|SHOW DATABASES}}; information is also returned by {{Command|SHOW SESSIONS}}.<br />
* Removed: {{Command|OPEN}}: path argument.<br />
<br />
;Version 7.3<br />
* Added: [[#XML Syntax|XML Syntax]] added.<br />
* Updated: {{Command|CHECK}} can now be used to create empty databases.<br />
* Updated: Names and paths in {{Command|OPEN}} and {{Command|LIST}} are now specified as separate arguments.<br />
<br />
;Version 7.2.1<br />
* Updated: permissions for {{Code|GET}} and {{Command|SET}} changed from {{Code|READ}} to {{Code|NONE}}.<br />
<br />
;Version 7.2<br />
* Updated: {{Command|CREATE INDEX}}, {{Command|DROP INDEX}} ({{Code|PATH}} argument removed. Path summary is always available now and updated with {{Command|OPTIMIZE}}).<br />
* Updated: permissions for {{Command|REPO DELETE}}, {{Command|REPO INSTALL}} and {{Command|REPO LIST}}.<br />
<br />
;Version 7.1<br />
* Updated: {{Command|KILL}} (killing sessions by specifying IP:port)<br />
<br />
;Version 7.0<br />
* Added: {{Command|FLUSH}}, {{Code|RETRIEVE}}, {{Code|STORE}}.<br />
* Updated: {{Command|ADD}}: simplified arguments.</div>CGhttps://docs.basex.org/index.php?title=Web_Application&diff=16888Web Application2023-12-01T16:39:10Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This page is part of the [[Getting Started]] Section. It describes how BaseX can be used to both provide simple APIs and build complex web applications.<br />
<br />
=Startup=<br />
<br />
* Run one of the {{Code|basexhttp}} or {{Code|basexhttp.bat}} scripts. Call the script with the {{Code|stop}} keyword to gracefully shut down the server.<br />
* If you have installed BaseX on ''Windows'', click on the '''BaseX HTTP Server (Start)''' icon.<br />
<br />
Both an instance of [https://www.eclipse.org/jetty/ Jetty] and BaseX [[Database Server]] will be started. By default, Jetty listens to the port 8080, and the [[Database Server|BaseX Database Server]] is accessible on port 1984. After startup, you can access a plain HTML welcome page via http://localhost:8080.<br />
<br />
Unless you have already chosen an admin password yet (e.g., via the Windows installer or a previous installation), you can do so by invoking the {{Command|PASSWORD}} command on your terminal:<br />
<br />
<pre><br />
basexhttp -c PASSWORD<br />
BaseX [HTTP Server]<br />
...<br />
[main] INFO org.eclipse.jetty.util.log - Logging initialized @239ms to org.eclipse.jetty.util.log.Slf4jLog<br />
...<br />
HTTP STOP Server was started (port: 8081).<br />
HTTP Server was started (port: 8080).<br />
Password: _<br />
</pre><br />
<br />
The Jetty logging level can be adjusted by adding the following properties to the start script:<br />
<br />
<pre lang="xml"><br />
-Dorg.eclipse.jetty.util.log.class=org.eclipse.jetty.util.log.StdErrLog -D{classref}.LEVEL=DEBUG<br />
</pre><br />
<br />
Various [[Command-Line_Options#HTTP Server|command-line options]] are available to simplify batch processing. The [[Start_Scripts|start script]] can be adjusted for individual purposes (e.g. if the default memory limit is too restrictive).<br />
<br />
BaseX can also be deployed as web servlet in a servlet container or with Maven:<br />
<br />
==Servlet Container==<br />
<br />
In order to deploy BaseX HTTP Services in a servlet container, you can download the WAR distribution of BaseX from the [https://basex.org/download download site], or compile it by calling <code>mvn compile war:war</code> in the <code>basex-api</code> directory. The WAR file can then be deployed following the instructions of the corresponding servlet container ([https://www.eclipse.org/jetty/documentation/current/quickstart-deploying-webapps.html Jetty], [https://tomcat.apache.org/tomcat-9.0-doc/deployer-howto.html Tomcat], etc.).<br />
<br />
You can configure the port, context path, etc. by following the instructions of the corresponding servlet container. This is needed if you want to replace the default URL path (e.g. http://localhost:8080/rest) with a custom one (e.g. http://localhost:8080/basex/rest).<br />
<br />
If you use Jetty (which is the default HTTP server of BaseX), the server configuration is defined in the {{Code|jetty.xml}} file, which is stored in the {{Code|WEB-INF}} directory next to the {{Code|web.xml}}. For detailed configuration, refer to the [https://www.eclipse.org/jetty/documentation/current/jetty-xml-config.html Jetty Documentation].<br />
<br />
To run on [https://tomcat.apache.org/ Apache Tomcat], start the Tomcat server and add any <code>*.war</code> distribution to deploy via the Tomcat web interface. By default, the interface is accessible via http://localhost:8080/manager/html/.<br />
<br />
==Maven==<br />
<br />
Check out the BaseX sources via [[Developing with Eclipse|Eclipse]] or [[Git]]. Execute <code>mvn install</code> in the main project directory and then <code>mvn install jetty:run</code> in the <code>basex-api</code> subdirectory. This will start a Jetty instance in which the servlets will be deployed.<br />
<br />
The same options as in the case of deployment apply in a servlet container. In this case, however, there is no WAR archive. Instead, Jetty looks up all files in the directory <code>[https://github.com/BaseXdb/basex/tree/master/basex-api/src/main/webapp basex-api/src/main/webapp]</code>. Jetty and servlet options can be configured in the {{Code|jetty.xml}} and {{Code|web.xml}} files as described above in the [[#Configuration|Servlet Container Configuration]]. The Jetty stop port can be changed in the [https://www.eclipse.org/jetty/documentation/current/jetty-maven-plugin.html Maven Jetty Plugin] sesion in the {{Code|pom.xml}} file.<br />
<br />
=Services=<br />
<br />
The following services are available and enabled by default:<br />
<br />
{| class="wikitable"<br />
|- valign="top"<br />
! Name<br />
! Standard Path<br />
! Description<br />
|- valign="top"<br />
| [[RESTXQ]]<br />
| <code>/</code><br />
| Write enriched APIs and full web applications with XQuery.<br />
|- valign="top"<br />
| [[WebSockets]]<br />
| <code>ws/</code><br />
| Bidirectional client/server communication.<br />
|- valign="top"<br />
| [[REST]]<br />
| <code>rest/</code><br />
| Straightforward access to XML databases and its resources.<br />
|- valign="top"<br />
| [[WebDAV]]<br />
| <code>webdav/</code><br />
| Database access via the file system.<br />
|- valign="top"<br />
| Default<br />
| <code>static/</code><br />
| Access to static server resources (HTML, JavaScript, CSS, images, …).<br />
|}<br />
<br />
The [[DBA]] is a web-based database administration interface written in RESTXQ. It allows you to create and administrate databases, evaluate queries in realtime, view log files, manage users, etc. It is embedded in the full distributions of BaseX, and it can be accessed after startup via http://localhost:8080/dba/.<br />
<br />
=Configuration=<br />
<br />
Unless BaseX is deployed as servlet, the location of the web application directory can be adjusted via the {{Option|WEBPATH}} option, and compression of HTTP responses can be enabled via the {{Option|GZIP}} option.<br />
<br />
Further database options can be defined as context parameters in the {{Code|web.xml}} file. The most important options for the web application context are:<br />
<br />
{| class="wikitable" <br />
|- valign="top"<br />
! width="160" | Option<br />
! width="90" |Default<br />
! Description<br />
|- valign="top"<br />
|{{Option|USER}}<br />
|{{Code|admin}}<br />
| If a user is specified, no credentials must be passed on by the client.<br />
|- valign="top"<br />
|{{Option|HTTPLOCAL}}<br />
|{{Code|false}}<br />
|Operation mode. By default, a database server instance will be started, as soon as the first HTTP service is called. The database server can be disabled by setting this flag to {{Code|true}}.<br />
|- valign="top"<br />
|{{Option|RESTXQPATH}}<br />
|{{Code|.}}<br />
|Relative or absolute directory referencing the [[RESTXQ]] modules. By default, the option points to the standard web application directory.<br />
|- valign="top"<br />
|{{Option|RESTPATH}}<br />
|{{Code|.}}<br />
|Relative or absolute directory referencing queries and command-scripts that can be invoked via the [[REST#GET_Requests|run operation]] of REST. By default, the option points to the standard web application directory.<br />
|- valign="top"<br />
|{{Option|AUTHMETHOD}}<br />
|{{Code|Basic}}<br />
|The default authentication method proposed by the server. The available methods are {{Code|Basic}} and {{Code|Digest}}.<br />
|}<br />
<br />
All options are prefixed with {{Code|org.basex.}}. Local file paths in options may be absolute or relative. If a relative path is specified, its root will be the servlet’s ({{Code|webapp}}) path:<br />
<br />
<pre lang="xml"><br />
<context-param><br />
<param-name>org.basex.dbpath</param-name><br />
<!-- will be rewritten to ..../webapp/WEB-INF/data --><br />
<param-value>WEB-INF/data</param-value><br />
</context-param><br />
<context-param><br />
<param-name>org.basex.repopath</param-name><br />
<!-- will be kept as is --><br />
<param-value>f:/basex/repository</param-value><br />
</context-param><br />
</pre> <br />
<br />
Context parameters can be requested from XQuery via [[Process_Module#proc:property-names|proc:property-names]] and [[Process_Module#proc:property|proc:property]]. How to set these options is specific to the servlet container. For example, in Jetty it can be done by [http://www.eclipse.org/jetty/documentation/current/override-web-xml.html overriding the web.xml] file. Another option is to directly edit the {{Code|WEB-INF/web.xml}} file in the WAR archive (WAR files are simple ZIP files). Refer to the sample [https://github.com/BaseXdb/basex/blob/master/basex-api/src/main/webapp/WEB-INF/web.xml web.xml] of the {{Code|basex-api}} package.<br />
<br />
To enable or disable a specific service, the corresponding servlet entry in the {{Code|web.xml}} file needs to be removed/commented.<br />
<br />
==Authentication==<br />
<br />
No credentials need to be supplied if a default user is assigned to a service in the {{Code|web.xml}} file. In the following example, the user {{Code|rest-user}} is specified for the REST service:<br />
<br />
<pre lang="xml"><br />
<servlet><br />
<servlet-name>REST</servlet-name><br />
<servlet-class>org.basex.http.rest.RESTServlet</servlet-class><br />
<init-param><br />
<param-name>org.basex.user</param-name><br />
<param-value>rest-user</param-value><br />
</init-param><br />
</servlet><br />
</pre><br />
<br />
If the HTTP server is started with no pre-defined user, the credentials must be passed on by the client via [https://en.wikipedia.org/wiki/Basic_access_authentication Basic Authentication] or [https://en.wikipedia.org/wiki/Digest_authentication Digest Authentication], depending on the chosen authentication method in the configuration.<br />
<br />
With cURL, internet browsers, and other tools, you can specify basic authentication credentials within the request string as plain text, using the format <code>USER:PASSWORD@URL</code>:<br />
<br />
: <code>http://admin:...@localhost:8080/</code><br />
<br />
Users are specified in a {{Code|users.xml}} file, which is stored in the database directory (see [[User Management]] for more information).<br />
<br />
=Changelog=<br />
<br />
;Version 10.0<br />
* Updated: The default admin password has been removed, and the default ports have been changed from 8984/8985 to 8080/8081.<br />
<br />
;Version 9.0<br />
* Updated: <code>jetty.xml</code> configuration file (required for Jetty 9).<br />
<br />
;Version 8.6<br />
* Updated: Authentication re-added to RESTXQ.<br />
* Updated: No password must be specified in the <code>web.xml</code> file anymore.<br />
* Updated: Server-side user and authentication method is now enforced (cannot be overwritten by client).<br />
<br />
;Version 8.0<br />
* Added: digest authentication<br />
* Updated: user management<br />
* Updated: default user/password disabled in web.xml<br />
<br />
;Version 7.7<br />
* Added: service-specific permissions<br />
<br />
;Version 7.5<br />
* Added: {{Code|jetty.xml}}: configuration for Jetty Server<br />
* Updated: {{Code|server}} replaced with {{Code|httplocal}} mode<br />
<br />
;Version 7.3<br />
* Updated: {{Code|client}} mode replaced with {{Code|server}} mode<br />
<br />
;Version 7.2<br />
* Web Application concept revised</div>CGhttps://docs.basex.org/index.php?title=Web_Module&diff=16887Web Module2023-12-01T16:39:07Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This [[Module Library|XQuery Module]] provides convenience functions for building web applications with [[RESTXQ]].<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/web</nowiki></code> namespace, which is statically bound to the {{Code|web}} prefix.<br/><br />
<br />
=Functions=<br />
<br />
==web:content-type==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>web:content-type(<br />
$path as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the content type of a path by analyzing its file suffix. <code>application/octet-stream</code> is returned if the file suffix is unknown.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code><nowiki>web:content-type("sample.mp3")</nowiki></code> returns <code>audio/mpeg</code><br />
|}<br />
<br />
==web:create-url==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>web:create-url(<br />
$href as xs:string,<br />
$parameters as map(*),<br />
$anchor as xs:string := ()<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a new URL from the specified {{Code|$href}} string, query string <code>$parameters</code> and an optional {{Code|$anchor}} reference. The keys and values of the map entries will be converted to strings, URL-encoded (see {{Function||web:encode-url}}), and appended to the URL as query parameters. If a map entry has more than a single item, all of them will be appended as single parameters.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code><nowiki>web:create-url('http://find.me', map { 'q': 'dog' })</nowiki></code> returns <code><nowiki>http://find.me?q=dog</nowiki></code><br />
* <code><nowiki>web:create-url('search', map { 'year': (2000,2001), 'title':() })</nowiki></code> returns <code>search?year=2000&year=2001</code><br />
|}<br />
<br />
==web:encode-url==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>web:encode-url(<br />
$string as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Encodes a string to a URL. Spaces are rewritten to {{Code|+}}; {{Code|*}}, {{Code|-}}, {{Code|.}} and {{Code|_}} are adopted; and all other non-ASCII characters and special characters are percent-encoded.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code><nowiki>web:encode-url("this is a test!.html")</nowiki></code> returns <code>this+is+a+test%21.html</code>.<br />
|}<br />
<br />
==web:decode-url==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>web:decode-url(<br />
$string as xs:string<br />
) as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Decodes a URL to the original string. Percent-encoded characters are decoded to their UTF8 codepoints, and {{Code|+}} characters are rewritten to spaces.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code><nowiki>web:decode-url("%E6%97%A5%E6%9C%AC%E8%AA%9E")</nowiki></code> returns <code>日本語</code>.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|invalid|#Errors}} the string contains invalid XML characters.<br />
|}<br />
<br />
==web:forward==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>web:forward(<br />
$path as xs:string,<br />
$parameters as map(*) := ()<br />
) as element(rest:forward)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a server-side [[RESTXQ#Forwards and Redirects|RESTXQ forward request]] to the specified {{Code|$path}}:<br />
* The client will not get notified of this forwarding.<br />
* Supplied query parameters will be attached to parameters of the current request.<br />
* The {{Code|$parameter}} argument is processed as described in {{Function||web:create-url}}.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
The function call <code><nowiki>web:forward('/a/b')</nowiki></code> creates the following result (which will be interpreted as forwarding if RESTXQ is used):<br />
<pre lang="xml"><br />
<rest:forward>/a/b</rest:forward><br />
</pre><br />
|}<br />
<br />
==web:redirect==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>web:redirect(<br />
$url as xs:string,<br />
$parameters as map(*) := (),<br />
$anchor as xs:string := ()<br />
) as element(rest:response)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a [[RESTXQ#Forwards and Redirects|RESTXQ redirection]] to the specified {{Code|$url}}. The returned response will only work if no other items are returned by the RESTXQ function.<br/>The {{Code|$parameters}} and {{Code|$anchor}} arguments are processed as described in (see {{Function||web:create-url}}).<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
The query <code><nowiki>web:redirect('/a/b')</nowiki></code> returns the following result (which will be interpreted as redirection if RESTXQ is used):<br />
<pre lang="xml"><br />
<rest:response xmlns:rest="http://exquery.org/ns/restxq"><br />
<http:response xmlns:http="http://expath.org/ns/http-client" status="302"><br />
<http:header name="location" value="/a/b"/><br />
</http:response><br />
</rest:response><br />
</pre><br />
|}<br />
<br />
==web:response-header==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>web:response-header(<br />
$output as map(*)? := (),<br />
$headers as map(*)? := (),<br />
$atts as map(*)? := ()<br />
) as element(rest:response)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a [[RESTXQ#Response|RESTXQ response header]].<br/><br />
Serialization parameters and header values can be supplied via the <code>$output</code> and <code>$headers</code> arguments, and status and message attributes can be attached to the HTTP response element with the <code>$atts</code> argument.<br />
* <code>media-type</code>: <code>application/octet-stream</code><br />
Header options can be supplied via the <code>$headers</code> argument. Empty string values can be specified to invalidate default values. By default, the following header options will be returned:<br />
* <code>Cache-Control</code>: <code>max-age=3600,public</code><br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* The function call <code>web:response-header()</code> returns:<br />
<pre lang="xml"><br />
<rest:response xmlns:rest="http://exquery.org/ns/restxq"><br />
<http:response xmlns:http="http://expath.org/ns/http-client"/><br />
<output:serialization-parameters xmlns:output="http://www.w3.org/2010/xslt-xquery-serialization"/><br />
</rest:response><br />
</pre><br />
* The following expression returns a media-type for binary data, a caching directive, and the OK status:<br/><br />
<pre lang='xquery'><br />
web:response-header(<br />
map { 'media-type': 'application/octet-stream' },<br />
map { 'Cache-Control': 'max-age=3600,public' },<br />
map { 'status': 200, 'message': 'OK' }<br />
)<br />
</pre><br />
* The following RESTXQ function returns the contents of a file to the client with correct media type:<br/><br />
<pre lang='xquery'><br />
declare %rest:path('media/{$file}') function local:get($file) {<br />
let $path := 'path/to/' || $file<br />
return (<br />
web:response-header(map { 'media-type': web:content-type($path) }),<br />
file:read-binary($path)<br />
)<br />
};<br />
</pre><br />
|}<br />
<br />
==web:error==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>web:error(<br />
$status as xs:integer,<br />
$message as xs:string<br />
) as none</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Raises an error with the QName {{Code|rest:error}}, the specified {{Code|$message}} and the specified {{Code|$status}} as error value.<br/>Calls to this function are equivalent to <code>fn:error(xs:QName('rest:error'), $message, $status)</code>.<br />
<br />
See [[RESTXQ#Raise Errors|RESTXQ: Raise Errors]] to learn how the function is helpful in web applications.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code><nowiki>web:error(404, "The requested resource cannot be found.")</nowiki></code><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|status|#Errors}} The supplied status code is invalid.<br />
|}<br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
|Description<br />
|- valign="top"<br />
|{{Code|invalid}}<br />
|A string contains invalid XML characters.<br />
|- valign="top"<br />
|{{Code|status}}<br />
|The supplied status code is invalid.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 9.3<br />
* Added: {{Function||web:error}}, {{Function||web:forward}}<br />
<br />
;Version 9.2<br />
* Updated: {{Function||web:create-url}}, {{Function||web:redirect}}: third argument added.<br />
<br />
;Version 9.0<br />
* Updated: {{Function||web:response-header}}: third argument added; default parameters removed.<br />
* Updated: error codes updated; errors now use the module namespace<br />
<br />
;Version 8.4<br />
* Updated: {{Function||web:response-header}}: serialization method <code>raw</code> was removed (now obsolete).<br />
<br />
;Version 8.2<br />
* Added: {{Function||web:encode-url}}, {{Function||web:decode-url}}.<br />
<br />
The module was introduced with Version 8.1.</div>CGhttps://docs.basex.org/index.php?title=XQuery_Extensions&diff=16885XQuery Extensions2023-12-01T16:39:03Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This article is part of the [[XQuery|XQuery Portal]]. It lists extensions and optimizations that are specific to the BaseX XQuery processor.<br />
<br />
=Expressions=<br />
<br />
{{Announce|Removed with Version 11}}: Elvis operator <code>?:</code>, in favor of the new <code>[https://qt4cg.org/specifications/xquery-40/xquery-40.html#id-otherwise otherwise]</code> expression.<br />
<br />
==Ternary If==<br />
<br />
The [https://en.wikipedia.org/wiki/%3F: ternary if] operator provides a short syntax for conditions. It is also called '''conditional operator''' or '''ternary operator'''. In most languages, the syntax is <code>a ? b : c</code>. As <code>?</code> and <code>:</code> have already been taken in XQuery, the syntax of Perl 6 is used:<br />
<br />
<pre lang='xquery'><br />
(: if/then/else :)<br />
if ($ok) then 1 else 0,<br />
(: ternary if :)<br />
$ok ?? 1 !! 0<br />
</pre><br />
<br />
The expression returns <code>ok</code> if the effective boolean value of <code>$test</code> is true, and it returns <code>fails</code> otherwise.<br />
<br />
==If Without Else==<br />
<br />
In XQuery 3.1, both branches of the <code>if</code> expression need to be specified. In many cases, only one branch is required, so the <code>else</code> branch was made optional in BaseX. If the second branch is omitted, an empty sequence will be returned if the effective boolean value of the test expression is false. Some examples:<br />
<br />
<pre lang='xquery'><br />
if (doc-available($doc)) then doc($doc),<br />
if (file:exists($file)) then file:delete($file),<br />
if (permissions:valid($user)) then <html>Welcome!</html><br />
</pre><br />
<br />
If conditions are nested, a trailing else branch will be associated with the innermost <code>if</code>:<br />
<br />
<pre lang='xquery'><br />
if ($a) then if($b) then '$a and $b is true' else 'only $a is true'<br />
</pre><br />
<br />
In general, if you have multiple or nested if expressions, additional parentheses can improve the readibility of your code:<br />
<br />
<pre lang='xquery'><br />
if ($a) then (<br />
if($b) then '$a and $b is true' else 'only $a is true'<br />
)<br />
</pre><br />
<br />
=Functions=<br />
<br />
==Regular Expressions==<br />
<br />
In analogy with Saxon, you can specify the flag {{Code|j}} to revert to Java’s default regex parser. For example, this allows you to use the word boundary option {{Code|\b}}, which has not been included in the XQuery grammar for regular expressions:<br />
<br />
'''Example:''' <br />
<pre lang='xquery'><br />
(: yields "!Hi! !there!" :)<br />
replace('Hi there', '\b', '!', 'j')<br />
</pre><br />
<br />
=Serialization=<br />
<br />
* <code>basex</code>is used as the default serialization method: nodes are serialized as XML, atomic values are serialized as string, and items of binary type are output in their native byte representation. Function items (including maps and arrays) are output just like with the [[XQuery 3.1#Adaptive Serialization|adaptive]] method.<br />
* With {{Code|csv}}, you can output XML nodes as CSV data (see the [[CSV Module]] for more details).<br />
* With {{Code|json}}, items are output as JSON as described in the [https://www.w3.org/TR/xslt-xquery-serialization-31/#json-output official specification]. If the root node is of type {{Code|element(json)}}, items are serialized as described for the {{Code|direct}} format in the [[JSON Module]].<br />
<br />
For more information and some additional BaseX-specific parameters, see the article on [[Serialization]].<br />
<br />
=Option Declarations=<br />
<br />
==Database Options==<br />
<br />
[[Options|Local database options]] can be set in the prolog of an XQuery main module. In the option declaration, options need to be bound to the [[Database Module]] namespace. All values will be reset after the evaluation of a query:<br />
<br />
<pre lang='xquery'><br />
declare option db:catalog 'etc/w3-catalog.xml';<br />
doc('doc.xml')<br />
</pre><br />
<br />
==XQuery Locks==<br />
<br />
If locks are declared in the query prolog of a module via the {{Code|basex:lock}} option, access to functions of this module locks will be controlled by the central transaction management. See [[Transaction Management#Options|Transaction Management]] for further details.<br />
<br />
=Pragmas=<br />
<br />
==BaseX Pragmas==<br />
<br />
{{Announce|Updated with Version 11}}: Renamed from {{Code|non-deterministic}} to {{Code|nondeterministic}}.<br />
<br />
Many optimizations in BaseX will only be performed if an expression is ''deterministic'' (i. e., if it always yields the same output and does not have side effects). By flagging an expression as nondeterministic, optimizations and query rewritings can be suppressed:<br />
<br />
<pre lang='xquery'><br />
sum( (# basex:nondeterministic #) {<br />
1 to 100000000<br />
})<br />
</pre><br />
<br />
This pragma can be helpful when debugging your code.<br />
<br />
In analogy with option declarations and function annotations, XQuery locks can also set via pragmas. See [[Transaction Management#Options|Transaction Management]] for details and examples.<br />
<br />
<pre lang='xquery'><br />
(# basex:write-lock CONFIGLOCK #) {<br />
file:write('config.xml', <config/>)<br />
}<br />
</pre><br />
<br />
==Database Pragmas==<br />
<br />
[[Options|Local database options]] can also be assigned via pragmas:<br />
<br />
* [[Indexes|Index access rewritings]] can be enforced. This is helpful if the name of a database is not static (see [[Indexes#Enforce Rewritings|Enforce Rewritings]] for more details):<br />
<br />
<pre lang='xquery'><br />
(# db:enforceindex #) {<br />
for $db in ('persons1', 'persons2', 'persons3')<br />
return db:get($db)//name[text() = 'John']<br />
}<br />
</pre><br />
<br />
* Node copying in node constructors can be disabled (see {{Option|COPYNODE}} for more details). The following query will consume much less memory than without pragma as the database nodes will not be fully duplicated, but only attached to the {{Code|xml}} parent element:<br />
<br />
<pre lang='xquery'><br />
file:write(<br />
'wrapped-db-nodes.xml',<br />
(# db:copynode false #) {<br />
<xml>{ db:get('huge') }</xml><br />
}<br />
)<br />
</pre><br />
<br />
* An XML catalog can be specified for URI rewritings. See the [[Catalog Resolver]] section for an example.<br />
<br />
=Annotations=<br />
<br />
==Function Inlining==<br />
<br />
{{Code|%basex:inline([limit])}} controls if functions will be inlined.<br />
<br />
If XQuery functions are ''inlined'', the function call will be replaced by a FLWOR expression, in which the function variables are bound to let clauses, and in which the function body is returned. This optimization triggers further query rewritings that will speed up your query. An example:<br />
<br />
'''Query:'''<br />
<br />
<pre lang='xquery'><br />
declare function local:square($a) { $a * $a };<br />
for $i in 1 to 3<br />
return local:square($i)<br />
</pre><br />
<br />
'''Query after function inlining:'''<br />
<br />
<pre lang='xquery'><br />
for $i in 1 to 3<br />
return<br />
let $a := $i<br />
return $a * $a<br />
</pre><br />
<br />
'''Query after further optimizations:'''<br />
<br />
<pre lang='xquery'><br />
for $i in 1 to 3<br />
return $i * $i<br />
</pre><br />
<br />
By default, XQuery functions will be ''inlined'' if the query body is not too large and does not exceed a fixed number of expressions, which can be adjusted via the {{Option|INLINELIMIT}} option.<br />
<br />
The annotation can be used to overwrite this global limit: Function inlining can be enforced if no argument is specified. Inlining will be disabled if {{Code|0}} is specified.<br />
<br />
'''Example:'''<br />
<br />
<pre lang='xquery'><br />
(: disable function inlining; the full stack trace will be shown... :)<br />
declare %basex:inline(0) function local:e() { error() };<br />
local:e()<br />
</pre><br />
<br />
'''Result:'''<br />
<br />
<pre lang="xml"><br />
Stopped at query.xq, 1/53:<br />
[FOER0000] Halted on error().<br />
<br />
Stack Trace:<br />
- query.xq, 2/9<br />
</pre><br />
<br />
==Lazy Evaluation==<br />
<br />
{{Code|%basex:lazy}} enforces lazy evaluation of a global variable. An example:<br />
<br />
'''Example:''' <br />
<pre lang='xquery'><br />
declare %basex:lazy variable $january := doc('does-not-exist.xml');<br />
if(month-from-date(current-date()) = 1) then $january else ()<br />
</pre><br />
<br />
The annotation ensures that an error is only raised if the condition yields true. Without the annotation, the error is always raised if the referenced document is not found.<br />
<br />
==XQuery Locks==<br />
<br />
In analogy with option declarations and pragmas, locks can also set via annotations. See [[Transaction Management#Annotations|Transaction Management]] for details and examples.<br />
<br />
=Namespaces=<br />
<br />
In XQuery, some namespaces are statically bound to prefixes. The following query requires no additional namespaces declarations in the query prolog:<br />
<br />
<pre lang='xquery'><br />
<xml:abc xmlns:prefix='uri' local:fn='x'/>,<br />
fn:exists(1)<br />
</pre><br />
<br />
In BaseX, various other namespaces are predefined. Apart from the namespaces that are listed on the [[Module Library]] page, the following namespaces are statically bound:<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Description<br />
! Prefix<br />
! Namespace URI<br />
|-<br />
| [[#Annotations|BaseX Annotations]], [[#Pragmas|Pragmas]], …<br />
| <code>basex</code><br />
| <code><nowiki>http://basex.org</nowiki></code><br />
|-<br />
| [[RESTXQ#Input Options|RESTXQ: Input Options]]<br />
| <code>input</code><br />
| <code><nowiki>http://basex.org/modules/input</nowiki></code><br />
|-<br />
| [[Repository#EXPath_Packaging|EXPath Packages]]<br />
| <code>pkg</code><br />
| <code><nowiki>http://expath.org/ns/pkg</nowiki></code><br />
|-<br />
| [[XQuery Errors]]<br />
| <code>err</code><br />
| <code><nowiki>http://www.w3.org/2005/xqt-errors</nowiki></code><br />
|-<br />
| [[Serialization]]<br />
| <code>output</code><br />
| <code><nowiki>http://www.w3.org/2010/xslt-xquery-serialization</nowiki></code><br />
|}<br />
<br />
=Suffixes=<br />
<br />
In BaseX, files with the suffixes {{Code|.xq}}, {{Code|.xqm}}, {{Code|.xqy}}, {{Code|.xql}}, {{Code|.xqu}} and {{Code|.xquery}} are treated as XQuery files. In XQuery, there are main and library modules:<br />
<br />
* Main modules have an expression as query body. Here is a minimum example:<br />
<br />
<pre lang='xquery'><br />
'Hello World!'<br />
</pre><br />
<br />
* Library modules start with a module namespace declaration and have no query body:<br />
<br />
<pre lang='xquery'><br />
module namespace hello = 'http://basex.org/examples/hello';<br />
<br />
declare function hello:world() {<br />
'Hello World!'<br />
};<br />
</pre><br />
<br />
We recommend {{Code|.xq}} as suffix for for main modules, and {{Code|.xqm}} for library modules. However, the actual module type will dynamically be detected when a file is opened and parsed.<br />
<br />
=Miscellaneous=<br />
<br />
Various other extensions are described in the articles on [[Full-Text#BaseX Features|XQuery Full Text]] and [[Updates|XQuery Update]].<br />
<br />
=Changelog=<br />
<br />
;Version 11:<br />
<br />
* Removed: Elvis operator <code>?:</code>, in favor of the new <code>[https://qt4cg.org/specifications/xquery-40/xquery-40.html#id-otherwise otherwise]</code> expression.<br />
* Updated: Renamed from {{Code|non-deterministic}} to {{Code|nondeterministic}}.<br />
<br />
;Version 9.1:<br />
<br />
* Added: New [[#Expressions|Expressions]]: Ternary if, elvis Operator, if without else<br />
* Added: XQuery Locks via pragmas and function annotations.<br />
* Added: [[#Regular expressions|Regular Expressions]], {{Code|j}} flag for using Java’s default regex parser.</div>CGhttps://docs.basex.org/index.php?title=Options&diff=16886Options2023-12-01T16:39:03Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This page is linked from the [[Getting Started]] Section.<br />
<br />
The options listed on this page influence the way how database [[Commands|commands]] are executed and XQuery expressions are evaluated. Two kinds of options exist:<br />
<br />
* '''[[#Global Options|Global Options]]''' are valid for all BaseX instances in the same JVM. This is particularly relevant if you are working with the client/server architecture.<br />
* '''Local options''' (all remaining ones) are specific to a client or session.<br />
<br />
Values of options are either ''strings'', ''numbers'' or ''booleans''. Options are ''static'' and not bound to a single operation (for example, the next command). Various ways exist to access and change options:<br />
<br />
* The current value of an option can be requested with the {{Command|SHOW OPTIONS}} command. Local options can be changed via {{Command|SET}} (all global options, except for {{Option|DEBUG}}, can only be changed at startup time). If an option is of type ''boolean'', and if no value is specified, its current value is inverted.<br />
<br />
* The {{Code|.basex}} [[Configuration#Configuration Files|configuration file]] is parsed by every new local BaseX instance. It contains all global options. Local options can be specified at the end of the file after the {{Code|Local Options}} comment:<br />
<br />
<pre lang="perl"><br />
# General Options<br />
DEBUG = false<br />
...<br />
<br />
# Local Options<br />
CATALOG = etc/w3-catalog.xml<br />
</pre><br />
<br />
* Initial values for global options can also be specified via system properties, which can be passed on with the [https://docs.oracle.com/en/java/javase/11/tools/java.html -D flag] on the command line, or with [https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/System.html#setProperty(java.lang.String,java.lang.String) System.setProperty()] before creating a BaseX instance. The specified keys need to be prefixed with {{Code|org.basex.}}. An example:<br />
<br />
<pre lang="perl"><br />
java -Dorg.basex.CATALOG=etc/w3-catalog.xml -cp basex.jar org.basex.BaseX -c"SHOW OPTIONS catalog"<br />
CATALOG: etc/w3-catalog.xml<br />
</pre><br />
<br />
* If the Mac OS X packaged application is used, global options can be set within the Info.plist file within the Contents folder of the application package. For example:<br />
<br />
<pre lang="xml"><br />
<key>JVMOptions</key><br />
<array><br />
<string>-Dorg.basex.CATALOG=etc/w3-catalog.xml</string><br />
</array><br />
</pre><br />
<br />
* In a [[Web Application]], the default can be adjusted in the {{Code|web.xml}} file as follows:<br />
<br />
<pre lang="xml"><br />
<context-param><br />
<param-name>org.basex.catalog</param-name><br />
<param-value>etc/w3-catalog.xml</param-value><br />
</context-param><br />
</pre><br />
<br />
* In XQuery, local options can be set via option declarations and [[XQuery Extensions#Pragmas|pragmas]].<br />
<br />
If options are changed by operations in the [[GUI]], the underlying commands are output in the [[GUI#Visualizations|Info View]].<br/><br />
<br />
=Global Options=<br />
<br />
Global options are constants. They can only be set in the configuration file or via system properties (see above). One exception is the [[#debug|DEBUG]] option, which can also be changed at runtime by users with [[User Management|admin permissions]].<br />
<br />
==General Options==<br />
<br />
===DEBUG===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|DEBUG [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Sends debug information to STDERR. This option can be enabled to get additional information for development and debugging purposes. It can also be triggered on the [[Command-Line Options#BaseX Standalone|command line]] via <code>-d</code>. Note that the amount of output can be overwhelming, so it should not be used productively.<br />
|}<br />
<br />
===DBPATH===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|DBPATH [path]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|<code><code>[[Configuration#Database Directory|{home}/data]]</code><br />
|- valign="top"<br />
| '''Summary'''<br />
|Points to the directory in which all databases are located.<br />
|}<br />
<br />
===LOGPATH===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|LOGPATH [path]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|<code>.logs</code><br />
|- valign="top"<br />
| '''Summary'''<br />
|Points to the directory where [[Logging|log files]] are stored. Relative paths are resolved against the {{Option|DBPATH}} directory.<br />
|}<br />
<br />
===REPOPATH===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|REPOPATH [path]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|<code>[[Configuration#Database Directory|{home}/repo]]</code><br />
|- valign="top"<br />
| '''Summary'''<br />
|Points to the [[Repository]], in which all XQuery modules are located.<br />
|}<br />
<br />
===LANG===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|LANG [language]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|English}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Specifies the interface language.<br />
|}<br />
<br />
===FAIRLOCK===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|FAIRLOCK [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Defines the locking strategy:<br />
* By default, non-fair locking is used. Read transactions are favored, and transactions that no not access databases can be evaluated even if the limit for parallel transactions (specified via {{Option|PARALLEL}}) is reached. This prevents update operations from blocking all other requests. For example, the DBA can still be used to see which jobs are running even if the queue is full.<br />
* When fair locking is enabled, read and write transactions are treated equally (first in, first out). This avoids starvation of updates and should be used when prompt evaluation of updates is critical.<br />
|}<br />
<br />
===CACHETIMEOUT===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|CACHETIMEOUT [seconds]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|3600}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Specifies how many seconds the results of [[Job Module|asynchronously executed jobs]] are cached in main memory.<br />
|}<br />
<br />
==Client/Server Architecture==<br />
<br />
===HOST===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|HOST [host]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|localhost}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|This host name is used by the client when connecting to a server. This option can also be changed when running the client on [[Command-Line Options#BaseX Client|command line]] via <code>-n</code>.<br />
|}<br />
<br />
===PORT===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|PORT [port]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|1984}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|This port is used by the client when connecting to a server. This option can also be changed when running the client on [[Command-Line Options#BaseX Client|command line]] via <code>-p</code>.<br />
|}<br />
<br />
===SERVERPORT===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|SERVERPORT [port]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|1984}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|This is the port the database server is listening to. This option can also be changed when running the server on [[Command-Line Options#BaseX Server|command line]] via <code>-p</code>.<br />
|}<br />
<br />
===USER===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|USER [name]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Represents a username, which is used for accessing the server or an HTTP service:<br />
* The default value is overwritten if a client specifies its own credentials.<br />
* If the default value is empty, login will only be possible if the client specifies credentials.<br />
* The option can also be changed on [[Command-Line Options#BaseX Client|command line]] via <code>-U</code>.<br />
|}<br />
<br />
===PASSWORD===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|PASSWORD [password]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Represents a password, which is used for accessing the server:<br />
* The default value is overwritten if a client specifies its own credentials.<br />
* If the default value is empty, authentication will only be possible if the client supplies credentials.<br />
* The option can also be changed on [[Command-Line Options#BaseX Client|command line]] via <code>-P</code>.<br />
* Please note that it is a security risk to specify your password in plain text.<br />
|}<br />
<br />
===AUTHMETHOD===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|AUTHMETHOD [method]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''Basic''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Specifies the default authentication method used by the [[Web Application|HTTP server]] for negotiating credentials. Allowed values are {{Code|Basic}}, {{Code|Digest}}, and {{Code|Custom}}:<br />
* If basic access is chosen, the client can still request digest authentication.<br />
* This is different for digest access, which cannot be overwritten.<br />
* With custom authentication, the server will not do any authentication.<br />
|}<br />
<br />
===SERVERHOST===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|SERVERHOST [host&#x7c;ip]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|This is the host name or IP address the server is bound to. If the option is set to an empty string (which is the default), the server accepts requests from all clients.<br />
|}<br />
<br />
===PROXYHOST===<br />
<br />
{| width='100%' width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|PROXYHOST [host]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty'' <br />
|- valign="top"<br />
| '''Summary'''<br />
|This is the host name of a proxy server. If the value is an empty string, it is ignored.<br />
|}<br />
<br />
===PROXYPORT===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|PROXYPORT [port]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|0}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|This is the port number of a proxy server. If the value is set to {{Code|0}}, it is ignored.<br />
|}<br />
<br />
===NONPROXYHOSTS===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|NONPROXYHOSTS [hosts]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|This is a list of hosts that should be directly accessed. If the value is an empty string, it is ignored.<br />
|}<br />
<br />
===IGNORECERT===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|IGNORECERT [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Disable SSL hostname verification and ignore untrusted certificates when connecting to servers.<br />
|}<br />
<br />
===TIMEOUT===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|TIMEOUT [seconds]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|30}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Specifies the maximum time a transaction triggered by a client may take. An operation is interrupted when it takes longer than the specified number of seconds. Running update operations are not affected by this timeout, as this would corrupt the integrity of the database. The timeout is disabled if the timeout is set to {{Code|0}}. It is ignored for operations with [[User Management|admin permissions]].<br />
|}<br />
<br />
===KEEPALIVE===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|KEEPALIVE [seconds]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|600}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Specifies the maximum time a client is remembered by the server. If there has been no interaction with a client for a longer time than specified by this timeout, it will be disconnected. Running operations will not be affected by this option. The check is disabled if the value is set to {{Code|0}}.<br />
|}<br />
<br />
===PARALLEL===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|PARALLEL [number]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|8}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Denotes the maximum allowed number of parallel [[Transaction Management|transactions]]:<br />
* If {{Option|FAIRLOCK}} is enabled, the number of parallel transactions will never exceed the specified value.<br />
* If the option is disabled (which is the default), the limit only applies to transactions that access databases.<br />
* The main reason for allowing parallel operations is to prevent slow transactions from blocking all other operations. A higher number of parallel operations may increase disk activity and thus slow down queries. In some cases, a single transaction may even give you better results than any parallel activity.<br />
|}<br />
<br />
===LOG===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|LOG [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|true}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Turns [[Logging]] of server operations and HTTP requests on/off. This option can also be changed when running the server on [[Command-Line Options#BaseX Server|command line]] via <code>-z</code>.<br />
|}<br />
<br />
===LOGMSGMAXLEN===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|LOGMSGMAXLEN [length]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|1000}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Specifies the maximum length of a single [[Logging|log message]].<br />
|}<br />
<br />
===LOGTRACE===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|LOGTRACE [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|true}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|If BaseX is running as a [[Web Application]], trace output (generated via {{Code|fn:trace}}, {{Function|Profiling|prof:dump}} and similar functions) is written to the [[Logging|database logs]]. When this option is disabled, trace output is redirected to standard error, as it is known from the standalone version of BaseX.<br />
|}<br />
<br />
==HTTP Services==<br />
<br />
Most HTTP options are defined in the {{Code|jetty.xml}} and {{Code|web.xml}} configuration files in the <code>[https://github.com/BaseXdb/basex/tree/master/basex-api/src/main/webapp/WEB-INF webapp/WEB-INF]</code> directory. Some additional BaseX-specific options exist that are set before the web server is started:<br />
<br />
===WEBPATH===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|WEBPATH [path]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|<code>[[Configuration#Database Directory|{home}/webapp]]</code><br />
|- valign="top"<br />
| '''Summary'''<br />
|Points to the directory in which all the [[Web Application]] contents are stored, including XQuery, Script, [[RESTXQ]] and configuration files:<br />
* The option is ignored if BaseX is deployed as [[Web Application#Servlet_Container|web servlet]].<br />
* It cannot be assigned via the {{Code|web.xml}} file, as it is evaluated before the configuration files are parsed.<br />
|}<br />
<br />
===GZIP===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|GZIP [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|<code>false</code><br />
|- valign="top"<br />
| '''Summary'''<br />
|Jetty provides a [https://www.eclipse.org/jetty/documentation/current/gzip-filter.html Gzip handler] for dynamically uncompressing requests and compressing responses. This feature can be enabled if Jetty is started via the [[Web Application|BaseX HTTP Server]]:<br />
* The option can also be enabled on [[Command-Line Options#HTTP Server|command line]] via <code>-g</code>.<br />
* It cannot be assigned via the {{Code|web.xml}} file, as the option is evaluated before the configuration files are parsed.<br />
* In addition to the [https://github.com/eclipse/jetty.project/blob/7cc552013eb4d05cb603ba0bc85d176c97957cd4/jetty-server/src/main/java/org/eclipse/jetty/server/handler/gzip/GzipHandler.java#L187-L211 Jetty defaults] (GET requests, exclusion of binaries, MSIE 6.0, etc.), POST and PUT requests are supported.<br />
|}<br />
<br />
===RESTXQPATH===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|RESTXQPATH [path]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Points to the directory which contains the [[RESTXQ]] modules of a web application. Relative paths are resolved against the {{Option|WEBPATH}} directory.<br />
|}<br />
<br />
===PARSERESTXQ===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|PARSERESTXQ}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|3}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Timeout after which the RESTXQ directory is parsed for changes:<br />
* If {{Code|0}} is specified, the directory is parsed every time a RESTXQ function is called.<br />
* A positive value defines the idle time, in seconds, after which parsing is enforced. The default value is {{Code|3}}: Changes in the RESTXQ directory is detected after 3 seconds without RESTXQ function calls.<br />
* Monitoring is completely disabled if a negative value is specified.<br />
<br />
See [[RESTXQ#Preliminaries|RESTXQ Preliminaries]] for more details.<br />
|}<br />
<br />
===RESTXQERRORS===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|RESTXQERRORS}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|true}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Reports parsing errors in XQuery modules in the RESTXQ directory and returns the full error message and stack trace to the client. By default, this option is enabled. In a production environment, it can be disabled to suppress errors that should not be seen by the user of an API (the full error information can still be looked up in the database logs). See [[RESTXQ#Error Handling|RESTXQ Error Handling]] for more details.<br />
|}<br />
<br />
===RESTPATH===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|RESTPATH [path]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Points to the directory which contains XQuery files and command scripts, which can be evaluated via the [[REST#GET Requests|REST run operation]]. Relative paths are resolved against the {{Option|WEBPATH}} directory.<br />
|}<br />
<br />
===HTTPLOCAL===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|HTTPLOCAL [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|By default, if BaseX is run as [[Web Application]], the database server instance is started in addition, which can then be addressed by [[Clients]] via the database port (see {{Option|PORT}}). If the option is set to {{Code|true}}, no database server is launched.<br />
|}<br />
<br />
===STOPPORT===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|STOPPORT [port]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|8081}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|If Jetty is started from BaseX, this is the port on which the [[Startup#BaseX HTTP Server|HTTP Server]] can be closed:<br />
* The listener for stopping the web server will only be started if the specified value is greater than {{Code|0}}.<br />
* The option is ignored if BaseX is used as a [[Web Application]] or started via [[Web Application#Maven|Maven]].<br />
* This option can also be changed when running the HTTP server on [[Command-Line Options#BaseX Server|command line]] via <code>-s</code>.<br />
|}<br />
<br />
=Create Options=<br />
<br />
==General==<br />
<br />
===MAINMEM===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|MAINMEM [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|If this option is turned on, new databases arecreated in main memory:<br />
* Most queries are evaluated faster in main-memory mode, but all data is lost if the BaseX instance in which the database was created is shut down.<br />
* It is not possible to store binary resources in a main-memory database.<br />
* A main-memory database will have no disk representation. However, it is possible to export the database via the {{Command|EXPORT}} command, and create a new database from the exported file in a second step.<br />
* This option will not be available for {{Function|Database|db:create}} because the database would not be accessible anymore after database creation, i.e., outside the query scope.<br />
|}<br />
<br />
===ADDCACHE===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|ADDCACHE [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|If this option is activated, data structures of documents will first be cached to disk before being added to the final database. This option is helpful when larger documents need to be added, and if the existing heuristics cannot estimate the input size (e.g. when adding directories or sending input streams).<br />
|}<br />
<br />
==Parsing==<br />
<br />
===CREATEFILTER===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|CREATEFILTER [filter]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|*.xml}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|File filter in the [[Commands#Glob Syntax|Glob Syntax]], which is applied whenever new databases are created, or resources are added to a database.<br />
|}<br />
<br />
===ADDARCHIVES===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|ADDARCHIVES [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|true}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|If this option is set to {{Code|true}}, files within archives (ZIP, GZIP, TAR, TGZ, DOCX, etc.) are parsed whenever new databases are created or resources are added to a database.<br />
|}<br />
<br />
===ARCHIVENAME===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|ARCHIVENAME [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|If this option is set to {{Code|true}}, the file name of parsed archives is included in the document paths.<br />
|}<br />
<br />
===SKIPCORRUPT===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|SKIPCORRUPT [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Skips corrupt (i.e., not well-formed) files while creating a database or adding new documents. If this option is activated, document updates are slowed down, as all files are parsed twice. Next, main-memory consumption is higher, as parsed files are cached in main memory.<br />
|}<br />
<br />
===ADDRAW===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|ADDRAW [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|If this option is enabled, all resources that are filtered out by the {{Option|CREATEFILTER}} option while being added to a database are stored as [[Binary Data|raw files]] instead (i.e., in their binary representation).<br />
|}<br />
<br />
===PARSER===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|PARSER [type]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|XML}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Defines a [[Parsers|parser]] for importing new files to the database. Available parsers are {{Code|XML}}, {{Code|JSON}}, {{Code|CSV}}, {{Code|TEXT}}, {{Code|HTML}}, and {{Code|RAW}}. HTML input is parsed as XML if [[Parsers#HTMLParser|Tagsoup]] is not found in the class path.<br />
|}<br />
<br />
===CSVPARSER===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|CSVPARSER [options]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Specifies the way how CSV data is parsed. Keys and values are delimited with <code>=</code>, and multiple options are delimited with <code>,</code>. The available options (except for the additional <code>encoding</code> option) are described in the [[CSV Module#Options|CSV Module]].<br />
|- valign="top"<br />
| '''Examples'''<br />
|<code>encoding=CP1252,header=true</code> parses the input as CP1252 and the first line as header.<br />
|}<br />
<br />
===JSONPARSER===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|JSONPARSER [options]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Specifies the way how JSON data is parsed. Keys and values are delimited with <code>=</code>, and multiple options are delimited with <code>,</code>. The available options (except for the additional <code>encoding</code> option) are described in the [[JSON Module#Options|JSON Module]].<br />
|- valign="top"<br />
| '''Examples'''<br />
|<code>format=jsonml,lax=yes</code> interprets the input as JSONML and uses lax parsing.<br />
|}<br />
<br />
===HTMLPARSER===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|HTMLPARSER [options]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Specifies the way how HTML data is parsed. Keys and values are delimited with <code>=</code>, and multiple options are delimited with <code>,</code>. The available options are described in the [[Parsers#Options|Parsers]] article.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>encoding=Shift-JIS,nons=true</code> parses the input as Shift-JIS and suppresses namespaces.<br />
* <code>lexical=true</code> preserves comments.<br />
|}<br />
<br />
==XML Parsing==<br />
<br />
===STRIPWS===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|STRIPWS [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Many XML documents include whitespaces that have been added to improve readability. This option controls the [https://www.w3.org/TR/REC-xml/#sec-white-space white-space processing mode] of the XML parser:<br />
* If the option is set to {{Code|true}}, leading and trailing whitespaces from text nodes are stripped and empty text nodes are discarded.<br />
* The flag should be disabled if a document contains [[Full-Text#Mixed Content|mixed content]].<br />
* The flag can also be turned on via the [[Command-Line Options#BaseX Standalone|command line]] and <code>-w</code>.<br />
* If the option is enabled, whitespaces of an element and its descendants can locally be preserved with the <code>xml:space="preserve"</code> attribute:<br />
<br />
<pre lang="xml"><br />
<xml><br />
<title><br />
Demonstrating the STRIPWS flag<br />
</title><br />
<text xml:space="preserve">To <b>be</b>, or not to <b>be</b>, that is the question.</text><br />
</xml><br />
</pre><br />
If whitespaces are stripped, <code>indent=yes</code> can be assigned to the {{Option|SERIALIZER}} option to get properly indented XML output.<br/><br />
See [[BaseX_10#Whitespaces|BaseX 10: Whitespaces]] for more information on whitespace handling.<br />
|}<br />
<br />
===STRIPNS===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|STRIPNS [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Strips all namespaces from an XML document while parsing.<br />
|}<br />
<br />
===INTPARSE===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|INTPARSE [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Uses the internal XML parser instead of the standard Java XML parser. Reasons for using the internal parser are:<br />
* Performance: Documents, in particular small ones, are parsed faster.<br />
* Entity support: More than 2000 [https://html.spec.whatwg.org/multipage/named-characters.html HTML character references] are decoded.<br />
* Fault tolerance: Invalid characters are replaced with the Unicode replacement character <code>FFFD</code> (&#xFFFD;).<br />
By default, Java’s Xerces parser is used as it supports all features of the XML standard, including {{Option|XINCLUDE}}, or recursive entity expansion if specified in DTDs.<br />
|}<br />
<br />
===DTD===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|DTD [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Parses referenced DTDs and resolves XML entities. By default, this option is switched to {{Code|false}}, as many DTDs are located externally, which may completely block the process of creating new databases. The {{Option|CATALOG}} option can be changed to locally resolve DTDs.<br />
|}<br />
<br />
===XINCLUDE===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|XINCLUDE [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|true}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Resolves XInclude inclusion tags and merges referenced XML documents. By default, this option is switched to {{Code|true}}. This option is only available if the standard Java XML Parser is used (see {{Option|INTPARSE}}).<br />
|}<br />
<br />
===CATALOG===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|CATALOG [path]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Semicolon-separated list of XML catalog files to resolve URIs. See [[Catalog Resolver]]s for more details.<br />
|}<br />
<br />
==Indexing==<br />
<br />
The following options control the creation of index structures. The current values are considered if a new database is created. See [[Indexes]] for more details.<br />
<br />
===TEXTINDEX===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|TEXTINDEX [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|true}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a text index whenever a new database is created. A text index speeds up queries with equality comparisons on text nodes. See [[Index#Text Index|Text Index]] for more details.<br />
|}<br />
<br />
===ATTRINDEX===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|ATTRINDEX [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|true}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates an attribute index whenever a new database is created. An attribute index speeds up queries with equality comparisons on attribute values. See [[Index#Attribute Index|Attribute Index]] for more details.<br />
|}<br />
<br />
===TOKENINDEX===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|TOKENINDEX [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|true}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a token index whenever a new database is created. A token index speeds up searches for single tokens in attribute values. See [[Index#Token Index|Token Index]] for more details.<br />
|}<br />
<br />
===FTINDEX===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|FTINDEX [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a full-text index whenever a new database is created. A full-text index speeds up queries with full-text expressions. See [[Index#Full-Text Index|Full-Text Index]] for more details.<br />
|}<br />
<br />
===TEXTINCLUDE===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|TEXTINCLUDE [names]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Defines name patterns for the parent elements of texts that are indexed. By default, all text nodes are indexed.<br/>Name patterns are separated by commas. See [[Indexes#Selective Indexing|Selective Indexing]] for more details.<br />
|}<br />
<br />
===ATTRINCLUDE===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|ATTRINCLUDE [names]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Defines name patterns for the attributes to be indexed. By default, all attribute nodes are indexed.<br/>Name patterns are separated by commas. See [[Indexes#Selective Indexing|Selective Indexing]] for more details.<br />
|}<br />
<br />
===TOKENINCLUDE===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|TOKENINCLUDE [names]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Defines name patterns for the attributes to be indexed. By default, tokens in all attribute nodes are indexed.<br/>Name patterns are separated by commas. See [[Indexes#Selective Indexing|Selective Indexing]] for more details.<br />
|}<br />
<br />
===FTINCLUDE===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|FTINCLUDE [names]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Defines name patterns for the parent elements of texts that are indexed. By default, all text nodes are indexed.<br/>Name patterns are separated by commas. See [[Indexes#Selective Indexing|Selective Indexing]] for more details.<br />
|}<br />
<br />
===MAXLEN===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|MAXLEN [int]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|96}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Specifies the maximum length for strings to be stored in [[Indexes|index structures]]. The value of this option is assigned once to a new database, and can only be changed by creating a new database or doing a [[Commands#OPTIMIZE|full optimization]].<br />
|}<br />
<br />
===MAXCATS===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|MAXCATS [int]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|100}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Specifies the maximum number of distinct values (categories) that are stored together with the element/attribute names or unique paths in the [[Index#Name Index|Name Index]] or [[Index#Path Index|Path Index]]. The value of this option is assigned once to a new database, and cannot be changed after that.<br />
|}<br />
<br />
===UPDINDEX===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|UPDINDEX [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|If turned on, incremental indexing is enabled:<br />
* The current value of this option is assigned to new databases. It can be changed for existing databases by running {{Command|OPTIMIZE}} with the {{Code|ALL}} keyword or {{Function|Database|db:optimize}} and {{Code|true()}} as second argument.<br />
* After each update, the value indexes are refreshed as well. Incremental updates are currently not available for the full-text index and database statistics.<br />
* Find more details in the article on [[Index#Updates|Index Structures]].<br />
|}<br />
<br />
===AUTOOPTIMIZE===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|AUTOOPTIMIZE [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|If turned on, auto optimization is applied to new databases:<br />
* With each update, outdated indexes and database statistics are recreated.<br />
* As a result, the index structures will always be up-to-date.<br />
* However, updates can take much longer, so this option should only be activated for medium-sized databases.<br />
* The value of this option is assigned once to a new database. It can be reassigned by running {{Command|OPTIMIZE}} or {{Function|Database|db:optimize}}.<br />
|}<br />
<br />
===SPLITSIZE===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|SPLITSIZE [num]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|0}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|This option affects the [[Indexes#Performance|construction]] of new value indexes. It controls the number of index build operations that are performed before writing partial index data to disk:<br />
* By default, if the value is set to {{Code|0}}, some heuristics are applied, based on the current memory consumption. Usually, this works fine.<br />
* If explicit garbage collection is disabled when running Java (e.g. via the JVM option {{Code|-XX:+DisableExplicitGC}}), you may need to choose a custom split size.<br />
* You can e.g. start with {{Code|1000000}} (one million) index operations and adjust this value in the next steps.<br />
* The larger the assigned value is, the fewer splits take place, and main-memory consumption increases.<br />
|}<br />
<br />
==Full-Text Indexing==<br />
<br />
===STEMMING===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|STEMMING [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|If {{Code|true}}, all tokens are stemmed during full-text indexing, using a language-specific stemmer implementation. By default, tokens will not be stemmed. See [[Indexes#Full-Text Index|Full-Text Index]] for more details.<br />
|}<br />
<br />
===CASESENS===<br />
<br />
{| width='100%'<br />
<br />
| width='120' | '''Signature'''<br />
|{{Code|CASESENS [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|If {{Code|true}}, the case of tokens is preserved during full-text indexing. By default, case is ignored (all tokens are indexed in lower case). See [[Indexes#Full-Text Index|Full-Text Index]] for more details.<br />
|}<br />
<br />
===DIACRITICS===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|DIACRITICS [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|If set to {{Code|true}}, diacritics are preserved during full-text indexing. By default, diacritics are removed. See [[Indexes#Full-Text Index|Full-Text Index]] for more details.<br />
|}<br />
<br />
===LANGUAGE===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|LANGUAGE [lang]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|en}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|The specified language will influence the way how texts are tokenized and stemmed. It can be the name of a language or a language code. See [[Indexes#Full-Text Index|Full-Text Index]] for more details.<br />
|}<br />
<br />
===STOPWORDS===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|STOPWORDS [path]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|If a text file with stop words is specified, frequently used terms contained in that file are ignored when a full-text index is created. A stop word list may decrease the size of the full-text index and speed up your queries. See [[Indexes#Full-Text Index|Full-Text Index]] for more details.<br />
|}<br />
<br />
=Query Options=<br />
<br />
===QUERYINFO===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|QUERYINFO [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Prints more information on internal query rewritings, optimizations, and performance. By default, this info is shown in the [[GUI#Visualizations|Info View]] in the GUI. It can also be activated on [[Command-Line Options#BaseX Standalone|command line]] via <code>-V</code>. <br />
|}<br />
<br />
===MIXUPDATES===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|MIXUPDATES}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Allows queries to both contain updating and non-updating expressions. All updating constraints are turned off, and nodes to be returned are copied before they are modified by an updating expression. By default, in compliance with the XQuery Update Facility, this option is set to {{Code|false}}. See [[XQuery Update#Returning Results|Returning Results]] for more details.<br />
|}<br />
<br />
===BINDINGS===<br />
<br />
{{Announce|Updated with Version 11}}: The Clark notation was replaced with the [[XQuery 3.0#Expanded QNames|Expanded QNames]] notation.<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|BINDINGS [vars]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Contains external variables to be bound to a query. The string must comply with the following rules:<br />
* Variable names and values must be separated by equality signs.<br />
* Multiple variables must be delimited by commas.<br />
* Commas in values must be duplicated.<br />
* Variables may optionally be introduced with a leading dollar sign.<br />
* If a variable uses a namespace different from the default namespace, it can be specified with the [[XQuery 3.0#Expanded QNames|Expanded QNames]] notation.<br />
This option can also be used on [[Command-Line Options#BaseX Standalone|command line]] with the flag <code>-b</code>.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>$a=1,$b=2</code> &nbsp; binds the values {{Code|1}} and {{Code|2}} to the variables $a and $b<br />
* <code>a=1,,2</code> &nbsp; binds the value {{Code|1,2}} to the variable $a<br />
* <code>{URI}a=x</code> &nbsp; binds the value {{Code|x}} to the variable $a with the namespace {{Code|URI}}.<br />
* In the following [[Commands#Command_Scripts| Command Script]], the value {{Code|hello world!}} is bound to the variable {{Code|$GREETING}}:<br />
<pre lang='xquery'><br />
SET BINDINGS GREETING="hello world!"<br />
XQUERY declare variable $GREETING external; $GREETING<br />
</pre><br />
|}<br />
<br />
===INLINELIMIT===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|INLINELIMIT}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|50}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|This option controls inlining of XQuery functions:<br />
* The XQuery compiler inlines functions to speed up query evaluation.<br />
* Inlining will only take place if a function body is not too large (i.e., if it does not contain too many expressions).<br />
* With this option, this maximum number of expressions can be specified.<br />
* Function inlining can be turned off by setting the value to {{Code|0}}.<br />
* The limit can be locally overwritten via the [[XQuery Extensions#Function Inlining|%basex:inline]] annotation (follow the link to get more information on function inlining).<br />
|}<br />
<br />
===UNROLLLIMIT===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|UNROLLLIMIT}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|5}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|This option controls the unroll limit:<br />
* Loops with few iterations are ''unrolled'' by the XQuery compiler to enable further optimizations.<br />
* If the limit is increased, more optimizations will take place, but the memory consumption and compile time will increase.<br />
* See [[XQuery Optimizations#Loop Unrolling|Loop Unrolling]] for more details.<br />
|}<br />
<br />
===ENFORCEINDEX===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|ENFORCEINDEX [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Enforces index rewritings in path expressions. See [[Indexes#Enforce Rewritings|Enforce Rewritings]] for details.<br />
|}<br />
<br />
===COPYNODE===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|COPYNODE [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|true}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|When creating new nodes in XQuery via [https://www.w3.org/TR/xquery-31/#id-constructors Node Constructors], copies of all enclosed nodes are created, and the copied nodes get new node identities. As a result, the following query yields <code>false</code>:<br />
<pre lang='xquery'><br />
let $a := <a/><br />
let $b := <b>{ $a }</b><br />
return $b/a is $a<br />
</pre><br />
This step can be expensive and memory consuming. If the option is disabled, child nodes will only be linked to the new parent nodes, and the upper query returns <code>true</code>.<br/>The option should be used carefully as it changes the semantics of XQuery. It should preferably be used in [[XQuery Extensions#Database Pragmas|Pragmas]].<br />
|}<br />
<br />
===TAILCALLS===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|TAILCALLS}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|256}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Specifies how many stack frames of [https://en.wikipedia.org/wiki/Tail_call tail-calls] are allowed on the stack at any time. When this limit is reached, tail-call optimization takes place and some call frames are eliminated. The feature can be turned off by setting the value to {{Code|-1}}.<br />
|}<br />
<br />
===WITHDB===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|WITHDB}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|true}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|By default, resources specified via {{Code|fn:doc}} and {{Code|fn:collection}} are looked up both in the database and in the file system. If you always use {{Function|Database|db:get}} to access databases, it is recommendable to disable this option:<br />
* No locks are created for the two functions (see [[Transaction Management#Limitations|limitations of database locking]] for more details).<br />
* Access to local and external resources is faster, as the database lookup is skipped.<br />
|}<br />
<br />
===DEFAULTDB===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|DEFAULTDB}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|If this option is turned on, paths specified in the {{Code|fn:doc}} and {{Code|fn:collection}} functions will first be resolved against a database that has been opened in the global context outside the query (e.g. by the {{Command|OPEN}} command). If the path does not match any existing resources, it is resolved as described in the article on [[Databases#Access Resources|accessing database resources]].<br />
|}<br />
<br />
===FORCECREATE===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|FORCECREATE [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|By enabling this option, database instances are created if the XQuery functions {{Code|fn:doc}} and {{Code|fn:collection}} are called.<br />
|}<br />
<br />
===CHECKSTRINGS===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|CHECKSTRINGS [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|true}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|By default, characters from external sources that are invalid in XML will trigger an error. If the option is set to <code>false</code>, these characters are replaced with the Unicode replacement character <code>FFFD</code> (&#xFFFD;). The option affects [[Java Bindings]] and string conversion and input functions such as {{Function|Archive|archive:create}}, {{Function|Archive|archive:extract-text}}, and {{Function|Archive|archive:update}}.<br />
|}<br />
<br />
===WRAPJAVA===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|WRAPJAVA [mode]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|some}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Defines the way how values that result from Java code invocation are converted to XQuery items. Allowed values: {{Code|none}}, {{Code|all}}, {{Code|some}}, {{Code|instance}}, and {{Code|void}}. See [[Java_Bindings#Conversion to XQuery|Java Bindings]] for further details.<br />
|}<br />
<br />
===LSERROR===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|LSERROR [error]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|0}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|This option specifies the maximum Levenshtein error for fuzzy full-text matching. By default, if {{Code|0}} is assigned, the error value is calculated dynamically. See [[Full-Text#Fuzzy_Querying|Fuzzy Querying]] for more details.<br />
|}<br />
<br />
===RUNQUERY===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|RUNQUERY [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|true}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Specifies if a query is executed or parsed only. This option can also be changed on [[Command-Line Options#BaseX Standalone|command line]] via <code>-R</code>.<br />
|}<br />
<br />
===RUNS===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|RUNS [num]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|1}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Specifies how often a query is evaluated. The result is serialized only once, and the measured times are averages of all runs. This option can also be changed on [[Command-Line Options#BaseX Standalone|command line]] via <code>-r</code>.<br />
|}<br />
<br />
=Serialization Options=<br />
<br />
===SERIALIZE===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|SERIALIZE [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|true}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Results of XQuery expressions are serialized if this option is turned on. For debugging purposes and performance measurements, this option can be set to {{Code|false}}. It can also be turned off on [[Command-Line Options#BaseX Standalone|command line]] via <code>-z</code>. <br />
|}<br />
<br />
===SERIALIZER===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|SERIALIZER [params]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Parameters for [[Serialization|serializing]] query results:<br />
* Variable names and values are separated by equality signs.<br />
* Multiple variables are delimited by commas.<br />
* Commas must be duplicated if they appear as literals in values.<br />
The option can also be used on [[Command-Line Options#BaseX Standalone|command line]] with the flag <code>-s</code>.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>indent=yes</code> : enables automatic indentation of XML nodes. This is recommended if whitespaces have been stripped from a document (see {{Options|STRIPWS}}).<br />
* <code>encoding=US-ASCII,omit-xml-declaration=no</code> : sets the encoding to {{Code|US-ASCII}} and prints the XML declaration.<br />
* <code>item-separator=,,</code> : separates serialized items by a single comma.<br />
|}<br />
<br />
===EXPORTER===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|EXPORTER [params]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|''empty''<br />
|- valign="top"<br />
| '''Summary'''<br />
|Contains parameters for exporting resources of a database and writing files after updates via the {{Option|WRITEBACK}} option. Keys and values are separated by equality signs, multiple parameters are delimited by commas. See [[Serialization]] for more details.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>indent=no,omit-xml-declaration=no</code> : disables automatic indentation of XML nodes, outputs the XML declaration.<br />
|}<br />
<br />
===XMLPLAN===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|XMLPLAN [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Prints the execution plan of an XQuery expression in its XML representation. This option can also be activated on [[Command-Line Options#BaseX Standalone|command line]] via <code>-x</code>. <br />
|}<br />
<br />
===FULLPLAN===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|FULLPLAN [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Attaches the file path, line and column of the expressions in the original query string to the query plan. Values (items and sequences) have no input information attached.<br />
|}<br />
<br />
=Other Options=<br />
<br />
===AUTOFLUSH===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|AUTOFLUSH [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|true}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Flushes database buffers to disk after each update. If this option is set to {{Code|false}}, bulk operations (multiple single updates) are evaluated faster. As a drawback, the chance of data loss increases if the database is not explicitly flushed via the {{Command|FLUSH}} command.<br />
|}<br />
<br />
===WRITEBACK===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|WRITEBACK [boolean]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|false}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Propagates updates on main-memory instances of files that have been retrieved via {{Code|fn:doc}} and {{Code|fn:collection}} back to disk:<br />
* This option can also be activated on [[Command-Line Options#BaseX Standalone|command line]] via <code>-u</code>.<br />
* Please take in mind that no backup is created from your original files.<br />
* The serialization options can be controlled via the {{Option|EXPORTER}} option.<br />
|}<br />
<br />
===MAXSTAT===<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|MAXSTAT [num]}}<br />
|- valign="top"<br />
| '''Default'''<br />
|{{Code|30}}<br />
|- valign="top"<br />
| '''Summary'''<br />
|Specifies the maximum number of index occurrences printed by the {{Command|INFO INDEX}} command.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 11.0<br />
* Removed: <code>TEXTPARSER</code><br />
* Updated: {{Option|BINDINGS}}: The Clark notation was replaced with the [[XQuery 3.0#Expanded QNames|Expanded QNames]] notation.<br />
<br />
;Version 10.0<br />
* Removed: <code>COMPPLAN</code>, <code>IGNOREHOSTNAME</code><br />
* Updated: {{Option|IGNORECERT}}: Additionally disable hostname verification.<br />
* Updated: {{Option|CATALOG}}: Renamed (before: {{Code|CATFILE}}).<br />
* Updated: {{Option|STRIPWS}}: Renamed (before: {{Code|CHOP}}), new default: false.<br />
<br />
;Version 9.7<br />
* Updated: {{Option|GZIP}}: Support for POST and PUT requests.<br />
<br />
;Version 9.6<br />
* Added: {{Option|UNROLLLIMIT}}, {{Option|WRAPJAVA}}<br />
<br />
;Version 9.5<br />
* Updated: {{Option|INLINELIMIT}}: default reduced to 50.<br />
* Updated: {{Option|RESTXQERRORS}}: additionally suppress stack trace in HTTP response<br />
<br />
;Version 9.4<br />
* Added: {{Option|LOGTRACE}}<br />
<br />
;Version 9.3<br />
* Added: {{Option|WITHDB}}, {{Option|GZIP}}<br />
<br />
;Version 9.2<br />
* Added: {{Option|RESTXQERRORS}}, {{Option|FULLPLAN}}<br />
* Removed: <code>DOTPLAN</code>, <code>DOTCOMPACT</code><br />
<br />
;Version 9.0<br />
* Added: {{Option|ENFORCEINDEX}}, {{Option|COPYNODE}}, {{Option|IGNOREHOSTNAME}}<br />
<br />
;Version 8.6<br />
* Added: {{Option|FAIRLOCK}}, {{Option|PARSERESTXQ}}<br />
* Removed: {{Code|GLOBALLOCK}} (exclusive use of database lock)<br />
* Removed: {{Code|QUERYPATH}} (will now be internally assigned)<br />
* Removed: {{Code|CACHERESTXQ}} (replaced with PARSERESTXQ)<br />
<br />
;Version 8.5<br />
* Added: {{Option|CACHETIMEOUT}}, {{Option|LOGPATH}}<br />
* Updated: {{Option|AUTHMETHOD}}: {{Code|custom}} value added.<br />
<br />
;Version 8.4<br />
* Added: {{Option|TOKENINDEX}}, {{Option|TOKENINCLUDE}}<br />
* Added: {{Option|SPLITSIZE}} (replacing <code>INDEXSPLITSIZE</code> and <code>FTINDEXSPLITSIZE</code>)<br />
* Removed: <code>INDEXSPLITSIZE</code>, <code>FTINDEXSPLITSIZE</code><br />
<br />
;Version 8.3<br />
* Added: {{Option|CACHERESTXQ}}, {{Option|TEXTINCLUDE}}, {{Option|ATTRINCLUDE}}, {{Option|FTINCLUDE}}, {{Option|ARCHIVENAME}}<br />
<br />
;Version 8.2<br />
* Removed: <code>EVENTPORT</code>, <code>CACHEQUERY</code><br />
<br />
;Version 8.1<br />
* Added: {{Option|IGNORECERT}}, {{Option|RESTPATH}}<br />
<br />
;Version 8.0<br />
* Added: {{Option|MIXUPDATES}}, {{Option|AUTOOPTIMIZE}}, {{Option|AUTHMETHOD}}, {{Option|XINCLUDE}}<br />
* Updated: {{Option|PROXYPORT}}: default set to 0; will be ignored. {{Option|PROXYHOST}}, {{Option|NONPROXYHOSTS}}: empty strings will be ignored.<br />
<br />
;Version 7.8.1<br />
* Updated: {{Option|ADDARCHIVES}}: parsing of TAR and TGZ files.<br />
<br />
;Version 7.8<br />
<br />
* Added: {{Option|CSVPARSER}}, {{Option|JSONPARSER}}, TEXTPARSER, {{Option|HTMLPARSER}}, {{Option|INLINELIMIT}}, {{Option|TAILCALLS}}, {{Option|DEFAULTDB}}, {{Option|RUNQUERY}}<br />
* Updated: {{Option|WRITEBACK}} only applies to main-memory document instances.<br />
* Updated: {{Option|DEBUG}} option can be changed at runtime by users with admin permissions.<br />
* Updated: default of {{Option|INTPARSE}} is now {{Code|false}}.<br />
* Removed: <code>HTMLOPT</code> (replaced with {{Option|HTMLPARSER}}), <code>PARSEROPT</code> (replaced with parser-specific options), <code>DOTDISPLAY</code>, <code>DOTTY</code><br />
<br />
;Version 7.7<br />
* Added: {{Option|ADDCACHE}}, {{Option|CHECKSTRINGS}}, {{Option|FTINDEXSPLITSIZE}}, {{Option|INDEXSPLITSIZE}}<br />
<br />
;Version 7.6<br />
* Added: {{Option|GLOBALLOCK}}<br />
* Added: store local options in configuration file after {{Code|# Local Options}} comments.<br />
<br />
;Version 7.5<br />
* Added: options can now be set via system properties<br />
* Added: a pragma expression can be used to locally change database options<br />
* Added: {{Option|USER}}, {{Option|PASSWORD}}, {{Option|LOG}}, {{Option|LOGMSGMAXLEN}}, {{Option|WEBPATH}}, {{Option|RESTXQPATH}}{{Option|HTTPLOCAL}}, {{Option|CREATEONLY}}, {{Option|STRIPNS}}<br />
* Removed: {{Code|HTTPPATH}}; {{Code|HTTPPORT}}: {{Code|jetty.xml}} configuration file is used instead<br />
* Removed: global options cannot be changed anymore during the lifetime of a BaseX instance<br />
<br />
;Version 7.3<br />
* Updated: {{Option|KEEPALIVE}}, {{Option|TIMEOUT}}: default values changed<br />
* Removed: {{Code|WILDCARDS}}; new index supports both fuzzy and wildcard queries<br />
* Removed: {{Code|SCORING}}; new scoring model will focus on lengths of text nodes and match options<br />
<br />
;Version 7.2<br />
* Added: {{Option|PROXYHOST}}, {{Option|PROXYPORT}}, {{Option|NONPROXYHOSTS}}, {{Option|HTMLOPT}}<br />
* Updated: {{Option|TIMEOUT}}: ignore timeout for admin users<br />
<br />
;Version 7.1<br />
* Added: {{Option|ADDRAW}}, {{Option|MAXLEN}}, {{Option|MAXCATS}}, {{Option|UPDINDEX}}<br />
* Updated: {{Option|BINDINGS}}<br />
<br />
;Version 7.0<br />
* Added: {{Option|SERVERHOST}}, {{Option|KEEPALIVE}}, {{Option|AUTOFLUSH}}, {{Option|QUERYPATH}}</div>CGhttps://docs.basex.org/index.php?title=HTML_Module&diff=16884HTML Module2023-12-01T16:39:00Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This [[Module Library|XQuery Module]] provides functions for converting HTML to XML. Conversion will only take place if TagSoup is included in the classpath (see [[Parsers#HTML Parser|HTML Parsing]] for more details).<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/html</nowiki></code> namespace, which is statically bound to the {{Code|html}} prefix.<br/><br />
<br />
=Functions=<br />
<br />
==html:doc==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>html:doc(<br />
$href as xs:string?,<br />
$options as map(*)? := map { }<br />
) as document-node()?</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Fetches the HTML document referred to by the given {{Code|$href}}, converts it to XML and returns a document node. The {{Code|$options}} argument can be used to set [[Parsers#Options|TagSoup Options]].<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|parse|#Errors}} the input cannot be converted to XML.<br />
|}<br />
<br />
==html:parse==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>html:parse(<br />
$value as xs:anyAtomicType,<br />
$options as map(*)? := map { }<br />
) as document-node()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Converts the HTML document specified by {{Code|$value}} to XML and returns a document node:<br/><br />
* The input may be of type {{Code|xs:string}}, {{Code|xs:base64Binary}}, or {{Code|xs:hexBinary}}.<br />
* If the input is passed on in its binary representation, and if no encoding option is supplied, the HTML parser will try to choose the correct encoding automatically.<br />
<br />
The {{Code|$options}} argument can be used to set [[Parsers#Options|TagSoup Options]].<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|parse|#Errors}} the input cannot be converted to XML.<br />
|}<br />
<br />
==html:parser==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|{{Code|'''html:parser'''() as xs:string}}<br/><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the name of the applied HTML parser (currently: {{Code|TagSoup}}). If an ''empty string'' is returned, TagSoup was not found in the classpath, and the input will be treated as well-formed XML.<br/><br />
|}<br />
<br />
=Examples=<br />
<br />
===Basic Example===<br />
<br />
The following query converts the specified string to an XML document node.<br />
<br />
;Query:<br />
<pre lang='xquery'><br />
html:parse("<html>")<br />
</pre><br />
<br />
;Result:<br />
<pre lang="xml"><br />
<html xmlns="http://www.w3.org/1999/xhtml"/><br />
</pre><br />
<br />
===Specifying Options===<br />
<br />
The next query creates an XML document with namespaces:<br />
<br />
;Query:<br />
<pre lang='xquery'><br />
html:parse("<a href='ok.html'/>", map { 'nons': false() })<br />
</pre><br />
<br />
;Result:<br />
<pre lang="xml"><br />
<html xmlns="http://www.w3.org/1999/xhtml"><br />
<body><br />
<a shape="rect" href="ok.html"/><br />
</body><br />
</html><br />
</pre><br />
<br />
===Parsing Binary Input===<br />
<br />
If the input encoding is unknown, the data to be processed can be passed on in its binary representation.<br />
The HTML parser will automatically try to detect the correct encoding:<br />
<br />
;Query:<br />
<pre lang='xquery'><br />
html:parse(fetch:binary("https://en.wikipedia.org"))<br />
</pre><br />
<br />
;Result:<br />
<pre lang="xml"><br />
<html xmlns="http://www.w3.org/1999/xhtml" class="client-nojs" dir="ltr" lang="en"><br />
<head><br />
<title>Wikipedia, the free encyclopedia</title><br />
<meta charset="UTF-8"/><br />
...<br />
</pre><br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
|Description<br />
|- valign="top"<br />
|{{Code|parse}}<br />
|The input cannot be converted to XML.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 9.4<br />
<br />
* Added: {{Function||html:doc}}<br />
<br />
;Version 9.0<br />
<br />
* Updated: error codes updated; errors now use the module namespace<br />
<br />
The module was introduced with Version 7.6.</div>CGhttps://docs.basex.org/index.php?title=Validation_Module&diff=16883Validation Module2023-12-01T16:38:56Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This [[Module Library|XQuery Module]] contains functions to perform validations against DTDs, XML Schema and RelaxNG. The documentation further describes how to use Schematron validation with BaseX.<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/validate</nowiki></code> namespace, which is statically bound to the {{Code|validate}} prefix.<br/><br />
<br />
=DTD Validation=<br />
<br />
Checks whether an XML document validates against a DTD. The input document can be specified as:<br />
<br />
* {{Code|xs:string}}, representing a URI (relative URIs will always be resolved against the static base URI of the query),<br />
* {{Code|xs:string}}, representing the resource in its string representation, or<br />
* {{Code|node()}}, representing the resource itself.<br />
<br />
If no DTD is supplied in a function, the XML document is expected to contain an embedded DTD doctype declaration.<br />
<br />
==validate:dtd==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>validate:dtd(<br />
$input as item(),<br />
$schema as xs:string? := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Validates the XML {{Code|$input}} document against a {{Code|$schema}} and returns an empty sequence or an error.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|error|#Errors}} the validation fails.<br/>{{Error|init|#Errors}} the validation process cannot be started.<br/>{{Error|not-found|#Errors}} no DTD validator is available.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|validate:dtd('doc.xml', 'doc.dtd')}} validates the document {{Code|doc.xml}} against the specified DTD file {{Code|doc.dtd}}.<br />
* The following example validates an invalid document against a DTD, which is specified as string:<br />
<pre lang='xquery'><br />
try {<br />
let $doc := <invalid/><br />
let $schema := '<!ELEMENT root (#PCDATA)>'<br />
return validate:dtd($doc, $schema)<br />
} catch validate:error {<br />
'DTD Validation failed.'<br />
}<br />
</pre><br />
|}<br />
<br />
==validate:dtd-info==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>validate:dtd-info(<br />
$input as item(),<br />
$schema as xs:string? := ()<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Validates the XML {{Code|$input}} document against a {{Code|$schema}} and returns warnings, errors and fatal errors in a string sequence.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|init|#Errors}} the validation process cannot be started.<br/>{{Error|not-found|#Errors}} no DTD validator is available.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>validate:dtd-info(<invalid/>, '<!ELEMENT root (#PCDATA)>')</code> returns:<br/><code>2:11: Element type "invalid" must be declared.</code>.<br />
|}<br />
<br />
==validate:dtd-report==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>validate:dtd-report(<br />
$input as item(),<br />
$schema as xs:string? := ()<br />
) as element(report)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Validates the XML {{Code|$input}} document against a {{Code|$schema}} and returns warnings, errors and fatal errors as XML.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|init|#Errors}} the validation process cannot be started.<br/>{{Error|not-found|#Errors}} no DTD validator is available.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* <code>validate:dtd-report(<invalid/>, '<!ELEMENT root (#PCDATA)>')</code> returns:<br />
<pre lang="xml"><br />
<report><br />
<status>invalid</status><br />
<message level="Error" line="2" column="11">Element type "invalid" must be declared.</message><br />
</report><br />
</pre><br />
|}<br />
<br />
=XML Schema Validation=<br />
<br />
Checks whether an XML document validates against an XML Schema. The input document and the schema can be specified as:<br />
<br />
* {{Code|xs:string}}, containing the path to the resource,<br />
* {{Code|xs:string}}, containing the resource in its string representation, or<br />
* {{Code|node()}}, containing the resource itself.<br />
<br />
If no schema is given, the input is expected to contain an {{Code|xsi:(noNamespace)schemaLocation}} attribute, as defined in [https://www.w3.org/TR/xmlschema-1/#xsi_schemaLocation W3C XML Schema].<br />
<br />
Different XML Schema processors are supported:<br />
<br />
* By default, the '''Java implementation''' of XML Schema 1.0 is used (it is based on an old version of Apache Xerces).<br />
* The latest version of '''Xerces2''' provides implementations of XML Schema 1.0 and 1.1. The processor will be applied if you download one of the [https://xerces.apache.org/mirrors.cgi#binary binary distributions] and copy the following libraries to the {{Code|lib/custom}} directory of the full distribution of BaseX:<br />
** {{Code|org.eclipse.wst.xml.xpath2.processor_1.2.0.jar}}<br />
** {{Code|cupv10k-runtime.jar}}<br />
** {{Code|xercesImpl.jar}}<br />
** {{Code|xml-apis.jar}}<br />
* '''Saxon Enterprise Edition''' will be used if you download the [https://www.saxonica.com/download/java.xml ZIP release] and if you copy {{Code|saxon9ee.jar}} and a valid license key to the classpath.<br />
<br />
==validate:xsd==<br />
<br />
{{Announce|Updated with Version 11:}} caching feature added.<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>validate:xsd(<br />
$input as item(),<br />
$schema as item()? := (),<br />
$options as map(*)? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Validates the XML {{Code|$input}} document against a {{Code|$schema}}. The supported {{Code|$options}} are:<br />
* {{Code|cache}}: Generated schemas are cached and reused.<br />
* All other options are handled as features and passed on to the validator.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|error|#Errors}} the validation fails.<br/>{{Error|init|#Errors}} the validation process cannot be started.<br/>{{Error|not-found|#Errors}} no XML Schema validator is available.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* Pass on document and schema as nodes:<br />
<pre lang='xquery'><br />
let $doc := <simple:root xmlns:simple='http://basex.org/simple'/><br />
let $schema :=<br />
<xs:schema xmlns:xs='http://www.w3.org/2001/XMLSchema' targetNamespace='http://basex.org/simple'><br />
<xs:element name='root'/><br />
</xs:schema><br />
return validate:xsd($doc, $schema)<br />
</pre><br />
* Validate all documents of a database against the specified schema, using the supplied feature:<br />
<pre lang='xquery'><br />
for $city in db:get('cities')<br />
return validate:xsd($city, 'city.xsd',<br />
map { 'http://javax.xml.XMLConstants/feature/secure-processing': true() }<br />
)<br />
</pre><br />
|}<br />
<br />
==validate:xsd-info==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>validate:xsd-info(<br />
$input as item(),<br />
$schema as item()? := (),<br />
$options as map(*)? := map { }<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Validates the XML {{Code|$input}} document against a {{Code|$schema}} and returns warnings, errors and fatal errors in a string sequence. See {{Function||validate:xsd}} for more information of the {{Code|$options}} parameter.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|init|#Errors}} the validation process cannot be started.<br/>{{Error|not-found|#Errors}} no XML Schema validator is available.<br />
|}<br />
<br />
==validate:xsd-report==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>validate:xsd-report(<br />
$input as item(),<br />
$schema as xs:string? := (),<br />
$options as map(*) := map { }<br />
) as element(report)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Validates the XML {{Code|$input}} document against a {{Code|$schema}} and returns warnings, errors and fatal errors as XML. See {{Function||validate:xsd}} for more information of the {{Code|$options}} parameter.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|init|#Errors}} the validation process cannot be started.<br/>{{Error|not-found|#Errors}} no XML Schema validator is available.<br />
|}<br />
<br />
==validate:xsd-processor==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>validate:xsd-processor() as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the name of the applied XSD processor.<br />
|}<br />
<br />
==validate:xsd-version==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>validate:xsd-version() as xs:string</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the supported version of XSD Schema.<br />
|}<br />
<br />
=RelaxNG Validation=<br />
<br />
Checks whether an XML document validates against a RelaxNG schema. The input document and the schema can be specified as:<br />
<br />
* {{Code|xs:string}}, containing the path to the resource,<br />
* {{Code|xs:string}}, containing the resource in its string representation, or<br />
* {{Code|node()}}, containing the resource itself.<br />
<br />
RelaxNG validation will be available if [https://relaxng.org/jclark/jing.html Jing] exists in the classpath. The latest version, {{Code|jing-20091111.jar}}, is included in the full distributions of BaseX. As Jing additionally supports [http://www.nvdl.org/ NVDL] validation, you can also use the functions to validate the input against NVDL schemas.<br />
<br />
==validate:rng==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>validate:rng(<br />
$input as item(),<br />
$schema as item(),<br />
$compact as xs:boolean? := map { }<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Validates the XML {{Code|$input}} document against a {{Code|$schema}}, using the XML or {{Code|$compact}} notation.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|error|#Errors}} the validation fails.<br/>{{Error|init|#Errors}} the validation process cannot be started.<br/>{{Error|not-found|#Errors}} the RelaxNG validator is not available.<br />
|- valign="top"<br />
| '''Examples'''<br />
|<br />
* {{Code|validate:rng('doc.xml', 'doc.rng')}} validates the document {{Code|doc.xml}} against the specified schema {{Code|doc.rng}}.<br />
|}<br />
<br />
==validate:rng-info==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>validate:rng-info(<br />
$input as item(),<br />
$schema as item(),<br />
$compact as xs:boolean? := map { }<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Validates the XML {{Code|$input}} document against a {{Code|$schema}}, using the XML or {{Code|$compact}} notation, and returns warnings, errors and fatal errors in a string sequence.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|init|#Errors}} the validation process cannot be started.<br/>{{Error|not-found|#Errors}} the RelaxNG validator is not available.<br />
|}<br />
<br />
==validate:rng-report==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>validate:rng-report(<br />
$input as item(),<br />
$schema as xs:string,<br />
$compact as xs:boolean? := map { }<br />
) as element(report)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Validates the XML {{Code|$input}} document against a {{Code|$schema}}, using the XML or {{Code|$compact}} notation, and returns warnings, errors and fatal errors as XML.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|init|#Errors}} the validation process cannot be started.<br/>{{Error|not-found|#Errors}} The RelaxNG validator is not available.<br />
|}<br />
<br />
=Schematron Validation=<br />
<br />
If you want to use Schematron for validating documents, install Vincent Lizzi’s excellent [https://github.com/Schematron/schematron-basex Schematron XQuery Module for BaseX]:<br />
<br />
<pre lang='xquery'><br />
repo:install('https://github.com/Schematron/schematron-basex/raw/master/dist/schematron-basex-1.2.xar')<br />
</pre><br />
<br />
The following query illustrates how documents are validated. It is directly taken from the GitHub project:<br />
<br />
<pre lang='xquery'><br />
import module namespace schematron = "http://github.com/Schematron/schematron-basex";<br />
<br />
let $sch := schematron:compile(doc('rules.sch'))<br />
let $svrl := schematron:validate(doc('document.xml'), $sch)<br />
return (<br />
schematron:is-valid($svrl),<br />
for $message in schematron:messages($svrl)<br />
return concat(schematron:message-level($message), ': ', schematron:message-description($message))<br />
)<br />
</pre><br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
|Description<br />
|- valign="top"<br />
|{{Code|error}}<br />
|The document cannot be validated against the specified schema.<br />
|- valign="top"<br />
|{{Code|init}}<br />
|The validation cannot be started.<br />
|- valign="top"<br />
|{{Code|not-found}}<br />
|No validator is available.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 11.0<br />
<br />
* Updated: {{Function||validate:xsd}}, {{Function||validate:xsd-info}}, {{Function||validate:xsd-report}}: caching feature added<br />
<br />
;Version 9.2<br />
<br />
* Added: {{Function||validate:xsd-processor}}, {{Function||validate:xsd-version}}<br />
* Updated: {{Function||validate:xsd}}, {{Function||validate:xsd-info}}, {{Function||validate:xsd-report}}: version argument was dropped (the latest version will always be used)<br />
<br />
;Version 9.0<br />
<br />
* Updated: error codes updated; errors now use the module namespace<br />
<br />
;Version 8.5<br />
<br />
* Updated: Relative URIs will always be resolved against the static base URI of the query<br />
<br />
;Version 8.3<br />
<br />
* Added: {{Function||validate:rng}}, {{Function||validate:rng-info}}<br />
* Added: {{Function||validate:dtd-report|dtd-report}}, {{Function||validate:xsd-report}}, {{Function||validate:rng-report}}<br />
<br />
;Version 7.6<br />
<br />
* Added: {{Function||validate:xsd-info}}, {{Function||validate:dtd-info}}<br />
<br />
The module was introduced with Version 7.3.</div>CGhttps://docs.basex.org/index.php?title=WebSockets&diff=16882WebSockets2023-12-01T16:38:55Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This page presents one of the [[Web Application]] services. It describes how to use the WebSockets API of BaseX. WebSocket is a communication protocol for providing '''full-duplex''' communication: Data can be sent in both directions and simultaneously.<br />
<br />
Please note that the current WebSocket implementation relies on Jetty’s WebSocket servlet API. Other web servers may be supported in future versions.<br />
<br />
=Introduction=<br />
<br />
==Protocol==<br />
<br />
Use WebSockets if you have to exchange data with a high frequency or if you have to send messages from the server to the client without techniques like [https://en.wikipedia.org/wiki/Polling_(computer_science) polling]. In contrast to REST, WebSockets use a single URL for the whole communication. <br />
<br />
The WebSocket protocol was standardized in [https://tools.ietf.org/html/rfc6455 RFC 6455] by the IETF. After an initial HTTP request, all communication takes place over a single TCP connection. Unlike the HTTP protocol, a connection will be kept alive, and a server can send unsolicited data to the client.<br />
<br />
For establishing a WebSocket connection, a handshake request is sent by the client. The web server returns a handshake response. If the handshake is successful, the persistent connection will be open until the client or the server closes it, an error occurs or a timeout happens. It is possible to transmit all kind of data, binary or text. '''The BaseX WebServer handles the handshake completely.''' You just have to define some limits of the connection in the <code>web.xml</code> and specify functions for WebSocket events like ''onConnect'' and ''onMessage''.<br />
<br />
Notice that there is no specification of a message protocol. The WebSocket protocol just specifies the message architecture but not how the payload of the messages is formatted. To agree on a format between the server and the client one can use sub-protocols.<br />
<br />
Some older browsers don’t support the WebSocket protocol. Therefore you can use fallback options like Ajax. JavaScript client libraries like SockJS can be used for building client applications. The library takes care of how to establish the real-time connection. If the WebSocket protocol isn’t supported, it uses polling. You have to provide server functions for the fallback solutions if you have to support fallbacks. <br />
<br />
==Preliminaries==<br />
<br />
There are a bunch of annotations depending to WebSockets for annotating XQuery functions. When a WebSocket message arrives at the server, an XQuery function will be invoked that matches the constraints indicated by its annotations.<br />
<br />
If a WebSocket function is requested (like connecting to the path <code>/</code>, sending a message to the path <code>/path</code>, …), the module directory and its subdirectories will be traversed, and all [[XQuery Extensions#Suffixes|XQuery files]] will be parsed for functions with WebSocket annotations. Subdirectories that include an {{Code|.ignore}} file will be skipped.<br />
<br />
To speed up processing, the functions of the existing XQuery modules are automatically cached in main memory. For further information on cache handling, check out the [[RESTXQ#Introduction|RESTXQ introduction]].<br />
<br />
==Configuration==<br />
<br />
* The WebSocket servlet can be enabled and disabled in the <code>web.xml</code> configuration file. You can specify further configuration options, such as <code>maxIdleTime</code>, <code>maxTextMessageSize</code>, and <code>maxBinaryMessageSize</code>.<br />
* The default limit for messges is 64 KB. If you a message exceeds the default or the specified limit, an error will be raised and the connection will be closed.<br />
<br />
=Annotations=<br />
<br />
To tag functions as WebSocket functions you have to use [[XQuery 3.0#Annotations|annotations]]. The annotation is written after the keyword ''declare'' and before the keyword ''function''. For the context of WebSockets there are some annotations listed below. Functions which are annotated with a WebSocket annotation will be called if the appropriate event occurs. For example, the function annotated with <code>ws:connect('/')</code> will be executed if a client establishes a connection with the WebSocket root path (which is, by default, <code>ws/</code>). By using annotations, it’s easy to provide an API for your WebSocket connection. You just have to specify what to do when a WebSocket Event occurs, annotate it with the corresponding annotation and the Servlet will do the rest for you. <br />
<br />
==ws:connect(path)==<br />
<br />
Called directly after a successful WebSocket handshake. The <code>path</code> specifies the path which a client is connected to:<br />
<br />
<pre lang='xquery'><br />
declare %ws:connect('/') function local:connect() { };<br />
</pre><br />
<br />
You can specify here how to handle your users, e. g. save a name as a WebSocket attribute. Furthermore, you can check header parameters for validity. <br />
<br />
==ws:message(path, message)==<br />
<br />
Called when a client message arrives at the server. The <code>path</code> specifies the path which a client is connected to. The <code>message</code> string contains the name of the variable to which the message will be bound:<br />
<br />
<pre lang='xquery'><br />
declare %ws:message('/', '{$info}') function local:message($info) { };<br />
</pre><br />
<br />
The value will be of type <code>xs:string</code> or <code>xs:base64Binary</code>. As there is no fixed message protocol, the client needs to take care of the message syntax.<br />
<br />
==ws:error(path, message)==<br />
<br />
Called when an error occurs. The <code>path</code> specifies the path which a client is connected to. The <code>message</code> string contains the name of the variable to which the message will be bound:<br />
<br />
<pre lang='xquery'><br />
declare %ws:error('/', '{$error}') function local:error($error) { };<br />
</pre><br />
<br />
Usually, errors happen because of bad/malformed incoming packets. The WebSocket connection gets closed after the error handling.<br />
<br />
==ws:close(path)==<br />
<br />
Called when the WebSocket closes. The <code>path</code> specifies the path which a client is connected to:<br />
<br />
<pre lang='xquery'><br />
declare %ws:close('/') function local:connect() { };<br />
</pre><br />
<br />
The WebSocket is already closed when this annotation is called so there can be no return. <br />
<br />
==ws:header-param(name, variable[, default])==<br />
<br />
For accessing connection-specific properties like the HTTP version. The value will be bound to the specified <code>variable</code>. If the property has no value, an optional <code>default</code> value will be assigned instead:<br />
<br />
<pre lang='xquery'><br />
declare<br />
%ws:close('host', '{$host}')<br />
%ws:header-param('host', '{$host}')<br />
function local:close($host) {<br />
admin:write-log('Connection was closed: ' || $host)<br />
};<br />
</pre><br />
<br />
The following parameters are available:<br />
<br />
{| class="wikitable" <br />
|- valign="top"<br />
! Name<br />
! Description<br />
|- valign="top"<br />
| <code>host</code><br />
| The host of the request URI.<br />
|- valign="top"<br />
| <code>http-version</code><br />
| The HTTP version used for the request.<br />
|- valign="top"<br />
| <code>is-secure</code><br />
| Indicates if the connection is secure.<br />
|- valign="top"<br />
| <code>origin</code><br />
| The WebSocket origin.<br />
|- valign="top"<br />
| <code>protocol-version</code><br />
| The version of the used protocol.<br />
|- valign="top"<br />
| <code>query-string</code><br />
| The query string of the request URI.<br />
|- valign="top"<br />
| <code>request-uri</code><br />
| The Request URI to use for this request.<br />
|- valign="top"<br />
| <code>sub-protocols</code><br />
| List of configured sub-protocols.<br />
|}<br />
<br />
General information on the request can be retrieved via the [[Request Module]].<br />
<br />
=Writing Applications=<br />
<br />
The [[WebSocket Module]] contains functions for interacting with other clients or manage specific clients. For example, you can store and access client-specific properties for a WebSocket connection or close the connection of clients.<br />
<br />
Note that one WebSocket connection can be opened per browser tab. In contrast, only one HTTP session exists for multiple tabs in in a browser. If you want to keep client-specific data on the web server, you can either store them in HTTP sessions or in the WebSocket connection.<br />
<br />
Note further that the results of functions annotated with <code>%ws:close</code> or <code>%ws:error</code> will not be transmitted to the client. Both annotations have rather been designed to gracefully close connections, write log data, remove clients from session data, etc.<br />
<br />
For keeping the connection alive it is recommendable to use heart-beats, and send regular pings to the server. There is no ideal timespan for sending pings: It should not be sent too often, but you should also consider possible network latencies.<br />
<br />
If your HTTP connection is secure, you should use the <code>wss</code> instead of the <code>ws</code> scheme.<br />
<br />
If you get the <code>[basex:ws] WebSocket connection required</code> error, you may be attempting to call WebSocket functions from a non-WebSocket context. If you use a proxy server, check in the configuration if WebSockets are enabled.<br />
<br />
=Examples=<br />
<br />
==Basic Example==<br />
<br />
The following chapter explains how to create a simple basic web application with WebSockets. You can find another example in the BaseX source code. <br />
<br />
First of all, you have to ensure that the <code>WsServlet</code> is enabled in your <code>web.xml</code> file. It will be enabled if you use the standard configuration of BaseX.<br />
<br />
For establishing a connection to the WebSocket server, it is necessary that the server provides at least one function annotated with a WebSocket annotation. Let’s start by using the annotation <code>%ws:connect('/')</code>. In the connect function, a bidirectional communication with the client can be initialized: attributes such as the id and name of a client can be set, or a welcome message can be emitted to other connected users, and so on.<br />
<br />
<pre lang='xquery'><br />
declare<br />
%ws:connect('/')<br />
function example:connect() as empty-sequence() {<br />
};<br />
</pre><br />
<br />
The connect function is sufficient for creating the persistent client/server connection. In order to something sensible with the connection, you should implement a function annotated with <code>%ws:message("/")</code>:<br />
<br />
<pre lang='xquery'><br />
import module namespace ws = 'http://basex.org/modules/ws';<br />
<br />
declare<br />
%ws:message('/', '{$message}')<br />
function example:message(<br />
$message as xs:string<br />
) as empty-sequence() {<br />
ws:emit($message)<br />
};<br />
</pre><br />
<br />
In the function above, the [[WebSocket Module]] is imported, and the function <code>ws:emit</code> is used for forwarding the message to all connected clients.<br />
<br />
The following client-side code demonstrates a basic application of the WebSocket connection:<br />
<br />
<pre lang="javascript"><br />
var ws = new WebSocket("ws://localhost:8080/ws");<br />
<br />
ws.onmessage = function(event) {<br />
alert(event.data);<br />
};<br />
<br />
function send(message) {<br />
ws.send(message);<br />
};<br />
</pre><br />
<br />
The <code>send</code> function can be called to pass on a string to the server.<br />
<br />
There are no heart-beats in this example. This means that the connection is terminated if nothing happens for 5 minutes (standard timeout). It will also be closed if you send a message that exceeds the standard text size.<br />
<br />
==Chat Application==<br />
<br />
In the full distributions of BaseX, you will find a little self-contained chat application that demonstrates how WebSockets can be used in practice.<br />
<br />
=Changelog=<br />
<br />
WebSockets werre introduced with Version 9.1.</div>CGhttps://docs.basex.org/index.php?title=XQuery_3.1&diff=16881XQuery 3.12023-12-01T16:38:54Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This article is part of the [[XQuery|XQuery Portal]]. It provides a summary of the most important features of the [https://www.w3.org/TR/xquery-31/ XQuery 3.1] Recommendation.<br />
<br />
=Maps=<br />
<br />
A ''map'' is a function that associates a set of keys with values, resulting in a collection of key/value pairs. Each key/value pair in a map is called an entry. A key is an arbitrary atomic value, and the associated value is an arbitrary sequence. Within a map, no two entries have the same key, when compared using the {{Code|eq}} operator. It is not necessary that all the keys should be mutually comparable (for example, they can include a mixture of integers and strings).<br />
<br />
Maps can be constructed as follows:<br />
<br />
<pre lang='xquery'><br />
map { }, (: empty map :)<br />
map { 'key': true(), 1984: (<a/>, <b/>) }, (: map with two entries :)<br />
map:merge( (: map with ten entries :)<br />
for $i in 1 to 10<br />
return map { $i: 'value' || $i }<br />
)<br />
</pre><br />
<br />
The function corresponding to the map has the signature {{Code|function($key as xs:anyAtomicType) as item()*}}. The expression {{Code|$map($key)}} returns the associated value; the function call {{Code|map:get($map, $key)}} is equivalent. For example, if {{Code|$books-by-isbn}} is a map whose keys are ISBNs and whose associated values are {{Code|book}} elements, then the expression {{Code|$books-by-isbn("0470192747")}} returns the {{Code|book}} element with the given ISBN. The fact that a map is a function item allows it to be passed as an argument to [[Higher-Order Functions]] that expect a function item as one of their arguments. As an example, the following query uses the higher-order function {{Code|fn:map($f, $seq)}} to extract all bound values from a map:<br />
<br />
<pre lang='xquery'><br />
let $map := map { 'foo': 42, 'bar': 'baz', 123: 456 }<br />
return fn:for-each(map:keys($map), $map)<br />
</pre><br />
<br />
This returns some permutation of {{Code|(42, 'baz', 456)}}.<br />
<br />
Because a map is a function item, functions that apply to functions also apply to maps. A map is an anonymous function, so {{Code|fn:function-name}} returns the empty sequence; {{Code|fn:function-arity}} always returns {{Code|1}}.<br />
<br />
Like all other values, maps are immutable. For example, the {{Function|Map|map:remove}} function creates a new map by removing an entry from an existing map, but the existing map is not changed by the operation. Like sequences, maps have no identity. It is meaningful to compare the contents of two maps, but there is no way of asking whether they are "the same map": two maps with the same content are indistinguishable.<br />
<br />
Maps may be compared using the {{Code|fn:deep-equal}} function. The [[Map Module]] describes the available set of map functions.<br />
<br />
=Arrays=<br />
<br />
An ''array'' is a function that associates a set of positions, represented as positive integer keys, with values. The first position in an array is associated with the integer {{Code|1}}. The values of an array are called its members. In the type hierarchy, array has a distinct type, which is derived from function. In BaseX, arrays (as well as sequences) are based on an efficient [https://en.wikipedia.org/wiki/Finger_tree Finger Tree] implementation.<br />
<br />
Arrays can be constructed in two ways. With the square bracket notation, the comma serves as delimiter:<br />
<br />
<pre lang='xquery'><br />
[], (: empty array :)<br />
[ (1, 2) ], (: array with single member :)<br />
[ 1 to 2, 3 ] (: array with two members; same as: [ (1, 2), 3 ] :)<br />
</pre><br />
<br />
With the {{Code|array}} keyword and curly brackets, the inner expression is evaluated as usual, and the resulting values will be the members of the array:<br />
<br />
<pre lang='xquery'><br />
array { }, (: empty array; same as: array { () } :) <br />
array { (1, 2) }, (: array with two members; same as: array { 1, 2 } :)<br />
array { 1 to 2, 3 } (: array with three members; same as: array { 1, 2, 3 } :)<br />
</pre><br />
<br />
The function corresponding to the array has the signature {{Code|function($index as xs:integer) as item()*}}. The expression {{Code|$array($index)}} returns an addressed member of the array. The following query returns the five array members {{Code|48 49 50 51 52}} as result:<br />
<br />
<pre lang='xquery'><br />
let $array := array { 48 to 52 }<br />
for $i in 1 to array:size($array)<br />
return $array($i)<br />
</pre><br />
<br />
Like all other values, arrays are immutable. For example, the {{Function|Array|array:reverse}} function creates a new array containing a re-ordering of the members of an existing array, but the existing array is not changed by the operation. Like sequences, arrays have no identity. It is meaningful to compare the contents of two arrays, but there is no way of asking whether they are "the same array": two arrays with the same content are indistinguishable.<br />
<br />
==Atomization==<br />
<br />
If an array is ''atomized'', all of its members will be atomized. As a result, an atomized item may now result in more than one item. Some examples:<br />
<br />
<pre lang='xquery'><br />
fn:data([1 to 2]) (: returns the sequence 1, 2 :)<br />
[ 'a', 'b', 'c' ] = 'b' (: returns true :)<br />
<a>{ [ 1, 2 ] }</a> (: returns <a>1 2</a> :)<br />
array { 1 to 2 } + 3 (: error: the left operand returns two items :)<br />
</pre><br />
<br />
Atomization also applies to function arguments. The following query returns 5, because the array will be atomized to a sequence of 5 integers:<br />
<br />
<pre lang='xquery'><br />
let $f := function($x as xs:integer*) { count($x) }<br />
return $f([1 to 5])<br />
</pre><br />
<br />
However, the next query returns 1, because the array is already of the general type {{Code|item()}}, and no atomization will take place:<br />
<br />
<pre lang='xquery'><br />
let $f := function($x as item()*) { count($x) }<br />
return $f([1 to 5])<br />
</pre><br />
<br />
Arrays can be compared with the {{Code|fn:deep-equal}} function. The [[Array Module]] describes the available set of array functions.<br />
<br />
=Lookup Operator=<br />
<br />
The lookup operator provides some syntactic sugar to access values of maps or array members. It is introduced by the question mark ({{Code|?}}) and followed by a specifier. The specifier can be:<br />
<br />
# A wildcard {{Code|*}},<br />
# The name of the key,<br />
# The integer offset, or<br />
# Any other parenthesized expression.<br />
<br />
The following example demonstrates the four alternatives:<br />
<br />
<pre lang='xquery'><br />
let $map := map { 'R': 'red', 'G': 'green', 'B': 'blue' }<br />
return (<br />
$map?* (: 1. returns all values; same as: map:keys($map) ! $map(.) :),<br />
$map?R (: 2. returns the value associated with the key 'R'; same as: $map('R') :),<br />
$map?('G','B') (: 3. returns the values associated with the key 'G' and 'B' :)<br />
),<br />
<br />
let $array := [ 'one', 'two', 'three' ]<br />
return (<br />
$array?* (: 1. returns all values; same as: (1 to array:size($array)) ! $array(.) :),<br />
$array?1 (: 2. returns the first value; same as: $array(1) :),<br />
$array?(2 to 3) (: 3. returns the second and third values; same as: (1 to 2) ! $array(.) :)<br />
)<br />
</pre><br />
<br />
The lookup operator can also be used without left operand. In this case, the context item will be used as input. This query returns {{Code|Akureyri}}:<br />
<br />
<pre lang='xquery'><br />
let $maps := (<br />
map { 'name': 'Guðrún', 'city': 'Reykjavík' },<br />
map { 'name': 'Hildur', 'city': 'Akureyri' }<br />
)<br />
return $maps[?name = 'Hildur'] ?city<br />
</pre><br />
<br />
=Arrow Operator=<br />
<br />
The arrow operator <code>=></code> provides a convenient alternative syntax for passing on functions to a value. The expression that precedes the operator will be supplied as first argument of the function that follows the arrow. If <code>$v</code> is a value and <code>f()</code> is a function, then <code>$v => f()</code> is equivalent to <code>f($v)</code>, and <code>$v => f($j)</code> is equivalent to <code>f($v, $j)</code>:<br />
<br />
<pre lang='xquery'><br />
(: Returns 3 :)<br />
count(('A', 'B', 'C')),<br />
('A', 'B', 'C') => count(),<br />
('A', 'B', 'C') => (function( $sequence) { count( $sequence)})(),<br />
<br />
(: Returns W-E-L-C-O-M-E :)<br />
string-join(tokenize(upper-case('w e l c o m e')), '-'),<br />
'w e l c o m e' => upper-case() => tokenize() => string-join('-'),<br />
<br />
(: Returns xfmdpnf :)<br />
codepoints-to-string(<br />
for $i in string-to-codepoints('welcome')<br />
return $i + 1<br />
),<br />
(for $i in 'welcome' => string-to-codepoints()<br />
return $i + 1) => codepoints-to-string()<br />
</pre><br />
<br />
The syntax makes nested function calls more readable, as it is easy to see if parentheses are balanced.<br />
<br />
=String Constructor=<br />
<br />
The string constructor has been inspired by [https://en.wikipedia.org/wiki/Here_document here document] literals of the Unix shell and script languages. It allows you to generate strings that contain various characters that would otherwise be interpreted as XQuery delimiters.<br />
<br />
The string constructors syntax uses two backticks and a square bracket for opening and closing a string:<br />
<br />
<pre lang='xquery'><br />
(: Returns "This is a 'new' & 'flexible' syntax." :)<br />
``["This is a 'new' & 'flexible' syntax."]``<br />
</pre><br />
<br />
XQuery expressions can be embedded via backticks and a curly bracket. The evaluated results will be separated with spaces, and all strings will eventually be concatenated:<br />
<br />
<pre lang='xquery'><br />
(: Returns »Count 1 2 3, and I will be there.« :)<br />
let $c := 1 to 3<br />
return ``[»Count `{ $c }`, and I will be there.«]``</pre><br />
<br />
=Serialization=<br />
<br />
Two [[Serialization]] methods have been added to the [https://www.w3.org/TR/xslt-xquery-serialization-31 Serialization spec]:<br />
<br />
==Adaptive Serialization==<br />
<br />
The {{Code|adaptive}} serialization provides an intuitive textual representation for all XDM types, including maps and arrays, functions, attributes, and namespaces. All items will be separated by the value of the {{Code|item-separator}} parameter, which by default is a newline character. It is utilized by the functions {{Function|Profiling|prof:dump}} and <code>[https://www.w3.org/TR/xpath-functions-31/#func-trace fn:trace]</code>.<br />
<br />
Example:<br />
<br />
<pre lang='xquery'><br />
declare option output:method 'adaptive';<br />
<element id='id0'/>/@id,<br />
xs:token("abc"),<br />
map { 'key': 'value' },<br />
true#0<br />
</pre><br />
<br />
Result:<br />
<br />
<pre lang="xml"><br />
id="id0"<br />
xs:token("abc"),<br />
map {<br />
"key": "value"<br />
}<br />
fn:true#0<br />
</pre><br />
<br />
==JSON Serialization==<br />
<br />
The new {{Code|json}} serialization output method can be used to serialize XQuery maps, arrays, atomic values and empty sequences as JSON.<br />
<br />
The {{Code|json}} output method has been introduced in BaseX before it was added to the official specification. It complies with the standard serialization rules and, at the same time, preserves the existing semantics:<br />
<br />
* If an XML node of type {{Code|element(json)}} is found, it will be serialized following the serialization rules of the [[JSON Module]].<br />
* Any other node or atomic value, map, array, or empty sequence will be serialized according to the [https://www.w3.org/TR/xslt-xquery-serialization-31/#json-output rules in the specification].<br />
<br />
The following two queries will both return the JSON snippet <code>{ "key": "value" }</code>:<br />
<br />
<pre lang='xquery'><br />
declare option output:method 'json';<br />
map { "key": "value" }<br />
</pre><br />
<br />
<pre lang='xquery'><br />
declare option output:method 'json';<br />
<json type='object'><br />
<key>value</key><br />
</json><br />
</pre><br />
<br />
=Functions=<br />
<br />
The following functions have been added in the [https://www.w3.org/TR/xpath-functions-31/ XQuery 3.1 Functions and Operators] Specification:<br />
<br />
==Map Functions==<br />
<br />
<code>map:merge</code>, <code>map:size</code>, <code>map:keys</code>, <code>map:contains</code>, <code>map:get</code>, <code>map:entry</code>, <code>map:put</code>, <code>map:remove</code>, <code>map:for-each</code><br />
<br />
Please check out the [[Map Module]] for more details.<br />
<br />
==Array Functions==<br />
<br />
<code>array:size</code>, <code>array:append</code>, <code>array:subarray</code>, <code>array:remove</code>, <code>array:insert-before</code>, <code>array:head</code>, <code>array:tail</code>, <code>array:reverse</code>, <code>array:join</code>, <code>array:flatten</code>, <code>array:for-each</code>, <code>array:filter</code>, <code>array:fold-left</code>, <code>array:fold-right</code>, <code>array:for-each-pair</code><br />
<br />
Please check out the [[Array Module]] for more details.<br />
<br />
==JSON Functions==<br />
<br />
With XQuery 3.1, native support for JSON objects was added. Strings and resources can be parsed to XQuery items and, as [[#JSON Serialization|shown above]], serialized back to their original form.<br />
<br />
===fn:parse-json===<br />
<br />
<pre>fn:parse-json(<br />
$json as xs:string?<br />
$options as map(*) := ()<br />
) as item()?</pre><br />
<br />
Parses the supplied string as JSON text and returns its item representation. The result may be a map, an array, a string, a double, a boolean, or an empty sequence. The allowed options can be looked up in the [https://www.w3.org/TR/xpath-functions-31/#func-parse-json specification].<br />
<br />
<pre lang='xquery'><br />
parse-json('{ "name": "john" }') (: yields { "name": "json" } :),<br />
parse-json('[ 1, 2, 4, 8, 16]') (: yields [ 1, 2, 4, 8, 16 ] :)<br />
</pre><br />
<br />
===fn:json-doc===<br />
<br />
<pre>fn:json-doc(<br />
$href as xs:string?<br />
$options as map(*) := ()<br />
) as item()?</pre><br />
<br />
Retrieves the text from the specified URI, parses the supplied string as JSON text and returns its item representation (see {{Function||fn:parse-json}} for more details).<br />
<br />
<pre lang='xquery'><br />
json-doc("http://ip.jsontest.com/")('ip') (: returns your IP address :)<br />
</pre><br />
<br />
===fn:json-to-xml===<br />
<br />
<pre>fn:json-to-xml(<br />
$json as xs:string?<br />
$options as map(*) := ()<br />
) as document-node()?</pre><br />
<br />
Converts a JSON string to an XML node representation. The allowed options can be looked up in the [https://www.w3.org/TR/xpath-functions-31/#func-json-to-xm specification].<br />
<br />
<pre lang='xquery'><br />
json-to-xml('{ "message": "world" }')<br />
<br />
(: result:<br />
<map xmlns="http://www.w3.org/2005/xpath-functions"><br />
<string key="message">world</string><br />
</map> :)<br />
</pre><br />
<br />
===fn:xml-to-json===<br />
<br />
<pre>fn:xml-to-json(<br />
$node as xs:string?<br />
$options as map(*) := ()<br />
) as xs:string?</pre><br />
<br />
Converts an XML node, whose format conforms to the results created by {{Function||fn:json-to-xml}}, to a JSON string representation. The allowed options can be looked up in the [https://www.w3.org/TR/xpath-functions-31/#func-xml-to-json specification].<br />
<br />
<pre lang='xquery'><br />
(: returns "JSON" :)<br />
xml-to-json(<string xmlns="http://www.w3.org/2005/xpath-functions">JSON</string>)<br />
</pre><br />
<br />
==fn:sort==<br />
<br />
<pre>fn:sort(<br />
$input as item()*,<br />
$collation as xs:string? := fn:default-collation(),<br />
$key as function(item()) as xs:anyAtomicType* := fn:data#1<br />
) as item()*</pre><br />
<br />
Returns a new sequence with sorted {{Code|$input}} items, using an optional {{Code|$collation}}. If a {{Code|$key}} function is supplied, it will be applied on all items. The items of the resulting values will be sorted using the semantics of the {{Code|lt}} expression.<br />
<br />
<pre lang='xquery'><br />
sort(reverse(1 to 3)) (: yields 1, 2, 3 :),<br />
reverse(sort(1 to 3)) (: returns the sorted order in descending order :),<br />
sort((3,-2,1), (), abs#1) (: yields 1, -2, 3 :),<br />
sort((1,2,3), (), function($x) { -$x }) (: yields 3, 2, 1 :),<br />
sort((1,'a')) (: yields an error, as strings and integers cannot be compared :)<br />
</pre><br />
<br />
==fn:contains-token==<br />
<br />
<pre>fn:contains-token(<br />
$value as xs:string*,<br />
$token as xs:string,<br />
$collation as xs:string? := fn:default-collation()<br />
) as xs:boolean</pre><br />
<br />
The supplied strings will be tokenized at whitespace boundaries. The function returns {{Code|true}} if one of the strings equals the supplied token, possibly under the rules of a supplied collation:<br />
<br />
<pre lang='xquery'><br />
contains-token(('a', 'b c', 'd'), 'c') (: yields true :)<br />
<xml class='one two'/>/contains-token(@class, 'one') (: yields true :)<br />
</pre><br />
<br />
==fn:parse-ietf-date==<br />
<br />
<pre>fn:parse-ietf-date(<br />
$value as xs:string?<br />
) as xs:dateTime?</pre><br />
<br />
Parses a string in the IETF format (which is widely used on the Internet) and returns a {{Code|xs:dateTime}} item:<br />
<br />
<pre lang='xquery'><br />
fn:parse-ietf-date('28-Feb-1984 07:07:07')" (: yields 1984-02-28T07:07:07Z :),<br />
fn:parse-ietf-date('Wed, 01 Jun 2001 23:45:54 +02:00')" (: yields 2001-06-01T23:45:54+02:00 :)<br />
</pre><br />
<br />
==fn:apply==<br />
<br />
<pre>fn:apply(<br />
$function as function(*),<br />
$arguments as array(*)<br />
) as item()*</pre><br />
<br />
The supplied {{Code|$function}} is invoked with the specified {{Code|$arguments}}. The arity of the function must be the same as the size of the array.<br />
<br />
Example:<br />
<br />
<pre lang='xquery'><br />
fn:apply(concat#5, array { 1 to 5 }) (: 12345 :)<br />
fn:apply(function($a) { sum($a) }, [ 1 to 5 ]) (: 15 :)<br />
fn:apply(count#1, [ 1,2 ]) (: error. the array has two members :)<br />
</pre><br />
<br />
==fn:random-number-generator==<br />
<br />
<pre>fn:random-number-generator(<br />
$seed as xs:anyAtomicType? := ()<br />
) as map(xs:string, item())</pre><br />
<br />
Creates a random number generator, using an optional seed. The returned map contains three entries:<br />
<br />
* {{Code|number}} is a random double between 0 and 1<br />
* {{Code|next}} is a function that returns another random number generator<br />
* {{Code|permute}} is a function that returns a random permutation of its argument<br />
<br />
The returned random generator is ''deterministic'': If the function is called twice with the same arguments and in the same execution scope, it will always return the same result.<br />
<br />
Example:<br />
<br />
<pre lang='xquery'><br />
let $rng := fn:random-number-generator()<br />
let $number := $rng('number') (: returns a random number :)<br />
let $next-rng := $rng('next')() (: returns a new generator :)<br />
let $next-number := $next-rng('number') (: returns another random number :)<br />
let $permutation := $rng('permute')(1 to 5) (: returns a random permutation of (1,2,3,4,5) :)<br />
return ($number, $next-number, $permutation)<br />
</pre><br />
<br />
==fn:format-number==<br />
<br />
The function has been extended to support scientific notation:<br />
<br />
<pre lang='xquery'><br />
format-number(1984.42, '00.0e0') (: yields 19.8e2 :)<br />
</pre><br />
<br />
==fn:tokenize==<br />
<br />
If no separator is specified as second argument, a string will be tokenized at whitespace boundaries:<br />
<br />
<pre lang='xquery'><br />
fn:tokenize(" a b c d") (: yields "a", "b", "c", "d" :)<br />
</pre><br />
<br />
==fn:trace==<br />
<br />
The second argument can now be omitted:<br />
<br />
<pre lang='xquery'><br />
fn:trace(<xml/>, "Node: ")/node() (: yields the debugging output "Node: <xml/>" :),<br />
fn:trace(<xml/>)/node() (: returns the debugging output "<xml/>" :)<br />
</pre><br />
<br />
==fn:string-join==<br />
<br />
The type of the first argument is now <code>xs:anyAtomicType*</code>, and all items will be implicitly cast to strings:<br />
<br />
<pre lang='xquery'><br />
fn:string-join(1 to 3) (: yields the string "123" :)<br />
</pre><br />
<br />
==fn:default-language==<br />
<br />
Returns the default language used for formatting numbers and dates. BaseX always returns {{Code|en}}.<br />
<br />
==Appendix==<br />
<br />
The three functions <code>fn:transform</code>, <code>fn:load-xquery-module</code> and <code>fn:collation-key</code> may be added in a future version of BaseX as their implementation might require the use of additional external libraries.<br />
<br />
=Binary Data=<br />
<br />
Items of type <code>xs:hexBinary</code> and <code>xs:base64Binary</code> can be compared against each other. The following queries all yield {{Code|true}}:<br />
<br />
<pre lang='xquery'><br />
xs:hexBinary('') < xs:hexBinary('bb'),<br />
xs:hexBinary('aa') < xs:hexBinary('bb'),<br />
max((xs:hexBinary('aa'), xs:hexBinary('bb'))) = xs:hexBinary('bb')<br />
</pre><br />
<br />
=Collations=<br />
<br />
XQuery 3.1 provides a default collation, which allows for a case-insensitive comparison of ASCII characters (<code>A-Z</code> = <code>a-z</code>). This query returns <code>true</code>:<br />
<br />
<pre lang='xquery'><br />
declare default collation 'http://www.w3.org/2005/xpath-functions/collation/html-ascii-case-insensitive';<br />
'HTML' = 'html'<br />
</pre><br />
<br />
If the [http://site.icu-project.org/download ICU Library] is downloaded and added to the classpath, the full [https://www.w3.org/TR/xpath-functions-31/#uca-collations Unicode Collation Algorithm] features become available in BaseX:<br />
<br />
<pre lang='xquery'><br />
(: returns 0 (both strings are compared as equal) :)<br />
compare('a-b', 'ab', 'http://www.w3.org/2013/collation/UCA?alternate=shifted')<br />
</pre><br />
<br />
=Enclosed Expressions=<br />
<br />
''Enclosed expression'' is the syntactical term for the expressions that are specified inside a function body, try/catch clauses, node constructors and some other expressions. In the following example expressions, its the empty sequence:<br />
<br />
<pre lang='xquery'><br />
declare function local:x() { () };<br />
try { () } catch * { () },<br />
element x { () },<br />
text { () }<br />
</pre><br />
<br />
With XQuery 3.1, the expression can be omitted. The following query is equivalent to the upper one:<br />
<br />
<pre lang='xquery'><br />
declare function local:x() { };<br />
try { } catch * { },<br />
element x { }<br />
text { }<br />
</pre><br />
<br />
=Changelog=<br />
<br />
;Version 8.6<br />
<br />
* Updated: Collation argument was inserted between first and second argument.<br />
<br />
;Version 8.4<br />
<br />
* Added: [[#String Constructors|String Constructors]], {{Code|fn:default-language}}, [[#Enclosed Expressions|Enclosed Expressions]]<br />
* Updated: [[#Adaptive Serialization|Adaptive Serialization]], {{Code|fn:string-join}}<br />
<br />
;Version 8.2<br />
<br />
* Added: {{Code|fn:json-to-xml}}, {{Code|fn:xml-to-json}}.<br />
<br />
;Version 8.1<br />
<br />
* Updated: arrays are now based on a [https://en.wikipedia.org/wiki/Finger_tree Finger Tree] implementation.<br />
<br />
Introduced with Version 8.0.</div>CGhttps://docs.basex.org/index.php?title=XQuery_Recipes&diff=16880XQuery Recipes2023-12-01T16:38:48Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This page contains code snippets that mainly originate from our [https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk basex-talk] mailing list.<br />
<br />
== Compact Notations ==<br />
<br />
<code>if</code>/<code>not</code>/<code>else</code> constructs can look pretty verbose in XQuery.<br />
However, some alternatives exist in order to make conditional code more compact:<br />
<br />
* The [[XQuery 3.0#Simple Map Operator|Simple Map Operator]] can be used to trigger an action if a value has a single item. The following two expressions are equivalent:<br />
<br />
<pre lang='xquery'><br />
let $s := "X" return (<br />
(: OLD :) if(count($s) = 1) then 'OK' else (),<br />
(: NEW :) $s ! 'OK'<br />
)<br />
</pre><br />
<br />
In some cases, also the first solution can be written more compact. If we know that our input will always have 0-1 items, we can write <code>if(exists($s))</code>. If our input will never be an empty string, a zero, etc, it’s sufficient to write <code>if($s)</code>.<br />
<br />
* If you want to choose the first non-empty item from two arguments, we can use the position predicate:<br />
<br />
<pre lang='xquery'><br />
let $s := "X" return (<br />
(: OLD :) if(exists($s)) then $s else 'default',<br />
(: NEW :) ($s, 'default')[1]<br />
)<br />
</pre><br />
<br />
Note that this only works if both of your inputs have zero or one items.<br />
<br />
== Computed Elements ==<br />
Returns dynamically named elements:<br />
<pre lang='xquery'><br />
let $root := "element"<br />
let $value := "hi"<br />
let $contents := <foo>Bar!</foo><br />
return element { $root } {<br />
attribute { "about" } { $value }, $contents<br />
}<br />
</pre><br />
<br />
The result is an XML fragment with <code>&lt;element&gt;</code> as root node:<br />
<pre lang="xml"><br />
<element about="hi"><br />
<foo>Bar!</foo><br />
</element><br />
</pre><br />
<br />
== Transform List to Tree ==<br />
This snippet transform a ''flat'' list of elements with <code>parentId</code>-references to a nested list.<br />
<br />
<pre lang='xquery'><br />
declare function local:link($entries as node()*, $id as xs:string) {<br />
let $entry := $entries[@id eq $id],<br />
$children := $entries[@parentId eq $id]<br />
return element entry {<br />
$entry/@*,<br />
for $child in $children<br />
return local:link($entries, $child/@id)<br />
}<br />
};<br />
<br />
let $entries :=<br />
<entries><br />
<entry id="entry1" /><br />
<entry id="entry2" parentId="entry1" /><br />
<entry id="entry3" parentId="entry1" /><br />
<entry id="entry4" parentId="entry2" /><br />
<entry id="entry5" parentId="entry2" /><br />
<entry id="entry6" parentId="entry3" /><br />
<entry id="entry7" parentId="entry3" /><br />
</entries><br />
return local:link($entries/entry, 'entry1')<br />
</pre><br />
results in<br />
<pre lang="xml"><br />
<entry id="entry1"><br />
<entry id="entry2" parentId="entry1"><br />
<entry id="entry4" parentId="entry2"/><br />
<entry id="entry5" parentId="entry2"/><br />
</entry><br />
<entry id="entry3" parentId="entry1"><br />
<entry id="entry6" parentId="entry3"/><br />
<entry id="entry7" parentId="entry3"/><br />
</entry><br />
</entry><br />
</pre><br />
<br />
== IP-Converter ==<br />
<br />
This snippet converts an IP address to its numeric representation:<br />
<br />
<pre lang='xquery'><br />
let $ip := '134.34.226.65'<br />
return fold-left(<br />
tokenize($ip, '\.')!xs:integer(.),<br />
0,<br />
function($n, $d) { 256 * $n + $d }<br />
)<br />
</pre><br />
results in<br />
<pre lang="xml"><br />
2250433089<br />
</pre><br />
<br />
== Count number of files ==<br />
<br />
This snippets returns the number of JPG files in a directory and its subdirectories:<br />
<br />
<pre lang='xquery'><br />
basex "count(file:list('.',true(),'*.jpg'))"</pre><br />
<br />
The Linux equivalent is<br />
<br />
<pre lang="xml"><br />
find . | grep \.jpg$ | wc -l<br />
</pre></div>CGhttps://docs.basex.org/index.php?title=Maven&diff=16879Maven2023-12-01T16:38:45Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This page is part of the [[Developer Section]]. It demonstrates how [https://maven.apache.org Maven] is used to compile and run BaseX, and embed it into other projects.<br />
<br />
==Using Maven==<br />
<br />
If you have [[Git|cloned our repository]] and installed Maven on your machine,<br />
you can run the following commands from all local repository directories:<br />
<br />
* <code>mvn compile</code>: the BaseX source files are compiled.<br />
* <code>mvn package</code>: JAR archives are created in the {{Code|target}} class directory, and all relevant libraries are created in the {{Code|lib}} directory. Packaging is useful if you want to use the start scripts.<br />
* <code>mvn install</code>: the JAR archive is installed to the local repository, and made available to other Maven projects. This is particularly useful if you are compiling a beta version of BaseX, for which no archives exist in the repositories.<br />
<br />
By adding the flag <code>-DskipTests</code> you can skip the JUnit tests and speed up packaging. You may as well use [[Developing with Eclipse|Eclipse and m2eclipse]] to compile the BaseX sources.<br />
<br />
There are several alternatives for starting BaseX:<br />
<br />
* type in {{Code|java -cp target/classes org.basex.BaseX}} in the {{Code|basex-core}} directory to start BaseX on the command-line mode,<br />
* type in {{Code|mvn jetty:run}} in the {{Code|basex-api}} directory to start BaseX with Jetty and the HTTP servers,<br />
* run one of the [[Start Scripts]] contained in the {{Code|etc}} directory<br />
<br />
==Artifacts==<br />
<br />
You can easily embed BaseX into your own Maven projects by adding the following XML snippets to your {{Code|pom.xml}} file:<br />
<br />
<pre lang="xml"><repositories><br />
<repository><br />
<id>basex</id><br />
<name>BaseX Maven Repository</name><br />
<url>http://files.basex.org/maven</url><br />
</repository><br />
</repositories></pre> <br />
<br />
===BaseX Main Package=== <br />
<br />
<pre lang="xml"><dependency><br />
<groupId>org.basex</groupId><br />
<artifactId>basex</artifactId><br />
<version>7.6</version><br />
</dependency></pre><br />
<br />
===APIs and Services=== <br />
<br />
<p>...including APIs and the [[REST]], [[RESTXQ]] and [[WebDAV]] services:</p> <br />
<pre lang="xml"><dependency><br />
<groupId>org.basex</groupId><br />
<artifactId>basex-api</artifactId><br />
<version>7.6</version><br />
</dependency></pre><br />
<br />
===XQJ API=== <br />
<br />
<p>The XQJ API is hosted at http://xqj.net:</p> <br />
<pre lang="xml"><br />
<repository><br />
<id>xqj</id><br />
<name>XQJ Maven Repository</name><br />
<url>http://xqj.net/maven</url><br />
</repository><br />
...<br />
<dependency><br />
<groupId>net.xqj</groupId><br />
<artifactId>basex-xqj</artifactId><br />
<version>1.2.0</version><br />
</dependency><br />
<dependency><br />
<groupId>com.xqj2</groupId><br />
<artifactId>xqj2</artifactId><br />
<version>0.1.0</version><br />
</dependency><br />
<dependency><br />
<groupId>javax.xml.xquery</groupId><br />
<artifactId>xqj-api</artifactId><br />
<version>1.0</version><br />
</dependency><br />
</pre></div>CGhttps://docs.basex.org/index.php?title=HTTP_Client_Module&diff=16878HTTP Client Module2023-12-01T16:38:42Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This [[Module Library|XQuery Module]] contains a single function to send HTTP requests and handle HTTP responses. The function {{Code|send-request}} is based on the [http://expath.org/spec/http-client EXPath HTTP Client Module]. It gives full control over the available request and response parameters. For simple GET requests, the [[Fetch Module]] may be sufficient.<br />
<br />
If <code><http:header name="Accept-Encoding" value="gzip"/></code> is specified and if the addressed web server provides support for the {{Code|gzip}} compression algorithm, the response will automatically be decompressed.<br />
<br />
Please note that BaseX provides extensions to the specification:<br />
<br />
* Since {{Announce|Version 11}}, {{Code|csv}}, {{Code|json}} and {{Code|html}} parser options can be supplied to influence the conversion of the response.<br />
<br />
Since BaseX 10, the module is based on the [https://openjdk.org/groups/net/httpclient/intro.html Java HTTP Client], which provides a better overall performance, uses internal connection pools and follows redirects across different protocols (http, https).<br />
<br />
=Conventions=<br />
<br />
All functions in this module are assigned to the <code><nowiki>http://expath.org/ns/http-client</nowiki></code> namespace, which is statically bound to the {{Code|http}} prefix.<br/><br />
All errors are assigned to the <code><nowiki>http://expath.org/ns/error</nowiki></code> namespace, which is statically bound to the {{Code|experr}} prefix.<br />
<br />
=Functions=<br />
<br />
==http:send-request==<br />
<br />
{{Announce|Updated with Version 11:}} {{Code|csv}}, {{Code|json}}, {{Code|html}} and {{Code|text}} attributes added.<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>http:send-request(<br />
$request as element(http:request)?,<br />
$href as xs:string? := (),<br />
$bodies as item()* := ()<br />
) as item()+</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Sends an HTTP request and interprets the corresponding response:<br />
* {{Code|$request}} contains an {{Code|<http:request/>}} element with a {{Code|method}} attribute, an {{Code|href}} attribute with the target URI, and optional header and body elements.<br />
* The request is either sent to the URI of the {{Code|$href}} argument or (if empty) to the URI supplied via the {{Code|href}} attribute.<br />
* In addition to the attributes of the official specification, {{Code|csv}}, {{Code|json}}, {{Code|html}} and {{Code|text}} attributes can be supplied to define how to convert the response body (see [[#Response Conversion|Response Conversion]] for an example).<br />
<br />
Notes:<br />
* Both basic and digest authentication is supported.<br />
* While the contents of the request can be supplied as child of the {{Code|http:body}} element, it is faster and safer to pass them on via the third argument.<br />
* Certificate verification can be globally disabled via the {{Option|IGNORECERT}} option.<br />
* For further information, please check out the [http://expath.org/spec/http-client EXPath] specification.<br />
|- valign="top"<br />
|'''Errors'''<br />
|{{Error|HC0001|#Errors}} an HTTP error occurred.<br/>{{Error|HC0002|#Errors}} error parsing the entity content as XML or HTML.<br/>{{Error|HC0003|#Errors}} with a multipart response, the override-media-type must be either a multipart media type or application/octet-stream.<br/>{{Error|HC0004|#Errors}} the src attribute on the body element is mutually exclusive with all other attribute (except the media-type).<br/>{{Error|HC0005|#Errors}} the request element is not valid.<br/>{{Error|HC0006|#Errors}} a timeout occurred waiting for the response.<br />
|}<br />
<br />
=Examples=<br />
<br />
==Status Only==<br />
<br />
Simple GET request. As the attribute {{Code|status-only}} is set to true, only the response element is returned.<br />
<br />
'''Query:'''<br />
<pre lang='xquery'>http:send-request(<http:request method='get' status-only='true'/>, 'http://basex.org')</pre><br />
'''Result:'''<br />
<pre lang="xml"><http:response status="200" message="OK"><br />
<http:header name="Date" value="Mon, 14 Mar 2011 20:55:53 GMT"/><br />
<http:header name="Content-Length" value="12671"/><br />
<http:header name="Expires" value="Mon, 14 Mar 2011 20:57:23 GMT"/><br />
<http:header name="Set-Cookie" value="fe_typo_user=d10c9552f9a784d1a73f8b6ebdf5ce63; path=/"/><br />
<http:header name="Connection" value="close"/><br />
<http:header name="Content-Type" value="text/html; charset=utf-8"/><br />
<http:header name="Server" value="Apache/2.2.16"/><br />
<http:header name="X-Powered-By" value="PHP/5.3.5"/><br />
<http:header name="Cache-Control" value="max-age=90"/><br />
<http:body media-type="text/html; charset=utf-8"/><br />
</http:response></pre><br />
<br />
==Google Homepage==<br />
<br />
Retrieve the Google search home page with a timeout of 10 seconds. In order to [[Parsers#HTMLParser|parse HTML]], TagSoup must be contained in the class path.<br />
<br />
'''Query:'''<br />
<pre lang='xquery'>http:send-request(<http:request method='get' href='http://www.google.com' timeout='10'/>)</pre><br />
'''Result:'''<br />
<pre lang="xml"><br />
<http:response status="200" message="OK"><br />
<http:header name="Date" value="Mon, 14 Mar 2011 22:03:25 GMT"/><br />
<http:header name="Transfer-Encoding" value="chunked"/><br />
<http:header name="Expires" value="-1"/><br />
<http:header name="X-XSS-Protection" value="1; mode=block"/><br />
<http:header name="Set-Cookie" value="...; expires=Tue, 13-Sep-2011 22:03:25 GMT; path=/; domain=.google.ch; HttpOnly"/><br />
<http:header name="Content-Type" value="text/html; charset=ISO-8859-1"/><br />
<http:header name="Server" value="gws"/><br />
<http:header name="Cache-Control" value="private, max-age=0"/><br />
<http:body media-type="text/html; charset=ISO-8859-1"/><br />
</http:response><br />
<html xmlns="http://www.w3.org/1999/xhtml"><br />
<head><br />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/><br />
<title>Google</title><br />
...<br />
</body><br />
</html><br />
</pre><br />
<br />
The response content type can also be overwritten in order to retrieve HTML pages and other textual data as plain string (using {{Code|text/plain}}) or in its binary representation (using {{Code|application/octet-stream}}). With the {{Code|http:header}} element, a custom user agent can be set. See the following example:<br />
<br />
'''Query:'''<br />
<pre lang='xquery'><br />
let $binary := http:send-request(<br />
<http:request method='get'<br />
override-media-type='application/octet-stream'<br />
href='http://www.google.com'><br />
<http:header name="User-Agent" value="Opera"/><br />
</http:request><br />
)[2]<br />
return try {<br />
html:parse($binary)<br />
} catch * {<br />
'Conversion to XML failed: ' || $err:description<br />
}<br />
</pre><br />
<br />
===SVG Data===<br />
<br />
Content-type ending with +xml, e.g. image/svg+xml.<br />
<br />
'''Query:'''<br />
<pre lang='xquery'>http:send-request(<http:request method='get'/>, 'http://upload.wikimedia.org/wikipedia/commons/6/6b/Bitmap_VS_SVG.svg')</pre><br />
<br />
'''Result:'''<br />
<pre lang="xml"><http:response status="200" message="OK"><br />
<http:header name="ETag" value="W/&quot;11b6d-4ba15ed4&quot;"/><br />
<http:header name="Age" value="9260"/><br />
<http:header name="Date" value="Mon, 14 Mar 2011 19:17:10 GMT"/><br />
<http:header name="Content-Length" value="72557"/><br />
<http:header name="Last-Modified" value="Wed, 17 Mar 2010 22:59:32 GMT"/><br />
<http:header name="Content-Type" value="image/svg+xml"/><br />
<http:header name="X-Cache-Lookup" value="MISS from knsq22.knams.wikimedia.org:80"/><br />
<http:header name="Connection" value="keep-alive"/><br />
<http:header name="Server" value="Sun-Java-System-Web-Server/7.0"/><br />
<http:header name="X-Cache" value="MISS from knsq22.knams.wikimedia.org"/><br />
<http:body media-type="image/svg+xml"/><br />
</http:response><br />
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="1063" height="638"><br />
<defs><br />
<linearGradient id="lg0"><br />
<stop stop-color="#3333ff" offset="0"/><br />
<stop stop-color="#3f3fff" stop-opacity="0" offset="1"/><br />
</linearGradient><br />
...<br />
</svg></pre><br />
<br />
==POST Request==<br />
<br />
POST request to the BaseX REST Service, specifying a username and password.<br />
<br />
'''Query:'''<br />
<pre lang='xquery'><br />
http:send-request(<br />
<http:request method='post' username='admin' password='admin'><br />
<http:body media-type='application/xml'/><br />
</http:request>,<br />
'http://localhost:8080/rest',<br />
<query><br />
<text><br />
<html>{<br />
for $i in 1 to 3<br />
return <div>Section {$i }</div><br />
}</html><br />
</text><br />
</query><br />
)<br />
</pre><br />
<br />
'''Result:'''<br />
<pre lang="xml"><br />
<http:response xmlns:http="http://expath.org/ns/http-client" status="200" message="OK"><br />
<http:header name="Content-Length" value="135"/><br />
<http:header name="Content-Type" value="application/xml"/><br />
<http:header name="Server" value="Jetty(6.1.26)"/><br />
<http:body media-type="application/xml"/><br />
</http:response><br />
<html><br />
<div>Section 1</div><br />
<div>Section 2</div><br />
<div>Section 3</div><br />
</html><br />
</pre><br />
<br />
==File Upload==<br />
<br />
Performs an HTML file upload. In the RESTXQ code, the uploaded file is written to the temporary directory:<br />
<br />
'''Query:'''<br />
<br />
<pre lang='xquery'><br />
let $path := 'file-to-be.uploaded'<br />
return http:send-request(<br />
<http:request method='POST'><br />
<http:multipart media-type='multipart/form-data'><br />
<http:header name='content-disposition'<br />
value='form-data; name="files"; filename="{ file:name($path) }"'/><br />
<http:body media-type='application/octet-stream'/><br />
</http:multipart><br />
</http:request>,<br />
'http://localhost:8080/write-to-temp',<br />
file:read-binary($path)<br />
)<br />
</pre><br />
<br />
'''RESTXQ service:'''<br />
<br />
<pre lang='xquery'><br />
declare<br />
%rest:POST<br />
%rest:path('/write-to-temp')<br />
%rest:form-param('files', '{$files}')<br />
function dba:file-upload(<br />
$files as map(xs:string, xs:base64Binary)<br />
) as empty-sequence() {<br />
map:for-each($files, function($file, $content) {<br />
file:write-binary(file:temp-dir() || $file, $content)<br />
});<br />
};<br />
</pre><br />
<br />
==Response Conversion==<br />
<br />
CSV, JSON and HTML responses are automatically converted to an XML representation. The target format can be influenced by supplying {{Code|csv}}, {{Code|json}} and {{Code|html}} attributes:<br />
<br />
'''Query:'''<br />
<br />
<pre lang='xquery'><br />
http:send-request(<http:request method='GET' href='http://localhost:8080/json' json='format=xquery,lax=true'/>)<br />
</pre><br />
<br />
'''Result:'''<br />
<pre lang="javascript"><br />
map { "abcde": 12345 }<br />
</pre><br />
<br />
Without the {{Code|json}} attribute, the response body is converted to the default XML representation:<br />
<br />
<pre lang="xml"><br />
<json type="object"><br />
<abcde>12345</abcde><br />
</json><br />
</pre><br />
<br />
'''RESTXQ service:'''<br />
<br />
<pre lang='xquery'><br />
declare<br />
%rest:path('json')<br />
%output:method('json')<br />
function local:json() {<br />
map { 'abcde': 12345 }<br />
};<br />
</pre><br />
<br />
See the [[CSV Module]], [[JSON Module]] and [[HTML Module]] for a list of the available options.<br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
|Description<br />
|- valign="top"<br />
|{{Code|HC0001}}<br />
|An HTTP error occurred.<br />
|- valign="top"<br />
|{{Code|HC0002}}<br />
|Error parsing the entity content as XML or HTML.<br />
|- valign="top"<br />
|{{Code|HC0003}}<br />
|With a multipart response, the override-media-type must be either a multipart media type or application/octet-stream.<br />
|- valign="top"<br />
|{{Code|HC0004}}<br />
|The src attribute on the body element is mutually exclusive with all other attribute (except the media-type).<br />
|- valign="top"<br />
|{{Code|HC0005}}<br />
|The request element is not valid.<br />
|- valign="top"<br />
|{{Code|HC0006}}<br />
|A timeout occurred waiting for the response.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 11.0<br />
* Updated: {{Function||http:send-request}}: {{Code|csv}}, {{Code|json}}, {{Code|html}} and {{Code|text}} attributes added.<br />
<br />
;Version 10.0<br />
* Updated: Implementation based on the new [https://openjdk.org/groups/net/httpclient/intro.html Java HTTP Client].<br />
<br />
;Version 9.0<br />
* Updated: support for gzipped content encoding<br />
<br />
;Version 8.0<br />
* Added: digest authentication<br />
<br />
;Version 7.6<br />
* Updated: {{Function||http:send-request}}: {{Code|HC0002}} is raised if the input cannot be parsed or converted to the final data type.<br />
* Updated: errors are using {{Code|text/plain}} as media-type.</div>CGhttps://docs.basex.org/index.php?title=Archive_Module&diff=16875Archive Module2023-12-01T16:38:40Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This [[Module Library|XQuery Module]] contains functions to handle archives (including ePub, Open Office, JAR, and many other formats). New ZIP and GZIP archives can be created, existing archives can be updated, and the archive entries can be listed and extracted. The {{Function||archive:extract-binary}} function includes an example for writing the contents of an archive to disk.<br />
<br />
=Conventions=<br />
<br />
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/archive</nowiki></code> namespace, which is statically bound to the {{Code|archive}} prefix.<br/><br />
<br />
=Content Handling=<br />
<br />
==archive:entries==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>archive:entries(<br />
$archive as xs:base64Binary<br />
) as element(archive:entry)*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the entry descriptors of the specified {{Code|$archive}}. A descriptor contains the following attributes, provided that they are available in the archive format:<br />
* {{Code|size}}: original file size<br />
* {{Code|last-modified}}: timestamp, formatted as xs:dateTime<br />
* {{Code|compressed-size}}: compressed file size<br />
An example:<br />
<pre lang="xml"><br />
<archive:entry size="1840" last-modified="2009-03-20T03:30:32" compressed-size="672"><br />
doc/index.html<br />
</archive:entry><br />
</pre><br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|error|#Errors}} archive creation failed.<br />
|- valign="top"<br />
|'''Examples'''<br />
|Sums up the file sizes of all entries of a JAR file:<br />
<pre lang='xquery'><br />
sum(archive:entries(file:read-binary('zip.zip'))/@size)<br />
</pre><br />
|}<br />
<br />
==archive:options==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>archive:options(<br />
$archive as xs:base64Binary<br />
) as map(*)</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Returns the options of the specified {{Code|$archive}} in the format specified by {{Function||archive:create}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|format|#Errors}} The archive format is not supported.<br/>{{Error|error|#Errors}} archive creation failed.<br />
|- valign="top"<br />
| '''Examples'''<br />
|A standard ZIP archive will return the following options:<br />
<pre lang='xquery'><br />
map {<br />
"format": "zip",<br />
"algorithm": "deflate"<br />
}<br />
</pre><br />
|}<br />
<br />
==archive:extract-text==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>archive:extract-text(<br />
$archive as xs:base64Binary,<br />
$entries as item()* := (),<br />
$encoding as xs:string := ()<br />
) as xs:string*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Extracts entries of the specified {{Code|$archive}} and returns them as texts.<br/>The returned entries can be limited via {{Code|$entries}}. The format of the argument is the same as for {{Function||archive:create}} (attributes will be ignored).<br/>The encoding of the input files can be specified via {{Code|$encoding}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|encode|#Errors}} the specified encoding is invalid or not supported, or the string conversion failed. Invalid XML characters will be ignored if {{Option|CHECKSTRINGS}} is turned off.<br/>{{Error|error|#Errors}} archive creation failed.<br />
|- valign="top"<br />
| '''Examples'''<br />
|The following expression extracts all {{Code|.txt}} files from an archive:<br />
<pre lang='xquery'><br />
let $archive := file:read-binary("documents.zip")<br />
for $entry in archive:entries($archive)[ends-with(., '.txt')]<br />
return archive:extract-text($archive, $entry)<br />
</pre><br />
|}<br />
<br />
==archive:extract-binary==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>archive:extract-binary(<br />
$archive as xs:base64Binary,<br />
$entries as item()* := ()<br />
) as xs:base64Binary*</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Extracts entries of the specified {{Code|$archive}} and returns them as binaries.<br/>The returned entries can be limited via {{Code|$entries}}. The format of the argument is the same as for {{Function||archive:create}} (attributes will be ignored).<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|error|#Errors}} archive creation failed.<br />
|- valign="top"<br />
| '''Examples'''<br />
|This example unzips all files of an archive to the current directory:<br />
<pre lang='xquery'><br />
let $archive := file:read-binary('archive.zip')<br />
let $entries := archive:entries($archive)<br />
let $contents := archive:extract-binary($archive)<br />
return for-each-pair($entries, $contents, function($entry, $content) {<br />
file:create-dir(replace($entry, "[^/]+$", "")),<br />
file:write-binary($entry, $content)<br />
})</pre><br />
|}<br />
<br />
=Updates=<br />
<br />
==archive:create==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature''' <br />
|<pre>archive:create(<br />
$entries as item(),<br />
$contents as item()*,<br />
$options as map(*)? := map { }<br />
) as xs:base64Binary</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates a new archive from the specified entries and contents.<br/>The {{Code|$entries}} argument contains meta information required to create new entries. All items may either be of type {{Code|xs:string}}, representing the entry name, or {{Code|element(archive:entry)}}, containing the name as text node and additional, optional attributes:<br />
* {{Code|last-modified}}: timestamp, specified as xs:dateTime (default: current time)<br />
* {{Code|compression-level}}: 0-9, 0 = uncompressed (default: 8)<br />
* {{Code|encoding}}: for textual entries (default: UTF-8)<br />
An example:<br />
<pre lang="xml"><br />
<archive:entry last-modified='2011-11-11T11:11:11'<br />
compression-level='8'<br />
encoding='US-ASCII'>hello.txt</archive:entry><br />
</pre><br />
The actual {{Code|$contents}} must be {{Code|xs:string}} or {{Code|xs:base64Binary}} items.<br/><br />
The {{Code|$options}} parameter contains archiving options:<br />
* {{Code|format}}: allowed values are {{Code|zip}} and {{Code|gzip}}. {{Code|zip}} is the default.<br />
* {{Code|algorithm}}: allowed values are {{Code|deflate}} and {{Code|stored}} (for the {{Code|zip}} format). {{Code|deflate}} is the default.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|number|#Errors}} the number of entries and contents differs.<br/>{{Error|format|#Errors}} the specified option or its value is invalid or not supported.<br/>{{Error|descriptor|#Errors}} entry descriptors contain invalid entry names, timestamps or compression levels.<br/>{{Error|encode|#Errors}} the specified encoding is invalid or not supported, or the string conversion failed. Invalid XML characters will be ignored if {{Option|CHECKSTRINGS}} is turned off.<br/>{{Error|single|#Errors}} the chosen archive format only allows single entries.<br/>{{Error|error|#Errors}} archive creation failed.<br />
|- valign="top"<br />
| '''Examples'''<br />
|The following one-liner creates an archive {{Code|archive.zip}} with one file {{Code|file.txt}}:<br />
<pre lang='xquery'><br />
archive:create(<archive:entry>file.txt</archive:entry>, 'Hello World')<br />
</pre><br />
The following function creates an archive {{Code|mp3.zip}}, which contains all MP3 files of a local directory:<br />
<pre lang='xquery'><br />
let $path := 'audio/'<br />
let $files := file:list($path, true(), '*.mp3')<br />
let $zip := archive:create($files,<br />
for $file in $files<br />
return file:read-binary($path || $file)<br />
)<br />
return file:write-binary('mp3.zip', $zip)</pre><br />
|}<br />
<br />
==archive:update==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>archive:update(<br />
$archive as xs:base64Binary,<br />
$entries as item()*,<br />
$contents as item()*<br />
) as xs:base64Binary</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Creates an updated version of the specified {{Code|$archive}} with new or replaced entries.<br/>The format of {{Code|$entries}} and {{Code|$contents}} is the same as for {{Function||archive:create}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|number|#Errors}} the number of entries and contents differs.<br/>{{Error|descriptor|#Errors}} entry descriptors contain invalid entry names, timestamps, compression levels or encodings.<br/>{{Error|encode|#Errors}} the specified encoding is invalid or not supported, or the string conversion failed. Invalid XML characters will be ignored if {{Option|CHECKSTRINGS}} is turned off.<br/>{{Error|modify|#Errors}} the entries of the given archive cannot be modified.<br/>{{Error|error|#Errors}} archive creation failed.<br />
|- valign="top"<br />
| '''Examples'''<br />
|This example replaces texts in a Word document:<br />
<pre lang='xquery'><br />
declare variable $input := "HelloWorld.docx";<br />
declare variable $output := "HelloUniverse.docx";<br />
declare variable $doc := "word/document.xml";<br />
<br />
let $archive := file:read-binary($input)<br />
let $entry :=<br />
copy $c := fn:parse-xml(archive:extract-text($archive, $doc))<br />
modify replace value of node $c//*[text() = "HELLO WORLD!"] with "HELLO UNIVERSE!"<br />
return fn:serialize($c)<br />
let $updated := archive:update($archive, $doc, $entry)<br />
return file:write-binary($output, $updated)<br />
</pre><br />
|}<br />
<br />
==archive:delete==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>archive:delete(<br />
$archive as xs:base64Binary,<br />
$entries as item()*<br />
) as xs:base64Binary</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|Deletes entries from an {{Code|$archive}}.<br/>The format of {{Code|$entries}} is the same as for {{Function||archive:create}}.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|modify|#Errors}} the entries of the given archive cannot be modified.<br/>{{Error|error|#Errors}} archive creation failed.<br />
|- valign="top"<br />
| '''Examples'''<br />
|This example deletes all HTML files in an archive and creates a new file:<br />
<pre lang='xquery'><br />
let $zip := file:read-binary('old.zip')<br />
let $entries := archive:entries($zip)[matches(., '\.x?html?$', 'i')]<br />
return file:write-binary('new.zip', archive:delete($zip, $entries))<br />
</pre><br />
|}<br />
<br />
=Convenience=<br />
<br />
==archive:create-from==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>archive:create-from(<br />
$path as xs:string,<br />
$options as map(*)? := map { },<br />
$entries as item()* := ()<br />
) as xs:base64Binary</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|This convenience function creates an archive from all files in the specified directory {{Code|$path}}.<br/>The {{Code|$options}} parameter contains archiving options, and the files to be archived can be limited via {{Code|$entries}}. The format of the two last arguments is identical to {{Function||archive:create}}, with two additional options:<br />
* {{Code|recursive}}: parse all files recursively (default: {{Code|true}}; ignored if entries are specified via the last argument).<br />
* {{Code|root-dir}}: use name of supplied directory as archive root directory (default: {{Code|false}}).<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|file:no-dir|File Module#Errors}} the specified path does not point to a directory.<br/>{{Error|file:is-dir|File Module#Errors}} one of the specified entries points to a directory.<br/>{{Error|file:not-found|File Module#Errors}} a specified entry does not exist.<br/>{{Error|error|#Errors}} archive creation failed.<br />
|- valign="top"<br />
| '''Examples'''<br />
|This example writes the files of a user’s home directory to <code>archive.zip</code>:<br />
<pre lang='xquery'><br />
let $zip := archive:create-from('/home/user/')<br />
return file:write-binary('archive.zip', $zip)<br />
</pre><br />
|}<br />
<br />
==archive:extract-to==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature'''<br />
|<pre>archive:extract-to(<br />
$path as xs:string,<br />
$archive as xs:base64Binary,<br />
$entries as item()* := ()<br />
) as empty-sequence()</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|This convenience function writes files of an {{Code|$archive}} directly to the specified directory {{Code|$path}}.<br/>The archive entries to be written can be restricted via {{Code|$entries}}. The format of the argument is the same as for {{Function||archive:create}} (attributes will be ignored).<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|error|#Errors}} archive creation failed.<br />
|- valign="top"<br />
| '''Examples'''<br />
|The following expression unzips all files of an archive to the current directory:<br />
<pre lang='xquery'><br />
archive:extract-to('.', file:read-binary('archive.zip'))<br />
</pre><br />
|}<br />
<br />
==archive:write==<br />
<br />
{| width='100%'<br />
|- valign="top"<br />
| width='120' | '''Signature''' <br />
|<pre>archive:write(<br />
$path as xs:string,<br />
$entries as item(),<br />
$contents as item()*,<br />
$options as map(*)? := map { }<br />
) as xs:base64Binary</pre><br />
|- valign="top"<br />
| '''Summary'''<br />
|This convenience function creates a new archive from the specified {{Code|$entries}} and {{Code|$contents}} and writes it disk.<br/> See {{Function||archive:create}} for more details.<br />
|- valign="top"<br />
| '''Errors'''<br />
|{{Error|number|#Errors}} the number of entries and contents differs.<br/>{{Error|format|#Errors}} the specified option or its value is invalid or not supported.<br/>{{Error|descriptor|#Errors}} entry descriptors contain invalid entry names, timestamps or compression levels.<br/>{{Error|encode|#Errors}} the specified encoding is invalid or not supported, or the string conversion failed. Invalid XML characters will be ignored if {{Option|CHECKSTRINGS}} is turned off.<br/>{{Error|single|#Errors}} the chosen archive format only allows single entries.<br/>{{Error|error|#Errors}} archive creation failed.<br />
|- valign="top"<br />
| '''Examples'''<br />
|All mp3 files from a directory are zipped and written to a file, along with an info file:<br />
<pre lang='xquery'><br />
let $files := file:children('music')[ends-with(., 'mp3')]<br />
return archive:write(<br />
'music.zip',<br />
('info.txt', $files ! file:name(.)),<br />
('Archive with MP3 files', $files ! file:read-binary(.))<br />
)<br />
</pre><br />
|}<br />
<br />
=Errors=<br />
<br />
{| class="wikitable" width="100%"<br />
! width="110"|Code<br />
|Description<br />
|- valign="top"<br />
|{{Code|descriptor}}<br />
|Entry descriptors contain invalid entry names, timestamps or compression levels.<br />
|- valign="top"<br />
|{{Code|encode}}<br />
|The specified encoding is invalid or not supported, or the string conversion failed. Invalid XML characters will be ignored if {{Option|CHECKSTRINGS}} is turned off.<br />
|- valign="top"<br />
|{{Code|error}}<br />
| processing failed.<br />
|- valign="top"<br />
|{{Code|format}}<br />
|The archive format or the specified option is invalid or not supported.<br />
|- valign="top"<br />
|{{Code|modify}}<br />
|The entries of the given archive cannot be modified.<br />
|- valign="top"<br />
|{{Code|number}}<br />
|The number of specified entries and contents differs.<br />
|- valign="top"<br />
|{{Code|single}}<br />
|The chosen archive format only allows single entries.<br />
|}<br />
<br />
=Changelog=<br />
<br />
;Version 9.6<br />
* Added: {{Function||archive:write}}<br />
<br />
;Version 9.0<br />
* Updated: {{Function||archive:create-from}}: options added<br />
* Updated: error codes updated; errors now use the module namespace<br />
<br />
;Version 8.5<br />
* Updated: {{Function||archive:options}}: map returned instead of element<br />
<br />
;Version 8.3<br />
* Added: {{Function||archive:create-from}}, {{Function||archive:extract-to}} (replaces <code>archive:write</code>)<br />
<br />
The module was introduced with Version 7.3.</div>CGhttps://docs.basex.org/index.php?title=Java_Bindings&diff=16876Java Bindings2023-12-01T16:38:40Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This article is part of the [[XQuery|XQuery Portal]]. It demonstrates different ways to invoke Java code from XQuery, and it presents extensions to access the current query context from Java.<br />
<br />
The Java Binding feature is an extensibility mechanism which enables developers<br />
to directly access Java variables and execute code from XQuery. Addressed Java code must either be contained in the Java classpath, or it must be located in the [[Repository]].<br />
<br />
Please bear in mind that the execution of Java code may cause side effects that conflict with the functional nature of XQuery, or may introduce new security risks to your project.<br />
<br />
Some more notes:<br />
* With the middle dot notation, three adjacent dots can be used to specify array types.<br />
* The path to the standard package {{Code|java.lang.}} can be omitted.<br />
* Java objects are wrapped into function items.<br />
* Results of constructor calls are always returned as function item.<br />
* With {{Option|WRAPJAVA}}, it can be controlled how Java values are converted to XQuery.<br />
<br />
=Identification=<br />
<br />
==Classes==<br />
<br />
A Java class is identified by a namespace URI. The original URI is rewritten as follows:<br />
<br />
# The [[#URI Rewriting|URI Rewriting]] steps are applied to the URI.<br />
# Slashes in the resulting URI are replaced with dots.<br />
# The last path segment of the URI is capitalized and rewritten to [https://en.wikipedia.org/wiki/CamelCase CamelCase].<br />
<br />
The normalization steps are skipped if the URI is prefixed with {{Code|java:}}. The path to the standard package {{Code|java.lang.}} can be omitted:<br />
<br />
* <code><nowiki>http://basex.org/modules/meta-data</nowiki></code> → <code>org.basex.modules.MetaData</code><br />
* <code>java:java.lang.String</code> → <code>java.lang.String</code><br />
* <code>StringBuilder</code> → <code>java.lang.StringBuilder</code><br />
<br />
==Functions and Variables==<br />
<br />
Java constructors, functions and variables can be referenced and evaluated by the existing XQuery function syntax:<br />
<br />
* The namespace of the function name identifies the Java class.<br />
* The local part of the name, which is rewritten to camel case, identifies a variable or function of that class.<br />
* The middle dot character <code>[https://www.fileformat.info/info/unicode/char/b7/index.htm ·]</code> (<code>&amp;#xB7;</code>, a valid character in XQuery names, but not in Java) can be used to append exact Java parameter types to the function name. Class types must be referenced by their full path. Three adjacent dots can be used to address an array argument.<br />
<br />
{| class="wikitable"<br />
|- valign="top"<br />
! Addressed code<br />
! XQuery<br />
! Java<br />
|- valign="top"<br />
| Variable<br />
| <code>Q{Integer}MIN_VALUE()</code><br />
| <code>Integer.MIN_VALUE</code><br />
|- valign="top"<br />
| Function<br />
| <code>Q{Object}hash-code($object)</code><br />
| <code>object.hashCode()</code><br />
|- valign="top"<br />
| Function with argument<br />
| <code>Q{String}split·String·int($string, ';', xs:int(3))</code><br />
| <code>string.split(";", 3)</code><br />
|- valign="top"<br />
| Constructor with array argument<br />
| <code>Q{String}new·byte...(xs:hexBinary('414243'))</code><br />
| <code>new String(new byte[] { 41, 42, 43 })</code><br />
|}<br />
<br />
As XQuery and Java have different type systems, XQuery arguments must be converted to equivalent Java values, and the result of a Java function is converted back to an XQuery value (see [[#Data Types|Data Types]]).<br />
<br />
If the Java function you want to address is not detected, you may need to cast your values to the target type. For example, if a Java function expects a primitive {{Code|int}} value, you will need to convert your XQuery integers to {{Code|xs:int}}.<br />
<br />
=Namespace Declarations=<br />
<br />
In the following example, the Java {{Code|Math}} class is referenced. When executed, the query returns the cosine of an angle by calling the static method {{Code|cos()}}, and the value of π by addressing the static variable via {{Code|PI()}}:<br />
<br />
<pre lang='xquery'><br />
declare namespace math = "java:java.lang.Math";<br />
math:cos(xs:double(0)), math:PI()<br />
</pre><br />
<br />
With the [[XQuery 3.0#Expanded QNames|Expanded QName]] notation of XQuery 3.0, the namespace can directly be embedded in the function call:<br />
<br />
<pre lang='xquery'><br />
Q{java:java.lang.Math}cos(xs:double(0))<br />
</pre><br />
<br />
The constructor of a class can be invoked by calling the virtual function {{Code|new()}}. Instance methods can then called by passing on the resulting Java object as first argument. In the following example, 256 bytes are written to the file {{Code|output.txt}}. First, a new {{Code|FileWriter}} instance is created, and its {{Code|write()}} function is called in the next step:<br />
<br />
<pre lang='xquery'><br />
declare namespace fw = 'java:java.io.FileWriter';<br />
let $file := fw:new('output.txt')<br />
return (<br />
for $i in 0 to 255<br />
return fw:write($file, xs:int($i)),<br />
fw:close($file)<br />
)<br />
</pre><br />
<br />
If the result of a Java call contains invalid XML characters, it will be rejected. The validity check can be disabled by setting {{Option|CHECKSTRINGS}} to false. In the example below, a file with a single {{Code|00}} byte is written, and this file will then be accessed by via Java functions:<br />
<br />
<pre lang='xquery'><br />
declare namespace br = 'java:java.io.BufferedReader';<br />
declare namespace fr = 'java:java.io.FileReader';<br />
<br />
declare option db:checkstrings 'false';<br />
<br />
(: write file :)<br />
file:write-binary('00.bin', xs:hexBinary('00')),<br />
(: read file :)<br />
let $br := br:new(fr:new('00.bin'))<br />
return (<br />
br:readLine($br), <br />
br:close($br)<br />
)<br />
</pre><br />
<br />
The option can also be specified via a pragma:<br />
<br />
<pre lang='xquery'><br />
(# db:checkstrings #) {<br />
br:new(fr:new('00.bin')) ! (br:readLine(.), br:close(.))<br />
}<br />
</pre><br />
<br />
=Module Imports=<br />
<br />
A Java class can be instantiated by ''importing'' them as a module: A new instance of the addressed class will be constructed, which can then be referenced in the query body.<br />
<br />
In the (side-effecting) example below, a HashSet instance is created, values are added, and the size of the set is returned. As {{Code|set:add()}} returns boolean values, {{Function|Profiling|prof:void}} is used to swallow the values:<br />
<br />
<pre lang='xquery'><br />
import module namespace set = "java:java.util.HashSet";<br />
prof:void(<br />
for $s in ("one", "two", "one")<br />
return set:add($s)<br />
),<br />
set:size()<br />
</pre><br />
<br />
The execution of imported classes is more efficient than the execution of instances that have been created via {{Code|new()}}. In turn, no arguments can be supplied in the import statement, and the construction will only be successful if the class can be instantiated without arguments.<br />
<br />
=Integration=<br />
<br />
Java classes can be coupled more closely to BaseX. If a class inherits the abstract [https://github.com/BaseXdb/basex/blob/master/basex-core/src/main/java/org/basex/query/QueryModule.java QueryModule] class, the two variables [https://github.com/BaseXdb/basex/blob/master/basex-core/src/main/java/org/basex/query/QueryContext.java queryContext] and [https://github.com/BaseXdb/basex/blob/master/basex-core/src/main/java/org/basex/query/StaticContext.java staticContext] get available, which provide access to the global and static context of a query.<br />
<br />
The [https://github.com/BaseXdb/basex/blob/master/basex-core/src/main/java/org/basex/query/QueryResource.java QueryResource] interface can be implemented to enforce finalizing operations, such as the closing of opened connections or resources in a module. Its {{Code|close()}} method will be called after the XQuery expression has been fully evaluated.<br />
<br />
==Annotations==<br />
<br />
The internal properties of functions can be assigned via annotations:<br />
<br />
* Java functions can only be executed by users with [[User_Management|Admin permissions]]. You can annotate a function with {{Code|@Requires(<Permission>)}} to also make it accessible to users with fewer privileges.<br />
* Java code is treated as ''nondeterministic'', as its behavior cannot be predicted by the XQuery processor. You may annotate a function as {{Code|@Deterministic}} if you know that it will have no side effects and will always yield the same result.<br />
* Java code is treated as ''context-independent''. If a function accesses the query context, it should be annotated as {{Code|@ContextDependent}}<br />
* Java code is treated as ''focus-independent''. If a function accesses the current context item, position or size, it should be annotated as {{Code|@FocusDependent}}<br />
<br />
In the following code, information from the static query context is returned by the first function, and a query exception is raised by the second function:<br />
<br />
<pre lang='xquery'><br />
import module namespace context = 'org.basex.examples.query.ContextModule';<br />
<br />
element user {<br />
context:user()<br />
},<br />
try {<br />
element to-int { context:to-int('abc') }<br />
} catch basex:error {<br />
element error { $err:description }<br />
}<br />
</pre><br />
<br />
The imported Java class is shown below:<br />
<br />
<pre lang="java"><br />
package org.basex.examples.query;<br />
<br />
import org.basex.query.*;<br />
import org.basex.query.value.item.*;<br />
import org.basex.util.*;<br />
<br />
/**<br />
* This example inherits the {@link QueryModule} class and<br />
* implements the QueryResource interface.<br />
*/<br />
public class ContextModule extends QueryModule implements QueryResource {<br />
/**<br />
* Returns the name of the logged-in user.<br />
* @return user string<br />
*/<br />
@Requires(Permission.NONE)<br />
@Deterministic<br />
@ContextDependent<br />
public String user() {<br />
return queryContext.context.user.name;<br />
}<br />
<br />
/**<br />
* Converts the specified string to an integer.<br />
* @param value string to be converted<br />
* @return resulting integer<br />
* @throws QueryException query exception<br />
*/<br />
@Requires(Permission.NONE)<br />
@Deterministic<br />
public int toInt(final String value) throws QueryException {<br />
try {<br />
return Integer.parseInt(value);<br />
} catch(NumberFormatException ex) {<br />
throw new QueryException("Integer conversion failed: " + value);<br />
}<br />
}<br />
<br />
@Override<br />
public void close() {<br />
// defined in QueryResource interface, will be called after query evaluation<br />
}<br />
}<br />
</pre><br />
<br />
The result will look as follows:<br />
<br />
<pre lang="xml"><br />
<user>admin</admin><br />
<error>Integer conversion failed: abc</error><br />
</pre><br />
<br />
Please visit the XQuery 3.0 specification if you want to get more insight into<br />
[https://www.w3.org/TR/xpath-functions-31/#properties-of-functions function properties].<br />
<br />
==Updates==<br />
<br />
The {{Code|@Updating}} annotation can be applied to mark Java functions that perform write or update operations:<br />
<br />
<pre lang="java"><br />
@Updating<br />
public void backup() {<br />
// ...<br />
}<br />
</pre><br />
<br />
An XQuery expression will be handled as an [[XQuery Update#Updating Expressions|updating expression]] if it calls an updating Java function. In contrast to XQuery update operations, the Java code will immediately be executed, but the result will be cached as if {{Function|Update|update:output}} was called.<br />
<br />
The annotation is particularly helpful if combined with a lock annotation.<br />
<br />
==Locking==<br />
<br />
By default, a Java function will be executed in parallel with other code. If a Java function performs sensitive operations, it is advisable to explicitly lock the code.<br />
<br />
===Java Locks===<br />
<br />
Java provides a handful of mechanism to control the execution of code. The concurrent execution of functions can be avoided with the {{Code|synchronized}} keyword. For more complex scenarios, the Lock, Semaphore and Atomic classes can be brought into play.<br />
<br />
===XQuery Locks===<br />
<br />
If you want to synchronize the execution of your code with BaseX locks, you can take advantage of the {{Code|@Lock}} annotation:<br />
<br />
<pre lang="java"><br />
@Lock("HEAVYIO")<br />
public void read() {<br />
// ...<br />
}<br />
<br />
@Updating<br />
@Lock("HEAVYIO")<br />
public void write() {<br />
// ...<br />
}<br />
</pre><br />
<br />
If an XQuery expression invokes {{Code|write()}}, any other query that calls {{Code|write()}} or {{Code|read()}} needs to wait for the query to be finished. The {{Code|read()}} function can be run in parallel; whereas queries will be queued if {{Code|write()}} is called.<br />
<br />
More details on concurrent querying can be found in the article on [[Transaction Management]].<br />
<br />
==Data Types==<br />
<br />
===Conversion to Java===<br />
<br />
Before Java code is executed, the arguments are converted to Java values, depending on the addressed function or constructor parameters. The accepted Java types and the original XQuery types are depicted in the second and first column of the table below.<br />
<br />
If a numeric value is supplied for which no exact matching is defined, it is cast to the appropriate type unless it exceeds its limits. The following two function calls are equivalent:<br />
<br />
<pre lang='xquery'><br />
(: exact match :)<br />
Q{String}codePointAt('ABC', xs:int(1)),<br />
(: xs:byte and xs:integer casts :)<br />
Q{String}codePointAt('ABC', xs:byte(1)),<br />
Q{String}codePointAt('ABC', 1)<br />
</pre><br />
<br />
===Conversion to XQuery===<br />
<br />
By default, Java values with the most common types (as shown in the second and third column of the table) are converted to XQuery values. All other values are returned as ''Java items'', which are function items with a wrapped Java value. The results of constructor calls are always returned as Java items.<br />
<br />
The conversion of the wrapped Java value to XQuery is enforced by invoking the function item: Values in {{Code|Iterator}} and {{Code|Iterable}} instances (Lists, Sets and Collections) are converted to items, and maps are converted to XQuery maps:<br />
<br />
<pre lang='xquery'><br />
declare namespace Scanner = 'java:java.util.Scanner';<br />
let $scanner := Scanner:new("A B C") => Scanner:useDelimiter(" ")<br />
return $scanner()<br />
</pre><br />
<br />
If no conversion is defined, a string is returned, resulting from the {{Code|toString()}} method of the object. This method is also called if the string representation of a Java item is requested:<br />
<br />
<pre lang='xquery'><br />
(: returns the string representations of a HashMap and an ArrayList instance :)<br />
'Map: ' || Q{java.util.HashMap}new(),<br />
string(Q{java:java.util.ArrayList}new())<br />
</pre><br />
<br />
The conversion can be further controlled with the {{Option|WRAPJAVA}} option. The following values exist:<br />
<br />
{| class="wikitable"<br />
|- valign="top"<br />
! Value<br />
! Description<br />
|- valign="top"<br />
| {{Code|some}}<br />
| The default: Java values of the most common types are converted, others are wrapped into Java items.<br />
|- valign="top"<br />
| {{Code|none}}<br />
| All Java values are converted. If no conversion is defined, a string is returned, resulting from the {{Code|toString()}} method.<br />
|- valign="top"<br />
| {{Code|all}}<br />
| Java values are wrapped into Java items (excluding those inheriting the internal type {{Code|org.basex.query.value.Value}}).<br />
|- valign="top"<br />
| {{Code|instance}}<br />
| If the method of a class instance was called, the Java value is ignored and the instance is wrapped into a Java item. Otherwise, the Java value is returned.<br />
|- valign="top"<br />
| {{Code|void}}<br />
| Java values are ignored, and an empty sequence is returned instead.<br />
|}<br />
<br />
In the following example, the result of the first function – a char array – is wrapped and passed on to a {{Code|CharBuffer}} function. Without the option, the single-value array would be converted to an {{Code|xs:unsignedShort}} item and the second function call would fail:<br />
<br />
<pre lang='xquery'><br />
(: Without the pragma, the result of toChars would be converted to an xs:unsignedShort item, and the second function call would fail :)<br />
<br />
(# db:wrapjava all #) {<br />
Q{Character}toChars(xs:int(33))<br />
=> Q{java.nio.CharBuffer}wrap()<br />
}<br />
</pre><br />
<br />
The next example demonstrates a use case for the {{Code|instance}} option:<br />
<br />
<pre lang='xquery'><br />
(: Thanks to the pragma, the function calls can be chained :)<br />
<br />
declare namespace set = 'java:java.util.HashSet';<br />
let $set := (# db:wrapjava instance #) {<br />
set:new()<br />
=> set:add('1')<br />
=> set:add('2')<br />
}<br />
return $set()<br />
</pre><br />
<br />
The {{Code|void}} option is helpful if side-effecting methods return values that do not contribute to the final result:<br />
<br />
<pre lang='xquery'><br />
(: Without the pragma, 100 booleans would be returned by the FLWOR expression :)<br />
<br />
declare namespace set = 'java:java.util.HashSet';<br />
let $set := set:new()<br />
return (<br />
(# db:wrapjava void #) {<br />
for $i in 1 to 100<br />
return set:add($set, $i)<br />
},<br />
$set()<br />
)<br />
</pre><br />
<br />
The irrelevant results could also be swallowed with {{Function|Profiling|prof:void}}.<br />
<br />
{| class="wikitable"<br />
|- valign="top"<br />
! XQuery input<br />
! Expected or returned Java type<br />
! XQuery output<br />
|- valign="top"<br />
| <code>item()*</code> (no conversion)<br />
| <code>org.basex.query.value.Value</code><br />
| <code>item()*</code> (no conversion)<br />
|- valign="top"<br />
| <code>empty-sequence()</code><br />
| <code>null</code><br />
| <code>empty-sequence()</code><br />
|- valign="top"<br />
| <code>xs:string</code>, <code>xs:untypedAtomic</code><br />
| <code>String</code><br />
| <code>xs:string</code><br />
|- valign="top"<br />
| <code>xs:unsignedShort</code><br />
| <code>char</code>, <code>Character</code><br />
| <code>xs:unsignedShort</code><br />
|- valign="top"<br />
| <code>xs:boolean</code><br />
| <code>boolean</code>, <code>Boolean</code><br />
| <code>xs:boolean</code><br />
|- valign="top"<br />
| <code>xs:byte</code><br />
| <code>byte</code>, <code>Byte</code><br />
| <code>xs:byte</code><br />
|- valign="top"<br />
| <code>xs:short</code><br />
| <code>short</code>, <code>Short</code><br />
| <code>xs:short</code><br />
|- valign="top"<br />
| <code>xs:int</code><br />
| <code>int</code>, <code>Integer</code><br />
| <code>xs:int</code><br />
|- valign="top"<br />
| <code>xs:integer</code>, <code>xs:long</code><br />
| <code>long</code>, <code>Long</code><br />
| <code>xs:integer</code><br />
|- valign="top"<br />
| <code>xs:unsignedLong</code><br />
| <code>java.math.BigInteger</code><br />
| <code>xs:unsignedLong</code> (lossy)<br />
|- valign="top"<br />
| <code>xs:decimal</code><br />
| <code>java.math.BigDecimal</code><br />
| <code>xs:decimal</code><br />
|- valign="top"<br />
| <code>xs:float</code><br />
| <code>float</code>, <code>Float</code><br />
| <code>xs:float</code><br />
|- valign="top"<br />
| <code>xs:double</code><br />
| <code>double</code>, <code>Double</code><br />
| <code>xs:double</code><br />
|- valign="top"<br />
| <code>xs:QName</code><br />
| <code>javax.xml.namespace.QName</code><br />
| <code>xs:QName</code><br />
|- valign="top"<br />
| <code>xs:anyURI</code><br />
| <code>java.net.URI</code>, <code>java.net.URL</code><br />
| <code>xs:anyURI</code><br />
|- valign="top"<br />
| <code>xs:date</code><br />
| <code>javax.xml.datatype.XMLGregorianCalendar</code><br />
| <code>xs:date</code><br />
|- valign="top"<br />
| <code>xs:duration</code><br />
| <code>javax.xml.datatype.Duration</code><br />
| <code>xs:duration</code><br />
|- valign="top"<br />
| <code>node()</code><br />
| <code>org.w3c.dom.Node</code><br />
| <code>node()</code><br />
|- valign="top"<br />
| <code>array(xs:boolean)</code><br />
| <code>boolean[]</code><br />
| <code>xs:boolean*</code><br />
|- valign="top"<br />
| <code>array(xs:string)</code><br />
| <code>String[]</code><br />
| <code>xs:string*</code><br />
|- valign="top"<br />
| <code>array(xs:unsignedShort)</code><br />
| <code>char[]</code><br />
| <code>xs:unsignedShort*</code><br />
|- valign="top"<br />
| <code>array(xs:short)</code><br />
| <code>short[]</code><br />
| <code>xs:short*</code><br />
|- valign="top"<br />
| <code>array(xs:int)</code><br />
| <code>int[]</code><br />
| <code>xs:int*</code><br />
|- valign="top"<br />
| <code>array(xs:integer)</code>, <code>array(xs:long)</code><br />
| <code>long[]</code><br />
| <code>xs:integer*</code><br />
|- valign="top"<br />
| <code>array(xs:float)</code><br />
| <code>float[]</code><br />
| <code>xs:float*</code><br />
|- valign="top"<br />
| <code>array(xs:double)</code><br />
| <code>double[]</code><br />
| <code>xs:double*</code><br />
|- valign="top"<br />
| <code>Object[]</code> (others)<br />
| <code>item()*</code><br />
| <code>array(*)</code> (others)<br />
|- valign="top"<br />
| <code>map(*)</code><br />
| java.util.HashMap<br />
| <code>Wrapped Java object</code><br />
|}<br />
<br />
==URI Rewriting==<br />
<br />
Before a Java class or module is accessed, its namespace URI will be normalized:<br />
<br />
# If the URI is a URL:<br />
## colons will be replaced with slashes,<br />
## in the URI authority, the order of all substrings separated by dots is reversed, and<br />
## dots in the authority and the path are replaced by slashes. If no path exists, a single slash is appended.<br />
# Otherwise, if the URI is a URN, colons will be replaced with slashes.<br />
# Characters other than letters, dots and slashes will be replaced with dashes.<br />
# If the resulting string ends with a slash, the {{Code|index}} string is appended.<br />
<br />
If the resulting path has no file suffix, it may point to either an XQuery module or a Java archive:<br />
<br />
* {{Code|<nowiki>http://basex.org/modules/hello/World</nowiki>}} → {{Code|org/basex/modules/hello/World}}<br />
* {{Code|<nowiki>http://www.example.com</nowiki>}} → {{Code|com/example/www/index}}<br />
* {{Code|a/little/example}} → {{Code|a/little/example}}<br />
* {{Code|a:b:c}} → {{Code|a/b/c}}<br />
<br />
Note that the mapping is not unique: Different URIs may result in the same path.<br />
<br />
=Changelog=<br />
<br />
; Version 9.6<br />
* Updated: Java Bindings revised (new mappings, Java functiom items, {{Option|WRAPJAVA}} option).<br />
<br />
; Version 9.4<br />
* Added: Annotation for [[#Updates|updating functions]].<br />
* Updated: Single annotation for read and write locks.<br />
<br />
; Version 8.4<br />
* Updated: Rewriting rules<br />
<br />
;Version 8.2<br />
* Added: [[#URI Rewriting|URI Rewriting]]: support for URNs<br />
<br />
; Version 8.0<br />
* Added: {{Code|QueryResource}} interface, called after a query has been fully evaluated.<br />
<br />
; Version 7.8<br />
* Added: Java locking annotations<br />
* Updated: {{Code|context}} variable has been split into {{Code|queryContext}} and {{Code|staticContext}}.<br />
<br />
; Version 7.2.1<br />
* Added: import of Java modules, context awareness<br />
* Added: [[#Packaging|Packaging]], [[#URI Rewriting|URI Rewriting]]</div>CGhttps://docs.basex.org/index.php?title=Full-Text&diff=16877Full-Text2023-12-01T16:38:40Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This article is part of the [[XQuery|XQuery Portal]]. It summarizes the features of the [https://www.w3.org/TR/xpath-full-text-10/ W3C XQuery Full Text] Recommendation, and custom features of the implementation in BaseX.<br />
<br />
Please read the separate [[Indexes#Full-Text Index|Full-Text Index]] section in our documentation if you want to learn how to evaluate full-text requests on large databases within milliseconds.<br />
<br />
=Introduction=<br />
<br />
The XQuery and XPath Full Text Recommendation (XQFT) is a feature-rich extension of the XQuery language. It can be used to both query XML documents and single strings for words and phrases. BaseX was the first query processor that supported all features of the specification.<br />
<br />
This section gives you a quick insight into the most important features of the language.<br />
<br />
This is a simple example for a basic full-text expression:<br />
<br />
<pre lang='xquery'><br />
"This is YOUR World" contains text "your world"<br />
</pre><br />
<br />
It yields {{Code|true}}, because the search string is ''tokenized'' before it is compared with the tokenized input string. In the tokenization process, several normalizations take place. Many of those steps can hardly be simulated with plain XQuery: as an example, upper/lower case and diacritics (umlauts, accents, etc.) are removed and an optional, language-dependent stemming algorithm is applied. Beside that, special characters such as whitespaces and punctuation marks will be ignored. Thus, this query also yields true:<br />
<br />
<pre lang='xquery'><br />
"Well... Done!" contains text "well, done"<br />
</pre><br />
<br />
The {{Code|occurs}} keyword comes into play when more than one occurrence of a token is to be found:<br />
<br />
<pre lang='xquery'><br />
"one and two and three" contains text "and" occurs at least 2 times<br />
</pre><br />
<br />
Various range modifiers are available: {{Code|exactly}}, {{Code|at least}}, {{Code|at most}}, and {{Code|from ... to ...}}.<br />
<br />
==Combining Results==<br />
<br />
In the given example, curly braces are used to combine multiple keywords:<br />
<br />
<pre lang='xquery'><br />
for $country in doc('factbook')//country<br />
where $country//religions[text() contains text { 'Sunni', 'Shia' } any]<br />
return $country/name<br />
</pre><br />
<br />
The query will output the names of all countries with a religion element containing {{Code|sunni}} or {{Code|shia}}. The {{Code|any}} keyword is optional; it can be replaced with:<br />
<br />
* {{Code|all}}: all strings need to be found<br />
* {{Code|any word}}: any of the single words within the specified strings need to be found<br />
* {{Code|all words}}: all single words within the specified strings need to be found<br />
* {{Code|phrase}}: all strings need to be found as a single phrase<br />
<br />
The keywords {{Code|ftand}}, {{Code|ftor}} and {{Code|ftnot}} can also be used to combine multiple query terms. The following query yields the same result as the last one does:<br />
<br />
<pre lang='xquery'><br />
doc('factbook')//country[descendant::religions contains text 'sunni' ftor 'shia']/name<br />
</pre><br />
<br />
The keywords {{Code|not in}} are special: they are used to find tokens which are not part of a longer token sequence:<br />
<br />
<pre lang='xquery'><br />
for $text in ("New York", "new conditions")<br />
return $text contains text "New" not in "New York"<br />
</pre><br />
<br />
Due to the complex data model of the XQuery Full Text spec, the usage of {{Code|ftand}} may lead to a high memory consumption. If you should encounter problems, simply use the {{Code|all}} keyword:<br />
<br />
<pre lang='xquery'><br />
doc('factbook')//country[descendant::religions contains text { 'Christian', 'Jewish'} all]/name<br />
</pre><br />
<br />
==Positional Filters==<br />
<br />
A popular retrieval operation is to filter texts by the distance of the searched words. In this query…<br />
<br />
<pre lang='xquery'><br />
<xml><br />
<text>There is some reason why ...</text><br />
<text>For some good yet unknown reason, ...</text><br />
<text>The reason why some people ...</text><br />
</xml>//text[. contains text { "some", "reason" } all ordered distance at most 3 words]<br />
</pre><br />
<br />
…the two first texts will be returned as result, because there are at most three words between {{Code|some}} and {{Code|reason}}. Additionally, the {{Code|ordered}} keyword ensures that the words are found in the specified order, which is why the third text is excluded. Note that {{Code|all}} is required here to guarantee that only those hits will be accepted that contain all searched words.<br />
<br />
The {{Code|window}} keyword is related: it accepts those texts in which all keyword occur within the specified number of tokens. Can you guess what is returned by the following query?<br />
<br />
<pre lang='xquery'><br />
("A C D", "A B C D E")[. contains text { "A", "E" } all window 3 words]<br />
</pre><br />
<br />
Sometimes it is interesting to only select texts in which all searched terms occur in the {{Code|same sentence}} or {{Code|paragraph}} (you can even filter for {{Code|different}} sentences/paragraphs). This is obviously not the case in the following example:<br />
<br />
<pre lang='xquery'><br />
'Mary told me, “I will survive!”.' contains text { 'will', 'told' } all words same sentence<br />
</pre><br />
<br />
By the way: In some examples above, the {{Code|words}} unit was used, but {{Code|sentences}} and {{Code|paragraphs}} would have been valid alternatives.<br />
<br />
Last but not least, three specifiers exist to filter results depending on the position of a hit:<br />
<br />
* {{Code|at start}} expects tokens to occur at the beginning of a text<br />
* {{Code|at end}} expects tokens to occur at the text end<br />
* {{Code|entire content}} only accepts texts which have no other words at the beginning or end<br />
<br />
==Match Options==<br />
<br />
As indicated in the introduction, the input and query texts are tokenized before they are compared with each other. During this process, texts are split into tokens, which are then normalized, based on the following matching options:<br />
<br />
* If {{Code|case}} is insensitive, no distinction is made between characters in upper and lower case. By default, the option is {{Code|insensitive}}; it can also be set to {{Code|sensitive}}:<br />
<pre lang='xquery'><br />
"Respect Upper Case" contains text "Upper" using case sensitive<br />
</pre><br />
* If {{Code|diacritics}} is insensitive, characters with and without diacritics (umlauts, characters with accents) are declared as identical. By default, the option is {{Code|insensitive}}; it can also be set to {{Code|sensitive}}:<br />
<pre lang='xquery'><br />
"'Äpfel' will not be found..." contains text "Apfel" using diacritics sensitive<br />
</pre><br />
* If {{Code|stemming}} is activated, words are shortened to a base form by a language-specific stemmer:<br />
<pre lang='xquery'><br />
"catch" contains text "catches" using stemming<br />
</pre><br />
* With the {{Code|stop words}} option, a list of words can be defined that will be ignored when tokenizing a string. This is particularly helpful if the full-text index takes too much space (a standard stopword list for English texts is provided in the directory {{Code|etc/stopwords.txt}} in the full distributions of BaseX, and available online at http://files.basex.org/etc/stopwords.txt):<br />
<pre lang='xquery'><br />
"You and me" contains text "you or me" using stop words ("and", "or"),<br />
"You and me" contains text "you or me" using stop words at "http://files.basex.org/etc/stopwords.txt"<br />
</pre><br />
* Related terms such as synonyms can be found with the sophisticated [[#Thesaurus|Thesaurus]] option.<br />
<br />
The {{Code|wildcards}} option facilitates search operations similar to simple regular expressions:<br />
<br />
* {{Code|.}} matches a single arbitrary character.<br />
* {{Code|.?}} matches either zero or one character.<br />
* {{Code|.*}} matches zero or more characters.<br />
* {{Code|.+}} matches one or more characters.<br />
* <code>.{min,max}</code> matches ''min''–''max'' number of characters.<br />
<br />
<pre lang='xquery'><br />
"This may be interesting in the year 2000" contains text { "interest.*", "2.{3,3}" } using wildcards<br />
</pre><br />
<br />
This was a quick introduction to XQuery Full Text; you are invited to explore the numerous other features of the language!<br />
<br />
=BaseX Features=<br />
<br />
==Languages==<br />
<br />
The chosen language determines how strings will be tokenized and stemmed. Either names (e.g. <code>English</code>, <code>German</code>) or codes (<code>en</code>, <code>de</code>) can be specified. <br />
A list of all language codes that are available on your system can be retrieved as follows:<br />
<br />
<pre lang='xquery'><br />
declare namespace locale = "java:java.util.Locale";<br />
distinct-values(locale:getAvailableLocales() ! locale:getLanguage(.))<br />
</pre><br />
<br />
By default, unless the languages codes <code>ja</code>, <code>ar</code>, <code>ko</code>, <code>th</code>, or <code>zh</code> are specified, a tokenizer for Western texts is used:<br />
<br />
* Whitespaces are interpreted as token delimiters.<br />
* Sentence delimiters are <code>.</code>, <code>!</code>, and <code>?</code>.<br />
* Paragraph delimiters are newlines (<code>&amp;#xa;</code>).<br />
<br />
The basic JAR file of BaseX comes with built-in stemming support for English, German, Greek and Indonesian. Some more languages are supported if the following libraries are found in the [[Startup#Distributions|classpath]]:<br />
<br />
* [https://files.basex.org/maven/org/apache/lucene-stemmers/3.4.0/lucene-stemmers-3.4.0.jar lucene-stemmers-3.4.0.jar] includes the Snowball and Lucene stemmers for the following languages: Arabic, Bulgarian, Catalan, Czech, Danish, Dutch, Finnish, French, Hindi, Hungarian, Italian, Latvian, Lithuanian, Norwegian, Portuguese, Romanian, Russian, Spanish, Swedish, Turkish.<br />
<br />
* [https://osdn.net/projects/igo/releases/ igo-0.4.3.jar]: [[Full-Text: Japanese|An additional article]] explains how Igo can be integrated, and how Japanese texts are tokenized and stemmed.<br />
<br />
The JAR files are included in the ZIP and EXE distributions of BaseX.<br />
<br />
The following two queries, which both return <code>true</code>, demonstrate that stemming depends on the selected language:<br />
<br />
<pre lang='xquery'><br />
"Indexing" contains text "index" using stemming,<br />
"häuser" contains text "haus" using stemming using language "German"<br />
</pre><br />
<br />
==Scoring==<br />
<br />
The XQuery Full Text Recommendation allows for the usage of scoring models and values within queries, with scoring being completely implementation-defined.<br />
<br />
The scoring model of BaseX takes into consideration the number of found terms, their frequency in a text, and the length of a text. The shorter the input text is, the higher scores will be:<br />
<br />
<pre lang='xquery'><br />
(: Score values: 1 0.62 0.45 :)<br />
for $text in ("A", "A B", "A B C")<br />
let score $score := $text contains text "A"<br />
order by $score descending<br />
return <hit score='{ format-number($score, "0.00") }'>{ $text }</hit><br />
</pre><br />
<br />
This simple approach has proven to consistently deliver good results, in particular when little is known about the structure of the queried XML documents.<br />
<br />
Scoring values can be further processed to compute custom values:<br />
<br />
<pre lang='xquery'><br />
let $terms := ('a', 'b')<br />
let $scores := ft:score($terms ! ('a b c' contains text { . }))<br />
return avg($scores)<br />
</pre><br />
<br />
Scoring is supported within full-text expressions, by {{Function|Full-Text|ft:search}}, and by simple predicate tests that can be rewritten to {{Function|Full-Text|ft:search}}:<br />
<br />
<pre lang='xquery'><br />
let $string := 'a b'<br />
return ft:score($string contains text 'a' ftand 'b'),<br />
<br />
for $n score $s in ft:search('factbook', 'orthodox')<br />
order by $s descending<br />
return $s || ': ' || $n,<br />
<br />
for $n score $s in db:get('factbook')//text()[. contains text 'orthodox']<br />
order by $s descending<br />
return $s || ': ' || $n<br />
</pre><br />
<br />
==Thesaurus==<br />
<br />
One or more thesaurus files can be specified in a full-text expression. The following query returns {{Code|false}}:<br />
<br />
<pre lang='xquery'><br />
'hardware' contains text 'computers'<br />
using thesaurus default<br />
</pre><br />
<br />
If a thesaurus is employed…<br />
<br />
<pre lang="xml"><br />
<thesaurus xmlns="http://www.w3.org/2007/xqftts/thesaurus"><br />
<entry><br />
<term>computers</term><br />
<synonym><br />
<term>hardware</term><br />
<relationship>NT</relationship><br />
</synonym><br />
</entry><br />
</thesaurus><br />
</pre><br />
<br />
…the result will be {{Code|true}}:<br />
<br />
<pre lang='xquery'><br />
'hardware' contains text 'computers'<br />
using thesaurus at 'thesaurus.xml'<br />
</pre><br />
<br />
Thesaurus files must comply with the [https://dev.w3.org/2007/xpath-full-text-10-test-suite/TestSuiteStagingArea/TestSources/thesaurus.xsd XSD Schema] of the XQFT Test Suite (but the namespace can be omitted). Apart from the relationship defined in [https://www.iso.org/standard/7776.html ISO 2788] (NT: narrower team, RT: related term, etc.), custom relationships can be used.<br />
<br />
The type of relationship and the level depth can be specified as well:<br />
<br />
<pre lang='xquery'><br />
(: BT: find broader terms; NT means narrower term :)<br />
'computers' contains text 'hardware'<br />
using thesaurus at 'x.xml' relationship 'BT' from 1 to 10 levels<br />
</pre><br />
<br />
More details can be found in the [https://www.w3.org/TR/xpath-full-text-10/#ftthesaurusoption specification].<br />
<br />
==Fuzzy Querying==<br />
<br />
In addition to the official recommendation, BaseX supports a fuzzy search feature. The XQFT grammar was enhanced by the <code>fuzzy</code> match option to allow for approximate results in full texts:<br />
<br />
'''Document 'doc.xml'''':<br />
<pre lang="xml"><br />
<doc><br />
<a>house</a><br />
<a>hous</a><br />
<a>haus</a><br />
</doc><br />
</pre> <br />
<br />
'''Query:'''<br />
<pre lang='xquery'><br />
//a[text() contains text 'house' using fuzzy]<br />
</pre><br />
<br />
'''Result:'''<br />
<pre lang="xml"><br />
<a>house</a><br />
<a>hous</a><br />
</pre><br />
<br />
Fuzzy search is based on the Levenshtein distance. The maximum number of allowed errors is calculated by dividing the token length of a specified query term by 4. The query above yields two results as there is no error between the query term “house” and the text node “house”, and one error between “house” and “hous”.<br />
<br />
A user-defined value can be adjusted globally via the {{Option|LSERROR}} option or via an additional argument:<br />
<br />
<pre lang='xquery'><br />
//a[text() contains text 'house' using fuzzy 3 errors]<br />
</pre><br />
<br />
=Mixed Content=<br />
<br />
When working with so-called narrative XML documents, such as HTML, [https://tei-c.org/ TEI], or [https://docbook.org/ DocBook] documents, you typically have ''mixed content'', i.e., elements containing a mix of text and markup, such as:<br />
<br />
<pre lang="xml"><br />
<p>This is only an illustrative <hi>example</hi>, not a <q>real</q> text.</p><br />
</pre><br />
<br />
Since the logical flow of the text is not interrupted by the child elements, you will typically want to search across elements, so that the above paragraph would match a search for “real text”. For more examples, see [https://www.w3.org/TR/xpath-full-text-10-use-cases/#Across XQuery and XPath Full Text 1.0 Use Cases].<br />
<br />
To enable this kind of searches, it is recommendable to:<br />
<br />
* Keep ''whitespace stripping'' turned off when importing XML documents. This can be done by ensuring that {{Option|STRIPWS}} is disabled. This can also be done in the GUI if a new database is created (''Database'' → ''New…'' → ''Parsing'' → ''Strip Whitespaces'').<br />
* Keep automatic indentation turned off. Ensure that the [[Serialization|serialization parameter]] {{Code|indent}} is set to {{Code|no}}.<br />
<br />
A query such as <code>//p[. contains text 'real text']</code> will then match the example paragraph above. However, the full-text index will '''not''' be used in this query, so it may take a long time. The full-text index would be used for the query <code>//p[text() contains text 'real text']</code>, but this query will not find the example paragraph because the matching text is split over two text nodes.<br />
<br />
Note that the node structure is ignored by the full-text tokenizer: The {{Code|contains text}} expression applies all full-text operations to the ''string value'' of its left operand. As a consequence, the {{Function|Full-Text|ft:mark}} and {{Function|Full-Text|ft:extract}} functions will only yield useful results if they are applied to single text nodes, as the following example demonstrates:<br />
<br />
<pre lang='xquery'><br />
(: Structure is ignored; no highlighting: :)<br />
ft:mark(//p[. contains text 'real'])<br />
(: Single text nodes are addressed: results will be highlighted: :)<br />
ft:mark(//p[.//text() contains text 'real'])<br />
</pre><br />
<br />
BaseX does '''not''' support the ''ignore option'' (<code>without content</code>) of the [https://www.w3.org/TR/xpath-full-text-10/#ftignoreoption W3C XQuery Full Text 1.0] Recommendation. If you want to ignore descendant element content, such as footnotes or other material that does not belong to the same logical text flow, you can build a second database from and exclude all information you want to avoid searching for. See the following example (visit [[XQuery Update]] to learn more about updates):<br />
<br />
<pre lang='xquery'><br />
let $docs := db:get('docs')<br />
return db:create(<br />
'index-db',<br />
$docs update delete node (<br />
.//footnote<br />
),<br />
$docs/db:path(.),<br />
map { 'ftindex': true() }<br />
)<br />
</pre><br />
<br />
=Functions=<br />
<br />
Some additional [[Full-Text Module|Full-Text Functions]] have been added to BaseX to extend the official language recommendation with useful features, such as explicitly requesting the score value of an item, marking the hits of a full-text request, or directly accessing the full-text index with the default index options.<br />
<br />
=Collations=<br />
<br />
See [[XQuery 3.1#Collations|XQuery 3.1]] for standard collation features.<br />
<br />
By default, string comparisons in XQuery are based on the Unicode codepoint order. The default namespace URI {{Code|http://www.w3.org/2003/05/xpath-functions/collation/codepoint}} specifies this ordering. In BaseX, the following URI syntax is supported to specify collations:<br />
<br />
<nowiki>http://basex.org/collation?lang=...;strength=...;decomposition=...</nowiki><br />
<br />
Semicolons can be replaced with ampersands; for convenience, the URL can be reduced to its ''query string component'' (including the question mark). All arguments are optional:<br />
<br />
{| class="wikitable"<br />
|-<br />
! width="190" | Argument<br />
! Description<br />
|-<br />
| {{Code|lang}}<br />
| A language code, selecting a Locale. It may be followed by a language variant. If no language is specified, the system’s default will be chosen. Examples: {{Code|de}}, {{Code|en-US}}.<br />
|-<br />
| {{Code|strength}}<br />
| Level of difference considered significant in comparisons. Four strengths are supported: {{Code|primary}}, {{Code|secondary}}, {{Code|tertiary}}, and {{Code|identical}}. As an example, in German:<br />
* "Ä" and "A" are considered primary differences,<br />
* "Ä" and "ä" are secondary differences,<br />
* "Ä" and "A&amp;#x308;" (see http://www.fileformat.info/info/unicode/char/308/index.htm) are tertiary differences, and<br />
* "A" and "A" are identical.<br />
|-<br />
| {{Code|decomposition}}<br />
| Defines how composed characters are handled. Three decompositions are supported: {{Code|none}}, {{Code|standard}}, and {{Code|full}}. More details are found in the [https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/text/Collator.html JavaDoc] of the JDK.<br />
|}<br />
<br />
'''Some Examples:'''<br />
<br />
* If a default collation is specified, it applies to all collation-dependent string operations in the query. The following expression yields <code>true</code>:<br />
<br />
<pre lang='xquery'><br />
declare default collation 'http://basex.org/collation?lang=de;strength=secondary';<br />
'Straße' = 'Strasse'<br />
</pre><br />
<br />
* Collations can also be specified in {{Code|order by}} and {{Code|group by}} clauses of FLWOR expressions. This query returns {{Code|à plutôt! bonjour!}}:<br />
<br />
<pre lang='xquery'><br />
for $w in ("bonjour!", "à plutôt!") order by $w collation "?lang=fr" return $w<br />
</pre><br />
<br />
* Various string function exists that take an optional collation as argument: The following functions give us {{Code|a}} and {{Code|1 2 3}} as results:<br />
<br />
<pre lang='xquery'><nowiki><br />
distinct-values(("a", "á", "à"), "?lang=it-IT;strength=primary"),<br />
index-of(("a", "á", "à"), "a", "?lang=it-IT;strength=primary")<br />
</nowiki></pre><br />
<br />
If the [http://site.icu-project.org/download ICU Library] is added to the classpath, the full [https://www.w3.org/TR/xpath-functions-31/#uca-collations Unicode Collation Algorithm] features become available:<br />
<br />
<pre lang='xquery'><br />
(: returns 0 (both strings are compared as equal) :)<br />
compare('a-b', 'ab', 'http://www.w3.org/2013/collation/UCA?alternate=shifted')<br />
</pre><br />
<br />
=Changelog=<br />
<br />
; Version 9.6<br />
* Updated: [[#Fuzzy_Querying|Fuzzy Querying]]: Specify Levenshtein error<br />
<br />
; Version 9.5:<br />
* Removed: Scoring propagation.<br />
<br />
; Version 9.2:<br />
* Added: Arabic stemmer.<br />
<br />
; Version 8.0:<br />
* Updated: [[#Scoring|Scores]] will be propagated by the {{Code|and}} and {{Code|or}} expressions and in predicates.<br />
<br />
; Version 7.7:<br />
* Added: [[#Collations|Collations]] support.<br />
<br />
; Version 7.3:<br />
* Removed: Trie index, which was specialized on wildcard queries. The fuzzy index now supports both wildcard and fuzzy queries.<br />
* Removed: TF/IDF scoring was discarded in favor of the internal scoring model.</div>CGhttps://docs.basex.org/index.php?title=XQuery_3.0&diff=16873XQuery 3.02023-12-01T16:38:39Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div>This article is part of the [[XQuery|XQuery Portal]]. It provides a summary of the most important features of the [https://www.w3.org/TR/xquery-30/ XQuery 3.0] Recommendation.<br />
<br />
=Enhanced FLWOR Expressions=<br />
<br />
Most clauses of FLWOR expressions can be specified in an arbitrary order: additional {{Code|let}} and {{Code|for}} clauses can be put after a {{Code|where}} clause, and multiple {{Code|where}}, {{Code|order by}} and {{Code|group by}} statements can be used. This means that many nested loops can now be rewritten to a single FLWOR expression.<br />
<br />
'''Example:''' <br />
<pre lang='xquery'><br />
for $country in db:get('factbook')//country<br />
where $country/@population > 100000000<br />
for $city in $country//city[population > 1000000]<br />
group by $name := $country/name[1]<br />
count $id<br />
return <country id='{ $id }' name='{ $name }'>{ $city/name }</country><br />
</pre><br />
<br />
==group by==<br />
<br />
FLWOR expressions have been extended to include the [https://www.w3.org/TR/xquery-30/#id-group-by group by] clause, which is well-established in SQL. <code>group by</code> can be used to apply value-based partitioning to query results:<br />
<br />
'''XQuery:''' <br />
<pre lang='xquery'><br />
for $ppl in doc('xmark')//people/person <br />
let $ic := $ppl/profile/@income<br />
let $income :=<br />
if($ic < 30000) then<br />
"challenge" <br />
else if($ic >= 30000 and $ic < 100000) then <br />
"standard" <br />
else if($ic >= 100000) then <br />
"preferred" <br />
else <br />
"na" <br />
group by $income<br />
order by $income<br />
return element { $income } { count($ppl) }<br />
</pre> <br />
<br />
This query is a rewrite of [https://www.ins.cwi.nl/projects/xmark/Assets/xmlquery.txt Query #20] contained in the [https://projects.cwi.nl/xmark/ XMark Benchmark Suite] to use <code>group by</code>.<br />
The query partitions the customers based on their income. <br />
<br />
'''Result:''' <br />
<pre lang="xml"><br />
<challenge>4731</challenge><br />
<na>12677</na><br />
<preferred>314</preferred><br />
<standard>7778</standard><br />
</pre><br />
<br />
In contrast to the relational GROUP BY statement, the XQuery counterpart concatenates the values of all non-grouping variables that belong to a specific group. In the context of our example, all nodes in <code>//people/person</code> that belong to the <code>preferred</code> partition are concatenated in <code class="brush:xquery">$ppl</code> after grouping has finished. You can see this effect by changing the return statement to:<br />
<br />
<pre lang='xquery'> <br />
...<br />
return element { $income } { $ppl }<br />
</pre><br />
<br />
'''Result:'''<br />
<pre lang="xml"><br />
<challenge><br />
<person id="person0"><br />
<name>Kasidit Treweek</name><br />
…<br />
<person id="personX"><br />
…<br />
</challenge><br />
</pre><br />
<br />
Moreover, a value can be assigned to the grouping variable. This is shown in the following example:<br />
<br />
'''XQuery:''' <br />
<pre lang='xquery'><br />
let $data :=<br />
<xml><br />
<person country='USA' name='John'/><br />
<person country='USA' name='Jack'/><br />
<person country='Germany' name='Johann'/><br />
</xml><br />
for $person in $data/person<br />
group by $country := $person/@country<br />
return element persons {<br />
attribute country { $country },<br />
for $name in $person/@name<br />
return element name { data($name) }<br />
}<br />
</pre><br />
<br />
'''Result:'''<br />
<pre lang="xml"><br />
<persons country="USA"><br />
<name>John</name><br />
<name>Jack</name><br />
</persons><br />
<persons country="Germany"><br />
<name>Johann</name><br />
</persons><br />
</pre><br />
<br />
==count==<br />
<br />
The {{Code|count}} clause enhances the FLWOR expression with a variable that enumerates the iterated tuples.<br />
<br />
<pre lang='xquery'><br />
for $n in (1 to 10)[. mod 2 = 1]<br />
count $c<br />
return <number count="{ $c }" number="{ $n }"/><br />
</pre><br />
<br />
==allowing empty==<br />
<br />
The {{Code|allowing empty}} provides functionality similar to outer joins in SQL:<br />
<br />
<pre lang='xquery'><br />
for $n allowing empty in ()<br />
return 'empty? ' || empty($n)<br />
</pre><br />
<br />
==window==<br />
<br />
Window clauses provide a rich set of variable declarations to process sub-sequences of iterated tuples. An example:<br />
<br />
<pre lang='xquery'><br />
for tumbling window $w in (2, 4, 6, 8, 10, 12, 14)<br />
start at $s when true()<br />
only end at $e when $e - $s eq 2<br />
return <window>{ $w }</window><br />
</pre><br />
<br />
More information on window clauses, and all other enhancements, can be found in the [https://www.w3.org/TR/xquery-30/#id-windows specification].<br />
<br />
=Function Items=<br />
<br />
One of the most distinguishing features added in ''XQuery 3.0'' are ''function items'', also known as ''lambdas'' or ''lambda functions''. They make it possible to abstract over functions and thus write more modular code.<br />
<br />
'''Examples:'''<br />
<br />
Function items can be obtained in three different ways:<br />
<br />
<ul><br />
<li>Declaring a new ''inline function'':<br />
<pre lang='xquery'>let $f := function($x, $y) { $x + $y }<br />
return $f(17, 25)</pre> <br />
'''Result:''' <code>42</code><br />
</li><br />
<li>Getting the function item of an existing (built-in or user-defined) XQuery function. The arity (number of arguments) has to be specified as there can be more than one function with the same name:<br />
<pre lang='xquery'>let $f := math:pow#2<br />
return $f(5, 2)</pre> <br />
'''Result:''' <code>25</code><br />
</li><br />
<li>''Partially applying'' another function or function item. This is done by supplying only some of the required arguments, writing the placeholder <code>?</code> in the positions of the arguments left out. The produced function item has one argument for every placeholder.<br />
<pre lang='xquery'>let $f := substring(?, 1, 3)<br />
return (<br />
$f('foo123'),<br />
$f('bar456')<br />
)</pre> <br />
'''Result:''' <code>foo bar</code><br />
</li><br />
</ul><br />
<br />
Function items can also be passed as arguments to and returned as results from functions. These so-called [[Higher-Order Functions]] like <code>for-each</code> and <code>fold-left</code> are discussed in more depth on their own Wiki page.<br />
<br />
=Simple Map Operator=<br />
<br />
The [https://www.w3.org/TR/xquery-30/#id-map-operator simple map] operator {{Code|!}} provides a compact notation for applying the results of a first to a second expression: the resulting items of the first expression are bound to the context item one by one, and the second expression is evaluated for each item. The map operator may be used as replacement for FLWOR expressions:<br />
<br />
'''Example:''' <br />
<pre lang='xquery'><br />
(: Simple map notation :)<br />
(1 to 10) ! element node { . },<br />
(: FLWOR notation :)<br />
for $i in 1 to 10<br />
return element node { $i }<br />
</pre><br />
<br />
In contrast to path expressions, the results of the map operator will not be made duplicate-free and returned in document order.<br />
<br />
=Try/Catch=<br />
<br />
The [https://www.w3.org/TR/xquery-30/#id-try-catch try/catch] construct can be used to handle errors at runtime:<br />
<br />
'''Example:''' <br />
<pre lang='xquery'><br />
try {<br />
1 + '2'<br />
} catch err:XPTY0004 {<br />
'Typing error: ' || $err:description<br />
} catch * {<br />
'Error [' || $err:code || ']: ' || $err:description<br />
}<br />
</pre><br />
'''Result:''' <code>Typing error: '+' operator: number expected, xs:string found.</code><br />
<br />
Within the scope of the catch clause, a number of variables are implicitly declared, giving information about the error that occurred:<br />
<br />
* {{Code|$err:code}} error code<br />
* {{Code|$err:description}}: error message<br />
* {{Code|$err:value}}: value associated with the error (optional)<br />
* {{Code|$err:module}}: URI of the module where the error occurred<br />
* {{Code|$err:line-number}}: line number where the error occurred<br />
* {{Code|$err:column-number}}: column number where the error occurred<br />
<br />
=Switch=<br />
<br />
The [https://www.w3.org/TR/xquery-30/#id-switch switch] statement is available in many other programming languages. It chooses one of several expressions to evaluate based on its input value.<br />
<br />
'''Example:''' <br />
<pre lang='xquery'><br />
for $fruit in ("Apple", "Pear", "Peach")<br />
return switch ($fruit)<br />
case "Apple" return "red"<br />
case "Pear" return "green"<br />
case "Peach" return "pink"<br />
default return "unknown"<br />
</pre> <br />
'''Result:''' <code>red green pink</code><br />
<br />
The expression to evaluate can correspond to multiple input values.<br />
<br />
'''Example:'''<br />
<pre lang='xquery'><br />
for $fruit in ("Apple", "Cherry")<br />
return switch ($fruit)<br />
case "Apple"<br />
case "Cherry"<br />
return "red"<br />
case "Pear"<br />
return "green"<br />
case "Peach"<br />
return "pink"<br />
default<br />
return "unknown"<br />
</pre><br />
'''Result:''' <code>red red</code><br />
<br />
=Expanded QNames=<br />
<br />
A ''QName'' can be prefixed with the letter {{Code|Q}}, the namespace URI wrapped in curly braces and the local name.<br />
<br />
'''Examples:'''<br />
* <code><nowiki>Q{http://www.w3.org/2005/xpath-functions/math}pi()</nowiki></code> returns the number π<br />
* <code>Q{java:java.io.FileOutputStream}new("output.txt")</code> creates a new Java file output stream<br />
<br />
=Namespace Constructors=<br />
<br />
New namespaces can be created via so-called 'Computed Namespace Constructors'.<br />
<br />
<pre lang='xquery'> <br />
element node { namespace pref { 'http://url.org/' } }<br />
</pre><br />
<br />
=String Concatenations=<br />
<br />
Two vertical bars <code>||</code> (also named ''pipe characters'') can be used to concatenate strings. This operator is a shortcut for the {{Code|concat()}} function.<br />
<br />
<pre lang='xquery'> <br />
'Hello' || ' ' || 'Universe'<br />
</pre><br />
<br />
=External Variables=<br />
<br />
Default values can be attached to external variable declarations. This way, an expression can also be evaluated if its external variables have not been bound to a new value.<br />
<br />
<pre lang='xquery'> <br />
declare variable $user external := "admin";<br />
"User:", $user<br />
</pre><br />
<br />
=Serialization=<br />
<br />
[[Serialization]] parameters can be defined within XQuery expressions. Parameters are placed in the query prolog and need to be specified as option declarations, using the <code>output</code> prefix.<br />
<br />
'''Example:''' <br />
<pre lang='xquery'><br />
declare namespace output = "http://www.w3.org/2010/xslt-xquery-serialization";<br />
declare option output:omit-xml-declaration "no";<br />
declare option output:method "xhtml";<br />
<html/><br />
</pre> <br />
'''Result:''' <code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;&lt;html&gt;&lt;/html&gt;</code><br />
<br />
In BaseX, the {{Code|output}} prefix is statically bound and can thus be omitted. Note that all namespaces need to be specified when using external APIs, such as [http://xqj.net/basex/ XQJ].<br />
<br />
=Context Item=<br />
<br />
The context item can be specified in the prolog of an XQuery expression:<br />
<br />
'''Example:''' <br />
<pre lang='xquery'><br />
declare context item := document {<br />
<xml><br />
<text>Hello</text><br />
<text>World</text><br />
</xml><br />
};<br />
<br />
for $t in .//text()<br />
return string-length($t)<br />
</pre> <br />
'''Result:''' <code>5 5</code><br />
<br />
=Annotations=<br />
<br />
XQuery 3.0 introduces annotations to declare properties associated with functions and variables. For instance, a function may be declared %public, %private, or %updating.<br />
<br />
'''Example:''' <br />
<pre lang='xquery'><br />
declare %private function local:max($x1, $x2) {<br />
if($x1 > $x2) then $x1 else $x2<br />
};<br />
<br />
local:max(2, 3)<br />
</pre><br />
<br />
=Functions=<br />
<br />
The following functions have been added in the [https://www.w3.org/TR/xpath-functions-31/ XQuery 3.0 Functions and Operators] Specification:<br />
<br />
<code>analyze-string</code>, <code>available-environment-variables</code>, <code>element-with-id</code>, <code>environment-variable</code>, <code>filter</code>, <code>fold-left</code>, <code>fold-right</code>, <code>for-each</code>, <code>for-each-pair</code>, <code>format-date</code>, <code>format-dateTime</code>, <code>format-integer</code>, <code>format-number</code>, <code>format-time</code>, <code>function-arity</code>, <code>function-lookup</code>, <code>function-name</code>, <code>generate-id</code>, <code>has-children</code>, <code>head</code>, <code>innermost</code>, <code>outermost</code>, <code>parse-xml</code>, <code>parse-xml-fragment</code>, <code>path</code>, <code>serialize</code>, <code>tail</code>, <code>unparsed-text</code>, <code>unparsed-text-available</code>, <code>unparsed-text-lines</code>, <code>uri-collection</code><br />
<br />
New signatures have been added for the following functions:<br />
<br />
<code>document-uri</code>, <code>string-join</code>, <code>node-name</code>, <code>round</code>, <code>data</code><br />
<br />
=Changelog=<br />
<br />
;Version 7.7<br />
<br />
* Added: [[#Enhanced FLWOR Expressions|Enhanced FLWOR Expressions]]<br />
<br />
;Version 7.3<br />
<br />
* Added: [[#Simple Map Operator|Simple Map Operator]]<br />
<br />
;Version 7.2<br />
<br />
* Added: [[#Annotations|Annotations]]<br />
* Updated: [[#Expanded QNames|Expanded QNames]]<br />
<br />
;Version 7.1<br />
<br />
* Added: [[#Expanded QNames|Expanded QNames]], [[#Namespace Constructors|Namespace Constructors]]<br />
<br />
;Version 7.0<br />
<br />
* Added: [[#String Concatenations|String Concatenations]]<br />
<br />
[[Category:XQuery]]</div>CGhttps://docs.basex.org/index.php?title=Query_Mode&diff=16874Query Mode2023-12-01T16:38:39Z<p>CG: Text replacement - "syntaxhighlight" to "pre"</p>
<hr />
<div><p>The query mode of the [[Clients]] allows you to bind external variables to a query<br />
and evaluate the query in an iterative manner. The <code>query()</code> function of the<br />
<code>Session</code> instance returns a new query instance.</p><br />
<br />
==Usage== <br />
<br />
The query execution works as follows:<br />
<br />
# Create a new session instance with hostname, port, username and password.<br />
# Call <code>query()</code> with your XQuery expression to get a query object.<br />
# Optionally bind variables to the query with one of the <code>bind()</code> functions.<br />
# Optionally bind a value to the context item via <code>context()</code>.<br />
# Iterate through the query object with the <code>more()</code> and <code>next()</code> functions.<br />
# As an alternative, call <code>execute()</code> to get the whole result at a time.<br />
# <code>info()</code> gives you information on query evaluation.<br />
# <code>options()</code> returns the query serialization parameters.<br />
# Don't forget to close the query with <code>close()</code>.<br />
<br />
==PHP Example==<br />
<br />
Taken from our [https://github.com/BaseXdb/basex/blob/master/basex-api/src/main/php/QueryBindExample.php repository]:<br />
<br />
<pre lang="php"><br />
<?php<br />
/*<br />
* This example shows how queries can be executed in an iterative manner.<br />
* Documentation: http://basex.org/api<br />
*<br />
* (C) BaseX Team 2005-15, BSD License<br />
*/<br />
include("BaseXClient.php");<br />
<br />
try {<br />
// create session<br />
$session = new Session("localhost", 1984, "admin", "...");<br />
<br />
try {<br />
// create query instance<br />
$input = 'declare variable $name external; '.<br />
'for $i in 1 to 10 return element { $name } { $i }';<br />
$query = $session->query($input);<br />
<br />
// bind variable<br />
$query->bind("$name", "number");<br />
<br />
// print result<br />
print $query->execute()."\n";<br />
<br />
// close query instance<br />
$query->close();<br />
<br />
} catch (Exception $e) {<br />
// print exception<br />
print $e->getMessage();<br />
}<br />
<br />
// close session<br />
$session->close();<br />
<br />
} catch (Exception $e) {<br />
// print exception<br />
print $e->getMessage();<br />
}<br />
?><br />
</pre><br />
<br />
=Changelog=<br />
<br />
;Version 7.2<br />
<br />
* Added: {{Code|context()}} function</div>CG