Database Module

From BaseX Documentation

Jump to: navigation, search

This XQuery Module contains functions for processing databases from within XQuery. Existing databases can be opened and listed, its contents can be directly accessed, documents can be added to and removed, etc.

Contents

Conventions

All functions and errors in this module are assigned to the http://basex.org/modules/db namespace, which is statically bound to the db prefix.

Database Nodes

Database nodes are XML nodes which are either stored in a persistent database, or which are a node of a main-memory database representation. XML fragments can be converted to a main-memory database by e. g. applying the update or transform expression on a node:

db:node-id(element hello { 'world' } update {})

General Functions

db:system

Signatures db:system() as element(system)
Summary Returns general information on the database system the current values of all global and local Options. The INFO command returns similar output.

db:option

Signatures db:option($name as xs:string) as xs:string
Summary Returns the current value (string, integer, boolean, map) of a global or local Option with the specified $name. The GET command works similar.
Examples
  • db:option('dbpath') returns the database path string.
  • db:option('serializer') returns a map with the current serialization parameters.
  • declare option db:chop 'true'; db:option('chop') returns true (irrespective of the global value).

db:info

Signatures db:info($db as xs:string) as element(database)
Summary Returns meta information on the database $db. The output is similar to the INFO DB command.
Errors open: the addressed database does not exist or could not be opened.

db:property

Signatures db:property($db as xs:string, $name as xs:string) as xs:anyAtomicType
Summary Returns the value (string, boolean, integer) of a property with the specified $name in the database $db. The available properties are the ones returned by db:info.
Errors property: the specified property is unknown.
Examples
  • db:property('db', 'size') returns the number of bytes occupied by the database db.
  • db:property('xmark', 'textindex') indicates if the xmark database has a text index.
  • db:property('discogs', 'uptodate') indicates if the database statistics and index structures of the discogs database are up-to-date.

db:list

Signatures db:list() as xs:string*
db:list($db as xs:string) as xs:string*
db:list($db as xs:string, $path as xs:string) as xs:string*
Summary The result of this function is dependent on the number of arguments:
  • Without arguments, the names of all databases are returned that are accessible to the current user.
  • If a database $db is specified, all documents and raw files of the specified database are returned.
  • The list of returned resources can be restricted by the $path argument.
Errors open: the addressed database does not exist or could not be opened.
Examples
  • db:list("docs") returns the names of all documents of a database named docs.

db:list-details

Signatures db:list-details() as element(database)*
db:list-details($db as xs:string) as element(resource)*
db:list-details($db as xs:string, $path as xs:string) as element(resource)*
Summary Without arguments, an element is returned for each database that is accessible to the current user:
  • An element has a value, which is the name of the database, and several attributes, which contain the number of stored resources, the modification date, the database size on disk (measured in bytes), and a path to the original database input.

If a database $db is specified, an element for each documents and raw file of the specified database is returned:

  • An element has a value, which is the name of the resource, and several attributes, which contain the content type, the modification date, the raw flag (which indicates if the resource is binary or XML), and the size of a resource.
  • The value of the size attribute depends on the resource type: for documents, it represents the number of nodes; for binary data, it represents the file size (measured in bytes).
  • Returned databases resources can be further restricted by the $path argument.
Errors open: the addressed database does not exist or could not be opened.
Examples
  • db:list-details("shop") returns the names plus additional info on all resources of a database named shop.

db:backups

Signatures db:backups() as element(backup)*
db:backups($db as xs:string) as element(backup)*
Summary Returns an element sequence containing all available database backups.
If a database $db is specified, the sequence will be restricted to the backups matching this database.
Examples
  • db:backups("factbook") returns all backups that have been made from the factbook database.

Read Operations

db:open

Signatures db:open($db as xs:string) as document-node()*
db:open($db as xs:string, $path as xs:string) as document-node()*
Summary Opens the database $db and returns all document nodes.
The document nodes to be returned can be filtered with the $path argument.
Errors open: the addressed database does not exist or could not be opened.
Examples
  • db:open("docs") returns all documents from the database named docs.
  • db:open("db", "one") returns all documents from the database named db located in the path one.
  • for $i in 1 to 3 return db:open("db" || $i)//item returns all item elements from the databases db1, db2 and db3.

db:open-pre

Signatures db:open-pre($db as xs:string, $pre as xs:integer) as node()
Summary Opens the database $db and returns the node with the specified $pre value.
The PRE value provides very fast access to an existing database node, but it will change whenever a node with a smaller pre values is added to or deleted from a database.
Errors open: the addressed database does not exist or could not be opened.
range: the specified pre value does not exist in the database.
Examples
  • db:open-pre("docs", 0) returns the first database node from the database named docs.

db:open-id

Signatures db:open-id($db as xs:string, $id as xs:integer) as node()
Summary Opens the database $db and returns the node with the specified $id value.
Each database node has a persistent ID value. Access to the node id can be sped up by turning on the UPDINDEX option.
Errors open: the addressed database does not exist or could not be opened.
range: the specified id value does not exist in the database.

db:node-pre

Signatures db:node-pre($nodes as node()*) as xs:integer*
Summary Returns the pre values of the nodes supplied by $nodes, which must all be database nodes.
The PRE value provides very fast access to an existing database node, but it will change whenever a node with a smaller pre values is added to or deleted from a database.
Errors node: $nodes contains a node which is not stored in a database.
Examples
  • db:node-pre(doc("input")) returns 0 if the database input contains a single document.

db:node-id

Signatures db:node-id($nodes as node()*) as xs:integer*
Summary Returns the id values of the nodes supplied by $nodes, which must all be database nodes.
Each database node has a persistent ID value. Access to the node id can be sped up by turning on the UPDINDEX option.
Errors node: $nodes contains a node which is not stored in a database.

db:retrieve

Signatures db:retrieve($db as xs:string, $path as xs:string) as xs:base64Binary
Summary Returns a binary resource addressed by the database $db and $path as streamable xs:base64Binary.
Errors open: the addressed database does not exist or could not be opened.
mainmem: the database is not persistent (stored on disk).
Examples
  • db:retrieve("DB", "music/01.mp3") returns the specified audio file as raw data.
  • stream:materialize(db:retrieve("DB", "music/01.mp3")) materializes the streamable result in main-memory before returning it.
  • convert:binary-to-string(db:retrieve("DB", "info.txt"), 'UTF-8') converts a binary database resource as UTF-8 text and returns a string.

db:export

Signatures db:export($db as xs:string, $path as xs:string) as empty-sequence()
db:export($db as xs:string, $path as xs:string, $params as item()) as empty-sequence()
Summary Exports the specified database $db to the specified file $path. Existing files will be overwritten. The $params argument contains serialization parameters (see Serialization for more details), which can either be specified
  • as children of an <output:serialization-parameters/> element, as defined for the fn:serialize() function; e.g.:
<output:serialization-parameters>
  <output:method value='xml'/>
  <output:cdata-section-elements value="div"/>
  ...
</output:serialization-parameters>
  • as map, which contains all key/value pairs:
map { "method": "xml", "cdata-section-elements": "div", ... }
Errors open: the addressed database does not exist or could not be opened.
Examples Export all files as text:
db:export("DB", "/home/john/xml/texts", map { 'method': 'text' })

The following query can be used to export parts of the database:

let $target := '/home/john/xml/target'
for $doc in db:open('DB', 'collection')
let $path := $target || db:path($doc)
return (
  file:create-dir(file:parent($path)),
  file:write($path, $doc)
)

Value Indexes

db:text

Updated with Version 9.1: Support for multiple string values.

Signatures db:text($db as xs:string, $strings as xs:string*) as text()*
Summary Returns all text nodes of the database $db that have one of the specified $strings as values and that are stored in the text index.
Errors open: the addressed database does not exist or could not be opened.
no-index: the index is not available.
Examples
  • db:text("DB", "QUERY")/.. returns the parents of all text nodes of the database DB that match the string QUERY.

db:text-range

Signatures db:text-range($db as xs:string, $min as xs:string, $max as xs:string) as text()*
Summary Returns all text nodes of the database $db that are located in between the $min and $max strings and that are stored in the text index.
Errors open: the addressed database does not exist or could not be opened.
no-index: the index is not available.
Examples
  • db:text-range("DB", "2000", "2001") returns all text nodes of the database DB that are found in between 2000 and 2001.

db:attribute

Updated with Version 9.1: Support for multiple string values.

Signatures db:attribute($db as xs:string, $strings as as xs:string*) as attribute()*
db:attribute($db as xs:string, $strings as xs:strings*, $name as xs:string) as attribute()*
Summary Returns all attribute nodes of the database $db that have one of the specified $strings as values and that are stored in the attribute index.
If $name is specified, the resulting attribute nodes are filtered by their attribute name.
Errors open: the addressed database does not exist or could not be opened.
no-index: the index is not available.
Examples
  • db:attribute("DB", "QUERY", "id")/.. returns the parents of all id attribute nodes of the database DB that have QUERY as string value.

db:attribute-range

Signatures db:attribute-range($db as xs:string, $min as xs:string, $max as xs:string) as attribute()*
db:attribute-range($db as xs:string, $min as xs:string, $max as xs:string, $name as xs:string) as attribute()*
Summary Returns all attributes of the database $db, the string values of which are larger than or equal to $min and smaller than or equal to $max and that are stored in the attribute index.
Errors open: the addressed database does not exist or could not be opened.
no-index: the index is not available.
Examples
  • db:attribute-range("DB", "id456", "id473", 'id') returns all @id attributes of the database DB that have a string value in between id456 and id473.

db:token

Updated with Version 9.1: Support for multiple string values.

Signatures db:token($db as xs:string, $tokens as xs:string*) as attribute()*
db:token($db as xs:string, $tokens as xs:string*, $name as xs:string) as attribute()*
Summary Returns all attribute nodes of the database $db the values of which contain one of the specified $tokens.
If $name is specified, the resulting attribute nodes are filtered by their attribute name.
Errors open: the addressed database does not exist or could not be opened.
no-index: the index is not available.
Examples
  • db:token("DB", "row", "class")/parent::div returns all div nodes of database DB with a class attribute that contains the token row.

Updates

Important note: All functions in this section are updating functions: they will not be immediately executed, but queued on the Pending Update List, which will be processed after the actual query has been evaluated. This means that the order in which the functions are specified in the query does usually not reflect the order in which the code will be evaluated.

db:create

Signatures db:create($db as xs:string) as empty-sequence()
db:create($db as xs:string, $inputs as item()*) as empty-sequence()
db:create($db as xs:string, $inputs as item()*, $paths as xs:string*) as empty-sequence()
db:create($db as xs:string, $inputs as item()*, $paths as xs:string*, $options as map(*)?) as empty-sequence()
Summary Creates a new database with name $db and adds initial documents specified via $inputs to the specified $paths:
  • $inputs may be strings or nodes:
    • nodes may be of any type except for attributes
    • strings can be a URI pointing to a file/directory or an XML string (which is detected by the leading < character)
    • a path must be specified if the input is not a file or directory reference
  • The parsing and indexing behavior can be controlled via $options:
  • An existing database will be overwritten.
  • Database creation takes place after most other update operations (see Pending Update List). As a consequence, a newly created database cannot be addressed in the same query.
Errors lock: a database is opened by another process.
name: the specified name is not a valid database name.
conflict: the same database was addressed more than once.
args: the number of specified inputs and paths differs.
Examples
  • db:create("DB") creates the empty database DB.
  • db:create("DB", "/home/dir/doc.xml") creates the database DB and adds the document /home/dir/doc.xml as initial content.
  • db:create("DB", <a/>, "doc.xml") creates the database DB and adds the document with content <a/> under the name doc.xml.
  • db:create("DB", "/home/dir/", "docs/dir") creates the database DB and adds the documents in /home/dir to the database under the path docs/dir.
  • db:create("DB", file:list('.'), (), map { 'ftindex': true() }) adds all files of the current working directory to a new database, preserving relative filesystem paths and creating a full-text index.

db:drop

Signatures db:drop($db as xs:string) as empty-sequence()
Summary Drops the database $db and all connected resources.
Errors open: the addressed database does not exist or could not be opened.
lock: a database is opened by another process.
conflict: the same database was addressed more than once.
Examples
  • db:drop("DB") drops the database DB.

db:add

Signatures db:add($db as xs:string, $input as item()) as empty-sequence()
db:add($db as xs:string, $input as item(), $path as xs:string) as empty-sequence()
db:add($db as xs:string, $input as item(), $path as xs:string, $options as map(*)?) as empty-sequence()
Summary Adds documents specified by $input to the database $db with the specified $path:
  • A document with the same path may occur more than once in a database. If you want to enforce single instances, use db:replace instead.
  • See db:create for more details on the input argument.
  • The parsing behavior can be controlled via $options:
    • allowed options are ADDCACHE and the parsing and XML parsing options, all in lower case
    • parsing options will only impact string input (URIs, XML strings), because nodes have already been parsed
Errors open: the addressed database does not exist or could not be opened.
Examples
  • db:add("DB", "/home/dir/doc.xml") adds the file /home/dir/doc.xml to the database DB.
  • db:add("DB", <a/>, "doc.xml") adds a document node to the database DB under the name doc.xml.
  • db:add("DB", "/home/dir", "docs/dir", map { 'addcache': true() }) adds all documents in /home/dir to the database DB under the path docs/dir. In order to reduce memory consumption, the files will be cached before being added to the database.

db:delete

Signatures db:delete($db as xs:string, $path as xs:string) as empty-sequence()
Summary Deletes resource(s), specified by $path, from the database $db.
Errors open: the addressed database does not exist or could not be opened.
path: the specified path is invalid.
Examples
  • db:delete("DB", "docs/dir/doc.xml") deletes the resource docs/dir/doc.xml from DB.
  • db:delete("DB", "docs/dir") deletes all resources from DB in the specified path docs/dir.

db:copy

Signatures db:copy($db as xs:string, $name as xs:string) as empty-sequence()
Summary Creates a copy of the database $db, which will be called $name.
Errors open: the addressed database does not exist or could not be opened.
lock: a database is opened by another process.
name: invalid database name.
conflict: the same database was addressed more than once.

db:alter

Signatures db:alter($db as xs:string, $name as xs:string) as empty-sequence()
Summary Renames the database $db to $name.
Errors open: the addressed database does not exist or could not be opened.
lock: a database is opened by another process.
name: invalid database name.
conflict: the same database was addressed more than once.

db:create-backup

Signatures db:create-backup($db as xs:string) as empty-sequence()
Summary Creates a backup of the database $db.
Errors open: the addressed database does not exist or could not be opened.
name: invalid database name.
conflict: the same database was addressed more than once.
Examples
  • db:create-backup("DB") creates a backup of the database DB.

db:drop-backup

Signatures db:drop-backup($name as xs:string) as empty-sequence()
Summary Drops all backups of the database with the specified $name. If the given $name points to a specific backup file, only this specific backup file is deleted.
Errors backup: No backup file found.
name: invalid database name.
conflict: the same database was addressed more than once.
Examples
  • db:drop-backup("DB") drops all backups of the database DB.
  • db:drop-backup("DB-2014-03-13-17-36-44") drops the specific backup file DB-2014-03-13-17-36-44.zip of the database DB.

db:restore

Signatures db:restore($name as xs:string) as empty-sequence()
Summary Restores the database with the specified $name. The $name may include the timestamp of the backup file.
Errors lock: a database is opened by another process.
name: invalid database name.
no-backup: No backup found.
conflict: the same database was addressed more than once.
Examples
  • db:restore("DB") restores the database DB.
  • db:restore("DB-2014-03-13-18-05-45") restores the database DB from the backup file with the given timestamp.

db:optimize

Signatures db:optimize($db as xs:string) as empty-sequence()
db:optimize($db as xs:string, $all as xs:boolean) as empty-sequence()
db:optimize($db as xs:string, $all as xs:boolean, $options as map(*)?) as empty-sequence()
Summary Optimizes the meta data and indexes of the database $db.
If $all is true, the complete database will be rebuilt.
The $options argument can be used to control indexing. The syntax is identical to the db:create function: Allowed options are all indexing and full-text options. UPDINDEX is only supported if $all is true.
Errors open: the addressed database does not exist or could not be opened.
Examples
  • db:optimize("DB") optimizes the database structures of the database DB.
  • db:optimize("DB", true(), map { 'ftindex': true() }) optimizes all database structures of the database DB and creates a full-text index.

db:rename

Signatures db:rename($db as xs:string, $source as xs:string, $target as xs:string) as empty-sequence()
Summary Moves all resources(s) of database $db, which are found in the supplied $source path, to the supplied $target path. The paths may point to single resources or directories. No updates will take place if a non-existing source path is supplied.
Errors open: the addressed database does not exist or could not be opened.
path: the specified source or target path, or one of its descendants, is invalid.
Examples
  • db:rename("DB", "docs/dir/doc.xml", "docs/dir/newdoc.xml") renames the resource docs/dir/doc.xml to docs/dir/newdoc.xml in the database DB.
  • db:rename("DB", "docs/dir", "docs/newdir") moves all resources in the database DB from docs/dir to {Code|docs/newdir}}.

db:replace

Signatures db:replace($db as xs:string, $path as xs:string, $input as item()) as empty-sequence()
db:replace($db as xs:string, $path as xs:string, $input as item(), $options as map(*)?) as empty-sequence()
Summary Replaces a resource, specified by $path, in the database $db with the contents of $input, or adds it as a new resource:
  • See db:create for more details on the input argument.
  • The parsing behavior can be controlled via $options:
    • allowed options are ADDCACHE and the parsing and XML parsing options, all in lower case
    • parsing options will only impact string input (URIs, XML strings), because nodes have already been parsed
  • For historical reasons, the order of the 2nd and 3rd argument is different to db:add and db:create
Errors open: the addressed database does not exist or could not be opened.
target: the path points to a directory.
Examples
  • db:replace("DB", "docs/dir/doc.xml", "/home/dir/doc.xml") replaces the content of the document docs/dir/doc.xml in the database DB with the content of the file /home/dir/doc.xml.
  • db:replace("DB", "docs/dir/doc.xml", "<a/>") replaces the content of the document docs/dir/doc.xml in the database DB with <a/>.
  • db:replace("DB", "docs/dir/doc.xml", document { <a/> }) replaces the content of the document docs/dir/doc.xml in the database DB with the specified document node.

The following query can be used to import files from a directory to a database:

let $source := '/home/john/xml/source'
for $file in file:list($source, true())
let $path := $source || $file
where not(file:is-dir($path))
return db:replace('db', $file, doc($path))

db:store

Signatures db:store($db as xs:string, $path as xs:string, $input as item()) as empty-sequence()
Summary Replaces a binary resource specified by $input in the database $db and the location specified by $path, or adds it as new resource.
Errors open: the addressed database does not exist or could not be opened.
mainmem: the database is not persistent (stored on disk).
Examples
  • db:store("DB", "video/sample.mov", file:read-binary('video.mov')) stores the addressed video file at the specified location.
  • With the following query, you can copy full directories:
let $db := 'db'
let $src-path := 'src/'
let $trg-path := 'trg/'
for $src in db:list($db, $src-path)
where db:is-raw($db, $src)
let $trg := $trg-path || substring-after($src, $src-path)
return db:store($db, $trg, db:retrieve($db, $src))

db:flush

Signatures db:flush($db as xs:string) as empty-sequence()
Summary Explicitly flushes the buffers of the database $db. This command is only useful if AUTOFLUSH has been set to false.
Errors open: the addressed database does not exist or could not be opened.

Helper Functions

db:name

Signatures db:name($node as node()) as xs:string
Summary Returns the name of the database in which the specified database node $node is stored.
Errors node: $nodes contains a node which is not stored in a database.

db:path

Signatures db:path($node as node()) as xs:string
Summary Returns the path of the database document in which the specified database node $node is stored.
Errors node: $nodes contains a node which is not stored in a database.

db:exists

Signatures db:exists($db as xs:string) as xs:boolean
db:exists($db as xs:string, $path as xs:string) as xs:boolean
Summary Checks if the database $db or the resource specified by $path exists. false is returned if a database directory has been addressed.
Examples
  • db:exists("DB") returns true if the database DB exists.
  • db:exists("DB", "resource") returns true if resource is an XML document or a raw file.

db:is-raw

Signatures db:is-raw($db as xs:string, $path as xs:string) as xs:boolean
Summary Checks if the specified resource in the database $db and the path $path exists, and if it is a binary resource.
Errors open: the addressed database does not exist or could not be opened.
Examples
  • db:is-raw("DB", "music/01.mp3") returns true.

db:is-xml

Signatures db:is-xml($db as xs:string, $path as xs:string) as xs:boolean
Errors open: the addressed database does not exist or could not be opened.
Summary Checks if the specified resource in the database $db and the path $path exists, and if it is an XML document.
Examples
  • db:is-xml("DB", "dir/doc.xml") returns true.

db:content-type

Signatures db:content-type($db as xs:string, $path as xs:string) as xs:string
Summary Retrieves the content type of a resource in the database $db and the path $path.
The file extension is used to recognize the content-type of a resource stored in the database. Content-type application/xml will be returned for any XML document stored in the database, regardless of its file name extension.
Errors open: the addressed database does not exist or could not be opened.
Examples
  • db:content-type("DB", "docs/doc01.pdf") returns application/pdf.
  • db:content-type("DB", "docs/doc01.xml") returns application/xml.
  • db:content-type("DB", "docs/doc01") returns application/xml, if db:is-xml("DB", "docs/doc01") returns true.

Errors

Code Description
args The number of specified inputs and paths differs.
conflict Multiple update operations point to the same target.
lock A database cannot be updated because it is opened by another process.
mainmem The addressed database is not persistent (stored on disk).
name The name of the specified database is invalid.
no-backup No backup exists for a database.
node The referenced XML node is no database node, i.e. it is neither stored in a database nor represented as database fragment.
no-index The database lacks an index structure required by the called function.
open The addressed database does not exist or could not be opened.
path The specified database path is invalid.
property A database property is unknown.
range The addressed database id or pre value is out of range.
target Path points to an invalid target.

Changelog

Version 9.0
Version 8.6
Version 8.4
Version 8.3
Version 8.2
Version 7.9
Version 7.8.2
Version 7.8
Version 7.7
Version 7.6
Version 7.5
Version 7.3
Version 7.2.1
Version 7.1
Version 7.0
Personal tools
Namespaces
Variants
Actions
Navigation
Print/export