Database Module

From BaseX Documentation
Revision as of 12:49, 27 November 2015 by CG (talk | contribs)
Jump to navigation Jump to search

This XQuery Module contains functions for processing databases from within XQuery. Existing databases can be opened and listed, its contents can be directly accessed, documents can be added to and removed, etc.

Conventions

All functions in this module are assigned to the http://basex.org/modules/db namespace, which is statically bound to the db prefix.
All errors are assigned to the http://basex.org/errors namespace, which is statically bound to the bxerr prefix.

Database Nodes

Database nodes are XML nodes which are either stored in a persistent database or part of a so-called database fragment. All XML fragments can be converted to database fragments by e. g. applying the transform expression on an XML fragment:

copy $c := element hello { 'world' } modify () return $c

General Functions

db:system

Signatures db:system() as element(system)
Summary Returns information on the database system, such as the database path and current database settings. The output is similar to the INFO command.

db:info

Signatures db:info($db as xs:string) as element(database)
Summary Returns meta information on the database $db. The output is similar to the INFO DB command.
Errors BXDB0002: the addressed database does not exist or could not be opened.

db:list

Signatures db:list() as xs:string*
db:list($db as xs:string) as xs:string*
db:list($db as xs:string, $path as xs:string) as xs:string*
Summary Returns a string sequence with the names of all databases:
  • If a database $db is specified, all documents and raw files of the specified database are returned.
  • The list of resources can be further restricted by the $path argument.
Errors BXDB0002: the addressed database does not exist or could not be opened.
Examples
  • db:list("docs") returns the names of all documents from the database named docs.

db:list-details

Signatures db:list-details() as element(database)*
db:list-details($db as xs:string) as element(resource)*
db:list-details($db as xs:string, $path as xs:string) as element(resource)*
Summary
  • If no argument is specified, a sequence of elements is returned. A single element contains the name of a database, the number of stored resources, the date of modification, and the database path.
  • If $db is specified, a sequence of elements is returned, comprising information on all resources of the addressed database. An element contains the name of the resource, the content type, the modified date, the raw flag and the size of a resource.
  • For binary resources, the size is the number of bytes; for document resources, the size is the number of nodes
  • Returned databases resources can be further restricted by the $path argument.
Errors BXDB0002: the addressed database does not exist or could not be opened.
Examples
  • db:list-details("docs") returns the names plus additional data of all documents from the database named docs.

db:backups

Signatures db:backups() as element(backup)*
db:backups($db as xs:string) as element(backup)*
Summary Returns an element sequence containing all available database backups.
If a database $db is specified, the sequence will be restricted to the backups matching this database.
Examples
  • db:backups("factbook") returns all backups that have been made from the factbook database.

Read Operations

db:open

Signatures db:open($db as xs:string) as document-node()*
db:open($db as xs:string, $path as xs:string) as document-node()*
Summary Opens the database $db and returns all document nodes.
The document nodes to be returned can be filtered with the $path argument.
Errors BXDB0002: the addressed database does not exist or could not be opened.
Examples
  • db:open("docs") returns all documents from the database named docs.
  • db:open("db", "one") returns all documents from the database named db located in the path one.
  • for $i in 1 to 3 return db:open("db" || $i)//item returns all item elements from the databases db1, db2 and db3.

db:open-pre

Signatures db:open-pre($db as xs:string, $pre as xs:integer) as node()
Summary Opens the database $db and returns the node with the specified $pre value.
The PRE value provides very fast access to an existing database node, but it will change whenever a node with a smaller pre values is added to or deleted from a database.
Errors BXDB0002: the addressed database does not exist or could not be opened.
BXDB0009: the specified pre value does not exist in the database.
Examples
  • db:open-pre("docs", 0) returns the first database node from the database named docs.

db:open-id

Signatures db:open-id($db as xs:string, $id as xs:integer) as node()
Summary Opens the database $db and returns the node with the specified $id value.
Each database node has a persistent ID value. Access to the node id can be sped up by turning on the UPDINDEX option.
Errors BXDB0002: the addressed database does not exist or could not be opened.
BXDB0009: the specified id value does not exist in the database.

db:node-pre

Signatures db:node-pre($nodes as node()*) as xs:integer*
Summary Returns the pre values of the nodes supplied by $nodes, which must all be database nodes.
The PRE value provides very fast access to an existing database node, but it will change whenever a node with a smaller pre values is added to or deleted from a database.
Errors BXDB0001: $nodes contains a node which is not stored in a database.
Examples
  • db:node-pre(doc("input")) returns 0 if the database input contains a single document.

db:node-id

Signatures db:node-id($nodes as node()*) as xs:integer*
Summary Returns the id values of the nodes supplied by $nodes, which must all be database nodes.
Each database node has a persistent ID value. Access to the node id can be sped up by turning on the UPDINDEX option.
Errors BXDB0001: $nodes contains a node which is not stored in a database.

db:retrieve

Signatures db:retrieve($db as xs:string, $path as xs:string) as xs:base64Binary
Summary Returns a binary resource addressed by the database $db and $path as streamable xs:base64Binary.
Errors BXDB0002: the addressed database does not exist or could not be opened.
BXDB0003: the database is not persistent (stored on disk).
FODC0002: the addressed resource cannot be retrieved.
FODC0007: the specified path is invalid.
Examples
  • db:retrieve("DB", "music/01.mp3") returns the specified audio file as raw data.
  • stream:materialize(db:retrieve("DB", "music/01.mp3")) materializes the streamable result in main-memory before returning it.

db:export

Signatures db:export($db as xs:string, $path as xs:string) as empty-sequence()
db:export($db as xs:string, $path as xs:string, $params as item()) as empty-sequence()
Summary Exports the specified database $db to the specified file $path. Existing files will be overwritten. The $params argument contains serialization parameters (see Serialization for more details), which can either be specified
  • as children of an <output:serialization-parameters/> element, as defined for the fn:serialize() function; e.g.:
<output:serialization-parameters>
  <output:method value='xml'/>
  <output:cdata-section-elements value="div"/>
  ...
</output:serialization-parameters>
  • as map, which contains all key/value pairs:
map { "method": "xml", "cdata-section-elements": "div", ... }
Errors BXDB0002: the addressed database does not exist or could not be opened.
Examples Export all files as text:
db:export("DB", "/home/john/xml/texts", map { 'method': 'text' })

The following query can be used to export parts of the database:

let $target := '/home/john/xml/target'
for $doc in db:open('DB', 'collection')
let $path := $target || db:path($doc)
return (
  file:create-dir(file:parent($path)),
  file:write($path, $doc)
)

Value Indexes

db:text

Signatures db:text($db as xs:string, $string as item()) as text()*
Summary Returns all text nodes of the database $db that have $string as their string value and that are stored in the text index.
Errors BXDB0002: the addressed database does not exist or could not be opened.
BXDB0004: the index is not available.
Examples
  • db:text("DB", "QUERY")/.. returns the parents of all text nodes of the database DB that match the string QUERY.

db:text-range

Signatures db:text-range($db as xs:string, $min as xs:string, $max as xs:string) as text()*
Summary Returns all text nodes of the database $db that are located in between the $min and $max strings and that are stored in the text index.
Errors BXDB0002: the addressed database does not exist or could not be opened.
BXDB0004: the index is not available.
Examples
  • db:text-range("DB", "2000", "2001") returns all text nodes of the database DB that are found in between 2000 and 2001.

db:attribute

Signatures db:attribute($db as xs:string, $string as item()) as attribute()*
db:attribute($db as xs:string, $string as item(), $attname as xs:string) as attribute()*
Summary Returns all attribute nodes of the database $db that have $string as string value and that are stored in the attribute index.
If $attname is specified, the resulting attribute nodes are filtered by their attribute name.
Errors BXDB0002: the addressed database does not exist or could not be opened.
BXDB0004: the index is not available.
Examples
  • db:attribute("DB", "QUERY", "id")/.. returns the parents of all id attribute nodes of the database DB that have QUERY as string value.

db:attribute-range

Signatures db:attribute-range($db as xs:string, $min as xs:string, $max as xs:string) as attribute()*
db:attribute-range($db as xs:string, $min as xs:string, $max as xs:string, $attname as xs:string) as attribute()*
Summary Returns all attributes of the database $db, the string values of which are larger than or equal to $min and smaller than or equal to $max and that are stored in the attribute index.
Errors BXDB0002: the addressed database does not exist or could not be opened.
BXDB0004: the index is not available.
Examples
  • db:attribute-range("DB", "id456", "id473", 'id') returns all @id attributes of the database DB that have a string value in between id456 and id473.

Updates

Important note: All functions in this section are updating functions: they will not be immediately executed, but queued on the Pending Update List, which will be processed after the actual query has been evaluated. This means that the order in which the functions are specified in the query does usually not reflect the order in which the code will be evaluated.

db:create

Signatures db:create($db as xs:string) as empty-sequence()
db:create($db as xs:string, $inputs as item()*) as empty-sequence()
db:create($db as xs:string, $inputs as item()*, $paths as xs:string*) as empty-sequence()
db:create($db as xs:string, $inputs as item()*, $paths as xs:string*, $options as map(*)) as empty-sequence()
Summary Creates a new database with name $db and adds initial documents specified via $inputs to the specified $paths:
  • $inputs may be strings or nodes:
    • nodes may be of any type except for attributes
    • strings may be file/directory references, or XML strings (which are detected by the leading < character)
    • a path must be specified if the input is not a file or directory reference
  • The $options argument can be used to change the indexing behavior:
    • allowed options are all indexing and full-text, parsing and XML parsing options in lower case
    • parsing options are only applied to string inputs (XML strings or file/directory references)
  • An existing database will be overwritten. Next, db:create will be placed last on the Pending Update List. As a consequence, a newly created database cannot be addressed in the same query.
Errors FODC0002: an input points to an unknown resource.
FOUP0001: an attribute was specified as input.
BXDB0007: a database is opened by another process.
BXDB0011: the specified name is not a valid database name.
BXDB0012: two db:create statements with the same database name were specified.
BXDB0013: the number of specified inputs and paths differs.
Examples
  • db:create("DB") creates the empty database DB.
  • db:create("DB", "/home/dir/doc.xml") creates the database DB and adds the document /home/dir/doc.xml as initial content.
  • db:create("DB", <a/>, "doc.xml") creates the database DB and adds the document with content <a/> under the name doc.xml.
  • db:create("DB", "/home/dir/", "docs/dir") creates the database DB and adds the documents in /home/dir to the database under the path docs/dir.
  • db:create("DB", file:list('.'), (), map { 'ftindex': true() }) adds all files of the current working directory to a new database, preserving relative filesystem paths and creating a full-text index.

db:drop

Signatures db:drop($db as xs:string) as empty-sequence()
Summary Drops the database $db and all connected resources.
Errors BXDB0002: the addressed database does not exist or could not be opened.
BXDB0007: a database is opened by another process.
Examples
  • db:drop("DB") drops the database DB.

db:add

Signatures db:add($db as xs:string, $input as item()) as empty-sequence()
db:add($db as xs:string, $input as item(), $path as xs:string) as empty-sequence()
db:add($db as xs:string, $input as item(), $path as xs:string, $options as map(*)) as empty-sequence()
Summary Adds documents specified by $input to the database $db with the specified $path:
  • A document with the same path may occur more than once in a database. If you want to enforce single instances, use db:replace instead.
  • For more information on the arguments, see db:create (note that the specified options must not be indexing and full-text options).
Errors BXDB0002: the addressed database does not exist or could not be opened.
FODC0002: the input points to an unknown resource.
FOUP0001: an attribute was specified as input.
Examples
  • db:add("DB", "/home/dir/doc.xml") adds the file /home/dir/doc.xml to the database DB.
  • db:add("DB", <a/>, "doc.xml") adds a document node to the database DB under the name doc.xml.
  • db:add("DB", "/home/dir", "docs/dir") adds all documents in /home/dir to the database DB under the path docs/dir.

db:delete

Signatures db:delete($db as xs:string, $path as xs:string) as empty-sequence()
Summary Deletes document(s), specified by $path, from the database $db.
Errors BXDB0002: the addressed database does not exist or could not be opened.
Examples
  • db:delete("DB", "docs/dir/doc.xml") deletes the document docs/dir/doc.xml in the database DB.
  • db:delete("DB", "docs/dir") deletes all documents with paths beginning with docs/dir in the database DB.

db:copy

Signatures db:copy($db as xs:string, $newname as xs:string) as empty-sequence()
Summary Creates a copy of the database specified by $db to $newname.
Errors BXDB0002: the addressed database does not exist or could not be opened.
BXDB0011: invalid database name.
BXDB0016: name of source and target database is equal.

db:alter

Signatures db:alter($db as xs:string, $newname as xs:string) as empty-sequence()
Summary Renames the database specified by $db to $newname.
Errors BXDB0002: the addressed database does not exist or could not be opened.
BXDB0011: invalid database name.
BXDB0016: name of source and target database is equal.

db:create-backup

Signatures db:create-backup($db as xs:string) as empty-sequence()
Summary Creates a backup of the database $db.
Errors BXDB0002: the addressed database does not exist or could not be opened.
BXDB0011: invalid database name.
Examples
  • db:create-backup("DB") creates a backup of the database DB.

db:drop-backup

Signatures db:drop-backup($name as xs:string) as empty-sequence()
Summary Drops all backups of the database with the specified $name. If the given $name points to a specific backup file, only this specific backup file is deleted.
Errors BXDB0002: No backup file found.
BXDB0011: invalid database name.
Examples
  • db:drop-backup("DB") drops all backups of the database DB.
  • db:drop-backup("DB-2014-03-13-17-36-44") drops the specific backup file DB-2014-03-13-17-36-44.zip of the database DB.

db:restore

Signatures db:restore($name as xs:string) as empty-sequence()
Summary Restores the database with the specified $name. The $name may include the timestamp of the backup file.
Errors BXDB0011: invalid database name.
BXDB0015: No backup found.
Examples
  • db:restore("DB") restores the database DB.
  • db:restore("DB-2014-03-13-18-05-45") restores the database DB from the backup file with the given timestamp.

db:optimize

Signatures db:optimize($db as xs:string) as empty-sequence()
db:optimize($db as xs:string, $all as xs:boolean) as empty-sequence()
db:optimize($db as xs:string, $all as xs:boolean, $options as map(*)) as empty-sequence()
Summary Optimizes the meta data and indexes of the database $db.
If $all is true, the complete database will be rebuilt.
The $options argument can be used to control indexing. The syntax is identical to the db:create function: Allowed options are all indexing and full-text options. UPDINDEX is only allowed if $all is true.
Errors BXDB0002: the addressed database does not exist or could not be opened.
FOUP0002: an error occurred while optimizing the database.
Examples
  • db:optimize("DB") optimizes the database structures of the database DB.
  • db:optimize("DB", true(), map { 'ftindex': true() }) optimizes all database structures of the database DB and creates a full-text index.

db:rename

Signatures db:rename($db as xs:string, $path as xs:string, $newpath as xs:string) as empty-sequence()
Summary Renames document(s), specified by $path to $newpath in the database $db.
Errors BXDB0002: the addressed database does not exist or could not be opened.
BXDB0008: new document names would be empty.
Examples
  • db:rename("DB", "docs/dir/doc.xml", "docs/dir/newdoc.xml") renames the document docs/dir/doc.xml to docs/dir/newdoc.xml in the database DB.
  • db:rename("DB", "docs/dir", "docs/newdir") renames all documents with paths beginning with docs/dir to paths beginning with docs/newdir in the database DB.

db:replace

Signatures db:replace($db as xs:string, $path as xs:string, $input as item()) as empty-sequence()
db:replace($db as xs:string, $path as xs:string, $input as item(), $options as map(*)) as empty-sequence()
Summary Replaces a document, specified by $path, in the database $db with the contents of $input, or adds it as a new document:
  • See db:add for more details on the arguments (for historical reasons, the order of the 2nd and 3rd argument differs).
Errors BXDB0002: the addressed database does not exist or could not be opened.
BXDB0014: the path points to a directory.
FODC0002: the input points to an unknown resource.
FOUP0001: an attribute was specified as input.
Examples
  • db:replace("DB", "docs/dir/doc.xml", "/home/dir/doc.xml") replaces the content of the document docs/dir/doc.xml in the database DB with the content of the file /home/dir/doc.xml.
  • db:replace("DB", "docs/dir/doc.xml", "<a/>") replaces the content of the document docs/dir/doc.xml in the database DB with <a/>.
  • db:replace("DB", "docs/dir/doc.xml", document { <a/> }) replaces the content of the document docs/dir/doc.xml in the database DB with the specified document node.

The following query can be used to import files from a directory to a database:

let $source := '/home/john/xml/source'
for $file in file:list($source, true())
let $path := $source || $file
where not(file:is-dir($path))
return db:replace('db', $file, doc($path))

db:store

Signatures db:store($db as xs:string, $path as xs:string, $input as item()) as empty-sequence()
Summary Stores a binary resource specified by $input in the database $db and the location specified by $path.
Errors BXDB0002: the addressed database does not exist or could not be opened.
BXDB0003: the database is not persistent (stored on disk).
FODC0007: the specified path is invalid.
FOUP0002: the resource cannot be stored at the specified location.
Examples
  • db:store("DB", "video/sample.mov", file:read-binary('video.mov')) stores the addressed video file at the specified location.

db:output

Signatures db:output($result as item()*) as empty-sequence()
Summary This function can be used to both perform updates and return results in a single query. The argument of the function will be evaluated, and the resulting items will be cached and returned after the updates on the pending update list have been processed. As nodes may be updated, they will be copied before being cached.
The function can only be used together with updating expressions; if the function is called within a transform expression, its results will be discarded.
Examples
  • db:output("Prices have been deleted."), delete node //price deletes all price elements in a database and returns an info message.

db:output-cache

Signatures db:output-cache() as item()*
Summary Returns the items that have been cached by db:output. It can be used to check which items will eventually be returned as result of an updating function.
This function is non-deterministic: Its will return different results before and after items have been cached. It is e. g. useful when writing unit tests.

db:flush

Signatures db:flush($db as xs:string) as empty-sequence()
Summary Explicitly flushes the buffers of the database $db. This command is only useful if AUTOFLUSH has been set to false.
Errors BXDB0002: the addressed database does not exist or could not be opened.

Helper Functions

db:name

Signatures db:name($node as node()) as xs:string
Summary Returns the name of the database in which the specified database node $node is stored.
Errors BXDB0001: $nodes contains a node which is not stored in a database.

db:path

Signatures db:path($node as node()) as xs:string
Summary Returns the path of the database document in which the specified database node $node is stored.
Errors BXDB0001: $nodes contains a node which is not stored in a database.

db:exists

Signatures db:exists($db as xs:string) as xs:boolean
db:exists($db as xs:string, $path as xs:string) as xs:boolean
Summary Checks if the database $db or the resource specified by $path exists. false is returned if a database directory has been addressed.
Examples
  • db:exists("DB") returns true if the database DB exists.
  • db:exists("DB", "resource") returns true if resource is an XML document or a raw file.

db:is-raw

Signatures db:is-raw($db as xs:string, $path as xs:string) as xs:boolean
Summary Checks if the specified resource in the database $db and the path $path exists, and if it is a binary resource.
Errors BXDB0002: the addressed database does not exist or could not be opened.
Examples
  • db:is-raw("DB", "music/01.mp3") returns true.

db:is-xml

Signatures db:is-xml($db as xs:string, $path as xs:string) as xs:boolean
Errors BXDB0002: the addressed database does not exist or could not be opened.
Summary Checks if the specified resource in the database $db and the path $path exists, and if it is an XML document.
Examples
  • db:is-xml("DB", "dir/doc.xml") returns true.

db:content-type

Signatures db:content-type($db as xs:string, $path as xs:string) as xs:string
Summary Retrieves the content type of a resource in the database $db and the path $path.
The file extension is used to recognize the content-type of a resource stored in the database. Content-type application/xml will be returned for any XML document stored in the database, regardless of its file name extension.
Errors BXDB0002: the addressed database does not exist or could not be opened.
FODC0002: the addressed resource is not found or cannot be retrieved.
Examples
  • db:content-type("DB", "docs/doc01.pdf") returns application/pdf.
  • db:content-type("DB", "docs/doc01.xml") returns application/xml.
  • db:content-type("DB", "docs/doc01") returns application/xml, if db:is-xml("DB", "docs/doc01") returns true.

Errors

Code Description
BXDB0001 The referenced XML node is no database node, i.e. it is neither stored in a database nor represented as database fragment.
BXDB0002 The addressed database does not exist or could not be opened.
BXDB0003 The addressed database is not persistent (stored on disk).
BXDB0004 The database lacks an index structure required by the called function.
BXDB0005 A query is expected to exclusively return database nodes of a single database.
BXDB0006 A database path addressed with doc() contains more than one document.
BXDB0007 A database cannot be updated because it is opened by another process.
BXDB0008 Database paths cannot be renamed to empty strings.
BXDB0009 The addressed database id or pre value is out of range.
BXDB0011 The name of the specified database is invalid.
BXDB0012 A database can only be created once.
BXDB0013 The number of specified inputs and paths differs.
BXDB0014 Path points to a directory.

Changelog

Version 8.3
  • Updated: db:list-details: attributes with name of database and date of backup added to results.
  • Updated: db:backups now include attributes with name of database and date of backup.
  • Updated: Value Indexes: raise error if no index exists.
Version 8.2
Version 7.9
Version 7.8.2
Version 7.8
Version 7.7
Version 7.6
  • Updated: db:create: allow more than one input and path.
Version 7.5
Version 7.3
Version 7.2.1
Version 7.1
Version 7.0