Difference between revisions of "Database Module"
m (Text replacement - "[[Jobs Module" to "[[Job Module") |
|||
Line 709: | Line 709: | ||
=Backups= | =Backups= | ||
− | {{Announce|Introduced with Version 10:}} Support for general data ([[User Management|registered users]], [[ | + | {{Announce|Introduced with Version 10:}} Support for general data ([[User Management|registered users]], [[Job Module#Services|scheduled services]] and [[Store Module|key-value stores]]). |
All functions in this section except for {{Function||db:backups}} are {{Function|Database|Updating Functions}}. | All functions in this section except for {{Function||db:backups}} are {{Function|Database|Updating Functions}}. | ||
Line 945: | Line 945: | ||
;Version 10 | ;Version 10 | ||
* Added: {{Function||db:get}}, {{Function||db:put}}, {{Function||db:type}}. | * Added: {{Function||db:get}}, {{Function||db:put}}, {{Function||db:type}}. | ||
− | * Added: [[#Backups|Backups]]: Support for general data ([[User Management|registered users]], [[ | + | * Added: [[#Backups|Backups]]: Support for general data ([[User Management|registered users]], [[Job Module#Services|scheduled services]] and [[Store Module|key-value stores]]). |
* Updated: {{Function||db:get}}, {{Function||db:get-id}}, {{Function||db:get-pre}} renamed (before: {{Code|db:open}}, {{Code|db:open-id}}, {{Code|db:open-pre}}) | * Updated: {{Function||db:get}}, {{Function||db:get-id}}, {{Function||db:get-pre}} renamed (before: {{Code|db:open}}, {{Code|db:open-id}}, {{Code|db:open-pre}}) | ||
* Updated: {{Function||db:put}} renamed (before: {{Code|db:replace}}); function signature aligned with {{Function||db:add}} (second and third argument swapped). | * Updated: {{Function||db:put}} renamed (before: {{Code|db:replace}}); function signature aligned with {{Function||db:add}} (second and third argument swapped). |
Revision as of 11:24, 28 July 2022
This XQuery Module contains functions for processing databases from within XQuery. Existing databases can be opened and listed, its contents can be directly accessed, documents can be added to and removed, etc.
Contents
Conventions
All functions and errors in this module are assigned to the http://basex.org/modules/db
namespace, which is statically bound to the db
prefix.
Database Nodes
In BaseX, two internal representations exist for nodes.
- XML fragments are generated by XQuery node constructors.db:b
- Database nodes are:
- stored in a persistent database on disk;
- nodes of a document that has been generated temporarily with
fn:doc
,fn:parse-xml
and other functions; or - result of a main-memory update operation.
Some operations are restricted to database nodes, but you can convert XML fragments to database nodes by applying an empty update or transform operation to a node. Two examples:
- Retrieve the internal node id of an XML fragment:
let $xml := <xml>hello world</xml> update {}
return db:node-id($xml/text())
- Puts a marker element around the result of a full-text request (see
ft:mark
for more details):
copy $p := <xml>hello world</xml>
modify ()
return ft:mark($p[text() contains text 'word'], 'b')
Updating Functions
Various functions in this module are updating. Updating functions will not be immediately executed, but queued on the Pending Update List, and processed after the remaining query has been evaluated. This means that the order in which the functions are specified in the query often does not reflect the order in which they will eventually be executed.
General Functions
db:system
Signatures | db:system() as element(system)
|
Summary | Returns general information on the database system the current values of all global and local Options. The INFO command returns similar output.
|
db:option
Signatures | db:option($name as xs:string) as xs:string
|
Summary | Returns the current value (string, integer, boolean, map) of a global or local Option with the specified $name . The SHOW OPTIONS command returns similar output.
|
Errors | option : the specified option is unknown.
|
Examples |
|
db:info
Signatures | db:info($db as xs:string) as element(database)
|
Summary | Returns meta information on the database $db . The output is similar to the INFO DB command.
|
Errors | open : the addressed database does not exist or could not be opened.
|
db:property
Signatures | db:property($db as xs:string, $name as xs:string) as xs:anyAtomicType
|
Summary | Returns the value (string, boolean, integer) of a property with the specified $name in the database $db . The available properties are the ones returned by db:info .
|
Errors | property : the specified property is unknown.
|
Examples |
|
db:list
Signatures | db:list() as xs:string* db:list($db as xs:string) as xs:string* db:list($db as xs:string, $path as xs:string) as xs:string*
|
Summary | The result of this function is dependent on the number of arguments:
|
Errors | open : the addressed database does not exist or could not be opened.
|
Examples |
|
db:list-details
Signatures | db:list-details() as element(database)* db:list-details($db as xs:string) as element(resource)* db:list-details($db as xs:string, $path as xs:string) as element(resource)*
|
Summary | Without arguments, an element is returned for each database that is accessible to the current user:
If a database
|
Errors | open : the addressed database does not exist or could not be opened.
|
Examples |
|
db:dir
Signatures | db:dir($db as xs:string, $path as xs:string) as element()*
|
Summary | Returns metadata on all directories and resources of the database $db in the specified directory $path . Two types of elements are returned:
Please note that directories are not stored in BaseX. Instead, they result implicitly from the paths of stored resources. |
Errors | open : the addressed database does not exist or could not be opened.path : the specified path is invalid.
|
Examples |
|
Read Operations
db:get
Updated with Version 10: Renamed (before: db:open
). Due to its widespread use, the old function name will be supported for some more time.
Signatures | db:get($db as xs:string) as document-node()* db:get($db as xs:string, $path as xs:string) as document-node()*
|
Summary | Returns all documents from the database $db , or only documents matching the specified $path .
|
Errors | open : the addressed database does not exist or could not be opened.
|
Examples |
|
db:get-pre
Updated with Version 10: Renamed (before: db:open-pre
).
Signatures | db:get-pre($db as xs:string, $pres as xs:integer*) as node()*
|
Summary | Returns all nodes from the database $db with the pre values $pres in distinct document order.The PRE value provides very fast access to an existing database node, but it will change whenever a node with a smaller pre values is added to or deleted from a database. |
Errors | open : the addressed database does not exist or could not be opened.range : the specified pre value does not exist in the database.
|
Examples |
|
db:get-id
Updated with Version 10: Renamed (before: open-id
).
Signatures | db:get-id($db as xs:string, $ids as xs:integer*) as node()*
|
Summary | Returns all nodes from the database $db with the pre values $ids in distinct document order.Each database node has a persistent ID value. Access to the node ID can be sped up by turning on the UPDINDEX option.
|
Errors | open : the addressed database does not exist or could not be opened.range : the specified ID value does not exist in the database.
|
db:get-binary
Updated with Version 10: renamed (before: db:retrieve
).
Signatures | db:get-binary($db as xs:string, $path as xs:string) as item()
|
Summary | Returns a map with all paths and binary resources of the database $db . A single xs:base64Binary item is returned if a $path is specified. All items are lazy, i.e., the actual data will only be retrieved if it is processed.
|
Errors | open : the addressed database does not exist or could not be opened.mainmem : the database is not persistent (stored on disk).
|
Examples |
|
db:get-value
Introduced with Version 10.
Signatures | db:get-value($db as xs:string, $path as xs:string) as item()*
|
Summary | Returns a map with all paths and values of the database $db . A single value is returned if a $path is specified.
|
Errors | open : the addressed database does not exist or could not be opened.mainmem : the database is not persistent (stored on disk).
|
Examples |
|
db:node-pre
Signatures | db:node-pre($nodes as node()*) as xs:integer*
|
Summary | Returns the pre values of the specified $nodes , which must all be database nodes.The PRE value provides very fast access to an existing database node, but it will change whenever a node with a smaller pre values is added to or deleted from a database. |
Errors | node : $nodes contains a node which is not stored in a database.
|
Examples |
|
db:node-id
Signatures | db:node-id($nodes as node()*) as xs:integer*
|
Summary | Returns the id values of the specified $nodes , which must all be database nodes.Each database node has a persistent ID value. Access to the node id can be sped up by turning on the UPDINDEX option.
|
Errors | node : $nodes contains a node which is not stored in a database.
|
db:export
Signatures | db:export($db as xs:string, $path as xs:string) as empty-sequence() db:export($db as xs:string, $path as xs:string, $params as item()) as empty-sequence() |
Summary | Exports the specified database $db to the specified file $path . Existing files will be overwritten.The $params argument contains serialization parameters. As with fn:serialize(), the parameters can be specified
<output:serialization-parameters>
<output:method value='xml'/>
<output:cdata-section-elements value="div"/>
...
</output:serialization-parameters>
map { "method": "xml", "cdata-section-elements": "div", ... }
|
Errors | open : the addressed database does not exist or could not be opened.
|
Examples | Export all files as text:db:export("DB", "/home/john/xml/texts", map { 'method': 'text' })
The following code can be used to export parts of the database: let $target := '/home/john/xml/target'
for $doc in db:get('DB', 'collection')
let $path := $target || db:path($doc)
return (
file:create-dir(file:parent($path)),
file:write($path, $doc)
)
|
Value Indexes
db:text
Signatures | db:text($db as xs:string, $strings as xs:string*) as text()*
|
Summary | Returns all text nodes of the database $db that have one of the specified $strings as values and that are stored in the text index.
|
Errors | open : the addressed database does not exist or could not be opened.no-index : the index is not available. |
Examples |
|
db:text-range
Signatures | db:text-range($db as xs:string, $min as xs:string, $max as xs:string) as text()*
|
Summary | Returns all text nodes of the database $db whose values are between $min and $max and that are stored in the text index.
|
Errors | open : the addressed database does not exist or could not be opened.no-index : the index is not available. |
Examples |
|
db:attribute
Signatures | db:attribute($db as xs:string, $strings as xs:string*) as attribute()* db:attribute($db as xs:string, $strings as xs:string*, $name as xs:string) as attribute()*
|
Summary | Returns all attribute nodes of the database $db that have one of the specified $strings as values and that are stored in the attribute index.If $name is specified, the resulting attribute nodes are filtered by their attribute name.
|
Errors | open : the addressed database does not exist or could not be opened.no-index : the index is not available. |
Examples |
|
db:attribute-range
Signatures | db:attribute-range($db as xs:string, $min as xs:string, $max as xs:string) as attribute()* db:attribute-range($db as xs:string, $min as xs:string, $max as xs:string, $name as xs:string) as attribute()*
|
Summary | Returns all attributes of the database $db , the string values of which are larger than or equal to $min and smaller than or equal to $max and that are stored in the attribute index.
|
Errors | open : the addressed database does not exist or could not be opened.no-index : the index is not available. |
Examples |
|
db:token
Signatures | db:token($db as xs:string, $tokens as xs:string*) as attribute()* db:token($db as xs:string, $tokens as xs:string*, $name as xs:string) as attribute()*
|
Summary | Returns all attribute nodes of the database $db the values of which contain one of the specified $tokens .If $name is specified, the resulting attribute nodes are filtered by their attribute name.
|
Errors | open : the addressed database does not exist or could not be opened.no-index : the index is not available. |
Examples |
|
Updates
All functions in this section are Updating Functions.
db:create
Signatures | db:create($db as xs:string) as empty-sequence() db:create($db as xs:string, $inputs as item()*) as empty-sequence() db:create($db as xs:string, $inputs as item()*, $paths as xs:string*) as empty-sequence() db:create($db as xs:string, $inputs as item()*, $paths as xs:string*, $options as map(*)?) as empty-sequence()
|
Summary | Creates a new database with name $db and adds initial documents specified via $inputs to the specified $paths :
|
Errors | lock : a database is opened by another process.name : the specified name is not a valid database name.conflict : the same database was addressed more than once.args : the number of specified inputs and paths differs.
|
Examples |
|
db:add
Signatures | db:add($db as xs:string, $input as item()) as empty-sequence() db:add($db as xs:string, $input as item(), $path as xs:string?) as empty-sequence() db:add($db as xs:string, $input as item(), $path as xs:string?, $options as map(*)?) as empty-sequence()
|
Summary | Adds documents specified by $input to the database $db with the specified $path :
|
Errors | open : the addressed database does not exist or could not be opened.
|
Examples |
|
db:put
Updated with Version 10: renamed (before: db:replace
); function signature aligned with db:add
(second and third argument swapped).
Signatures | db:put($db as xs:string, $input as item(), $path as xs:string) as empty-sequence() db:put($db as xs:string, $input as item(), $path as xs:string, $options as map(*)?) as empty-sequence()
|
Summary | Replaces a resource, specified by $path , in the database $db with the contents of $input , or adds it as a new resource:
|
Errors | open : the addressed database does not exist or could not be opened.target : the path points to a directory.
|
Examples |
The following query can be used to import files from a directory to a database: let $source := '/home/john/xml/source'
for $file in file:list($source, true())
let $path := $source || $file
where not(file:is-dir($path))
return db:put('db', doc($path), $file)
|
db:put-binary
Updated with Version 10: renamed (before: db:put
); function signature aligned with db:add
(second and third argument swapped).
Signatures | db:put-binary($db as xs:string, $input as item(), $path as xs:string) as empty-sequence()
|
Summary | Stores a binary resource specified by $input in the database $db at the specified $path . Existing resources are overwritten.
|
Errors | open : the addressed database does not exist or could not be opened.mainmem : the database is not persistent (stored on disk).
|
Examples |
let $db := 'db'
let $src-path := 'src/'
let $trg-path := 'trg/'
for $src in db:list($db, $src-path)
where db:type($db, $src) = 'binary'
let $trg := $trg-path || substring-after($src, $src-path)
return db:put-binary($db, db:get-binary($db, $src), $trg)
|
db:put-value
Introduced with Version 10.
Signatures | db:put-value($db as xs:string, $input as item()*, $path as xs:string) as empty-sequence()
|
Summary | Stores a value specified by $input in the database $db at the specified $path . Existing resources are overwritten. The value can be an arbitrary sequence of atomic items, nodes, maps, and arrays.
|
Errors | open : the addressed database does not exist or could not be opened.mainmem : the database is not persistent (stored on disk).
|
Examples |
db:put-value(
'factbook',
map:merge(
for $country in db:get('factbook')//country
return map:entry($country/@name, $country//city/name ! string())
),
'cities'
)
|
db:delete
Signatures | db:delete($db as xs:string, $path as xs:string) as empty-sequence()
|
Summary | Deletes resource(s), specified by $path , from the database $db .
|
Errors | open : the addressed database does not exist or could not be opened.path : the specified path is invalid.
|
Examples |
|
db:copy
Signatures | db:copy($db as xs:string, $name as xs:string) as empty-sequence()
|
Summary | Creates a copy of the database $db , which will be called $name .
|
Errors | open : the addressed database does not exist or could not be opened.lock : a database is opened by another process.name : invalid database name.conflict : the same database was addressed more than once.
|
db:alter
Signatures | db:alter($db as xs:string, $name as xs:string) as empty-sequence()
|
Summary | Renames the database $db to $name .
|
Errors | open : the addressed database does not exist or could not be opened.lock : a database is opened by another process.name : invalid database name.conflict : the same database was addressed more than once.
|
db:optimize
Signatures | db:optimize($db as xs:string) as empty-sequence() db:optimize($db as xs:string, $all as xs:boolean) as empty-sequence() db:optimize($db as xs:string, $all as xs:boolean, $options as map(*)?) as empty-sequence()
|
Summary | Optimizes the metadata and indexes of the database $db .If $all is true , the complete database will be rebuilt.The $options argument can be used to control indexing. The syntax is identical to the db:create function: Allowed options are all indexing and full-text options. UPDINDEX is only supported if $all is true .
|
Errors | open : the addressed database does not exist or could not be opened.
|
Examples |
|
db:rename
Signatures | db:rename($db as xs:string, $source as xs:string, $target as xs:string) as empty-sequence()
|
Summary | Moves all resources(s) of database $db , which are found in the supplied $source path, to the supplied $target path. The paths may point to single resources or directories. No updates will take place if a non-existing source path is supplied.
|
Errors | open : the addressed database does not exist or could not be opened.path : the specified source or target path, or one of its descendants, is invalid.
|
Examples |
|
db:flush
Signatures | db:flush($db as xs:string) as empty-sequence()
|
Summary | Explicitly flushes the buffers of the database $db . This command is only useful if AUTOFLUSH has been set to false .
|
Errors | open : the addressed database does not exist or could not be opened.
|
db:drop
Signatures | db:drop($db as xs:string) as empty-sequence()
|
Summary | Drops the database $db and all connected resources.
|
Errors | open : the addressed database does not exist or could not be opened.lock : a database is opened by another process.conflict : the same database was addressed more than once.
|
Examples |
|
Backups
Introduced with Version 10: Support for general data (registered users, scheduled services and key-value stores).
All functions in this section except for db:backups
are Updating Functions
.
db:create-backup
Updated with Version 10: Options argument added.
Signatures | db:create-backup($db as xs:string) as empty-sequence() db:create-backup($db as xs:string, $options as map(*)) as empty-sequence()
|
Summary | Creates a backup of the database $db . If no name is supplied, general data will be backed up. The following $options are available:
|
Errors | open : the addressed database does not exist or could not be opened.name : invalid database name.conflict : the same database was addressed more than once.
|
Examples |
|
db:drop-backup
Signatures | db:drop-backup($name as xs:string) as empty-sequence()
|
Summary | Drops all backups of the database with the specified $name . If the name ends with a timestamp, only the specified backup file will be deleted. If no name is supplied, backups with general data are addressed.
|
Errors | backup : No backup file found.name : invalid database name.conflict : the same database was addressed more than once.
|
Examples |
|
db:alter-backup
Signatures | db:alter-backup($name as xs:string, $new-name as xs:string) as empty-sequence()
|
Summary | Renames all backups of the database with the specified $name to $new-name . If the name ends with a date, only the specified backup file will be renamed.
|
Errors | backup : No backup file found.name : invalid database name.conflict : the same database was addressed more than once.
|
Examples |
|
db:restore
Signatures | db:restore($name as xs:string) as empty-sequence()
|
Summary | Restores the database with the specified $name . The $name may include the timestamp of the backup file. If no name is supplied, general data will be restored. If general data is restored, it will only be available after BaseX has been restarted.
|
Errors | lock : a database is opened by another process.name : invalid database name.no-backup : No backup found.conflict : the same database was addressed more than once.
|
Examples |
|
db:backups
Signatures | db:backups() as element(backup)* db:backups($db as xs:string) as element(backup)*
|
Summary | Returns an element sequence containing all available database backups with timestamp, file size and comment. If a database $db is specified, the sequence will be restricted to the backups matching this database.
|
Examples |
|
Helper Functions
db:name
Signatures | db:name($node as node()) as xs:string
|
Summary | Returns the name of the database in which the specified database node $node is stored.
|
Errors | node : $nodes contains a node which is not stored in a database.
|
db:path
Signatures | db:path($node as node()) as xs:string
|
Summary | Returns the path of the database document in which the specified database node $node is stored.
|
Errors | node : $nodes contains a node which is not stored in a database.
|
db:exists
Signatures | db:exists($db as xs:string) as xs:boolean db:exists($db as xs:string, $path as xs:string) as xs:boolean
|
Summary | Checks if the database $db or the resource specified by $path exists. false is returned if a database directory has been addressed.
|
Examples |
|
db:type
Introduced with BaseX 10: Replaces db:is-raw
and db:is-xml
.
Signatures | db:type($db as xs:string, $path as xs:string) as xs:boolean
|
Summary | Returns the type of a resource – xml , binary , or value – in the database $db at the specified $path .
|
Errors | open : the addressed database does not exist or could not be opened.
|
Examples |
|
db:content-type
Signatures | db:content-type($db as xs:string, $path as xs:string) as xs:string
|
Summary | Retrieves the content type of a resource in the database $db and the path $path .The file extension is used to recognize the content-type of a resource stored in the database. Content-type application/xml will be returned for any XML document stored in the database, regardless of its file name extension.
|
Errors | open : the addressed database does not exist or could not be opened.
|
Examples |
|
Errors
Code | Description |
---|---|
args
|
The number of specified inputs and paths differs. |
conflict
|
Multiple update operations point to the same target. |
lock
|
A database cannot be updated because it is opened by another process. |
mainmem
|
The addressed database is not persistent (stored on disk). |
name
|
The name of the specified database is invalid. |
no-backup
|
No backup exists for a database. |
node
|
The referenced XML node is no database node, i.e. it is neither stored in a database nor represented as database fragment. |
no-index
|
The database lacks an index structure required by the called function. |
open
|
The addressed database does not exist or could not be opened. |
option
|
The specified option is unknown. |
path
|
The specified database path is invalid. |
property
|
The specified database property is unknown. |
range
|
The addressed database id or pre value is out of range. |
target
|
Path points to an invalid target. |
Changelog
- Version 10
- Added:
db:get
,db:put
,db:type
. - Added: Backups: Support for general data (registered users, scheduled services and key-value stores).
- Updated:
db:get
,db:get-id
,db:get-pre
renamed (before:db:open
,db:open-id
,db:open-pre
) - Updated:
db:put
renamed (before:db:replace
); function signature aligned withdb:add
(second and third argument swapped). - Updated:
db:put-binary
renamed (before:db:store
); function signature aligned withdb:add
(second and third argument swapped). - Updated:
db:get-binary
renamed (before:db:retrieve
). - Updated:
db:backups
,db:create-backup
: Options added. - Removed:
db:is-raw
,db:is-raw
(new:db:type
).
- Version 9.3
- Added:
db:alter-backup
- Updated:
db:open-id
,db:open-pre
: support for multiple integers
- Version 9.2
- Version 9.0
- Added:
db:option
- Updated: db:output renamed to
update:output
, db:output-cache renamed toupdate:cache
- Updated: error codes updated; errors now use the module namespace
- Version 8.6
- Added:
db:property
- Version 8.4
- Version 8.3
- Updated:
db:list-details
: attributes with name of database and date of backup added to results. - Updated:
db:backups
now include attributes with name of database and date of backup. - Updated:
Value Indexes
: raise error if no index exists.
- Version 8.2
- Added:
db:output-cache
- Removed: db:event
- Version 7.9
- Updated: parsing options added to
db:create
,db:add
anddb:replace
. - Updated: allow
UPDINDEX
if$all
istrue
.
- Version 7.8.2
- Added:
db:alter
,db:copy
,db:create-backup
,db:drop-backup
,db:restore
- Version 7.8
- Removed: db:fulltext (use
ft:search
instead)
- Version 7.7
- Added:
db:export
,db:name
,db:path
- Updated:
$options
argument added todb:create
anddb:optimize
. - Updated: the functions no longer accept database nodes as reference. Instead, the name of a database must now be specified.
- Version 7.6
- Updated:
db:create
: allow more than one input and path.
- Version 7.5
- Updated:
db:add
: input nodes will be automatically converted to document nodes - Added:
db:backups
- Added:
db:create
- Added:
db:drop
- Version 7.3
- Added:
db:flush
- Version 7.2.1
- Added:
db:text-range
,db:attribute-range
,db:output
- Version 7.1
- Added:
db:list-details
,db:content-type
- Updated:
db:info
,db:system
,db:retrieve
- Version 7.0