Bureaucrats, editor, reviewer, Administrators
13,550
edits
Changes
Jump to navigation
Jump to search
New databases Databases can be created via commands[[Commands]], via [[XQuery]], in the [[GUI]], or and with any of ourvarious [[Developing|APIs]]. If some an initial input is specified along with the a create operation, it some time can be saved, as the specified resources will be added to the database in a bulk operation:
Database must follow the The database name is composed of a restricted set of characters (see [[Valid Names|valid names constraints]]).Various [[parsersParsers]] can be chosen selected to influence control the database creationimport process, or to convert data of different formats input type to XML. '''Note:''' A main-memory only database can be created using the the <code>SET MAINMEM true</code> command before calling <code>CREATE DB</code> ([[Databases#In Memory Database|see below]] for more).
The <code>[[Commands#RETRIEVE|RETRIEVE]]</code> command and If the API you use does not support binary output (which is e.g. the <code>case for various [[Database Module#db:retrieveClients|db:retrieveClient]]</code> function language bindings), you can be used convert your binary data to return files in their native byte its string representation.before returning it to the client:
If the API you use does not support <syntaxhighlight lang="xquery">string(db:get-binary output (this is e'multimedia', 'sample.g. the case for various [[Clients|Client]] language bindingsavi')), you need to convert your binary data to its string representation before returning it to the client:</syntaxhighlight>
<pre class="brush:xquery">string(db:retrieve('multimedia', 'sample.avi'))</pre>=XQuery Values==
==HTTP Services==With {{Function|Database|db:get-value}}, XQuery values can be retrieved. In the following example, we assume that an XQuery map {{Code|cities}} was stored in an {{Code|indexes}} database:
* With [[REST]] and [[WebDAV]], all database resources can be requested in a uniform way<syntaxhighlight lang="xquery">let $city-map := db:get-value('indexes', no matter if they are well'cities')return $city-formed XML documents or binary files.map?Chile</syntaxhighlight>
The [[Options#AUTOFLUSH{{Option|AUTOFLUSH]] option }} can be turned off before ''bulk operations'' (i.e. , before a large number of numerous new resources is are added to the database).
The [[Options#ADDCACHEIf {{Option|ADDCACHE]] option will first cache }} is enabled, the input will be cached before adding it is added to the database. This is helpful when the input documents to be added are expected to eat up consume too much main -memory.
The With the following commands create [[Commands#Command Scripts|command script]], an empty databaseis created, add two resourcesare added (one directly, explicitly flush data structures to diskanother one cached), and finally delete all inserted datais exported to the file system:
STORE BINARY PUT TO images/ 123.jpgFLUSHDELETE EXPORT /path/to/file-system/
You may as well use the BaseX-specific [[Database Module|==XQuery Database Functions]] to create, add, replace, and delete XML documents:==
Last but not least, XML documents All resources stored in a database can also be added via the GUI and the ''Databaseexported'' menu, i.e., written back to disk, e.g., as follows:
=* Commands: {{Command|EXPORT}} writes all resources to the specified target directory.* GUI: Go to ''Database'' → ''Export Data='', choose the target directory and press ''OK''.* XQuery: Use {{Function|Database|db:export}}.* WebDAV: Locate the database directory (or a subdirectory of it) and copy all contents to another location.
All resources stored in a database can be ''exported'', i.e., written back to disk. This can be done in several ways:=Main-Memory Databases=
* Commands: <code>[[Commands#EXPORTA database can be created in main-memory by enabling the {{Option|EXPORT]]</code> writes all resources to MAINMEM}} option. Next, in the specified target directory* GUI: Go to ''Database'' → ''Export''standalone context, choose the target directory and press ''OK''* WebDAV: Locate the database directory (or a submain-directory of it) and copy all contents to another locationmemory database can be created, which can then be accessed by subsequent commands.
=In Memory Database=If a BaseX server is started, and if a database is created in its context at startup time, e.g., with the [[Command-Line Options|command-line option -c]] and a {{Command|CREATE DB}} call, BaseX clients can then access and update this database:
* In the standalone context, a main-memory database can be created (using <codesyntaxhighlight lang="perl"># Serverbasexserver -c"SET mainmem on" -c"CREATE DB</code>), which can then be accessed by subsequent commandsmainmem document.xml"* If a BaseX server instance is [Server]Server was started, and if a database is created in its context (using <code>CREATE DB</code>), other BaseX client instances can access (and update) this database (using OPEN, dbport:open, etc.1984) as long as no other database is opened/created by the server.* You can force an ordinary database to being loaded to memeory by using <code>dbMAINMEM:open(trueDatabase 'some-dbmainmem') update {}</code>created in 1782.80 ms.
'''Note# ClientbasexclientUsername:''' main-memory database instances are also created by the invocation of <code>doc(...)</code> or <code>collection(Password: ...BaseX [Client]Try 'help' to get more information.> XQUERY count(db:get('mainmem')</code>, if the argument is not a/*)1876462database (no matter which value is set for MAINMEM)Query executed in 0.97 ms. In other words:the same internal representation is used for main-memory databases anddocuments</collections generated via XQuery.syntaxhighlight>
→Changelog
This page is part of the [[Getting Started]] Section.
In BaseX, a ''database'' is a pretty light-weight concept and can be comparedto a ''collection''. It contains an arbitrary number of may contain one or more '''resources''',which are addressed by their a unique database path. Resources There is no explicit layer for collections: Instead, collections are implicitly created and deleted, and collections result from the existence of documents in specific paths. As a single database is restricted to 2 billion XML nodes (see [[Statistics]]), but resources can easily be distributed across multiple database instances. Multiple databases can either beaddressed (queried, updated) by a single XQuery expression. Three different resource types exist: {|class="wikitable"|- valign="top"| '''XML documentsResource Type''' or | '''raw filesDescription''' (binaries)|- valign="top"| XML Documents| The default resource type. The storage and index features are optimized for XML contents, or any other contents stored in an XML representation.Some information on |- valign="top"| Binary Data| Binary data: Raw data of any type, stored in its binary representation. See [[Binary Data]] for more information.|- valign="top"| XQuery Values| Introduced with {{Announce|Version 10}}: Results of XQuery expressions, stored in a binary datarepresentation for fast retrieval. All value types are supported, including maps and arrays, but excluding any other [[Higher-Order Functions#Function Items|function items]] can be found on an extra page.|}
=Create Databases=
* [[Startup#BaseX Standalone|ConsoleCommand-Line]]: <code>CREATE DB db documents /path/to/resources</code> will add initial documents : Add resources in the specified path to a databasenamed {{Code|documents}}.* [[Startup#BaseX GUI|GUI]]: Go to ''Database'' → ''New'', press ''BrowseBrowse…'' to choose an initial file or directory, and press ''OK''.
=Access Resources=
{| class="wikitable"
|-valign="top"
!Function
!Example
!Description
|-valign="top"|[[{{Function|Database Module#db:open|db:open]]get}}|{{Code|db:openget("db", "path/to/docs")}}
|Returns all documents that are found in the database {{Code|db}} at the (optional) path {{Code|path/to/docs}}.
|-valign="top"|<code>[http://www.xqueryfunctions.com/xq/fn_collection.html fn:collection]</code>
|{{Code|collection("db/path/to/docs")}}
|Returns all documents at the location {{Code|path/to/docs}} in the database {{Code|db}}.<br/>If no path is specified after the database, all documents in the database will be returned.<br/>If no argument is specified, all documents of the database will be returned that has been opened in the global context.
|-valign="top"|<code>[http://www.xqueryfunctions.com/xq/fn_doc.html fn:doc]</code>
|{{Code|doc("db/path/to/doc.xml")}}
|Returns the document at the location {{Code|path/to/docs}} in the database {{Code|db}}.<br/>An error is raised if the specified yields zero or more than one document.
You can access multiple databases in a single query:
<pre classsyntaxhighlight lang="brush:xquery">
for $i in 1 to 100
return db:openget('books' || $i)//book/title</presyntaxhighlight>
If the [[Options#DEFAULTDB{{Option|DEFAULTDB]] }} option is turned on, the path argument of the {{Code|fn:doc}} or {{Code|fn:collection}} function functions will first be resolved against the globally opened database.
Two more functions are available for retrieving information on database nodes:
!Example
!Description
|-valign="top"|[[{{Function|Database Module#db:name|db:name]]}}
|{{Code|db:name($node)}}
|Returns the name of the database in which the specified {{Code|$node}} is stored.
|-valign="top"|[[{{Function|Database Module#db:path|db:path]]}}
|{{Code|db:path($node)}}
|Returns the path of the database document in which the specified {{Code|$node}} is stored.
The {{Code|fn:document-uri}} and {{Code|fn:base-uri}} functions return URIs that can also be reused as arguments for the {{Code|fn:doc}} and {{Code|fn:collection}} functions. As a result, the following example query always returns {{Code|true}}:
<pre classsyntaxhighlight lang="brush:xquery">
every $c in collection('anyDB')
satisfies doc-available(document-uri($c))
</presyntaxhighlight>
If the argument of {{Code|fn:doc}} or {{Code|fn:collection}} does not start with a valid database name, or if the addressed database does not exist, the string is interpreted as URI reference, and the documents found at this location will be returned. Examples:
* {{Code|doc("http://web.de")}}: retrieves the addressed URI and returns it as a main-memory document node.
* {{Code|doc("myfile.xml")}}: retrieves the given file from the file system and returns it as a main-memory document node. Note that updates to main-memory nodes are not automatically written back to disk unless the <code>[[Options#WRITEBACK{{Option|WRITEBACK]]</code> }} option is set.
* {{Code|collection("/path/to/docs")}}: returns a main-memory collection with all XML documents found at the addressed file path.
==Raw FilesBinary Data== The {{Command|BINARY GET}} command and the {{Function|Database|db:get-binary}} function can be used to return files in their native byte representation.
=Update Resources=
==Commands==
Once you have created a database, additional commands exist to modify its contents:
* XML documents can be added with the <code>[[Commands#ADD{{Command|PUT}} and {{Command|ADD]]</code> command.* Raw files are added with <code>[[Commands#STORE|STORE]]</code>}} commands.* Existing resources can be replaced Binary data is stored with the <code>[[Commands#REPLACE{{Command|REPLACE]]</code> commandBINARY PUT}}.* Resources of all types can be deleted via <code>[[Commands#DELETE{{Command|DELETE]]</code>}}.
<pre>
SET ADDCACHE true
ADD /path/to/xml/documents
</pre>
You can also use functions from the [[Database Module]] to add, replace, or delete XML documents: <pre classsyntaxhighlight lang="brushxquery">db:add('documents', '/path/to/xml/resources/')</syntaxhighlight> Function from other modules, such as the [[File Module]], can be utilized to filter the input. With the following code, all files that contain numbers in the filename are selected, and stored as XML. If an input file contains no well-formed XML, it is stored as binary resource, and the error message is stored as a string value: <syntaxhighlight lang="xquery">let $db := 'documents'let $root := "'/path/to/xmlresources/documents/"'for $file path in file:list($root)where matches($path, '\d+')return try { db:addput($db, fetch:doc($root || $path), $path)} catch * { db:put-binary($db, $root || $path, $path), db:put-value($db, $err:description, $path || '.error')}</syntaxhighlight> The error messages can e.g. be analyzed in a second step: <syntaxhighlight lang="databasexquery">let $errors := db:get-value('documents')for $filename in map:keys($errors)where ends-with($filename, '.error')return $root filename || ': ' || $fileerrors?($filename)</presyntaxhighlight> =Export Database=
Additional notes:
* You can force an ordinary database, or parts of it, to being temporarily copied to memory by applying an empty [[XQuery_Update#Main-Memory_Updates|main-memory update]] on a database node: <code>db:get('some-db') update { }</code>
* If you open local or remote documents with <code>fn:doc</code> or <code>fn:collection</code>, the resulting internal representation is identical to those of main-memory database instances (regardless of which value is set for {{Option|MAINMEM}}).
=Changelog=
;Version 10.0
* Added: New resource type for XQuery values.
;Version 8.4
* Updated: [[#Raw Files|Raw Files]]: Items of binary type can be output without specifying the obsolete <code>raw</code> serialization method.
;Version 7.2.1
* Updated: {{Code|fn:document-uri}} and {{Code|fn:base-uri}} now return strings that can be reused with {{Code|fn:doc}} or {{Code|fn:collection}} to reopen the original document.
