Difference between revisions of "BaseX 10"

From BaseX Documentation
Jump to navigation Jump to search
(35 intermediate revisions by the same user not shown)
Line 1: Line 1:
''Draft, yet to be finalized''
+
After 15 years of continuous development, the first double-digit version of BaseX sees the light of day.
 
 
--------
 
 
 
After 15 years of continuous development, the first double-digit version of BaseX is about to see the light of day soon.
 
  
 
We have taken the version jump as an opportunity to perform some major refactorings of BaseX, both under the hood and on API and XQuery level. Before migrating your projects to the new version, some adjustments may be required, so please read this article carefully.
 
We have taken the version jump as an opportunity to perform some major refactorings of BaseX, both under the hood and on API and XQuery level. Before migrating your projects to the new version, some adjustments may be required, so please read this article carefully.
Line 9: Line 5:
 
=Prerequisites=
 
=Prerequisites=
  
BaseX 10 requires Java 11 or later to run. Databases created with BaseX 10 are backward compatible and can still be opened with BaseX 9.
+
BaseX 10 requires Java 11 or later to run. Databases created with the new version are backward compatible and can still be opened with BaseX 9.
  
=Applications=
+
=Migrating Applications=
  
 
The following modifications might be relevant when migrating existing applications:
 
The following modifications might be relevant when migrating existing applications:
  
 
* The default ports for web applications have been changed from 8984/8985 to 8080/8081.
 
* The default ports for web applications have been changed from 8984/8985 to 8080/8081.
* If a new application of BaseX is deployed, the {{Code|admin}} user can only be used after a custom password has been assigned, e.g., via the {{Command|PASSWORD}} command.
+
* The default admin password has been removed. The {{Code|admin}} user can only be used if a password has been assigned, e.g., via the {{Command|PASSWORD}} command.
 +
* The conventions for functions in [[Clients]] in other programming languages were revised.
 
* The <code>IGNOREHOSTNAME</code> option was dropped and merged with {{Option|IGNORECERT}}.
 
* The <code>IGNOREHOSTNAME</code> option was dropped and merged with {{Option|IGNORECERT}}.
* The conventions for functions in [[Clients]] in other programming languages were revised.
 
  
 
=Storage=
 
=Storage=
Line 24: Line 20:
 
==Whitespaces==
 
==Whitespaces==
  
With BaseX 10, all whitespaces are now preserved when importing XML resources, unless whitespace stripping is enabled.
+
All whitespaces are now preserved when importing XML resources, unless whitespace stripping is enabled.
  
The notorious {{Code|CHOP}} option was removed to prevent conflicting behavior caused by earlier installations. It was replaced by a new {{Code|STRIPWS}} option, which defaults to {{Code|false}}. In addition, the new default of the [[Serialization|serialization parameter]] {{Code|indent}} is {{Code|no}}.
+
The notorious {{Code|CHOP}} option was removed to prevent conflicting behavior caused by earlier installations. It was replaced by a new {{Option|STRIPWS}} option, which defaults to {{Code|false}}. In addition, the new default of the [[Serialization|serialization parameter]] {{Code|indent}} is {{Code|no}}.
  
 
Please be warned that the new default can throw off existing applications. If you want to restore the old behavior, you should assign the following values in your [[Configuration|.basex configuration file]], or the {{Code|web.xml}} file of your [[Web Application]]:
 
Please be warned that the new default can throw off existing applications. If you want to restore the old behavior, you should assign the following values in your [[Configuration|.basex configuration file]], or the {{Code|web.xml}} file of your [[Web Application]]:
Line 32: Line 28:
 
<syntaxhighlight lang="xquery">
 
<syntaxhighlight lang="xquery">
 
STRIPWS: true
 
STRIPWS: true
SERIALIZER: indent=no
+
SERIALIZER: indent=yes
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
In the GUI editor, a [[Shortcuts#Editor Shortcuts|shortcut]] and an icon were added to switch result indentation on and off.
  
 
In addition, databases may considerably increase in size, as whitespaces used for indenting an XML document will be interpreted and stored as additional text nodes. If your XML resources are structured and have no [[Full-Text#Mixed Content|mixed content]], it is advisable to enable whitespaces stripping when importing them to a database.
 
In addition, databases may considerably increase in size, as whitespaces used for indenting an XML document will be interpreted and stored as additional text nodes. If your XML resources are structured and have no [[Full-Text#Mixed Content|mixed content]], it is advisable to enable whitespaces stripping when importing them to a database.
Line 47: Line 45:
 
   'factbook',
 
   'factbook',
 
   map:merge(
 
   map:merge(
     for $country in db:open('factbook')//country
+
     for $country in db:get('factbook')//country
 
     return map:entry($country/@name, $country//city/name ! string())
 
     return map:entry($country/@name, $country//city/name ! string())
 
   ),
 
   ),
Line 64: Line 62:
 
==Backups==
 
==Backups==
  
The [[Commands#Backups|Backup Commands]] and [[Database_Module#Backups|Backup Functions]] were enhanced to back up general data: [[User Management|registered users]], [[Jobs Module#Services|scheduled services]], [[Store Module|key-value stores]].
+
The [[Commands#Backups|Backup Commands]] and [[Database_Module#Backups|Backup Functions]] were enhanced to back up general data: [[User Management|registered users]], [[Job Module#Services|scheduled services]], [[Store Module|key-value stores]].
 
 
=Commands=
 
 
 
The following commands have been revised:
 
 
 
* Added: {{Command|GET}}: Retrieves a single XML document.
 
* Added: {{Command|DIR}}: Lists directories and resources.
 
* Renamed: {{Code|GET}} → {{Command|SHOW OPTIONS}}
 
* Renamed: {{Code|STORE}} → {{Command|BINARY PUT}}
 
* Renamed: {{Code|RETRIEVE}} → {{Command|BINARY GET}}
 
* Renamed: {{Code|REPLACE}} → {{Command|PUT}}
 
  
 
=XQuery=
 
=XQuery=
Line 81: Line 68:
 
==Compilation==
 
==Compilation==
  
Several internal steps are performed when a query is executed (see [[XQuery Optimizations#Introduction|XQuery Optimizations]] for more details):
+
The compilation has been split up into multiple steps to improve locking.
  
1. The query is parsed, i.e., the original query string is transformed to an executable tree representation.
+
So far, several internal steps were already performed when executing a query (see [[XQuery Optimizations#Introduction|XQuery Optimizations]] for more details):
2. External values that are passed on by APIs are bound to variables and the query context. External values can be names of databases, or contribute to a name that will be assembled later on in the query.
+
 
3. The query is compiled and evaluated.
+
# The query is parsed, i.e., the original query string is transformed to an executable tree representation.
 +
# External values that are passed on by APIs are bound to variables and the query context. External values can be names of databases, or contribute to a name that will later on be constructed in the query.
 +
# The query is compiled and evaluated.
  
 
The [[Transaction_Management|transaction manager]] gathers the names of the databases that will be accessed by a query. If it is not possible to uniquely identify all databases that may be opened by the query, global locking will be applied, and all databases will be locked. Detection can fail if the names of databases depend on external input. It can also fail if a query is too complex to associate character strings with database operations.
 
The [[Transaction_Management|transaction manager]] gathers the names of the databases that will be accessed by a query. If it is not possible to uniquely identify all databases that may be opened by the query, global locking will be applied, and all databases will be locked. Detection can fail if the names of databases depend on external input. It can also fail if a query is too complex to associate character strings with database operations.
  
With BaseX 10, compilation has been split into two separate steps:
+
The compilation phase now comprises two separate steps:
  
# ''Compilation'' of logical, context-independent operations. External values are bound to the query, and deterministic code is rewritten, simplified and pre-evaluated.
+
# ''Compilation'' of logical, context-independent (static) operations. External values are bound to the query, and deterministic code is rewritten, simplified and pre-evaluated.
# ''Optimization'' of physical, context-based operations. Addressed databases are opened and checked for available indexes; current date/time is retrieved. The resulting code is further rewritten and optimized.
+
# ''Optimization'' of physical, context-based (dynamic) operations. Databases are opened and checked for available indexes; current date/time is retrieved. The resulting code is further rewritten and optimized.
  
 
Lock detection will be performed after the first step, and the code resulting from this step offers much more insight into which specific databases need to be locked. As a result, local locks can be applied to many more queries than before, and many queries can now run in parallel. An example:
 
Lock detection will be performed after the first step, and the code resulting from this step offers much more insight into which specific databases need to be locked. As a result, local locks can be applied to many more queries than before, and many queries can now run in parallel. An example:
Line 98: Line 87:
 
<syntaxhighlight lang="xquery">
 
<syntaxhighlight lang="xquery">
 
declare variable $n external;
 
declare variable $n external;
db:open('names-' || $n)
+
db:get('names-' || $n)
 
</syntaxhighlight>
 
</syntaxhighlight>
  
After the query has been parsed, a user-specific value (e.g., {{Code|123}}) will be bound to {{Code|$n}}. The variable will be inlined by the compiler, and the argument of {{Code|db:open}} will be pre-evaluated to {{Code|names123}}. It is then easy for the lock detector to collect the name of the database that needs to read-locked before the query is eventually executed.
+
After the query has been parsed, a user-specific value (e.g., {{Code|123}}) will be bound to {{Code|$n}}. The variable will be inlined by the compiler, and the argument of {{Function||db:get}} will be pre-evaluated to {{Code|names123}}. It is then easy for the lock detector to collect the name of the database that needs to read-locked before the query is eventually executed.
  
 
Another positive side effect of two-step compilation is that productive environments get faster in general: Queries can be compiled in parallel, and it’s only the optimization and evaluation of a query that may need to be delayed by locking.
 
Another positive side effect of two-step compilation is that productive environments get faster in general: Queries can be compiled in parallel, and it’s only the optimization and evaluation of a query that may need to be delayed by locking.
Line 107: Line 96:
 
==Main-Memory Updates==
 
==Main-Memory Updates==
  
XQuery Updates provides constructs to [[XQuery Update#Main-Memory Updates|update XML nodes in main memory]]. The data structures for in-memory representations of XML resources have been revised, such that updates can be performed orders of magnitudes faster than before. With BaseX 9.x, the following query runs for 6-7 minutes, whereas it can now be computed in 3 seconds:
+
XQuery Update provides constructs to [[XQuery Update#Main-Memory Updates|update XML nodes in main memory]]. The data structures for in-memory representations of XML resources have been revised, such that updates can be performed orders of magnitudes faster than before. With BaseX 9.x, the following query runs for several minutes, whereas it can now be computed in a few seconds:
  
 
<syntaxhighlight lang="xquery">
 
<syntaxhighlight lang="xquery">
Line 117: Line 106:
 
}
 
}
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
==Key-Value Store==
 +
 +
A new [[Store Module]] provides functions to organize values in a persistent main-memory key-value store. The store allows you to speed up access to frequently accessed data.
 +
 +
; Store data:
 +
 +
<syntaxhighlight lang="xquery">
 +
let $email := map:merge(
 +
  for $address in db:get('addressbook')//address
 +
  return map:entry($address/name, $address/email)
 +
)
 +
return store:put('emails', $email)
 +
</syntaxhighlight>
 +
 +
; Retrieve data:
 +
 +
<syntaxhighlight lang="xquery">
 +
let $name := 'Richard David James'
 +
return store:get('email')($name)
 +
</syntaxhighlight>
 +
 +
The store is persistent: Its contents are written to disk if BaseX is shut down, and retrieved again after a restart.
  
 
==Modules==
 
==Modules==
  
The new [[Store Module]] provides a persistent main-memory key-value store for speeding up operations on frequently accessed data.
+
Functions of all modules, excluding the [[File Module]], now consistently resolve relative URI references against the static base URI, and not the current working directory.
 +
 
 +
Various modules and functions have been revised, added, renamed or removed:
  
Some existing query modules have been revised:
+
{| class="wikitable"
 +
|-
 +
! Description
 +
! BaseX 10
 +
! BaseX 9
 +
|-
 +
| Retrieve XML resources
 +
| {{Function|Database|db:get}}
 +
| {{Code|db:open}}
 +
|-
 +
| Retrieve nodes with specified pre values
 +
| {{Function|Database|db:get-pre}}
 +
| {{Code|db:open-pre}}
 +
|-
 +
| Retrieve nodes with specified IDs
 +
| {{Function|Database|db:get-id}}
 +
| {{Code|db:open-id}}
 +
|-
 +
| Retrieve binary resources
 +
| {{Function|Database|db:get-binary}}
 +
| {{Code|db:retrieve}}
 +
|-
 +
| Retrieve value resources
 +
| {{Function|Database|db:get-value}}
 +
| ''new''
 +
|-
 +
| Add or replace resource
 +
| {{Function|Database|db:put}}, arguments swapped!
 +
| {{Code|db:replace}}
 +
|-
 +
| Add or replace binary resource
 +
| {{Function|Database|db:put-binary}}, arguments swapped!
 +
| {{Code|db:store}}
 +
|-
 +
| Add or replace value resource
 +
| {{Function|Database|db:put-value}}
 +
| ''new''
 +
|-
 +
| Get resource type
 +
| {{Function|Database|db:type}}
 +
| {{Code|db:is-raw}}, {{Code|db:is-xml}}
 +
|-
 +
| Fetch XML document
 +
| {{Function|Fetch|fetch:doc}}
 +
| {{Code|fetch:xml}}
 +
|-
 +
| Convert binary data to XML
 +
| {{Function|Fetch|fetch:binary-doc}}
 +
| {{Code|fetch:xml-binary}}
 +
|-
 +
| Module: Process Geo data
 +
| ''removed''
 +
| Geo Module
 +
|-
 +
| XQuery jobs
 +
| [[Job Module]]
 +
| Jobs Module
 +
|-
 +
| Return variable bindings of a job
 +
| {{Function|Job|job:bindings}}
 +
| ''new''
 +
|-
 +
| Return variable bindings of a job
 +
| {{Function|Job|job:remove}}
 +
| {{Code|jobs:stop}}
 +
|-
 +
| Module: Main-memory key-value store
 +
| [[Store Module]]
 +
| ''new''
 +
|-
 +
| Module: String computations
 +
| [[String Module]]
 +
| Strings Module
 +
|-
 +
| Format string
 +
| {{Function|String|string:format}}
 +
| {{Code|out:format}}
 +
|-
 +
| Return control characters
 +
| {{Function|String|string:cr}}, {{Function|String|string:nl}}, {{Function|String|string:tab}}
 +
| {{Code|out:cr}}, {{Code|out:nl}}, {{Code|out:tab}}
 +
|-
 +
| Module: Process ZIP files
 +
| ''removed''
 +
| ZIP Module
 +
|}
  
* [[Database Module]]:
+
=Commands=
** Added: {{Function|Database|db:put-value}}, {{Function|Database|db:get-value}}.
 
** {{Code|db:open}}/{{Code|db:open-id}}/{{Code|db:open-pre}} → {{Function|Database|db:get}}/{{Function|Database|db:get-id}}/{{Function|Database|db:get-pre}}
 
** {{Code|db:replace}} → {{Function|Database|db:put}}: Second and third arguments have been swapped.
 
** {{Code|db:store}} → {{Function|Database|db:put-binary}}: Second and third arguments have been swapped.
 
** {{Code|db:retrieve}} → {{Function|Database|db:get-binary}}
 
** {{Code|db:is-raw}}/{{Code|db:is-xml}} → {{Function|Database|db:type}}
 
* [[Fetch Module]]:
 
** {{Code|fetch:xml}} → {{Function|Database|fetch:doc}}
 
** {{Code|fetch:xml-binary}} → {{Function|Database|fetch:binary-doc}}
 
* The Jobs Module has been renamed to [[Job Module]].
 
* The Strings Module has been renamed to [[String Module]].
 
* The deprecated ZIP Module has been removed.
 
  
Functions of all modules, excluding the [[File Module]], now consistently resolve relative URI references against the static base URI, and not the current working directory.
+
The following commands have been revised:
 +
 
 +
{| class="wikitable"
 +
|-
 +
! Description
 +
! BaseX 10
 +
! BaseX 9
 +
|-
 +
| List directories and resources.
 +
| {{Command|DIR}}
 +
| ''new''
 +
|-
 +
| Retrieve single XML document
 +
| {{Command|GET}}
 +
| ''new''
 +
|-
 +
| Retrieve binary resource
 +
| {{Command|BINARY GET}}
 +
| {{Code|RETRIEVE}}
 +
|-
 +
| Add or replace resources
 +
| {{Command|PUT}}
 +
| {{Code|REPLACE}}
 +
|-
 +
| Store binary resource
 +
| {{Command|BINARY PUT}}
 +
| {{Code|STORE}}
 +
|-
 +
| Returns current option values
 +
| {{Command|SHOW OPTIONS}}
 +
| {{Code|GET}}
 +
|-
 +
| Lists jobs
 +
| ''removed''
 +
| {{Command|JOBS LIST}}
 +
|-
 +
| Returns a job result
 +
| ''removed''
 +
| {{Command|JOBS RESULT}}
 +
|-
 +
| Stops a job
 +
| ''removed''
 +
| {{Command|JOBS STOP}}
 +
|}
  
 
=HTTP Requests=
 
=HTTP Requests=
  
HTTP requests in BaseX take advantage of the new [https://openjdk.org/groups/net/httpclient/intro.html Java HTTP Client]. This client provides a better overall performance, uses internal connection pools and follows redirects across different protocols (http, https).
+
HTTP requests in BaseX are now based on the new [https://openjdk.org/groups/net/httpclient/intro.html Java HTTP Client]. This client provides a better overall performance, uses internal connection pools and follows redirects across different protocols (http, https).
  
 
HTTP operations are, among others, performed by:
 
HTTP operations are, among others, performed by:
Line 151: Line 281:
 
* the {{Command|CREATE DB}} and {{Command|REPO INSTALL}} commands.
 
* the {{Command|CREATE DB}} and {{Command|REPO INSTALL}} commands.
  
=[[REST]]=
+
=Catalogs=
  
Results in the {{Code|rest}} namespace are now returned unprefixed:
+
From early on, catalog resolvers had been neglected both in BaseX and Java. This has changed: The new [https://docs.oracle.com/en/java/javase/11/core/xml-catalog-api1.html XML Catalog API] from Java is universally used to resolve references to external resources. As an alternative, Norman Walsh’s [https://xmlresolver.org/ Enhanced XML Resolver] is utilized if it is found in the classpath.
  
<syntaxhighlight lang="xml">
+
The option for supplying the XML catalog was renamed from <code>CATFILE</code> to {{Option|CATALOG}}. See [[Catalog Resolver]] for more details.
<!-- before -->
+
 
<rest:databases xmlns:rest="http://basex.org/rest"/>
+
=Graphical User Interface=
  
<!-- before -->
+
The graphical user interface of BaseX has been revised and made more consistent.
<databases xmlns="http://basex.org/rest"/>
 
</syntaxhighlight>
 
  
When listing the resources of a database, {{Code|dir}} elements are returned for resources that are located in subdirectories.
+
The icons were replaced by scalable ones, building upon the [https://openjdk.org/jeps/263 HiDPI graphics support for Windows and Linux].
  
=Catalogs=
+
=REST=
  
From early on, catalog resolvers had been neglected both in BaseX and Java. This has changed: The new [https://docs.oracle.com/en/java/javase/11/core/xml-catalog-api1.html XML Catalog API] from Java is now universally used to resolve references to external resources. As an alternative, Norman Walsh’s [https://xmlresolver.org/ Enhanced XML Resolver] will be used if it is found in the classpath.
+
Results in the {{Code|rest}} namespace are now returned without prefix:
  
The option for supplying the XML catalog was renamed from <code>CATFILE</code> to {{Option|CATALOG}}. See [[Catalog Resolver]] for more details.
+
<syntaxhighlight lang="xml">
 +
<!-- before -->
 +
<rest:databases xmlns:rest="http://basex.org/rest"/>
  
<syntaxhighlight lang="xquery">
+
<!-- now -->
-Djavax.xml.catalog.resolve=continue
+
<databases xmlns="http://basex.org/rest"/>
Q{java:System}setProperty('javax.xml.catalog.resolve', 'continue')
 
 
</syntaxhighlight>
 
</syntaxhighlight>
  
=Graphical User Interface=
+
When listing the resources of a database, {{Code|dir}} elements are returned for resources that are located in subdirectories. See [[REST]] for more details.
 
 
The graphical user interface of BaseX has been revised and made more consistent.
 
 
 
The icons were replaced by scalable ones, building upon the [https://openjdk.org/jeps/263 HiDPI graphics support for Windows and Linux].
 

Revision as of 15:55, 4 August 2022

After 15 years of continuous development, the first double-digit version of BaseX sees the light of day.

We have taken the version jump as an opportunity to perform some major refactorings of BaseX, both under the hood and on API and XQuery level. Before migrating your projects to the new version, some adjustments may be required, so please read this article carefully.

Prerequisites

BaseX 10 requires Java 11 or later to run. Databases created with the new version are backward compatible and can still be opened with BaseX 9.

Migrating Applications

The following modifications might be relevant when migrating existing applications:

  • The default ports for web applications have been changed from 8984/8985 to 8080/8081.
  • The default admin password has been removed. The admin user can only be used if a password has been assigned, e.g., via the PASSWORD command.
  • The conventions for functions in Clients in other programming languages were revised.
  • The IGNOREHOSTNAME option was dropped and merged with IGNORECERT.

Storage

Whitespaces

All whitespaces are now preserved when importing XML resources, unless whitespace stripping is enabled.

The notorious CHOP option was removed to prevent conflicting behavior caused by earlier installations. It was replaced by a new STRIPWS option, which defaults to false. In addition, the new default of the serialization parameter indent is no.

Please be warned that the new default can throw off existing applications. If you want to restore the old behavior, you should assign the following values in your .basex configuration file, or the web.xml file of your Web Application:

<syntaxhighlight lang="xquery"> STRIPWS: true SERIALIZER: indent=yes </syntaxhighlight>

In the GUI editor, a shortcut and an icon were added to switch result indentation on and off.

In addition, databases may considerably increase in size, as whitespaces used for indenting an XML document will be interpreted and stored as additional text nodes. If your XML resources are structured and have no mixed content, it is advisable to enable whitespaces stripping when importing them to a database.

Value Resources

In addition to XML and binary resources, a third resource type has been added: XQuery values (atomic items and nodes, sequences, maps, arrays) can now be stored in databases as well. The db:put-value and db:get-value can be used to store to and retrieve values.

The new feature can e.g. be used to store maps in a database:

<syntaxhighlight lang="xquery"> db:put-value(

 'factbook',
 map:merge(
   for $country in db:get('factbook')//country
   return map:entry($country/@name, $country//city/name ! string())
 ),
 'cities'

) </syntaxhighlight>

…and use them as index later on:

<syntaxhighlight lang="xquery"> let $cities := db:get-value('factbook', 'cities') for $country in ('Japan', 'Indonesia', 'Malaysia') return $country || ': ' || string-join($cities?($country), ', ') </syntaxhighlight>

Backups

The Backup Commands and Backup Functions were enhanced to back up general data: registered users, scheduled services, key-value stores.

XQuery

Compilation

The compilation has been split up into multiple steps to improve locking.

So far, several internal steps were already performed when executing a query (see XQuery Optimizations for more details):

  1. The query is parsed, i.e., the original query string is transformed to an executable tree representation.
  2. External values that are passed on by APIs are bound to variables and the query context. External values can be names of databases, or contribute to a name that will later on be constructed in the query.
  3. The query is compiled and evaluated.

The transaction manager gathers the names of the databases that will be accessed by a query. If it is not possible to uniquely identify all databases that may be opened by the query, global locking will be applied, and all databases will be locked. Detection can fail if the names of databases depend on external input. It can also fail if a query is too complex to associate character strings with database operations.

The compilation phase now comprises two separate steps:

  1. Compilation of logical, context-independent (static) operations. External values are bound to the query, and deterministic code is rewritten, simplified and pre-evaluated.
  2. Optimization of physical, context-based (dynamic) operations. Databases are opened and checked for available indexes; current date/time is retrieved. The resulting code is further rewritten and optimized.

Lock detection will be performed after the first step, and the code resulting from this step offers much more insight into which specific databases need to be locked. As a result, local locks can be applied to many more queries than before, and many queries can now run in parallel. An example:

<syntaxhighlight lang="xquery"> declare variable $n external; db:get('names-' || $n) </syntaxhighlight>

After the query has been parsed, a user-specific value (e.g., 123) will be bound to $n. The variable will be inlined by the compiler, and the argument of db:get will be pre-evaluated to names123. It is then easy for the lock detector to collect the name of the database that needs to read-locked before the query is eventually executed.

Another positive side effect of two-step compilation is that productive environments get faster in general: Queries can be compiled in parallel, and it’s only the optimization and evaluation of a query that may need to be delayed by locking.

Main-Memory Updates

XQuery Update provides constructs to update XML nodes in main memory. The data structures for in-memory representations of XML resources have been revised, such that updates can be performed orders of magnitudes faster than before. With BaseX 9.x, the following query runs for several minutes, whereas it can now be computed in a few seconds:

<syntaxhighlight lang="xquery"> <xml>{

 (1 to 1000000) ! <child/>

}</xml> update {

 for $child at $pos in child
 return insert node text { $pos } into $child

} </syntaxhighlight>

Key-Value Store

A new Store Module provides functions to organize values in a persistent main-memory key-value store. The store allows you to speed up access to frequently accessed data.

Store data

<syntaxhighlight lang="xquery"> let $email := map:merge(

 for $address in db:get('addressbook')//address
 return map:entry($address/name, $address/email)

) return store:put('emails', $email) </syntaxhighlight>

Retrieve data

<syntaxhighlight lang="xquery"> let $name := 'Richard David James' return store:get('email')($name) </syntaxhighlight>

The store is persistent: Its contents are written to disk if BaseX is shut down, and retrieved again after a restart.

Modules

Functions of all modules, excluding the File Module, now consistently resolve relative URI references against the static base URI, and not the current working directory.

Various modules and functions have been revised, added, renamed or removed:

Description BaseX 10 BaseX 9
Retrieve XML resources db:get db:open
Retrieve nodes with specified pre values db:get-pre db:open-pre
Retrieve nodes with specified IDs db:get-id db:open-id
Retrieve binary resources db:get-binary db:retrieve
Retrieve value resources db:get-value new
Add or replace resource db:put, arguments swapped! db:replace
Add or replace binary resource db:put-binary, arguments swapped! db:store
Add or replace value resource db:put-value new
Get resource type db:type db:is-raw, db:is-xml
Fetch XML document fetch:doc fetch:xml
Convert binary data to XML fetch:binary-doc fetch:xml-binary
Module: Process Geo data removed Geo Module
XQuery jobs Job Module Jobs Module
Return variable bindings of a job job:bindings new
Return variable bindings of a job job:remove jobs:stop
Module: Main-memory key-value store Store Module new
Module: String computations String Module Strings Module
Format string string:format out:format
Return control characters string:cr, string:nl, string:tab out:cr, out:nl, out:tab
Module: Process ZIP files removed ZIP Module

Commands

The following commands have been revised:

Description BaseX 10 BaseX 9
List directories and resources. DIR new
Retrieve single XML document GET new
Retrieve binary resource BINARY GET RETRIEVE
Add or replace resources PUT REPLACE
Store binary resource BINARY PUT STORE
Returns current option values SHOW OPTIONS GET
Lists jobs removed JOBS LIST
Returns a job result removed JOBS RESULT
Stops a job removed JOBS STOP

HTTP Requests

HTTP requests in BaseX are now based on the new Java HTTP Client. This client provides a better overall performance, uses internal connection pools and follows redirects across different protocols (http, https).

HTTP operations are, among others, performed by:

Catalogs

From early on, catalog resolvers had been neglected both in BaseX and Java. This has changed: The new XML Catalog API from Java is universally used to resolve references to external resources. As an alternative, Norman Walsh’s Enhanced XML Resolver is utilized if it is found in the classpath.

The option for supplying the XML catalog was renamed from CATFILE to CATALOG. See Catalog Resolver for more details.

Graphical User Interface

The graphical user interface of BaseX has been revised and made more consistent.

The icons were replaced by scalable ones, building upon the HiDPI graphics support for Windows and Linux.

REST

Results in the rest namespace are now returned without prefix:

<syntaxhighlight lang="xml"> <rest:databases xmlns:rest="http://basex.org/rest"/>

<databases xmlns="http://basex.org/rest"/> </syntaxhighlight>

When listing the resources of a database, dir elements are returned for resources that are located in subdirectories. See REST for more details.