Main Page » Changelog

Changelog

The following features have been added, updated, or removed in BaseX 13:

Startup

  • Updated: Switched to Java 21

Graphical User Interface

  • Added: Editor: Code completion, function signatures, marked occurrences, code formatting, go to declaration, Markdown highlighting; word wrapping can be disabled; files changed on disk are reloaded.
  • Updated: Introduction: The Map view is disabled by default.
  • Added: Project View: Search and replace in all files, with backup and undo.
  • Added: FlatLaf: Support for the FlatLaf library.
  • Updated: Editor: Faster rendering of the editor and text panels.
  • Updated: Search and Replace: Count hits, improved interaction, scroll bar integration.
  • Updated: Look and Feel: Scaling relative to the operating system.
  • Updated: Startup: Single GUI instance per project directory.
  • Updated: Introduction: Panel sizes and arrangement are preserved across restarts; the menu bar was restructured.
  • Removed: Introduction: Fullscreen mode.

Shortcuts

  • Added: Multiple cursors (Alt Click, Alt J, Ctrl Alt Shift ↑/↓)

Command-Line Options

  • Added: -L to start a database server instance in addition (HTTP server).
  • Updated: stop: Servers are stopped via the network instead of a stop file, and only from the same host.

Catalog Resolver

  • Updated: Catalogs are only applied if external resources may be accessed.

Commands

  • Updated: Permissions: CREATE was merged into ADMIN.
  • Updated: SET: STOPWORDS and WRITEBACK can only be assigned with ADMIN permission.
  • Updated: ALTER BACKUP, ALTER DB, COPY, CREATE BACKUP, DROP BACKUP, DROP DB, RESTORE: Local permissions of the addressed databases are respected.
  • Updated: PUT, BINARY PUT: Replace or ignore new resources.
  • Updated: INFO: Libraries section. No permission required (before: READ).
  • Updated: LIST: Input paths are listed for all users.
  • Updated: INSPECT: Optional database name, additional checks, new output format.
  • Updated: OPTIMIZE: Incrementally updated indexes are compacted, unchanged indexes are skipped.
  • Updated: REPO INSTALL: Install BaseX modules by name.

Options

  • Updated: Create Options: XQuery functions ignore options assigned with SET, in .basex or as system properties.
  • Updated: STOPWORDS, WRITEBACK: Can only be assigned with admin permissions.
  • Added: FTMIXED: Index the string values of elements with mixed content.
  • Added: THESINDEX: Index the thesaurus documents of a database.
  • Added: AUTHALGORITHMS: Configure the password algorithms for which hashes are stored.
  • Added: TRUSTEXTERNAL: Default for the trust-external option of XQuery functions.
  • Added: XSILOCATION: Retrieve schemas referenced by xsi:schemaLocation attributes.
  • Added: RETAINLOCATION: Retain the source location of parsed nodes.
  • Added: REPLACE: PUT: replace or ignore existing resources.
  • Added: CACHEMAX: Maximum number of entries stored in a cache.
  • Added: CACHETTL: Lifetime of cache entries.
  • Added: HTTPPORT: Port of the HTTP server.
  • Added: HTTPHOST: Host name or IP address the HTTP server is bound to.
  • Updated: XMLPLAN: Choose the stage of the query plan.
  • Updated: UPDINDEX: New default: true. The full-text index is updatable as well.
  • Updated: HTTPLOCAL: New default: true.
  • Updated: PARALLEL: New default: 32.
  • Updated: STRIPWS: Only whitespace-only text nodes are discarded; whitespace in mixed content is preserved.
  • Updated: INTPARSE: Improved DTD support.
  • Updated: ADDARCHIVES: Parsing of Zstandard- and XZ-compressed files.
  • Updated: LOG: SLF4J levels warn and error for HTTP client and server errors.
  • Removed: MIXUPDATES: Use job:execute to run updating and non-updating queries side by side.
  • Removed: LSERROR: Use using fuzzy N errors or the errors option of the full-text functions.
  • Removed: FAIRLOCK: Non-fair locking is always used.
  • Removed: SPLITSIZE: Index structures are always split based on the available main memory.
  • Removed: ADDCACHE: Documents are moved to disk while they are being added if they get too large for main memory.
  • Removed: TAILCALLS: Tail calls are always eliminated.
  • Removed: FORCECREATE: Use db:create to create databases.
  • Removed: CREATEONLY: Use CLOSE to close a database after its creation.
  • Removed: OPTPLAN: Use XMLPLAN with compiled to output the plan before optimization.
  • Removed: CHECKSTRINGS: Invalid characters in strings from external sources are always rejected.
  • Removed: AUTHMETHOD: Digest value.

Configuration

  • Updated: The ancestors of the current user directory are checked as well.
  • Updated: .basex is never rewritten; only changed options are updated. Local options no longer need a separate section.

Integrating oXygen

  • Added: Instructions for accessing database resources via WebDAV were re-added.

Integrating Eclipse

  • Added: Instructions for accessing database resources via WebDAV were re-added.

XQuery 4.0

  • Added: JNodes: XPath navigation on map and array structures. Example: json-doc(…)//city.
  • Added: Destructuring: Assign components of sequences, arrays and maps to variables
  • Added: Record declarations: but with operator for updating record values. Example: $record but with { 'key': 'value' }.
  • Added: Trace Clause: Emit diagnostic output per tuple. Example: for $i in 1 to 3 trace $i return $i.
  • Added: Axes: additional axes that include the context node, e.g. preceding-or-self.
  • Added: Node Comparisons: New operators and aliases.
  • Added: Unicode Operators: × (U+00D7) and ÷ (U+00F7).
  • Updated: Record types: A record has a fixed set of fields. Optional fields (field?) and the wildcard (*) were removed, field access is strict, and subtyping is width-invariant.
  • Updated: Record declarations: Declared record types are nominative. Declarations with identical fields define distinct types.
  • Updated: Map Constructors: Syntax generalized to allow arbitrary expressions: { $map-entry }
  • Updated: Methods: Annotation %method replaced by =?> operator
  • Updated: Dynamic Function Calls: Dynamic evaluation of function sequences.
  • Updated: Function Declarations: local namespace prefix can be omitted.
  • Updated: Default Element Namespace: new keyword fixed and special value "##any".
  • Updated: QName literals: QName literals with prefix.
  • Updated: Casting: Casting from strings follows XSD 1.1; casts to xs:anyURI never fail.
  • Updated: node() and xnode() Types: node() also matches JNodes; xnode() matches XML nodes.
  • Updated: order by: NaN precedes all numbers, also with empty greatest.
  • Removed: Map and array filter was removed.
  • Removed: gnode() type (use node()).

Invisible XML

  • Updated: Now using Markup Blitz 1.13.

Functions

Standard Functions

  • Added: fn:atomic-type-annotation, fn:build-dateTime, fn:build-uri, fn:days-in-month, fn:drop-while, fn:duration-to-seconds, fn:element-to-map-plan, fn:element-to-map, fn:graphemes, fn:index-of-substring, fn:jkey, fn:jtree, fn:jvalue, fn:location, fn:map-to-element, fn:matching-segments, fn:node-type-annotation, fn:pad-string, fn:parts-of-dateTime, fn:regex, fn:scan, fn:schema-type, fn:substring-after-last, fn:substring-before-last, fn:system-properties, fn:transform, fn:trim-space, fn:unix-dateTime, fn:xsd-validator.
  • Updated: fn:analyze-string, fn:char, fn:codepoints-to-string, fn:collation, fn:decode-from-uri, fn:deep-equal, fn:doc, fn:doc-available, fn:insert-separator, fn:load-xquery-module, fn:parse-uri, fn:parse-xml, fn:parse-xml-fragment, fn:random-number-generator, fn:replace, fn:unparsed-text
  • Updated: Conventions: Option values must match the declared types; boolean options require booleans.
  • Removed: fn:csv-to-arrays (use fn:parse-csv), fn:function-identity, fn:intersperse, fn:sequence-join, fn:subsequence-where (use fn:drop-while, fn:take-while)

Map Functions

  • Updated: map:filter, map:for-each: Positional callback argument added.
  • Removed: map:keys-where, map:of-pairs, map:pairs.

Array Functions

  • Added: array:sort-by, array:sort-with.
  • Updated: array:members, array:of-members: JNodes are used instead of value records.
  • Removed: array:join: $separator parameter.

Archive Functions

  • Updated: archive:create: Requires admin permission.
  • Added: TAR archives (tar, tar.gz) can be created, updated and read. Zstandard and XZ compression (zstd, xz, tar.zst, tar.xz) are supported for creating and reading archives.
  • Updated: archive:create, archive:update, archive:write, archive:refresh: Duplicate entry names are rejected.

Binary Functions

  • Added: bin:infer-encoding: New function for inferring the encoding of binary data.
  • Added: bin:rotate: New function for cyclic bit rotation.
  • Added: bin:is-bit-set: New function for testing a single bit.
  • Added: bin:set-bits: New function for setting or clearing bits.
  • Added: bin:count-bits-set: New function for counting one-bits.
  • Added: bin:to-base64url, bin:from-base64url: New functions for Base64url conversions.

Cache Functions

  • Added: New module.

Conversion Functions

  • Updated: convert:dateTime-to-integer: A value without a timezone is interpreted in the implicit timezone.
  • Removed: convert:binary-to-string, convert:string-to-base64, convert:string-to-hex in favor of Binary Functions.
  • Removed: convert:integer-to-dayTime, convert:dayTime-to-integer in favor of fn:seconds.

Cryptographic Functions

  • Added: crypto:pbkdf2.
  • Updated: crypto:encrypt, crypto:decrypt: AES-GCM and RSA added; DES removed; algorithm names are case-insensitive.
  • Updated: crypto:generate-signature: RSA_SHA256, RSA_SHA512 and DSA_SHA256 added; defaults changed to SHA256 and RSA_SHA256.
  • Updated: crypto:generate-signature: Keystores require admin permission.

CSV Functions

  • Added: Options: comment-marker option.
  • Updated: Options: Support for backslash-escape sequences. Quotes in unquoted fields are adopted as literal characters. field-delimiter renamed back to separator.
  • Removed: Options: row-delimiter (rows are always delimited by a newline), skip-empty, strict-quoting; xquery and w3-arrays formats (use w3).

Database Functions

  • Added: db:counts: Count distinct values of nodes.
  • Added: db:inspect: Check storage structures for inconsistencies.
  • Updated: db:add: Multiple inputs are accepted; binary and value resources can be added.
  • Updated: db:create: Binary and value resources can be stored as initial contents.
  • Updated: db:put, db:put-value: Replace or ignore new resources.
  • Updated: db:put-binary: Replace or ignore new resources; $input restricted to binary items and strings (strings are interpreted as URIs).
  • Removed: db:create, db:add, db:put: addcache option; large inputs are moved to disk automatically.
  • Updated: db:get, db:text: $database can be addressed by a compact node.
  • Updated: db:system: libraries element.
  • Updated: db:list-details: The input path is returned to all users.

File Functions

  • Updated: file:read-text, file:read-text-lines, file:descendants: New $options parameter.
  • Updated: file:size: New $recursive parameter.
  • Updated: file:delete: Non-existing paths are ignored.
  • Updated: file:write-text-lines, file:append-text-lines: $values parameter renamed to $lines.
  • Updated: file:resolve-path: Absolute paths are normalized as well; a trailing directory separator is preserved.
  • Removed: file:read-text-lines: $offset and $length parameters removed.

Full-Text Functions

  • Added: ft:languages: List the recognized languages and the availability of stemmers and tokenizers.
  • Updated: ft:search: New stop-words option; full-text options declared in the query prolog are inherited; elements are returned if the index contains the string values of elements.
  • Updated: ft:contains: New stop-words and occurs options; full-text options declared in the query prolog are inherited.
  • Updated: ft:tokens: New fuzzy and errors options for looking up index tokens with a bounded number of errors.
  • Updated: ft:mark, ft:extract: Constructed nodes are accepted; the name of the marker element must be an NCName.
  • Updated: ft:count: Constructed nodes are accepted.
  • Updated: ft:thesaurus: Terms are normalized with the full-text options; the thesaurus index of a database is used; the queried term is not returned anymore; by default, one level is traversed; levels accepts a range and denotes the returned levels.
  • Updated: Tokenization: Letters that denote multiple characters are expanded (ß → ss, æ → ae, …).
  • Updated: Database Functions: Rebuild indexes of older versions to apply the standardized token normalization.

Geo Functions

  • Added: Module re-added with GML 3 support.

HTML Functions

  • Added: Validator.nu option fail-on-error.
  • Updated: If no parser is available, the HTML input will be wrapped in an html element.

HTTP Client Functions

  • Added: http:get, http:head, http:delete, http:options, http:post, http:put, http:patch, http:query, http:send: map-based interface with typed options and record results.
  • Added: http:send-request: proxy, verify and xml attributes.
  • Updated: http:send-request: follow-redirect accepts a maximum number of redirects; auth-method is case-insensitive; timeout accepts fractions of a second.
  • Updated: Compressed responses are requested by default; redirects are no longer forwarded to another origin with credentials, and downgrades from https to http are not followed.
  • Added: http:send-request: cookies attribute for collecting and resending cookies.
  • Added: http:send-request: href and version attributes in the http:response element; SHA-256 and SHA-512-256 digest authentication.
  • Updated: http:send-request: response header names are returned in lower case; application/xquery bodies are returned as strings; a BaseX user agent is sent by default.
  • Updated: http:send-request: binary response bodies are returned as lazy items; file-based request payloads are streamed.

Job Functions

  • Added: job:next, job:info
  • Updated: job:eval: A function item can be invoked instead of a query. Cron syntax and daylight saving changes are supported. permission, timeout, memory and serializer options added. A service must have a unique id; memory, timeout and permission are rejected.
  • Updated: job:execute: A function item can be invoked instead of a query. If the calling query holds locks, the job may only read databases read-locked by it.
  • Updated: job:list-details: cron, address and session attributes added.

Cron Syntax

  • Added: New article added.

JSON Functions

  • Added: W3-Mapping: New conversion format.
  • Added: Options: Option mapping for the w3-mapping format. Serialization option escape-solidus, parsing option json-lines.
  • Updated: Options: number-parser was replaced with number-format.
  • Removed: Options: Conversion formats xquery and basic (use w3 and w3-xml).

Process Functions

  • Updated: proc:system: New options binary, normalize-newlines and fallback; binary input; invalid characters are rejected.
  • Updated: proc:execute: New options binary, normalize-newlines and fallback; binary input; invalid characters are rejected.

Profiling Functions

  • Updated: prof:track: allocated option added.
  • Updated: prof:human: arbitrary numbers and durations accepted.

Random Functions

  • Updated: random:seeded-double, random:seeded-integer, random:seeded-permutation: The same seed yields different values than before. random:integer, random:seeded-integer: The maximum is no longer limited to 2147483647.

Repository Functions

  • Updated: repo:install: Install BaseX modules by name.

Request Functions

  • Added: request:body, request:query-map, request:form-map
  • Updated: Functions can also be called in WebSocket handlers and asynchronous jobs.

Sessions Functions

  • Added: sessions:list-details

SQL Functions

  • Added: sql:execute-batch: executes a prepared statement over multiple parameter sets.
  • Updated: sql:execute: generated-keys and null options added.
  • Updated: sql:execute-prepared: Parameters can be supplied as an array; null option added.
  • Updated: sql:prepare: generated-keys option added.

Store Functions

  • Added: store:info, store:close
  • Updated: All opened stores are now kept in main memory.

String Functions

  • Added: string:closest, string:levenshtein-distance, string:partial-ratio, string:token-sort-ratio, string:token-set-ratio, string:ngram-similarity, string:ngrams
  • Updated: The similarity functions and string:ngrams adopt the $options of ft:tokenize (case, diacritics, stemming, language).
  • Updated: string:soundex: a string without letters yields an empty string (before: 0000).
  • Removed: string:format in favor of fn:pad-string.

Unit Functions

  • Added: unit:timeout: abort tests that exceed a time limit.
  • Updated: unit:test: multiple expected error codes can be specified.

User Functions

  • Updated: user:drop: Requires admin permission.

Web Functions

  • Updated: web:error: $message can be an arbitrary item; serialization $options added.
  • Updated: web:response-header: message attribute is ignored by RESTXQ (deprecated Servlet API feature).
  • Updated: web:redirect: Status parameter added.
  • Removed: web:forward: server-side forwarding was dropped.

WebSocket Functions

  • Added: ws:list-details, ws:names, ws:ping
  • Updated: ws:eval: A function item can be invoked instead of a query; job options and serialization parameters.
  • Updated: ws:close: Optional close status and reason.

XQuery Functions

  • Added: xquery:for-each: New function for parallel iteration.
  • Added: xquery:reduce: New function for parallel aggregation.
  • Added: xquery:fork-any: New function returning the first available result.
  • Updated: xquery:eval: A function item can be invoked instead of a query. Allow decimal places for timeout value. The permission option also applies to returned function items.
  • Updated: xquery:eval-update: A function item can be invoked instead of a query.
  • Updated: xquery:fork-join: report and timeout options added.
  • Updated: xquery:parse: optimize option added.

Full-Text

  • Added: Full-Text Index: The index is updatable with UPDINDEX.
  • Updated: Mixed Content: Elements with mixed content can be indexed, and their matches are highlighted.
  • Updated: Thesaurus: Terms are normalized with the full-text options. Thesauri can be stored and indexed in databases. SKOS thesauri are supported. By default, one level is traversed. The query term is not returned as thesaurus term anymore.
  • Updated: Fuzzy Querying: With using fuzzy 0 errors, only exact matches are found.

Updates

  • Removed: MIXUPDATES option. Use job:execute to combine updating and non-updating operations.

Indexes

  • Added: Full-Text Index: The index is updatable with UPDINDEX. FTMIXED indexes the string values of elements with mixed content.
  • Added: Thesaurus Index: Thesauri stored in databases can be indexed with THESINDEX.
  • Updated: Full-Text Index: Faster fuzzy querying. Token normalization was standardized (rebuild indexes of older versions to apply it).
  • Updated: Text Index, Attribute Index: integer comparisons (equality and ranges) on integer-typed nodes are rewritten for value-index access.
  • Updated: Path Index, Name Index, Updates: insertions keep the path index, name index and statistics up to date.

XQuery Optimizations

  • Added: Node Construction.

Java Bindings

  • Updated: Module Imports: Classes that do not extend QueryModule require admin permission.
  • Removed: @ContextDependent annotation.
  • Removed: Permission.CREATE (use Permission.ADMIN).

Repository

  • Added: Web Applications: RESTXQ applications can be packaged as archives
  • Updated: Java classes are cached after the first import.
  • Updated: Listing: absolute paths are returned
  • Added: BaseX Modules: installation by name

XQuery Extensions

  • Added: %basex:memo annotation for the Memoization of function results.
  • Added: %basex:eval annotation for the Evaluation in Other Queries.
  • Updated: Debugging: The output is serialized with the BaseX method instead of the adaptive method.
  • Removed: The regular expression flags j, ! and ;.
  • Removed: The nondeterministic prefix for dynamic function calls: calls of functions that are statically unknown are now treated as nondeterministic.

Serialization

  • Added: Standard Parameters: indent-unit, line-ending, json-lines, escape-solidus, csv-header, csv-quote-character, csv-separator, canonical for Canonical XML output.
  • Added: Custom Parameters: expression for output as XQuery expression.
  • Updated: Standard Parameters: The csv output method is part of the official specification: it is the inverse of fn:parse-csv. indent-attributes is now a standard serialization parameter. xhtml: HTML5 is serialized by default. html-version, version: Control characters that the requested version does not allow now raise an error. adaptive: Numbers are output as JSON numbers, QNames as #prefix:local.
  • Removed: Custom Parameters: indents, newline and tabulator (use indent-unit and line-ending).

Web Application

  • Added: WebDAV has been re-added as a RESTXQ application.
  • Removed: Digest authentication.
  • Updated: BaseX is now based on the Jetty EE10 environment (Servlet 6.0) and can be deployed in any Jakarta EE 10 servlet container.
  • Updated: Static resources are served by a servlet of BaseX; the directory is specified via the path parameter.
  • Updated: No database server is started anymore by default (see HTTPLOCAL).
  • Updated: The HTTP port is specified via HTTPPORT; the jetty.xml configuration file is optional.
  • Updated: Session cookies are sent with SameSite=Lax.

RESTXQ

  • Updated: HTML Form Fields: Size limit for URL-encoded forms.
  • Added: Web Archives: applications can be deployed as ZIP archives.
  • Added: Default HTTP Methods: Support for QUERY method added.
  • Updated: Paths: %rest:path can be specified more than once.
  • Updated: Default HTTP Methods: Functions with method annotations are preferred.
  • Updated: Custom HTTP Methods: Allow header of OPTIONS lists the methods of the requested path; HEAD prefers GET functions.
  • Updated: Content Types: application/xquery bodies are converted to strings.
  • Added: Input Options: option declarations for all functions of a module.
  • Updated: Output: Without media-type and method, the content type is derived from the result.
  • Updated: Response: Specific status codes for requests that match no function; 403 for permission errors.
  • Updated: Scope Error Handlers: %rest:error can be combined with %rest:path.
  • Removed: Server-side forwarding (<rest:forward> element and web:forward function) was dropped.

Permissions

  • Updated: Security functions are also invoked for WebSocket handshakes.

WebSockets

  • Added: Path templates in WebSocket annotations.
  • Updated: Writing Applications: WebSocket connections are closed when their HTTP session ends.
  • Added: Permission checks for WebSocket handshakes.
  • Added: %ws:subprotocol annotation: sub-protocol negotiation.
  • Added: Serialization parameters for the results of WebSocket functions.
  • Updated: All Request Functions can be used in WebSocket handlers.
  • Updated: %ws:error annotation: also invoked for errors of WebSocket functions and of ws:eval. Without such a function, no error message will be sent to the client.
  • Updated: %ws:close annotation: close status and reason can be bound to variables.
  • Removed: %ws:header-param annotation (use the Request Functions instead).

REST

  • Added: QUERY Method: Evaluate the request body as XQuery expression.
  • Added: PUT Method: Replace or ignore new resources.

WebDAV

  • Added: WebDAV is now written in XQuery and runs as a RESTXQ application. Locks are kept in main memory; the ~webdav database is not used anymore.

DBA

  • Added: Stores view; Caches panel of the Activity view.
  • Updated: Full redesign: side-by-side panels, Workspace, Activity, CodeMirror 6, use of WebSockets.

Clients

  • Updated: Digest replaced with salted authentication.
  • Removed: Support for digest and cram-md5 authentication, and the outdated clients.

Server Protocol

  • Updated: Digest replaced with salted authentication
  • Removed: Digest authentication of clients of Version 8 to 12, and the client-side fallback to cram-md5 authentication

Java Examples

  • Removed: XML:DB API

User Management

  • Added: Trusted Code: Code can be untrusted.
  • Updated: Permissions: Permissions are also checked for databases that have already been opened.
  • Updated: Permissions: The create permission was merged into admin.
  • Added: Password algorithms can be configured via AUTHALGORITHMS, including PBKDF2 encryption.
  • Removed: The unsalted digest (MD5) algorithm.

Transaction Management

  • Updated: Updating queries acquire read locks for databases that are only read.
  • Removed: FAIRLOCK option: Non-fair locking is always used.

Logging

  • Updated: Uniform notation of client addresses; IPv6 addresses behind proxies are recognized.

Node Storage

  • Updated: Updatable Indexes: Segmented layout of the value indexes, including the full-text index.
  • Updated: Compact storage of namespaces, incremental persistence of the ID-PRE mapping, parallel reading of the node table.
⚡Generated with XQuery