Difference between revisions of "JSON Module"

From BaseX Documentation
Jump to navigation Jump to search
Line 128: Line 128:
 
|}
 
|}
  
==json:serialize==
+
==json:doc==
 +
 
 +
{{Mark|Introduced with BaseX 9.4:}}
  
 
{| width='100%'
 
{| width='100%'
 
|-
 
|-
 
| width='120' | '''Signatures'''
 
| width='120' | '''Signatures'''
|{{Func|json:serialize|$input as item()?|xs:string}}<br/>{{Func|json:serialize|$input as item()?, $options as map(*)?|xs:string}}
+
|{{Func|json:doc|$uri as xs:string|item()?}}<br />{{Func|json:doc|$uri as xs:string, $options as map(*)?|item()?}}<br />
 
|-
 
|-
 
| '''Summary'''
 
| '''Summary'''
|Serializes the specified {{Code|$input}} as JSON, using the specified {{Code|$options}}, and returns the result as string:
+
|Fetches the JSON document referred to by the given {{Code|$uri}} and converts it to an XQuery value. The {{Code|$options}} argument can be used to control the way the input is converted.
* The input is expected to conform to the results that are created by [[#json:parse|json:parse()]].
 
* Non-conforming items will be serialized as specified in the [[XQuery 3.1#JSON Serialization|json output method]] of the official recommendation.
 
Values can also be serialized as JSON with the standard [[Serialization]] feature of XQuery:
 
* The parameter {{Code|method}} needs to be set to {{Code|json}}, and
 
* the options presented in this article need to be assigned to the {{Code|json}} parameter.
 
 
|-
 
|-
 
| '''Errors'''
 
| '''Errors'''
|{{Error|serialize|#Errors}} the specified node cannot be serialized as JSON document.
+
|{{Error|parse|#Errors}} the specified input cannot be parsed as JSON document.<br/>{{Error|options|#Errors}} the specified options are conflicting.
 
|}
 
|}
  
==json:doc==
+
==json:serialize==
 
 
{{Mark|Introduced with BaseX 9.4:}}
 
  
 
{| width='100%'
 
{| width='100%'
 
|-
 
|-
 
| width='120' | '''Signatures'''
 
| width='120' | '''Signatures'''
|{{Func|json:doc|$uri as xs:string|item()?}}<br />{{Func|json:doc|$uri as xs:string, $options as map(*)?|item()?}}<br />
+
|{{Func|json:serialize|$input as item()?|xs:string}}<br/>{{Func|json:serialize|$input as item()?, $options as map(*)?|xs:string}}
 
|-
 
|-
 
| '''Summary'''
 
| '''Summary'''
|Fetches the JSON document referred to by the given {{Code|$uri}} and converts it to an XQuery value. The {{Code|$options}} argument can be used to control the way the input is converted.
+
|Serializes the specified {{Code|$input}} as JSON, using the specified {{Code|$options}}, and returns the result as string:
 +
* The input is expected to conform to the results that are created by [[#json:parse|json:parse()]].
 +
* Non-conforming items will be serialized as specified in the [[XQuery 3.1#JSON Serialization|json output method]] of the official recommendation.
 +
Values can also be serialized as JSON with the standard [[Serialization]] feature of XQuery:
 +
* The parameter {{Code|method}} needs to be set to {{Code|json}}, and
 +
* the options presented in this article need to be assigned to the {{Code|json}} parameter.
 
|-
 
|-
 
| '''Errors'''
 
| '''Errors'''
|{{Error|parse|#Errors}} the specified input cannot be parsed as JSON document.<br/>{{Error|options|#Errors}} the specified options are conflicting.
+
|{{Error|serialize|#Errors}} the specified node cannot be serialized as JSON document.
 
|}
 
|}
  

Revision as of 12:25, 3 April 2020

This XQuery Module contains functions to parse and serialize JSON data JSON (JavaScript Object Notation) is a popular data exchange format for applications written in JavaScript. As there are notable differences between JSON and XML, or XQuery data types, no mapping exists that guarantees a lossless, bidirectional conversion between JSON and XML. For this reason, we offer various mappings, all of which are suited to different use cases.

Conventions

All functions and errors in this module are assigned to the http://basex.org/modules/json namespace, which is statically bound to the json prefix.

Conversion Formats

A little advice: in the Database Creation dialog of the GUI, if you select JSON Parsing and switch to the Parsing tab, you can see the effects of some of the conversion options.

Direct

The direct conversion format allows a lossless conversion from JSON to XML and back. The transformation is based on the following rules:

  • The resulting document has a <json> root node.
  • Object pairs are represented via elements. The name of a pair is rewritten to an element name:
    • Empty names are represented by a single underscore (_). Existing underscores are rewritten to two underscores (__), and characters that are not valid in element names are rewritten to an underscore and the character’s four-digit Unicode.
    • If the lax option is set to true, invalid characters are simply replaced with underscores or (when invalid as first character of an element name) prefixed with an underscore. The resulting names are better readable, but cannot always be converted back to their original form.
  • Array entries are also represented via elements. _ is used as element name.
  • Object and array values are stored in text nodes.
  • The types of values are represented via type attributes:
    • The existing types are string, number, boolean, null, object, and array.
    • As most values are strings, the string type is by default omitted.

Attributes

The attributes format is lossless, too. The transformation based on the following rules:

  • The resulting document has a <json> root node.
  • Object pairs are represented via <pair> elements. The name of a pair is stored in a name attribute.
  • Array entries are represented via <item> elements.
  • Object and array values are stored in text nodes.
  • The types of values are represented via type attributes:
    • The existing types are string, number, boolean, null, object, and array.
    • As most values are strings, the string type is by default omitted.

Basic

The basic format is another lossless format. It converts a JSON document to an XML node and vice versa. The conversion rules are the same as for fn:json-to-xml.

JsonML

The jsonml format is designed to convert XML to JSON and back, using the JsonML dialect. JsonML allows the transformation of arbitrary XML documents, but namespaces, comments and processing instructions will be discarded in the transformation process. More details are found in the official JsonML documentation.

XQuery

The xquery format is lossless, too. It converts JSON data to an XQuery value (a map, array, string, number, boolean, or empty sequence) and vice versa. The conversion rules are the same as for fn:parse-json.

The resulting representation consumes less memory than XML-based formats, and values can be directly accessed without conversion. Thus, it is recommendable for very large inputs and for efficient ad-hoc processing.

Options

The following options are available (the Direction column indicates if an option applies to parsing, serialization, or both operations):

Option Description Allowed Default Direction
format Specifies the format for converting JSON data. direct, attributes, jsonml, xquery direct parse, serialize
liberal Determines if minor deviations from RFC 7159 will be ignored. yes, no no parse
merge This option is considered when direct or attributes conversion is used:
  • If a name has the same type throughout the data, the type attribute will be omitted. Instead, the name will be listed in additional, type-specific attributes in the root node.
  • The attributes are named by their type in plural (numbers, booleans, nulls, objects and arrays), and the attribute value contains all names with that type, separated by whitespaces.
yes, no no parse, serialize
strings Indicates if type attributes will be added for strings. yes, no yes parse, serialize
lax Specifies if a lax approach is used to convert QNames to JSON names. yes, no no parse, serialize
escape Indicates if escaped characters are expanded (for example, \n becomes a single x0A character, while \u20AC becomes the character ). yes, no yes parse
escape Indicates if characters are escaped whenever the JSON syntax requires it. This option can be set to no if strings are already in escaped form and no further escaping is permitted. yes, no yes serialize
indent Indicates if whitespace should be added to the output with the aim of improving human legibility. If the parameter is set as in the query prolog, it overrides the indent serialization parameter. yes, no yes serialize

Functions

json:parse

Signatures json:parse($string as xs:string?) as item()?
json:parse($string as xs:string?, $options as map(*)?) as item()?
Summary Converts the JSON $string to an XQuery value. If the input can be successfully parsed, it can be serialized back to the original JSON representation. The $options argument can be used to control the way the input is converted.
Errors parse: the specified input cannot be parsed as JSON document.
options: the specified options are conflicting.

json:doc

Template:Mark

Signatures json:doc($uri as xs:string) as item()?
json:doc($uri as xs:string, $options as map(*)?) as item()?
Summary Fetches the JSON document referred to by the given $uri and converts it to an XQuery value. The $options argument can be used to control the way the input is converted.
Errors parse: the specified input cannot be parsed as JSON document.
options: the specified options are conflicting.

json:serialize

Signatures json:serialize($input as item()?) as xs:string
json:serialize($input as item()?, $options as map(*)?) as xs:string
Summary Serializes the specified $input as JSON, using the specified $options, and returns the result as string:
  • The input is expected to conform to the results that are created by json:parse().
  • Non-conforming items will be serialized as specified in the json output method of the official recommendation.

Values can also be serialized as JSON with the standard Serialization feature of XQuery:

  • The parameter method needs to be set to json, and
  • the options presented in this article need to be assigned to the json parameter.
Errors serialize: the specified node cannot be serialized as JSON document.

Examples

BaseX Format

Example 1: Adds all JSON documents in a directory to a database

Query: <syntaxhighlight lang="xquery"> let $database := "database" for $name in file:list('.', false(), '*.json') let $file := file:read-text($name) let $json := json:parse($file) return db:add($database, $json, $name) </syntaxhighlight>

Example 2: Converts a simple JSON string to XML and back

Query: <syntaxhighlight lang="xquery"> json:parse('{}') </syntaxhighlight>

Result: <syntaxhighlight lang="xml"> <json type="object"/> </syntaxhighlight>

Query: <syntaxhighlight lang="xquery"> (: serialize result as plain text :) declare option output:method 'text'; json:serialize(<json type="object"/>) </syntaxhighlight>

Result: <syntaxhighlight lang="xquery"> { } </syntaxhighlight>

Example 3: Converts a JSON string with simple objects and arrays

Query: <syntaxhighlight lang="xquery"> json:parse('{

 "title": "Talk On Travel Pool",
 "link": "http://www.flickr.com/groups/talkontravel/pool/",
 "description": "Travel and vacation photos from around the world.",
 "modified": "2014-02-02T11:10:27Z",
 "generator": "http://www.flickr.com/"

}') </syntaxhighlight>

Result: <syntaxhighlight lang="xml"> <json type="object">

 <title>Talk On Travel Pool</title>
 <link>http://www.flickr.com/groups/talkontravel/pool/</link>
 <description>Travel and vacation photos from around the world.</description>
 <modified>2014-02-02T11:10:27Z</modified>
 <generator>http://www.flickr.com/</generator>

</json> </syntaxhighlight>

Example 4: Converts a JSON string with different data types

Query: <syntaxhighlight lang="xquery"> let $options := map { 'merge': true() } return json:parse('{

 "first_name": "John",
 "last_name": "Smith",
 "age": 25,
 "address": {
   "street": "21 2nd Street",
   "city": "New York",
   "code": 10021
 },
 "phone": [
   {
     "type": "home",
     "number": "212 555-1234"
   },
   {
     "type": "mobile",
     "number": 1327724623
   }
 ]

}', $options) </syntaxhighlight>

Result: <syntaxhighlight lang="xml"> <json numbers="age code" arrays="phone" objects="json address value">

 <first__name>John</first__name>
 <last__name>Smith</last__name>
 <age>25</age>
 <address>
   <street>21 2nd Street</street>
   <city>New York</city>
   10021
 </address>
 <phone>
   <_>
     <type>home</type>
     <number>212 555-1234</number>
   </_>
   <_>
     <type>mobile</type>
     <number type="number">1327724623</number>
   </_>
 </phone>

</json> </syntaxhighlight>

JsonML Format

Example 1: Converts all XML documents in a database to the JsonML format and writes them to disk

Query: <syntaxhighlight lang="xquery"> for $doc in collection('json') let $name := document-uri($doc) let $json := json:serialize($doc, map { 'format': 'jsonml' }) return file:write($name, $json) </syntaxhighlight>

Example 2: Converts an XML document with elements and text

Query: <syntaxhighlight lang="xquery"> json:serialize(doc('flickr.xml'), map { 'format': 'jsonml' }) </syntaxhighlight>

flickr.xml: <syntaxhighlight lang="xml"> <flickr>

 <title>Talk On Travel Pool</title>
 <link>http://www.flickr.com/groups/talkontravel/pool/</link>
 <description>Travel and vacation photos from around the world.</description>
 <modified>2014-02-02T11:10:27Z</modified>
 <generator>http://www.flickr.com/</generator>

</flickr> </syntaxhighlight>

Result: <syntaxhighlight lang="json"> ["flickr",

 ["title",
   "Talk On Travel Pool"],
 ["link",
   "http://www.flickr.com/groups/talkontravel/pool/"],
 ["description",
   "Travel and vacation photos from around the world."],
 ["modified",
   "2014-02-02T11:10:27Z"],
 ["generator",
   "http://www.flickr.com/"]]

</syntaxhighlight>

Example 3: Converts a document with nested elements and attributes to JsonML

Query: <syntaxhighlight lang="xquery"> json:serialize(doc('input.xml'), map { 'format': 'jsonml' }) </syntaxhighlight>

input.xml: <syntaxhighlight lang="xml"> <address id='1'>

 <last_name>Smith</last_name>
 <age>25</age>
 <address xmlns='will be dropped as well'>
   <street>21 2nd Street</street>
   <city>New York</city>
   10021
 </address>
 <phone type='home'>212 555-1234</phone>

</address> </syntaxhighlight>

Result: <syntaxhighlight lang="json"> ["address", {"id":"1"},

 ["last_name",
   "Smith"],
 ["age",
   "25"],
 ["address",
   ["street",
     "21 2nd Street"],
   ["city",
     "New York"],
   ["code",
     "10021"]],
 ["phone", {"type":"home"},
   "212 555-1234"]]

</syntaxhighlight>

XQuery Format

Example 1: Converts a JSON string to XQuery

Query: <syntaxhighlight lang="xquery"> let $input := '{

 "Title": "Drinks",
 "Author": [ "Jim Daniels", "Jack Beam" ]

}' let $data := json:parse($input, map { 'format': 'xquery' }) return map:for-each($data, function($k, $v) {

 $k || ': ' || string-join($v, ', ')

}) </syntaxhighlight>

Result: <syntaxhighlight lang="json"> Author: Jim Daniels, Jack Beam Title: Drinks </syntaxhighlight>

Example 2: Converts XQuery data to JSON

Query: <syntaxhighlight lang="xquery"> for $item in (

 true(),
 'ABC',
 array { 1 to 5 },
 map { "Key": "Value" }

) return json:serialize(

 $item,
 map { 'format': 'xquery', 'indent': 'no' }

) </syntaxhighlight>

Result: <syntaxhighlight lang="json"> true "ABC" [1,2,3,4,5] {"Key":"Value"} </syntaxhighlight>

Errors

Code Description
options The specified options are conflicting.
parse The specified input cannot be parsed as JSON document.
serialize The specified node cannot be serialized as JSON document.

Changelog

Version 9.4
Version 9.1
  • Updated: json:parse can be called with empty sequence.
Version 9.0
  • Updated: map format renamed to xquery.
  • Updated: error codes updated; errors now use the module namespace
Version 8.4
  • Updated: unescape changed to escape.
Version 8.2
  • Added: Conversion format basic.
Version 8.0
  • Updated: Serialization aligned with the json output method of the official specification.
  • Added: liberal option.
  • Removed: spec option.
Version 7.8
  • Removed: json:parse-ml, json:serialize-ml.
  • Updated: json:parse now returns a document node instead of an element, or an XQuery map if format is set to .map.
Version 7.7.2

The module was introduced with Version 7.0.