Difference between revisions of "JSON Module"

From BaseX Documentation
Jump to navigation Jump to search
m (Text replacement - "syntaxhighlight" to "pre")
Tags: Mobile web edit Mobile edit
 
(166 intermediate revisions by 8 users not shown)
Line 1: Line 1:
[http://www.json.org/ JSON (JavaScript Object Notation)] is a popular data exchange format for applications written in JavaScript. This module contains [[Querying#Functions|XQuery functions]] to parse and serialize JSON documents. All functions are preceded by the <code>json:</code> prefix, which is linked to the <code><nowiki>http://www.basex.org/json</nowiki></code> namespace. The JSON module will be officially supported with the upcoming <font color='orangered'>Version 6.8</font> of BaseX.
+
This [[Module Library|XQuery Module]] contains functions to parse and serialize JSON data [https://www.json.org/ JSON (JavaScript Object Notation)] is a popular data exchange format for applications written in JavaScript. As there are notable differences between JSON and XML, or XQuery data types, no mapping exists that guarantees a lossless, bidirectional conversion between JSON and XML. For this reason, we offer various mappings, all of which are suited to different use cases.
  
As there are notable differences between JSON and XML, no mapping exists that guarantees a lossless, bidirectional conversion between JSON and XML. For this reason, we offer two sets of functions in this module:
+
=Conventions=
  
=JSON=
+
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/json</nowiki></code> namespace, which is statically bound to the {{Code|json}} prefix.<br/>
  
[[JSON Functions#json:parse|json:parse]] and [[JSON Functions#json:serialize|json:serialize]] facilitate a lossless conversion from JSON to XML and back. The transformation is based on the following rules:
+
==Conversion Formats==
# The resulting document has a <code><json/></code> root node.  
+
 
# Names (keys) of objects are represented as elements:
+
'''A little advice''': in the Database Creation dialog of the GUI, if you select JSON Parsing and switch to the ''Parsing'' tab, you can see the effects of some of the conversion options.
## Empty names are represented by a single underscore (<code>&lt;_&gt;...&lt;/_&gt;</code>).
+
 
## Underscore characters are rewritten to two underscores (<code>__</code>).
+
===Direct===
## A character that cannot be represented as NCName character is rewritten to an underscore and its four-digit Unicode.
+
 
# As arrays have no names, <code>&lt;value/&gt;</code> is used as element name.
+
The {{Code|direct}} conversion format allows a lossless conversion from JSON to XML and back. The transformation is based on the following rules:
# JSON values are represented as text nodes.
+
 
# The types of values are represented in attributes:
+
* The resulting document has a {{Code|json}} root node.  
## The value types ''number'', ''boolean'', ''null'', ''object'' and ''array'' are represented by a <code>type</code> attribute.
+
* Object pairs are represented via elements. The name of a pair is encoded, as described in the [[Conversion Module#Keys|Conversion Module]], and used as element name.
## The ''string'' type is omitted, as it is treated as default type.
+
* Array entries are also represented via elements, with {{Code|_}} as element name.
## If a name has the same type throughout the document, the <code>type</code> attribute will be omitted. Instead, the name will be listed in additional, type-specific attributes in the root node. The attributes are named by their type in the plural (''numbers'', ''booleans'', ''nulls'', ''objects'' and ''arrays''), and the attribute value contains all names with that type, separated by whitespaces.
+
* Object and array values are stored in text nodes.
 +
* The types of values are represented via {{Code|type}} attributes:
 +
** The existing types are ''string'', ''number'', ''boolean'', ''null'', ''object'', and ''array''.
 +
** As most values are strings, the ''string'' type is by default omitted.
 +
 
 +
===Attributes===
 +
 
 +
The {{Code|attributes}} format is lossless, too. The transformation based on the following rules:
 +
 
 +
* The resulting document has a {{Code|json}} root node.
 +
* Object pairs are represented via {{Code|pair}} elements. The name of a pair is stored in a {{Code|name}} attribute.
 +
* Array entries are represented via {{Code|item}} elements.
 +
* Object and array values are stored in text nodes.
 +
* The types of values are represented via {{Code|type}} attributes:
 +
** The existing types are ''string'', ''number'', ''boolean'', ''null'', ''object'', and ''array''.
 +
** As most values are strings, the ''string'' type is by default omitted.
 +
 
 +
===Basic===
 +
 
 +
The {{Code|basic}} format is another lossless format. It converts a JSON document to an XML node and vice versa. The conversion rules are the same as for [[XQuery 3.1#fn:json-to-xml|fn:json-to-xml]].
 +
 
 +
===JsonML===
 +
 
 +
The {{Code|jsonml}} format is designed to convert XML to JSON and back, using the JsonML dialect. JsonML allows the transformation of arbitrary XML documents, but namespaces, comments and processing instructions will be discarded in the transformation process. More details are found in the official [http://jsonml.org/XML JsonML documentation].
 +
 
 +
===XQuery===
 +
 
 +
The {{Code|xquery}} format is lossless, too. It converts JSON data to an XQuery value (a map, array, string, number, boolean, or empty sequence) and vice versa. The conversion rules are the same as for [[XQuery 3.1#fn:parse-json|fn:parse-json]].
 +
 
 +
The resulting representation consumes less memory than XML-based formats, and values can be directly accessed without conversion. Thus, it is recommendable for very large inputs and for efficient ad-hoc processing.
 +
 
 +
==Options==
 +
 
 +
The following options are available (the ''Direction'' column indicates if an option applies to parsing, serialization, or both operations):
 +
 
 +
{| class="wikitable sortable" width="100%"
 +
|- valign="top"
 +
! width="140" | Option
 +
! width="50%" | Description
 +
! Allowed
 +
! Default
 +
! Direction
 +
|- valign="top"
 +
| {{Code|format}}
 +
| Specifies the format for converting JSON data ([[#Conversion Formats|see above]]).
 +
| {{Code|direct}}, {{Code|attributes}}, {{Code|basic}}, {{Code|jsonml}}, {{Code|xquery}}
 +
| {{Code|direct}}
 +
| ''parse'', ''serialize''
 +
|- valign="top"
 +
| {{Code|liberal}}
 +
| Determines if minor deviations from [https://www.rfc-editor.org/rfc/rfc7159.txt RFC 7159] will be ignored.
 +
| {{Code|yes}}, {{Code|no}}
 +
| {{Code|no}}
 +
| ''parse''
 +
|- valign="top"
 +
| {{Code|merge}}
 +
| This option is considered when {{Code|direct}} or {{Code|attributes}} conversion is used:<br/>
 +
* If a name has the same type throughout the data, the {{Code|type}} attribute will be omitted. Instead, the name will be listed in additional, type-specific attributes in the root node.
 +
* The attributes are named by their type in plural (''numbers'', ''booleans'', ''nulls'', ''objects'' and ''arrays''), and the attribute value contains all names with that type, separated by whitespaces.
 +
| {{Code|yes}}, {{Code|no}}
 +
| {{Code|no}}
 +
| ''parse'', ''serialize''
 +
|- valign="top"
 +
| {{Code|strings}}
 +
| Indicates if {{Code|type}} attributes will be added for strings.
 +
| {{Code|yes}}, {{Code|no}}
 +
| {{Code|yes}}
 +
| ''parse'', ''serialize''
 +
|- valign="top"
 +
| {{Code|lax}}
 +
| Specifies if [[Conversion Module#Keys|lax conversion rules]] are used to convert QNames to JSON names.
 +
| {{Code|yes}}, {{Code|no}}
 +
| {{Code|no}}
 +
| ''parse'', ''serialize''
 +
|- valign="top"
 +
| {{Code|escape}}
 +
| Indicates if escaped characters are expanded (for example, {{Code|\n}} becomes a single {{Code|x0A}} character, while {{Code|\u20AC}} becomes the character {{Code|€}}).
 +
| {{Code|yes}}, {{Code|no}}
 +
| {{Code|yes}}
 +
| ''parse''
 +
|- valign="top"
 +
| {{Code|escape}}
 +
| Indicates if characters are escaped whenever the JSON syntax requires it. This option can be set to {{Code|no}} if strings are already in escaped form and no further escaping is permitted.
 +
| {{Code|yes}}, {{Code|no}}
 +
| {{Code|yes}}
 +
| ''serialize''
 +
|- valign="top"
 +
| {{Code|indent}}
 +
| Indicates if whitespace should be added to the output with the aim of improving human legibility. If the parameter is set as in the query prolog, it overrides the {{Code|indent}} [[Serialization|serialization parameter]].
 +
| {{Code|yes}}, {{Code|no}}
 +
| {{Code|no}}
 +
| ''serialize''
 +
|}
 +
 
 +
=Functions=
 +
 
 +
==json:doc==
 +
 
 +
{| width='100%'
 +
|- valign="top"
 +
| width='120' | '''Signature'''
 +
|<pre>json:doc(
 +
  $href    as xs:string,
 +
  $options  as map(*)?    := map { }
 +
) as item()?</pre>
 +
|- valign="top"
 +
| '''Summary'''
 +
|Fetches the JSON document referred to by the given {{Code|$href}} and converts it to an XQuery value. The {{Code|$options}} argument can be used to control the way the input is converted.
 +
|- valign="top"
 +
| '''Errors'''
 +
|{{Error|parse|#Errors}} the specified input cannot be parsed as JSON document.<br/>{{Error|options|#Errors}} the specified options are conflicting.
 +
|}
  
 
==json:parse==
 
==json:parse==
{|
+
 
|-
+
{| width='100%'
| valign='top' width='90' | '''Signatures'''
+
|- valign="top"
|<code><b>json:parse</b>($input as xs:string()) as element(json)</code>
+
| width='120' | '''Signature'''
|-
+
|<pre>json:parse(
| valign='top' | '''Summary'''
+
  $value  as xs:string?,
|Converts the JSON document specified by <code>$input</code> to XML, and returns the result as <code>element(json)</code> instance. The converted XML document is both well readable and lossless, i.e., the converted document can be serialized back to the original JSON representation.
+
  $options  as map(*)?    := map { }
|-
+
) as item()?</pre>
| valign='top' | '''Errors'''
+
|- valign="top"
|<b>[[XQuery Errors#BaseX Errors|BASX0015]]</b> is raised if the specified input cannot be parsed as JSON document.
+
| '''Summary'''
 +
|Converts the JSON {{Code|$value}} to an XQuery value. If the input can be successfully parsed, it can be serialized back to the original JSON representation. The {{Code|$options}} argument can be used to control the way the input is converted.
 +
|- valign="top"
 +
| '''Errors'''
 +
|{{Error|parse|#Errors}} the specified input cannot be parsed as JSON document.<br/>{{Error|options|#Errors}} the specified options are conflicting.
 
|}
 
|}
  
 
==json:serialize==
 
==json:serialize==
{|
+
 
|-
+
{| width='100%'
| valign='top' width='90' | '''Signatures'''
+
|- valign="top"
|<code><b>json:serialize</b>($input as node()) as xs:string()</code>
+
| width='120' | '''Signature'''
|-
+
|<pre>json:serialize(
| valign='top' | '''Summary'''
+
  $input   as item()?,
|Serializes the node specified by <code>$input</code> as JSON, and returns the result as <code>xs:string</code> instance. The serialized node must conform to the syntax specified by the [[JSON Functions#json:parse|json:parse()]] function.<br />XML documents can also be serialized as JSON if the [[Serialization|Serialization Option]] <code>"method"</code> is set to <code>"json"</code>.
+
  $options  as map(*)?  := map { }
|-
+
) as xs:string</pre>
| valign='top' | '''Errors'''
+
|- valign="top"
|<b>[[XQuery Errors#BaseX Errors|BASX0016]]</b> is raised if the specified node cannot be serialized as JSON document.
+
| '''Summary'''
 +
|Serializes the specified {{Code|$input}} as JSON, using the specified {{Code|$options}}, and returns the result as string:
 +
* The input is expected to conform to the results that are created by {{Function||json:parse}}.
 +
* Non-conforming items will be serialized as specified in the [[XQuery 3.1#JSON Serialization|json output method]] of the official recommendation.
 +
Values can also be serialized as JSON with the standard [[Serialization]] feature of XQuery:
 +
* The parameter {{Code|method}} needs to be set to {{Code|json}}, and
 +
* the options presented in this article need to be assigned to the {{Code|json}} parameter.
 +
|- valign="top"
 +
| '''Errors'''
 +
|{{Error|serialize|#Errors}} the specified node cannot be serialized as JSON document.
 
|}
 
|}
  
==Examples==
+
=Examples=
 +
 
 +
==BaseX Format==
  
 
'''Example 1: Adds all JSON documents in a directory to a database'''
 
'''Example 1: Adds all JSON documents in a directory to a database'''
  
 
'''Query:'''
 
'''Query:'''
<pre class="brush:xquery">
+
<pre lang='xquery'>
 
let $database := "database"
 
let $database := "database"
 
for $name in file:list('.', false(), '*.json')
 
for $name in file:list('.', false(), '*.json')
 
let $file := file:read-text($name)
 
let $file := file:read-text($name)
 
let $json := json:parse($file)
 
let $json := json:parse($file)
return db:add($database, document { $json }, $name)  
+
return db:add($database, $json, $name)  
 
</pre>
 
</pre>
  
'''Example 2: Converts a simple JSON string to XML'''
+
'''Example 2: Converts a simple JSON string to XML and back'''
  
 
'''Query:'''
 
'''Query:'''
<pre class="brush:xquery">
+
<pre lang='xquery'>
 
json:parse('{}')
 
json:parse('{}')
 
</pre>
 
</pre>
  
 
'''Result:'''
 
'''Result:'''
<pre class="brush:xml">
+
<pre lang="xml">
<json objects="json"/>
+
<json type="object"/>
 +
</pre>
 +
 
 +
'''Query:'''
 +
<pre lang='xquery'>
 +
(: serialize result as plain text :)
 +
declare option output:method 'text';
 +
json:serialize(<json type="object"/>)
 +
</pre>
 +
 
 +
'''Result:'''
 +
<pre lang='xquery'>
 +
{ }
 
</pre>
 
</pre>
  
Line 72: Line 210:
  
 
'''Query:'''
 
'''Query:'''
<pre class="brush:xquery">
+
<pre lang='xquery'>
 
json:parse('{
 
json:parse('{
 
   "title": "Talk On Travel Pool",
 
   "title": "Talk On Travel Pool",
 
   "link": "http://www.flickr.com/groups/talkontravel/pool/",
 
   "link": "http://www.flickr.com/groups/talkontravel/pool/",
 
   "description": "Travel and vacation photos from around the world.",
 
   "description": "Travel and vacation photos from around the world.",
   "modified": "2009-02-02T11:10:27Z",
+
   "modified": "2014-02-02T11:10:27Z",
 
   "generator": "http://www.flickr.com/"
 
   "generator": "http://www.flickr.com/"
 
}')
 
}')
Line 83: Line 221:
  
 
'''Result:'''
 
'''Result:'''
<pre class="brush:xml">
+
<pre lang="xml">
<json objects="json">
+
<json type="object">
 
   <title>Talk On Travel Pool</title>
 
   <title>Talk On Travel Pool</title>
 
   <link>http://www.flickr.com/groups/talkontravel/pool/</link>
 
   <link>http://www.flickr.com/groups/talkontravel/pool/</link>
 
   <description>Travel and vacation photos from around the world.</description>
 
   <description>Travel and vacation photos from around the world.</description>
   <modified>2009-02-02T11:10:27Z</modified>
+
   <modified>2014-02-02T11:10:27Z</modified>
 
   <generator>http://www.flickr.com/</generator>
 
   <generator>http://www.flickr.com/</generator>
 
</json>
 
</json>
Line 96: Line 234:
  
 
'''Query:'''
 
'''Query:'''
<pre class="brush:xquery">
+
<pre lang='xquery'>
json:parse('{
+
let $options := map { 'merge': true() }
 +
return json:parse('{
 
   "first_name": "John",
 
   "first_name": "John",
 
   "last_name": "Smith",
 
   "last_name": "Smith",
Line 113: Line 252:
 
     {
 
     {
 
       "type": "mobile",
 
       "type": "mobile",
       "number": 001327724623
+
       "number": 1327724623
 
     }
 
     }
 
   ]
 
   ]
}')
+
}', $options)
 
</pre>
 
</pre>
  
 
'''Result:'''
 
'''Result:'''
<pre class="brush:xml">
+
<pre lang="xml">
 
<json numbers="age code" arrays="phone" objects="json address value">
 
<json numbers="age code" arrays="phone" objects="json address value">
 
   <first__name>John</first__name>
 
   <first__name>John</first__name>
Line 131: Line 270:
 
   </address>
 
   </address>
 
   <phone>
 
   <phone>
     <value>
+
     <_>
 
       <type>home</type>
 
       <type>home</type>
 
       <number>212 555-1234</number>
 
       <number>212 555-1234</number>
     </value>
+
     </_>
     <value>
+
     <_>
 
       <type>mobile</type>
 
       <type>mobile</type>
       <number type="number">001327724623</number>
+
       <number type="number">1327724623</number>
     </value>
+
     </_>
 
   </phone>
 
   </phone>
</json></pre>
+
</json>
 +
</pre>
  
=JsonML=
+
==JsonML Format==
  
[[JSON Functions#json:serialize-ml|json:serialize-ml]] and [[JSON Functions#json:parse-ml|json:parse-ml]] are used to transform XML to JSON and back, using the [http://jsonml.org JsonML] dialect. JsonML can be used to transform arbitrary XML documents, but namespaces, comments and processing instructions will be discarded in the transformation process. More details are found in the official [http://jsonml.org/XML JsonML documentation].
+
'''Example 1: Converts all XML documents in a database to the JsonML format and writes them to disk'''
 
 
==json:serialize-ml==
 
{|
 
|-
 
| valign='top' width='90' | '''Signatures'''
 
|<code><b>json:serialize-ml</b>($input as node()) as xs:string()</code>
 
|-
 
| valign='top' | '''Summary'''
 
|Serializes the node specified by <code>$input</code> and returns the result as <code>xs:string</code> instance.<br />XML documents can also be output in the JsonML format by setting the [[Serialization|Serialization Option]] <code>"method"</code> to <code>"jsonml"</code>.
 
|-
 
| valign='top' | '''Errors'''
 
|<b>[[XQuery Errors#BaseX Errors|BASX0016]]</b> is raised if the specified value cannot be serialized.
 
|}
 
 
 
==json:parse-ml==
 
{|
 
|-
 
| valign='top' width='90' | '''Signatures'''
 
|<code><b>json:parse-ml</b>($input as xs:string()) as element()</code>
 
|-
 
| valign='top' | '''Summary'''
 
|Converts the [http://jsonml.org JsonML] document specified by <code>$input</code> to XML, and returns the result as <code>element()</code> instance. The JSON input must conform to the JsonML specification to be successfully converted.
 
|-
 
| valign='top' | '''Errors'''
 
|<b>[[XQuery Errors#BaseX Errors|BASX0015]]</b> is raised if the specified input cannot be parsed as JsonML instance.
 
|}
 
 
 
==Examples==
 
 
 
'''Example 1: Converts all XML documents in a database to JsonML and writes them to disk'''
 
  
 
'''Query:'''
 
'''Query:'''
<pre class="brush:xquery">
+
<pre lang='xquery'>
 
for $doc in collection('json')
 
for $doc in collection('json')
 
let $name := document-uri($doc)
 
let $name := document-uri($doc)
let $json := json:serialize($doc)
+
let $json := json:serialize($doc, map { 'format': 'jsonml' })
 
return file:write($name, $json)
 
return file:write($name, $json)
 
</pre>
 
</pre>
  
'''Example 2: Converts a simple XML fragment to the JsonML format'''
+
'''Example 2: Converts an XML document with elements and text'''
 
 
'''Query:'''
 
<pre class="brush:xquery">
 
json:serialize-ml(<xml/>)
 
</pre>
 
 
 
'''Result:'''
 
<pre>
 
["xml"]
 
</pre>
 
 
 
'''Example 3: Converts an XML document with elements and text'''
 
  
 
'''Query:'''
 
'''Query:'''
<pre class="brush:xquery">
+
<pre lang='xquery'>
json:serialize-ml(doc('flickr.xml'))
+
json:serialize(doc('flickr.xml'), map { 'format': 'jsonml' })
 
</pre>
 
</pre>
  
 
'''flickr.xml:'''
 
'''flickr.xml:'''
<pre class="brush:xml">
+
<pre lang="xml">
 
<flickr>
 
<flickr>
 
   <title>Talk On Travel Pool</title>
 
   <title>Talk On Travel Pool</title>
 
   <link>http://www.flickr.com/groups/talkontravel/pool/</link>
 
   <link>http://www.flickr.com/groups/talkontravel/pool/</link>
 
   <description>Travel and vacation photos from around the world.</description>
 
   <description>Travel and vacation photos from around the world.</description>
   <modified>2009-02-02T11:10:27Z</modified>
+
   <modified>2014-02-02T11:10:27Z</modified>
 
   <generator>http://www.flickr.com/</generator>
 
   <generator>http://www.flickr.com/</generator>
 
</flickr>
 
</flickr>
Line 215: Line 313:
  
 
'''Result:'''
 
'''Result:'''
<pre>
+
<pre lang="json">
 
["flickr",
 
["flickr",
 
   ["title",
 
   ["title",
 
     "Talk On Travel Pool"],
 
     "Talk On Travel Pool"],
 
   ["link",
 
   ["link",
     "http:\/\/www.flickr.com\/groups\/talkontravel\/pool\/"],
+
     "http://www.flickr.com/groups/talkontravel/pool/"],
 
   ["description",
 
   ["description",
 
     "Travel and vacation photos from around the world."],
 
     "Travel and vacation photos from around the world."],
 
   ["modified",
 
   ["modified",
     "2009-02-02T11:10:27Z"],
+
     "2014-02-02T11:10:27Z"],
 
   ["generator",
 
   ["generator",
     "http:\/\/www.flickr.com\/"]]
+
     "http://www.flickr.com/"]]
 
</pre>
 
</pre>
  
'''Example 4: Converts a document with nested elements and attributes'''
+
'''Example 3: Converts a document with nested elements and attributes to JsonML'''
  
 
'''Query:'''
 
'''Query:'''
<pre class="brush:xquery">
+
<pre lang='xquery'>
json:serialize-ml(doc('input.xml'))
+
json:serialize(doc('input.xml'), map { 'format': 'jsonml' })
 
</pre>
 
</pre>
  
 
'''input.xml:'''
 
'''input.xml:'''
<pre class="brush:xml">
+
<pre lang="xml">
 
<address id='1'>
 
<address id='1'>
 
   <!-- comments will be discarded -->
 
   <!-- comments will be discarded -->
Line 252: Line 350:
  
 
'''Result:'''
 
'''Result:'''
<pre>
+
<pre lang="json">
 
["address", {"id":"1"},
 
["address", {"id":"1"},
 
   ["last_name",
 
   ["last_name",
Line 268: Line 366:
 
     "212 555-1234"]]
 
     "212 555-1234"]]
 
</pre>
 
</pre>
 +
 +
==XQuery Format==
 +
 +
'''Example 1: Converts a JSON string to XQuery'''
 +
 +
'''Query:'''
 +
<pre lang='xquery'>
 +
let $input := '{
 +
  "Title": "Drinks",
 +
  "Author": [ "Jim Daniels", "Jack Beam" ]
 +
}'
 +
let $data := json:parse($input, map { 'format': 'xquery' })
 +
return map:for-each($data, function($k, $v) {
 +
  $k || ': ' || string-join($v, ', ')
 +
})
 +
</pre>
 +
 +
'''Result:'''
 +
<pre lang="json">
 +
Author: Jim Daniels, Jack Beam
 +
Title: Drinks
 +
</pre>
 +
 +
'''Example 2: Converts XQuery data to JSON'''
 +
 +
'''Query:'''
 +
<pre lang='xquery'>
 +
for $item in (
 +
  true(),
 +
  'ABC',
 +
  array { 1 to 5 },
 +
  map { "Key": "Value" }
 +
)
 +
return json:serialize(
 +
  $item,
 +
  map { 'format': 'xquery', 'indent': 'no' }
 +
)
 +
</pre>
 +
 +
'''Result:'''
 +
<pre lang="json">
 +
true
 +
"ABC"
 +
[1,2,3,4,5]
 +
{"Key":"Value"}
 +
</pre>
 +
 +
=Errors=
 +
 +
{| class="wikitable" width="100%"
 +
! width="110"|Code
 +
|Description
 +
|- valign="top"
 +
|{{Code|options}}
 +
|The specified options are conflicting.
 +
|- valign="top"
 +
|{{Code|parse}}
 +
|The specified input cannot be parsed as JSON document.
 +
|- valign="top"
 +
|{{Code|serialize}}
 +
|The specified node cannot be serialized as JSON document.
 +
|}
 +
 +
=Changelog=
 +
 +
;Version 10.0
 +
* Updated: {{Code|indent}}: Default changed from {{Code|yes}} to {{Code|no}}.
 +
 +
;Version 9.4
 +
* Added: {{Function||json:doc}}
 +
 +
; Version 9.1
 +
* Updated: {{Function||json:parse}} can be called with empty sequence.
 +
 +
;Version 9.0
 +
* Updated: <code>map</code> format renamed to <code>xquery</code>.
 +
* Updated: error codes updated; errors now use the module namespace
 +
 +
;Version 8.4
 +
* Updated: <code>unescape</code> changed to <code>escape</code>.
 +
 +
;Version 8.2
 +
* Added: Conversion format <code>[[#Basic|basic]]</code>.
 +
 +
;Version 8.0
 +
* Updated: Serialization aligned with the {{Code|json}} output method of the official specification.
 +
* Added: {{Code|liberal}} option.
 +
* Removed: {{Code|spec}} option.
 +
 +
;Version 7.8
 +
* Removed: {{Code|json:parse-ml}}, {{Code|json:serialize-ml}}.
 +
* Updated: {{Code|json:parse}} now returns a document node instead of an element, or an XQuery map if {{Code|format}} is set to {{Code|.map}}.
 +
 +
;Version 7.7.2
 +
* Updated: {{Code|$options}} argument added to {{Function||json:parse}} and {{Function||json:serialize}}.
 +
* Updated: {{Function||json:parse-ml}} and {{Function||json:serialize-ml}} are now ''deprecated''.
 +
 +
The module was introduced with Version 7.0.

Latest revision as of 17:38, 1 December 2023

This XQuery Module contains functions to parse and serialize JSON data JSON (JavaScript Object Notation) is a popular data exchange format for applications written in JavaScript. As there are notable differences between JSON and XML, or XQuery data types, no mapping exists that guarantees a lossless, bidirectional conversion between JSON and XML. For this reason, we offer various mappings, all of which are suited to different use cases.

Conventions[edit]

All functions and errors in this module are assigned to the http://basex.org/modules/json namespace, which is statically bound to the json prefix.

Conversion Formats[edit]

A little advice: in the Database Creation dialog of the GUI, if you select JSON Parsing and switch to the Parsing tab, you can see the effects of some of the conversion options.

Direct[edit]

The direct conversion format allows a lossless conversion from JSON to XML and back. The transformation is based on the following rules:

  • The resulting document has a json root node.
  • Object pairs are represented via elements. The name of a pair is encoded, as described in the Conversion Module, and used as element name.
  • Array entries are also represented via elements, with _ as element name.
  • Object and array values are stored in text nodes.
  • The types of values are represented via type attributes:
    • The existing types are string, number, boolean, null, object, and array.
    • As most values are strings, the string type is by default omitted.

Attributes[edit]

The attributes format is lossless, too. The transformation based on the following rules:

  • The resulting document has a json root node.
  • Object pairs are represented via pair elements. The name of a pair is stored in a name attribute.
  • Array entries are represented via item elements.
  • Object and array values are stored in text nodes.
  • The types of values are represented via type attributes:
    • The existing types are string, number, boolean, null, object, and array.
    • As most values are strings, the string type is by default omitted.

Basic[edit]

The basic format is another lossless format. It converts a JSON document to an XML node and vice versa. The conversion rules are the same as for fn:json-to-xml.

JsonML[edit]

The jsonml format is designed to convert XML to JSON and back, using the JsonML dialect. JsonML allows the transformation of arbitrary XML documents, but namespaces, comments and processing instructions will be discarded in the transformation process. More details are found in the official JsonML documentation.

XQuery[edit]

The xquery format is lossless, too. It converts JSON data to an XQuery value (a map, array, string, number, boolean, or empty sequence) and vice versa. The conversion rules are the same as for fn:parse-json.

The resulting representation consumes less memory than XML-based formats, and values can be directly accessed without conversion. Thus, it is recommendable for very large inputs and for efficient ad-hoc processing.

Options[edit]

The following options are available (the Direction column indicates if an option applies to parsing, serialization, or both operations):

Option Description Allowed Default Direction
format Specifies the format for converting JSON data (see above). direct, attributes, basic, jsonml, xquery direct parse, serialize
liberal Determines if minor deviations from RFC 7159 will be ignored. yes, no no parse
merge This option is considered when direct or attributes conversion is used:
  • If a name has the same type throughout the data, the type attribute will be omitted. Instead, the name will be listed in additional, type-specific attributes in the root node.
  • The attributes are named by their type in plural (numbers, booleans, nulls, objects and arrays), and the attribute value contains all names with that type, separated by whitespaces.
yes, no no parse, serialize
strings Indicates if type attributes will be added for strings. yes, no yes parse, serialize
lax Specifies if lax conversion rules are used to convert QNames to JSON names. yes, no no parse, serialize
escape Indicates if escaped characters are expanded (for example, \n becomes a single x0A character, while \u20AC becomes the character ). yes, no yes parse
escape Indicates if characters are escaped whenever the JSON syntax requires it. This option can be set to no if strings are already in escaped form and no further escaping is permitted. yes, no yes serialize
indent Indicates if whitespace should be added to the output with the aim of improving human legibility. If the parameter is set as in the query prolog, it overrides the indent serialization parameter. yes, no no serialize

Functions[edit]

json:doc[edit]

Signature
json:doc(
  $href     as xs:string,
  $options  as map(*)?    := map { }
) as item()?
Summary Fetches the JSON document referred to by the given $href and converts it to an XQuery value. The $options argument can be used to control the way the input is converted.
Errors parse: the specified input cannot be parsed as JSON document.
options: the specified options are conflicting.

json:parse[edit]

Signature
json:parse(
  $value   as xs:string?,
  $options  as map(*)?     := map { }
) as item()?
Summary Converts the JSON $value to an XQuery value. If the input can be successfully parsed, it can be serialized back to the original JSON representation. The $options argument can be used to control the way the input is converted.
Errors parse: the specified input cannot be parsed as JSON document.
options: the specified options are conflicting.

json:serialize[edit]

Signature
json:serialize(
  $input    as item()?,
  $options  as map(*)?  := map { }
) as xs:string
Summary Serializes the specified $input as JSON, using the specified $options, and returns the result as string:
  • The input is expected to conform to the results that are created by json:parse.
  • Non-conforming items will be serialized as specified in the json output method of the official recommendation.

Values can also be serialized as JSON with the standard Serialization feature of XQuery:

  • The parameter method needs to be set to json, and
  • the options presented in this article need to be assigned to the json parameter.
Errors serialize: the specified node cannot be serialized as JSON document.

Examples[edit]

BaseX Format[edit]

Example 1: Adds all JSON documents in a directory to a database

Query:

let $database := "database"
for $name in file:list('.', false(), '*.json')
let $file := file:read-text($name)
let $json := json:parse($file)
return db:add($database, $json, $name) 

Example 2: Converts a simple JSON string to XML and back

Query:

json:parse('{}')

Result:

<json type="object"/>

Query:

(: serialize result as plain text :)
declare option output:method 'text';
json:serialize(<json type="object"/>)

Result:

{ }

Example 3: Converts a JSON string with simple objects and arrays

Query:

json:parse('{
  "title": "Talk On Travel Pool",
  "link": "http://www.flickr.com/groups/talkontravel/pool/",
  "description": "Travel and vacation photos from around the world.",
  "modified": "2014-02-02T11:10:27Z",
  "generator": "http://www.flickr.com/"
}')

Result:

<json type="object">
  <title>Talk On Travel Pool</title>
  <link>http://www.flickr.com/groups/talkontravel/pool/</link>
  <description>Travel and vacation photos from around the world.</description>
  <modified>2014-02-02T11:10:27Z</modified>
  <generator>http://www.flickr.com/</generator>
</json>

Example 4: Converts a JSON string with different data types

Query:

let $options := map { 'merge': true() }
return json:parse('{
  "first_name": "John",
  "last_name": "Smith",
  "age": 25,
  "address": {
    "street": "21 2nd Street",
    "city": "New York",
    "code": 10021
  },
  "phone": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "mobile",
      "number": 1327724623
    }
  ]
}', $options)

Result:

<json numbers="age code" arrays="phone" objects="json address value">
  <first__name>John</first__name>
  <last__name>Smith</last__name>
  <age>25</age>
  <address>
    <street>21 2nd Street</street>
    <city>New York</city>
    <code>10021</code>
  </address>
  <phone>
    <_>
      <type>home</type>
      <number>212 555-1234</number>
    </_>
    <_>
      <type>mobile</type>
      <number type="number">1327724623</number>
    </_>
  </phone>
</json>

JsonML Format[edit]

Example 1: Converts all XML documents in a database to the JsonML format and writes them to disk

Query:

for $doc in collection('json')
let $name := document-uri($doc)
let $json := json:serialize($doc, map { 'format': 'jsonml' })
return file:write($name, $json)

Example 2: Converts an XML document with elements and text

Query:

json:serialize(doc('flickr.xml'), map { 'format': 'jsonml' })

flickr.xml:

<flickr>
  <title>Talk On Travel Pool</title>
  <link>http://www.flickr.com/groups/talkontravel/pool/</link>
  <description>Travel and vacation photos from around the world.</description>
  <modified>2014-02-02T11:10:27Z</modified>
  <generator>http://www.flickr.com/</generator>
</flickr>

Result:

["flickr",
  ["title",
    "Talk On Travel Pool"],
  ["link",
    "http://www.flickr.com/groups/talkontravel/pool/"],
  ["description",
    "Travel and vacation photos from around the world."],
  ["modified",
    "2014-02-02T11:10:27Z"],
  ["generator",
    "http://www.flickr.com/"]]

Example 3: Converts a document with nested elements and attributes to JsonML

Query:

json:serialize(doc('input.xml'), map { 'format': 'jsonml' })

input.xml:

<address id='1'>
  <!-- comments will be discarded -->
  <last_name>Smith</last_name>
  <age>25</age>
  <address xmlns='will be dropped as well'>
    <street>21 2nd Street</street>
    <city>New York</city>
    <code>10021</code>
  </address>
  <phone type='home'>212 555-1234</phone>
</address>

Result:

["address", {"id":"1"},
  ["last_name",
    "Smith"],
  ["age",
    "25"],
  ["address",
    ["street",
      "21 2nd Street"],
    ["city",
      "New York"],
    ["code",
      "10021"]],
  ["phone", {"type":"home"},
    "212 555-1234"]]

XQuery Format[edit]

Example 1: Converts a JSON string to XQuery

Query:

let $input := '{
  "Title": "Drinks",
  "Author": [ "Jim Daniels", "Jack Beam" ]
}'
let $data := json:parse($input, map { 'format': 'xquery' })
return map:for-each($data, function($k, $v) {
  $k || ': ' || string-join($v, ', ')
})

Result:

Author: Jim Daniels, Jack Beam
Title: Drinks

Example 2: Converts XQuery data to JSON

Query:

for $item in (
  true(),
  'ABC',
  array { 1 to 5 },
  map { "Key": "Value" }
)
return json:serialize(
  $item,
  map { 'format': 'xquery', 'indent': 'no' }
)

Result:

true
"ABC"
[1,2,3,4,5]
{"Key":"Value"}

Errors[edit]

Code Description
options The specified options are conflicting.
parse The specified input cannot be parsed as JSON document.
serialize The specified node cannot be serialized as JSON document.

Changelog[edit]

Version 10.0
  • Updated: indent: Default changed from yes to no.
Version 9.4
Version 9.1
  • Updated: json:parse can be called with empty sequence.
Version 9.0
  • Updated: map format renamed to xquery.
  • Updated: error codes updated; errors now use the module namespace
Version 8.4
  • Updated: unescape changed to escape.
Version 8.2
  • Added: Conversion format basic.
Version 8.0
  • Updated: Serialization aligned with the json output method of the official specification.
  • Added: liberal option.
  • Removed: spec option.
Version 7.8
  • Removed: json:parse-ml, json:serialize-ml.
  • Updated: json:parse now returns a document node instead of an element, or an XQuery map if format is set to .map.
Version 7.7.2

The module was introduced with Version 7.0.