JSON Functions

This module contains functions to parse and serialize JSON data. JSON (JavaScript Object Notation) is a popular data exchange format for applications written in JavaScript. As there are notable differences between JSON and XML, or XDM types, no single mapping is ideal for all use cases. For this reason, we offer various mappings, each of which is suited to different purposes. See Parsers if you want to import JSON documents into a database.

Conventions

All functions and errors are in the http://basex.org/modules/json namespace, to which the json prefix is statically bound.

Conversion Formats

Direct

The direct conversion format allows a lossless conversion from JSON to XML and back. The transformation is based on the following rules:

• The resulting document has a json root node.
• Object pairs are represented via elements. The name of a pair is encoded, as described for the Conversion Functions, and used as element name.
• Array entries are also represented via elements, with _ as element name.
• Object and array values are stored in text nodes.
• The types of values are represented via type attributes:
  • The existing types are string, number, boolean, null, object, and array.
  • As most values are strings, the string type is by default omitted.

Attributes

The attributes format is lossless, too. The transformation is based on the following rules:

• The resulting document has a json root node.
• Object pairs are represented via pair elements. The name of a pair is stored in a name attribute.
• Array entries are represented via item elements.
• Object and array values are stored in text nodes.
• The types of values are represented via type attributes:
  • The existing types are string, number, boolean, null, object, and array.
  • As most values are strings, the string type is by default omitted.

JsonML

The jsonml format is designed to convert XML to JSON and back, using the JsonML dialect. JsonML allows the transformation of arbitrary XML documents, but namespaces, comments and processing instructions will be discarded in the transformation process. More details are found in the official JsonML documentation.

W3

The w3 format is lossless, too. It converts JSON data to an XDM value (a map, array, string, number, boolean, or empty sequence) and vice versa. The conversion rules are the same as for fn:parse-json.

The resulting representation consumes less memory than XML-based formats, and values can be directly accessed without conversion. Thus, it is recommended for very large inputs and for efficient ad-hoc processing.

W3-XML

The w3-xml format is another lossless format. It converts a JSON document to an XML node and vice versa. The conversion rules are the same as for fn:json-to-xml.

W3-Mapping

Added: New conversion format.

The w3-mapping format converts JSON data to XML without type attributes, and vice versa. The conversion rules are the same as for fn:map-to-element and fn:element-to-map, and their options (including a conversion plan that controls the element layouts) can be supplied with the mapping option:

• When parsing, the JSON input is converted to an XDM value (as with the w3 format), which is then converted to an element. The result is a document node.
• When serializing, document and element nodes are converted to maps, which are then serialized. Other items are serialized unchanged.
• The input must be an object with a single entry, the key of which becomes the name of the root element. Other inputs can be converted by assigning a root element name with the root key of the mapping option.
• Keys are not encoded: a key that is no valid element name is rejected.
• The JSON value null is represented by an element with an xsi:nil="true" attribute.
• Numbers are parsed with the adaptive number format by default.

The format is not lossless: When XML is serialized, the types of values are inferred from their string values, and an element that occurs only once is serialized as a single value. A conversion plan can be used to ensure that JSON data is serialized back to its original representation.

Options

Added: Serialization option escape-solidus, parsing option json-lines.

Updated: number-parser was replaced with number-format.

Removed: Conversion formats xquery and basic (use w3 and w3-xml).

Added: Option mapping for the w3-mapping format.

The following options are available (the Direction column indicates if an option applies to parsing, serialization, or both operations):

Functions

json:doc

json:doc(  $source   as xs:string,  $options  as map(*)?  := {}) as item()*create

json:parse

json:parse(  $value    as xs:string?,  $options  as map(*)?  := {}) as item()*

json:serialize

json:serialize(  $input    as item()?,  $options  as map(*)?  := {}) as xs:string

Examples

Direct Format

Example 1: Adds all JSON documents in a directory to a database.

let $database := "database"
for $name in file:list('.', false(), '*.json')
let $file := file:read-text($name)
let $json := json:parse($file)
return db:add($database, $json, $name)

Example 2: Converts a simple JSON string to XML and back.

json:parse('{}')

<json type="object"/>

(: serialize result as plain text :)
declare option output:method 'text';
json:serialize(<json type="object"/>)

{ }

Example 3: Converts a JSON string with simple objects and arrays

json:parse('{
  "title": "Talk On Travel Pool",
  "link": "http://www.flickr.com/groups/talkontravel/pool/",
  "description": "Travel and vacation photos from around the world."
}')

<json type="object">
  <title>Talk On Travel Pool</title>
  <link>http://www.flickr.com/groups/talkontravel/pool/</link>
  <description>Travel and vacation photos from around the world.</description>
</json>

Example 4: Converts a JSON string with different data types

let $options := { 'merge': true() }
return json:parse('{
  "first_name": "John",
  "last_name": "Smith",
  "age": 25,
  "address": {
    "street": "21 2nd Street",
    "city": "New York",
    "code": 10021
  },
  "phone": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "mobile",
      "number": 1327724623
    }
  ]
}', $options)

<json objects="json address _" arrays="phone" numbers="age code">
  <first__name>John</first__name>
  <last__name>Smith</last__name>
  <age>25</age>
  <address>
    <street>21 2nd Street</street>
    <city>New York</city>
    <code>10021</code>
  </address>
  <phone>
    <_>
      <type>home</type>
      <number>212 555-1234</number>
    </_>
    <_>
      <type>mobile</type>
      <number type="number">1327724623</number>
    </_>
  </phone>
</json>

JsonML Format

Example 1: Converts all XML documents in a database to the JsonML format and writes them to disk.

for $doc in collection('json')
let $name := document-uri($doc)
let $json := json:serialize($doc, { 'format': 'jsonml' })
return file:write($name, $json)

Example 2: Converts an XML document with elements and text.

json:serialize(doc('flickr.xml'), { 'format': 'jsonml' })

<flickr>
  <title>Talk On Travel Pool</title>
  <link>http://www.flickr.com/groups/talkontravel/pool/</link>
  <description>Travel and vacation photos from around the world.</description>
</flickr>

[ "flickr",
  [ "title", "Talk On Travel Pool" ],
  [ "link", "http://www.flickr.com/groups/talkontravel/pool/" ],
  ["description", "Travel and vacation photos from around the world." ]
]

Example 3: Converts a document with nested elements and attributes to JsonML:

json:serialize(doc('input.xml'), { 'format': 'jsonml' })

<address id='1'>
  <!-- comments will be discarded -->
  <last_name>Smith</last_name>
  <age>25</age>
  <address xmlns='will be dropped as well'>
    <street>21 2nd Street</street>
    <city>New York</city>
    <code>10021</code>
  </address>
  <phone type='home'>212 555-1234</phone>
</address>

[ "address", { "id": "1" },
  [ "last_name", "Smith" ],
  [ "age", "25"],
  [ "address",
    [ "street", "21 2nd Street" ],
    [ "city", "New York" ],
    [ "code", "10021" ]
  ],
  [ "phone", { "type": "home" }, "212 555-1234" ]
]

W3 Format

Example 1: Converts a JSON string to XQuery.

let $input := '{
  "Title": "Drinks",
  "Author": [ "Jim Daniels", "Jack Beam" ]
}'
let $data := json:parse($input, { 'format': 'w3' })
return map:for-each($data, fn($key, $value) {
  $key || ': ' || string-join($value, ', ')
})

Title: Drinks
Author: Jim Daniels, Jack Beam

Example 2: Converts XDM data to JSON.

for $item in (
  true(),
  'ABC',
  array { 1 to 5 },
  { "Key": "Value" }
)
return json:serialize(
  $item,
  { 'format': 'w3', 'indent': false() }
)

true
"ABC"
[1,2,3,4,5]
{"Key":"Value"}

W3-Mapping Format

Example 1: Converts a JSON string to XML and back, using a conversion plan.

let $json := '{
  "library": {
    "name": "City Library",
    "books": [
      { "title": "Dune", "year": 1965 },
      { "title": "Solaris", "year": 1961 }
    ]
  }
}'
let $options := {
  'format': 'w3-mapping',
  'mapping': {
    'plan': {
      'books': { 'layout': 'list', 'child': 'book' }
    }
  }
}
let $xml := json:parse($json, $options)
return (
  $xml,
  json:serialize($xml, $options)
)

<library>
  <name>City Library</name>
  <books>
    <book>
      <title>Dune</title>
      <year>1965</year>
    </book>
    <book>
      <title>Solaris</title>
      <year>1961</year>
    </book>
  </books>
</library>
{"library":{"name":"City Library","books":[{"title":"Dune","year":1965},{"title":"Solaris","year":1961}]}}

Example 2: Converts a JSON array, using a root element name.

json:parse('[ "a", "b" ]', {
  'format': 'w3-mapping',
  'mapping': {
    'root': 'list',
    'plan': { 'list': { 'layout': 'list', 'child': 'entry' } }
  }
})

<list>
  <entry>a</entry>
  <entry>b</entry>
</list>

Errors

Changelog

• Added: Serialization option escape-solidus, parsing option json-lines.
• Added: Conversion format w3-mapping, with the option mapping.
• Updated: number-parser was replaced with number-format.
• Removed: Conversion formats xquery and basic (use w3 and w3-xml).

• Updated: W3 Conversion Format: Renamed (old name: xquery), w3-xml: Renamed (old name: basic).

• Added: New number-parser, fallback, duplicates and null options (derived from fn:parse-json).

• Updated: indent: Default changed from yes to no.

• Added: json:doc

• Updated: json:parse can be called with empty sequence.

• Updated: map format renamed to xquery.
• Updated: error codes updated; errors now use the module namespace

• Updated: unescape changed to escape.

• Added: Conversion format basic.

• Added: liberal option.
• Updated: Serialization aligned with the json output method of the official specification.
• Removed: spec option.

• Updated: json:parse now returns a document node instead of an element, or an XQuery map if format is set to map.
• Removed: json:parse-ml, json:serialize-ml.

• Updated: $options argument added to json:parse and json:serialize.
• Updated: json:parse-ml, json:serialize-ml are now deprecated.

• Added: New module added.