Main Page » Mailing List » January 2015

The JSON serialization controversy

Monday, 5 January 2015 · 9 messages · last reply 12 Jan 2015
FG
Florent Gallaire5 Jan 2015 05:46 #
Whether my database this XML file:
<?xml version="1.0" encoding="UTF-8"?>
<mots>
  <mot>
    <nom>iatrogène</nom>
    <racine>gène</racine>
  </mot>
  <mot>
    <nom>hippiatre</nom>
    <racine>hippo</racine>
  </mot>
</mots>
The only easy way to have JSON serialization with BaseX is to use the "format=jsonml" option which returns: ["mots", ["mot", ["nom", "iatrogène"], ["racine", "gène"]], ["mot", ["nom", "hippiatre"], ["racine", "hippo"]]] This output is not at all what I need, and think I'm not the only one in this case. A quick search on Google give us simple online tools which have an useful output:
http://codebeautify.org/xmltojson
{
    "mots": {
        "mot": [
            {
                "nom": "iatrogène",
                "racine": "gène"
            },
            {
                "nom": "hippiatre",
                "racine": "hippo"
            }
        ]
    }
}

http://www.utilities-online.info/xmltojson
{
  "mots": {
    "mot": [
      {
        "nom": "iatrogène",
        "racine": "gène"
      },
      {
        "nom": "hippiatre",
        "racine": "hippo"
      }
    ]
  }
}
Could we have this "easyjson" option in BaseX ? I know that it has not at all the power of lossless conversion from JSON to XML and back, like the "direct" and "attributes" options, but who really cares ? I really think it's what most users need today to use BaseX with client side JSON oriented frameworks. Best regards Florent
-- FLOSS Engineer & Lawyer
DK
Dirk Kirsten5 Jan 2015 08:38 #
Hello Florent, JsonML isn't the only way to generate JSON with BaseX; it isn't even the default one. Please take a look at our documentation at http://docs.basex.org/wiki/JSON to see what is available. We agree that JsonML isn't the most convenient format, that's why the BaseX-specific direct format is the default one. Cheers, Dirk
Quoted text · Florent Gallaire · 77 lines
On 01/05/2015 05:46 AM, Florent Gallaire wrote: > Whether my database this XML file: > > <?xml version="1.0" encoding="UTF-8"?> > <mots> > <mot> > <nom>iatrogène</nom> > <racine>gène</racine> > </mot> > <mot> > <nom>hippiatre</nom> > <racine>hippo</racine> > </mot> > </mots> > > The only easy way to have JSON serialization with BaseX is to use the > "format=jsonml" option which returns: > ["mots", > ["mot", > ["nom", > "iatrogène"], > ["racine", > "gène"]], > ["mot", > ["nom", > "hippiatre"], > ["racine", > "hippo"]]] > > This output is not at all what I need, and think I'm not the only one > in this case. > A quick search on Google give us simple online tools which have an > useful output: > > http://codebeautify.org/xmltojson > { > "mots": { > "mot": [ > { > "nom": "iatrogène", > "racine": "gène" > }, > { > "nom": "hippiatre", > "racine": "hippo" > } > ] > } > } > > http://www.utilities-online.info/xmltojson > { > "mots": { > "mot": [ > { > "nom": "iatrogène", > "racine": "gène" > }, > { > "nom": "hippiatre", > "racine": "hippo" > } > ] > } > } > > Could we have this "easyjson" option in BaseX ? I know that it has not > at all the power of lossless conversion from JSON to XML and back, > like the "direct" and "attributes" options, but who really cares ? > > I really think it's what most users need today to use BaseX with > client side JSON oriented frameworks. > > Best regards > > Florent >
-- Dirk Kirsten, BaseX GmbH, http://basex.org |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
FG
Florent Gallaire5 Jan 2015 09:31 #
Hello Dirk, Thank for your answer, but "direct" and "attribute" format are not easy to use, I could even say that they are too hard to use for normal people. And they need a lot of extra work: <json> root node, _ elements, etc. I just would like an easy way, with no extra work to process arbitrary XML. An easy use like the jsonml option, but with an useful output. Cheers Florent
Quoted text · Dirk Kirsten · 93 lines
On Mon, Jan 5, 2015 at 8:38 AM, Dirk Kirsten <dk at basex.org> wrote: > Hello Florent, > > JsonML isn't the only way to generate JSON with BaseX; it isn't even the default one. Please take a look at our documentation at http://docs.basex.org/wiki/JSON to see what is available. We agree that JsonML isn't the most convenient format, that's why the BaseX-specific direct format is the default one. > > Cheers, > Dirk > > On 01/05/2015 05:46 AM, Florent Gallaire wrote: >> Whether my database this XML file: >> >> <?xml version="1.0" encoding="UTF-8"?> >> <mots> >> <mot> >> <nom>iatrogène</nom> >> <racine>gène</racine> >> </mot> >> <mot> >> <nom>hippiatre</nom> >> <racine>hippo</racine> >> </mot> >> </mots> >> >> The only easy way to have JSON serialization with BaseX is to use the >> "format=jsonml" option which returns: >> ["mots", >> ["mot", >> ["nom", >> "iatrogène"], >> ["racine", >> "gène"]], >> ["mot", >> ["nom", >> "hippiatre"], >> ["racine", >> "hippo"]]] >> >> This output is not at all what I need, and think I'm not the only one >> in this case. >> A quick search on Google give us simple online tools which have an >> useful output: >> >> http://codebeautify.org/xmltojson >> { >> "mots": { >> "mot": [ >> { >> "nom": "iatrogène", >> "racine": "gène" >> }, >> { >> "nom": "hippiatre", >> "racine": "hippo" >> } >> ] >> } >> } >> >> http://www.utilities-online.info/xmltojson >> { >> "mots": { >> "mot": [ >> { >> "nom": "iatrogène", >> "racine": "gène" >> }, >> { >> "nom": "hippiatre", >> "racine": "hippo" >> } >> ] >> } >> } >> >> Could we have this "easyjson" option in BaseX ? I know that it has not >> at all the power of lossless conversion from JSON to XML and back, >> like the "direct" and "attributes" options, but who really cares ? >> >> I really think it's what most users need today to use BaseX with >> client side JSON oriented frameworks. >> >> Best regards >> >> Florent >> > > -- > Dirk Kirsten, BaseX GmbH, http://basex.org > |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 >
-- FLOSS Engineer & Lawyer
CG
Christian Grün6 Jan 2015 15:45 #
Hi Florent, Experience has taught us that feature requests that are straightforward to include lead to many new requests that are much harder to satisfy. The same applies to JSON serialization: There are just too many different solutions for serializing XML as JSON (see e.g. [1]), and all of them have their strengths and weaknesses. Here is a trivial example: I think it's surprising, if not questionable, how the following document is serialized by one of the converters you cited [2]:
  <xml>
    <a>x</a>
    <b></b>
    <a>z</a>
 </xml>

  { "xml": { "a": [ "x", "z" ] }
The "b" element is simply discarded, and the "a" elements are represented as array (because a JSON key is unique). But I immeditalely agree that the JSONML output is not helpful and counter-intuitive in nearly all cases, and I also agree that the overhead imposed by our alternative JSON mapper is inconvenient if existing XML snippets are to be serialized. I can think of two solutions: a) We provide some more JSON serializers/parsers in BaseX, which allow you to choose one that best matches a given use case b) We motivate users to write their own JSON serializers in XQuery... Which solution would you prefer? Well, I can guess ;) Christian PS: In our own use cases, we integrate custom JSON mappers that do exactly what we need. [1] http://onwebdevelopment.blogspot.de/2008/05/converting-xml-to-json.html [2] http://www.utilities-online.info/xmltojson
Quoted text · Florent Gallaire · 115 lines
On Mon, Jan 5, 2015 at 9:31 AM, Florent Gallaire <fgallaire at gmail.com> wrote: > Hello Dirk, > > Thank for your answer, but "direct" and "attribute" format are not > easy to use, I could even say that they are too hard to use for normal > people. > > And they need a lot of extra work: <json> root node, _ elements, etc. > I just would like an easy way, with no extra work to process arbitrary > XML. > > An easy use like the jsonml option, but with an useful output. > > Cheers > > Florent > > On Mon, Jan 5, 2015 at 8:38 AM, Dirk Kirsten <dk at basex.org> wrote: >> Hello Florent, >> >> JsonML isn't the only way to generate JSON with BaseX; it isn't even the default one. Please take a look at our documentation at http://docs.basex.org/wiki/JSON to see what is available. We agree that JsonML isn't the most convenient format, that's why the BaseX-specific direct format is the default one. >> >> Cheers, >> Dirk >> >> On 01/05/2015 05:46 AM, Florent Gallaire wrote: >>> Whether my database this XML file: >>> >>> <?xml version="1.0" encoding="UTF-8"?> >>> <mots> >>> <mot> >>> <nom>iatrogène</nom> >>> <racine>gène</racine> >>> </mot> >>> <mot> >>> <nom>hippiatre</nom> >>> <racine>hippo</racine> >>> </mot> >>> </mots> >>> >>> The only easy way to have JSON serialization with BaseX is to use the >>> "format=jsonml" option which returns: >>> ["mots", >>> ["mot", >>> ["nom", >>> "iatrogène"], >>> ["racine", >>> "gène"]], >>> ["mot", >>> ["nom", >>> "hippiatre"], >>> ["racine", >>> "hippo"]]] >>> >>> This output is not at all what I need, and think I'm not the only one >>> in this case. >>> A quick search on Google give us simple online tools which have an >>> useful output: >>> >>> http://codebeautify.org/xmltojson >>> { >>> "mots": { >>> "mot": [ >>> { >>> "nom": "iatrogène", >>> "racine": "gène" >>> }, >>> { >>> "nom": "hippiatre", >>> "racine": "hippo" >>> } >>> ] >>> } >>> } >>> >>> http://www.utilities-online.info/xmltojson >>> { >>> "mots": { >>> "mot": [ >>> { >>> "nom": "iatrogène", >>> "racine": "gène" >>> }, >>> { >>> "nom": "hippiatre", >>> "racine": "hippo" >>> } >>> ] >>> } >>> } >>> >>> Could we have this "easyjson" option in BaseX ? I know that it has not >>> at all the power of lossless conversion from JSON to XML and back, >>> like the "direct" and "attributes" options, but who really cares ? >>> >>> I really think it's what most users need today to use BaseX with >>> client side JSON oriented frameworks. >>> >>> Best regards >>> >>> Florent >>> >> >> -- >> Dirk Kirsten, BaseX GmbH, http://basex.org >> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz >> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: >> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle >> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 >> > > > > -- > FLOSS Engineer & Lawyer
CG
Christian Grün6 Jan 2015 21:29 #
Hi Florent,
Quoted text · Christian Grün · 1 line
> b) We motivate users to write their own JSON serializers in XQuery...
As I failed miserably in my attempt to motivate any user out there (it could be that I was too impatient, though), I just wrote a simple XQuery function for serializing XML to JSON. The JSON output is similar to the one generated by your online tools. Please be warned: The code is experimental (it ignores the famous 10% of corner cases), so any patches are welcome. Have fun, Christian
FG
Florent Gallaire10 Jan 2015 02:25 #
Thanks a lot for your work Chrisitian. Will the code be merged in the master ? Best regards Florent
Quoted text · Christian Grün · 15 lines
On Tue, Jan 6, 2015 at 9:29 PM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Florent, > >> b) We motivate users to write their own JSON serializers in XQuery... > > As I failed miserably in my attempt to motivate any user out there (it > could be that I was too impatient, though), I just wrote a simple > XQuery function for serializing XML to JSON. The JSON output is > similar to the one generated by your online tools. Please be warned: > The code is experimental (it ignores the famous 10% of corner cases), > so any patches are welcome. > > Have fun, > Christian
-- FLOSS Engineer & Lawyer
CG
Christian Grün10 Jan 2015 02:49 #
I'm not sure yet. Does it work for you do far?
Quoted text · Florent Gallaire · 30 lines
Am 10.01.2015 02:25 schrieb "Florent Gallaire" <fgallaire at gmail.com>: > Thanks a lot for your work Chrisitian. > Will the code be merged in the master ? > > > Best regards > > Florent > > On Tue, Jan 6, 2015 at 9:29 PM, Christian Grün > <christian.gruen at gmail.com> wrote: > > Hi Florent, > > > >> b) We motivate users to write their own JSON serializers in XQuery... > > > > As I failed miserably in my attempt to motivate any user out there (it > > could be that I was too impatient, though), I just wrote a simple > > XQuery function for serializing XML to JSON. The JSON output is > > similar to the one generated by your online tools. Please be warned: > > The code is experimental (it ignores the famous 10% of corner cases), > > so any patches are welcome. > > > > Have fun, > > Christian > > > > -- > FLOSS Engineer & Lawyer >
FG
Florent Gallaire12 Jan 2015 17:19 #
Hello Christian, I'm working with BaseX 7.9 and I have a message error: Stopped at /home/../basex/webapp/simple-json.xqm, 33/27: [XPST0003] Incomplete 'if' expression. With the BaseX 8.0 dev version I don't have this error. Florent
Quoted text · Christian Grün · 34 lines
On Sat, Jan 10, 2015 at 2:49 AM, Christian Grün <christian.gruen at gmail.com> wrote: > I'm not sure yet. Does it work for you do far? > > Am 10.01.2015 02:25 schrieb "Florent Gallaire" <fgallaire at gmail.com>: > >> Thanks a lot for your work Chrisitian. >> Will the code be merged in the master ? >> >> >> Best regards >> >> Florent >> >> On Tue, Jan 6, 2015 at 9:29 PM, Christian Grün >> <christian.gruen at gmail.com> wrote: >> > Hi Florent, >> > >> >> b) We motivate users to write their own JSON serializers in XQuery... >> > >> > As I failed miserably in my attempt to motivate any user out there (it >> > could be that I was too impatient, though), I just wrote a simple >> > XQuery function for serializing XML to JSON. The JSON output is >> > similar to the one generated by your online tools. Please be warned: >> > The code is experimental (it ignores the famous 10% of corner cases), >> > so any patches are welcome. >> > >> > Have fun, >> > Christian >> >> >> >> -- >> FLOSS Engineer & Lawyer
-- FLOSS Engineer & Lawyer
CG
Christian Grün12 Jan 2015 17:27 #
Hi Florent, The module uses XQuery 3.1 arrays; those have only been introduced by BaseX 8.0. Hope this helps, Christian
Quoted text · Florent Gallaire · 52 lines
On Mon, Jan 12, 2015 at 5:19 PM, Florent Gallaire <fgallaire at gmail.com> wrote: > Hello Christian, > > I'm working with BaseX 7.9 and I have a message error: > > Stopped at /home/../basex/webapp/simple-json.xqm, 33/27: > [XPST0003] Incomplete 'if' expression. > > With the BaseX 8.0 dev version I don't have this error. > > Florent > > > On Sat, Jan 10, 2015 at 2:49 AM, Christian Grün > <christian.gruen at gmail.com> wrote: >> I'm not sure yet. Does it work for you do far? >> >> Am 10.01.2015 02:25 schrieb "Florent Gallaire" <fgallaire at gmail.com>: >> >>> Thanks a lot for your work Chrisitian. >>> Will the code be merged in the master ? >>> >>> >>> Best regards >>> >>> Florent >>> >>> On Tue, Jan 6, 2015 at 9:29 PM, Christian Grün >>> <christian.gruen at gmail.com> wrote: >>> > Hi Florent, >>> > >>> >> b) We motivate users to write their own JSON serializers in XQuery... >>> > >>> > As I failed miserably in my attempt to motivate any user out there (it >>> > could be that I was too impatient, though), I just wrote a simple >>> > XQuery function for serializing XML to JSON. The JSON output is >>> > similar to the one generated by your online tools. Please be warned: >>> > The code is experimental (it ignores the famous 10% of corner cases), >>> > so any patches are welcome. >>> > >>> > Have fun, >>> > Christian >>> >>> >>> >>> -- >>> FLOSS Engineer & Lawyer > > > > -- > FLOSS Engineer & Lawyer

Written for BaseX 7.9. BaseX may have changed since: please check the current documentation.

« Question / Suggestion : Template mechanisms and operator overloading for XQUERY 3.++ ?Feature request: RNG and Schematron validation »
⚡Generated with XQuery