Main Page » Mailing List » April 2014

JSON serialization via REST API

Monday, 7 April 2014 · 8 messages · last reply 8 Apr 2014
FG
Florent Gallaire7 Apr 2014 17:41 #
Hello, I'm new in the XML databases world, evaluating BaseX and eXist-db for a new project. BaseX seems really clean and lightweight. I would like to use it via the REST API and with a json output. I test with the factbook.xml example database and the method parameter set to "json" : http://localhost:8984/rest/factbook?query=//city/name&wrap=yes works fine but http://localhost:8984/rest/factbook?query=//city/name&wrap=yes&method=json returns [BXJS0002] JSON serializer: <json> expected as root node. I don't understand why. I need json serialization whatever is the XML returned by the query. And I don't understand how to use the json options via the REST API. http://docs.basex.org/wiki/JSON_Module An example could be fine to help. Best regards Florent
-- FLOSS Engineer & Lawyer
DK
Dirk Kirsten7 Apr 2014 18:48 #
Hello Florent, there are actually is an example given (Example 4) in the doc page you mention, although it is just in the other direction (i.e. from json to xml). But as the JSON serialization is lossless the same rules apply in the other direction as well. So if you want to serialize to JSON, you will have to need a <json/> element as root, e.g.
  <json objects="json">
    <myvalue>test</myvalue>
  </json>
You might be wondering why it is necessary to have such a format and the XML is not simply automatically converted to JSON. This is directed to the different nature of JSON and XML; whereas JSON has different types such as numbers and strings, XML just uses Strings. So the conversion can not automatically detect what value should be a string or a number. If you are currently evaluating let me add that you might want to take a look at RestXQ (see https://docs.basex.org/wiki/RESTXQ for more details). This way you could also write one simply Rest interface which automatically wraps your data into a <json/> element (either very simply because you have just strings or by certain rules)) and delivers JSON back do you. Something like this (untested!):
  declare
    %rest:path("")
    %rest:query-param("q", "{$q}")
    %output:method("json")
    function page:json($q as xs:string)
  {
    <json objects="json">{xquery:eval($q)}</json>
  };
Of course you will have a much cleaner architecture if you define certain REST endpoints for your application and not simply evaluate every incoming query (which also introduces certain security problems, so don't run this in production!). Hope you stay with BaseX :-) Cheers, Dirk
Quoted text · Florent Gallaire · 28 lines
On 07/04/14 17:41, Florent Gallaire wrote: > Hello, > > I'm new in the XML databases world, evaluating BaseX and eXist-db for > a new project. > > BaseX seems really clean and lightweight. > > I would like to use it via the REST API and with a json output. > > I test with the factbook.xml example database and the method parameter > set to "json" : > http://localhost:8984/rest/factbook?query=//city/name&wrap=yes > works fine but > http://localhost:8984/rest/factbook?query=//city/name&wrap=yes&method=json > returns [BXJS0002] JSON serializer: <json> expected as root node. > > I don't understand why. I need json serialization whatever is the XML > returned by the query. > > And I don't understand how to use the json options via the REST API. > http://docs.basex.org/wiki/JSON_Module > An example could be fine to help. > > Best regards > > Florent >
-- Dirk Kirsten, BaseX GmbH, http://basex.org |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
FG
Florent Gallaire8 Apr 2014 05:39 #
Hello,
Quoted text · Dirk Kirsten · 4 lines
> For me "json" serialization of XML (i.e. XML->JSON) is a false problem > because you don't lose informations, so the format=json should always > work. If you want to stay backward compatible, maybe a new value for > the format parameter like format=djson for dirtyjson could be useful ?
I just realize that : "Version 7.8 Removed: json:parse-ml, json:serialize-ml Updated: json:parse now returns a document node instead of an element, or an XQuery map if format is set to map. Version 7.7.2 Updated: $options argument added to json:parse and json:serialize Updated: json:parse-ml and json:serialize-ml are now deprecated" So BaseX 7.7.2 and the depreciated method=jsonml is just working fine and doing what I want. http://localhost:8984/rest/factbook?query=//city/name&wrap=yes&method=jsonml Really, why have you removed that ? Because there is the new options argument...it's more complex but ok if knew how to use it with REST API, please help me ^^ : http://localhost:8984/rest/factbook?query=//city/name&wrap=yes&method=json&options=?? Cheers Florent
Quoted text · Dirk Kirsten · 45 lines
> Cheers > > Florent > > >> On 07/04/14 17:41, Florent Gallaire wrote: >>> Hello, >>> >>> I'm new in the XML databases world, evaluating BaseX and eXist-db for >>> a new project. >>> >>> BaseX seems really clean and lightweight. >>> >>> I would like to use it via the REST API and with a json output. >>> >>> I test with the factbook.xml example database and the method parameter >>> set to "json" : >>> http://localhost:8984/rest/factbook?query=//city/name&wrap=yes >>> works fine but >>> http://localhost:8984/rest/factbook?query=//city/name&wrap=yes&method=json >>> returns [BXJS0002] JSON serializer: <json> expected as root node. >>> >>> I don't understand why. I need json serialization whatever is the XML >>> returned by the query. >>> >>> And I don't understand how to use the json options via the REST API. >>> http://docs.basex.org/wiki/JSON_Module >>> An example could be fine to help. >>> >>> Best regards >>> >>> Florent >>> >> >> -- >> Dirk Kirsten, BaseX GmbH, http://basex.org >> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz >> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: >> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle >> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 > > > > -- > FLOSS Engineer & Lawyer
-- FLOSS Engineer & Lawyer
FG
Florent Gallaire8 Apr 2014 06:51 #
Quoted text · Florent Gallaire · 4 lines
> Really, why have you removed that ? Because there is the new options > argument...it's more complex but ok if knew how to use it with REST > API, please help me ^^ : > http://localhost:8984/rest/factbook?query=//city/name&wrap=yes&method=json&options=??
So I finally found how to do that via REST : http://localhost:8984/rest/factbook?query=//city/name&wrap=yes&method=json&json=format=jsonml I'm not very happy with the syntax, which is more heavy than the old method=jsonml, and less clear (json=format=jsonml). But you can add other options like indent=no : http://localhost:8984/rest/factbook?query=//city/name&wrap=yes&method=json&json=format=jsonml,indent=no Maybe a shortcut method=jsonml could be back for simplicity ? And add an example to the wiki ^^ Florent
Quoted text · Florent Gallaire · 54 lines
> Cheers > > Florent > >> Cheers >> >> Florent >> >> >>> On 07/04/14 17:41, Florent Gallaire wrote: >>>> Hello, >>>> >>>> I'm new in the XML databases world, evaluating BaseX and eXist-db for >>>> a new project. >>>> >>>> BaseX seems really clean and lightweight. >>>> >>>> I would like to use it via the REST API and with a json output. >>>> >>>> I test with the factbook.xml example database and the method parameter >>>> set to "json" : >>>> http://localhost:8984/rest/factbook?query=//city/name&wrap=yes >>>> works fine but >>>> http://localhost:8984/rest/factbook?query=//city/name&wrap=yes&method=json >>>> returns [BXJS0002] JSON serializer: <json> expected as root node. >>>> >>>> I don't understand why. I need json serialization whatever is the XML >>>> returned by the query. >>>> >>>> And I don't understand how to use the json options via the REST API. >>>> http://docs.basex.org/wiki/JSON_Module >>>> An example could be fine to help. >>>> >>>> Best regards >>>> >>>> Florent >>>> >>> >>> -- >>> Dirk Kirsten, BaseX GmbH, http://basex.org >>> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz >>> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: >>> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle >>> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 >> >> >> >> -- >> FLOSS Engineer & Lawyer > > > > -- > FLOSS Engineer & Lawyer
-- FLOSS Engineer & Lawyer
DK
Dirk Kirsten8 Apr 2014 12:08 #
Hi Florent,
Quoted text · Florent Gallaire · 7 lines
> I have some data in XML format with daily updates. I don't control > this data, they come to me as XML, I don't want to convert them and > make some mistakes, so I want to use a XML database. If I just add a > <json/> element as root, I fix nothing about the string/integer > conversion so I don't understand in what this is useful. And if XML > only has strings there's no problem to go to JSON, they become strings > in JSON !
While this might be true in your _specific_ case, it is not true in general. Even if you only have strings and don't care about numbers, other people do very much care about that. So we need to provide a mapping. Also the <json objects="json" /> element already adds mapping information, as it also could be <json arrays="json"/>. In the first case you get back an JSON object, in the latter one an array. How could we be able to determine that without any mapping information? You will never be able to do a totally automatical lossless JSON<>XML conversion. However, you are able to write your own mapping functions which are useful for your specific use case
Quoted text · Florent Gallaire · 3 lines
> I have seen RestXQ which is probably excellent but it can't help me, > because I just want a REST interface with JSON, so I can change my > backend without frontend modificiation if needed.
I don't really get your argument here. RestXQ enables you to build a REST interface and you can in fact change your backend without any frontend modifications. In fact, it is much better at this job than the REST interface. As you already discovered, sometimes parameters in newer versions of BaseX change. Using REST you will have to update all REST calls in your frontend. Using RestXQ, you simply adopt your backend code. Also, you would not need all the format information in each and every REST call, as they would be defined in your backend RestXQ code. I agree with you that the new options syntax for JSON output might be more verbose in this case. But it also gives much more flexibility and is much easier to handle especially in a RestXQ context and I am very happy we have this new syntax. Please also keep in mind that the majority of our users are using RestXQ rather than REST. As I already indicated I personally don't see many good reasons to use REST instead of RestXQ for application development. Hence, I very much doubt that we will switch back to the old method=jsonml syntax. Cheers, Dirk
-- Dirk Kirsten, BaseX GmbH, http://basex.org |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
FG
Florent Gallaire8 Apr 2014 14:23 #
Quoted text · Dirk Kirsten · 3 lines
> You will never be able to do a totally automatical lossless JSON<>XML > conversion. However, you are able to write your own mapping functions > which are useful for your specific use case
OK.
Quoted text · Dirk Kirsten · 8 lines
> I don't really get your argument here. RestXQ enables you to build a > REST interface and you can in fact change your backend without any > frontend modifications. In fact, it is much better at this job than the > REST interface. As you already discovered, sometimes parameters in newer > versions of BaseX change. Using REST you will have to update all REST > calls in your frontend. Using RestXQ, you simply adopt your backend > code. Also, you would not need all the format information in each and > every REST call, as they would be defined in your backend RestXQ code.
Yes, you're right. The problem is more that I'm afraid to be dependant (and to learn) yet an other technologie. And if tomorrow the data given to me are no more XML, I will change my database and some URL in my client and it's OK. If I'm based on some other (great) techno, this could be more tougher. But I will watch at it again, and if some few lines of RestXQ simplify my work, I will probably use it :-)
Quoted text · Dirk Kirsten · 8 lines
> I agree with you that the new options syntax for JSON output might be > more verbose in this case. But it also gives much more flexibility and > is much easier to handle especially in a RestXQ context and I am very > happy we have this new syntax. Please also keep in mind that the > majority of our users are using RestXQ rather than REST. As I already > indicated I personally don't see many good reasons to use REST instead > of RestXQ for application development. Hence, I very much doubt that we > will switch back to the old method=jsonml syntax.
I think that the back of method=jsonml as a shortcut or the default json set at jsonml instead of direct is a plus for new adopters, because this help to have easily something that work, and so stay with BaseX. I think that most of the non-JAVA non-XML people (as I am) just want that. I was evaluating eXist first, they are doing easy JSON serialization for output: http://exist-db.org/exist/apps/wiki/blogs/eXist/JSONSerializer But as I don't found how to do it via REST, I switched to BaseX. Do you have some informations about the performance penalty to do JSON conversion at the BaseX level ? Cheers Florent
-- FLOSS Engineer & Lawyer
DK
Dirk Kirsten8 Apr 2014 17:21 #
Hi Florent,
Quoted text · Florent Gallaire · 2 lines
> Yes, you're right. The problem is more that I'm afraid to be dependant > (and to learn) yet an other technologie.
true; of course there always is a learning curve. But we are commited to open technologies and RestXQ is an open standard and in fact it is also not that much complicated. In the end it is mainly a few annotations to server your XQueries at server side.
Quoted text · Florent Gallaire · 4 lines
> > And if tomorrow the data given to me are no more XML, I will change my > database and some URL in my client and it's OK. If I'm based on some > other (great) techno, this could be more tougher.
That's true, but if you change your database technology in the future you will always have to make changes. If you just use a simply and easy RestXQ layer that won't be much. You would have the same if you have complicated REST calls, which you would have to rewrite. In the end you can use RestXQ not only as database layer, but as business logic as it is a full-fledged language. Of course, if you write lots of business logic than and in some time decide to not use it any more it will be a major effort to switch. But this also applies if you want to replace any other business logic language.
Quoted text · Florent Gallaire · 5 lines
> I think that the back of method=jsonml as a shortcut or the default > json set at jsonml instead of direct is a plus for new adopters, > because this help to have easily something that work, and so stay with > BaseX. I think that most of the non-JAVA non-XML people (as I am) just > want that.
I see your point. It is always a trade-off between simplicity for beginners and flexibility for experts. As JsonML is not our default JSON conversion algorithm I think it is ok to have a bit more difficult syntax.
Quoted text · Florent Gallaire · 2 lines
> Do you have some informations about the performance penalty to do JSON > conversion at the BaseX level ?
I don't think we have a specific numbers on this at it heavily depends on your data. Simplest thing will probably to give it a try for your specific data set. In general I would say it should not be a major bottleneck. But of course if you want a super-fast JSON database there are more suitable solutions than BaseX. Cheers, Dirk
-- Dirk Kirsten, BaseX GmbH, http://basex.org |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
CG
Christian Grün8 Apr 2014 17:43 #
Hi Florent,
Quoted text · Dirk Kirsten · 7 lines
>> Do you have some informations about the performance penalty to do JSON >> conversion at the BaseX level ? > > I don't think we have a specific numbers on this at it heavily depends > on your data. Simplest thing will probably to give it a try for your > specific data set. In general I would say it should not be a major > bottleneck.
I agree with Dirk; the conversion between XML and JSON is optimized pretty well and shouldn't cause any considerable overhead. Cheers, Christian

Written for BaseX 7.8. BaseX may have changed since: please check the current documentation.

« Is there a way to identify an inline function in BaseX?setting global options »
⚡Generated with XQuery