parse-json() function reorder keys in the resulting map{}

Am 16.12.2024 um 13:42 schrieb Oleksandr Shpak: > Hi team, > > Found one "issue" with key ordering in the resulting map. Need to > clarify, is it normal behavior or a bug. > > json:parse('{"b":1, "a":3}', { 'format': 'xquery' }) returns > {"a":3.0e0,"b":1.0e0} > > but > json:parse('{"b":1, "a":3}') > <json type="object"><b type="number">1</b><a type="number">3</a></json> > > As you see, the standard function returns keys not in the order as it > was set in the original JSON string. I didn't find any requirement in > xQuery spec that the resulting map entries should be sorted by the key > name. > > Is it a bug or a feature? > If a feature I would be really happy to see one additional param to > preserve key ordering. >

Am 16.12.2024 um 14:15 schrieb Martin Honnen: > There is a big debate in the working group developing XQuery, XPath > and XSLT 4 on whether to introduce a new map type that preserves key > insertion/creation order > >

On Mon, Dec 16, 2024 at 3:31 PM Martin Honnen <martin.honnen at gmx.de> wrote: > > Am 16.12.2024 um 14:15 schrieb Martin Honnen: > > There is a big debate in the working group developing XQuery, XPath > > and XSLT 4 on whether to introduce a new map type that preserves key > > insertion/creation order > > > > > See https://github.com/qt4cg/qtspecs/issues/1651 >

Am 16.12.2024 um 15:19 schrieb Oleksandr Shpak: > Hi Martin, > > I found even more weird situation > parse-json('{"1:aa": 1, "2:bb": 2}') > returns > {"2:bb":2.0e0,"1:aa":1.0e0} > > Seems like order is inversed > But, > parse-json('{"aa": 1, "b": 2, "ab": 3}') > returns > {"aa":1.0e0,"ab":3.0e0,"b":2.0e0} > > and, finally > parse-json('{"aa": 1, "b": 2, "ab": 3, "1:aa": 1, "2:bb": 2}') > returns > {"aa":1.0e0,"ab":3.0e0,"b":2.0e0,"2:bb":2.0e0,"1:aa":1.0e0} > > 🤯 > > It seems like a bug to me ;) > >

> > If we are talking about the JSON standard, not xQuery, devs will > expect the function which behaves in the same way as it does in many > other languages. >

On Mon, Dec 16, 2024 at 4:37 PM Martin Honnen <martin.honnen at gmx.de> wrote: > > Am 16.12.2024 um 15:19 schrieb Oleksandr Shpak: > > Hi Martin, > > I found even more weird situation > parse-json('{"1:aa": 1, "2:bb": 2}') > returns > {"2:bb":2.0e0,"1:aa":1.0e0} > > Seems like order is inversed > But, > parse-json('{"aa": 1, "b": 2, "ab": 3}') > returns > {"aa":1.0e0,"ab":3.0e0,"b":2.0e0} > > and, finally > parse-json('{"aa": 1, "b": 2, "ab": 3, "1:aa": 1, "2:bb": 2}') > returns > {"aa":1.0e0,"ab":3.0e0,"b":2.0e0,"2:bb":2.0e0,"1:aa":1.0e0} > > 🤯 > > It seems like a bug to me ;) > > > As I said, in XPath 3.1 and XQuery 3.1 maps are unordered collections, you > might get any order when parsing a particular JSON into an XDM 3.1 map. > > If it is any help, in the BaseX fiddle you probably get/see the order > preservation you want/expect > <https://fiddle.basex.org/?share=%28%27query%21%275.**8.A6%21%27%27%7Emode%21%27XQuery+%7BBaseX6Type%21%27xml%27%29*A%7D%2C%5Cn579b024ab03.71%3A92%3Abb020%5C%21+4875parse-json%7B%22%286%7D%27%7Econtext7%5C%278%2C+9aa014A%29%22%01A9876540.*_> > . > > > > > If we are talking about the JSON standard, not xQuery, devs will expect > the function which behaves in the same way as it does in many other > languages. > > Well, I have been around long enough to remember JavaScript/ECMAScript > objects as unordered so some features change in the course of time, I also > think, Python, for instance only recently switched to some order > preservation in its dictionary type. > > I don't know what to suggest for the current BaseX release, unfortunately > http://www.woerteler.de/xquery/modules/ordered-map.html is not accessible. >

On Mon, Dec 16, 2024 at 4:01 PM Oleksandr Shpak <shadowkin at gmail.com> wrote: > Since ECMAScript v5 if I remember correctly object keys are ordered, > because under the hood List object should be used to store keys. > > > On Mon, Dec 16, 2024 at 4:37 PM Martin Honnen <martin.honnen at gmx.de> > wrote: > >> >> Am 16.12.2024 um 15:19 schrieb Oleksandr Shpak: >> >> Hi Martin, >> >> I found even more weird situation >> parse-json('{"1:aa": 1, "2:bb": 2}') >> returns >> {"2:bb":2.0e0,"1:aa":1.0e0} >> >> Seems like order is inversed >> But, >> parse-json('{"aa": 1, "b": 2, "ab": 3}') >> returns >> {"aa":1.0e0,"ab":3.0e0,"b":2.0e0} >> >> and, finally >> parse-json('{"aa": 1, "b": 2, "ab": 3, "1:aa": 1, "2:bb": 2}') >> returns >> {"aa":1.0e0,"ab":3.0e0,"b":2.0e0,"2:bb":2.0e0,"1:aa":1.0e0} >> >> 🤯 >> >> It seems like a bug to me ;) >> >> >> As I said, in XPath 3.1 and XQuery 3.1 maps are unordered collections, >> you might get any order when parsing a particular JSON into an XDM 3.1 map. >> >> If it is any help, in the BaseX fiddle you probably get/see the order >> preservation you want/expect >> <https://fiddle.basex.org/?share=%28%27query%21%275.**8.A6%21%27%27%7Emode%21%27XQuery+%7BBaseX6Type%21%27xml%27%29*A%7D%2C%5Cn579b024ab03.71%3A92%3Abb020%5C%21+4875parse-json%7B%22%286%7D%27%7Econtext7%5C%278%2C+9aa014A%29%22%01A9876540.*_> >> . >> >> >> >> >> If we are talking about the JSON standard, not xQuery, devs will expect >> the function which behaves in the same way as it does in many other >> languages. >> >> Well, I have been around long enough to remember JavaScript/ECMAScript >> objects as unordered so some features change in the course of time, I also >> think, Python, for instance only recently switched to some order >> preservation in its dictionary type. >> >> I don't know what to suggest for the current BaseX release, unfortunately >> http://www.woerteler.de/xquery/modules/ordered-map.html is not >> accessible. >> > > > -- > s0rr0w >

On Mon, Dec 16, 2024 at 5:18 PM Christian Grün <christian.gruen at gmail.com> wrote: > Hi Oleksandr, > > As Martin has summarized, the the current behavior is not a bug as the > XQuery 3.x specification does not define order on maps. It is up to the > implementation to choose an internal order for entries (usually realized > via an immutable/persistent hash structure) that maximizes access time. > With XQuery 4, as indicated, this is about to change. As you may have seen > if you have clicked the Fiddle link, a proof-of-concept implementation is > available now that demonstrates the new behavior. It is also included in > the latest snapshot of BaseX [1]. We will further pursue it in BaseX, no > matter how the spec evolves. > > As for JavaScript, the JSON specification (RFC 8259) states that the order > of key/value pairs is not guaranteed. In ECMAScript, however, orderedness > was introduced with ES6. > > Hope this helps, > Christian > > [1] https://files.basex.org/releases/latest/ > > > On Mon, Dec 16, 2024 at 4:01 PM Oleksandr Shpak <shadowkin at gmail.com> > wrote: > >> Since ECMAScript v5 if I remember correctly object keys are ordered, >> because under the hood List object should be used to store keys. >> >> >> On Mon, Dec 16, 2024 at 4:37 PM Martin Honnen <martin.honnen at gmx.de> >> wrote: >> >>> >>> Am 16.12.2024 um 15:19 schrieb Oleksandr Shpak: >>> >>> Hi Martin, >>> >>> I found even more weird situation >>> parse-json('{"1:aa": 1, "2:bb": 2}') >>> returns >>> {"2:bb":2.0e0,"1:aa":1.0e0} >>> >>> Seems like order is inversed >>> But, >>> parse-json('{"aa": 1, "b": 2, "ab": 3}') >>> returns >>> {"aa":1.0e0,"ab":3.0e0,"b":2.0e0} >>> >>> and, finally >>> parse-json('{"aa": 1, "b": 2, "ab": 3, "1:aa": 1, "2:bb": 2}') >>> returns >>> {"aa":1.0e0,"ab":3.0e0,"b":2.0e0,"2:bb":2.0e0,"1:aa":1.0e0} >>> >>> 🤯 >>> >>> It seems like a bug to me ;) >>> >>> >>> As I said, in XPath 3.1 and XQuery 3.1 maps are unordered collections, >>> you might get any order when parsing a particular JSON into an XDM 3.1 map. >>> >>> If it is any help, in the BaseX fiddle you probably get/see the order >>> preservation you want/expect >>> <https://fiddle.basex.org/?share=%28%27query%21%275.**8.A6%21%27%27%7Emode%21%27XQuery+%7BBaseX6Type%21%27xml%27%29*A%7D%2C%5Cn579b024ab03.71%3A92%3Abb020%5C%21+4875parse-json%7B%22%286%7D%27%7Econtext7%5C%278%2C+9aa014A%29%22%01A9876540.*_> >>> . >>> >>> >>> >>> >>> If we are talking about the JSON standard, not xQuery, devs will expect >>> the function which behaves in the same way as it does in many other >>> languages. >>> >>> Well, I have been around long enough to remember JavaScript/ECMAScript >>> objects as unordered so some features change in the course of time, I also >>> think, Python, for instance only recently switched to some order >>> preservation in its dictionary type. >>> >>> I don't know what to suggest for the current BaseX release, >>> unfortunately http://www.woerteler.de/xquery/modules/ordered-map.html >>> is not accessible. >>> >> >> >> -- >> s0rr0w >> >