Request and JSON parsing
Thursday, 25 April 2024 · 6 messages · last reply 26 Apr 2024
GD
Günter Dunz-Wolff25 Apr 2024 21:58 #
Hi there,
via RESTXQ I request a vectorstore. The response is an JSON-Object.
For example:
With the following curl:
curl https://vector.io/query-data \
-H "Authorization: Bearer ABcFMGhetc" \
-d '{ "data": "Du hast dem Menschen ein so freies, herrliches und üppiges Leben bestimmt", "topK": 2, "includeVectors": false, "includeMetadata": true }'
I'm getting this object:
{
"result" : [ {
"id" : "ba02277",
"score" : 0.9949091,
"metadata" : {"genre":"ba","blatt":"1810-01","autor":"Kleist","titel":"Zoroaster.","link":"https://kleist-digital.de/etc","text":"freies, herrliches und üppiges Leben bestimmt."}
}, {
"id" : "ba01711",
"score" : 0.8891792,
"metadata" : {"genre":"ba","blatt":"1810-29","autor":"Ompteda","titel":"Fragmente","link":"https://kleist-digital.de/etc","text":"So mit dem Menschen im Moralischen."}
} ]
}
1.
Is the request in my RESTXQ in the right form?
http:send-request(
<http:request
method="post"
href="{$prod:url}/query-data">
<http:header
name="Authorization"
value="Bearer {$prod:token}"/>
<http:body
media-type="application/json">
{{
"data": "{$query}",
"topK": 20,
"includeVectors": false,
"includeMetadata": true
}}
</http:body>
</http:request>
)
2.
I'm getting a full Object (but I dont know, if its the raw JSON-Object like above), but I am not able, to parse it, to get a list in html.
In the documentation https://docs.basex.org/wiki/JSON_Module is an example:
let $input := '{
"Title": "Drinks",
"Author": [ "Jim Daniels", "Jack Beam" ]
}'
let $data := json:parse($input, map { 'format': 'xquery' })
return map:for-each($data, function($k, $v) {
$k || ': ' || string-join($v, ', ')
})
But my Object is more complex and I don't know, how to get two and more datasets with $k and $v out of the object.
I want to have the following list:
<div>
<li>
<ul>
<li><span>Score: </span> <span>result score</span></li>
<li><span>Blatt: </span> <span>result blatt</span></li>
<li><span>Autor: </span> <span>result autor</span></li>
<li><span>Titel: </span> <span>result titel</span></li>
<li><span>Text: </span> <span>result text</span></li>
<li><span>Link: </span> <span>result link</span></li>
</ul>
</li>
<ul>
<li></li>
</ul>
etc.
</div>Thanks for any help
Günter
MH
Martin Honnen25 Apr 2024 22:29 #
Quoted text · Günter Dunz-Wolff · 81 lines
On 25/04/2024 21:58, Günter Dunz-Wolff wrote: > Hi there, > > via RESTXQ I request a vectorstore. The response is an JSON-Object. > > For example: > With the following curl: > > curl https://vector.io/query-data \ > > -H "Authorization: Bearer ABcFMGhetc" \ > > -d '{ "data": "Du hast dem Menschen ein so freies, herrliches und > üppiges Leben bestimmt", "topK": 2, "includeVectors": false, > "includeMetadata": true }' > > > I'm getting this object: > > > { > > "result" : [ { > > "id" : "ba02277", > > "score" : 0.9949091, > > "metadata" : > {"genre":"ba","blatt":"1810-01","autor":"Kleist","titel":"Zoroaster.","link":"https://kleist-digital.de/etc","text":"freies, > herrliches und üppiges Leben bestimmt."} > > }, { > > "id" : "ba01711", > > "score" : 0.8891792, > > "metadata" : > {"genre":"ba","blatt":"1810-29","autor":"Ompteda","titel":"Fragmente","link":"https://kleist-digital.de/etc","text":"So > mit dem Menschen im Moralischen."} > > } ] > > } > > > > <div> > > <li> > > <ul> > > <li><span>Score: </span> <span>result score</span></li> > > <li><span>Blatt: </span> <span>result blatt</span></li> > > <li><span>Autor: </span> <span>result autor</span></li> > > <li><span>Titel: </span> <span>result titel</span></li> > > <li><span>Text: </span> <span>result text</span></li> > > <li><span>Link: </span> <span>result link</span></li> > > </ul> > > </li> > > <ul> > > <li></li> > > </ul> > > etc. > > </div> > >
Parse the result (if it is not parsed and you have it as JSON string) with parse-json($result)) then run XQuery like
<div>
{
parse-json($result)?result?*!(let $score := ?score return ?metadata ! <ul>
<li><span>Score: </span> <span>{$score}</span></li>
<li><span>Blatt: </span> <span>{?blatt}</span></li>
<li><span>Autor: </span> <span>{?autor}</span></li>
<li><span>Titel: </span> <span>{?titel}</span></li>
<li><span>Text: </span> <span>{?text}</span></li>
<li><span>Link: </span> <span>{?link}</span></li>
</ul>
)
}
</div>and it should give you something like
<div>
<ul>
<li>
<span>Score: </span>
<span>0.9949091</span>
</li>
<li>
<span>Blatt: </span>
<span>1810-01</span>
</li>
<li>
<span>Autor: </span>
<span>Kleist</span>
</li>
<li>
<span>Titel: </span>
<span>Zoroaster.</span>
</li>
<li>
<span>Text: </span>
<span>freies, herrliches und üppiges Leben bestimmt.</span>
</li>
<li>
<span>Link: </span>
<span>https://kleist-digital.de/etc</span>
</li>
</ul>
<ul>
<li>
<span>Score: </span>
<span>0.8891792</span>
</li>
<li>
<span>Blatt: </span>
<span>1810-29</span>
</li>
<li>
<span>Autor: </span>
<span>Ompteda</span>
</li>
<li>
<span>Titel: </span>
<span>Fragmente</span>
</li>
<li>
<span>Text: </span>
<span>So mit dem Menschen im Moralischen.</span>
</li>
<li>
<span>Link: </span>
<span>https://kleist-digital.de/etc</span>
</li>
</ul>
</div>It is not quite the result you showed but I have left out the outer
li elements as an li inside a div doesn't make sense
Fiddle
https://bxfiddle.cloud.basexgmbh.de/?share=%28%27query%21%27%3Cdiv%3EH%28H%3Fresult%3F*%27%3A%7Blet+%24V+%3A%3D+%3FV+return+%3Fmetadata+%27_%3CulUScore5%24V3BlattYblatt3AutorYautor3TiwlYtiwl3TextYwxt3LinkYlink%29IJIli%3EHHIul%3EH%7DH%29HIdiv%3E%27%7Econwxt%21%27%28C6resultG%5BW2277z9949091%2CCE-01NautorRKleistNtiwlRZoroaswr.ND6freies%2C+herrliches+und+%C3%BCppiges+Leben+bestimmt.6%29C%29%2CW1711z8891792%2CCE-29NautorROmpwdaNtiwlRFragmenwND6So+mit+dem+Menschen+im+Moralischen.6%29C%29+%5DHH%29%27%7Emode%21%27XQuery+%7BBaseX%7D%27%7EconwxtType%21%27json%27%293%29IJIliU5_IJ+%3CJ%286%5C%27CHHPDlinkRhttps%3A%2F%2Fkleist-digital.de%2FetcNwxt%5C%21EP6metadataG%286genreRbaNblattR1810G6+_H%5CnI%3C%2FJspan%3EN6%2C6P++R%5C%216U%3EHH%5Ct%3Cli%3E%3CJVscoreW+%28CP6idG6ba0Y5%3F_%3A+wtez6%2CCP6VG0.%01zw_YWVURPNJIHGEDC653_CG
Hi Günter,
Quoted text · Günter Dunz-Wolff · 3 lines
2. I'm getting a full Object (but I dont know, if its the raw JSON-Object > like above), but I am not able, to parse it, to get a list in html. >
Could you share the result of your response with us (possibly shortened)?
It would be interesting to learn something about the type of the response body. What do you get if you
inspect:type($response[2]) ?let $data := json:parse($input, map { 'format': 'xquery' })Quoted text · Günter Dunz-Wolff · 7 lines
> > return map:for-each($data, function($k, $v) { > > $k || ': ' || string-join($v, ', ') > > }) >
If you use
json:parse($input) without a specific format, you’ll get an XML representation of the JSON data, which is usually simpler to postprocess.
Hope this helps
ChristianGD
Günter Dunz-Wolff26 Apr 2024 11:05 #
Hi Christian, hi Martin,
I checked the response-typ. It's "document-node()".
The structure is like this:
<json type="object">
<result type="array">
<_ type="object">
<id>ba03177</id>
<score type="number">0.83175087</score>
<metadata type="object">
<genre>ba</genre>
<blatt>1810-40</blatt>
<autor>Heinrich von Kleist</autor>
<titel>Die heilige Cäcilie</titel>
<link>
"https://kleist-digital.de/berliner-abendblaetter-etc"
<text>Die Aebtissinn , die schon, in der Stunde der Mitternacht, durch einen Freund, von der Gefahr, etc</text>
</metadata>
<!--_-->
etc.I hope this helps. How to parse it, to get my proposed html-structure?
Thanks for further help.
Günter
Quoted text · Christian Grün · 22 lines
> Am 25.04.2024 um 23:08 schrieb Christian Grün <christian.gruen at gmail.com>: > > Hi Günter, > >> 2. I'm getting a full Object (but I dont know, if its the raw JSON-Object like above), but I am not able, to parse it, to get a list in html. > > > Could you share the result of your response with us (possibly shortened)? > > It would be interesting to learn something about the type of the response body. What do you get if you inspect:type($response[2]) ? > >> let $data := json:parse($input, map { 'format': 'xquery' }) >> return map:for-each($data, function($k, $v) { >> $k || ': ' || string-join($v, ', ') >> }) > > > If you use json:parse($input) without a specific format, you’ll get an XML representation of the JSON data, which is usually simpler to postprocess. > > Hope this helps > Christian >
CG
Christian Grün26 Apr 2024 11:21 #
Thanks. Here’s one way to do it:
let $body := document {
<json type="object">
<result type="array">
<_ type="object">
<id>ba03177</id>
<score type="number">0.83175087</score>
<metadata type="object">
<genre>ba</genre>
<blatt>1810-40</blatt>
<autor>Heinrich von Kleist</autor>
<titel>Die heilige Cäcilie</titel>
<link>https://kleist-digital.de/...</link>
<text>Die Aebtissinn, ...</text>
</metadata>
</_>
</result>
</json>
}
return <div>
<ul>{
for $result in $body/json/result/_
return <ul>{
for $field in ('Score', 'Blatt', 'Autor', 'Titel', 'Text', 'Link')
return <li>
<span>{ $field }: </span>
<span>{ data($result//*[name() = lower-case($field)]) }</span>
</li>
}</ul>
}</ul>
</div>Another one is…
return <div>
<ul>{
for $result in $body/json/result/_
return <ul>{
for $text in $result//text()
let $field := name($text/..)
return <li>
<span>{ $field ! (upper-case(substring(., 1, 1)) || substring(.,
2)) }: </span>
<span>{ $text }</span>
</li>
}</ul>
}</ul>
</div>…but of course you can also output only the relevant fields.
Hope this helps,
Christian
GD
Günter Dunz-Wolff26 Apr 2024 11:55 #
Hi Christian,
now it's working. Thanks a lot!!
Regards
Günter
Quoted text · Christian Grün · 56 lines
> Am 26.04.2024 um 11:21 schrieb Christian Grün <christian.gruen at gmail.com>: > > Thanks. Here’s one way to do it: > > let $body := document { > <json type="object"> > <result type="array"> > <_ type="object"> > <id>ba03177</id> > <score type="number">0.83175087</score> > <metadata type="object"> > <genre>ba</genre> > <blatt>1810-40</blatt> > <autor>Heinrich von Kleist</autor> > <titel>Die heilige Cäcilie</titel> > <link>https://kleist-digital.de/.. <https://kleist-digital.de/>.</link> > <text>Die Aebtissinn, ...</text> > </metadata> > </_> > </result> > </json> > } > return <div> > <ul>{ > for $result in $body/json/result/_ > return <ul>{ > for $field in ('Score', 'Blatt', 'Autor', 'Titel', 'Text', 'Link') > return <li> > <span>{ $field }: </span> > <span>{ data($result//*[name() = lower-case($field)]) }</span> > </li> > }</ul> > }</ul> > </div> > > Another one is… > > return <div> > <ul>{ > for $result in $body/json/result/_ > return <ul>{ > for $text in $result//text() > let $field := name($text/..) > return <li> > <span>{ $field ! (upper-case(substring(., 1, 1)) || substring(., 2)) }: </span> > <span>{ $text }</span> > </li> > }</ul> > }</ul> > </div> > > …but of course you can also output only the relevant fields. > > Hope this helps, > Christian >
Written for BaseX 10.7. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery