Main Page » Mailing List » January 2013

BaseX: Support for JSON

Monday, 21 January 2013 · 6 messages · last reply 22 Jan 2013
AN
Abishek N21 Jan 2013 21:56 #
Hello I am using twitter4j streaming api to capture tweets and feeding them into a basex database, converting them into xml using json api. While using the json:parse api, i get BXJS0001 error often. I assume it is because of encoding. Is there an option available in the json: api that would allow me to specify the enoding technique? Also if i store the documents as a json, instead would it facilitate json query? Thanks, Abishek
CG
Christian Grün22 Jan 2013 09:29 #
Hi Abishek, our JSON Module assumes that the input is a correctly encoded Unicode string. More specifically, it takes an xs:string as argument, which should usually already be valid Unicode. You could start and have a look at the rejected JSON strings by wrapping your json function call with try/catch:
  for $json in ('...') return
  try {
    json:parse($json)
  } catch bxerr:BXJS0001 {
    'Error message: ' || $err:description || out:nl() ||
    'Input string: ' || $json
  }
What’s the error message you are getting as result? Christian
Quoted text · Abishek N · 13 lines
> I am using twitter4j streaming api to capture tweets and feeding > them into a basex database, converting them into xml using json api. > While using the json:parse api, i get BXJS0001 error often. I assume > it is because of encoding. Is there an option available in the json: > api that would allow me to specify the enoding technique? > Also if i store the documents as a json, instead would it facilitate json query? > > Thanks, > Abishek > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
AW
Andreas Weiler22 Jan 2013 09:38 #
Hi Abishek, you have to use the raw JSON content of the status object from Twitter4J. DataObjectFactory.getRawJSON(status) I didn't have any issues with parsing and storing that string. -- Andreas
Quoted text · Christian Grün · 39 lines
Am 22.01.2013 um 09:29 schrieb Christian Grün: > Hi Abishek, > > our JSON Module assumes that the input is a correctly encoded Unicode > string. More specifically, it takes an xs:string as argument, which > should usually already be valid Unicode. > > You could start and have a look at the rejected JSON strings by > wrapping your json function call with try/catch: > > for $json in ('...') return > try { > json:parse($json) > } catch bxerr:BXJS0001 { > 'Error message: ' || $err:description || out:nl() || > 'Input string: ' || $json > } > > What’s the error message you are getting as result? > Christian > ___________________________ > >> I am using twitter4j streaming api to capture tweets and feeding >> them into a basex database, converting them into xml using json api. >> While using the json:parse api, i get BXJS0001 error often. I assume >> it is because of encoding. Is there an option available in the json: >> api that would allow me to specify the enoding technique? >> Also if i store the documents as a json, instead would it facilitate json query? >> >> Thanks, >> Abishek >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
AN
Abishek N22 Jan 2013 17:56 #
I did not get the exception for every tweet that I was trying to capture. half them got through while the rest errored out. most of them error out stating this Error [bxerr:BXJS0001]: JSON parser (1:3817): Char "0" found, " expected on my eclipse obviously with different line numbers. I am actually using JSONObject.toString to get the Json string Is there a way i can make it print the whole stack trace?
Quoted text · Andreas Weiler · 66 lines
On Tue, Jan 22, 2013 at 3:38 AM, Andreas Weiler <andreas.weiler at uni-konstanz.de> wrote: > Hi Abishek, > > you have to use the raw JSON content of the status object from Twitter4J. > DataObjectFactory.getRawJSON(status) > > I didn't have any issues with parsing and storing that string. > > -- Andreas > > Am 22.01.2013 um 09:29 schrieb Christian Grün: > > Hi Abishek, > > our JSON Module assumes that the input is a correctly encoded Unicode > string. More specifically, it takes an xs:string as argument, which > should usually already be valid Unicode. > > You could start and have a look at the rejected JSON strings by > wrapping your json function call with try/catch: > > for $json in ('...') return > try { > json:parse($json) > } catch bxerr:BXJS0001 { > 'Error message: ' || $err:description || out:nl() || > 'Input string: ' || $json > } > > What’s the error message you are getting as result? > Christian > ___________________________ > > I am using twitter4j streaming api to capture tweets and feeding > > them into a basex database, converting them into xml using json api. > > While using the json:parse api, i get BXJS0001 error often. I assume > > it is because of encoding. Is there an option available in the json: > > api that would allow me to specify the enoding technique? > > Also if i store the documents as a json, instead would it facilitate json > query? > > > Thanks, > > Abishek > > _______________________________________________ > > BaseX-Talk mailing list > > BaseX-Talk at mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >
-- Regards Abishek
AN
Abishek Nin reply to Andreas Weiler, 22 Jan 2013, 09:3822 Jan 2013 18:07 #
Andreas, I created a new onStatus method which would pass the jason object itself on twitter4j. Using this jason object I do JSONObject.toString operation to convert the JSON to string. I validated this string with proper json to xml editors and this is perfect. Nearly half of the JSON strings dont get parsed and throws an error while doing a json:parse on the xquery side. Thanks Abishek
Quoted text · Andreas Weiler · 66 lines
On Tue, Jan 22, 2013 at 3:38 AM, Andreas Weiler <andreas.weiler at uni-konstanz.de> wrote: > Hi Abishek, > > you have to use the raw JSON content of the status object from Twitter4J. > DataObjectFactory.getRawJSON(status) > > I didn't have any issues with parsing and storing that string. > > -- Andreas > > Am 22.01.2013 um 09:29 schrieb Christian Grün: > > Hi Abishek, > > our JSON Module assumes that the input is a correctly encoded Unicode > string. More specifically, it takes an xs:string as argument, which > should usually already be valid Unicode. > > You could start and have a look at the rejected JSON strings by > wrapping your json function call with try/catch: > > for $json in ('...') return > try { > json:parse($json) > } catch bxerr:BXJS0001 { > 'Error message: ' || $err:description || out:nl() || > 'Input string: ' || $json > } > > What’s the error message you are getting as result? > Christian > ___________________________ > > I am using twitter4j streaming api to capture tweets and feeding > > them into a basex database, converting them into xml using json api. > > While using the json:parse api, i get BXJS0001 error often. I assume > > it is because of encoding. Is there an option available in the json: > > api that would allow me to specify the enoding technique? > > Also if i store the documents as a json, instead would it facilitate json > query? > > > Thanks, > > Abishek > > _______________________________________________ > > BaseX-Talk mailing list > > BaseX-Talk at mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >
-- Regards Abishek
AN
Abishek N22 Jan 2013 19:03 #
I see where the problem is now, it was not able to parse character &, I URLEncoded the json and was able to resolve it thanks for your help. I was able to find this out by printing the json which was received by the xqm module and the json got truncated after &.
Quoted text · Abishek N · 83 lines
On Tue, Jan 22, 2013 at 11:07 AM, Abishek N <abishek.n at gmail.com> wrote: > Andreas, > I created a new onStatus method which would pass the jason object > itself on twitter4j. Using this jason object I do JSONObject.toString > operation to convert the JSON to string. I validated this string with > proper json to xml editors and this is perfect. > > Nearly half of the JSON strings dont get parsed and throws an error > while doing a json:parse on the xquery side. > Thanks > Abishek > > On Tue, Jan 22, 2013 at 3:38 AM, Andreas Weiler > <andreas.weiler at uni-konstanz.de> wrote: >> Hi Abishek, >> >> you have to use the raw JSON content of the status object from Twitter4J. >> DataObjectFactory.getRawJSON(status) >> >> I didn't have any issues with parsing and storing that string. >> >> -- Andreas >> >> Am 22.01.2013 um 09:29 schrieb Christian Grün: >> >> Hi Abishek, >> >> our JSON Module assumes that the input is a correctly encoded Unicode >> string. More specifically, it takes an xs:string as argument, which >> should usually already be valid Unicode. >> >> You could start and have a look at the rejected JSON strings by >> wrapping your json function call with try/catch: >> >> for $json in ('...') return >> try { >> json:parse($json) >> } catch bxerr:BXJS0001 { >> 'Error message: ' || $err:description || out:nl() || >> 'Input string: ' || $json >> } >> >> What’s the error message you are getting as result? >> Christian >> ___________________________ >> >> I am using twitter4j streaming api to capture tweets and feeding >> >> them into a basex database, converting them into xml using json api. >> >> While using the json:parse api, i get BXJS0001 error often. I assume >> >> it is because of encoding. Is there an option available in the json: >> >> api that would allow me to specify the enoding technique? >> >> Also if i store the documents as a json, instead would it facilitate json >> query? >> >> >> Thanks, >> >> Abishek >> >> _______________________________________________ >> >> BaseX-Talk mailing list >> >> BaseX-Talk at mailman.uni-konstanz.de >> >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> >> > > > > -- > Regards Abishek
-- Regards Abishek

Written for BaseX 7.5. BaseX may have changed since: please check the current documentation.

« Use of BaseXHTTPS »
⚡Generated with XQuery