Reading JSON

> I have noticed different speeds when running the following functions (from slowest to fastest): > > parse-json(unparsed-text('example.txt')) > json-doc("example.txt") > parse-json(file:read-text('example.txt')) > > similarly for documents on the web: > > parse-doc('http://example.com/text') > parse-json(fetch:text('http://example.com/text')) > > Does this make sense to you? Is there any recommendation to follow? Despite the runtime speed difference, I admit I love writing one single function (json-doc) to get map conversion. Everything is so immediate :) > > Best, > Giuseppe > > > > > > Universität Leipzig > Institute of Computer Science, Digital Humanities > Augustusplatz 10 > 04109 Leipzig > Deutschland > E-mail: celano at informatik.uni-leipzig.de > E-mail: giuseppegacelano at gmail.com > Web site 1: http://www.dh.uni-leipzig.de/wo/team/ > Web site 2: https://sites.google.com/site/giuseppegacelano/ >

> On Aug 14, 2017, at 5:55 PM, Christian Grün <christian.gruen at gmail.com> wrote: > > Hi Giuseppe, > > The semantics of the used functions are slightly different, but I > guess the differences in terms of performance should be rather > marginal. Just in case: Have you have done some in-depth comparisons > that I could have a look at? Did you work with large input files, or > did you run the queries repeatedly and look at the average times? > > Cheers, > Christian > > > >> I have noticed different speeds when running the following functions (from slowest to fastest): >> >> parse-json(unparsed-text('example.txt')) >> json-doc("example.txt") >> parse-json(file:read-text('example.txt')) >> >> similarly for documents on the web: >> >> parse-doc('http://example.com/text') >> parse-json(fetch:text('http://example.com/text')) >> >> Does this make sense to you? Is there any recommendation to follow? Despite the runtime speed difference, I admit I love writing one single function (json-doc) to get map conversion. Everything is so immediate :) >> >> Best, >> Giuseppe >> >> >> >> >> >> Universität Leipzig >> Institute of Computer Science, Digital Humanities >> Augustusplatz 10 >> 04109 Leipzig >> Deutschland >> E-mail: celano at informatik.uni-leipzig.de >> E-mail: giuseppegacelano at gmail.com >> Web site 1: http://www.dh.uni-leipzig.de/wo/team/ >> Web site 2: https://sites.google.com/site/giuseppegacelano/ >> >

On Mon, Aug 14, 2017 at 6:18 PM, Giuseppe Celano <celano at informatik.uni-leipzig.de> wrote: > Hi Christian, > > The latter option. I just opened a file and run the same query repeatedly. It is not an in-depth comparison at all, but the times shown in the Query Info were clearly different (even if just ms). > > Best, > Giuseppe > > > > Universität Leipzig > Institute of Computer Science, Digital Humanities > Augustusplatz 10 > 04109 Leipzig > Deutschland > E-mail: celano at informatik.uni-leipzig.de > E-mail: giuseppegacelano at gmail.com > Web site 1: http://www.dh.uni-leipzig.de/wo/team/ > Web site 2: https://sites.google.com/site/giuseppegacelano/ > >> On Aug 14, 2017, at 5:55 PM, Christian Grün <christian.gruen at gmail.com> wrote: >> >> Hi Giuseppe, >> >> The semantics of the used functions are slightly different, but I >> guess the differences in terms of performance should be rather >> marginal. Just in case: Have you have done some in-depth comparisons >> that I could have a look at? Did you work with large input files, or >> did you run the queries repeatedly and look at the average times? >> >> Cheers, >> Christian >> >> >> >>> I have noticed different speeds when running the following functions (from slowest to fastest): >>> >>> parse-json(unparsed-text('example.txt')) >>> json-doc("example.txt") >>> parse-json(file:read-text('example.txt')) >>> >>> similarly for documents on the web: >>> >>> parse-doc('http://example.com/text') >>> parse-json(fetch:text('http://example.com/text')) >>> >>> Does this make sense to you? Is there any recommendation to follow? Despite the runtime speed difference, I admit I love writing one single function (json-doc) to get map conversion. Everything is so immediate :) >>> >>> Best, >>> Giuseppe >>> >>> >>> >>> >>> >>> Universität Leipzig >>> Institute of Computer Science, Digital Humanities >>> Augustusplatz 10 >>> 04109 Leipzig >>> Deutschland >>> E-mail: celano at informatik.uni-leipzig.de >>> E-mail: giuseppegacelano at gmail.com >>> Web site 1: http://www.dh.uni-leipzig.de/wo/team/ >>> Web site 2: https://sites.google.com/site/giuseppegacelano/ >>> >> >

> On Aug 15, 2017, at 1:28 AM, Christian Grün <christian.gruen at gmail.com> wrote: > > Hi Giuseppe, > > I did some tests on command-line: > > basex -v -z -r100 "parse-json(unparsed-text('example.json'))" > basex -v -z -r100 "parse-json(file:read-text('example.json'))" > basex -v -z -r100 "json-doc('example.json')" > > I tested the calls with a small and a large file (10 KB, 1.5 MB), and > evaluation times were very similar, so I guess I need some more input > to reproduce your results. > > Best, > Christian > > > > On Mon, Aug 14, 2017 at 6:18 PM, Giuseppe Celano > <celano at informatik.uni-leipzig.de> wrote: >> Hi Christian, >> >> The latter option. I just opened a file and run the same query repeatedly. It is not an in-depth comparison at all, but the times shown in the Query Info were clearly different (even if just ms). >> >> Best, >> Giuseppe >> >> >> >> Universität Leipzig >> Institute of Computer Science, Digital Humanities >> Augustusplatz 10 >> 04109 Leipzig >> Deutschland >> E-mail: celano at informatik.uni-leipzig.de >> E-mail: giuseppegacelano at gmail.com >> Web site 1: http://www.dh.uni-leipzig.de/wo/team/ >> Web site 2: https://sites.google.com/site/giuseppegacelano/ >> >>> On Aug 14, 2017, at 5:55 PM, Christian Grün <christian.gruen at gmail.com> wrote: >>> >>> Hi Giuseppe, >>> >>> The semantics of the used functions are slightly different, but I >>> guess the differences in terms of performance should be rather >>> marginal. Just in case: Have you have done some in-depth comparisons >>> that I could have a look at? Did you work with large input files, or >>> did you run the queries repeatedly and look at the average times? >>> >>> Cheers, >>> Christian >>> >>> >>> >>>> I have noticed different speeds when running the following functions (from slowest to fastest): >>>> >>>> parse-json(unparsed-text('example.txt')) >>>> json-doc("example.txt") >>>> parse-json(file:read-text('example.txt')) >>>> >>>> similarly for documents on the web: >>>> >>>> parse-doc('http://example.com/text') >>>> parse-json(fetch:text('http://example.com/text')) >>>> >>>> Does this make sense to you? Is there any recommendation to follow? Despite the runtime speed difference, I admit I love writing one single function (json-doc) to get map conversion. Everything is so immediate :) >>>> >>>> Best, >>>> Giuseppe >>>> >>>> >>>> >>>> >>>> >>>> Universität Leipzig >>>> Institute of Computer Science, Digital Humanities >>>> Augustusplatz 10 >>>> 04109 Leipzig >>>> Deutschland >>>> E-mail: celano at informatik.uni-leipzig.de >>>> E-mail: giuseppegacelano at gmail.com >>>> Web site 1: http://www.dh.uni-leipzig.de/wo/team/ >>>> Web site 2: https://sites.google.com/site/giuseppegacelano/ >>>> >>> >> >

On Tue, Aug 15, 2017 at 11:17 AM, Giuseppe Celano <celano at informatik.uni-leipzig.de> wrote: > Hi Christian, > > I confirm that the command you use always outputs very similar time results, even when you manually repeat it for 3 or 4 times, i.e., running it without using "-r". However, if you run (manually) for a few times each of the queries from the GUI, I get differences (total time from the Query Info Panel, for a 333 KB file): > > parse-json(file:read-text('text.txt') = around 40 ms > json-doc('text.txt') = around 80 ms > parse-json(unparsed-text('text.txt') = around 120 ms > > Looking at the values reported in the Query Info Panel, the "compiling time" seems to be responsible for that. > > Best, > Giuseppe > > Universität Leipzig > Institute of Computer Science, Digital Humanities > Augustusplatz 10 > 04109 Leipzig > Deutschland > E-mail: celano at informatik.uni-leipzig.de > E-mail: giuseppegacelano at gmail.com > Web site 1: http://www.dh.uni-leipzig.de/wo/team/ > Web site 2: https://sites.google.com/site/giuseppegacelano/ > >> On Aug 15, 2017, at 1:28 AM, Christian Grün <christian.gruen at gmail.com> wrote: >> >> Hi Giuseppe, >> >> I did some tests on command-line: >> >> basex -v -z -r100 "parse-json(unparsed-text('example.json'))" >> basex -v -z -r100 "parse-json(file:read-text('example.json'))" >> basex -v -z -r100 "json-doc('example.json')" >> >> I tested the calls with a small and a large file (10 KB, 1.5 MB), and >> evaluation times were very similar, so I guess I need some more input >> to reproduce your results. >> >> Best, >> Christian >> >> >> >> On Mon, Aug 14, 2017 at 6:18 PM, Giuseppe Celano >> <celano at informatik.uni-leipzig.de> wrote: >>> Hi Christian, >>> >>> The latter option. I just opened a file and run the same query repeatedly. It is not an in-depth comparison at all, but the times shown in the Query Info were clearly different (even if just ms). >>> >>> Best, >>> Giuseppe >>> >>> >>> >>> Universität Leipzig >>> Institute of Computer Science, Digital Humanities >>> Augustusplatz 10 >>> 04109 Leipzig >>> Deutschland >>> E-mail: celano at informatik.uni-leipzig.de >>> E-mail: giuseppegacelano at gmail.com >>> Web site 1: http://www.dh.uni-leipzig.de/wo/team/ >>> Web site 2: https://sites.google.com/site/giuseppegacelano/ >>> >>>> On Aug 14, 2017, at 5:55 PM, Christian Grün <christian.gruen at gmail.com> wrote: >>>> >>>> Hi Giuseppe, >>>> >>>> The semantics of the used functions are slightly different, but I >>>> guess the differences in terms of performance should be rather >>>> marginal. Just in case: Have you have done some in-depth comparisons >>>> that I could have a look at? Did you work with large input files, or >>>> did you run the queries repeatedly and look at the average times? >>>> >>>> Cheers, >>>> Christian >>>> >>>> >>>> >>>>> I have noticed different speeds when running the following functions (from slowest to fastest): >>>>> >>>>> parse-json(unparsed-text('example.txt')) >>>>> json-doc("example.txt") >>>>> parse-json(file:read-text('example.txt')) >>>>> >>>>> similarly for documents on the web: >>>>> >>>>> parse-doc('http://example.com/text') >>>>> parse-json(fetch:text('http://example.com/text')) >>>>> >>>>> Does this make sense to you? Is there any recommendation to follow? Despite the runtime speed difference, I admit I love writing one single function (json-doc) to get map conversion. Everything is so immediate :) >>>>> >>>>> Best, >>>>> Giuseppe >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Universität Leipzig >>>>> Institute of Computer Science, Digital Humanities >>>>> Augustusplatz 10 >>>>> 04109 Leipzig >>>>> Deutschland >>>>> E-mail: celano at informatik.uni-leipzig.de >>>>> E-mail: giuseppegacelano at gmail.com >>>>> Web site 1: http://www.dh.uni-leipzig.de/wo/team/ >>>>> Web site 2: https://sites.google.com/site/giuseppegacelano/ >>>>> >>>> >>> >> >

On Mon, Aug 14, 2017 at 3:01 PM, Giuseppe Celano <celano at informatik.uni-leipzig.de> wrote: > Hi, > > I have noticed different speeds when running the following functions (from slowest to fastest): > > parse-json(unparsed-text('example.txt')) > json-doc("example.txt") > parse-json(file:read-text('example.txt')) > > similarly for documents on the web: > > parse-doc('http://example.com/text') > parse-json(fetch:text('http://example.com/text')) > > Does this make sense to you? Is there any recommendation to follow? Despite the runtime speed difference, I admit I love writing one single function (json-doc) to get map conversion. Everything is so immediate :) > > Best, > Giuseppe > > > > > > Universität Leipzig > Institute of Computer Science, Digital Humanities > Augustusplatz 10 > 04109 Leipzig > Deutschland > E-mail: celano at informatik.uni-leipzig.de > E-mail: giuseppegacelano at gmail.com > Web site 1: http://www.dh.uni-leipzig.de/wo/team/ > Web site 2: https://sites.google.com/site/giuseppegacelano/ >