BaseX xpath querying takes more time than SAX parser takes
Wednesday, 20 February 2013 · 4 messages · last reply 22 Feb 2013
EM
elangovan MuthuSwamy20 Feb 2013 11:56 #
Hi All,
I am trying to use BaseX Client and server with large XML files(13 MB size). I have created a database for the XML and evaluated an xpath in that. When I executed an xpath it takes around 250 milliseconds to fetch the results. In case of SAX parser I could get the response for the same xpath within 5 seconds.
I used following BaseXClient code for this.
BaseXClient session = new BaseXClient("localhost", 1984, "admin", "admin");
// code to fetch the large XML file size. Read from the file. String xmlString = .......
InputStream bais = new ByteArrayInputStream(xmlString.getBytes());
session.create("database", bais);String expression = "/configuration/protocol[starts-with(name, 'http')]/name";
*session.query(expression).execute();* // this line takes 250 milliseconds to give the result. Where as same xpath can be executed within 5 milliseconds in SAX parser.
Am I basically missing anything or something else.
Please advice me.
Thanks in advance.
--
Regards,
Elango.
CG
Christian Grün22 Feb 2013 11:21 #
Hi Elango,
to say anything on this, we probably need more information on how you are processing your data via SAX, and how you receive your result via session.query(expression).execute(). Do you have some self-contained code that allows us to get more insight into how you are testing?
Best,
Christian
Quoted text · elangovan MuthuSwamy · 37 lines
On Wed, Feb 20, 2013 at 11:56 AM, elangovan MuthuSwamy <elangovan.h at gmail.com> wrote: > Hi All, > > I am trying to use BaseX Client and server with large XML files(13 MB size). > I have created a database for the XML and evaluated an xpath in that. When I > executed an xpath it takes around 250 milliseconds to fetch the results. In > case of SAX parser I could get the response for the same xpath within 5 > seconds. > > I used following BaseXClient code for this. > > BaseXClient session = new BaseXClient("localhost", 1984, "admin", "admin"); > > // code to fetch the large XML file size. Read from the file. > String xmlString = ....... > > InputStream bais = new ByteArrayInputStream(xmlString.getBytes()); > session.create("database", bais); > > String expression = "/configuration/protocol[starts-with(name, > 'http')]/name"; > > session.query(expression).execute(); // this line takes 250 milliseconds to > give the result. Where as same xpath can be executed within 5 milliseconds > in SAX parser. > > Am I basically missing anything or something else. > > > Please advice me. > > Thanks in advance. > > -- > Regards, > Elango.
CG
Christian Grün22 Feb 2013 13:08 #
…glad to hear it works.
Quoted text · Christian Grün · 74 lines
> Thanks Christian for your response, > > I have done a mistake in my code. I have executed different xpaths against > both causes the time difference. Now I have executed same xpath against both > and BaseX xpath evaluation is faster than SAX. > > Regards, > Elango. > > > On Fri, Feb 22, 2013 at 3:51 PM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> Hi Elango, >> >> to say anything on this, we probably need more information on how you >> are processing your data via SAX, and how you receive your result via >> session.query(expression).execute(). Do you have some self-contained >> code that allows us to get more insight into how you are testing? >> >> Best, >> Christian >> >> >> On Wed, Feb 20, 2013 at 11:56 AM, elangovan MuthuSwamy >> <elangovan.h at gmail.com> wrote: >> > Hi All, >> > >> > I am trying to use BaseX Client and server with large XML files(13 MB >> > size). >> > I have created a database for the XML and evaluated an xpath in that. >> > When I >> > executed an xpath it takes around 250 milliseconds to fetch the results. >> > In >> > case of SAX parser I could get the response for the same xpath within 5 >> > seconds. >> > >> > I used following BaseXClient code for this. >> > >> > BaseXClient session = new BaseXClient("localhost", 1984, "admin", >> > "admin"); >> > >> > // code to fetch the large XML file size. Read from the file. >> > String xmlString = ....... >> > >> > InputStream bais = new ByteArrayInputStream(xmlString.getBytes()); >> > session.create("database", bais); >> > >> > String expression = "/configuration/protocol[starts-with(name, >> > 'http')]/name"; >> > >> > session.query(expression).execute(); // this line takes 250 milliseconds >> > to >> > give the result. Where as same xpath can be executed within 5 >> > milliseconds >> > in SAX parser. >> > >> > Am I basically missing anything or something else. >> > >> > >> > Please advice me. >> > >> > Thanks in advance. >> > >> > -- >> > Regards, >> > Elango. > > > > > -- > Regards, > Elango.
JR
Jan Vlčinský (TamTam Research)in reply to elangovan MuthuSwamy, 20 Feb 2013, 11:5622 Feb 2013 16:41 #
Hi Elango
I noticed, your code runs faster after you have corrected evaluated XPath, but anyway, one observation from my practice.
In case, you need only one shot processing of the file, and your task fits for use of SAX parser, then the SAX parser will be the most efficient and the fastest.
BaseX will perform much better, if your queries are complex in structure, as it will use very efficient indices.
But if your task is of kind "run through the xml and pick up some values here and there, sum them up and have the result", then SAX is really difficult to beat, because that is what it by design does.
In case, you have to process the same file few times, then XQuery may complete, this depends on real case and scenario.
With best regards
Jan
*Jan Vlčinský*
TamTam Research s.r.o.
Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic
tel: +420-597 602 024; mob: +420-608 979 040
skype: janvlcinsky; GoogleTalk: jan.vlcinsky at gmail.com
http://cz.linkedin.com/in/vlcinsky
2013/2/20 elangovan MuthuSwamy <elangovan.h at gmail.com>
Quoted text · elangovan MuthuSwamy · 42 lines
> Hi All, > > I am trying to use BaseX Client and server with large XML files(13 MB > size). I have created a database for the XML and evaluated an xpath in > that. When I executed an xpath it takes around 250 milliseconds to fetch > the results. In case of SAX parser I could get the response for the same > xpath within 5 seconds. > > I used following BaseXClient code for this. > > BaseXClient session = new BaseXClient("localhost", 1984, "admin", "admin"); > > // code to fetch the large XML file size. Read from the file. > String xmlString = ....... > > InputStream bais = new ByteArrayInputStream(xmlString.getBytes()); > session.create("database", bais); > > String expression = "/configuration/protocol[starts-with(name, > 'http')]/name"; > > *session.query(expression).execute();* // this line takes 250 > milliseconds to give the result. Where as same xpath can be executed within > 5 milliseconds in SAX parser. > > Am I basically missing anything or something else. > > > Please advice me. > > Thanks in advance. > > -- > Regards, > Elango. > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >
Written for BaseX 7.6. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery