Xquery returns 'Out of Main Memory'
Monday, 3 February 2014 · 2 messages
CG
Hi Daiane,
due to some nifty details in the XQFT spec., it is hardly possible to make "ftand" really efficent. But there is one way out: you can use the "all" keyword. Please find more details here:
http://docs.basex.org/wiki/Full-Text#Combining_Results
Hope this helps,
Christian
Quoted text · DAIANE ROBERTA CANDIDA · 185 lines
On Mon, Feb 3, 2014 at 1:59 PM, DAIANE ROBERTA CANDIDA <dairc_8 at hotmail.com> wrote: > Hello, > > The query below returns 'Out of Main Memory': > > XQUERY > collection("/home/arwen/daiane_oracle/loading_inex2009/xml_pages/")//article[.//text() > contains text "Nobel" ftand "prize"] > > > I am using basexclient. > > I have a collection of xml files that have 14GB size. > -------------------------------------------------------------------- >> LIST > name Resources Size Input Path > ---------------------------------------------------------------------- > bd_inex2009 730283 16263937902 > --------------------------------------------------------------------- > > I dumped those files by the commands below: > > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles1/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles2/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles3/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles4/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles5/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles6/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles7/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles8/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles9/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles10/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles11/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles12/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles13/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles14/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles15/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles16/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles17/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles18/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles19/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles20/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles21/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles22/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles23/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles24/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles25/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles26/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles27/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles28/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles29/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles30/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles31/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles32/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles33/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles34/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles35/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles36/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles37/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles38/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles39/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles40/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles41/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles42/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles43/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles44/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles45/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles46/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles47/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles48/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles49/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles50/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles51/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles52/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles53/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles54/; > > > ---------------------------------------------------------------------------------------- > > The command INFO returns: > INFO > General Information > Version: 7.7.2 > Used Memory: 79 MB > > Main Options > DBPATH: /home/arwen/BaseXData > DEBUG: false > EVENTPORT: 1985 > GLOBALLOCK: false > HOST: localhost > HTTPLOCAL: false > KEEPALIVE: 0 > LANG: English > LANGKEYS: false > LOG: true > LOGMSGMAXLEN: 1000 > NONPROXYHOSTS: > PARALLEL: 8 > PASSWORD: > PORT: 1984 > PROXYHOST: > PROXYPORT: 80 > REPOPATH: /home/arwen/BaseXRepo > RESTXQPATH: > SERVERHOST: > SERVERPORT: 1984 > STOPPORT: 8985 > TIMEOUT: 0 > USER: > WEBPATH: /home/arwen/BaseXWeb > > Options > ADDARCHIVES: true > ADDCACHE: false > ADDRAW: false > ATTRINDEX: true > AUTOFLUSH: true > BINDINGS: > CACHEQUERY: false > CASESENS: false > CATFILE: > CHECKSTRINGS: true > CHOP: true > COMPPLAN: true > CREATEFILTER: *.xml > CREATEONLY: false > DIACRITICS: false > DOTCOMPACT: false > DOTDISPLAY: true > DOTPLAN: false > DOTTY: dotty > DTD: false > EXPORTER: > FORCECREATE: false > FTINDEX: false > FTINDEXSPLITSIZE: 0 > HTMLOPT: > html=false,omit-xml-declaration=false,method=xml,nons=false,nobogons=false,nodefaults=false,nocolons=false,norestart=false,ignorable=false,emptybogons=false,any=false,norootbogons=false,nocdata=false,lexical=false,encoding=utf-8 > INDEXSPLITSIZE: 0 > INLINELIMIT: 0 > INTPARSE: true > LANGUAGE: en > LSERROR: 0 > MAINMEM: false > MAXCATS: 100 > MAXHITS: -1 > MAXLEN: 96 > MAXSTAT: 30 > PARSER: xml > PARSEROPT: > encoding=UTF-8,flat=false,header=false,jsonml=false,lines=true,separator=comma > QUERYINFO: false > QUERYPATH: > RUNS: 1 > SERIALIZE: true > SERIALIZER: > SKIPCORRUPT: false > STEMMING: false > STOPWORDS: > STRIPNS: false > TAILCALLS: 256 > TEXTINDEX: true > UPDINDEX: false > WRITEBACK: false > XMLPLAN: false > XQUERY3: true > > ----------------------------------------------------------- > > How can I run XQuery in that collection? > > Regards, > > Daiane > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >
DC
DAIANE ROBERTA CANDIDA3 Feb 2014 14:59 #
Hello,
The query below returns 'Out of Main Memory':
XQUERY collection("/home/arwen/daiane_oracle/loading_inex2009/xml_pages/")//article[.//text() contains text "Nobel" ftand "prize"]
I am using basexclient.
I have a collection of xml files that have 14GB size.
--------------------------------------------------------------------
Quoted text · 1 line
> LIST
name Resources Size Input Path
---------------------------------------------------------------------- bd_inex2009 730283 16263937902
---------------------------------------------------------------------
I dumped those files by the commands below:
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles1/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles2/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles3/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles4/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles5/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles6/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles7/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles8/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles9/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles10/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles11/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles12/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles13/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles14/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles15/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles16/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles17/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles18/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles19/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles20/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles21/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles22/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles23/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles24/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles25/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles26/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles27/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles28/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles29/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles30/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles31/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles32/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles33/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles34/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles35/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles36/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles37/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles38/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles39/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles40/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles41/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles42/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles43/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles44/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles45/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles46/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles47/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles48/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles49/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles50/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles51/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles52/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles53/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles54/;----------------------------------------------------------------------------------------
The command INFO returns:
INFO
General Information
Version: 7.7.2
Used Memory: 79 MB
Main Options
DBPATH: /home/arwen/BaseXData
DEBUG: false
EVENTPORT: 1985
GLOBALLOCK: false
HOST: localhost
HTTPLOCAL: false
KEEPALIVE: 0
LANG: English
LANGKEYS: false
LOG: true
LOGMSGMAXLEN: 1000
NONPROXYHOSTS:
PARALLEL: 8
PASSWORD:
PORT: 1984
PROXYHOST:
PROXYPORT: 80
REPOPATH: /home/arwen/BaseXRepo
RESTXQPATH:
SERVERHOST:
SERVERPORT: 1984
STOPPORT: 8985
TIMEOUT: 0
USER:
WEBPATH: /home/arwen/BaseXWeb
Options
ADDARCHIVES: true
ADDCACHE: false
ADDRAW: false
ATTRINDEX: true
AUTOFLUSH: true
BINDINGS:
CACHEQUERY: false
CASESENS: false
CATFILE:
CHECKSTRINGS: true
CHOP: true
COMPPLAN: true
CREATEFILTER: *.xml
CREATEONLY: false
DIACRITICS: false
DOTCOMPACT: false
DOTDISPLAY: true
DOTPLAN: false
DOTTY: dotty
DTD: false
EXPORTER:
FORCECREATE: false
FTINDEX: false
FTINDEXSPLITSIZE: 0
HTMLOPT: html=false,omit-xml-declaration=false,method=xml,nons=false,nobogons=false,nodefaults=false,nocolons=false,norestart=false,ignorable=false,emptybogons=false,any=false,norootbogons=false,nocdata=false,lexical=false,encoding=utf-8 INDEXSPLITSIZE: 0
INLINELIMIT: 0
INTPARSE: true
LANGUAGE: en
LSERROR: 0
MAINMEM: false
MAXCATS: 100
MAXHITS: -1
MAXLEN: 96
MAXSTAT: 30
PARSER: xml
PARSEROPT: encoding=UTF-8,flat=false,header=false,jsonml=false,lines=true,separator=comma QUERYINFO: false
QUERYPATH:
RUNS: 1
SERIALIZE: true
SERIALIZER:
SKIPCORRUPT: false
STEMMING: false
STOPWORDS:
STRIPNS: false
TAILCALLS: 256
TEXTINDEX: true
UPDINDEX: false
WRITEBACK: false
XMLPLAN: false
XQUERY3: true
-----------------------------------------------------------
How can I run XQuery in that collection?
Regards,
DaianeWritten for BaseX 7.7. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery