Xquery returns 'Out of Main Memory'
Hi Daiane,
due to some nifty details in the XQFT spec., it is hardly possible to make "ftand" really efficent. But there is one way out: you can use the "all" keyword. Please find more details here:
http://docs.basex.org/wiki/Full-Text#Combining_Results
Hope this helps, Christian
Quoted text · DAIANE ROBERTA CANDIDA · 185 lines
On Mon, Feb 3, 2014 at 1:59 PM, DAIANE ROBERTA CANDIDA <dairc_8 at hotmail.com> wrote: > Hello, > > The query below returns 'Out of Main Memory': > > XQUERY > collection("/home/arwen/daiane_oracle/loading_inex2009/xml_pages/")//article[.//text() > contains text "Nobel" ftand "prize"] > > > I am using basexclient. > > I have a collection of xml files that have 14GB size. > -------------------------------------------------------------------- >> LIST > name Resources Size Input Path > ---------------------------------------------------------------------- > bd_inex2009 730283 16263937902 > --------------------------------------------------------------------- > > I dumped those files by the commands below: > > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles1/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles2/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles3/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles4/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles5/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles6/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles7/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles8/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles9/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles10/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles11/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles12/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles13/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles14/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles15/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles16/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles17/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles18/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles19/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles20/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles21/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles22/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles23/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles24/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles25/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles26/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles27/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles28/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles29/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles30/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles31/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles32/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles33/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles34/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles35/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles36/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles37/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles38/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles39/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles40/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles41/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles42/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles43/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles44/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles45/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles46/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles47/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles48/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles49/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles50/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles51/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles52/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles53/; > ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles54/; > > > ---------------------------------------------------------------------------------------- > > The command INFO returns: > INFO > General Information > Version: 7.7.2 > Used Memory: 79 MB > > Main Options > DBPATH: /home/arwen/BaseXData > DEBUG: false > EVENTPORT: 1985 > GLOBALLOCK: false > HOST: localhost > HTTPLOCAL: false > KEEPALIVE: 0 > LANG: English > LANGKEYS: false > LOG: true > LOGMSGMAXLEN: 1000 > NONPROXYHOSTS: > PARALLEL: 8 > PASSWORD: > PORT: 1984 > PROXYHOST: > PROXYPORT: 80 > REPOPATH: /home/arwen/BaseXRepo > RESTXQPATH: > SERVERHOST: > SERVERPORT: 1984 > STOPPORT: 8985 > TIMEOUT: 0 > USER: > WEBPATH: /home/arwen/BaseXWeb > > Options > ADDARCHIVES: true > ADDCACHE: false > ADDRAW: false > ATTRINDEX: true > AUTOFLUSH: true > BINDINGS: > CACHEQUERY: false > CASESENS: false > CATFILE: > CHECKSTRINGS: true > CHOP: true > COMPPLAN: true > CREATEFILTER: *.xml > CREATEONLY: false > DIACRITICS: false > DOTCOMPACT: false > DOTDISPLAY: true > DOTPLAN: false > DOTTY: dotty > DTD: false > EXPORTER: > FORCECREATE: false > FTINDEX: false > FTINDEXSPLITSIZE: 0 > HTMLOPT: > html=false,omit-xml-declaration=false,method=xml,nons=false,nobogons=false,nodefaults=false,nocolons=false,norestart=false,ignorable=false,emptybogons=false,any=false,norootbogons=false,nocdata=false,lexical=false,encoding=utf-8 > INDEXSPLITSIZE: 0 > INLINELIMIT: 0 > INTPARSE: true > LANGUAGE: en > LSERROR: 0 > MAINMEM: false > MAXCATS: 100 > MAXHITS: -1 > MAXLEN: 96 > MAXSTAT: 30 > PARSER: xml > PARSEROPT: > encoding=UTF-8,flat=false,header=false,jsonml=false,lines=true,separator=comma > QUERYINFO: false > QUERYPATH: > RUNS: 1 > SERIALIZE: true > SERIALIZER: > SKIPCORRUPT: false > STEMMING: false > STOPWORDS: > STRIPNS: false > TAILCALLS: 256 > TEXTINDEX: true > UPDINDEX: false > WRITEBACK: false > XMLPLAN: false > XQUERY3: true > > ----------------------------------------------------------- > > How can I run XQuery in that collection? > > Regards, > > Daiane > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >
Hello,
The query below returns 'Out of Main Memory':
XQUERY collection("/home/arwen/daiane_oracle/loading_inex2009/xml_pages/")//article[.//text() contains text "Nobel" ftand "prize"]
I am using basexclient.
I have a collection of xml files that have 14GB size. --------------------------------------------------------------------
Quoted text · 1 line
> LIST
name Resources Size Input Path ---------------------------------------------------------------------- bd_inex2009 730283 16263937902 ---------------------------------------------------------------------
I dumped those files by the commands below:
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles1/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles2/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles3/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles4/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles5/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles6/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles7/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles8/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles9/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles10/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles11/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles12/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles13/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles14/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles15/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles16/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles17/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles18/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles19/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles20/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles21/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles22/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles23/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles24/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles25/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles26/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles27/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles28/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles29/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles30/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles31/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles32/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles33/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles34/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles35/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles36/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles37/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles38/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles39/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles40/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles41/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles42/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles43/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles44/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles45/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles46/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles47/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles48/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles49/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles50/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles51/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles52/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles53/;
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles54/;----------------------------------------------------------------------------------------
The command INFO returns: INFO General Information Version: 7.7.2 Used Memory: 79 MB
Main Options
DBPATH: /home/arwen/BaseXData
DEBUG: false
EVENTPORT: 1985
GLOBALLOCK: false
HOST: localhost
HTTPLOCAL: false
KEEPALIVE: 0
LANG: English
LANGKEYS: false
LOG: true
LOGMSGMAXLEN: 1000
NONPROXYHOSTS:
PARALLEL: 8
PASSWORD:
PORT: 1984
PROXYHOST:
PROXYPORT: 80
REPOPATH: /home/arwen/BaseXRepo
RESTXQPATH:
SERVERHOST:
SERVERPORT: 1984
STOPPORT: 8985
TIMEOUT: 0
USER:
WEBPATH: /home/arwen/BaseXWeb
Options
ADDARCHIVES: true
ADDCACHE: false
ADDRAW: false
ATTRINDEX: true
AUTOFLUSH: true
BINDINGS:
CACHEQUERY: false
CASESENS: false
CATFILE:
CHECKSTRINGS: true
CHOP: true
COMPPLAN: true
CREATEFILTER: *.xml
CREATEONLY: false
DIACRITICS: false
DOTCOMPACT: false
DOTDISPLAY: true
DOTPLAN: false
DOTTY: dotty
DTD: false
EXPORTER:
FORCECREATE: false
FTINDEX: false
FTINDEXSPLITSIZE: 0
HTMLOPT: html=false,omit-xml-declaration=false,method=xml,nons=false,nobogons=false,nodefaults=false,nocolons=false,norestart=false,ignorable=false,emptybogons=false,any=false,norootbogons=false,nocdata=false,lexical=false,encoding=utf-8 INDEXSPLITSIZE: 0
INLINELIMIT: 0
INTPARSE: true
LANGUAGE: en
LSERROR: 0
MAINMEM: false
MAXCATS: 100
MAXHITS: -1
MAXLEN: 96
MAXSTAT: 30
PARSER: xml
PARSEROPT: encoding=UTF-8,flat=false,header=false,jsonml=false,lines=true,separator=comma QUERYINFO: false
QUERYPATH:
RUNS: 1
SERIALIZE: true
SERIALIZER:
SKIPCORRUPT: false
STEMMING: false
STOPWORDS:
STRIPNS: false
TAILCALLS: 256
TEXTINDEX: true
UPDINDEX: false
WRITEBACK: false
XMLPLAN: false
XQUERY3: true
-----------------------------------------------------------
How can I run XQuery in that collection?
Regards,
Daiane
Written for BaseX 7.7. BaseX may have changed since: please check the current documentation.