Load xml files to BaseX in a loop
Sunday, 10 February 2013 · 2 messages
NS
Nicholas OR Rita Stevens10 Feb 2013 01:59 #
I am new at BaseX so please pardon the newbie question.
I am trying to run an xquery script to load a set of xml files in a directory into to BaseX 7.5.
If I run replace.xq, shown below, it fails with Out of Main Memory
declare %updating function local:upd($path,$file) {
db:replace("eams",$path,$file)
};
let $files:=for $file in file:list("d:\xmlfiles") return
"d:\xmlfiles\"||$file
for $file in $files return local:upd("pd",$file)But if I run file replace.bxs, it updates with no problem
<commands>
<replace path="p1">d:\xmlfiles\drr1.xml</replace>
<replace path="p1">d:\xmlfiles\drr2.xml</replace>
<replace path="p1">d:\xmlfiles\drr3.xml</replace>
<replace path="p1">d:\xmlfiles\dw1.xml</replace>
<replace path="p1">d:\xmlfiles\dlp1.xml</replace>
</commands>(The same set of files is being processed by both scripts; I checked)
The only idea I have is to use xquery to generate the file replace.bxs and then run it.
A better way?
Thanks.
CG
Christian Grün10 Feb 2013 23:38 #
Hi Nicholas,
Quoted text · Nicholas OR Rita Stevens · 1 line
> If I run replace.xq, shown below, it fails with Out of Main Memory
as you already experienced in your tests, the replace command takes less memory than the db:replace() function, as the second one will first cache all update operations on the Pending Update List [1] before actually executing them.
Quoted text · Nicholas OR Rita Stevens · 2 lines
> The only idea I have is to use xquery to generate the file replace.bxs and > then run it.
This is indeed a popular approach. If all of your files are stored in a single directory, you could as well try to delete your documents and add all files via a single
db:add call: db:delete("database"),
db:add("database", "/path/to/dir")Hope this helps,
Christian
[1] http://docs.basex.org/wiki/XQuery_Update#Pending_Update_List
Written for BaseX 7.6. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery