running server commands from a script?
Friday, 2 December 2011 · 2 messages · last reply 3 Dec 2011
CS
C. M. Sperberg-McQueen2 Dec 2011 22:33 #
From time to time, I find myself wanting to run BaseX server commands from a shell script. For example, if I have two or three documents I'd like to add to a database, it's easy enough to do so from the GUI interface, or from the interactive server prompt. But if it's five or ten documents (and even more so if it's a few hundred or a few thousand documents), entering the commands interactively can become tedious and error prone. When the files are available on the local file system, I can use wildcards and I can add a directory at a time. But when the files are on a remote server and identified by URI instead of file name, I guess that won't work.
What I think I'm looking for is the ability to specify a sequence of BaseX server commands and have it run that sequence of commands -- or some other way of being able to specify a list of filenames or URIs and have them all added to a given database.
For example
CREATE DATABASE demo
OPEN demo
ADD http://example.com/docs/a.xml
ADD http://example.com/docs/b.xml
ADD http://example.com/docs/c.xml
ADD http://example.com/docs/d.xml
ADD http://example.com/docs/e.xml
...
ADD http://example.com/docs/z.xml
RENAME http://example.com/docs/ /
# I'd use ADD to a.xml http://example.com/docs/a.xml, but
# that doesn't seem to have the effect I expect. But
# RENAME works for what I need
There is certainly a lot of BaseX documentation I have not read, but I have not found any mention of this kind of thing. Have I overlooked it?
One technique that I guess would work would be to exploit the new REST interface (very nice, by the way!) and write a bash script that went something like this:
curl http://localhost:8984/rest?command=CREATE+DATABASE+demo
curl http://localhost:8984/rest/demo?command=ADD+http://example.com/docs/a.xml
curl http://localhost:8984/rest/demo?command=ADD+http://example.com/docs/b.xml
curl http://localhost:8984/rest/demo?command=ADD+http://example.com/docs/c.xml
curl http://localhost:8984/rest/demo?command=ADD+http://example.com/docs/d.xml
curl http://localhost:8984/rest/demo?command=ADD+http://example.com/docs/e.xml
...
curl http://localhost:8984/rest/demo?command=ADD+http://example.com/docs/z.xml
curl http://localhost:8984/rest/demo?command=RENAME+http://example.com/docs/+/or rather
CURL="curl --user X:secretpasswd"
$CURL http://localhost:8984/rest?command=CREATE+DATABASE+demo
etc.But if there is s nicer way to handle this case, I'd be glad to learn about it.
Perhaps this is when I should buckle down and learn how to do this kind of thing with XQuery Update?
Thanks!
Michael Sperberg-McQueen
--
****************************************************************
* C. M. Sperberg-McQueen, Black Mesa Technologies LLC
* http://www.blackmesatech.com
* http://cmsmcq.com/mib
* http://balisage.net
****************************************************************
CG
Christian Grün3 Dec 2011 04:02 #
Quoted text · C. M. Sperberg-McQueen · 7 lines
> From time to time, I find myself wanting to run BaseX server commands > from a shell script. For example, if I have two or three documents > I'd like to add to a database, it's easy enough to do so from the GUI > interface, or from the interactive server prompt. But if it's five or > ten documents (and even more so if it's a few hundred or a few > thousand documents), entering the commands interactively can become > tedious and error prone. [...]
Here are two approaches that might do what you need:
a) you can separate multiple commands by semi-colons. an example:
basex -c "create db test <a/>; info db; drop db test"
b) commands can also be copied to an extra file and then redirected to basex:
basex < basex-commands.txt
Quoted text · C. M. Sperberg-McQueen · 2 lines
> # I'd use ADD to a.xml http://example.com/docs/a.xml, but > # that doesn't seem to have the effect I expect.
This should work as well (at least with the latest version I'm having here):
basex -c "create db db; add to 123.xml
http://files.basex.org/BaseXPADFile.xml; list db"
Input Path Type Content-Type Size
---------------------------------------
123.xml xml application/xml 245
..what happens in your case?
Quoted text · C. M. Sperberg-McQueen · 2 lines
> Perhaps this is when I should buckle down and learn how to do this kind of thing > with XQuery Update?
;) Unfortunately, that's not possible with XQuery Update anyway, as there's no way to add new documents to a collection (but for some reason it's actually legal to add several root elements to a document node). As a matter of fact, we are internally using exactly the same update operations for adding documents to a database, which can be called from the database commands and our XQuery database module [1]. This is a small example for adding 10,000 dummy documents to a database 'DB':
for $i in 1 to 10000
let $doc := document { <xml>{ $i }</xml> }
return db:add('DB', $doc, concat($i, '.xml'))Feel free to ask for more,
Christian
[1] http://docs.basex.org/wiki/Database_Module
Written for BaseX 7.0. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery