right way to create a db in the middle of processing?

On Thu, Nov 24, 2022 at 4:53 PM Graydon Saunders <graydonish at gmail.com> wrote: > Hello -- > > So I've got a pattern where I want to: > > 1. perform some processing using proc:execute() on a directory tree of XML > files (easy) > 2. load the result of the processing (a parallel structure tree of XML > files) into a new db > (in principle, easy; db:create() does this) > 3. extract information from the newly created db and write that to a file > (easy) > 4. use proc:execute() to run different processing on the file written in > step 3 (easy, I think) > 5. load the result (thankfully a single file) and process it with a query > (easy, I can stuff that in a function) > > Ideally, this winds up as something invoked from a single query file as a > sequence of functions because its eventual fate is to be part of an > automated test that would ideally be a "run this one thing, look at the > boolean result". > > I hang up on step 2; so far as I can tell, there isn't a way to say "go > create a database and then make it usable to these other modules that are > guaranteed not to happen until db:create() has completed" but this seems > like such a common thing to want to do that I feel like I must be missing > something. > > Thanks! > Graydon >

On Fri, Nov 25, 2022 at 05:39:45PM +0100, Johan Mörén scripsit: > Take a look at https://docs.basex.org/wiki/Commands#Command_Scripts > > To my knowledge, this is the way to do it.

On Sun, Nov 27, 2022 at 04:41:01PM +0100, Christian Grün scripsit: > Hi Graydon,

From: BaseX-Talk <basex-talk-bounces at mailman.uni-konstanz.de> on behalf of Graydon <graydonish at gmail.com> Date: Sunday, November 27, 2022 at 11:15 AM To: Christian Grün <christian.gruen at gmail.com> Cc: BaseX <BaseX-Talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] right way to create a db in the middle of processing? [External Email] On Sun, Nov 27, 2022 at 04:41:01PM +0100, Christian Grün scripsit: > Hi Graydon, Hi Christian, [customary patient careful explanation of side-effect free language design snipped] I am sorry; I didn't even do a terrible job of explaining the use case, because that was what my head was stuck in. Mea culpa! What I want is something similar to transform() but for a main query module, so I'm going to call it query(). I want this because I frequently find myself using XQuery to test XML content sets, and keeping the individual tests to an individual query module is much simpler than trying to stuff everything into an eventually enormous set of functions. But then they all have to be run individually and the specific results aggregated together somehow to get the overall result. It's possible to do this with the scripting functionality but I find that gets hard to maintain and I'd really like an XQuery layer that allows using multiple modules in a single at-least-notional query. (The quality improvement I've experienced in XSLT from using transform() and keeping every step of the transformation small and comprehensible has been large; such an improvement may not generalize well, but if it does, being able to treat a query module as a function seems highly worthwhile for implementing anything complex in XQuery.) I am not sure if my hypothetical query() function would be equivalent to fn:load-xquery-module($module-uri as xs:string, $options as map(*)) as map(*) from the 4.0 version of the functions spec at https://www.saxonica.com/qt4specs/FO/Overview-diff.html#func-load-xquery-module<https://www.saxonica.com/qt4specs/FO/Overview-diff.html#func-load-xquery-module> My idea would be fn:query($module-uri as xs:string, $context as item()*, $options as map(*)) as map(*) where a db is explicitly available as a value for $context. (I realize that document-node()* is not the type of a BaseX db, but given the optimizer will substitute db:get() for collection() I figure it's close...) So it would be possible to do: let $contentSet as xs:string := fn:query('load-set.xq', (), $options)?db-name let $test1Result as map(*) := fn:query('test1.xq', $contentSet, $test-options) let $test2Result as map(*) := fn:query('test2.xq', $contentSet, $test-options) And so on. I emphatically don't want to be able to update a DB and then query it further in a single query module; I am sharply aware I'm not smart enough not to mess that up! Does that make any more sense? Thanks! -- Graydon Saunders | graydonish at gmail.com Þæs oferéode, ðisses swá mæg. -- Deor ("That passed, so may this.")

> What I want is something similar to transform() but for a main query module,

> fn:load-xquery-module($module-uri as xs:string, $options as map(*)) as

> I want this because I > frequently find myself using XQuery to test XML content sets, and > keeping the individual tests to an individual query module is much > simpler than trying to stuff everything into an eventually enormous set > of functions. But then they all have to be run individually and the > specific results aggregated together somehow to get the overall result. > > It's possible to do this with the scripting functionality but I find > that gets hard to maintain and I'd really like an XQuery layer that > allows using multiple modules in a single at-least-notional query. > > (The quality improvement I've experienced in XSLT from using transform() > and keeping every step of the transformation small and comprehensible > has been large; such an improvement may not generalize well, but if it > does, being able to treat a query module as a function seems highly > worthwhile for implementing anything complex in XQuery.) > > I am not sure if my hypothetical query() function would be equivalent to > > fn:load-xquery-module($module-uri as xs:string, $options as map(*)) as > map(*) > > from the 4.0 version of the functions spec at > https://www.saxonica.com/qt4specs/FO/Overview-diff.html#func-load-xquery-module > > My idea would be > > fn:query($module-uri as xs:string, $context as item()*, $options as > map(*)) as map(*) > > where a db is explicitly available as a value for $context. (I realize > that document-node()* is not the type of a BaseX db, but given the > optimizer will substitute db:get() for collection() I figure it's > close...) > > So it would be possible to do: > > let $contentSet as xs:string := > fn:query('load-set.xq', (), $options)?db-name > > let $test1Result as map(*) := > fn:query('test1.xq', $contentSet, $test-options) > > let $test2Result as map(*) := > fn:query('test2.xq', $contentSet, $test-options) > > And so on. > > I emphatically don't want to be able to update a DB and then query it > further in a single query module; I am sharply aware I'm not smart > enough not to mess that up! > > Does that make any more sense? > > Thanks! > > > -- > Graydon Saunders | graydonish at gmail.com > Þæs oferéode, ðisses swá mæg. > -- Deor ("That passed, so may this.") >

On Mon, Nov 28, 2022 at 11:13:39AM +0100, Christian Grün scripsit: > Hi Graydon,

> > What I want is something similar to transform() but for a main query module, > > so I'm going to call it query(). > > Did you have a look at xquery:eval? [1] It allows you to supply a query as > string or URI and pass on a context or variables:

> > fn:load-xquery-module($module-uri as xs:string, $options as map(*)) as > map(*) > > The function is not available in BaseX, but inspect:functions works > similarly [2]: The functions of the module at the specified URI will be > returned as a sequence (back then when the function was added, maps were > not available yet).

> If you have read-only and updating queries, Eliot’s approach of chaining > queries is definitely worth looking at.

On Sun, Nov 27, 2022 at 08:58:59PM +0000, Eliot Kimber scripsit: > I implemented the “BaseX Orchestration” project for exactly this > purpose: Enable doing operations that involve the creation or update > of databases in series from a single driving script.