Main Page » Mailing List » September 2026

Java code in XQuery?

Wednesday, 16 September 2026 · 7 messages · last reply 18 Sep 2026
JC
Jan Ctibor16 Sep 2026 13:13 #
Hi everybody, I have a question for both the BaseX team and everybody else: what is the good/best vs. bad/worst practice when integrating Java code to BaseX (XQuery)? XQuery has some limitations and peculiarities - most famosly the invariable variables. I think I am not the only one, who initilly, after comming to XQuery from the realm of "standard" programming languages, struggled with this pecculiarity. And this as well as some other features of XQuery still makes some things imposible or very difficult to achieve with pure XQuery. But then there's the possibility to include Java classes in the XQuery project. And those Java classes are by default non-deterministic. With only minimal effort, this can provide those "features" missing in XQuery - like true, changable variables accessible from any module of the query being executed - in Java, in this specific case, its can be a simple class with some dictionary or map variable and setter/getter functions, litteraly a few lines of code (in the most simple form, of course). It seems to work well, as far as I have tested it, but here is my question: Is this a good idea? It feels a bit like "cheating" on the XQuery. Is it something you would rather discourage, or is it fine to do so? Is it a common practise, or something rather avoided by the developers in the BaseX community? What are the potential security or performance (or any other) risks of such a meddling of deterministic XQueries with non-deterministic Java code? Some risks are mentioned in then BaseX documentation, but it is rather vague. Thanks for hints and suggestions, Jan Ctibor
GS
Graydon Saunders16 Sep 2026 14:54 #
Quoted text · Jan Ctibor · 2 lines
On Wed, Sep 16, 2026, at 07:13, Jan Ctibor via BaseX-Talk wrote: > XQuery has some limitations and peculiarities - most famosly the *invariable variables*. I think I am not the only one, who initilly, after comming to XQuery from the realm of "standard" programming languages, struggled with this pecculiarity. And this as well as some other features of XQuery still makes some things imposible or very difficult to achieve with *pure XQuery.*
There's a saying about how one can write Fortran in any language. "Variables won't, constants aren't" is a core design element of declarative languages in general, not just XQuery; someone advocating for the use of declarative languages would argue that it's an important feature of the approach to language design. In XQuery, trying to extend the language to allow imperative-style variables would get tangled in the distinction between the https://www.w3.org/TR/xquery-31/#dt-static-context and the dynamic context. That strikes me as likely to be unhelpful; the distinction between static and dynamic context is a core part of the language design. Local exceptions to those rules aren't going to make debugging simpler. I'd be more inclined to ask what you're trying to do that you find difficult or impossible?
-- Graydon Saunders | graydonish at fastmail.com Þæs oferéode, ðisses swá mæg. -- Deor ("That passed, so may this.")
WP
Wendell Piez16 Sep 2026 16:09 #
Hello Jan and BaseX list, Thank you for asking this useful and important question. I agree entirely with what Graydon says. Whatever questions you have about how things might be done ('imperatively'), might also be framed in terms of requirements for XQuery, and those will be interesting if nothing else. But, also, you ask
Quoted text · Graydon Saunders · 1 line
> What are the potential security or performance (or any other) risks of
such a meddling of deterministic XQueries with non-deterministic Java code? I like how this question *almost* answers itself. It could be asked about 'meddling with' (nice phrase) any deterministic X with any non-deterministic Y. Apart from X and Y themselves, XY is no longer deterministic. In addition, you now have XQuery that is now dependent on a *specific* non-deterministic externality, namely your library. This introduces risk and costs related to design, testing, maintenance. Take care not to underestimate that commitment. Especially compared to (ideal alternative scenario) XQuery that 'just works' in a way that (was) new to you. *Sometimes* that non-deterministic external component might present an attractive potential ROI in return to balance that risk profile: I don't know, that is for you to judge. :-) My 2 francs, thanks again -- Cheers, Wendell
Quoted text · Graydon Saunders · 34 lines
On Wed, Sep 16, 2026 at 8:54 AM Graydon Saunders via BaseX-Talk < basex-talk at mailman.uni-konstanz.de> wrote: > > On Wed, Sep 16, 2026, at 07:13, Jan Ctibor via BaseX-Talk wrote: > > XQuery has some limitations and peculiarities - most famosly the *invariable > variables*. I think I am not the only one, who initilly, after comming to > XQuery from the realm of "standard" programming languages, struggled with > this pecculiarity. And this as well as some other features of XQuery still > makes some things imposible or very difficult to achieve with *pure > XQuery.* > > There's a saying about how one can write Fortran in any language. > > "Variables won't, constants aren't" is a core design element of > declarative languages in general, not just XQuery; someone advocating for > the use of declarative languages would argue that it's an important feature > of the approach to language design. In XQuery, trying to extend the > language to allow imperative-style variables would get tangled in the > distinction between the https://www.w3.org/TR/xquery-31/#dt-static-context and > the dynamic context. That strikes me as likely to be unhelpful; the > distinction between static and dynamic context is a core part of the > language design. Local exceptions to those rules aren't going to make > debugging simpler. > > I'd be more inclined to ask what you're trying to do that you find > difficult or impossible? > > -- > Graydon Saunders | graydonish at fastmail.com > Þæs oferéode, ðisses swá mæg. > -- Deor ("That passed, so may this.") > >
-- ...Wendell Piez... ...wendellpiez.com... ...pellucidliterature.org... ...raventracks.org... ... xproc.zone... ...github.com/wendellpiez...
JC
Jan Ctibor16 Sep 2026 17:18 #
Dear Wendell and Graydon, thanks for your answers. I actually don't know, whether what I have done can be called "meddling" - its rather putting values aside to Java object and then retriving them all on the end of the execution. Actually, probably only the retrieving function can be seen as non-deterministic. My use-case: I'm building a RESTXQ application, that grows really complex and I want to send back to the client, along with the actual output, some kind of log of what happened during the execution - which functions were called, what was the important values, where an error was encounterd etc. If I ran this in the standalone mode, I would use the message (or prof:dump in older versions) function, but since the output of this function can't be retrieved in the RESTXQ (or can it be retrieved somehow?), I was wondering, how to do this. One way would be to write the messages to a file and then read it (given some deterministic way would be found, how to name this file for each call to the server) - but the file:read-... functions are after all non- deterministic as well! Then it seem to me as more elegant solution than to use files to write a small class in Java, in which I store those messages and then retrieve them, really a few lines of code. How could this be achieved in a fully deterministic way? I suppouse, you should probably have to pass a sequence (or map or whatever) of the already existing messages as an parameter to each and every function you would want to log, then add new message and return the new sequence along with the "actual" result of the function, so it could be passed to the next function and so on and on. This could work in theory, but in practise it would be probably a nightmare to mantain this through the whole code. (But maybe I lack imagination or knowledge and this could be in fact be done in a simpler and a more elegant way.) So, my present case could be probably resolved even without Java, just using the file module - but this is non-deterministic anyway. And, by the way, it brings up the question: is using file module "dangerous" in the same way as using Java code? I fully understand, that real "meddling" could lead to unforeseen results. But where is the line? Is calling an external function in Java by itself dangerous? Or really only a non-deterministic one, which gives different results with the same input? In the former version of my project, I had to rewrite substantial parts of the program in python (and retrieve data from BX externally), because it seemed to me impossilbe to achieve my goals with XQuery - despite I was stil only working with XML files. I'd love to rewrite it all in XQuery with some Java code inside it, but I don't know, if it is a good and safe idea... Best regards Jan ---------- Původní e‑mail ---------- Od: Wendell Piez via BaseX-Talk <basex-talk at mailman.uni-konstanz.de> Komu: BaseX <basex-talk at mailman.uni-konstanz.de> Datum: 16. 9. 2026 16:11:29 Předmět: [basex-talk] Re: Java code in XQuery? " Hello Jan and BaseX list, Thank you for asking this useful and important question. I agree entirely with what Graydon says. Whatever questions you have about how things might be done ('imperatively'), might also be framed in terms of requirements for XQuery, and those will be interesting if nothing else. But, also, you ask
Quoted text · Wendell Piez · 1 line
> What are the potential security or performance (or any other) risks of
such a meddling of deterministic XQueries with non-deterministic Java code? I like how this question *almost* answers itself. It could be asked about 'meddling with' (nice phrase) any deterministic X with any non-deterministic Y. Apart from X and Y themselves, XY is no longer deterministic. In addition, you now have XQuery that is now dependent on a *specific* non- deterministic externality, namely your library. This introduces risk and costs related to design, testing, maintenance. Take care not to underestimate that commitment. Especially compared to (ideal alternative scenario) XQuery that 'just works' in a way that (was) new to you. *Sometimes* that non-deterministic external component might present an attractive potential ROI in return to balance that risk profile: I don't know, that is for you to judge. :-) My 2 francs, thanks again -- Cheers, Wendell
Quoted text · Graydon Saunders · 2 lines
On Wed, Sep 16, 2026 at 8:54 AM Graydon Saunders via BaseX-Talk <basex-talk@ mailman.uni-konstanz.de(mailto:basex-talk at mailman.uni-konstanz.de)> wrote:
"
Quoted text · Jan Ctibor · 1 line
On Wed, Sep 16, 2026, at 07:13, Jan Ctibor via BaseX-Talk wrote:
" XQuery has some limitations and peculiarities - most famosly the invariable variables. I think I am not the only one, who initilly, after comming to XQuery from the realm of "standard" programming languages, struggled with this pecculiarity. And this as well as some other features of XQuery still makes some things imposible or very difficult to achieve with pure XQuery. " There's a saying about how one can write Fortran in any language. "Variables won't, constants aren't" is a core design element of declarative languages in general, not just XQuery; someone advocating for the use of declarative languages would argue that it's an important feature of the approach to language design. In XQuery, trying to extend the language to allow imperative-style variables would get tangled in the distinction between the https://www.w3.org/TR/xquery-31/#dt-static-context (https://www.w3.org/TR/xquery-31/#dt-static-context) and the dynamic context. That strikes me as likely to be unhelpful; the distinction between static and dynamic context is a core part of the language design. Local exceptions to those rules aren't going to make debugging simpler. I'd be more inclined to ask what you're trying to do that you find difficult or impossible?
-- Graydon Saunders | graydonish at fastmail.com(mailto:graydonish at fastmail.com) Þæs oferéode, ðisses swá mæg. -- Deor ("That passed, so may this.") " -- ...Wendell Piez... ...wendellpiez.com... ...pellucidliterature.org... ...raventracks.org... ... xproc.zone... ...github.com/wendellpiez.(http://github.com/wendellpiez.) .. "
GS
Graydon Saunders16 Sep 2026 18:29 #
Quoted text · Jan Ctibor · 2 lines
On Wed, Sep 16, 2026, at 11:18, Jan Ctibor via BaseX-Talk wrote: > How could this be achieved in a fully deterministic way? I suppouse, you should probably have to pass a sequence (or map or whatever) of the already existing messages as an parameter to each and every function you would want to log, then add new message and return the new sequence along with the "actual" result of the function, so it could be passed to the next function and so on and on. This could work in theory, but in practise it would be probably a nightmare to mantain this through the whole code. (But maybe I lack imagination or knowledge and this could be in fact be done in a simpler and a more elegant way.)
My first thought for this is that it's easy to create a database and it's easy to create a unique database name by appending a UUID to a standard string (you may well want something more elaborate than 'report' || random:uuid() but that approach does work.) The DB name does need to be passed through the function chain and you would need to make friends with the rules for updating expressions but presuming the individual steps are their own modules, a pattern like
let $queryNameList as xs:string+ :=  ('step-1.xq', 'step-2.xq','step-3.xq','step-4.xq')
let $queryURIList as xs:anyURI+ := $queryNameList ! xs:anyURI(.)
let $bindings as map(*) := (: bindings to pass values to the queries we're about to run :)
    {... 'messages-db': 'report' || random:uuid() }
return for $query in $queryURIList
    return job:execute($query,$bindings)
in the outermost module leaves you at the end with a db full of messages you can return and then delete the message db. Which might well leave the RESTQ the outer-outermost thing, outside the above pattern so you wind up with the response getting both whatever the main logic does and the db contents. Or two dbs, one populated by the main logic and one populated by the messaging. But I don't know much about the shape of what you're specifically doing so I ought to hush. I use this pattern to load before-and-after document versions to individual dbs, adjust the contents of those dbs, and then compare them to report if any words got lost during a conversion process. Once one acclimatizes to "an expression will return one thing and only one thing, that's core language design" and "a sequence is one thing; so is a map; so is the return value of an updating expression on an arbitrary db" and "every db change is its own module", I find that's a powerful pattern. (Well, and the BaseX "a db is a lightweight object, one db per document isn't inherently silly" approach; that's not core XQuery but it's also an acclimatization.) -- Graydon
JC
Jan Ctibor18 Sep 2026 13:53 #
Dear Graydon, thank you for your suggestion: it is actually a interesting idea to use a dedicated database for this purpose. However: you suggest using both random:uuid() and job:execute() - yes, the code inside the single modules is stil deterministic, but the program as whole is not, because those functions are not deterministic. Or am I wrong? It moves us actually a bit further from my original Question - we don't need to talk about custom Java classes at all. There are many non-deterministic functions already incorporated in the XQuery - like file:read() - or whole modules: random, job, proccess. Some of those functions can have some kind of side-effects: writing to a file (which you can read later), using one job:execute to change a db and than another job:execute to read this changed database... So my questions are: 1) Has the determinism and functional-declarativness of XQuery actually been already compromised by adoption of functions mentioned above? 2) Are those functions actually a part of XQuery language, or are they rather to be considered a part of a specific XQuery implementation (aka BaseX,eXist etc.?) 3) If so, is it a problem? Should we strive for pure functional-declarative deterministic XQuery code with so many "traps" and "enticements" all around? Are we actually writing applications in "XQuery", or rather in "BaseX XQuery"? 4) Am I overthinking? Am I seeing problems where there aren’t any? 5) Am I completely off the mark? Jan ---------- Původní e‑mail ---------- Od: Graydon Saunders via BaseX-Talk <basex-talk at mailman.uni-konstanz.de> Komu: Jan Ctibor <CtiborJan at seznam.cz> Kopie: BaseX <basex-talk at mailman.uni-konstanz.de> Datum: 16. 9. 2026 18:31:21 Předmět: [basex-talk] Re: Java code in XQuery? "
Quoted text · Jan Ctibor · 1 line
On Wed, Sep 16, 2026, at 11:18, Jan Ctibor via BaseX-Talk wrote:
" How could this be achieved in a fully deterministic way? I suppouse, you should probably have to pass a sequence (or map or whatever) of the already existing messages as an parameter to each and every function you would want to log, then add new message and return the new sequence along with the "actual" result of the function, so it could be passed to the next function and so on and on. This could work in theory, but in practise it would be probably a nightmare to mantain this through the whole code. (But maybe I lack imagination or knowledge and this could be in fact be done in a simpler and a more elegant way.) " My first thought for this is that it's easy to create a database and it's easy to create a unique database name by appending a UUID to a standard string (you may well want something more elaborate than 'report' || random: uuid() but that approach does work.) The DB name does need to be passed through the function chain and you would need to make friends with the rules for updating expressions but presuming the individual steps are their own modules, a pattern like let $queryNameList as xs:string+ := ('step-1.xq', 'step-2.xq','step-3.xq',' step-4.xq')
let $queryURIList as xs:anyURI+ := $queryNameList ! xs:anyURI(.)
let $bindings as map(*) := (: bindings to pass values to the queries we're about to run :) {... 'messages-db': 'report' || random:uuid() } return for $query in $queryURIList return job:execute($query,$bindings) in the outermost module leaves you at the end with a db full of messages you can return and then delete the message db. Which might well leave the RESTQ the outer-outermost thing, outside the above pattern so you wind up with the response getting both whatever the main logic does and the db contents. Or two dbs, one populated by the main logic and one populated by the messaging. But I don't know much about the shape of what you're specifically doing so I ought to hush. I use this pattern to load before-and-after document versions to individual dbs, adjust the contents of those dbs, and then compare them to report if any words got lost during a conversion process. Once one acclimatizes to "an expression will return one thing and only one thing, that's core language design" and "a sequence is one thing; so is a map; so is the return value of an updating expression on an arbitrary db" and "every db change is its own module", I find that's a powerful pattern. (Well, and the BaseX "a db is a lightweight object, one db per document isn' t inherently silly" approach; that's not core XQuery but it's also an acclimatization.) -- Graydon "
GS
Graydon Saunders18 Sep 2026 15:46 #
Quoted text · Jan Ctibor · 2 lines
On Fri, Sep 18, 2026, at 07:53, Jan Ctibor via BaseX-Talk wrote: > However: you suggest using both random:uuid() and job:execute() - yes, the code inside the single modules is stil deterministic, but the program as whole is not, because those functions are not deterministic. Or am I wrong?
I don't think you're wrong. random:uuid is a convenience; you can get the avoidance of collision of db names by other means. (decide how many simultaneous operations you can have, multiply that by a suitably chosen prime and round-robin through a fixed list of names, for example.) job:execute() is a way to treat query modules like functions. This is a convenience in the sense that it allows maintainable code to be written, rather than requiring everything you need to do to go into a single module. I don't think it's strictly required in some formal sense and I don't think something like it couldn't be part of the language specification.
Quoted text · Jan Ctibor · 1 line
> 1) Has the determinism and functional-declarativness of XQuery actually been already compromised by adoption of functions mentioned above?
Once you set out to actually do something with a programming language, it needs to write to a file system (or deal with the idea that the computer running it doesn't have a continuous power supply in some life-of-the-universe sense). The pure abstract determinism of XQuery was compromised by `doc()` from XPath if it wasn't compromised by being run on imperfect physical hardware.
Quoted text · Jan Ctibor · 1 line
> 2) Are those functions actually a part of *XQuery* language, or are they rather to be considered a part of a specific XQuery implementation (aka *BaseX*,eXist etc.?)
This gets into graduated answers; I think in principle all of the extension functions could be formalized, and some of them (file:read(), etc.) are, so it turns into how much you think the libraries are part of the language.
Quoted text · Jan Ctibor · 1 line
> 3) If so, is it a problem? Should we strive for pure functional-declarative deterministic XQuery code with so many "traps" and "enticements" all around? Are we actually writing applications in "XQuery", or rather in "BaseX XQuery"?
As a practical matter, you're writing in $IMPLEMENTATION XQuery for some value of $IMPLEMENTATION. This might not matter much for a given application, but given that there implementation-defined behaviors in the spec it is expected to matter which implementation you select. From my perspective, the notable thing about BaseX is that it works (comes very close to implementing the full spec in a performant way) and that it has this nifty DB concept. (Which is, at least as I understand it, not part of the spec! `collection()` is part of the spec; "there's a database" is not.) Since (again as far as I know) nothing else presents that combination, accepting that I'm going to be writing against this particular implementation is easy.
Quoted text · Jan Ctibor · 2 lines
> 4) Am I overthinking? Am I seeing problems where there aren’t any? > 5) Am I completely off the mark?
I think it depends on what your objectives are. If you're after a demonstration of a principle, you've got to think a lot about where exactly the boundaries are (reality can have a fire in the server room if it feels like it; there will always be some boundaries). If you're trying to do a thing well enough for some particular purpose, that gets into "this one time" versus "for a lot of people, into the indefinite future". I'm arguing from a "and future-you will be tired and stressed already" take on (what I understand of) "for a lot of people, into the indefinite future". -- Graydon

Written for BaseX 12.4. BaseX may have changed since: please check the current documentation.

« multiple import of the same module[ANN] IOXML meeting Fri 25 Sep @ 1500 UTC »
⚡Generated with XQuery