Recursive query performance
Tuesday, 14 February 2012 · 2 messages · last reply 15 Feb 2012
CD
Charles Duffy14 Feb 2012 16:50 #
Howdy, all --
First, a note that bears saying -- I'm very, *very* excited by BaseX; this is my first use of the database, and I'm already extremely impressed.
My immediate use case centers around the following pair of functions:
declare function local:immediate_requirements($e as element(project)) as
element(project)* {
doc("inventory.xml")/report/project[
./classifiers/ivy/publications/artifact/@name =
$e/classifiers/ivy/dependencies/dependency/@name
]
};
declare function local:requirements($e as element(project))
as element(project)* {
let $direct := local:immediate_requirements($e)
return $direct | $direct/local:requirements(.)
};However -- while evaluating immediate_requirements() is very fast (on the scale of 72ms for one of the larger project elements in my sample corpus, and a total under 1600ms ms to evaluate for _every_ project element in said corpus), evaluating the recursive requirements() function for even a single top-level project can require as much as two minutes to execute.
Should I be writing this function differently?
My expectation was that, due to XQuery's functional nature, the database should be able to avoid needing to evaluate local:immediate_requirements more than once for a single input element, capping the worst-case execution time not far over the cost of evaluating that single-stage evaluation once over every possible project element in the document; however, true worst-case execution time is obviously far worse.
Thank you very much for any guidance on offer, and for this excellent tool.
CG
Christian Grün15 Feb 2012 00:09 #
Dear Charles,
Quoted text · Charles Duffy · 2 lines
> First, a note that bears saying -- I'm very, *very* excited by BaseX; > this is my first use of the database, and I'm already extremely impressed.
thanks for the positive feedback, always appreciated!
Quoted text · Charles Duffy · 8 lines
> My immediate use case centers around the following pair of functions: > [...] > My expectation was that, due to XQuery's functional nature, the database > should be able to avoid needing to evaluate local:immediate_requirements > more than once for a single input element, capping the worst-case > execution time not far over the cost of evaluating that single-stage > evaluation once over every possible project element in the document; > however, true worst-case execution time is obviously far worse.
While it's often difficult to give general advise on how to optimize XQuery expressions without looking at the actual data, it may help in your case to cache often requested database nodes in a variable and pass them on to the functions - or even put them in a global variable to get sure they'll only be evaluated once. Just a quick guess, your feedback is welcome.
Christian
Written for BaseX 7.1. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery