Difference between revisions of "Job Module"
m (Text replacement - "syntaxhighlight" to "pre") |
|||
(11 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
− | + | This [[Module Library|XQuery Module]] provides functions for registering new query jobs and orchestrating existing jobs. Jobs can be queries, commands, operations performed by a database client, and HTTP requests. | |
− | |||
− | This [[Module Library|XQuery Module]] provides functions for | ||
=Conventions= | =Conventions= | ||
Line 11: | Line 9: | ||
A job can be registered as ''service'' by supplying the {{Code|service}} option to {{Function||job:eval}}: | A job can be registered as ''service'' by supplying the {{Code|service}} option to {{Function||job:eval}}: | ||
− | < | + | <pre lang='xquery'> |
(: register job as service; will be run every day at 1 am :) | (: register job as service; will be run every day at 1 am :) | ||
job:eval('db:drop("tmp")', (), map { 'id':'cleanup', 'start':'01:00:00', 'interval':'P1D', 'service': true() }), | job:eval('db:drop("tmp")', (), map { 'id':'cleanup', 'start':'01:00:00', 'interval':'P1D', 'service': true() }), | ||
Line 21: | Line 19: | ||
(: unregister job :) | (: unregister job :) | ||
job:remove('cleanup', map { 'service': true() }) | job:remove('cleanup', map { 'service': true() }) | ||
− | </ | + | </pre> |
'''Some more notes:''' | '''Some more notes:''' | ||
Line 31: | Line 29: | ||
=Executing Jobs= | =Executing Jobs= | ||
− | There are cases in which a client does not, or cannot, wait until a request is fully processed. The client may be a browser, which sends an HTTP request to the server to start another time-consuming query job. The functions in this section allow you to register | + | There are cases in which a client does not, or cannot, wait until a request is fully processed. The client may be a browser, which sends an HTTP request to the server to start another time-consuming query job. The functions in this section allow you to register new query jobs and access existing ones. Jobs can be executed immediately (i.e., as soon as a [[Transaction Management#Concurrency Control|free slot is available]]) or scheduled for repeated execution. Each registered job gets a job ID, and the ID can be used to retrieve a query result, stop a job, or wait for its termination. |
==job:eval== | ==job:eval== | ||
Line 37: | Line 35: | ||
{| width='100%' | {| width='100%' | ||
|- valign="top" | |- valign="top" | ||
− | | width='120' | ''' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>job:eval( |
+ | $query as xs:anyAtomicType, | ||
+ | $bindings as map(*)? := map { }, | ||
+ | $options as map(*)? := map { } | ||
+ | ) as xs:string</pre> | ||
|- valign="top" | |- valign="top" | ||
| '''Summary''' | | '''Summary''' | ||
− | |Schedules the evaluation | + | |Schedules the evaluation a new query job for the supplied {{Code|$query}} (of type {{Code|xs:string}}, or of type {{Code|xs:anyURI}} if points to a resource), and returns a job ID. The job will be queued until a free slot is available, and the query result can be cached. Queries can be updating, and variables and the context value can be declared via {{Code|$bindings}} (see {{Function|XQuery|xquery:eval}} for more details). The following {{Code|$options}} can be supplied: |
* {{Code|cache}}: indicates if the query result will be cached or ignored (default: {{Code|false}}): | * {{Code|cache}}: indicates if the query result will be cached or ignored (default: {{Code|false}}): | ||
** The result will be cached in main-memory until it is fetched via {{Function||job:result}}, or until {{Option|CACHETIMEOUT}} is exceeded. | ** The result will be cached in main-memory until it is fetched via {{Function||job:result}}, or until {{Option|CACHETIMEOUT}} is exceeded. | ||
Line 63: | Line 65: | ||
| | | | ||
* Cache query result. The returned ID can be used to pick up the result with {{Function||job:result}}: | * Cache query result. The returned ID can be used to pick up the result with {{Function||job:result}}: | ||
− | < | + | <pre lang='xquery'> |
job:eval("1+3", (), map { 'cache': true() }) | job:eval("1+3", (), map { 'cache': true() }) | ||
− | </ | + | </pre> |
* A happy birthday mail will be sent at the given date: | * A happy birthday mail will be sent at the given date: | ||
− | < | + | <pre lang='xquery'> |
job:eval("import module namespace mail='mail'; mail:send('Happy birthday!')", | job:eval("import module namespace mail='mail'; mail:send('Happy birthday!')", | ||
(), map { 'start': '2018-09-01T06:00:00' })}} | (), map { 'start': '2018-09-01T06:00:00' })}} | ||
− | </ | + | </pre> |
* The following [[RESTXQ]] functions can be called to execute a query at 2 am every day. An ID will be returned by the first function, which can be used to stop the scheduler via the second function: | * The following [[RESTXQ]] functions can be called to execute a query at 2 am every day. An ID will be returned by the first function, which can be used to stop the scheduler via the second function: | ||
− | < | + | <pre lang='xquery'> |
declare %rest:POST("{$query}") %rest:path('/start-scheduling') function local:start($query) { | declare %rest:POST("{$query}") %rest:path('/start-scheduling') function local:start($query) { | ||
job:eval($query, (), map { 'start': '02:00:00', 'interval': 'P1D' }) | job:eval($query, (), map { 'start': '02:00:00', 'interval': 'P1D' }) | ||
Line 79: | Line 81: | ||
job:remove($id) | job:remove($id) | ||
}; | }; | ||
− | </ | + | </pre> |
* Query execution is scheduled for every second, and for 10 seconds in total. As the query itself will take 1.5 seconds, it will only be executed every second time: | * Query execution is scheduled for every second, and for 10 seconds in total. As the query itself will take 1.5 seconds, it will only be executed every second time: | ||
− | < | + | <pre lang='xquery'> |
job:eval("prof:sleep(1500)", (), map { 'interval': 'PT1S', 'end': 'PT10S' }) | job:eval("prof:sleep(1500)", (), map { 'interval': 'PT1S', 'end': 'PT10S' }) | ||
− | </ | + | </pre> |
* The query in the specified file will be evaluated once: | * The query in the specified file will be evaluated once: | ||
− | < | + | <pre lang='xquery'> |
job:eval(xs:anyURI('cleanup.xq')) | job:eval(xs:anyURI('cleanup.xq')) | ||
− | </ | + | </pre> |
* The following expression, if stored in a file, will be evaluated every 5 seconds: | * The following expression, if stored in a file, will be evaluated every 5 seconds: | ||
− | < | + | <pre lang='xquery'> |
job:eval( | job:eval( | ||
static-base-uri(), | static-base-uri(), | ||
Line 95: | Line 97: | ||
map { 'start': 'PT5S' } | map { 'start': 'PT5S' } | ||
) | ) | ||
− | </ | + | </pre> |
|} | |} | ||
Line 102: | Line 104: | ||
{| width='100%' | {| width='100%' | ||
|- valign="top" | |- valign="top" | ||
− | | width='120' | ''' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>job:result( |
+ | $id as xs:string, | ||
+ | $options as map(*)? := map { } | ||
+ | ) as item()*</pre> | ||
|- valign="top" | |- valign="top" | ||
| '''Summary''' | | '''Summary''' | ||
|Returns the cached result of a job with the specified job {{Code|$id}}: | |Returns the cached result of a job with the specified job {{Code|$id}}: | ||
* If the original job has raised an error, the cached error will be raised instead. | * If the original job has raised an error, the cached error will be raised instead. | ||
− | * | + | * The cached result or error will be dropped after it has been retrieved. |
− | * If the result has | + | * If the result has not been cached or if it has been dropped, an empty sequence is returned. |
+ | The following {{Code|$options}} can be supplied: | ||
+ | * {{Code|keep}}: Keep the cached result or error after retrieval. | ||
+ | |||
|- valign="top" | |- valign="top" | ||
| '''Examples''' | | '''Examples''' | ||
| | | | ||
* The following [[RESTXQ]] function will either return the result of a previously started job or raise an error: | * The following [[RESTXQ]] function will either return the result of a previously started job or raise an error: | ||
− | < | + | <pre lang='xquery'> |
declare %rest:path('/result/{$id}') function local:result($id) { | declare %rest:path('/result/{$id}') function local:result($id) { | ||
job:result($id) | job:result($id) | ||
}; | }; | ||
− | </ | + | </pre> |
* The following query demonstrates how the results of an executed query can be returned within the same query (see below why you should avoid this pattern in practice): | * The following query demonstrates how the results of an executed query can be returned within the same query (see below why you should avoid this pattern in practice): | ||
− | < | + | <pre lang='xquery'> |
let $query := job:eval('(1 to 10000000)[. = 1]', map { }, map { 'cache': true() }) | let $query := job:eval('(1 to 10000000)[. = 1]', map { }, map { 'cache': true() }) | ||
return ( | return ( | ||
Line 126: | Line 134: | ||
job:result($query) | job:result($query) | ||
) | ) | ||
− | </ | + | </pre> |
Queries of this kind can cause deadlocks! If the original query and the new query perform updates on the same database, the second query will only be run after the first one has been executed, and the first query will wait for the second query forever. You should resort to {{Function|XQuery|xquery:fork-join}} if you want to have full control on parallel query execution. | Queries of this kind can cause deadlocks! If the original query and the new query perform updates on the same database, the second query will only be run after the first one has been executed, and the first query will wait for the second query forever. You should resort to {{Function|XQuery|xquery:fork-join}} if you want to have full control on parallel query execution. | ||
|} | |} | ||
==job:remove== | ==job:remove== | ||
− | |||
− | |||
{| width='100%' | {| width='100%' | ||
|- valign="top" | |- valign="top" | ||
− | | width='120' | ''' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>job:remove( |
+ | $id as xs:string, | ||
+ | $options as map(*)? := map { } | ||
+ | ) as empty-sequence()</pre> | ||
|- valign="top" | |- valign="top" | ||
| '''Summary''' | | '''Summary''' | ||
Line 153: | Line 162: | ||
{| width='100%' | {| width='100%' | ||
|- valign="top" | |- valign="top" | ||
− | | width='120' | ''' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>job:wait( |
+ | $id as xs:string | ||
+ | ) as empty-sequence()</pre> | ||
|- valign="top" | |- valign="top" | ||
| '''Summary''' | | '''Summary''' | ||
Line 171: | Line 182: | ||
{| width='100%' | {| width='100%' | ||
|- valign="top" | |- valign="top" | ||
− | | width='120' | ''' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>job:current() as xs:string</pre> |
|- valign="top" | |- valign="top" | ||
| '''Summary''' | | '''Summary''' | ||
Line 182: | Line 193: | ||
{| width='100%' | {| width='100%' | ||
|- valign="top" | |- valign="top" | ||
− | | width='120' | ''' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>job:list() as xs:string*</pre> |
|- valign="top" | |- valign="top" | ||
| '''Summary''' | | '''Summary''' | ||
Line 196: | Line 207: | ||
{| width='100%' | {| width='100%' | ||
|- valign="top" | |- valign="top" | ||
− | | width='120' | ''' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>job:list-details( |
+ | $id as xs:string := () | ||
+ | ) as element(job)*</pre> | ||
|- valign="top" | |- valign="top" | ||
| '''Summary''' | | '''Summary''' | ||
Line 211: | Line 224: | ||
| '''Examples''' | | '''Examples''' | ||
| <code>job:list-details()</code> returns information on the currently running job and possibly others: | | <code>job:list-details()</code> returns information on the currently running job and possibly others: | ||
− | < | + | <pre lang="xml"> |
<job id="job1" type="XQuery" state="running" user="admin" duration="PT0.001S"> | <job id="job1" type="XQuery" state="running" user="admin" duration="PT0.001S"> | ||
XQUERY job:list-details() | XQUERY job:list-details() | ||
</job> | </job> | ||
− | </ | + | </pre> |
|} | |} | ||
==job:bindings== | ==job:bindings== | ||
− | |||
− | |||
{| width='100%' | {| width='100%' | ||
|- valign="top" | |- valign="top" | ||
− | | width='120' | ''' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>job:bindings( |
+ | $id as xs:string | ||
+ | ) as map(*)</pre> | ||
|- valign="top" | |- valign="top" | ||
| '''Summary''' | | '''Summary''' | ||
Line 235: | Line 248: | ||
{| width='100%' | {| width='100%' | ||
|- valign="top" | |- valign="top" | ||
− | | width='120' | ''' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>job:finished( |
+ | $id as xs:string | ||
+ | ) as xs:boolean</pre> | ||
|- valign="top" | |- valign="top" | ||
| '''Summary''' | | '''Summary''' | ||
Line 248: | Line 263: | ||
{| width='100%' | {| width='100%' | ||
|- valign="top" | |- valign="top" | ||
− | | width='120' | ''' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>job:services() as element(job)*</pre> |
|- valign="top" | |- valign="top" | ||
| '''Summary''' | | '''Summary''' | ||
Line 291: | Line 306: | ||
* Updated: Renamed from ''Jobs Module'' to ''Job Module''. The namespace URI has been updated as well. | * Updated: Renamed from ''Jobs Module'' to ''Job Module''. The namespace URI has been updated as well. | ||
* Updated: {{Function||job:remove}} renamed from {{Code|jobs:stop}}. | * Updated: {{Function||job:remove}} renamed from {{Code|jobs:stop}}. | ||
+ | * Updated: {{Function||job:result}}: options argument added. | ||
* Added: {{Function||job:bindings}} | * Added: {{Function||job:bindings}} | ||
Latest revision as of 18:39, 1 December 2023
This XQuery Module provides functions for registering new query jobs and orchestrating existing jobs. Jobs can be queries, commands, operations performed by a database client, and HTTP requests.
Contents
Conventions[edit]
All functions in this module are assigned to the http://basex.org/modules/job
namespace, which is statically bound to the job
prefix. Errors will be bound to the same prefix.
Services[edit]
A job can be registered as service by supplying the service
option to job:eval
:
(: register job as service; will be run every day at 1 am :)
job:eval('db:drop("tmp")', (), map { 'id':'cleanup', 'start':'01:00:00', 'interval':'P1D', 'service': true() }),
(: list registered services :)
job:services(),
(: result: <job base-uri="..." id="cleanup" interval="P1D" start="01:00:00">db:drop("tmp")</job> :)
(: unregister job :)
job:remove('cleanup', map { 'service': true() })
Some more notes:
- All job services will be scheduled for evaluation when the BaseX server or BaseX HTTP server is started.
- If a job service is outdated (e.g. because a supplied end time has been exceeded), it will be removed from the jobs file at startup time.
- The job definitions are stored in a
jobs.xml
file in the database directory. It can also be edited manually.
Executing Jobs[edit]
There are cases in which a client does not, or cannot, wait until a request is fully processed. The client may be a browser, which sends an HTTP request to the server to start another time-consuming query job. The functions in this section allow you to register new query jobs and access existing ones. Jobs can be executed immediately (i.e., as soon as a free slot is available) or scheduled for repeated execution. Each registered job gets a job ID, and the ID can be used to retrieve a query result, stop a job, or wait for its termination.
job:eval[edit]
Signature | job:eval( $query as xs:anyAtomicType, $bindings as map(*)? := map { }, $options as map(*)? := map { } ) as xs:string |
Summary | Schedules the evaluation a new query job for the supplied $query (of type xs:string , or of type xs:anyURI if points to a resource), and returns a job ID. The job will be queued until a free slot is available, and the query result can be cached. Queries can be updating, and variables and the context value can be declared via $bindings (see xquery:eval for more details). The following $options can be supplied:
|
Errors | overflow : Query execution is rejected because too many jobs are queued or being executed. CACHETIMEOUT can be decreased if the default setting is too restrictive.range : A specified time or duration is out of range.id : The specified ID is invalid or has already been assigned.options : The specified options are conflicting.
|
Examples |
job:eval("1+3", (), map { 'cache': true() })
job:eval("import module namespace mail='mail'; mail:send('Happy birthday!')",
(), map { 'start': '2018-09-01T06:00:00' })}}
declare %rest:POST("{$query}") %rest:path('/start-scheduling') function local:start($query) {
job:eval($query, (), map { 'start': '02:00:00', 'interval': 'P1D' })
};
declare %rest:path('/stop-scheduling/{$id}') function local:stop($id) {
job:remove($id)
};
job:eval("prof:sleep(1500)", (), map { 'interval': 'PT1S', 'end': 'PT10S' })
job:eval(xs:anyURI('cleanup.xq'))
job:eval(
static-base-uri(),
map { },
map { 'start': 'PT5S' }
)
|
job:result[edit]
Signature | job:result( $id as xs:string, $options as map(*)? := map { } ) as item()* |
Summary | Returns the cached result of a job with the specified job $id :
The following
|
Examples |
declare %rest:path('/result/{$id}') function local:result($id) {
job:result($id)
};
let $query := job:eval('(1 to 10000000)[. = 1]', map { }, map { 'cache': true() })
return (
job:wait($query),
job:result($query)
)
Queries of this kind can cause deadlocks! If the original query and the new query perform updates on the same database, the second query will only be run after the first one has been executed, and the first query will wait for the second query forever. You should resort to |
job:remove[edit]
Signature | job:remove( $id as xs:string, $options as map(*)? := map { } ) as empty-sequence() |
Summary | Triggers the cancelation of a job with the specified $id , cancels a scheduled job or removes a cached result. Unknown IDs are ignored. All jobs are gracefully stopped; it is up to the process to decide when it is safe to shut down. The following $options can be supplied:
|
Examples |
|
job:wait[edit]
Signature | job:wait( $id as xs:string ) as empty-sequence() |
Summary | Waits for the completion of a job with the specified $id :
|
Errors | self : The current job is addressed. |
Listing Jobs[edit]
job:current[edit]
Signature | job:current() as xs:string |
Summary | Returns the ID of the current job. |
job:list[edit]
Signature | job:list() as xs:string* |
Summary | Returns the IDs of all jobs that are currently registered. The list includes scheduled, queued, running, stopped, and finished jobs with cached results. |
Examples | job:list() returns the same job ID as job:current if no other job is registered.
|
job:list-details[edit]
Signature | job:list-details( $id as xs:string := () ) as element(job)* |
Summary | Returns information on all jobs that are currently registered, or on a job with the specified $id (or an empty sequence if this job is not found). The list includes scheduled, queued, running jobs, and cached jobs. A string representation of the job, or its URI, will be returned as a value. The returned elements have additional attributes:
|
Examples | job:list-details() returns information on the currently running job and possibly others:
<job id="job1" type="XQuery" state="running" user="admin" duration="PT0.001S">
XQUERY job:list-details()
</job>
|
job:bindings[edit]
Signature | job:bindings( $id as xs:string ) as map(*) |
Summary | Returns the variable bindings of an existing job with the specified $id . If no variables have been bound to this job, an empty map is returned.
|
job:finished[edit]
Signature | job:finished( $id as xs:string ) as xs:boolean |
Summary | Indicates if the evaluation of an already running job with the specified $id has finished. As the IDs of finished jobs will usually be discarded, unless caching is enabled, the function will also return true for unknown jobs.
|
job:services[edit]
Signature | job:services() as element(job)* |
Summary | Returns a list of all jobs that have been persistently registered as Services. |
Errors | services : Registered services cannot be parsed. |
Errors[edit]
Code | Description |
---|---|
options
|
The specified options are conflicting. |
id
|
The specified ID is invalid or has already been assigned. |
overflow
|
Too many queries or query results are queued. |
range
|
A specified time or duration is out of range. |
running
|
A query is still running. |
self
|
The current job cannot be addressed. |
service
|
Registered services cannot be parsed, added or removed. |
Changelog[edit]
- Version 10.0
- Updated: Renamed from Jobs Module to Job Module. The namespace URI has been updated as well.
- Updated:
job:remove
renamed fromjobs:stop
. - Updated:
job:result
: options argument added. - Added:
job:bindings
- Version 9.7
- Updated:
job:result
: return empty sequence if no result is cached.
- Version 9.5
- Updated:
job:eval
: integers added as valid start and end times.
- Version 9.4
- Updated:
job:eval
: option added for writing log entries. - Updated:
job:list-details
: interval added.
- Version 9.2
- Deleted: job:invoke (merged with
job:eval
)
- Version 9.1
- Updated:
job:list-details
: registration time added.
- Version 9.0
- Added:
job:invoke
, Services
- Version 8.6
- Updated:
job:eval
:id
option added.
The module was introduced with Version 8.5.