Difference between revisions of "Job Module"
(21 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
− | This [[Module Library|XQuery Module]] provides functions for organizing queued and | + | This [[Module Library|XQuery Module]] provides functions for organizing scheduled, queued, running and cached jobs. Jobs can be commands, queries, client or HTTP requests. |
=Conventions= | =Conventions= | ||
Line 26: | Line 26: | ||
|- | |- | ||
| '''Summary''' | | '''Summary''' | ||
− | |Returns the ids of all jobs that are currently queued | + | |Returns the ids of all jobs that are currently registered. The list includes scheduled, queued, running jobs, and cached jobs. |
|- | |- | ||
| '''Examples''' | | '''Examples''' | ||
| | | | ||
− | * <code>jobs:list()</code> returns the same job id as [[#jobs:current|jobs:current]] if no other job is | + | * <code>jobs:list()</code> returns the same job id as [[#jobs:current|jobs:current]] if no other job is registered. |
* <code>jobs:list()[. != jobs:current()] ! jobs:stop(.)</code> stops and invalidates all asynchronous queries and results except for the current one. | * <code>jobs:list()[. != jobs:current()] ! jobs:stop(.)</code> stops and invalidates all asynchronous queries and results except for the current one. | ||
|} | |} | ||
Line 42: | Line 42: | ||
|- | |- | ||
| '''Summary''' | | '''Summary''' | ||
− | |Returns information on all jobs that are currently queued | + | |Returns information on all jobs that are currently registered. The list includes scheduled, queued, running jobs, and cached jobs. A string representation of the job will be returned in the text node. The returned elements have additional attributes: |
+ | * <code>id</code>: job id | ||
+ | * <code>type</code>: type of the job (command, query, REST, RESTXQ, etc.) | ||
+ | * <code>state</code>: current state of the job (scheduled, queued, running, or cached) | ||
+ | * <code>user</code>: the user who started the job | ||
+ | * <code>duration</code>: evaluation time (for running and cached jobs) | ||
+ | * <code>start</code>: dateTime string with next start (for jobs that will be executed repeatedly) | ||
|- | |- | ||
| '''Examples''' | | '''Examples''' | ||
Line 48: | Line 54: | ||
* <code>jobs:list-details()</code> returns information on the currently running job and possibly others: | * <code>jobs:list-details()</code> returns information on the currently running job and possibly others: | ||
<pre class="brush:xml"> | <pre class="brush:xml"> | ||
− | <job id="job1" type="XQuery | + | <job id="job1" type="XQuery" state="running" user="admin" duration="PT0.001S"> |
XQUERY jobs:list-details() | XQUERY jobs:list-details() | ||
</job> | </job> | ||
Line 62: | Line 68: | ||
|- | |- | ||
| '''Summary''' | | '''Summary''' | ||
− | |Indicates if the evaluation of | + | |Indicates if the evaluation of an already running job with the specified {{Code|$id}} has finished. As the ids of finished jobs will usually be discarded, unless caching is enabled, the function will also return <code>true</code> for unknown jobs. |
− | + | * <code>false</code> indicates that the job id is scheduled, queued, or currently running. | |
− | * <code>false</code> indicates that the job id is | + | * <code>true</code> will be returned if the job has either finished, or if the id is unknown (because the ids of all finished jobs will not be cached). |
|} | |} | ||
Line 75: | Line 81: | ||
|- | |- | ||
| '''Summary''' | | '''Summary''' | ||
− | | | + | |Triggeres the cancelation of a job with the specified {{Code|$id}}, drops the cached result of a query, or cancels a scheduled job. Unknown ids are ignored. All jobs are gracefully stopped; it is up to the process to decide when it is safe to shut down. |
+ | |} | ||
+ | |||
+ | ==jobs:wait== | ||
+ | |||
+ | {| width='100%' | ||
+ | |- | ||
+ | | width='120' | '''Signatures''' | ||
+ | |{{Func|jobs:wait|$id as xs:string|empty-sequence()}} | ||
+ | |- | ||
+ | | '''Summary''' | ||
+ | |Waits for the completion of a job with the specified {{Code|$id}}. If the function is called with the id of a queued job, or repeatedly executed job, it may stall and never terminate. | ||
+ | |- | ||
+ | | '''Errors''' | ||
+ | |{{Error|self|#Errors}} if the current job is addressed.<br/> | ||
|} | |} | ||
Line 82: | Line 102: | ||
Asynchronous query execution is recommendable if a client does not, or cannot, wait until a request is fully processed. This is e. g. the case with web browsers, which will usually cancel a request after a specific timeout. In such cases, you can use asynchronous execution to trigger another server-side process, which will start the time-consuming process, and fetch the result later on as soon as it is available. | Asynchronous query execution is recommendable if a client does not, or cannot, wait until a request is fully processed. This is e. g. the case with web browsers, which will usually cancel a request after a specific timeout. In such cases, you can use asynchronous execution to trigger another server-side process, which will start the time-consuming process, and fetch the result later on as soon as it is available. | ||
− | ==jobs: | + | ==jobs:eval== |
+ | |||
+ | {{Mark|Updated with Version 8.6}}: <code>id</code> option added. | ||
{| width='100%' | {| width='100%' | ||
|- | |- | ||
| width='120' | '''Signatures''' | | width='120' | '''Signatures''' | ||
− | |{{Func|jobs: | + | |{{Func|jobs:eval|$query as xs:string|xs:string}}<br />{{Func|jobs:eval|$query as xs:string, $bindings as map(*)?|xs:string}}<br />{{Func|jobs:eval|$query as xs:string, $bindings as map(*)?, $options as map(xs:string, xs:string)|xs:string}}<br /> |
|- | |- | ||
| '''Summary''' | | '''Summary''' | ||
− | |Schedules the | + | |Schedules the evaluation of the supplied {{Code|$query}} and returns a query id. The query will be queued, and the result will optionally be cached. Queries can be updating. Variables and context items can be declared via {{Code|$bindings}} (see [[XQuery Module#xquery:eval|xquery:eval]] for more details). The {{Code|$options}} parameter contains scheduling options: |
* {{Code|cache}}: indicates if the query result will be cached or ignored (default: <code>false</code>): | * {{Code|cache}}: indicates if the query result will be cached or ignored (default: <code>false</code>): | ||
** The result will be cached in main-memory until it is fetched via [[#jobs:result|jobs:result]], or until {{Option|CACHETIMEOUT}} is exceeded. | ** The result will be cached in main-memory until it is fetched via [[#jobs:result|jobs:result]], or until {{Option|CACHETIMEOUT}} is exceeded. | ||
Line 99: | Line 121: | ||
** If a dateTime is specified, the query will be executed at this date. Examples for valid values are: <code>2018-12-31T23:59:59</code> (New Year's Eve 2018, close to midnight). An error will be raised if the specified time lies in the past. | ** If a dateTime is specified, the query will be executed at this date. Examples for valid values are: <code>2018-12-31T23:59:59</code> (New Year's Eve 2018, close to midnight). An error will be raised if the specified time lies in the past. | ||
* {{Code|interval}}: a dayTimeDuration string can be specified to execute the query periodically. An error is raised if the specified interval is less than one second (<code>PT1S</code>). If the next scheduled call is due, and if a query with the same id is still running, it will be skipped. | * {{Code|interval}}: a dayTimeDuration string can be specified to execute the query periodically. An error is raised if the specified interval is less than one second (<code>PT1S</code>). If the next scheduled call is due, and if a query with the same id is still running, it will be skipped. | ||
− | * {{Code|base-uri}}: | + | * {{Code|end}}: scheduling can be stopped after a given time or duration. The string format is the same as for {{Code|start}}. An error is raised if the resulting end time is smaller than the start time. |
+ | * {{Code|base-uri}}: sets the [https://www.w3.org/TR/xquery-31/#dt-static-base-uri base-uri property] for the query. This URI will be used when resolving relative URIs, such as with {{Code|fn:doc}}. | ||
+ | * {{Code|id}}: sets a custom job id. The id must not start with the standard <code>job</code> prefix, and it can only be assigned if no job with the same name exists. | ||
|- | |- | ||
| '''Errors''' | | '''Errors''' | ||
− | |{{Error|overflow|#Errors}} Query execution is rejected, because too many jobs are queued or being executed. {{Option|CACHETIMEOUT}} can be decreased if the default setting is too restrictive.<br/>{{Error|range|#Errors}} A specified time or duration is out of range. | + | |{{Error|overflow|#Errors}} Query execution is rejected, because too many jobs are queued or being executed. {{Option|CACHETIMEOUT}} can be decreased if the default setting is too restrictive.<br/>{{Error|range|#Errors}} A specified time or duration is out of range.<br/>{{Error|id|#Errors}} The specified id is invalid or has already been assigned. |
|- | |- | ||
| '''Examples''' | | '''Examples''' | ||
| | | | ||
− | * Cache query result. | + | * Cache query result. The returned id can be used to pick up the result with [[#jobs:result|jobs:result]]: |
<pre class='brush:xquery'> | <pre class='brush:xquery'> | ||
− | jobs: | + | jobs:eval("1+3", (), map { 'cache': true() }) |
</pre> | </pre> | ||
* A happy birthday mail will be sent at the given date: | * A happy birthday mail will be sent at the given date: | ||
<pre class="brush:xquery"> | <pre class="brush:xquery"> | ||
− | jobs: | + | jobs:eval("import module namespace mail='mail'; mail:send('Happy birthday!')", |
(), map { 'start': '2018-09-01T06:00:00' })}} | (), map { 'start': '2018-09-01T06:00:00' })}} | ||
</pre> | </pre> | ||
− | * The following [[RESTXQ]] functions can be called to execute a | + | * The following [[RESTXQ]] functions can be called to execute a query at 2 am every day. An id will be returned by the first function, which can be used to stop the scheduler via the second function: |
<pre class='brush:xquery'> | <pre class='brush:xquery'> | ||
declare %rest:POST("{$query}") %rest:path('/start-scheduling') function local:start($query) { | declare %rest:POST("{$query}") %rest:path('/start-scheduling') function local:start($query) { | ||
− | jobs: | + | jobs:eval($query, (), map { 'start': '02:00:00', 'interval': 'P1D' }) |
}; | }; | ||
declare %rest:path('/stop-scheduling/{$id}') function local:stop($id) { | declare %rest:path('/stop-scheduling/{$id}') function local:stop($id) { | ||
Line 124: | Line 148: | ||
}; | }; | ||
</pre> | </pre> | ||
− | * Query execution is scheduled for every second. As the query itself will take 1.5 seconds, it will only be executed every second time: | + | * Query execution is scheduled for every second, and for 10 seconds in total. As the query itself will take 1.5 seconds, it will only be executed every second time: |
<pre class="brush:xquery"> | <pre class="brush:xquery"> | ||
− | jobs: | + | jobs:eval("prof:sleep(1500)", (), map { 'interval': 'PT1S', 'end': 'PT10S' }) |
</pre> | </pre> | ||
|} | |} | ||
Line 138: | Line 162: | ||
|- | |- | ||
| '''Summary''' | | '''Summary''' | ||
− | |Returns the cached result of a | + | |Returns the cached result of a job with the specified job {{Code|$id}}: |
* Results can only be retrieved once. After retrieval, the cached result will be dropped. | * Results can only be retrieved once. After retrieval, the cached result will be dropped. | ||
− | * If the original | + | * If the original job has raised an error, the cached error will be raised instead. |
|- | |- | ||
| '''Errors''' | | '''Errors''' | ||
Line 147: | Line 171: | ||
| '''Examples''' | | '''Examples''' | ||
| | | | ||
− | * The following [[RESTXQ]] function will either return the result of a previously started | + | * The following [[RESTXQ]] function will either return the result of a previously started job or raise an error: |
<pre class='brush:xquery'> | <pre class='brush:xquery'> | ||
declare %rest:path('/result/{$id}') function local:result($id) { | declare %rest:path('/result/{$id}') function local:result($id) { | ||
Line 153: | Line 177: | ||
}; | }; | ||
</pre> | </pre> | ||
− | * The following query demonstrates how the results of an asynchronously executed query can be returned within the same query | + | * The following query demonstrates how the results of an asynchronously executed query can be returned within the same query: |
<pre class='brush:xquery'> | <pre class='brush:xquery'> | ||
− | let $query := jobs: | + | let $query := jobs:eval('(1 to 10000000)[. = 1]', map{}, map{ 'cache': true() }) |
return ( | return ( | ||
− | + | jobs:wait($query), | |
− | |||
− | |||
− | |||
− | |||
jobs:result($query) | jobs:result($query) | ||
) | ) | ||
</pre> | </pre> | ||
− | + | Please note that this query can easily cause a deadlock if the asynchronously executed query will be queued. In practice. In practice, you should avoid this pattern and resort to [[XQuery Module#xquery:fork-join|xquery:fork-join]] if you want to do things in parallel. | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
|} | |} | ||
Line 193: | Line 203: | ||
| Too many queries or query results are queued. | | Too many queries or query results are queued. | ||
|- | |- | ||
− | {{Code|range}} | + | |{{Code|range}} |
| A specified time or duration is out of range. | | A specified time or duration is out of range. | ||
+ | |- | ||
+ | |{{Code|id}} | ||
+ | | The specified is invalid or has already been assigned. | ||
+ | |- | ||
+ | |{{Code|self}} | ||
+ | | The current job cannot be addressed. | ||
|} | |} | ||
=Changelog= | =Changelog= | ||
+ | |||
+ | ;Version 8.6 | ||
+ | |||
+ | * Updated: [[#jobs:eval|jobs:eval]]: <code>id</code> option added. | ||
The module was introduced with Version 8.5. | The module was introduced with Version 8.5. |
Revision as of 11:27, 4 February 2017
This XQuery Module provides functions for organizing scheduled, queued, running and cached jobs. Jobs can be commands, queries, client or HTTP requests.
Contents
Conventions
All functions in this module are assigned to the http://basex.org/modules/jobs
namespace, which is statically bound to the jobs
prefix. Errors will be bound to the same prefix.
Functions
jobs:current
Signatures | jobs:current() as xs:string
|
Summary | Returns the id of the current job. |
jobs:list
Signatures | jobs:list() as xs:string*
|
Summary | Returns the ids of all jobs that are currently registered. The list includes scheduled, queued, running jobs, and cached jobs. |
Examples |
|
jobs:list-details
Signatures | jobs:list-details() as element(job)* jobs:list-details($id as xs:string) as element(job)*
|
Summary | Returns information on all jobs that are currently registered. The list includes scheduled, queued, running jobs, and cached jobs. A string representation of the job will be returned in the text node. The returned elements have additional attributes:
|
Examples |
<job id="job1" type="XQuery" state="running" user="admin" duration="PT0.001S"> XQUERY jobs:list-details() </job> |
jobs:finished
Signatures | jobs:finished($id as xs:string) as xs:boolean
|
Summary | Indicates if the evaluation of an already running job with the specified $id has finished. As the ids of finished jobs will usually be discarded, unless caching is enabled, the function will also return true for unknown jobs.
|
jobs:stop
Signatures | jobs:stop($id as xs:string) as empty-sequence()
|
Summary | Triggeres the cancelation of a job with the specified $id , drops the cached result of a query, or cancels a scheduled job. Unknown ids are ignored. All jobs are gracefully stopped; it is up to the process to decide when it is safe to shut down.
|
jobs:wait
Signatures | jobs:wait($id as xs:string) as empty-sequence()
|
Summary | Waits for the completion of a job with the specified $id . If the function is called with the id of a queued job, or repeatedly executed job, it may stall and never terminate.
|
Errors | self : if the current job is addressed. |
Asynchronous Execution
Asynchronous query execution is recommendable if a client does not, or cannot, wait until a request is fully processed. This is e. g. the case with web browsers, which will usually cancel a request after a specific timeout. In such cases, you can use asynchronous execution to trigger another server-side process, which will start the time-consuming process, and fetch the result later on as soon as it is available.
jobs:eval
Template:Mark: id
option added.
Signatures | jobs:eval($query as xs:string) as xs:string jobs:eval($query as xs:string, $bindings as map(*)?) as xs:string jobs:eval($query as xs:string, $bindings as map(*)?, $options as map(xs:string, xs:string)) as xs:string |
Summary | Schedules the evaluation of the supplied $query and returns a query id. The query will be queued, and the result will optionally be cached. Queries can be updating. Variables and context items can be declared via $bindings (see xquery:eval for more details). The $options parameter contains scheduling options:
|
Errors | overflow : Query execution is rejected, because too many jobs are queued or being executed. CACHETIMEOUT can be decreased if the default setting is too restrictive.range : A specified time or duration is out of range.id : The specified id is invalid or has already been assigned.
|
Examples |
jobs:eval("1+3", (), map { 'cache': true() })
jobs:eval("import module namespace mail='mail'; mail:send('Happy birthday!')", (), map { 'start': '2018-09-01T06:00:00' })}}
declare %rest:POST("{$query}") %rest:path('/start-scheduling') function local:start($query) { jobs:eval($query, (), map { 'start': '02:00:00', 'interval': 'P1D' }) }; declare %rest:path('/stop-scheduling/{$id}') function local:stop($id) { jobs:stop($id) };
jobs:eval("prof:sleep(1500)", (), map { 'interval': 'PT1S', 'end': 'PT10S' }) |
jobs:result
Signatures | jobs:result($id as xs:string) as item()*
|
Summary | Returns the cached result of a job with the specified job $id :
|
Errors | running : the job is still running.unknown : the supplied id is unknown: The id is unknown, or the result has already been retrieved. |
Examples |
declare %rest:path('/result/{$id}') function local:result($id) { jobs:result($id) };
let $query := jobs:eval('(1 to 10000000)[. = 1]', map{}, map{ 'cache': true() }) return ( jobs:wait($query), jobs:result($query) ) Please note that this query can easily cause a deadlock if the asynchronously executed query will be queued. In practice. In practice, you should avoid this pattern and resort to xquery:fork-join if you want to do things in parallel. |
Errors
Code | Description |
---|---|
unknown
|
The supplied query id is unknown or not available anymore. |
running
|
A query is still running. |
overflow
|
Too many queries or query results are queued. |
range
|
A specified time or duration is out of range. |
id
|
The specified is invalid or has already been assigned. |
self
|
The current job cannot be addressed. |
Changelog
- Version 8.6
- Updated: jobs:eval:
id
option added.
The module was introduced with Version 8.5.