Difference between revisions of "Job Module"
(19 intermediate revisions by 2 users not shown) | |||
Line 5: | Line 5: | ||
All functions in this module are assigned to the <code><nowiki>http://basex.org/modules/jobs</nowiki></code> namespace, which is statically bound to the {{Code|jobs}} prefix. Errors will be bound to the same prefix. | All functions in this module are assigned to the <code><nowiki>http://basex.org/modules/jobs</nowiki></code> namespace, which is statically bound to the {{Code|jobs}} prefix. Errors will be bound to the same prefix. | ||
− | =Functions= | + | =Services= |
+ | |||
+ | A job can be registered as ''service'' by supplying the {{Code|service}} option to {{Function|Jobs|jobs:eval}}: | ||
+ | |||
+ | <pre class="brush:xquery"> | ||
+ | (: register job as service; will be run every day at 1 am :) | ||
+ | jobs:eval('db:drop("tmp")', (), map { 'id':'cleanup', 'start':'01:00:00', 'interval':'P1D', 'service': true() }), | ||
+ | |||
+ | (: list registered services :) | ||
+ | jobs:services(), | ||
+ | (: result: <job base-uri="..." id="cleanup" interval="P1D" start="01:00:00">db:drop("tmp")</job> :) | ||
+ | |||
+ | (: unregister job :) | ||
+ | jobs:stop('cleanup', map { 'service': true() }) | ||
+ | </pre> | ||
+ | |||
+ | '''Some more notes:''' | ||
+ | |||
+ | * All job services will be scheduled for evaluation when the BaseX server or BaseX HTTP server is started. | ||
+ | * If a job service is outdated (e.g. because a supplied end time has been exceeded), it will be removed from the jobs file at startup time. | ||
+ | * Job services can be updated: If a new job is registered, and if there is already a job with the same id, the old entry will be replaced. | ||
+ | * The job definitions are stored in a {{Code|jobs.xml}} file in the database directory. It can also be edited manually. | ||
+ | |||
+ | =Basic Functions= | ||
==jobs:current== | ==jobs:current== | ||
Line 26: | Line 49: | ||
|- | |- | ||
| '''Summary''' | | '''Summary''' | ||
− | |Returns the ids of all jobs that are currently registered. The list includes scheduled, queued, running, stopped, and | + | |Returns the ids of all jobs that are currently registered. The list includes scheduled, queued, running, stopped, and finished jobs with cached results. |
|- | |- | ||
| '''Examples''' | | '''Examples''' | ||
Line 43: | Line 66: | ||
* <code>id</code>: job id | * <code>id</code>: job id | ||
* <code>type</code>: type of the job (command, query, REST, RESTXQ, etc.) | * <code>type</code>: type of the job (command, query, REST, RESTXQ, etc.) | ||
− | * <code>state</code>: current state of the job | + | * <code>state</code>: current state of the job: <code>scheduled</code>, <code>queued</code>, <code>running</code>, <code>cached</code> |
− | * <code>user</code>: | + | * <code>user</code>: user who started the job |
− | * <code>duration</code>: evaluation time ( | + | * <code>duration</code>: evaluation time (included if a job is running or if the result was cached) |
− | * <code>start</code>: | + | * <code>start</code>: next start of job (included if a job will be executed repeatedly) |
+ | * <code>time</code>: time when job was registered | ||
|- | |- | ||
| '''Examples''' | | '''Examples''' | ||
− | | | + | | <code>jobs:list-details()</code> returns information on the currently running job and possibly others: |
− | |||
<pre class="brush:xml"> | <pre class="brush:xml"> | ||
<job id="job1" type="XQuery" state="running" user="admin" duration="PT0.001S"> | <job id="job1" type="XQuery" state="running" user="admin" duration="PT0.001S"> | ||
Line 71: | Line 94: | ||
|} | |} | ||
− | ==jobs: | + | ==jobs:services== |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
{| width='100%' | {| width='100%' | ||
|- | |- | ||
| width='120' | '''Signatures''' | | width='120' | '''Signatures''' | ||
− | |{{Func|jobs: | + | |{{Func|jobs:services||element(job)*}} |
|- | |- | ||
| '''Summary''' | | '''Summary''' | ||
− | | | + | |Returns a list of all jobs that have been persistently registered as [[#Services|Services]]. |
− | |||
− | |||
|- | |- | ||
| '''Errors''' | | '''Errors''' | ||
− | |{{Error| | + | |{{Error|services|#Errors}} Registered services cannot be parsed.<br/> |
|} | |} | ||
− | = | + | =Execution= |
− | + | There are cases in which a client does not, or cannot, wait until a request is fully processed. The client may be a browser, which sends an HTTP request to the server in order to start another time-consuming query job. The functions in this section allow you to register a new query job from a running query. Jobs can be executed immediately (i.e., as soon as the [[Transaction Management#Concurrency Control|Concurrency Control]] allows it) or scheduled for repeated execution. Each registered job gets a job id, and the id can be used to retrieve a query result, stop a job, or wait for its termination. | |
==jobs:eval== | ==jobs:eval== | ||
+ | |||
+ | {{Mark|Updated with 9.2}}: First argument can be a URI (jobs:invoke was removed). | ||
{| width='100%' | {| width='100%' | ||
|- | |- | ||
| width='120' | '''Signatures''' | | width='120' | '''Signatures''' | ||
− | |{{Func|jobs:eval|$query as xs: | + | |{{Func|jobs:eval|$query as xs:anyAtomicItem|xs:string}}<br />{{Func|jobs:eval|$query as xs:anyAtomicItem, $bindings as map(*)?|xs:string}}<br />{{Func|jobs:eval|$query as xs:anyAtomicItem, $bindings as map(*)?, $options as map(*)?|xs:string}}<br /> |
|- | |- | ||
| '''Summary''' | | '''Summary''' | ||
− | |Schedules the evaluation of the supplied {{Code|$query}} and returns a query id. The query will be queued, and the result will optionally be cached. Queries can be updating. | + | |Schedules the evaluation of the supplied {{Code|$query}} and returns a query id. The query will be queued, and the result will optionally be cached. Queries can be updating. The query can be a URI or a string, and variables and context items can be declared via {{Code|$bindings}} (see [[XQuery Module#xquery:eval|xquery:eval]] for more details). The following {{Code|$options}} can be supplied: |
* {{Code|cache}}: indicates if the query result will be cached or ignored (default: <code>false</code>): | * {{Code|cache}}: indicates if the query result will be cached or ignored (default: <code>false</code>): | ||
** The result will be cached in main-memory until it is fetched via [[#jobs:result|jobs:result]], or until {{Option|CACHETIMEOUT}} is exceeded. | ** The result will be cached in main-memory until it is fetched via [[#jobs:result|jobs:result]], or until {{Option|CACHETIMEOUT}} is exceeded. | ||
Line 125: | Line 134: | ||
* {{Code|base-uri}}: sets the [https://www.w3.org/TR/xquery-31/#dt-static-base-uri base-uri property] for the query. This URI will be used when resolving relative URIs, such as with {{Code|fn:doc}}. | * {{Code|base-uri}}: sets the [https://www.w3.org/TR/xquery-31/#dt-static-base-uri base-uri property] for the query. This URI will be used when resolving relative URIs, such as with {{Code|fn:doc}}. | ||
* {{Code|id}}: sets a custom job id. The id must not start with the standard <code>job</code> prefix, and it can only be assigned if no job with the same name exists. | * {{Code|id}}: sets a custom job id. The id must not start with the standard <code>job</code> prefix, and it can only be assigned if no job with the same name exists. | ||
+ | * {{Code|service}}: additionally registers the job as [[#Services|service]]. Registered services must have no variable bindings. | ||
|- | |- | ||
| '''Errors''' | | '''Errors''' | ||
Line 153: | Line 163: | ||
jobs:eval("prof:sleep(1500)", (), map { 'interval': 'PT1S', 'end': 'PT10S' }) | jobs:eval("prof:sleep(1500)", (), map { 'interval': 'PT1S', 'end': 'PT10S' }) | ||
</pre> | </pre> | ||
− | * The following expression, if stored | + | * The query in the specified file will be evaluated once: |
+ | <pre class='brush:xquery'> | ||
+ | jobs:eval(xs:anyURI('cleanup.xq')) | ||
+ | </pre> | ||
+ | * The following expression, if stored in a file, will be evaluated every 5 seconds: | ||
<pre class="brush:xquery"> | <pre class="brush:xquery"> | ||
jobs:eval( | jobs:eval( | ||
− | + | static-base-uri(), | |
map { }, | map { }, | ||
map { 'start': 'PT5S' } | map { 'start': 'PT5S' } | ||
) | ) | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
</pre> | </pre> | ||
|} | |} | ||
Line 209: | Line 200: | ||
}; | }; | ||
</pre> | </pre> | ||
− | * The following query demonstrates how the results of an | + | * The following query demonstrates how the results of an executed query can be returned within the same query (see below why you should avoid this pattern in practice): |
<pre class='brush:xquery'> | <pre class='brush:xquery'> | ||
− | let $query := jobs:eval('(1 to 10000000)[. = 1]', map{}, map{ 'cache': true() }) | + | let $query := jobs:eval('(1 to 10000000)[. = 1]', map { }, map { 'cache': true() }) |
return ( | return ( | ||
jobs:wait($query), | jobs:wait($query), | ||
Line 217: | Line 208: | ||
) | ) | ||
</pre> | </pre> | ||
− | + | Queries of this kind can cause deadlocks! If the original query and the new query perform updates on the same database, the second query will only be run after the first one has been executed, and the first query will wait for the second query forever. You should resort to [[XQuery Module#xquery:fork-join|xquery:fork-join]] if you want to have full control on parallel query execution. | |
+ | |} | ||
+ | |||
+ | ==jobs:stop== | ||
+ | |||
+ | {| width='100%' | ||
+ | |- | ||
+ | | width='120' | '''Signatures''' | ||
+ | |{{Func|jobs:stop|$id as xs:string|empty-sequence()}} | ||
+ | |- | ||
+ | | '''Summary''' | ||
+ | |Triggers the cancelation of a job with the specified {{Code|$id}}, drops the cached result of a query, or cancels a scheduled job. Unknown ids are ignored. All jobs are gracefully stopped; it is up to the process to decide when it is safe to shut down. The following {{Code|$options}} can be supplied: | ||
+ | * {{Code|service}}: additionally removes the job from the [[#Services|job services]] list. | ||
+ | |- | ||
+ | | '''Examples''' | ||
+ | | <code>jobs:list()[. != jobs:current()] ! jobs:stop(.)</code> stops and discards all jobs except for the current one. | ||
+ | |} | ||
+ | |||
+ | ==jobs:wait== | ||
+ | |||
+ | {| width='100%' | ||
+ | |- | ||
+ | | width='120' | '''Signatures''' | ||
+ | |{{Func|jobs:wait|$id as xs:string|empty-sequence()}} | ||
+ | |- | ||
+ | | '''Summary''' | ||
+ | |Waits for the completion of a job with the specified {{Code|$id}}: | ||
+ | * The function will terminate immediately if the job id is unknown. This is the case if a future job has not been queued yet, or if the id has already been discarded after job evaluation. | ||
+ | * If the function is called with the id of a queued job, or repeatedly executed job, it may stall and never terminate. | ||
+ | |- | ||
+ | | '''Errors''' | ||
+ | |{{Error|self|#Errors}} The current job is addressed.<br/> | ||
|} | |} | ||
Line 243: | Line 265: | ||
|{{Code|self}} | |{{Code|self}} | ||
| The current job cannot be addressed. | | The current job cannot be addressed. | ||
+ | |- | ||
+ | |{{Code|service}} | ||
+ | | Registered services cannot be parsed, added or removed. | ||
|- | |- | ||
|{{Code|unknown}} | |{{Code|unknown}} | ||
Line 249: | Line 274: | ||
=Changelog= | =Changelog= | ||
+ | |||
+ | ;Version 9.2 | ||
+ | |||
+ | * Deleted: jobs:invoke (merged with [[#jobs:eval|jobs:eval]]) | ||
+ | |||
+ | ;Version 9.1 | ||
+ | |||
+ | * Updated: {{Function|Jobs|jobs:list-details}}: registration time added. | ||
;Version 9.0 | ;Version 9.0 | ||
− | * Added: [[# | + | * Added: {{Function|Jobs|jobs:invoke}}, [[#Services|Services]] |
;Version 8.6 | ;Version 8.6 |
Revision as of 13:56, 8 July 2019
This XQuery Module provides functions for organizing scheduled, queued, running and cached jobs. Jobs can be commands, queries, client or HTTP requests.
Contents
Conventions
All functions in this module are assigned to the http://basex.org/modules/jobs
namespace, which is statically bound to the jobs
prefix. Errors will be bound to the same prefix.
Services
A job can be registered as service by supplying the service
option to jobs:eval
:
(: register job as service; will be run every day at 1 am :) jobs:eval('db:drop("tmp")', (), map { 'id':'cleanup', 'start':'01:00:00', 'interval':'P1D', 'service': true() }), (: list registered services :) jobs:services(), (: result: <job base-uri="..." id="cleanup" interval="P1D" start="01:00:00">db:drop("tmp")</job> :) (: unregister job :) jobs:stop('cleanup', map { 'service': true() })
Some more notes:
- All job services will be scheduled for evaluation when the BaseX server or BaseX HTTP server is started.
- If a job service is outdated (e.g. because a supplied end time has been exceeded), it will be removed from the jobs file at startup time.
- Job services can be updated: If a new job is registered, and if there is already a job with the same id, the old entry will be replaced.
- The job definitions are stored in a
jobs.xml
file in the database directory. It can also be edited manually.
Basic Functions
jobs:current
Signatures | jobs:current() as xs:string
|
Summary | Returns the id of the current job. |
jobs:list
Signatures | jobs:list() as xs:string*
|
Summary | Returns the ids of all jobs that are currently registered. The list includes scheduled, queued, running, stopped, and finished jobs with cached results. |
Examples | jobs:list() returns the same job id as jobs:current if no other job is registered.
|
jobs:list-details
Signatures | jobs:list-details() as element(job)* jobs:list-details($id as xs:string) as element(job)*
|
Summary | Returns information on all jobs that are currently registered, or on a job with the specified $id (or an empty sequence if this job is not found). The list includes scheduled, queued, running jobs, and cached jobs. A string representation of the job, or its URI, will be returned as value. The returned elements have additional attributes:
|
Examples | jobs:list-details() returns information on the currently running job and possibly others:
<job id="job1" type="XQuery" state="running" user="admin" duration="PT0.001S"> XQUERY jobs:list-details() </job> |
jobs:finished
Signatures | jobs:finished($id as xs:string) as xs:boolean
|
Summary | Indicates if the evaluation of an already running job with the specified $id has finished. As the ids of finished jobs will usually be discarded, unless caching is enabled, the function will also return true for unknown jobs.
|
jobs:services
Signatures | jobs:services() as element(job)*
|
Summary | Returns a list of all jobs that have been persistently registered as Services. |
Errors | services : Registered services cannot be parsed. |
Execution
There are cases in which a client does not, or cannot, wait until a request is fully processed. The client may be a browser, which sends an HTTP request to the server in order to start another time-consuming query job. The functions in this section allow you to register a new query job from a running query. Jobs can be executed immediately (i.e., as soon as the Concurrency Control allows it) or scheduled for repeated execution. Each registered job gets a job id, and the id can be used to retrieve a query result, stop a job, or wait for its termination.
jobs:eval
Template:Mark: First argument can be a URI (jobs:invoke was removed).
Signatures | jobs:eval($query as xs:anyAtomicItem) as xs:string jobs:eval($query as xs:anyAtomicItem, $bindings as map(*)?) as xs:string jobs:eval($query as xs:anyAtomicItem, $bindings as map(*)?, $options as map(*)?) as xs:string |
Summary | Schedules the evaluation of the supplied $query and returns a query id. The query will be queued, and the result will optionally be cached. Queries can be updating. The query can be a URI or a string, and variables and context items can be declared via $bindings (see xquery:eval for more details). The following $options can be supplied:
|
Errors | overflow : Query execution is rejected, because too many jobs are queued or being executed. CACHETIMEOUT can be decreased if the default setting is too restrictive.range : A specified time or duration is out of range.id : The specified id is invalid or has already been assigned.options : The specified options are conflicting.
|
Examples |
jobs:eval("1+3", (), map { 'cache': true() })
jobs:eval("import module namespace mail='mail'; mail:send('Happy birthday!')", (), map { 'start': '2018-09-01T06:00:00' })}}
declare %rest:POST("{$query}") %rest:path('/start-scheduling') function local:start($query) { jobs:eval($query, (), map { 'start': '02:00:00', 'interval': 'P1D' }) }; declare %rest:path('/stop-scheduling/{$id}') function local:stop($id) { jobs:stop($id) };
jobs:eval("prof:sleep(1500)", (), map { 'interval': 'PT1S', 'end': 'PT10S' })
jobs:eval(xs:anyURI('cleanup.xq'))
jobs:eval( static-base-uri(), map { }, map { 'start': 'PT5S' } ) |
jobs:result
Signatures | jobs:result($id as xs:string) as item()*
|
Summary | Returns the cached result of a job with the specified job $id :
|
Errors | running : the job is still running.unknown : the supplied id is unknown: The id is unknown, or the result has already been retrieved. |
Examples |
declare %rest:path('/result/{$id}') function local:result($id) { jobs:result($id) };
let $query := jobs:eval('(1 to 10000000)[. = 1]', map { }, map { 'cache': true() }) return ( jobs:wait($query), jobs:result($query) ) Queries of this kind can cause deadlocks! If the original query and the new query perform updates on the same database, the second query will only be run after the first one has been executed, and the first query will wait for the second query forever. You should resort to xquery:fork-join if you want to have full control on parallel query execution. |
jobs:stop
Signatures | jobs:stop($id as xs:string) as empty-sequence()
|
Summary | Triggers the cancelation of a job with the specified $id , drops the cached result of a query, or cancels a scheduled job. Unknown ids are ignored. All jobs are gracefully stopped; it is up to the process to decide when it is safe to shut down. The following $options can be supplied:
|
Examples | jobs:list()[. != jobs:current()] ! jobs:stop(.) stops and discards all jobs except for the current one.
|
jobs:wait
Signatures | jobs:wait($id as xs:string) as empty-sequence()
|
Summary | Waits for the completion of a job with the specified $id :
|
Errors | self : The current job is addressed. |
Errors
Code | Description |
---|---|
options
|
The specified options are conflicting. |
id
|
The specified id is invalid or has already been assigned. |
overflow
|
Too many queries or query results are queued. |
range
|
A specified time or duration is out of range. |
running
|
A query is still running. |
self
|
The current job cannot be addressed. |
service
|
Registered services cannot be parsed, added or removed. |
unknown
|
The supplied query id is unknown or not available anymore. |
Changelog
- Version 9.2
- Deleted: jobs:invoke (merged with jobs:eval)
- Version 9.1
- Updated:
jobs:list-details
: registration time added.
- Version 9.0
- Added:
jobs:invoke
, Services
- Version 8.6
- Updated: jobs:eval:
id
option added.
The module was introduced with Version 8.5.