Difference between revisions of "Lazy Module"
m (moved Streaming Module to Lazy Module) |
|||
Line 1: | Line 1: | ||
− | This [[Module Library|XQuery Module]] contains functions for handling '' | + | This [[Module Library|XQuery Module]] contains functions for handling ''lazy'' items. |
− | In contrast to standard XQuery items, a | + | In contrast to standard XQuery items, a lazy item contains a reference to the actual data, and the data itself will only be retrieved if it is requested. Hence, possible errors will be postponed, and no memory will be occupied by a lazy item as long as its contents has not been reqested yet. |
− | The following BaseX functions return | + | The following BaseX functions return lazy items: |
− | * | + | * Lazy Base64 binaries: |
** <code>[[Database Module#db:retrieve|db:retrieve]]</code> | ** <code>[[Database Module#db:retrieve|db:retrieve]]</code> | ||
** <code>[[Fetch Module#fetch:binary|fetch:binary]]</code> | ** <code>[[Fetch Module#fetch:binary|fetch:binary]]</code> | ||
** <code>[[File Module#file:read-binary|file:read-binary]]</code> | ** <code>[[File Module#file:read-binary|file:read-binary]]</code> | ||
− | * | + | * Lazy strings: |
** <code>[[Fetch Module#fetch:text|fetch:text]]</code> | ** <code>[[Fetch Module#fetch:text|fetch:text]]</code> | ||
** <code>[[File Module#file:read-text|file:read-text]]</code> | ** <code>[[File Module#file:read-text|file:read-text]]</code> | ||
− | Some functions are capable of consuming items in a ''streamable'' fashion: data will | + | Some functions are capable of consuming the contents of lazy items in a ''streamable'' fashion: data will not be cached, but instead passed on to another target (file, the calling expression, etc.). The following streaming functions are currently available: |
− | * <code>[[Conversion Module#convert:binary-to-bytes|convert:binary-to-bytes]]</code> | + | * [[Archive Module]] (most functions) |
− | * <code>[[Database Module#db:store|db:store]]</code> | + | * Conversion Module: <code>[[Conversion Module#convert:binary-to-bytes|convert:binary-to-bytes]]</code>, <code>[[Conversion Module#convert:binary-to-string|convert:binary-to-string]]</code> |
− | * <code>[[File Module#file:write-binary|file:write-binary]]</code> | + | * Database Module: <code>[[Database Module#db:store|db:store]]</code> |
− | + | * File Module: <code>[[File Module#file:write-binary|file:write-binary]]</code>, <code>[[File Module#file:write-text|file:write-text]] </code> (if no encoding is specified) | |
+ | * [[Hashing Module]] (all functions) | ||
The XQuery expression below serves as an example on how large files can be downloaded and written to a file with constant memory consumption: | The XQuery expression below serves as an example on how large files can be downloaded and written to a file with constant memory consumption: | ||
Line 26: | Line 27: | ||
file:write-binary('output.data', fetch:binary('http://files.basex.org/xml/xmark111mb.zip')) | file:write-binary('output.data', fetch:binary('http://files.basex.org/xml/xmark111mb.zip')) | ||
</pre> | </pre> | ||
+ | |||
+ | If lazy items are serialized, they will be streamed as well. | ||
=Conventions= | =Conventions= | ||
− | All functions in this module are assigned to the <code><nowiki>http://basex.org/modules/ | + | All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/lazy</nowiki></code> namespace, which is statically bound to the {{Code|lazy}} prefix.<br/> |
− | |||
=Functions= | =Functions= | ||
− | == | + | ==lazy:cache== |
{| width='100%' | {| width='100%' | ||
|- | |- | ||
| width='120' | '''Signatures''' | | width='120' | '''Signatures''' | ||
− | |{{Func| | + | |{{Func|lazy:cache|$item as item()|item()}} |
|- | |- | ||
| '''Summary''' | | '''Summary''' | ||
− | | | + | |Caches the contents of a lazy {{Code|$item}}:<br /> |
− | * | + | * contents of lazy items will be retrieved and cached in the item. |
− | * | + | * non-lazy items or lazy items with cached contents will simply be passed through. |
− | + | Caching of a lazy items is advisable if the the targeted contents will not be available anymore at a later stage. | |
|- | |- | ||
| '''Example''' | | '''Example''' | ||
− | |In the following example, a file will be deleted before its content is returned. To avoid a “file not found” error, the content | + | |In the following example, a file will be deleted before its content is returned. To avoid a “file not found” error when serializing the result, the content must be cached: |
<pre class="brush:xquery"> | <pre class="brush:xquery"> | ||
let $file := 'data.txt' | let $file := 'data.txt' | ||
− | let $ | + | let $text := lazy:cache(file:read-text($file)) |
− | return (file:delete($file), $ | + | return (file:delete($file), $text) |
</pre> | </pre> | ||
|} | |} | ||
− | == | + | ==lazy:is-lazy== |
+ | |||
{| width='100%' | {| width='100%' | ||
|- | |- | ||
| width='120' | '''Signatures''' | | width='120' | '''Signatures''' | ||
− | |{{Func|stream:is- | + | |{{Func|stream:is-lazy|$item as item()|xs:boolean}} |
|- | |- | ||
| '''Summary''' | | '''Summary''' | ||
− | |Checks whether the specified {{Code|$item}} is | + | |Checks whether the specified {{Code|$item}} is lazy. |
+ | |} | ||
+ | |||
+ | ==lazy:is-cached== | ||
+ | |||
+ | {| width='100%' | ||
+ | |- | ||
+ | | width='120' | '''Signatures''' | ||
+ | |{{Func|stream:is-cached|$item as item()|xs:boolean}} | ||
+ | |- | ||
+ | | '''Summary''' | ||
+ | |Checks whether the contents of the specified {{Code|$item}} are cached. The function will always return {{Code|true}} for non-lazy items. | ||
|} | |} | ||
=Changelog= | =Changelog= | ||
+ | |||
+ | ;Version 9.0 | ||
+ | |||
+ | * Updated: Renamed from Streaming Module to Lazy Module. | ||
+ | * Added: [[#lazy:is-cached|lazy:is-cached]] | ||
;Version 8.0 | ;Version 8.0 | ||
− | * | + | |
+ | * Updated: [[#stream:materialize|stream:materialize]] extended to sequences. | ||
This module was introduced with Version 7.7. | This module was introduced with Version 7.7. |
Revision as of 17:50, 21 November 2017
This XQuery Module contains functions for handling lazy items.
In contrast to standard XQuery items, a lazy item contains a reference to the actual data, and the data itself will only be retrieved if it is requested. Hence, possible errors will be postponed, and no memory will be occupied by a lazy item as long as its contents has not been reqested yet.
The following BaseX functions return lazy items:
- Lazy Base64 binaries:
- Lazy strings:
Some functions are capable of consuming the contents of lazy items in a streamable fashion: data will not be cached, but instead passed on to another target (file, the calling expression, etc.). The following streaming functions are currently available:
- Archive Module (most functions)
- Conversion Module:
convert:binary-to-bytes
,convert:binary-to-string
- Database Module:
db:store
- File Module:
file:write-binary
,file:write-text
(if no encoding is specified) - Hashing Module (all functions)
The XQuery expression below serves as an example on how large files can be downloaded and written to a file with constant memory consumption:
file:write-binary('output.data', fetch:binary('http://files.basex.org/xml/xmark111mb.zip'))
If lazy items are serialized, they will be streamed as well.
Conventions
All functions and errors in this module are assigned to the http://basex.org/modules/lazy
namespace, which is statically bound to the lazy
prefix.
Functions
lazy:cache
Signatures | lazy:cache($item as item()) as item()
|
Summary | Caches the contents of a lazy $item :
Caching of a lazy items is advisable if the the targeted contents will not be available anymore at a later stage. |
Example | In the following example, a file will be deleted before its content is returned. To avoid a “file not found” error when serializing the result, the content must be cached:
let $file := 'data.txt' let $text := lazy:cache(file:read-text($file)) return (file:delete($file), $text) |
lazy:is-lazy
Signatures | stream:is-lazy($item as item()) as xs:boolean
|
Summary | Checks whether the specified $item is lazy.
|
lazy:is-cached
Signatures | stream:is-cached($item as item()) as xs:boolean
|
Summary | Checks whether the contents of the specified $item are cached. The function will always return true for non-lazy items.
|
Changelog
- Version 9.0
- Updated: Renamed from Streaming Module to Lazy Module.
- Added: lazy:is-cached
- Version 8.0
- Updated: stream:materialize extended to sequences.
This module was introduced with Version 7.7.