Difference between revisions of "Lazy Module"

From BaseX Documentation
Jump to navigation Jump to search
m (Text replacement - "</syntaxhighlight>" to "</pre>")
Tags: Mobile web edit Mobile edit
 
(41 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
This [[Module Library|XQuery Module]] contains functions for handling ''streamable'' items.
+
This [[Module Library|XQuery Module]] contains functions for handling ''lazy'' items.
  
−
In contrast to standard XQuery items, streamable items may take up much less space, because they only contain a reference to the actual data. The data itself will only be retrieved if it is required by another expression, or if an item is serialized. Serialization of streamable items takes constant space.
+
In contrast to standard XQuery items, a lazy item contains a reference to the actual data, and the data itself will only be retrieved if it is processed. Hence, possible errors will be postponed, and no memory will be occupied by a lazy item as long as its content has not been requested yet.
  
−
The following BaseX functions return streamable items:
+
The following BaseX functions return lazy items:
  
−
* <code>[[Database Module#db:retrieve|db:retrieve]]</code>
+
* Lazy Base64 binaries:
−
* <code>[[Fetch Module#fetch:binary|fetch:binary]]</code>
+
** {{Function|Fetch|fetch:binary}}
−
* <code>[[Fetch Module#fetch:text|fetch:text]]</code>
+
** {{Function|File|file:read-binary}}
−
* <code>[[File Module#file:read-binary|file:read-binary]]</code>
+
** {{Function|Database|db:get-binary}}
−
* <code>[[File Module#file:read-text|file:read-text]]</code>
 
  
−
Some functions are capable of consuming items in a ''streamable'' fashion: data will never be cached, but instead passed on to another target (file, the calling expression, etc.). The following streaming functions are currently available:
+
* Lazy strings:
 +
** {{Function|Fetch|fetch:text}}
 +
** {{Function|File|file:read-text}}
  
−
* <code>[[Conversion Module#convert:binary-to-bytes|convert:binary-to-bytes]]</code>
+
Some functions are capable of consuming the contents of lazy items in a ''streamable'' fashion: data will not be cached, but instead passed on to another target (file, the calling expression, etc.). The following streaming functions are currently available:
−
* <code>[[Database Module#db:store|db:store]]</code>
 
−
* <code>[[File Module#file:write-binary|file:write-binary]]</code>
 
−
* <code>[[Fetch Module#file:write-text|file:write-text]]</code>
 
  
−
The XQuery expression serves as an example on how large files can be downloaded and written to a file with constant memory consumption:
+
* [[Archive Module]] (most functions)
 +
* Conversion Module: {{Function|Conversion|convert:binary-to-string}}
 +
* File Module: {{Function|File|file:write-binary-text}}, {{Function|File|file:write-text}} (if no encoding is specified)
 +
* Database Module: {{Function|Database|db:put-binary}}
 +
* [[Hashing Module]] (all functions)
  
−
<pre class="brush:xquery">
+
The XQuery expression below serves as an example on how large files can be downloaded and written to a file with constant memory consumption:
 +
 
 +
<pre lang='xquery'>
 
file:write-binary('output.data', fetch:binary('http://files.basex.org/xml/xmark111mb.zip'))
 
file:write-binary('output.data', fetch:binary('http://files.basex.org/xml/xmark111mb.zip'))
 
</pre>
 
</pre>
 +
 +
If lazy items are serialized, they will be streamed as well.
  
 
=Conventions=
 
=Conventions=
  
−
All functions in this module are assigned to the {{Code|http://basex.org/modules/stream}} namespace, which is statically bound to the {{Code|stream}} prefix.<br/>
+
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/lazy</nowiki></code> namespace, which is statically bound to the {{Code|lazy}} prefix.<br/>
−
All errors are assigned to the {{Code|http://basex.org/errors}} namespace, which is statically bound to the {{Code|bxerr}} prefix.
 
  
 
=Functions=
 
=Functions=
  
−
==stream:materialize==
+
==lazy:cache==
 +
 
 
{| width='100%'
 
{| width='100%'
−
|-
+
|- valign="top"
−
| width='90' | '''Signatures'''
+
| width='120' | '''Signature'''
−
|{{Func|stream:materialize|$item as item()|item()}}
+
|<pre>lazy:cache(
−
|-
+
  $input  as item()*,
 +
  $lazy  as xs:boolean?  := false()
 +
) as item()*</pre>
 +
|- valign="top"
 
| '''Summary'''
 
| '''Summary'''
−
|Returns a materialized instance of the specified {{Code|$item}}.<br />If an item is streamable, its content will be retrieved, and a new item containing its data will be returned. Other, non-streamable items will simply be passed through.
+
|Caches the data of lazy {{Code|$input}} items:<br/>
 +
* data of lazy items are retrieved and cached inside the item.
 +
* non-lazy items, or lazy items with cached data, are simply passed through.
 +
* If {{Code|$lazy}} is set to {{Code|true()}}, caching is deferred until the data is eventually requested. Streaming will be disabled: Data will be cached before a stream is returned.
 +
Caching is advisable if an item is processed more than once, or if the data may not be available anymore at a later stage.
 +
|- valign="top"
 +
| '''Example'''
 +
|In the following example, a file is deleted before its content is returned. To avoid a “file not found” error when serializing the result, the content must be cached:
 +
<pre lang='xquery'>
 +
let $file := 'data.txt'
 +
let $text := lazy:cache(file:read-text($file))
 +
return (file:delete($file), $text)
 +
</pre>
 
|}
 
|}
  
−
==stream:is-streamable==
+
==lazy:is-lazy==
 +
 
 
{| width='100%'
 
{| width='100%'
−
|-
+
|- valign="top"
−
| width='90' | '''Signatures'''
+
| width='120' | '''Signature'''
−
|{{Func|stream:is-streamable|$item as item()|item()}}
+
|<pre>lazy:is-lazy(
−
|-
+
  $item as item()
 +
) as xs:boolean</pre>
 +
|- valign="top"
 
| '''Summary'''
 
| '''Summary'''
−
|Checks if the specified {{Code|$item}} is streamable.
+
|Checks whether the specified {{Code|$item}} is lazy.
 +
|}
 +
 
 +
==lazy:is-cached==
 +
 
 +
{| width='100%'
 +
|- valign="top"
 +
| width='120' | '''Signature'''
 +
|<pre>lazy:is-cached(
 +
  $item  as item()
 +
) as xs:boolean</pre>
 +
|- valign="top"
 +
| '''Summary'''
 +
|Checks whether the contents of the specified {{Code|$item}} are cached. The function will always return {{Code|true}} for non-lazy items.
 
|}
 
|}
  
 
=Changelog=
 
=Changelog=
 +
 +
;Version 9.1
 +
 +
* Updated: {{Function||lazy:cache}}: {{Code|$lazy}} argument added; support for sequences.
 +
 +
;Version 9.0
 +
 +
* Updated: Renamed from Streaming Module to Lazy Module.
 +
* Added: {{Function||lazy:is-cached}}
 +
 +
;Version 8.0
 +
 +
* Updated: {{Function||stream:materialize}} extended to sequences.
  
 
This module was introduced with Version 7.7.
 
This module was introduced with Version 7.7.
−
 
−
[[Category:XQuery]]
 

Latest revision as of 18:33, 1 December 2023

This XQuery Module contains functions for handling lazy items.

In contrast to standard XQuery items, a lazy item contains a reference to the actual data, and the data itself will only be retrieved if it is processed. Hence, possible errors will be postponed, and no memory will be occupied by a lazy item as long as its content has not been requested yet.

The following BaseX functions return lazy items:

Some functions are capable of consuming the contents of lazy items in a streamable fashion: data will not be cached, but instead passed on to another target (file, the calling expression, etc.). The following streaming functions are currently available:

The XQuery expression below serves as an example on how large files can be downloaded and written to a file with constant memory consumption: 

file:write-binary('output.data', fetch:binary('http://files.basex.org/xml/xmark111mb.zip'))

If lazy items are serialized, they will be streamed as well.

Conventions[edit]

All functions and errors in this module are assigned to the http://basex.org/modules/lazy namespace, which is statically bound to the lazy prefix.

Functions[edit]

lazy:cache[edit]

Signature 
lazy:cache(
  $input  as item()*,
  $lazy   as xs:boolean?  := false()
) as item()*
Summary Caches the data of lazy $input items:
  • data of lazy items are retrieved and cached inside the item.
  • non-lazy items, or lazy items with cached data, are simply passed through.
  • If $lazy is set to true(), caching is deferred until the data is eventually requested. Streaming will be disabled: Data will be cached before a stream is returned.

Caching is advisable if an item is processed more than once, or if the data may not be available anymore at a later stage.

Example In the following example, a file is deleted before its content is returned. To avoid a “file not found” error when serializing the result, the content must be cached: 
let $file := 'data.txt'
let $text := lazy:cache(file:read-text($file))
return (file:delete($file), $text)

lazy:is-lazy[edit]

Signature 
lazy:is-lazy(
  $item  as item()
) as xs:boolean
Summary Checks whether the specified $item is lazy.

lazy:is-cached[edit]

Signature 
lazy:is-cached(
  $item  as item()
) as xs:boolean
Summary Checks whether the contents of the specified $item are cached. The function will always return true for non-lazy items.

Changelog[edit]

Version 9.1
  • Updated: lazy:cache: $lazy argument added; support for sequences.
Version 9.0
  • Updated: Renamed from Streaming Module to Lazy Module.
  • Added: lazy:is-cached
Version 8.0

This module was introduced with Version 7.7.

Retrieved from "http://docs.basex.org/index.php?title=Lazy_Module&oldid=16809"