Difference between revisions of "Fetch Module"
Line 12: | Line 12: | ||
{| width='100%' | {| width='100%' | ||
− | |- | + | |- valign="top" |
| width='120' | '''Signatures''' | | width='120' | '''Signatures''' | ||
|{{Func|fetch:binary|$uri as xs:string|xs:base64Binary}}<br/> | |{{Func|fetch:binary|$uri as xs:string|xs:base64Binary}}<br/> | ||
− | |- | + | |- valign="top" |
| '''Summary''' | | '''Summary''' | ||
|Fetches the resource referred to by the given URI and returns it as [[Lazy Module|lazy]] {{Code|xs:base64Binary}} item. | |Fetches the resource referred to by the given URI and returns it as [[Lazy Module|lazy]] {{Code|xs:base64Binary}} item. | ||
− | |- | + | |- valign="top" |
| '''Errors''' | | '''Errors''' | ||
|{{Error|open|#Errors}} the URI could not be resolved, or the resource could not be retrieved. | |{{Error|open|#Errors}} the URI could not be resolved, or the resource could not be retrieved. | ||
− | |- | + | |- valign="top" |
| '''Examples''' | | '''Examples''' | ||
| | | | ||
Line 31: | Line 31: | ||
{| width='100%' | {| width='100%' | ||
− | |- | + | |- valign="top" |
| width='120' | '''Signatures''' | | width='120' | '''Signatures''' | ||
|{{Func|fetch:text|$uri as xs:string|xs:string}}<br/>{{Func|fetch:text|$uri as xs:string, $encoding as xs:string|xs:string}}<br/>{{Func|fetch:text|$uri as xs:string, $encoding as xs:string, $fallback as xs:boolean|xs:string}}<br/> | |{{Func|fetch:text|$uri as xs:string|xs:string}}<br/>{{Func|fetch:text|$uri as xs:string, $encoding as xs:string|xs:string}}<br/>{{Func|fetch:text|$uri as xs:string, $encoding as xs:string, $fallback as xs:boolean|xs:string}}<br/> | ||
− | |- | + | |- valign="top" |
| '''Summary''' | | '''Summary''' | ||
|Fetches the resource referred to by the given {{Code|$uri}} and returns it as [[Lazy Module|lazy]] {{Code|xs:string}} item: | |Fetches the resource referred to by the given {{Code|$uri}} and returns it as [[Lazy Module|lazy]] {{Code|xs:string}} item: | ||
* The UTF-8 default encoding can be overwritten with the optional {{Code|$encoding}} argument. | * The UTF-8 default encoding can be overwritten with the optional {{Code|$encoding}} argument. | ||
* By default, invalid characters will be rejected. If {{Code|$fallback}} is set to true, these characters will be replaced with the Unicode replacement character <code>FFFD</code> (�). | * By default, invalid characters will be rejected. If {{Code|$fallback}} is set to true, these characters will be replaced with the Unicode replacement character <code>FFFD</code> (�). | ||
− | |- | + | |- valign="top" |
| '''Errors''' | | '''Errors''' | ||
|{{Error|open|#Errors}} the URI could not be resolved, or the resource could not be retrieved.<br/>{{Error|encoding|#Errors}} the specified encoding is not supported, or unknown. | |{{Error|open|#Errors}} the URI could not be resolved, or the resource could not be retrieved.<br/>{{Error|encoding|#Errors}} the specified encoding is not supported, or unknown. | ||
− | |- | + | |- valign="top" |
| '''Examples''' | | '''Examples''' | ||
| | | | ||
Line 53: | Line 53: | ||
{| width='100%' | {| width='100%' | ||
− | |- | + | |- valign="top" |
| width='120' | '''Signatures''' | | width='120' | '''Signatures''' | ||
|{{Func|fetch:doc|$uri as xs:string|document-node()}}<br/>{{Func|fetch:doc|$uri as xs:string, $options as map(*)?|document-node()}} | |{{Func|fetch:doc|$uri as xs:string|document-node()}}<br/>{{Func|fetch:doc|$uri as xs:string, $options as map(*)?|document-node()}} | ||
− | |- | + | |- valign="top" |
| '''Summary''' | | '''Summary''' | ||
|Fetches the resource referred to by the given {{Code|$uri}} and returns it as a document node.<br/>The {{Code|$options}} argument can be used to change the parsing behavior. Allowed options are all [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options in lower case.<br/>The function differs from {{Code|fn:doc}} in various aspects: | |Fetches the resource referred to by the given {{Code|$uri}} and returns it as a document node.<br/>The {{Code|$options}} argument can be used to change the parsing behavior. Allowed options are all [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options in lower case.<br/>The function differs from {{Code|fn:doc}} in various aspects: | ||
Line 62: | Line 62: | ||
* A document created by this function will be garbage-collected as soon as it is not referenced anymore. | * A document created by this function will be garbage-collected as soon as it is not referenced anymore. | ||
* URIs will not be resolved against existing databases. As a result, it will not trigger any locks (see [[Transaction Management#Limitations|limitations of database locking]] for more details). | * URIs will not be resolved against existing databases. As a result, it will not trigger any locks (see [[Transaction Management#Limitations|limitations of database locking]] for more details). | ||
− | |- | + | |- valign="top" |
| '''Errors''' | | '''Errors''' | ||
|{{Error|open|#Errors}} the URI could not be resolved, or the resource could not be retrieved. | |{{Error|open|#Errors}} the URI could not be resolved, or the resource could not be retrieved. | ||
− | |- | + | |- valign="top" |
| '''Examples''' | | '''Examples''' | ||
| | | | ||
Line 87: | Line 87: | ||
{| width='100%' | {| width='100%' | ||
− | |- | + | |- valign="top" |
| width='120' | '''Signatures''' | | width='120' | '''Signatures''' | ||
|{{Func|fetch:binary-doc|$input as xs:anyAtomicType|document-node()}}<br/>{{Func|fetch:binary-doc|$data as xs:anyAtomicType, $options as map(*)?|document-node()}} | |{{Func|fetch:binary-doc|$input as xs:anyAtomicType|document-node()}}<br/>{{Func|fetch:binary-doc|$data as xs:anyAtomicType, $options as map(*)?|document-node()}} | ||
− | |- | + | |- valign="top" |
| '''Summary''' | | '''Summary''' | ||
|Converts the specified {{Code|$input}} ({{Code|xs:base64Binary}}, {{Code|xs:hexBinary}}) to XML and returns it as a document node.<br/>In contrast to {{Code|fn:parse-xml}}, which expects a string, the input can be arbitrarily encoded. The encoding will be derived from the XML declaration or (in case of UTF-16 or UTF-32) from the first bytes of the input.<br/>The {{Code|$options}} argument can be used to change the parsing behavior. Allowed options are all [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options in lower case. | |Converts the specified {{Code|$input}} ({{Code|xs:base64Binary}}, {{Code|xs:hexBinary}}) to XML and returns it as a document node.<br/>In contrast to {{Code|fn:parse-xml}}, which expects a string, the input can be arbitrarily encoded. The encoding will be derived from the XML declaration or (in case of UTF-16 or UTF-32) from the first bytes of the input.<br/>The {{Code|$options}} argument can be used to change the parsing behavior. Allowed options are all [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options in lower case. | ||
− | |- | + | |- valign="top" |
| '''Examples''' | | '''Examples''' | ||
| | | | ||
Line 111: | Line 111: | ||
fetch:binary-doc(convert:string-to-base64("<xml/>", "UTF16")) | fetch:binary-doc(convert:string-to-base64("<xml/>", "UTF16")) | ||
</syntaxhighlight> | </syntaxhighlight> | ||
− | |- | + | |- valign="top" |
| '''Errors''' | | '''Errors''' | ||
|{{Error|open|#Errors}} the input could not be parsed. | |{{Error|open|#Errors}} the input could not be parsed. | ||
Line 119: | Line 119: | ||
{| width='100%' | {| width='100%' | ||
− | |- | + | |- valign="top" |
| width='120' | '''Signatures''' | | width='120' | '''Signatures''' | ||
|{{Func|fetch:content-type|$uri as xs:string|xs:string}}<br/> | |{{Func|fetch:content-type|$uri as xs:string|xs:string}}<br/> | ||
− | |- | + | |- valign="top" |
| '''Summary''' | | '''Summary''' | ||
|Returns the content-type (also called mime-type) of the resource specified by {{Code|$uri}}: | |Returns the content-type (also called mime-type) of the resource specified by {{Code|$uri}}: | ||
* If a remote resource is addressed, the request header will be evaluated. | * If a remote resource is addressed, the request header will be evaluated. | ||
* If the addressed resource is locally stored, the content-type will be guessed based on the file extension. | * If the addressed resource is locally stored, the content-type will be guessed based on the file extension. | ||
− | |- | + | |- valign="top" |
| '''Errors''' | | '''Errors''' | ||
|{{Error|open|#Errors}} the URI could not be resolved, or the resource could not be retrieved. | |{{Error|open|#Errors}} the URI could not be resolved, or the resource could not be retrieved. | ||
− | |- | + | |- valign="top" |
| '''Examples''' | | '''Examples''' | ||
| | | | ||
Line 141: | Line 141: | ||
! width="110"|Code | ! width="110"|Code | ||
|Description | |Description | ||
− | |- | + | |- valign="top" |
|{{Code|encoding}} | |{{Code|encoding}} | ||
|The specified encoding is not supported, or unknown. | |The specified encoding is not supported, or unknown. | ||
− | |- | + | |- valign="top" |
|{{Code|open}} | |{{Code|open}} | ||
|The URI could not be resolved, or the resource could not be retrieved. | |The URI could not be resolved, or the resource could not be retrieved. |
Revision as of 14:17, 20 July 2022
This XQuery Module provides simple functions to fetch the content of resources identified by URIs. Resources can be stored locally or remotely and e.g. use the file://
or http://
scheme. If more control over HTTP requests is required, the HTTP Client Module can be used. With the HTML Module, retrieved HTML documents can be converted to XML.
Contents
Conventions
All functions and errors in this module are assigned to the http://basex.org/modules/fetch
namespace, which is statically bound to the fetch
prefix.
URI arguments can point be URLs or point to local files. Relative file paths will be resolved against the current working directory (for more details, have a look at the File Module).
Functions
fetch:binary
Signatures | fetch:binary($uri as xs:string) as xs:base64Binary |
Summary | Fetches the resource referred to by the given URI and returns it as lazy xs:base64Binary item.
|
Errors | open : the URI could not be resolved, or the resource could not be retrieved.
|
Examples |
|
fetch:text
Signatures | fetch:text($uri as xs:string) as xs:string fetch:text($uri as xs:string, $encoding as xs:string) as xs:string fetch:text($uri as xs:string, $encoding as xs:string, $fallback as xs:boolean) as xs:string |
Summary | Fetches the resource referred to by the given $uri and returns it as lazy xs:string item:
|
Errors | open : the URI could not be resolved, or the resource could not be retrieved.encoding : the specified encoding is not supported, or unknown.
|
Examples |
|
fetch:doc
Signatures | fetch:doc($uri as xs:string) as document-node() fetch:doc($uri as xs:string, $options as map(*)?) as document-node()
|
Summary | Fetches the resource referred to by the given $uri and returns it as a document node.The $options argument can be used to change the parsing behavior. Allowed options are all parsing and XML parsing options in lower case.The function differs from fn:doc in various aspects:
|
Errors | open : the URI could not be resolved, or the resource could not be retrieved.
|
Examples |
<syntaxhighlight lang="xquery"> fetch:doc("http://en.wikipedia.org", map { 'stripws': true() }) </syntaxhighlight>
<syntaxhighlight lang="xquery"> fetch:doc( 'http://basex.org/', map { 'parser': 'html', 'htmlparser': map { 'nons': false() } } ) </syntaxhighlight> |
fetch:binary-doc
Signatures | fetch:binary-doc($input as xs:anyAtomicType) as document-node() fetch:binary-doc($data as xs:anyAtomicType, $options as map(*)?) as document-node()
|
Summary | Converts the specified $input (xs:base64Binary , xs:hexBinary ) to XML and returns it as a document node.In contrast to fn:parse-xml , which expects a string, the input can be arbitrarily encoded. The encoding will be derived from the XML declaration or (in case of UTF-16 or UTF-32) from the first bytes of the input.The $options argument can be used to change the parsing behavior. Allowed options are all parsing and XML parsing options in lower case.
|
Examples |
<syntaxhighlight lang="xquery"> fetch:binary-doc(file:read-binary('doc.xml')) </syntaxhighlight>
<syntaxhighlight lang="xquery"> fetch:binary-doc(convert:string-to-base64( "<?xml version='1.0' encoding='CP1252'?><xml>touché</xml>", "CP1252" )) </syntaxhighlight>
<syntaxhighlight lang="xquery"> fetch:binary-doc(convert:string-to-base64("<xml/>", "UTF16")) </syntaxhighlight> |
Errors | open : the input could not be parsed.
|
fetch:content-type
Signatures | fetch:content-type($uri as xs:string) as xs:string |
Summary | Returns the content-type (also called mime-type) of the resource specified by $uri :
|
Errors | open : the URI could not be resolved, or the resource could not be retrieved.
|
Examples |
|
Errors
Code | Description |
---|---|
encoding
|
The specified encoding is not supported, or unknown. |
open
|
The URI could not be resolved, or the resource could not be retrieved. |
Changelog
- Version 10.0
- Updated:
fetch:doc
renamed (before:fetch:xml
). - Updated:
fetch:binary-doc
renamed (before:fetch:xml-binary
).
- Version 9.0
- Added:
fetch:xml-binary
- Updated: error codes updated; errors now use the module namespace
- Version 8.5
- Updated:
fetch:text
:$fallback
argument added.
- Version 8.0
- Added:
fetch:xml
The module was introduced with Version 7.6.