Bureaucrats, editor, reviewer, Administrators
13,550
edits
Changes
Jump to navigation
Jump to search
{{Mark|Updated with Version 9.0}}:
{{Mark|Introduced with Version 9.0:}}
{{Mark|Updated with Version 9.0}}:
no edit summary
This [[Module Library|XQuery Module]] provides simple functions to fetch the content of resources identified by URIs. Resources can be stored locally or remotely and e.g. use the {{Code|file://}} or {{Code|http://}} scheme. If more control over HTTP requests is required, the [[HTTP Client Module]] can be used. With the [[HTML Module]], retrieved HTML documents can be converted to XML.
=Conventions=
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/fetch</nowiki></code> namespace, which is statically bound to the {{Code|fetch}} prefix.<br/>
|-
| '''Summary'''
|Fetches the resource referred to by the given URI and returns it as [[Streaming Lazy Module|streamablelazy]] {{Code|xs:base64Binary}}item.
|-
| '''Errors'''
|
* <code><nowiki>fetch:binary("http://images.trulia.com/blogimg/c/5/f/4/679932_1298401950553_o.jpg")</nowiki></code> returns the addressed image.
* <code><nowiki>streamlazy:materializecache(fetch:binary("http://en.wikipedia.org"))</nowiki></code> returns a materialized representation of enforces the streamable resultfetch operation (otherwise, it will be delayed until requested first).
|}
|-
| '''Summary'''
|Fetches the resource referred to by the given {{Code|$uri}} and returns it as [[Streaming Lazy Module|streamablelazy]] {{Code|xs:string}}item:
* The UTF-8 default encoding can be overwritten with the optional {{Code|$encoding}} argument.
* By default, invalid characters will be rejected. If {{Code|$fallback}} is set to true, these characters will be replaced with the Unicode replacement character <code>FFFD</code> (�).
* <code><nowiki>fetch:text("http://en.wikipedia.org")</nowiki></code> returns a string representation of the English Wikipedia main HTML page.
* <code><nowiki>fetch:text("http://www.bbc.com","US-ASCII",true())</nowiki></code> returns the BBC homepage in US-ASCII with all non-US-ASCII characters replaced with �.
* <code><nowiki>streamlazy:materializecache(fetch:text("http://en.wikipedia.org"))</nowiki></code> returns a materialized representation of enforces the streamable resultfetch operation (otherwise, it will be delayed until requested first).
|}
|-
| width='120' | '''Signatures'''
|{{Func|fetch:xml|$uri as xs:string|document-node()}}<br/>{{Func|fetch:xml|$uri as xs:string, $options as map(*)?|document-node()}}
|-
| '''Summary'''
|Fetches the resource referred to by the given {{Code|$uri}} and returns it as XML document node.<br/>In contrast to <code>fn:doc</code>, each function call returns a different document node. As a consequence, document instances created by this function will not be kept in memory until the end of query evaluation.<br/>The {{Code|$options}} argument can be used to change the parsing behavior. Allowed options are all [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options in lower case.<br/>The function is different to <code>fn:doc</code> in various aspects:* As it is non-deterministic, a new document node will be created by each call of this function.* A document created by this function will be garbage-collected as soon as it is not referenced anymore.* URIs will not be resolved against existing databases. As a result, it will not trigger any locks (see [[Transaction Management#Limitations|limitations of database locking]] for more details).
|-
| '''Errors'''
|
* Retrieve an XML representation of the English Wikipedia main HTML page, chop all whitespace nodes:
<pre classsyntaxhighlight lang="brush:xquery">
fetch:xml("http://en.wikipedia.org", map { 'chop': true() })
</presyntaxhighlight>
* Return a document located in the current base directory:
<pre classsyntaxhighlight lang="brush:xquery">
fetch:xml(file:base-dir() || "example.xml")
</presyntaxhighlight>* Return a web page as XML, preserve namespaces:<syntaxhighlight lang="xquery">fetch:xml( 'http://basex.org/', map { 'parser': 'html', 'htmlparser': map { 'nons': false() } })</syntaxhighlight>
|}
==fetch:xml-binary==
{| width='100%'
|-
| width='120' | '''Signatures'''
|{{Func|fetch:xml-binary|$data as xs:base64Binary|document-node()}}<br/>{{Func|fetch:xml-binary|$data as xs:base64Binary, $options as map(*)?|document-node()}}
|-
| '''Summary'''
|Parses binary {{Code|$data}} and returns it as XML document node.<br/>In contrast to fn:parse-xml, which expects an XQuery string, the input of this function can be arbitrarily encoded. The encoding will be derived from the XML declaration or (in case of UTF16 or UTF32) from the first bytes of the input.<br/>The {{Code|$options}} argument can be used to change the parsing behavior. Allowed options are all [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options in lower case.
|-
| '''Examples'''
|
* Retrieves file input as binary data and parses it as XML:
<pre classsyntaxhighlight lang="brush:xquery">
fetch:xml-binary(file:read-binary('doc.xml'))
</presyntaxhighlight>
* Encodes a string as CP1252 and parses it as XML. The input and the string {{Code|touché}} will be correctly decoded because of the XML declaration:
<pre classsyntaxhighlight lang="brush:xquery">
fetch:xml-binary(convert:string-to-base64(
"<?xml version='1.0' encoding='CP1252'?><xml>touché</xml>",
"CP1252"
))
</presyntaxhighlight>
* Encodes a string as UTF16 and parses it as XML. The document will be correctly decoded, as the first bytes of the data indicate that the input must be UTF16:
<pre classsyntaxhighlight lang="brush:xquery">
fetch:xml-binary(convert:string-to-base64("<xml/>", "UTF16"))
</presyntaxhighlight>
|}
=Errors=
{| class="wikitable" width="100%"
