HTTP Client Module

This XQuery Module contains a single function to send HTTP requests and handle HTTP responses. The function send-request is based on the EXPath HTTP Client Module. It gives full control over the available request and response parameters. For simple GET requests, the Fetch Module may be sufficient.

If  is specified and if the addressed web server provides support for the gzip compression algorithm, the response will automatically be decompressed.

=Conventions=

All functions in this module are assigned to the  namespace, which is statically bound to the http prefix. All errors are assigned to the  namespace, which is statically bound to the experr prefix.

=Functions=

http:send-request
=Examples=

Status Only
Simple GET request. As the attribute status-only is set to true, only the response element is returned.

Query: http:send-request(, 'http://basex.org') Result:            

Google Homepage
Retrieve the Google search home page with a timeout of 10 seconds. In order to parse HTML, TagSoup must be contained in the class path.

Query: http:send-request(<http:request method='get' href='http://www.google.com' timeout='10'/>) Result: <pre class="brush:xml">  <http:header name="Date" value="Mon, 14 Mar 2011 22:03:25 GMT"/> <http:header name="Transfer-Encoding" value="chunked"/> <http:header name="Expires" value="-1"/> <http:header name="X-XSS-Protection" value="1; mode=block"/> <http:header name="Set-Cookie" value="...; expires=Tue, 13-Sep-2011 22:03:25 GMT; path=/; domain=.google.ch; HttpOnly"/> <http:header name="Content-Type" value="text/html; charset=ISO-8859-1"/> <http:header name="Server" value="gws"/> <http:header name="Cache-Control" value="private, max-age=0"/> <http:body media-type="text/html; charset=ISO-8859-1"/> </http:response> <html xmlns="http://www.w3.org/1999/xhtml"> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/> Google ...

The response content type can also be overwritten in order to retrieve HTML pages and other textual data as plain string (using text/plain) or in its binary representation (using application/octet-stream). With the http:header element, a custom user agent can be set. See the following example:

Query: <pre class="brush:xquery"> let $binary := http:send-request(  <http:request method='get'     override-media-type='application/octet-stream'     href='http://www.google.com'>    <http:header name="User-Agent" value="Opera"/>  </http:request> )[2] return try { html:parse($binary) } catch * { 'Conversion to XML failed: ' || $err:description }

SVG Data
Content-type ending with +xml, e.g. image/svg+xml.

Query: <pre class="brush:xquery">http:send-request(<http:request method='get'/>, 'http://upload.wikimedia.org/wikipedia/commons/6/6b/Bitmap_VS_SVG.svg')

Result: <pre class="brush:xml"> <http:header name="ETag" value="W/&quot;11b6d-4ba15ed4&quot;"/> <http:header name="Age" value="9260"/> <http:header name="Date" value="Mon, 14 Mar 2011 19:17:10 GMT"/> <http:header name="Content-Length" value="72557"/> <http:header name="Last-Modified" value="Wed, 17 Mar 2010 22:59:32 GMT"/> <http:header name="Content-Type" value="image/svg+xml"/> <http:header name="X-Cache-Lookup" value="MISS from knsq22.knams.wikimedia.org:80"/> <http:header name="Connection" value="keep-alive"/> <http:header name="Server" value="Sun-Java-System-Web-Server/7.0"/> <http:header name="X-Cache" value="MISS from knsq22.knams.wikimedia.org"/> <http:body media-type="image/svg+xml"/> </http:response> <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="1063" height="638"> <linearGradient id="lg0"> <stop stop-color="#3333ff" offset="0"/> <stop stop-color="#3f3fff" stop-opacity="0" offset="1"/> </linearGradient> ...

POST Request
POST request to the BaseX REST Service, specifying a username and password.

Query: <pre class="brush:xquery"> http:send-request( <http:request method='post' username='admin' password='admin'>    <http:body media-type='application/xml'/>  </http:request>,  'http://localhost:8984/rest',      {        for $i in 1 to 3        return Section {$i }       } )

Result: <pre class="brush:xml">  <http:header name="Content-Length" value="135"/> <http:header name="Content-Type" value="application/xml"/> <http:header name="Server" value="Jetty(6.1.26)"/> <http:body media-type="application/xml"/> </http:response> Section 1 Section 2 Section 3

File Upload
Performs an HTML file upload. In the RESTXQ code, the uploaded file is written to the temporary directory:

Query:

<pre class="brush:xquery"> let $path := 'file-to-be.uploaded' return http:send-request( <http:request method='POST'>    <http:multipart media-type='multipart/form-data'>      <http:header name='content-disposition'        value='form-data; name="files"; filename="{ file:name($path) }"'/>      <http:body media-type='application/octet-stream'/>    </http:multipart>  </http:request>,  'http://localhost:8984/write-to-temp',  file:read-binary($path) )

RESTXQ service:

<pre class="brush:xquery"> declare %rest:POST %rest:path('/write-to-temp') %rest:form-param('files', '{$files}') function dba:file-upload( $files  as map(xs:string, xs:base64Binary) ) as empty-sequence { map:for-each($files, function($file, $content) {   file:write-binary(file:temp-dir || $file, $content)  }); };

=Errors=

=Changelog=


 * Version 9.0
 * Updated: support for gzipped content encoding


 * Version 8.0
 * Added: digest authentication


 * Version 7.6
 * Updated: http:send-request: HC0002 is raised if the input cannot be parsed or converted to the final data type.
 * Updated: errors are using text/plain as media-type.