Difference between revisions of "WebSockets"

From BaseX Documentation
Jump to navigation Jump to search
(24 intermediate revisions by 3 users not shown)
Line 1: Line 1:
This page presents one of the [[Web Application]] services. It describes how to use the WebSockets API of BaseX. WebSocket is a communication protocol for providing full-duplex communication.  
+
This page presents one of the [[Web Application]] services. It describes how to use the WebSockets API of BaseX. WebSocket is a communication protocol for providing '''full-duplex''' communication: Data can be sent in both directions and simultaneously.
  
After an initial HTTP-request it communicates over a single TCP connection. The WebSocket protocol was standardized in RFC 6455 by the IETF. It creates the possibility to transponse real-time data from the server to the client and the client to the server.  
+
Please note that the current WebSocket implementation relies on Jetty’s WebSocket servlet API. Other web servers may be supported in future versions.
  
In contrast to the HTTP protocol it enables the server to send unsolicited data to the client. For establishing a WebSocket connection there is a WebSocket handshake request, send by the client. The server returns a WebSocket handshake response. This will be handled by jetty. After a successful handshake, the annotation <code>ws:connect</code> will be called. The persistent connection is now open until an the client or the server closes it, an error occures or a timeout happens. How long this timeout goes can be set in the web.xml. It is possible to transmit all kind of data, binary or text.  
+
=Introduction=
 +
 
 +
==Protocol==
 +
 
 +
Use WebSockets if you have to exchange data with a high frequency or if you have to send messages from the server to the client without techniques like [polling https://en.wikipedia.org/wiki/Polling_(computer_science)]. In contrast to REST, WebSockets use a single URL for the whole communication.  
  
For further information about the WebSocket protocol check out the RFC6455 spec.
+
The WebSocket protocol was standardized in [https://tools.ietf.org/html/rfc6455 RFC 6455] by the IETF. After an initial HTTP request, all communication takes place over a single TCP connection. Unlike the HTTP protocol, a connection will be kept alive, and a server can send unsolicited data to the client.
  
=Introduction=
+
For establishing a WebSocket connection, a handshake request is sent by the client. The web server returns a handshake response. If the handshake is successful, the persistent connection will be open until the client or the server closes it, an error occurs or a timeout happens. It is possible to transmit all kind of data, binary or text. '''The BaseX WebServer handles the handshake completely.''' You just have to define some limits of the connection in the <code>web.xml</code> and specify functions for WebSocket events like ''onConnect'' and ''onMessage''.
 +
 
 +
Notice that there is no specification of a message protocol. The WebSocket protocol just specifies the message architecture but not how the payload of the messages is formatted. To agree on a format between the server and the client one can use sub-protocols.
 +
 
 +
Some older browsers don’t support the WebSocket protocol. Therefore you can use fallback options like Ajax. JavaScript client libraries like SockJS can be used for building client applications. The library takes care of how to establish the real-time connection. If the WebSocket protocol isn’t supported, it uses polling. You have to provide server functions for the fallback solutions if you have to support fallbacks.
  
 
==Preliminaries==
 
==Preliminaries==
Line 13: Line 21:
 
There are a bunch of annotations depending to WebSockets for annotating XQuery functions. When a WebSocket message arrives at the server, an XQuery function will be invoked that matches the constraints indicated by its annotations.
 
There are a bunch of annotations depending to WebSockets for annotating XQuery functions. When a WebSocket message arrives at the server, an XQuery function will be invoked that matches the constraints indicated by its annotations.
  
If a WebSocket function is requested (like connecting to the path '/', message to the path '/path', ...), the module directory and its sub-directories will be traversed, and all [[XQuery Extensions#Suffixes|XQuery files]] will be parsed for functions with WebSocket annotations. Sub-directories that include an {{Code|.ignore}} the file will be skipped.
+
If a WebSocket function is requested (like connecting to the path <code>/</code>, sending a message to the path <code>/path</code>, ), the module directory and its sub-directories will be traversed, and all [[XQuery Extensions#Suffixes|XQuery files]] will be parsed for functions with WebSocket annotations. Sub-directories that include an {{Code|.ignore}} file will be skipped.
  
To speed up processing, the functions of the existing XQuery modules are automatically cached in main memory:
+
To speed up processing, the functions of the existing XQuery modules are automatically cached in main memory. For further information on cache handling, check out the [[RESTXQ#Introduction|RESTXQ introduction]].
* Functions will be invalidated and parsed again if the timestamp of their module changes.
 
  
==Examples==
+
==Configuration==
<pre class="brush:xquery">
+
 
module namespace page = 'http://basex.org/modules/web-page';
+
* The WebSocket servlet can be enabled and disabled in the <code>web.xml</code> configuration file. You can specify further configuration options, such as <code>maxIdleTime</code>, <code>maxTextMessageSize</code>, and <code>maxBinaryMessageSize</code>.
 +
* The default limit for messges is 64 KB. If you a message exceeds the default or the specified limit, an error will be raised and the connection will be closed.
 +
 
 +
=Annotations=
 +
 
 +
To tag functions as WebSocket functions you have to use [[XQuery 3.0#Annotations|annotations]]. The annotation is written after the keyword ''declare'' and before the keyword ''function''. For the context of WebSockets there are some annotations listed below. Functions which are annotated with a WebSocket annotation will be called if the appropriate event occurs. For example, the function annotated with <code>ws:connect('/')</code> will be executed if a client establishes a connection with the WebSocket root path (which is, by default, <code>ws/</code>). By using annotations, it’s easy to provide an API for your WebSocket connection. You just have to specify what to do when a WebSocket Event occurs, annotate it with the corresponding annotation and the Servlet will do the rest for you.
 +
 
 +
==%ws:connect(path)==
 +
 
 +
Called directly after a successful WebSocket handshake. The <code>path</code> specifies the path which a client is connected to:
 +
 
 +
<syntaxhighlight lang="xquery">
 +
declare %ws:connect('/') function local:connect() { };
 +
</syntaxhighlight>
 +
 
 +
You can specify here how to handle your users, e. g. save a name as a WebSocket attribute. Furthermore, you can check header parameters for validity.
 +
 
 +
==%ws:message(path, message)==
 +
 
 +
Called when a client message arrives at the server. The <code>path</code> specifies the path which a client is connected to. The <code>message</code> string contains the name of the variable to which the message will be bound:
 +
 
 +
<syntaxhighlight lang="xquery">
 +
declare %ws:message('/', '{$info}') function local:message($info) { };
 +
</syntaxhighlight>
 +
 
 +
The value will be of type <code>xs:string</code> or <code>xs:base64Binary</code>. As there is no fixed message protocol, the client needs to take care of the message syntax.
 +
 
 +
==%ws:error(path, message)==
 +
 
 +
Called when an error occurs. The <code>path</code> specifies the path which a client is connected to. The <code>message</code> string contains the name of the variable to which the message will be bound:
 +
 
 +
<syntaxhighlight lang="xquery">
 +
declare %ws:error('/', '{$error}') function local:error($error) { };
 +
</syntaxhighlight>
 +
 
 +
Usually, errors happen because of bad/malformed incoming packets. The WebSocket connection gets closed after the error handling.
 +
 
 +
==%ws:close(path)==
 +
 
 +
Called when the WebSocket closes. The <code>path</code> specifies the path which a client is connected to:
 +
 
 +
<syntaxhighlight lang="xquery">
 +
declare %ws:close('/') function local:connect() { };
 +
</syntaxhighlight>
 +
 
 +
The WebSocket is already closed when this annotation is called so there can be no return.
 +
 
 +
==%ws:header-param(name, variable[, default])==
 +
 
 +
For accessing connection-specific properties like the HTTP version. The value will be bound to the specified <code>variable</code>. If the property has no value, an optional <code>default</code> value will be assigned instead:
 +
 
 +
<syntaxhighlight lang="xquery">
 +
declare
 +
  %ws:close('host', '{$host}')
 +
  %ws:header-param('host', '{$host}')
 +
function local:close($host) {
 +
  admin:write-log('Connection was closed: ' || $host)
 +
};
 +
</syntaxhighlight>
 +
 
 +
The following parameters are available:
 +
 
 +
{| class="wikitable"
 +
|- valign="top"
 +
! Name
 +
! Description
 +
|- valign="top"
 +
| <code>host</code>
 +
| The host of the request URI.
 +
|- valign="top"
 +
| <code>http-version</code>
 +
| The HTTP version used for the request.
 +
|- valign="top"
 +
| <code>is-secure</code>
 +
| Indicates if the connection is secure.
 +
|- valign="top"
 +
| <code>origin</code>
 +
| The WebSocket origin.
 +
|- valign="top"
 +
| <code>protocol-version</code>
 +
| The version of the used protocol.
 +
|- valign="top"
 +
| <code>query-string</code>
 +
| The query string of the request URI.
 +
|- valign="top"
 +
| <code>request-uri</code>
 +
| The Request URI to use for this request.
 +
|- valign="top"
 +
| <code>sub-protocols</code>
 +
| List of configured sub-protocols.
 +
|}
 +
 
 +
General information on the request can be retrieved via the [[Request Module]].
 +
 
 +
=Writing Applications=
 +
 
 +
The [[WebSocket Module]] contains functions for interacting with other clients or manage specific clients. For example, you can store and access client-specific properties for a WebSocket connection or close the connection of clients.
 +
 
 +
Note that one WebSocket connection can be opened per browser tab. In contrast, only one HTTP session exists for multiple tabs in in a browser. If you want to keep client-specific data on the web server, you can either store them in HTTP sessions or in the WebSocket connection.
 +
 
 +
Note further that the results of functions annotated with <code>%ws:close</code> or <code>%ws:error</code> will not be transmitted to the client. Both annotations have rather been designed to gracefully close connections, write log data, remove clients from session data, etc.
 +
 
 +
For keeping the connection alive it is recommendable to use heart-beats, and send regular pings to the server. There is no ideal timespan for sending pings: It should not be sent too often, but you should also consider possible network latencies.
 +
 
 +
If your HTTP connection is secure, you should use the <code>wss</code> instead of the <code>ws</code> scheme.
 +
 
 +
If you get the <code>[basex:ws] WebSocket connection required</code> error, you may be attempting to call WebSocket functions from a non-WebSocket context. If you use a proxy server, check in the configuration if WebSockets are enabled.
 +
 
 +
=Examples=
 +
 
 +
==Basic Example==
 +
 
 +
The following chapter explains how to create a simple basic web application with WebSockets. You can find another example in the BaseX source code.
 +
 
 +
First of all, you have to ensure that the <code>WsServlet</code> is enabled in your <code>web.xml</code> file. It will be enabled if you use the standard configuration of BaseX.
 +
 
 +
For establishing a connection to the WebSocket server, it is necessary that the server provides at least one function annotated with a WebSocket annotation. Let’s start by using the annotation <code>%ws:connect('/')</code>. In the connect function, a bidirectional communication with the client can be initialized: attributes such as the id and name of a client can be set, or a welcome message can be emitted to other connected users, and so on.
 +
 +
<syntaxhighlight lang="xquery">
 
declare
 
declare
   %ws:connect("/")
+
   %ws:connect('/')
  function wsdba:connect(
+
function example:connect() as empty-sequence() {
  ) {
+
};
  (: Do something after a client connects to the path "/" :)
+
</syntaxhighlight>
  };
+
 
</pre>
+
The connect function is sufficient for creating the persistent client/server connection. In order to something sensible with the connection, you should implement a function annotated with <code>%ws:message("/")</code>:
 +
 
 +
<syntaxhighlight lang="xquery">
 +
import module namespace ws = 'http://basex.org/modules/ws'
  
<pre class="brush:xquery">
 
module namespace page = 'http://basex.org/modules/web-page';
 
 
declare
 
declare
   %ws:message("/","{$message}")
+
   %ws:message('/', '{$message}')
  function chat:message(
+
function example:message(
    $message as xs:string
+
  $message as xs:string
  ){
+
) as empty-sequence() {
    (: Do something if a message arrives at the server :)
+
  ws:emit($message)
  };
+
};
</pre>
+
</syntaxhighlight>
  
<pre class="brush:xquery">
+
In the function above, the [[WebSocket Module]] is imported, and the function <code>ws:emit</code> is used for forwarding the message to all connected clients.
module namespace page = 'http://basex.org/modules/web-page';
 
(: Import the websocket module :)
 
import module namespace websocket = "http://basex.org/modules/Websocket";
 
declare
 
  %ws:close("/")
 
  function chat:close(
 
  ){
 
      (: Build a close message with the id of the client who closes the connection :)
 
      let $client-id := websocket:id()
 
      let $msg := json:serialize(
 
                                <json type="object">
 
                                  <type>WebsocketClosed</type>
 
                                  <idThatClosed>{$client-id}</idThatClosed>
 
                                </json>
 
                                )
 
      (: Broadcast the message to all connected users except the client who closes the connection :)
 
      return websocket:broadcast($msg)
 
  };
 
</pre>
 
  
=Usage=
+
The following client-side code demonstrates a basic application of the WebSocket connection:
* Enable the WebSocket servlet in the web.xml. You can set here the maxIdleTime, maxTextMessageSize and maxBinaryMessageSize too.
 
* If you get a message that exceeds the maxTextMessageSize/maxBinaryMessageSize or, if not set, the default messageSize of Jetty of 65 536 bytes (64 kB) then the connection will close. In this case, the websocket:error annotation will be called.
 
<pre class="brush:xml">
 
<servlet>
 
    <servlet-name>wsservlet</servlet-name>
 
    <servlet-class>org.basex.http.ws.WsServlet</servlet-class>
 
    <init-param>
 
      <param-name>maxIdleTime</param-name>
 
      <param-value>100000</param-value>
 
    </init-param>
 
    <init-param>
 
      <param-name>maxTextMessageSize</param-name>
 
      <param-value>3000</param-value>
 
    </init-param>
 
</servlet>
 
<servlet-mapping>
 
<servlet-name>wsservlet</servlet-name>
 
<url-pattern>/ws/*</url-pattern>
 
</servlet-mapping>
 
</pre>
 
* Annotate your specific XQuery-Functions with WebSocketAnnotations.
 
  
=Annotations=
+
<syntaxhighlight lang="javascript">
==websocket:connect(path)==
+
var ws = new WebSocket("ws://localhost:8984/ws");
Called when a client WebSocket connection successfully connected to the server. The <code>path</code> specifies the path the client connected to.
+
 
 +
ws.onmessage = function(event) {
 +
  alert(event.data);
 +
};
  
==websocket:message(path,message)==
+
function send(message) {
Called when a <code>message</code> arrives at the server. The <code>path</code> specifies the <code>path</code> the client is connected to. The <code>message</code> is the <code>message</code> sent by the client. Could be a text-message or a binary-message.
+
  ws.send(message);
 +
};
 +
</syntaxhighlight>
  
==websocket:close(path)==
+
The <code>send</code> function can be called to pass on a string to the server.
Called when the WebSocket closes. The <code>path</code> specifies the <code>path</code> the client is connected to.
 
The WebSocket is already closed when this annotation is called so there can be no return.
 
  
==websocket:error(path,message)==
+
There are no heart-beats in this example. This means that the connection is terminated if nothing happens for 5 minutes (standard timeout). It will also be closed if you send a message that exceeds the standard text size.
Called when an error occurred. Usually, this happens because of bad/malformed incoming packets. The <code>path</code> specifies the <code>path</code> the client is connected to. The <code>message</code> is the error-message.
 
The WebSocket gets closed after an error occurred.
 
  
==websocket:path(name,variable[,default]==
+
==Chat Application==
The value of the first parameter will be assigned to the variable specified as the second parameter. The third parameter can be a default value.
 
  
==websocket:header-param(name,variable[,default]==
+
In the full distributions of BaseX, you will find a little self-contained chat application that demonstrates how WebSockets can be used in practice.
For accessing specific parameters like the Http-Version or the Sec-WebSocket-Version.
 
  
=Parameters=
+
=Changelog=
* Http-Version  -> f.e.: ```%ws:param("Http-Version", "{$version}")```
 
* Origin
 
* Protocol-Version
 
* QueryString
 
* IsSecure
 
* RequestURI
 
* Host
 
* Sec-WebSocket-Version
 
* offset -> just for binary-Messages
 
* len -> just for binary-Messages
 
  
=Tipps=
+
WebSockets werre introduced with Version 9.1.
* For interacting with other clients or manage specific clients you should checkout the [[WebSocket Module]] and the [[WebSockets Module]] as well.
 

Revision as of 14:18, 27 February 2020

This page presents one of the Web Application services. It describes how to use the WebSockets API of BaseX. WebSocket is a communication protocol for providing full-duplex communication: Data can be sent in both directions and simultaneously.

Please note that the current WebSocket implementation relies on Jetty’s WebSocket servlet API. Other web servers may be supported in future versions.

Introduction

Protocol

Use WebSockets if you have to exchange data with a high frequency or if you have to send messages from the server to the client without techniques like [polling https://en.wikipedia.org/wiki/Polling_(computer_science)]. In contrast to REST, WebSockets use a single URL for the whole communication.

The WebSocket protocol was standardized in RFC 6455 by the IETF. After an initial HTTP request, all communication takes place over a single TCP connection. Unlike the HTTP protocol, a connection will be kept alive, and a server can send unsolicited data to the client.

For establishing a WebSocket connection, a handshake request is sent by the client. The web server returns a handshake response. If the handshake is successful, the persistent connection will be open until the client or the server closes it, an error occurs or a timeout happens. It is possible to transmit all kind of data, binary or text. The BaseX WebServer handles the handshake completely. You just have to define some limits of the connection in the web.xml and specify functions for WebSocket events like onConnect and onMessage.

Notice that there is no specification of a message protocol. The WebSocket protocol just specifies the message architecture but not how the payload of the messages is formatted. To agree on a format between the server and the client one can use sub-protocols.

Some older browsers don’t support the WebSocket protocol. Therefore you can use fallback options like Ajax. JavaScript client libraries like SockJS can be used for building client applications. The library takes care of how to establish the real-time connection. If the WebSocket protocol isn’t supported, it uses polling. You have to provide server functions for the fallback solutions if you have to support fallbacks.

Preliminaries

There are a bunch of annotations depending to WebSockets for annotating XQuery functions. When a WebSocket message arrives at the server, an XQuery function will be invoked that matches the constraints indicated by its annotations.

If a WebSocket function is requested (like connecting to the path /, sending a message to the path /path, …), the module directory and its sub-directories will be traversed, and all XQuery files will be parsed for functions with WebSocket annotations. Sub-directories that include an .ignore file will be skipped.

To speed up processing, the functions of the existing XQuery modules are automatically cached in main memory. For further information on cache handling, check out the RESTXQ introduction.

Configuration

  • The WebSocket servlet can be enabled and disabled in the web.xml configuration file. You can specify further configuration options, such as maxIdleTime, maxTextMessageSize, and maxBinaryMessageSize.
  • The default limit for messges is 64 KB. If you a message exceeds the default or the specified limit, an error will be raised and the connection will be closed.

Annotations

To tag functions as WebSocket functions you have to use annotations. The annotation is written after the keyword declare and before the keyword function. For the context of WebSockets there are some annotations listed below. Functions which are annotated with a WebSocket annotation will be called if the appropriate event occurs. For example, the function annotated with ws:connect('/') will be executed if a client establishes a connection with the WebSocket root path (which is, by default, ws/). By using annotations, it’s easy to provide an API for your WebSocket connection. You just have to specify what to do when a WebSocket Event occurs, annotate it with the corresponding annotation and the Servlet will do the rest for you.

%ws:connect(path)

Called directly after a successful WebSocket handshake. The path specifies the path which a client is connected to:

declare %ws:connect('/') function local:connect() { };

You can specify here how to handle your users, e. g. save a name as a WebSocket attribute. Furthermore, you can check header parameters for validity.

%ws:message(path, message)

Called when a client message arrives at the server. The path specifies the path which a client is connected to. The message string contains the name of the variable to which the message will be bound:

declare %ws:message('/', '{$info}') function local:message($info) { };

The value will be of type xs:string or xs:base64Binary. As there is no fixed message protocol, the client needs to take care of the message syntax.

%ws:error(path, message)

Called when an error occurs. The path specifies the path which a client is connected to. The message string contains the name of the variable to which the message will be bound:

declare %ws:error('/', '{$error}') function local:error($error) { };

Usually, errors happen because of bad/malformed incoming packets. The WebSocket connection gets closed after the error handling.

%ws:close(path)

Called when the WebSocket closes. The path specifies the path which a client is connected to:

declare %ws:close('/') function local:connect() { };

The WebSocket is already closed when this annotation is called so there can be no return.

%ws:header-param(name, variable[, default])

For accessing connection-specific properties like the HTTP version. The value will be bound to the specified variable. If the property has no value, an optional default value will be assigned instead:

declare
  %ws:close('host', '{$host}')
  %ws:header-param('host', '{$host}')
function local:close($host) {
  admin:write-log('Connection was closed: ' || $host)
};

The following parameters are available:

Name Description
host The host of the request URI.
http-version The HTTP version used for the request.
is-secure Indicates if the connection is secure.
origin The WebSocket origin.
protocol-version The version of the used protocol.
query-string The query string of the request URI.
request-uri The Request URI to use for this request.
sub-protocols List of configured sub-protocols.

General information on the request can be retrieved via the Request Module.

Writing Applications

The WebSocket Module contains functions for interacting with other clients or manage specific clients. For example, you can store and access client-specific properties for a WebSocket connection or close the connection of clients.

Note that one WebSocket connection can be opened per browser tab. In contrast, only one HTTP session exists for multiple tabs in in a browser. If you want to keep client-specific data on the web server, you can either store them in HTTP sessions or in the WebSocket connection.

Note further that the results of functions annotated with %ws:close or %ws:error will not be transmitted to the client. Both annotations have rather been designed to gracefully close connections, write log data, remove clients from session data, etc.

For keeping the connection alive it is recommendable to use heart-beats, and send regular pings to the server. There is no ideal timespan for sending pings: It should not be sent too often, but you should also consider possible network latencies.

If your HTTP connection is secure, you should use the wss instead of the ws scheme.

If you get the [basex:ws] WebSocket connection required error, you may be attempting to call WebSocket functions from a non-WebSocket context. If you use a proxy server, check in the configuration if WebSockets are enabled.

Examples

Basic Example

The following chapter explains how to create a simple basic web application with WebSockets. You can find another example in the BaseX source code.

First of all, you have to ensure that the WsServlet is enabled in your web.xml file. It will be enabled if you use the standard configuration of BaseX.

For establishing a connection to the WebSocket server, it is necessary that the server provides at least one function annotated with a WebSocket annotation. Let’s start by using the annotation %ws:connect('/'). In the connect function, a bidirectional communication with the client can be initialized: attributes such as the id and name of a client can be set, or a welcome message can be emitted to other connected users, and so on.

declare
  %ws:connect('/')
function example:connect() as empty-sequence() {
};

The connect function is sufficient for creating the persistent client/server connection. In order to something sensible with the connection, you should implement a function annotated with %ws:message("/"):

import module namespace ws = 'http://basex.org/modules/ws'

declare
  %ws:message('/', '{$message}')
function example:message(
  $message  as xs:string
) as empty-sequence() {
  ws:emit($message)
};

In the function above, the WebSocket Module is imported, and the function ws:emit is used for forwarding the message to all connected clients.

The following client-side code demonstrates a basic application of the WebSocket connection:

var ws = new WebSocket("ws://localhost:8984/ws");

ws.onmessage = function(event) {
  alert(event.data);
};

function send(message) {
  ws.send(message);
};

The send function can be called to pass on a string to the server.

There are no heart-beats in this example. This means that the connection is terminated if nothing happens for 5 minutes (standard timeout). It will also be closed if you send a message that exceeds the standard text size.

Chat Application

In the full distributions of BaseX, you will find a little self-contained chat application that demonstrates how WebSockets can be used in practice.

Changelog

WebSockets werre introduced with Version 9.1.