Securing All Endpoints
Monday, 18 January 2021 · 12 messages · last reply 27 Jan 2021
AL
Adam Law18 Jan 2021 03:19 #
Golang has great middleware functionality for endpoints that allows a person to readily secure all of them. I was wondering there would be anyway doing similar in BaseX for restxq & websockets. For example by requiring all requests to have a particular query parameter (very basic example). ... to protect say against browser / javascript port scanning.
Eg
https://stackoverflow.com/questions/26204485/gorilla-mux-custom-middleware and check that request contains a particular parameter (very basic security)
It seems that I have to secure each endpoint individually.
Adam
CG
Christian Grün18 Jan 2021 10:26 #
Hi Adam,
You could have a loot at our Permissions layer [1]. It can be used for all kinds of checks, not only those specific to authentication. I think we should generalize the concept in a future version of BaseX and introduce additional post-hooks.
Hope this helps?
Christian
[1] https://docs.basex.org/wiki/Permissions
Quoted text · Adam Law · 9 lines
On Mon, Jan 18, 2021 at 3:19 AM Adam Law <adamjameslaw at gmail.com> wrote: > > Golang has great middleware functionality for endpoints that allows a person to readily secure all of them. I was wondering there would be anyway doing similar in BaseX for restxq & websockets. For example by requiring all requests to have a particular query parameter (very basic example). ... to protect say against browser / javascript port scanning. > > Eg https://stackoverflow.com/questions/26204485/gorilla-mux-custom-middleware and check that request contains a particular parameter (very basic security) > > It seems that I have to secure each endpoint individually. > > Adam
AL
Adam Law18 Jan 2021 13:16 #
Hi Christian,
Many thanks for your reply. Pre and Post Hooks would be great
Here is some crazy stuff I thought of (just a skeleton). It should allow say a code search for %((rest)|(ws)).*:((path)|(connect))to ensure that all endpoints have permissions middleware of some sort.
Hope it is not too silly.
Adam
/---------------------------------
import module namespace authMiddleware = 'http://middleware..auth' at '[...].xqm'
declare
%rest:preHook:(authMiddleware:authoriseFail2BanRest):path("/main/admin"):result($vs_authKeyResult)
%rest:postHook:(noCaching:noCacheFunc):(htmlTextConversion:text2HTMLFunc)
%output:method("text")
function local:admin($vs_authKeyResult) {
let $authorized := 'Welcome to the admin page'
let $unauthorized := 'Your IP has been logged with fail2ban'
let $result := if ($vs_authKeyResult=true()) then $authorized else
$unauthorized
return $result
};declare
%ws:preHook:(authMiddleware:authoriseFail2BanWS):connect('/'):result($vs_authKeyResult)
function local:connect($vs_authKeyResult) as empty-sequence() {
let $id := ws:id()
let $message := json:serialize(map {
'type': 'Connect',
'id': $id
})
return if ($vs_authKeyResult=true()) then ws:broadcast($message) else ()
};/---------------------------------
module namespace authMiddleware = 'http://middleware.auth';
import module namespace loggingMiddleware = 'http://middleware.logging' at '[...].xqm'
declare
%preHook:(loggingMiddleware:functionLog):result($vs_logResult) (:a
postHook here would be before the calling function local:admin :)
%rest:query-param("vs_authKey", " {$vs_authKey}")
function authMiddleware:authoriseFail2BanRest($vs_authKey,$vs_logResult) {
let $result := if ($vs_authKey='YKJKhh') then true() else false()
let $result := if ($vs_logResult='override') then true() else result()
return $result
};
/---------------------------------
module namespace loggingMiddleware = 'http://middleware.logging';
module namespace myUpdatingOutput = 'http://function.updatingOutput';
declare
%rest:query-param("vs_logAside", " {$vs_logAside}")
function loggingMiddleware:functionLog($vs_logAside) {
let $result := if (vs_logAside != 'adminOverride*75x4558') then
myUpdatingOutput:logwrite(request:remote-address()) else 'override'
return $result
};Quoted text · Christian Grün · 2 lines
> >
AL
Another thought - a 'catch all' path for fail2ban to catch attempted accessess to undefined paths.
I haven't tested this to see if there is such a thing in BaseX https://stackoverflow.com/questions/50282541/catch-all-url-in-golang
Quoted text · Christian Grün · 32 lines
On Mon, Jan 18, 2021 at 5:26 PM Christian Grün <christian.gruen at gmail.com> wrote: > Hi Adam, > > You could have a loot at our Permissions layer [1]. It can be used for > all kinds of checks, not only those specific to authentication. I > think we should generalize the concept in a future version of BaseX > and introduce additional post-hooks. > > Hope this helps? > Christian > > [1] https://docs.basex.org/wiki/Permissions > > > > On Mon, Jan 18, 2021 at 3:19 AM Adam Law <adamjameslaw at gmail.com> wrote: > > > > Golang has great middleware functionality for endpoints that allows a > person to readily secure all of them. I was wondering there would be > anyway doing similar in BaseX for restxq & websockets. For example by > requiring all requests to have a particular query parameter (very basic > example). ... to protect say against browser / javascript port scanning. > > > > Eg > https://stackoverflow.com/questions/26204485/gorilla-mux-custom-middleware > and check that request contains a particular parameter (very basic security) > > > > It seems that I have to secure each endpoint individually. > > > > Adam >
CG
Christian Grün18 Jan 2021 13:33 #
You could have a look at the "Catch HTTP Errors" section in our documentation.
[1] https://docs.basex.org/wiki/RESTXQ#Error_Handling
Quoted text · Adam Law · 31 lines
On Mon, Jan 18, 2021 at 1:30 PM Adam Law <adamjameslaw at gmail.com> wrote: > > Another thought - a 'catch all' path for fail2ban to catch attempted accessess to undefined paths. > I haven't tested this to see if there is such a thing in BaseX > https://stackoverflow.com/questions/50282541/catch-all-url-in-golang > > On Mon, Jan 18, 2021 at 5:26 PM Christian Grün <christian.gruen at gmail.com> wrote: >> >> Hi Adam, >> >> You could have a loot at our Permissions layer [1]. It can be used for >> all kinds of checks, not only those specific to authentication. I >> think we should generalize the concept in a future version of BaseX >> and introduce additional post-hooks. >> >> Hope this helps? >> Christian >> >> [1] https://docs.basex.org/wiki/Permissions >> >> >> >> On Mon, Jan 18, 2021 at 3:19 AM Adam Law <adamjameslaw at gmail.com> wrote: >> > >> > Golang has great middleware functionality for endpoints that allows a person to readily secure all of them. I was wondering there would be anyway doing similar in BaseX for restxq & websockets. For example by requiring all requests to have a particular query parameter (very basic example). ... to protect say against browser / javascript port scanning. >> > >> > Eg https://stackoverflow.com/questions/26204485/gorilla-mux-custom-middleware and check that request contains a particular parameter (very basic security) >> > >> > It seems that I have to secure each endpoint individually. >> > >> > Adam
AL
Quoted text · Adam Law · 5 lines
> > Logging would have to know the point in the pre chain somehow to be able >>> to log the call from 'local:admin' (in loggingMiddleware:functionLog ) >>> >>>>
AL
More Silly
import module namespace authMiddleware = 'http://middleware..auth' at '[...].xqm'
declare
%rest:preHook:(authMiddleware:authoriseFail2BanRest:input("/main/admin",,)):path("/main/admin"):result($vs_authKeyResult)
%rest:query-param("vbJson", " {$vbJson}")
%if ($vbJSON = 'true') then
%rest:postHook:(noCaching:noCacheFunc):(JSONConversion:xml2JOSNFunc:input($resultdifferent))
%output:method("xml")
function local:admin($vs_authKeyResult) {
let $authorized := 'Welcome to the admin page'
let $unauthorized := 'Your IP has been logged with fail2ban'
let $resultdifferent := if ($vs_authKeyResult=true()) then
<jsonRep>{$authorized}</jsonRep> else <jsonRep>{$unauthorized}</jsonRep>{
return <xml>{$resultdifferent/*}</xml>
};/---------------------------------
module namespace authMiddleware = 'http://middleware.auth';
import module namespace loggingMiddleware = 'http://middleware.logging' at '[...].xqm'
declare
%variable $vsPath external = "/";
%rest:query-param("vs_logAside", " {$vs_logAside}")
%preHook:(loggingMiddleware:functionLog:input($vsPath,$vs_logAside)):result($vs_logResult)
(:a postHook here would be before the calling function local:admin :)
%rest:query-param("vs_authKey", " {$vs_authKey}")
function
authMiddleware:authoriseFail2BanRest($vsPath,$vs_authKey,$vs_logResult) {
let $result := if ($vs_authKey='YKJKhh') then true() else false()
let $result := if ($vs_logResult='override') then true() else result()
return $result
};
/---------------------------------
module namespace loggingMiddleware = 'http://middleware.logging';
module namespace myUpdatingOutput = 'http://function.updatingOutput';
declare
%variable $vsPath external = "/";
%variable $vs_logAside external = "/";
function loggingMiddleware:functionLog($vsPath,$vs_logAside) {
let $result := if (vs_logAside != 'adminOverride*75x4558') then
myUpdatingOutput:logwrite($vsPath,request:remote-address()) else 'override'
return $result
};AL
Adam Law19 Jan 2021 05:22 #
Hi Christian,
Thanks for the permissions pointer- it works well.
As an aside I had to use
let $vbFailure1 := ($perm?authorization != 'example' or exists(
$perm?authorization )=false())Rather than
let $vbFailure1 := ($perm?authorization != 'example' )To handle the case of no Authorisation. I am not sure why.
I should be able to use
request:cookie for ws (when I need to) which is great.
I wasn't able to work out a catchall rest:path solution but it is not really important for me.
Thanks again for the pointer.
AdamQuoted text · Christian Grün · 48 lines
On Mon, Jan 18, 2021 at 8:33 PM Christian Grün <christian.gruen at gmail.com> wrote: > You could have a look at the "Catch HTTP Errors" section in our > documentation. > > [1] https://docs.basex.org/wiki/RESTXQ#Error_Handling > > On Mon, Jan 18, 2021 at 1:30 PM Adam Law <adamjameslaw at gmail.com> wrote: > > > > Another thought - a 'catch all' path for fail2ban to catch attempted > accessess to undefined paths. > > I haven't tested this to see if there is such a thing in BaseX > > https://stackoverflow.com/questions/50282541/catch-all-url-in-golang > > > > On Mon, Jan 18, 2021 at 5:26 PM Christian Grün < > christian.gruen at gmail.com> wrote: > >> > >> Hi Adam, > >> > >> You could have a loot at our Permissions layer [1]. It can be used for > >> all kinds of checks, not only those specific to authentication. I > >> think we should generalize the concept in a future version of BaseX > >> and introduce additional post-hooks. > >> > >> Hope this helps? > >> Christian > >> > >> [1] https://docs.basex.org/wiki/Permissions > >> > >> > >> > >> On Mon, Jan 18, 2021 at 3:19 AM Adam Law <adamjameslaw at gmail.com> > wrote: > >> > > >> > Golang has great middleware functionality for endpoints that allows a > person to readily secure all of them. I was wondering there would be > anyway doing similar in BaseX for restxq & websockets. For example by > requiring all requests to have a particular query parameter (very basic > example). ... to protect say against browser / javascript port scanning. > >> > > >> > Eg > https://stackoverflow.com/questions/26204485/gorilla-mux-custom-middleware > and check that request contains a particular parameter (very basic security) > >> > > >> > It seems that I have to secure each endpoint individually. > >> > > >> > Adam >
AL
Adam Law19 Jan 2021 06:44 #
Does anyone have any experience running basex (windows) - jetty on https with hot reload
https://github.com/eclipse/jetty.project/issues/5019.
AL
Adam Law20 Jan 2021 02:47 #
This is how I appear to be able to set SSL on 8446 in jetty.xml but ssl-reload (hot reload) is beyond me.
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN"
"http://www.eclipse.org/jetty/configure_9_3.dtd">
<Configure id="Server" class="org.eclipse.jetty.server.Server">
<New id="httpConfig" class="org.eclipse.jetty.server.HttpConfiguration">
<Set name="secureScheme">https</Set>
<Set name="securePort">8446</Set>
</New>
<New id="httpsConfig"
class="org.eclipse.jetty.server.HttpConfiguration">
<Call name="addCustomizer">
<Arg>
<New
class="org.eclipse.jetty.server.SecureRequestCustomizer" />
</Arg>
</Call>
</New>
<New id="connector" class="org.eclipse.jetty.server.ServerConnector">
<Arg name="server">
<Ref refid="Server" />
</Arg>
<Arg name="factories">
<Array type="org.eclipse.jetty.server.ConnectionFactory">
<Item>
<New
class="org.eclipse.jetty.server.HttpConnectionFactory">
<Arg name="config">
<Ref refid="httpConfig" />
</Arg>
</New>
</Item>
</Array>
</Arg>
<Set name="host">0.0.0.0</Set>
<Set name="port">8984</Set>
<Set name="idleTimeout">60000</Set>
<Set name="reuseAddress">true</Set>
</New>
<New id="sslContextFactory"
class="org.eclipse.jetty.util.ssl.SslContextFactory$Server">
<Set
name="KeyStorePath">D:/D_Ser/CP_Certificates/CertifyTheWebACME.store</Set>
<Set name="KeyStorePassword">Password</Set>
</New>
<New id="sslConnectionFactory"
class="org.eclipse.jetty.server.SslConnectionFactory">
<Arg name="sslContextFactory">
<Ref refid="sslContextFactory" />
</Arg>
<Arg name="next">http/1.1</Arg>
</New>
<New id="sslConnector" class="org.eclipse.jetty.server.ServerConnector">
<Arg name="server">
<Ref refid="Server" />
</Arg>
<Arg name="factories">
<Array type="org.eclipse.jetty.server.ConnectionFactory">
<Item>
<Ref refid="sslConnectionFactory" />
</Item>
<Item>
<New
class="org.eclipse.jetty.server.HttpConnectionFactory">
<Arg name="config">
<Ref refid="httpsConfig" />
</Arg>
</New>
</Item>
</Array>
</Arg>
<Set name="port">8446</Set>
</New>
<Call name="setConnectors">
<Arg>
<Array type="org.eclipse.jetty.server.ServerConnector">
<Item>
<Ref refid="connector" />
</Item>
<Item>
<Ref refid="sslConnector" />
</Item>
</Array>
</Arg>
</Call>
<!-- Default connector. The Jetty stop port can be specified in the
.basex or pom.xml configuration file. -->
</Configure>CG
Hi Adam,
Quoted text · Adam Law · 1 line
> let $vbFailure1 := ($perm?authorization != 'example' or exists( $perm?authorization )=false())
If you want, you can simplify this to:
let $vbFailure1 := not($perm?authorization = 'example')If $perm?authorization yields no item (i.e., an “empty sequence”), the comparison () != 'example' will give you false, because there is no item on the left-hand side of the comparison that matches 'example'.
Hope this helps,
Christian
AL
Adam Law27 Jan 2021 09:24 #
Hi Christian - thanks for the explanation.
Adam
Quoted text · Christian Grün · 18 lines
On Tue, Jan 26, 2021 at 6:26 PM Christian Grün <christian.gruen at gmail.com> wrote: > Hi Adam, > > > let $vbFailure1 := ($perm?authorization != 'example' or exists( > $perm?authorization )=false()) > > If you want, you can simplify this to: > > let $vbFailure1 := not($perm?authorization = 'example') > > If $perm?authorization yields no item (i.e., an “empty sequence”), the > comparison () != 'example' will give you false, because there is no > item on the left-hand side of the comparison that matches 'example'. > > Hope this helps, > Christian >
Written for BaseX 9.4. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery