Securing All Endpoints

Monday, 18 January 2021 · 12 messages · last reply 27 Jan 2021
AL
Adam Law18 Jan 2021 03:19 #
Golang has great middleware functionality for endpoints that allows a person to readily secure all of them. I was wondering there would be anyway doing similar in BaseX for restxq & websockets. For example by requiring all requests to have a particular query parameter (very basic example). ... to protect say against browser / javascript port scanning. Eg https://stackoverflow.com/questions/26204485/gorilla-mux-custom-middleware and check that request contains a particular parameter (very basic security) It seems that I have to secure each endpoint individually. Adam
CG
Christian Grün18 Jan 2021 10:26 #
Hi Adam, You could have a loot at our Permissions layer [1]. It can be used for all kinds of checks, not only those specific to authentication. I think we should generalize the concept in a future version of BaseX and introduce additional post-hooks. Hope this helps? Christian [1] https://docs.basex.org/wiki/Permissions
Quoted text · Adam Law · 9 lines
On Mon, Jan 18, 2021 at 3:19 AM Adam Law <adamjameslaw at gmail.com> wrote: > > Golang has great middleware functionality for endpoints that allows a person to readily secure all of them. I was wondering there would be anyway doing similar in BaseX for restxq & websockets. For example by requiring all requests to have a particular query parameter (very basic example). ... to protect say against browser / javascript port scanning. > > Eg https://stackoverflow.com/questions/26204485/gorilla-mux-custom-middleware and check that request contains a particular parameter (very basic security) > > It seems that I have to secure each endpoint individually. > > Adam
AL
Adam Law18 Jan 2021 13:16 #
Hi Christian, Many thanks for your reply. Pre and Post Hooks would be great Here is some crazy stuff I thought of (just a skeleton). It should allow say a code search for %((rest)|(ws)).*:((path)|(connect))to ensure that all endpoints have permissions middleware of some sort. Hope it is not too silly. Adam /--------------------------------- import module namespace authMiddleware = 'http://middleware..auth' at '[...].xqm' declare
%rest:preHook:(authMiddleware:authoriseFail2BanRest):path("/main/admin"):result($vs_authKeyResult)
  %rest:postHook:(noCaching:noCacheFunc):(htmlTextConversion:text2HTMLFunc)
  %output:method("text")
function local:admin($vs_authKeyResult) {
  let $authorized := 'Welcome to the admin page'
  let $unauthorized := 'Your IP has been logged with fail2ban'
  let $result := if ($vs_authKeyResult=true()) then $authorized else
$unauthorized
  return $result
};
declare
%ws:preHook:(authMiddleware:authoriseFail2BanWS):connect('/'):result($vs_authKeyResult)
function local:connect($vs_authKeyResult) as empty-sequence() {
  let $id := ws:id()
  let $message := json:serialize(map {
    'type': 'Connect',
    'id': $id
  })
  return if ($vs_authKeyResult=true()) then ws:broadcast($message) else ()
};
/--------------------------------- module namespace authMiddleware = 'http://middleware.auth'; import module namespace loggingMiddleware = 'http://middleware.logging' at '[...].xqm'
declare
%preHook:(loggingMiddleware:functionLog):result($vs_logResult)  (:a
postHook here would be before the calling function local:admin :)
%rest:query-param("vs_authKey", " {$vs_authKey}")
function authMiddleware:authoriseFail2BanRest($vs_authKey,$vs_logResult) {
  let $result := if ($vs_authKey='YKJKhh') then true() else false()
  let $result := if ($vs_logResult='override') then true() else result()
  return $result
};

/---------------------------------
module namespace loggingMiddleware = 'http://middleware.logging';
module namespace myUpdatingOutput = 'http://function.updatingOutput';

declare
%rest:query-param("vs_logAside", " {$vs_logAside}")
function loggingMiddleware:functionLog($vs_logAside) {
  let $result := if (vs_logAside != 'adminOverride*75x4558') then
myUpdatingOutput:logwrite(request:remote-address()) else 'override'
  return $result
};
Quoted text · Christian Grün · 2 lines
> >
AL
Adam Lawin reply to Christian Grün, 18 Jan 2021, 10:2618 Jan 2021 13:30 #
Another thought - a 'catch all' path for fail2ban to catch attempted accessess to undefined paths. I haven't tested this to see if there is such a thing in BaseX https://stackoverflow.com/questions/50282541/catch-all-url-in-golang
Quoted text · Christian Grün · 32 lines
On Mon, Jan 18, 2021 at 5:26 PM Christian Grün <christian.gruen at gmail.com> wrote: > Hi Adam, > > You could have a loot at our Permissions layer [1]. It can be used for > all kinds of checks, not only those specific to authentication. I > think we should generalize the concept in a future version of BaseX > and introduce additional post-hooks. > > Hope this helps? > Christian > > [1] https://docs.basex.org/wiki/Permissions > > > > On Mon, Jan 18, 2021 at 3:19 AM Adam Law <adamjameslaw at gmail.com> wrote: > > > > Golang has great middleware functionality for endpoints that allows a > person to readily secure all of them. I was wondering there would be > anyway doing similar in BaseX for restxq & websockets. For example by > requiring all requests to have a particular query parameter (very basic > example). ... to protect say against browser / javascript port scanning. > > > > Eg > https://stackoverflow.com/questions/26204485/gorilla-mux-custom-middleware > and check that request contains a particular parameter (very basic security) > > > > It seems that I have to secure each endpoint individually. > > > > Adam >
CG
Christian Grün18 Jan 2021 13:33 #
You could have a look at the "Catch HTTP Errors" section in our documentation. [1] https://docs.basex.org/wiki/RESTXQ#Error_Handling
Quoted text · Adam Law · 31 lines
On Mon, Jan 18, 2021 at 1:30 PM Adam Law <adamjameslaw at gmail.com> wrote: > > Another thought - a 'catch all' path for fail2ban to catch attempted accessess to undefined paths. > I haven't tested this to see if there is such a thing in BaseX > https://stackoverflow.com/questions/50282541/catch-all-url-in-golang > > On Mon, Jan 18, 2021 at 5:26 PM Christian Grün <christian.gruen at gmail.com> wrote: >> >> Hi Adam, >> >> You could have a loot at our Permissions layer [1]. It can be used for >> all kinds of checks, not only those specific to authentication. I >> think we should generalize the concept in a future version of BaseX >> and introduce additional post-hooks. >> >> Hope this helps? >> Christian >> >> [1] https://docs.basex.org/wiki/Permissions >> >> >> >> On Mon, Jan 18, 2021 at 3:19 AM Adam Law <adamjameslaw at gmail.com> wrote: >> > >> > Golang has great middleware functionality for endpoints that allows a person to readily secure all of them. I was wondering there would be anyway doing similar in BaseX for restxq & websockets. For example by requiring all requests to have a particular query parameter (very basic example). ... to protect say against browser / javascript port scanning. >> > >> > Eg https://stackoverflow.com/questions/26204485/gorilla-mux-custom-middleware and check that request contains a particular parameter (very basic security) >> > >> > It seems that I have to secure each endpoint individually. >> > >> > Adam
AL
Adam Lawin reply to Adam Law, 18 Jan 2021, 13:1618 Jan 2021 15:35 #
Quoted text · Adam Law · 5 lines
> > Logging would have to know the point in the pre chain somehow to be able >>> to log the call from 'local:admin' (in loggingMiddleware:functionLog ) >>> >>>>
AL
Adam Lawin reply to Adam Law, 18 Jan 2021, 13:1619 Jan 2021 05:20 #
More Silly import module namespace authMiddleware = 'http://middleware..auth' at '[...].xqm' declare
%rest:preHook:(authMiddleware:authoriseFail2BanRest:input("/main/admin",,)):path("/main/admin"):result($vs_authKeyResult)
  %rest:query-param("vbJson", " {$vbJson}")
  %if ($vbJSON = 'true') then
%rest:postHook:(noCaching:noCacheFunc):(JSONConversion:xml2JOSNFunc:input($resultdifferent))

  %output:method("xml")
function local:admin($vs_authKeyResult) {
  let $authorized := 'Welcome to the admin page'
  let $unauthorized := 'Your IP has been logged with fail2ban'
  let $resultdifferent := if ($vs_authKeyResult=true()) then
<jsonRep>{$authorized}</jsonRep> else <jsonRep>{$unauthorized}</jsonRep>{
  return <xml>{$resultdifferent/*}</xml>
};
/--------------------------------- module namespace authMiddleware = 'http://middleware.auth'; import module namespace loggingMiddleware = 'http://middleware.logging' at '[...].xqm'
declare
%variable $vsPath external = "/";
%rest:query-param("vs_logAside", " {$vs_logAside}")
%preHook:(loggingMiddleware:functionLog:input($vsPath,$vs_logAside)):result($vs_logResult)
 (:a postHook here would be before the calling function local:admin :)
%rest:query-param("vs_authKey", " {$vs_authKey}")
function
authMiddleware:authoriseFail2BanRest($vsPath,$vs_authKey,$vs_logResult) {
  let $result := if ($vs_authKey='YKJKhh') then true() else false()
  let $result := if ($vs_logResult='override') then true() else result()
  return $result
};

/---------------------------------
module namespace loggingMiddleware = 'http://middleware.logging';
module namespace myUpdatingOutput = 'http://function.updatingOutput';
declare
%variable $vsPath external = "/";
%variable $vs_logAside external = "/";
function loggingMiddleware:functionLog($vsPath,$vs_logAside) {
  let $result := if (vs_logAside != 'adminOverride*75x4558') then
myUpdatingOutput:logwrite($vsPath,request:remote-address()) else 'override'
  return $result
};
AL
Adam Law19 Jan 2021 05:22 #
Hi Christian, Thanks for the permissions pointer- it works well. As an aside I had to use
   let $vbFailure1 := ($perm?authorization !=  'example' or exists(
$perm?authorization )=false())
Rather than
  let $vbFailure1 := ($perm?authorization !=  'example' )
To handle the case of no Authorisation. I am not sure why. I should be able to use request:cookie for ws (when I need to) which is great. I wasn't able to work out a catchall rest:path solution but it is not really important for me. Thanks again for the pointer. Adam
Quoted text · Christian Grün · 48 lines
On Mon, Jan 18, 2021 at 8:33 PM Christian Grün <christian.gruen at gmail.com> wrote: > You could have a look at the "Catch HTTP Errors" section in our > documentation. > > [1] https://docs.basex.org/wiki/RESTXQ#Error_Handling > > On Mon, Jan 18, 2021 at 1:30 PM Adam Law <adamjameslaw at gmail.com> wrote: > > > > Another thought - a 'catch all' path for fail2ban to catch attempted > accessess to undefined paths. > > I haven't tested this to see if there is such a thing in BaseX > > https://stackoverflow.com/questions/50282541/catch-all-url-in-golang > > > > On Mon, Jan 18, 2021 at 5:26 PM Christian Grün < > christian.gruen at gmail.com> wrote: > >> > >> Hi Adam, > >> > >> You could have a loot at our Permissions layer [1]. It can be used for > >> all kinds of checks, not only those specific to authentication. I > >> think we should generalize the concept in a future version of BaseX > >> and introduce additional post-hooks. > >> > >> Hope this helps? > >> Christian > >> > >> [1] https://docs.basex.org/wiki/Permissions > >> > >> > >> > >> On Mon, Jan 18, 2021 at 3:19 AM Adam Law <adamjameslaw at gmail.com> > wrote: > >> > > >> > Golang has great middleware functionality for endpoints that allows a > person to readily secure all of them. I was wondering there would be > anyway doing similar in BaseX for restxq & websockets. For example by > requiring all requests to have a particular query parameter (very basic > example). ... to protect say against browser / javascript port scanning. > >> > > >> > Eg > https://stackoverflow.com/questions/26204485/gorilla-mux-custom-middleware > and check that request contains a particular parameter (very basic security) > >> > > >> > It seems that I have to secure each endpoint individually. > >> > > >> > Adam >
AL
Adam Law19 Jan 2021 06:44 #
Does anyone have any experience running basex (windows) - jetty on https with hot reload https://github.com/eclipse/jetty.project/issues/5019.
AL
Adam Law20 Jan 2021 02:47 #
This is how I appear to be able to set SSL on 8446 in jetty.xml but ssl-reload (hot reload) is beyond me.
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN"
  "http://www.eclipse.org/jetty/configure_9_3.dtd">

<Configure id="Server" class="org.eclipse.jetty.server.Server">

    <New id="httpConfig" class="org.eclipse.jetty.server.HttpConfiguration">
        <Set name="secureScheme">https</Set>
        <Set name="securePort">8446</Set>
    </New>

    <New id="httpsConfig"
class="org.eclipse.jetty.server.HttpConfiguration">
        <Call name="addCustomizer">
            <Arg>
                <New
class="org.eclipse.jetty.server.SecureRequestCustomizer" />
            </Arg>
        </Call>
    </New>

    <New id="connector" class="org.eclipse.jetty.server.ServerConnector">
        <Arg name="server">
            <Ref refid="Server" />
        </Arg>
        <Arg name="factories">
            <Array type="org.eclipse.jetty.server.ConnectionFactory">
                <Item>
                    <New
class="org.eclipse.jetty.server.HttpConnectionFactory">
                        <Arg name="config">
                            <Ref refid="httpConfig" />
                        </Arg>
                    </New>
                </Item>
            </Array>
        </Arg>
        <Set name="host">0.0.0.0</Set>
        <Set name="port">8984</Set>
        <Set name="idleTimeout">60000</Set>
        <Set name="reuseAddress">true</Set>
    </New>

    <New id="sslContextFactory"
class="org.eclipse.jetty.util.ssl.SslContextFactory$Server">
        <Set
name="KeyStorePath">D:/D_Ser/CP_Certificates/CertifyTheWebACME.store</Set>
        <Set name="KeyStorePassword">Password</Set>
    </New>

    <New id="sslConnectionFactory"
class="org.eclipse.jetty.server.SslConnectionFactory">
        <Arg name="sslContextFactory">
            <Ref refid="sslContextFactory" />
        </Arg>
        <Arg name="next">http/1.1</Arg>
    </New>

    <New id="sslConnector" class="org.eclipse.jetty.server.ServerConnector">
        <Arg name="server">
            <Ref refid="Server" />
        </Arg>
        <Arg name="factories">
            <Array type="org.eclipse.jetty.server.ConnectionFactory">
                <Item>
                    <Ref refid="sslConnectionFactory" />
                </Item>
                <Item>
                    <New
class="org.eclipse.jetty.server.HttpConnectionFactory">
                        <Arg name="config">
                            <Ref refid="httpsConfig" />
                        </Arg>
                    </New>
                </Item>
            </Array>
        </Arg>
        <Set name="port">8446</Set>
    </New>

    <Call name="setConnectors">
        <Arg>
            <Array type="org.eclipse.jetty.server.ServerConnector">
                <Item>
                    <Ref refid="connector" />
                </Item>
                <Item>
                    <Ref refid="sslConnector" />
                </Item>
            </Array>
        </Arg>
    </Call>

    <!-- Default connector. The Jetty stop port can be specified in the
.basex or pom.xml configuration file.  -->

</Configure>
CG
Christian Grünin reply to Adam Law, 19 Jan 2021, 05:2226 Jan 2021 11:25 #
Hi Adam,
Quoted text · Adam Law · 1 line
> let $vbFailure1 := ($perm?authorization != 'example' or exists( $perm?authorization )=false())
If you want, you can simplify this to:
  let $vbFailure1 := not($perm?authorization =  'example')
If $perm?authorization yields no item (i.e., an “empty sequence”), the comparison () != 'example' will give you false, because there is no item on the left-hand side of the comparison that matches 'example'. Hope this helps, Christian
AL
Adam Law27 Jan 2021 09:24 #
Hi Christian - thanks for the explanation. Adam
Quoted text · Christian Grün · 18 lines
On Tue, Jan 26, 2021 at 6:26 PM Christian Grün <christian.gruen at gmail.com> wrote: > Hi Adam, > > > let $vbFailure1 := ($perm?authorization != 'example' or exists( > $perm?authorization )=false()) > > If you want, you can simplify this to: > > let $vbFailure1 := not($perm?authorization = 'example') > > If $perm?authorization yields no item (i.e., an “empty sequence”), the > comparison () != 'example' will give you false, because there is no > item on the left-hand side of the comparison that matches 'example'. > > Hope this helps, > Christian >

⚡Generated with XQuery