Main Page » Mailing List » November 2013

Decoupling custom code with restxq

Thursday, 28 November 2013 · 7 messages · last reply 2 Dec 2013
WS
Wiemer, Sebastian28 Nov 2013 13:14 #
Hi, we are developing an application based on BaseX with RESTXQ. The application uses the BaseX RESTXQ URL mechanism to decouple extensions from product code. application.xqm: (this is shipped product code)
declare %restxq:path(„dispatch/do/{$key}")

  function app:defaultReq($key) {
(: default code :) The custom code simply has a more specific RESTXQ path specified, so it is used instead of the shipped code. custom.xqm: (this is the customers extension)
declare %restxq:path(„dispatch/do/something_special")

  function custom:customReq() {
(: some custom code :) }; So just by dropping a file like „custom.xqm" in the restxq path, we can customize the system. Without having to use a config file or changing our shipped application code. Removing the file from the path removes the custom functions. And: we can mix in updating functions without having to know about this in advance. The drawback is that we can only use this for HTTPS requests from the client, but not from the server, because we use HTTPS. So we cannot use RESTXQ requests server side with http:send-request. (Or do we?) I tried to workaround by setting up jetty to allow not secured HTTP connections from 127.0.0.1 by setting up a second connector. That works (it’s possible to use the site locally with HTTP), but I could not get around our form-based authentication with http:send-request. As far as I can see, only BASIC authentication is possible (not an option for us). Is there another way? A java extension perhaps? Sebastian
CG
Christian Grün30 Nov 2013 18:11 #
Hi Sebastian, thanks for your e-mail; I’m pleased to see you are still working with BaseX.
Quoted text · Wiemer, Sebastian · 2 lines
> So just by dropping a file like „custom.xqm" in the restxq path, we can > customize the system.
…sounds like a nice idea.
Quoted text · Wiemer, Sebastian · 3 lines
> The drawback is that we can only use this for HTTPS requests from the > client, but not from the server, because we use HTTPS. So we cannot use > RESTXQ requests server side with http:send-request. (Or do we?)
I guess I have no answer to that, as I haven’t actually come across this challenge yet. This might be completely off the track, but maybe it’s also a question of how Jetty is configured? Did you already check out the jetty.xml file and possible settings? Christian
WS
Wiemer, Sebastian30 Nov 2013 22:30 #
Hi Christian,
Quoted text · Christian Grün · 5 lines
> Hi Sebastian, > > > > thanks for your e-mail; I’m pleased to see you are still working with BaseX.
And we still love BaseX :-) Thanks for keeping up the good work! BTW: I unfortunately missed your and Alexanders presentation at the MarkUp Forum: are there any slides available?
Quoted text · Christian Grün · 26 lines
> > So just by dropping a file like „custom.xqm" in the restxq path, we can > > > > customize the system. > > > > …sounds like a nice idea. > > > > > > The drawback is that we can only use this for HTTPS requests from the > > > > client, but not from the server, because we use HTTPS. So we cannot use > > > > RESTXQ requests server side with http:send-request. (Or do we?) > > > > I guess I have no answer to that, as I haven’t actually come across > > this challenge yet. This might be completely off the track, but maybe > > it’s also a question of how Jetty is configured? Did you already check > > out the jetty.xml file and possible settings?
Yes, I checked jetty configuration (but I gave up), and it seems to me, that jetty.xml or web.xml might not be the right place for a solution after all. What Florian and I probably are trying to accomplish here is a way to introduce aspect oriented or rule based programming to XQuery to decouple custom extensions from product code. This is similar to imported modules in XSLT: If you import template rules, which are more specific, they will be applied first. Could this be applied to XQuery as well? (But in XSLT you would have to import the custom code and hence change your code. We can do that in XQuery as well, but it is not completely decoupled, is it?) Florian and I came around with an idea, that does not use HTTP requests and maybe clarify our goal: how about introducing a „dispatch" annotation to XQuery? Let’s assume we have a function „age“ that takes an age as integer and a name as string and prints a message. A customization would be to be more specific for teenagers. But we don’t want to change the code. the annotation „dispatch:declare“ declares a function to be potentially dispatched: if there is not a more specific function with a „dispatch:replace“ annotation, than that function is called. Maybe "dispatch:prepend" and "dispatch:append“ would be nice as well. Also it would come in very handy if the replacing function is allowed be updating or not updating. product.xqm %dispatch:declare(age($age,$name))
declare function prod:age($age as xs:integer,$name as xs:string){

  $name||" is "||$age||" years old."
};
prod:age(14,'Emily’) = „Emily is 14 years old."
custom.xqm %dispatch:replace(age(19>$age>13,$name))
declare function custom:age($age as xs:integer,$name as xs:string){

  $name||" is a "||$age||“ years old teenager."
}; So a use of „prod:age“ would yield the result of „custom:age“:
prod:age(14,'Emily’) = „Emily is a 14 years old teenager.“
Comments on this idea are very appreciated. Sebastian
Quoted text · Christian Grün · 1 line
> Christian
JM
jean-marc Mercier1 Dec 2013 00:44 #
Hi Sebastian, On this problem, a quick comment coming into my mind, since I faced a comparable problem recently. Without changing XQUERY or BaseX, it is possible to emulate overloading using the introspection module of BaseX. But you need a map which keys and values are functions (not possible with BaseX map module, but possible with XQUERY). With such a map, keys are computed using the BaseX introspection module for your prod functions, since values are computed using the BaseX introspection module for your custom functions. All calls are then concentrated through a dispatcher : something like
call($function,$argument) { if contains($MapOfFunctions, $function) then
map:get($MapOfFunctions,$function)(argument) else $function($argument) }.
I tested this trick very recently actually, to overload the xquery:eval BaseX function. I named the resulting function evil_eval (a very dangerous function) ! Hope this help Cheers Jean-Marc 2013/11/30 Wiemer, Sebastian <Sebastian.Wiemer at adesso.de>
Quoted text · Wiemer, Sebastian · 98 lines
> Hi Christian, > > Hi Sebastian, > > thanks for your e-mail; I’m pleased to see you are still working with > BaseX. > > And we still love BaseX :-) > Thanks for keeping up the good work! > > BTW: I unfortunately missed your and Alexanders presentation at the > MarkUp Forum: are there any slides available? > > > > So just by dropping a file like „custom.xqm" in the restxq path, we can > customize the system. > > > …sounds like a nice idea. > > The drawback is that we can only use this for HTTPS requests from the > client, but not from the server, because we use HTTPS. So we cannot use > RESTXQ requests server side with http:send-request. (Or do we?) > > > I guess I have no answer to that, as I haven’t actually come across > this challenge yet. This might be completely off the track, but maybe > it’s also a question of how Jetty is configured? Did you already check > out the jetty.xml file and possible settings? > > > Yes, I checked jetty configuration (but I gave up), and it seems to me, > that jetty.xml or web.xml might not be the right place for a solution after > all. > > What Florian and I probably are trying to accomplish here is a way to > introduce aspect oriented or rule based > programming to XQuery to decouple custom extensions from product code. > > This is similar to imported modules in XSLT: If you import template > rules, which are more specific, > they will be applied first. Could this be applied to XQuery as well? (But > in XSLT you would have to > import the custom code and hence change your code. We can do that in > XQuery as well, but it > is not completely decoupled, is it?) > > Florian and I came around with an idea, that does not use HTTP requests > and maybe clarify > our goal: how about introducing a „dispatch" annotation to XQuery? > > Let’s assume we have a function „age“ that takes an age as integer and a > name as string and prints a message. > A customization would be to be more specific for teenagers. But we don’t > want to change the code. > the annotation „dispatch:declare“ declares a function to be potentially > dispatched: if there is not a more specific > function with a „dispatch:replace“ annotation, than that function is > called. > Maybe "dispatch:prepend" and "dispatch:append“ would be nice as well. > Also it would come in very handy if the replacing function is allowed be > updating or not updating. > > product.xqm > %dispatch:declare(age($age,$name)) > declare function prod:age($age as xs:integer,$name as xs:string){ > $name||" is "||$age||" years old." > }; > > prod:age(14,'Emily’) = „Emily is 14 years old." > > custom.xqm > %dispatch:replace(age(19>$age>13,$name)) > declare function custom:age($age as xs:integer,$name as xs:string){ > $name||" is a "||$age||“ years old teenager." > }; > > > So a use of „prod:age“ would yield the result of „custom:age“: > *prod*:age(14,'Emily’) = „Emily is a 14 years old *teenager*.“ > > > Comments on this idea are very appreciated. > > Sebastian > > > Christian > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >
WS
Wiemer, Sebastian2 Dec 2013 12:08 #
Quoted text · jean-marc Mercier · 1 line
> Hi Sebastian,
Hi Jean-Marc,
Quoted text · jean-marc Mercier · 2 lines
> On this problem, a quick comment coming into my mind, since I faced a comparable problem recently. Without changing XQUERY or BaseX, it is possible to emulate overloading using the introspection module of BaseX. But you need a map which keys and values > are functions (not possible with BaseX map module, but possible with XQUERY).
Thank you for you input. Using the inspection module helped. It’s not perfect, because the dispatcher still has to import custom code modules, but at least I don’t have to change my code everywhere I want to use the custom functions: —— — dispatch module module namespace dispatch="http://extensions/dispatch"; import module namespace custom="http://extensions/custom“; (: is it possible to read this via query:invoke? or something :)
declare function dispatch:fun($id as xs:string,$arg1){

    let $default
:= for $f in inspect:functions()
where exists(inspect:function($f)[annotation[@name='dispatch:default'][literal[1]=$id]]) return $f
    let $custom :=  for $f in inspect:functions()
where exists(inspect:function($f)[annotation[@name='dispatch:custom'][literal[1]=$id][literal[2]=$arg1]]) return $f return if (count($custom)=1) then $custom($arg1) else $default($arg1) }; ——— custom module module namespace custom="http://extensions/custom"; import module namespace dispatch="http://extensions/dispatch";
declare %dispatch:custom('someid','13')
function custom:canbenamedwhatever($t as item()) {'custom'}; — — — test run import module namespace dispatch="http://extensions/dispatch";
declare %dispatch:default('someid')
function local:NameAsYouLike($t as item()) { 'default' };
dispatch:fun('someid',12) (: yields „default“ :)
,dispatch:fun('someid',13) (: yields „custom“ :) This is not a generalized solution, but it meets my requirements (i.e. only one argument). Pro/cons + I wrote my first annotation that actually does something ;-) + the code is reasonably well decoupled - I have to call the dispatcher directly - no automagic no function overload - the dispatcher has to import the custom modules, it’t just one line, but still this is a dependency.
Quoted text · jean-marc Mercier · 1 line
> With such a map, keys are computed using the BaseX introspection module for your prod functions, since values are computed using the BaseX introspection module for your custom functions.
By „map of functions“ did you mean what "inspection:functions()“ is doing?
Quoted text · jean-marc Mercier · 13 lines
> All calls are then concentrated through a dispatcher : something like > > > > > > call($function,$argument) { if contains($MapOfFunctions, $function) then map:get($MapOfFunctions,$function)(argument) else $function($argument) }. > > > > > > I tested this trick very recently actually, to overload the xquery:eval BaseX function. I named the resulting function evil_eval (a very dangerous function) !
eval ist evil enough… ;-) I don’t get it: how would you be able to overload the query eval function?
Quoted text · jean-marc Mercier · 1 line
> Hope this help
Yes it did.
Quoted text · jean-marc Mercier · 7 lines
> Cheers > > > > > > Jean-Marc
Have fun, Sebastian
Quoted text · jean-marc Mercier · 213 lines
> 2013/11/30 Wiemer, Sebastian <Sebastian.Wiemer at adesso.de> > > > > Hi Christian, > > > > > > > > > > > > > Hi Sebastian, > > > > > > > > > > > > thanks for your e-mail; I’m pleased to see you are still working with BaseX. > > > > > > > > And we still love BaseX :-) > > > > Thanks for keeping up the good work! > > > > > > > > > > > > BTW: I unfortunately missed your and Alexanders presentation at the MarkUp Forum: are there any slides available? > > > > > > > > > > > > > > > > > > > > > > So just by dropping a file like „custom.xqm" in the restxq path, we can > > > > > > > > customize the system. > > > > > > > > > > > > …sounds like a nice idea. > > > > > > > > > > > > > > > > The drawback is that we can only use this for HTTPS requests from the > > > > > > > > client, but not from the server, because we use HTTPS. So we cannot use > > > > > > > > RESTXQ requests server side with http:send-request. (Or do we?) > > > > > > > > > > > > I guess I have no answer to that, as I haven’t actually come across > > > > > > this challenge yet. This might be completely off the track, but maybe > > > > > > it’s also a question of how Jetty is configured? Did you already check > > > > > > out the jetty.xml file and possible settings? > > > > > > > > > > > > > > > > Yes, I checked jetty configuration (but I gave up), and it seems to me, that jetty.xml or web.xml might not be the right place for a solution after all. > > > > > > > > > > > > What Florian and I probably are trying to accomplish here is a way to introduce aspect oriented or rule based > > > > programming to XQuery to decouple custom extensions from product code. > > > > > > > > > > > > This is similar to imported modules in XSLT: If you import template rules, which are more specific, > > > > they will be applied first. Could this be applied to XQuery as well? (But in XSLT you would have to > > > > import the custom code and hence change your code. We can do that in XQuery as well, but it > > > > is not completely decoupled, is it?) > > > > > > > > > > > > Florian and I came around with an idea, that does not use HTTP requests and maybe clarify > > > > our goal: how about introducing a „dispatch" annotation to XQuery? > > > > > > > > > > > > Let’s assume we have a function „age“ that takes an age as integer and a name as string and prints a message. > > > > A customization would be to be more specific for teenagers. But we don’t want to change the code. > > > > the annotation „dispatch:declare“ declares a function to be potentially dispatched: if there is not a more specific > > > > function with a „dispatch:replace“ annotation, than that function is called. > > > > Maybe "dispatch:prepend" and "dispatch:append“ would be nice as well. > > > > Also it would come in very handy if the replacing function is allowed be updating or not updating. > > > > > > > > > > > > product.xqm > > > > %dispatch:declare(age($age,$name)) > > > > > > declare function prod:age($age as xs:integer,$name as xs:string){ > > > > $name||" is "||$age||" years old." > > > > }; > > > > > > > > > > > > prod:age(14,'Emily’) = „Emily is 14 years old." > > > > > > > > > > > > custom.xqm > > > > > > %dispatch:replace(age(19>$age>13,$name)) > > > > declare function custom:age($age as xs:integer,$name as xs:string){ > > > > > > $name||" is a "||$age||“ years old teenager." > > > > }; > > > > > > > > > > > > > > > > > > > > So a use of „prod:age“ would yield the result of „custom:age“: > > > > > > prod:age(14,'Emily’) = „Emily is a 14 years old teenager.“ > > > > > > > > > > > > > > > > > > > > > > > > Comments on this idea are very appreciated. > > > > > > > > > > > > Sebastian > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Christian > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > BaseX-Talk mailing list > > > > BaseX-Talk at mailman.uni-konstanz.de > > > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
CG
Christian Grünin reply to Wiemer, Sebastian, 30 Nov 2013, 22:302 Dec 2013 12:12 #
Hi Sebastian,
Quoted text · Wiemer, Sebastian · 2 lines
> BTW: I unfortunately missed your and Alexanders presentation at the MarkUp > Forum: are there any slides available?
The slides should soon be available here: http://www.markupforum.de/archiv.html
Quoted text · Wiemer, Sebastian · 7 lines
> This is similar to imported modules in XSLT: If you import template rules, > which are more specific, > they will be applied first. Could this be applied to XQuery as well? (But in > XSLT you would have to > import the custom code and hence change your code. We can do that in XQuery > as well, but it > is not completely decoupled, is it?)
As Jean-Marc pointed out, the Inspection Module may help out. The module is fairly new, so we are interested in your (and everyone’s) feedback if you would like to have some functions added.. Christian
JM
jean-marc Mercier2 Dec 2013 13:11 #
Hi Sebastian, A quick comment for your code : it is better to use it sparsely, because inspect:functions is quite slow. If you want to improve perfs, create first a map. Here is the code that I am using for this kind of "overloading"
declare variable $common:MapIntrospecting := common:map_inspect_functions();

declare function common:name-function($fun as
function(*)){fn:data(inspect:function($fun)/@name)};

declare function common:map_inspect_functions() { map:new(for $fun in
inspect:functions() return map:entry(common:name-function($fun),$fun)) };
(this one has to be modified accordingly to your %dispatch annotation, i.e.
add where exists(inspect:function($f)[annotation[@name='dispatch:
default'][literal[1]=$id]]))
and the dispatcher becomes simply
declare function common:dispatcher(nodes as node()*,$fun as *function(*)*)
as item()* {
           let $disptchedfun := map:get($common:MapIntrospecting,
*common:name-function($fun)*) return
           if (empty($disptchedfun)) then $fun($nodes) else
$disptchedfun($nodes) )
};
Quoted text · Christian Grün · 1 line
>By „map of functions“ did you mean what "inspection:functions()“ is doing?
No, I am talking about a map that can take any item a keys, and any item as values. For instance, in the code above, I would prefer to write the following dispatcher
declare function common:dispatcher(nodes as node()*,$fun as *function(*)*)
as item()* {
           let $disptchedfun := map:get(common:MapIntrospecting(),$fun)
return
           if (empty($disptchedfun)) then $fun($nodes) else
$disptchedfun($nodes) )
};
Its is simply a question of performance : in the first mechanism, you have first to "serialize" the function as a string (calling the inspect:function), then to look to a map(string, function), to finally get the correct function call. In the second you directly access to the function, avoiding an expensive call (inspect:function). However, most probably this enhancement would be of less impact in your code, if you don't call this dispatcher trillions of time.
Quoted text · Christian Grün · 1 line
>I don’t get it: how would you be able to overload the query eval function?
exactly as above. It is something a little bit more elaborate than the following code
declare function common:evil_eval($nodes as node()*,$expr as xs:string) as
item()* {
try {

           let $fun := map:get($common:MapIntrospecting,$expr) return
           if (empty($fun)) then xquery:eval('$xml'||$expr,map{'$xml' :=
$nodes})
           else $fun($nodes)
}
catch *("ouch")
 };
Hope this helps
Cheers, Jean-Marc 2013/12/2 Christian Grün <christian.gruen at gmail.com>
Quoted text · Christian Grün · 30 lines
> Hi Sebastian, > > > BTW: I unfortunately missed your and Alexanders presentation at the > MarkUp > > Forum: are there any slides available? > > The slides should soon be available here: > http://www.markupforum.de/archiv.html > > > This is similar to imported modules in XSLT: If you import template > rules, > > which are more specific, > > they will be applied first. Could this be applied to XQuery as well? > (But in > > XSLT you would have to > > import the custom code and hence change your code. We can do that in > XQuery > > as well, but it > > is not completely decoupled, is it?) > > As Jean-Marc pointed out, the Inspection Module may help out. The > module is fairly new, so we are interested in your (and everyone’s) > feedback if you would like to have some functions added.. > > Christian > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

Written for BaseX 7.7. BaseX may have changed since: please check the current documentation.

« Reg : Group By ClauseNew user requiring help in population of database »
⚡Generated with XQuery