http:send-request POST xml body is inheriting the http-client namespace

On Wed, Sep 14, 2016 at 1:35 PM, chrisis <chrisisanon at gmail.com> wrote: > I'm using BaseX 8.4 but have recreated the issue with 8.5.4 beta 8d90b66 > > I'm POSTing xml files to a RESTXQ function using the following script > > let $fpath := 'C:\incoming\SAQD\mo' > let $targetBasex := 'http://localhost:8984/saqd/mo/' > let $list := sort(file:list($fpath), function($fname) > {fn:analyze-string($fname,'.*_(.*?)\.xml$')/fn:match/fn:group[@nr="1"]/string()}) > for $file in $list > let $abspath := $fpath||'\'||$file > let $body := try {fetch:xml($abspath, map { 'chop': true() }) } > catch * {()} > > return if ( not(empty($body)) ) then (http:send-request(<http:request > href="{$targetBasex||$file}" > method='POST'> > <http:body media-type="text/xml">{$body}</http:body> > </http:request>), file:move($abspath,$fpath||'\processed\'||$file) ) > else () > > The XML file has no namespace declarations. > The RESTXQ function does a db:update using the body received and the > http-client namespace has been added to the root node. > > If I POST to the same function using curl the stored document is unchanged. > The function is as follows. > > declare > %rest:path("/saqd/mo/{$fname}") > (: %rest:consumes("application/xml", "text/xml") :) > %rest:POST("{$body}") > %updating > function saqd:mo($fname, $body as document-node()) > { > try {(db:replace('SAQD', '/MO/'||$fname, $body ), > db:output(http:send-request(<http:request > href="{'http://example.org:8082/home/VALIDSCHEMA/xml/current/Metoffice_Data/'||$fname}" > method='PUT' username='validuser' password='validpassword' > send-authorization='true'> > <http:body media-type='text/xml'>{$body}</http:body> > </http:request>)) > ) > } > catch * {db:output( > <rest:response> > <http:response status="500" message="{$err:code||' : '||$err:description}"> > </http:response> > </rest:response>)} > }; > > > The file PUT inside the db:output statement also gets the namespace added > i.e. it happens even when the file was POSTed with curl so the recieved > document still had no namespaces. > > Is this expected behaviour, a bug, or am I using http:send-request > incorrectly? > > Regards, > > chris sweeney >

On Thu, Sep 15, 2016 at 2:34 PM, chrisis <chrisisanon at gmail.com> wrote: > Hi Christian > > Sorry, chopped that out of the real thing in a bit of a hurry and made a bit > of a mess of it. > Here's something slightly closer to a SSCE. > > > Module under WEBPATH > > xquery version "3.1" encoding "utf-8"; > module namespace example = 'http://localhost/example'; > declare > %rest:path("/example/post/{$fname}") > %rest:POST("{$body}") > %updating > function example:post($fname, $body as document-node()) > {db:replace('exampledb', '/post/'||$fname, $body) }; > > > Script run in BaseX GUI > > let $file := 'C:\temp\xq\example.xml' > let $fname := 'example.xml' > let $targetrestxqproc := 'http://localhost:8984/example/post/' > let $body := fetch:xml($file, map { 'chop': true() }) > return http:send-request(<http:request href="{$targetrestxqproc||$fname}" > method='POST'> > <http:body media-type='text/xml'>{$body}</http:body> > </http:request>) > > Result in exampledb is > <example xmlns:http="http://expath.org/ns/http-client"/>

On Fri, Sep 16, 2016 at 4:06 PM, Christian Grün <christian.gruen at gmail.com> wrote: > Hm. I can’t reproduce it.. > > 1. I stored your module as 'example.xqm' in the WEBPATH directory > 2. I started basexhttp > 3. I created an empty 'exampledb' database in the GUI and closed it again > 4. I ran your script in the GUI (after rewriting fetch:xml(...) with <example/>) > 5. I ran db:open('exambledb') > > Did I do something wrong? > > By the way, if possible, remove everything from your examples that > required manual post-processing. Could you please check the following > code and tell me how the returned XML looks like? > > _ example.xqm ___________ > > module namespace example = 'http://localhost/example'; > declare > %rest:path("/example") > %rest:POST("{$body}") > function example:post($body) { > $body > }; > > _ test.xqm ___________ > > http:send-request( > <http:request href="http://localhost:8984/example" method='POST'> > <http:body media-type='text/xml'> > <example/> > </http:body> > </http:request> > )[2] > > On my system, I get "<example/>". > > > > On Thu, Sep 15, 2016 at 2:34 PM, chrisis <chrisisanon at gmail.com> wrote: >> Hi Christian >> >> Sorry, chopped that out of the real thing in a bit of a hurry and made a bit >> of a mess of it. >> Here's something slightly closer to a SSCE. >> >> >> Module under WEBPATH >> >> xquery version "3.1" encoding "utf-8"; >> module namespace example = 'http://localhost/example'; >> declare >> %rest:path("/example/post/{$fname}") >> %rest:POST("{$body}") >> %updating >> function example:post($fname, $body as document-node()) >> {db:replace('exampledb', '/post/'||$fname, $body) }; >> >> >> Script run in BaseX GUI >> >> let $file := 'C:\temp\xq\example.xml' >> let $fname := 'example.xml' >> let $targetrestxqproc := 'http://localhost:8984/example/post/' >> let $body := fetch:xml($file, map { 'chop': true() }) >> return http:send-request(<http:request href="{$targetrestxqproc||$fname}" >> method='POST'> >> <http:body media-type='text/xml'>{$body}</http:body> >> </http:request>) >> >> Result in exampledb is >> <example xmlns:http="http://expath.org/ns/http-client"/>

> On 16 Sep 2016, at 17:15, Christian Grün <christian.gruen at gmail.com> wrote: > > Thanks for the hints. I can finally agree it’s a bug ;) I have added a > GitHub issue for it [1]. The problem only occurs with "database nodes" > (which are the ones created via "fn:doc", "fetch:xml", or by applying > transformations via "copy $c := ..." or "... update {}". > > This looks to me like a generall weakness of the EXPath spec in > general: A namespace declaration will always be valid for all > descendant nodes of an element. As it doesn’t make sense to keep the > http namespace when supplying bodies via http:send-request, we are > explicitly removing the namespace from the body node before sending it > out. As there is no such functionality in XQuery like “removing > namespaces”, this operation is not well-defined, and it’s probably the > reason for this bug. > > I hope I’ll find time to fix this soon. > > Cheers, > Christian > > [1] https://github.com/BaseXdb/basex/issues/1353 > > > On Fri, Sep 16, 2016 at 4:06 PM, Christian Grün > <christian.gruen at gmail.com> wrote: >> Hm. I can’t reproduce it.. >> >> 1. I stored your module as 'example.xqm' in the WEBPATH directory >> 2. I started basexhttp >> 3. I created an empty 'exampledb' database in the GUI and closed it again >> 4. I ran your script in the GUI (after rewriting fetch:xml(...) with <example/>) >> 5. I ran db:open('exambledb') >> >> Did I do something wrong? >> >> By the way, if possible, remove everything from your examples that >> required manual post-processing. Could you please check the following >> code and tell me how the returned XML looks like? >> >> _ example.xqm ___________ >> >> module namespace example = 'http://localhost/example'; >> declare >> %rest:path("/example") >> %rest:POST("{$body}") >> function example:post($body) { >> $body >> }; >> >> _ test.xqm ___________ >> >> http:send-request( >> <http:request href="http://localhost:8984/example" method='POST'> >> <http:body media-type='text/xml'> >> <example/> >> </http:body> >> </http:request> >> )[2] >> >> On my system, I get "<example/>". >> >> >> >> On Thu, Sep 15, 2016 at 2:34 PM, chrisis <chrisisanon at gmail.com> wrote: >>> Hi Christian >>> >>> Sorry, chopped that out of the real thing in a bit of a hurry and made a bit >>> of a mess of it. >>> Here's something slightly closer to a SSCE. >>> >>> >>> Module under WEBPATH >>> >>> xquery version "3.1" encoding "utf-8"; >>> module namespace example = 'http://localhost/example'; >>> declare >>> %rest:path("/example/post/{$fname}") >>> %rest:POST("{$body}") >>> %updating >>> function example:post($fname, $body as document-node()) >>> {db:replace('exampledb', '/post/'||$fname, $body) }; >>> >>> >>> Script run in BaseX GUI >>> >>> let $file := 'C:\temp\xq\example.xml' >>> let $fname := 'example.xml' >>> let $targetrestxqproc := 'http://localhost:8984/example/post/' >>> let $body := fetch:xml($file, map { 'chop': true() }) >>> return http:send-request(<http:request href="{$targetrestxqproc||$fname}" >>> method='POST'> >>> <http:body media-type='text/xml'>{$body}</http:body> >>> </http:request>) >>> >>> Result in exampledb is >>> <example xmlns:http="http://expath.org/ns/http-client"/>

On Fri, Sep 16, 2016 at 6:28 PM, chris sweeney <chrisisanon at gmail.com> wrote: > Thanks for the explanation. > I’ll keep an eye out for the fix but I can easily work round this issue for now. > > cheers, > > chris > >> On 16 Sep 2016, at 17:15, Christian Grün <christian.gruen at gmail.com> wrote: >> >> Thanks for the hints. I can finally agree it’s a bug ;) I have added a >> GitHub issue for it [1]. The problem only occurs with "database nodes" >> (which are the ones created via "fn:doc", "fetch:xml", or by applying >> transformations via "copy $c := ..." or "... update {}". >> >> This looks to me like a generall weakness of the EXPath spec in >> general: A namespace declaration will always be valid for all >> descendant nodes of an element. As it doesn’t make sense to keep the >> http namespace when supplying bodies via http:send-request, we are >> explicitly removing the namespace from the body node before sending it >> out. As there is no such functionality in XQuery like “removing >> namespaces”, this operation is not well-defined, and it’s probably the >> reason for this bug. >> >> I hope I’ll find time to fix this soon. >> >> Cheers, >> Christian >> >> [1] https://github.com/BaseXdb/basex/issues/1353 >> >> >> On Fri, Sep 16, 2016 at 4:06 PM, Christian Grün >> <christian.gruen at gmail.com> wrote: >>> Hm. I can’t reproduce it.. >>> >>> 1. I stored your module as 'example.xqm' in the WEBPATH directory >>> 2. I started basexhttp >>> 3. I created an empty 'exampledb' database in the GUI and closed it again >>> 4. I ran your script in the GUI (after rewriting fetch:xml(...) with <example/>) >>> 5. I ran db:open('exambledb') >>> >>> Did I do something wrong? >>> >>> By the way, if possible, remove everything from your examples that >>> required manual post-processing. Could you please check the following >>> code and tell me how the returned XML looks like? >>> >>> _ example.xqm ___________ >>> >>> module namespace example = 'http://localhost/example'; >>> declare >>> %rest:path("/example") >>> %rest:POST("{$body}") >>> function example:post($body) { >>> $body >>> }; >>> >>> _ test.xqm ___________ >>> >>> http:send-request( >>> <http:request href="http://localhost:8984/example" method='POST'> >>> <http:body media-type='text/xml'> >>> <example/> >>> </http:body> >>> </http:request> >>> )[2] >>> >>> On my system, I get "<example/>". >>> >>> >>> >>> On Thu, Sep 15, 2016 at 2:34 PM, chrisis <chrisisanon at gmail.com> wrote: >>>> Hi Christian >>>> >>>> Sorry, chopped that out of the real thing in a bit of a hurry and made a bit >>>> of a mess of it. >>>> Here's something slightly closer to a SSCE. >>>> >>>> >>>> Module under WEBPATH >>>> >>>> xquery version "3.1" encoding "utf-8"; >>>> module namespace example = 'http://localhost/example'; >>>> declare >>>> %rest:path("/example/post/{$fname}") >>>> %rest:POST("{$body}") >>>> %updating >>>> function example:post($fname, $body as document-node()) >>>> {db:replace('exampledb', '/post/'||$fname, $body) }; >>>> >>>> >>>> Script run in BaseX GUI >>>> >>>> let $file := 'C:\temp\xq\example.xml' >>>> let $fname := 'example.xml' >>>> let $targetrestxqproc := 'http://localhost:8984/example/post/' >>>> let $body := fetch:xml($file, map { 'chop': true() }) >>>> return http:send-request(<http:request href="{$targetrestxqproc||$fname}" >>>> method='POST'> >>>> <http:body media-type='text/xml'>{$body}</http:body> >>>> </http:request>) >>>> >>>> Result in exampledb is >>>> <example xmlns:http="http://expath.org/ns/http-client"/> >