Send multipart binary with http client

> Hi, > > I can't figure out how to send a file in binary (not base64) when > using multipart [1] > > On the body, I've tried method="basex" because the docs says this > would write raw bytes to the request > > Regards, > Max > > > [1] > > declare function _:send-multipart($url, $file-name, $path) as item()* { > > > let $uri := file:path-to-uri($path || $file-name) > let $type := fetch:content-type($uri) > > let $binary := file:read-binary($path || $file-name) > let $boundary_id := concat("batch_", random:uuid()) > > let $req := > <http:request method="post" send-authorization="false" > override-media-type="text/plain"> > > <http:header name="Accept" value="application/json"/> > <http:header name="Accept-Encoding" value="gzip, deflate"/> > > <http:multipart media-type="multipart/form-data" > boundary="{$boundary_id}"> > > <http:header name="Content-Transfer-Encoding" value="binary"/> > <http:header name="Content-Disposition" value="form-data; > name='FILE'; filename='{$file-name}'"/> > <http:header name="Content-Type" value="{$type}"/> > > <!-- method="basex" method="adaptive" ? --> > <http:body media-type="application/octet-stream" method="binary"> > {$binary} > </http:body> > > </http:multipart> > </http:request> > > let $resp := http:send-request($req, $url) > return json:parse($resp[2]) > > };

Am 30.01.2017 4:24 nachm. schrieb "Maximilian Gärber" <mgaerber at arcor.de>:

> Hi, > > I can't figure out how to send a file in binary (not base64) when > using multipart [1] > > On the body, I've tried method="basex" because the docs says this > would write raw bytes to the request > > Regards, > Max > > > [1] > > declare function _:send-multipart($url, $file-name, $path) as item()* { > > > let $uri := file:path-to-uri($path || $file-name) > let $type := fetch:content-type($uri) > > let $binary := file:read-binary($path || $file-name) > let $boundary_id := concat("batch_", random:uuid()) > > let $req := > <http:request method="post" send-authorization="false" > override-media-type="text/plain"> > > <http:header name="Accept" value="application/json"/> > <http:header name="Accept-Encoding" value="gzip, deflate"/> > > <http:multipart media-type="multipart/form-data" > boundary="{$boundary_id}"> > > <http:header name="Content-Transfer-Encoding" value="binary"/> > <http:header name="Content-Disposition" value="form-data; > name='FILE'; filename='{$file-name}'"/> > <http:header name="Content-Type" value="{$type}"/> > > <!-- method="basex" method="adaptive" ? --> > <http:body media-type="application/octet-stream" method="binary"> > {$binary} > </http:body> > > </http:multipart> > </http:request> > > let $resp := http:send-request($req, $url) > return json:parse($resp[2]) > > };

> Hi Max, > > Do you have an MCVE for us? > > Thanks in advance, > Christian > > > Am 30.01.2017 4:24 nachm. schrieb "Maximilian Gärber" <mgaerber at arcor.de>: > > Hi, > > this seems related to https://github.com/BaseXdb/basex/issues/1203 > > Problem 1: When setting Content-Transfer-Encoding manually (via > <http:header>), it will appear twice in the request that is sent > > Problem 2: I cannot override base64 because it happens automatically > based on the content I am sending and for some reason the web server > that receives this request does not recognize the base64 encoding > correctly > > Regards, > > Max > > > > 2017-01-30 15:35 GMT+01:00 Maximilian Gärber <mgaerber at arcor.de>: >> Hi, >> >> I can't figure out how to send a file in binary (not base64) when >> using multipart [1] >> >> On the body, I've tried method="basex" because the docs says this >> would write raw bytes to the request >> >> Regards, >> Max >> >> >> [1] >> >> declare function _:send-multipart($url, $file-name, $path) as item()* { >> >> >> let $uri := file:path-to-uri($path || $file-name) >> let $type := fetch:content-type($uri) >> >> let $binary := file:read-binary($path || $file-name) >> let $boundary_id := concat("batch_", random:uuid()) >> >> let $req := >> <http:request method="post" send-authorization="false" >> override-media-type="text/plain"> >> >> <http:header name="Accept" value="application/json"/> >> <http:header name="Accept-Encoding" value="gzip, deflate"/> >> >> <http:multipart media-type="multipart/form-data" >> boundary="{$boundary_id}"> >> >> <http:header name="Content-Transfer-Encoding" value="binary"/> >> <http:header name="Content-Disposition" value="form-data; >> name='FILE'; filename='{$file-name}'"/> >> <http:header name="Content-Type" value="{$type}"/> >> >> <!-- method="basex" method="adaptive" ? --> >> <http:body media-type="application/octet-stream" method="binary"> >> {$binary} >> </http:body> >> >> </http:multipart> >> </http:request> >> >> let $resp := http:send-request($req, $url) >> return json:parse($resp[2]) >> >> }; > >

> Hi, > > the decision for base64 is now hardcoded: > https://github.com/BaseXdb/basex/commit/b48318abaf3a406d6e34253694c8e33e66287b32#diff-c7debdf57b8d834609533e844f1bf674R252 > > Browser for example just send files in raw binary, so most other > software (including jetty) does not know how to handle form-data with > base64 encoding. > > > > Regards, > Max > > > > 2017-01-30 19:16 GMT+01:00 Christian Grün <christian.gruen at gmail.com>: >> Hi Max, >> >> Do you have an MCVE for us? >> >> Thanks in advance, >> Christian >> >> >> Am 30.01.2017 4:24 nachm. schrieb "Maximilian Gärber" <mgaerber at arcor.de>: >> >> Hi, >> >> this seems related to https://github.com/BaseXdb/basex/issues/1203 >> >> Problem 1: When setting Content-Transfer-Encoding manually (via >> <http:header>), it will appear twice in the request that is sent >> >> Problem 2: I cannot override base64 because it happens automatically >> based on the content I am sending and for some reason the web server >> that receives this request does not recognize the base64 encoding >> correctly >> >> Regards, >> >> Max >> >> >> >> 2017-01-30 15:35 GMT+01:00 Maximilian Gärber <mgaerber at arcor.de>: >>> Hi, >>> >>> I can't figure out how to send a file in binary (not base64) when >>> using multipart [1] >>> >>> On the body, I've tried method="basex" because the docs says this >>> would write raw bytes to the request >>> >>> Regards, >>> Max >>> >>> >>> [1] >>> >>> declare function _:send-multipart($url, $file-name, $path) as item()* { >>> >>> >>> let $uri := file:path-to-uri($path || $file-name) >>> let $type := fetch:content-type($uri) >>> >>> let $binary := file:read-binary($path || $file-name) >>> let $boundary_id := concat("batch_", random:uuid()) >>> >>> let $req := >>> <http:request method="post" send-authorization="false" >>> override-media-type="text/plain"> >>> >>> <http:header name="Accept" value="application/json"/> >>> <http:header name="Accept-Encoding" value="gzip, deflate"/> >>> >>> <http:multipart media-type="multipart/form-data" >>> boundary="{$boundary_id}"> >>> >>> <http:header name="Content-Transfer-Encoding" value="binary"/> >>> <http:header name="Content-Disposition" value="form-data; >>> name='FILE'; filename='{$file-name}'"/> >>> <http:header name="Content-Type" value="{$type}"/> >>> >>> <!-- method="basex" method="adaptive" ? --> >>> <http:body media-type="application/octet-stream" method="binary"> >>> {$binary} >>> </http:body> >>> >>> </http:multipart> >>> </http:request> >>> >>> let $resp := http:send-request($req, $url) >>> return json:parse($resp[2]) >>> >>> }; >> >>

On Wed, Feb 1, 2017 at 2:57 PM, Maximilian Gärber <mgaerber at arcor.de> wrote: > Hi, > > the decision for base64 is now hardcoded: > https://github.com/BaseXdb/basex/commit/b48318abaf3a406d6e34253694c8e33e66287b32#diff-c7debdf57b8d834609533e844f1bf674R252 > > Browser for example just send files in raw binary, so most other > software (including jetty) does not know how to handle form-data with > base64 encoding. > > > > Regards, > Max > > > > 2017-01-30 19:16 GMT+01:00 Christian Grün <christian.gruen at gmail.com>: >> Hi Max, >> >> Do you have an MCVE for us? >> >> Thanks in advance, >> Christian >> >> >> Am 30.01.2017 4:24 nachm. schrieb "Maximilian Gärber" <mgaerber at arcor.de>: >> >> Hi, >> >> this seems related to https://github.com/BaseXdb/basex/issues/1203 >> >> Problem 1: When setting Content-Transfer-Encoding manually (via >> <http:header>), it will appear twice in the request that is sent >> >> Problem 2: I cannot override base64 because it happens automatically >> based on the content I am sending and for some reason the web server >> that receives this request does not recognize the base64 encoding >> correctly >> >> Regards, >> >> Max >> >> >> >> 2017-01-30 15:35 GMT+01:00 Maximilian Gärber <mgaerber at arcor.de>: >>> Hi, >>> >>> I can't figure out how to send a file in binary (not base64) when >>> using multipart [1] >>> >>> On the body, I've tried method="basex" because the docs says this >>> would write raw bytes to the request >>> >>> Regards, >>> Max >>> >>> >>> [1] >>> >>> declare function _:send-multipart($url, $file-name, $path) as item()* { >>> >>> >>> let $uri := file:path-to-uri($path || $file-name) >>> let $type := fetch:content-type($uri) >>> >>> let $binary := file:read-binary($path || $file-name) >>> let $boundary_id := concat("batch_", random:uuid()) >>> >>> let $req := >>> <http:request method="post" send-authorization="false" >>> override-media-type="text/plain"> >>> >>> <http:header name="Accept" value="application/json"/> >>> <http:header name="Accept-Encoding" value="gzip, deflate"/> >>> >>> <http:multipart media-type="multipart/form-data" >>> boundary="{$boundary_id}"> >>> >>> <http:header name="Content-Transfer-Encoding" value="binary"/> >>> <http:header name="Content-Disposition" value="form-data; >>> name='FILE'; filename='{$file-name}'"/> >>> <http:header name="Content-Type" value="{$type}"/> >>> >>> <!-- method="basex" method="adaptive" ? --> >>> <http:body media-type="application/octet-stream" method="binary"> >>> {$binary} >>> </http:body> >>> >>> </http:multipart> >>> </http:request> >>> >>> let $resp := http:send-request($req, $url) >>> return json:parse($resp[2]) >>> >>> }; >> >>

> Hi Max, > > Thanks for the hint. I’ll be glad to fix this, but I think I need an > example, or step-by-step description, how to reproduce this quickly. > Do you think that would be possible? > > Cheers from the other side of the lake, > Christian > > > > On Wed, Feb 1, 2017 at 2:57 PM, Maximilian Gärber <mgaerber at arcor.de> wrote: >> Hi, >> >> the decision for base64 is now hardcoded: >> https://github.com/BaseXdb/basex/commit/b48318abaf3a406d6e34253694c8e33e66287b32#diff-c7debdf57b8d834609533e844f1bf674R252 >> >> Browser for example just send files in raw binary, so most other >> software (including jetty) does not know how to handle form-data with >> base64 encoding. >> >> >> >> Regards, >> Max >> >> >> >> 2017-01-30 19:16 GMT+01:00 Christian Grün <christian.gruen at gmail.com>: >>> Hi Max, >>> >>> Do you have an MCVE for us? >>> >>> Thanks in advance, >>> Christian >>> >>> >>> Am 30.01.2017 4:24 nachm. schrieb "Maximilian Gärber" <mgaerber at arcor.de>: >>> >>> Hi, >>> >>> this seems related to https://github.com/BaseXdb/basex/issues/1203 >>> >>> Problem 1: When setting Content-Transfer-Encoding manually (via >>> <http:header>), it will appear twice in the request that is sent >>> >>> Problem 2: I cannot override base64 because it happens automatically >>> based on the content I am sending and for some reason the web server >>> that receives this request does not recognize the base64 encoding >>> correctly >>> >>> Regards, >>> >>> Max >>> >>> >>> >>> 2017-01-30 15:35 GMT+01:00 Maximilian Gärber <mgaerber at arcor.de>: >>>> Hi, >>>> >>>> I can't figure out how to send a file in binary (not base64) when >>>> using multipart [1] >>>> >>>> On the body, I've tried method="basex" because the docs says this >>>> would write raw bytes to the request >>>> >>>> Regards, >>>> Max >>>> >>>> >>>> [1] >>>> >>>> declare function _:send-multipart($url, $file-name, $path) as item()* { >>>> >>>> >>>> let $uri := file:path-to-uri($path || $file-name) >>>> let $type := fetch:content-type($uri) >>>> >>>> let $binary := file:read-binary($path || $file-name) >>>> let $boundary_id := concat("batch_", random:uuid()) >>>> >>>> let $req := >>>> <http:request method="post" send-authorization="false" >>>> override-media-type="text/plain"> >>>> >>>> <http:header name="Accept" value="application/json"/> >>>> <http:header name="Accept-Encoding" value="gzip, deflate"/> >>>> >>>> <http:multipart media-type="multipart/form-data" >>>> boundary="{$boundary_id}"> >>>> >>>> <http:header name="Content-Transfer-Encoding" value="binary"/> >>>> <http:header name="Content-Disposition" value="form-data; >>>> name='FILE'; filename='{$file-name}'"/> >>>> <http:header name="Content-Type" value="{$type}"/> >>>> >>>> <!-- method="basex" method="adaptive" ? --> >>>> <http:body media-type="application/octet-stream" method="binary"> >>>> {$binary} >>>> </http:body> >>>> >>>> </http:multipart> >>>> </http:request> >>>> >>>> let $resp := http:send-request($req, $url) >>>> return json:parse($resp[2]) >>>> >>>> }; >>> >>>

> On the body, I've tried method="basex" because the docs says this

On Wed, Feb 1, 2017 at 3:47 PM, Maximilian Gärber <mgaerber at arcor.de> wrote: > Hi, > > just try to send for example an image file (e.g. attached) with the > code below [1]. It will always be base64 (and when I uncomment the > "Content-Transfer-Encoding" header it will appear twice) > > You would need something like Fiddler > (https://www.telerik.com/download/fiddler) to intercept the request an > see what is going on. > > This is the shortest example I could create. > > > Best, Max > > > [1] > > let $name := "grass.jpg" > let $path := "upload/files/" > let $url := concat("http://localhost:4567", "/file/meta") > return _:send-multipart($url, $name, $path) > > > declare function _:send-multipart($url, $file-name, $path) as item()* { > > > let $uri := file:path-to-uri($path || $file-name) > let $type := fetch:content-type($uri) > let $binary := file:read-binary($path || $file-name) > > let $req := > <http:request method="post" send-authorization="false" > override-media-type="text/plain"> > > <http:header name="Accept" value="application/json"/> > <http:header name="Accept-Encoding" value="gzip, deflate"/> > > <http:multipart media-type="multipart/form-data"> > > <!-- <http:header name="Content-Transfer-Encoding" value="binary"/> --> > <http:header name="Content-Disposition" value='form-data; > name="FILE"; filename="{$file-name}"'/> > > <http:body media-type="{$type}"/> > > </http:multipart> > </http:request> > > let $resp := http:send-request($req, $url, $binary) > return json:parse($resp[2]) > }; > > 2017-02-01 15:01 GMT+01:00 Christian Grün <christian.gruen at gmail.com>: >> Hi Max, >> >> Thanks for the hint. I’ll be glad to fix this, but I think I need an >> example, or step-by-step description, how to reproduce this quickly. >> Do you think that would be possible? >> >> Cheers from the other side of the lake, >> Christian >> >> >> >> On Wed, Feb 1, 2017 at 2:57 PM, Maximilian Gärber <mgaerber at arcor.de> wrote: >>> Hi, >>> >>> the decision for base64 is now hardcoded: >>> https://github.com/BaseXdb/basex/commit/b48318abaf3a406d6e34253694c8e33e66287b32#diff-c7debdf57b8d834609533e844f1bf674R252 >>> >>> Browser for example just send files in raw binary, so most other >>> software (including jetty) does not know how to handle form-data with >>> base64 encoding. >>> >>> >>> >>> Regards, >>> Max >>> >>> >>> >>> 2017-01-30 19:16 GMT+01:00 Christian Grün <christian.gruen at gmail.com>: >>>> Hi Max, >>>> >>>> Do you have an MCVE for us? >>>> >>>> Thanks in advance, >>>> Christian >>>> >>>> >>>> Am 30.01.2017 4:24 nachm. schrieb "Maximilian Gärber" <mgaerber at arcor.de>: >>>> >>>> Hi, >>>> >>>> this seems related to https://github.com/BaseXdb/basex/issues/1203 >>>> >>>> Problem 1: When setting Content-Transfer-Encoding manually (via >>>> <http:header>), it will appear twice in the request that is sent >>>> >>>> Problem 2: I cannot override base64 because it happens automatically >>>> based on the content I am sending and for some reason the web server >>>> that receives this request does not recognize the base64 encoding >>>> correctly >>>> >>>> Regards, >>>> >>>> Max >>>> >>>> >>>> >>>> 2017-01-30 15:35 GMT+01:00 Maximilian Gärber <mgaerber at arcor.de>: >>>>> Hi, >>>>> >>>>> I can't figure out how to send a file in binary (not base64) when >>>>> using multipart [1] >>>>> >>>>> On the body, I've tried method="basex" because the docs says this >>>>> would write raw bytes to the request >>>>> >>>>> Regards, >>>>> Max >>>>> >>>>> >>>>> [1] >>>>> >>>>> declare function _:send-multipart($url, $file-name, $path) as item()* { >>>>> >>>>> >>>>> let $uri := file:path-to-uri($path || $file-name) >>>>> let $type := fetch:content-type($uri) >>>>> >>>>> let $binary := file:read-binary($path || $file-name) >>>>> let $boundary_id := concat("batch_", random:uuid()) >>>>> >>>>> let $req := >>>>> <http:request method="post" send-authorization="false" >>>>> override-media-type="text/plain"> >>>>> >>>>> <http:header name="Accept" value="application/json"/> >>>>> <http:header name="Accept-Encoding" value="gzip, deflate"/> >>>>> >>>>> <http:multipart media-type="multipart/form-data" >>>>> boundary="{$boundary_id}"> >>>>> >>>>> <http:header name="Content-Transfer-Encoding" value="binary"/> >>>>> <http:header name="Content-Disposition" value="form-data; >>>>> name='FILE'; filename='{$file-name}'"/> >>>>> <http:header name="Content-Type" value="{$type}"/> >>>>> >>>>> <!-- method="basex" method="adaptive" ? --> >>>>> <http:body media-type="application/octet-stream" method="binary"> >>>>> {$binary} >>>>> </http:body> >>>>> >>>>> </http:multipart> >>>>> </http:request> >>>>> >>>>> let $resp := http:send-request($req, $url) >>>>> return json:parse($resp[2]) >>>>> >>>>> }; >>>> >>>>