http:send-request - problem with form data

On Mon, Jul 2, 2018 at 11:11 PM Bogdan Bogucki <bbogucki at hussar.pl> wrote: > > Hello, > > > > I have problem with http:send-request. I want to send post method request with form data but I encountered problem with characters encoding. > > Example: > > > > Request: > > for $x in http:send-request( > > <http:request method='post' href= > > 'http://localhost:3333'> > > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"/> > > <http:body media-type='application/x-www-form-urlencoded; charset=UTF-8'> > > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> > > </http:body> > > </http:request> > > )[2]/json/data/results/content > > return $x > > > > Optimized Query: > > util:item-at(http:send-request(element Q{http://expath.org/ns/http-client}request { (attribute method { ("post") }, attribute href { ("http://localhost:3333") }, element Q{http://expath.org/ns/http-client}header { (attribute name { ("User-Agent") }, attribute value { ("Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36") }) }, element Q{http://expath.org/ns/http-client}body { (attribute media-type { ("application/x-www-form-urlencoded; charset=UTF-8") }, "&#xA; start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100&#xA; ") }) }), 2)/json/data/results/content > > > > Request send: > > POST / HTTP/1.1 > > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36 > > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 > > Host: copart.com:3333 > > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 > > Connection: keep-alive > > Content-Length: 75 > > > > > > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 > > > > Payload is not correctly encoded it should be: > > start=0&length=100&sort=date_type&page=0&size=100 > > > > How can I force function http:send-request to serialize CDATA without escaping ? > > > > > > Best Regards > > Bogdan Bogucki > >

On Mon, Jul 2, 2018 at 11:11 PM Bogdan Bogucki <bbogucki at hussar.pl> wrote: > > Hello, > > > > I have problem with http:send-request. I want to send post method request with form data but I encountered problem with characters encoding. > > Example: > > > > Request: > > for $x in http:send-request( > > <http:request method='post' href= > > 'http://localhost:3333'> > > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; > Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > Safari/537.36"/> > > <http:body media-type='application/x-www-form-urlencoded; > charset=UTF-8'> > > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> > > </http:body> > > </http:request> > > )[2]/json/data/results/content > > return $x > > > > Optimized Query: > > util:item-at(http:send-request(element Q{http://expath.org/ns/http-client}request { (attribute method { ("post") }, attribute href { ("http://localhost:3333") }, element Q{http://expath.org/ns/http-client}header { (attribute name { ("User-Agent") }, attribute value { ("Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36") }) }, element Q{http://expath.org/ns/http-client}body { (attribute media-type { ("application/x-www-form-urlencoded; charset=UTF-8") }, "&#xA; start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100&#xA; ") }) }), 2)/json/data/results/content > > > > Request send: > > POST / HTTP/1.1 > > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > Safari/537.36 > > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 > > Host: copart.com:3333 > > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 > > Connection: keep-alive > > Content-Length: 75 > > > > > > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 > > > > Payload is not correctly encoded it should be: > > start=0&length=100&sort=date_type&page=0&size=100 > > > > How can I force function http:send-request to serialize CDATA without escaping ? > > > > > > Best Regards > > Bogdan Bogucki > >

On Tue, Jul 3, 2018 at 10:14 PM Bogdan Bogucki <bbogucki at hussar.pl> wrote: > > Hello, > > I don't want pass values in query string. Server side expect parameters in payload. > HTML Form parameters in POST method are passed in payload not in URI. > This is the reason why I put parameters in body. Problem is escaping & character with &amp; > How can I force BaseX to not escape characters in CDATA element ? > > This is my code: > <http:request method='post' href='http://localhost:3333'> > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"/> > <http:body media-type='application/x-www-form-urlencoded; charset=UTF-8'> > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> > </http:body> > </http:request> > > On server side I received following request: > > POST / HTTP/1.1 > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36 > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 > > Host: copart.com:3333 > > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 > > Connection: keep-alive > Content-Length: 75 > > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 > > I expect that CDATA will be not escaped, or I am wrong ? > > > Pozdrawiam > Bogdan Bogucki > > HUSSAR SYSTEMS > > NIP: 532-170-12-00 > Regon: 142624070 > > e-mail: bbogucki at hussar.pl > tel. kom.: +48 607 409 301 > www: www.hussar.pl > > -----Wiadomość oryginalna----- > Od: Christian Grün [mailto:christian.gruen at gmail.com] > Wysłano: 3 lipca 2018 12:11 > Do: bbogucki at hussar.pl > DW: BaseX > Temat: Re: [basex-talk] http:send-request - problem with form data > > Hi Bogdan, > > If you want your arguments parsed as HTTP query parameters, it should suffice to attach them to the URL: > > let $url := 'http://localhost:3333' > let $params := ``[start=0&length=100&sort=date_type&page=0&size=100]`` > return http:send-request( > <http:request method='post' href='{ $url || '?' || $params }'> ... > > If this is not what you need, you might need to give us more information on how the request will be handled server-side (for example, do you use RESTXQ?). > > Best, > Christian > > > > > On Mon, Jul 2, 2018 at 11:11 PM Bogdan Bogucki <bbogucki at hussar.pl> wrote: > > > > Hello, > > > > > > > > I have problem with http:send-request. I want to send post method request with form data but I encountered problem with characters encoding. > > > > Example: > > > > > > > > Request: > > > > for $x in http:send-request( > > > > <http:request method='post' href= > > > > 'http://localhost:3333'> > > > > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; > > Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > > Safari/537.36"/> > > > > <http:body media-type='application/x-www-form-urlencoded; > > charset=UTF-8'> > > > > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> > > > > </http:body> > > > > </http:request> > > > > )[2]/json/data/results/content > > > > return $x > > > > > > > > Optimized Query: > > > > util:item-at(http:send-request(element Q{http://expath.org/ns/http-client}request { (attribute method { ("post") }, attribute href { ("http://localhost:3333") }, element Q{http://expath.org/ns/http-client}header { (attribute name { ("User-Agent") }, attribute value { ("Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36") }) }, element Q{http://expath.org/ns/http-client}body { (attribute media-type { ("application/x-www-form-urlencoded; charset=UTF-8") }, "&#xA; start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100&#xA; ") }) }), 2)/json/data/results/content > > > > > > > > Request send: > > > > POST / HTTP/1.1 > > > > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) > > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > > Safari/537.36 > > > > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 > > > > Host: copart.com:3333 > > > > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 > > > > Connection: keep-alive > > > > Content-Length: 75 > > > > > > > > > > > > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 > > > > > > > > Payload is not correctly encoded it should be: > > > > start=0&length=100&sort=date_type&page=0&size=100 > > > > > > > > How can I force function http:send-request to serialize CDATA without escaping ? > > > > > > > > > > > > Best Regards > > > > Bogdan Bogucki > > > > >

On Tue, Jul 3, 2018 at 10:14 PM Bogdan Bogucki <bbogucki at hussar.pl> wrote: > > Hello, > > I don't want pass values in query string. Server side expect parameters in payload. > HTML Form parameters in POST method are passed in payload not in URI. > This is the reason why I put parameters in body. Problem is escaping & > character with &amp; How can I force BaseX to not escape characters in CDATA element ? > > This is my code: > <http:request method='post' href='http://localhost:3333'> > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"/> > <http:body media-type='application/x-www-form-urlencoded; charset=UTF-8'> > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> > </http:body> > </http:request> > > On server side I received following request: > > POST / HTTP/1.1 > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > Safari/537.36 > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 > > Host: copart.com:3333 > > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 > > Connection: keep-alive > Content-Length: 75 > > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 > > I expect that CDATA will be not escaped, or I am wrong ? > > > Pozdrawiam > Bogdan Bogucki > > HUSSAR SYSTEMS > > NIP: 532-170-12-00 > Regon: 142624070 > > e-mail: bbogucki at hussar.pl > tel. kom.: +48 607 409 301 > www: www.hussar.pl > > -----Wiadomość oryginalna----- > Od: Christian Grün [mailto:christian.gruen at gmail.com] > Wysłano: 3 lipca 2018 12:11 > Do: bbogucki at hussar.pl > DW: BaseX > Temat: Re: [basex-talk] http:send-request - problem with form data > > Hi Bogdan, > > If you want your arguments parsed as HTTP query parameters, it should suffice to attach them to the URL: > > let $url := 'http://localhost:3333' > let $params := ``[start=0&length=100&sort=date_type&page=0&size=100]`` > return http:send-request( > <http:request method='post' href='{ $url || '?' || $params }'> ... > > If this is not what you need, you might need to give us more information on how the request will be handled server-side (for example, do you use RESTXQ?). > > Best, > Christian > > > > > On Mon, Jul 2, 2018 at 11:11 PM Bogdan Bogucki <bbogucki at hussar.pl> wrote: > > > > Hello, > > > > > > > > I have problem with http:send-request. I want to send post method request with form data but I encountered problem with characters encoding. > > > > Example: > > > > > > > > Request: > > > > for $x in http:send-request( > > > > <http:request method='post' href= > > > > 'http://localhost:3333'> > > > > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT > > 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) > > Chrome/67.0.3396.99 Safari/537.36"/> > > > > <http:body media-type='application/x-www-form-urlencoded; > > charset=UTF-8'> > > > > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> > > > > </http:body> > > > > </http:request> > > > > )[2]/json/data/results/content > > > > return $x > > > > > > > > Optimized Query: > > > > util:item-at(http:send-request(element Q{http://expath.org/ns/http-client}request { (attribute method { ("post") }, attribute href { ("http://localhost:3333") }, element Q{http://expath.org/ns/http-client}header { (attribute name { ("User-Agent") }, attribute value { ("Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36") }) }, element Q{http://expath.org/ns/http-client}body { (attribute media-type { ("application/x-www-form-urlencoded; charset=UTF-8") }, "&#xA; start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100&#xA; ") }) }), 2)/json/data/results/content > > > > > > > > Request send: > > > > POST / HTTP/1.1 > > > > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) > > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > > Safari/537.36 > > > > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 > > > > Host: copart.com:3333 > > > > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 > > > > Connection: keep-alive > > > > Content-Length: 75 > > > > > > > > > > > > > > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 > > > > > > > > Payload is not correctly encoded it should be: > > > > start=0&length=100&sort=date_type&page=0&size=100 > > > > > > > > How can I force function http:send-request to serialize CDATA without escaping ? > > > > > > > > > > > > Best Regards > > > > Bogdan Bogucki > > > > >

Bogdan Bogucki <bbogucki at hussar.pl> schrieb am Mi., 4. Juli 2018, 21:56: > Hello, > > It works with body as a third parameter. Body has to be defined in http > request because without that third parameter is ignored. > > Working example: > > for $x in http:send-request( > <http:request method='post'> > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; > Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > Safari/537.36"/> > <http:body media-type='application/x-www-form-urlencoded; > charset=UTF-8'/> > </http:request> > ,'http:localhost:3333', 'start=0&amp;length=100&amp; > ')[2]/data/results/content > return $x > > Thank you Christian for help. > > Best Regards > Bogdan Bogucki > > > -----Wiadomość oryginalna----- > Od: Christian Grün [mailto:christian.gruen at gmail.com] > Wysłano: 4 lipca 2018 18:31 > Do: bbogucki at hussar.pl > DW: BaseX > Temat: Re: [basex-talk] http:send-request - problem with form data > > Hi Bogdan, > > I didn’t try it, but what happens if you specify the body as third > argument? > > Cheers, > Christian > > > On Tue, Jul 3, 2018 at 10:14 PM Bogdan Bogucki <bbogucki at hussar.pl> wrote: > > > > Hello, > > > > I don't want pass values in query string. Server side expect parameters > in payload. > > HTML Form parameters in POST method are passed in payload not in URI. > > This is the reason why I put parameters in body. Problem is escaping & > > character with &amp; How can I force BaseX to not escape characters in > CDATA element ? > > > > This is my code: > > <http:request method='post' href='http://localhost:3333'> > > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; > Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > Safari/537.36"/> > > <http:body media-type='application/x-www-form-urlencoded; > charset=UTF-8'> > > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> > > </http:body> > > </http:request> > > > > On server side I received following request: > > > > POST / HTTP/1.1 > > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) > > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > > Safari/537.36 > > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 > > > > Host: copart.com:3333 > > > > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 > > > > Connection: keep-alive > > Content-Length: 75 > > > > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 > > > > I expect that CDATA will be not escaped, or I am wrong ? > > > > > > Pozdrawiam > > Bogdan Bogucki > > > > HUSSAR SYSTEMS > > > > NIP: 532-170-12-00 > > Regon: 142624070 > > > > e-mail: bbogucki at hussar.pl > > tel. kom.: +48 607 409 301 > > www: www.hussar.pl > > > > -----Wiadomość oryginalna----- > > Od: Christian Grün [mailto:christian.gruen at gmail.com] > > Wysłano: 3 lipca 2018 12:11 > > Do: bbogucki at hussar.pl > > DW: BaseX > > Temat: Re: [basex-talk] http:send-request - problem with form data > > > > Hi Bogdan, > > > > If you want your arguments parsed as HTTP query parameters, it should > suffice to attach them to the URL: > > > > let $url := 'http://localhost:3333' > > let $params := ``[start=0&length=100&sort=date_type&page=0&size=100]`` > > return http:send-request( > > <http:request method='post' href='{ $url || '?' || $params }'> ... > > > > If this is not what you need, you might need to give us more information > on how the request will be handled server-side (for example, do you use > RESTXQ?). > > > > Best, > > Christian > > > > > > > > > > On Mon, Jul 2, 2018 at 11:11 PM Bogdan Bogucki <bbogucki at hussar.pl> > wrote: > > > > > > Hello, > > > > > > > > > > > > I have problem with http:send-request. I want to send post method > request with form data but I encountered problem with characters encoding. > > > > > > Example: > > > > > > > > > > > > Request: > > > > > > for $x in http:send-request( > > > > > > <http:request method='post' href= > > > > > > 'http://localhost:3333'> > > > > > > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT > > > 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) > > > Chrome/67.0.3396.99 Safari/537.36"/> > > > > > > <http:body media-type='application/x-www-form-urlencoded; > > > charset=UTF-8'> > > > > > > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> > > > > > > </http:body> > > > > > > </http:request> > > > > > > )[2]/json/data/results/content > > > > > > return $x > > > > > > > > > > > > Optimized Query: > > > > > > util:item-at(http:send-request(element Q{ > http://expath.org/ns/http-client}request { (attribute method { ("post") > }, attribute href { ("http://localhost:3333") }, element Q{ > http://expath.org/ns/http-client}header { (attribute name { > ("User-Agent") }, attribute value { ("Mozilla/5.0 (Windows NT 6.1; Win64; > x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > Safari/537.36") }) }, element Q{http://expath.org/ns/http-client}body { > (attribute media-type { ("application/x-www-form-urlencoded; > charset=UTF-8") }, "&#xA; > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100&#xA; > ") }) }), 2)/json/data/results/content > > > > > > > > > > > > Request send: > > > > > > POST / HTTP/1.1 > > > > > > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) > > > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > > > Safari/537.36 > > > > > > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 > > > > > > Host: copart.com:3333 > > > > > > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 > > > > > > Connection: keep-alive > > > > > > Content-Length: 75 > > > > > > > > > > > > > > > > > > > > > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 > > > > > > > > > > > > Payload is not correctly encoded it should be: > > > > > > start=0&length=100&sort=date_type&page=0&size=100 > > > > > > > > > > > > How can I force function http:send-request to serialize CDATA without > escaping ? > > > > > > > > > > > > > > > > > > Best Regards > > > > > > Bogdan Bogucki > > > > > > > > > >

On Wed, Jul 4, 2018 at 10:05 PM Christian Grün <christian.gruen at gmail.com> wrote: > > And thanks for the confirmation. I will try to find out why the two alternatives make a difference. > > > > Bogdan Bogucki <bbogucki at hussar.pl> schrieb am Mi., 4. Juli 2018, 21:56: >> >> Hello, >> >> It works with body as a third parameter. Body has to be defined in http request because without that third parameter is ignored. >> >> Working example: >> >> for $x in http:send-request( >> <http:request method='post'> >> <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"/> >> <http:body media-type='application/x-www-form-urlencoded; charset=UTF-8'/> >> </http:request> >> ,'http:localhost:3333', 'start=0&amp;length=100&amp; ')[2]/data/results/content >> return $x >> >> Thank you Christian for help. >> >> Best Regards >> Bogdan Bogucki >> >> >> -----Wiadomość oryginalna----- >> Od: Christian Grün [mailto:christian.gruen at gmail.com] >> Wysłano: 4 lipca 2018 18:31 >> Do: bbogucki at hussar.pl >> DW: BaseX >> Temat: Re: [basex-talk] http:send-request - problem with form data >> >> Hi Bogdan, >> >> I didn’t try it, but what happens if you specify the body as third argument? >> >> Cheers, >> Christian >> >> >> On Tue, Jul 3, 2018 at 10:14 PM Bogdan Bogucki <bbogucki at hussar.pl> wrote: >> > >> > Hello, >> > >> > I don't want pass values in query string. Server side expect parameters in payload. >> > HTML Form parameters in POST method are passed in payload not in URI. >> > This is the reason why I put parameters in body. Problem is escaping & >> > character with &amp; How can I force BaseX to not escape characters in CDATA element ? >> > >> > This is my code: >> > <http:request method='post' href='http://localhost:3333'> >> > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"/> >> > <http:body media-type='application/x-www-form-urlencoded; charset=UTF-8'> >> > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> >> > </http:body> >> > </http:request> >> > >> > On server side I received following request: >> > >> > POST / HTTP/1.1 >> > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) >> > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 >> > Safari/537.36 >> > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 >> > >> > Host: copart.com:3333 >> > >> > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 >> > >> > Connection: keep-alive >> > Content-Length: 75 >> > >> > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 >> > >> > I expect that CDATA will be not escaped, or I am wrong ? >> > >> > >> > Pozdrawiam >> > Bogdan Bogucki >> > >> > HUSSAR SYSTEMS >> > >> > NIP: 532-170-12-00 >> > Regon: 142624070 >> > >> > e-mail: bbogucki at hussar.pl >> > tel. kom.: +48 607 409 301 >> > www: www.hussar.pl >> > >> > -----Wiadomość oryginalna----- >> > Od: Christian Grün [mailto:christian.gruen at gmail.com] >> > Wysłano: 3 lipca 2018 12:11 >> > Do: bbogucki at hussar.pl >> > DW: BaseX >> > Temat: Re: [basex-talk] http:send-request - problem with form data >> > >> > Hi Bogdan, >> > >> > If you want your arguments parsed as HTTP query parameters, it should suffice to attach them to the URL: >> > >> > let $url := 'http://localhost:3333' >> > let $params := ``[start=0&length=100&sort=date_type&page=0&size=100]`` >> > return http:send-request( >> > <http:request method='post' href='{ $url || '?' || $params }'> ... >> > >> > If this is not what you need, you might need to give us more information on how the request will be handled server-side (for example, do you use RESTXQ?). >> > >> > Best, >> > Christian >> > >> > >> > >> > >> > On Mon, Jul 2, 2018 at 11:11 PM Bogdan Bogucki <bbogucki at hussar.pl> wrote: >> > > >> > > Hello, >> > > >> > > >> > > >> > > I have problem with http:send-request. I want to send post method request with form data but I encountered problem with characters encoding. >> > > >> > > Example: >> > > >> > > >> > > >> > > Request: >> > > >> > > for $x in http:send-request( >> > > >> > > <http:request method='post' href= >> > > >> > > 'http://localhost:3333'> >> > > >> > > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT >> > > 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) >> > > Chrome/67.0.3396.99 Safari/537.36"/> >> > > >> > > <http:body media-type='application/x-www-form-urlencoded; >> > > charset=UTF-8'> >> > > >> > > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> >> > > >> > > </http:body> >> > > >> > > </http:request> >> > > >> > > )[2]/json/data/results/content >> > > >> > > return $x >> > > >> > > >> > > >> > > Optimized Query: >> > > >> > > util:item-at(http:send-request(element Q{http://expath.org/ns/http-client}request { (attribute method { ("post") }, attribute href { ("http://localhost:3333") }, element Q{http://expath.org/ns/http-client}header { (attribute name { ("User-Agent") }, attribute value { ("Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36") }) }, element Q{http://expath.org/ns/http-client}body { (attribute media-type { ("application/x-www-form-urlencoded; charset=UTF-8") }, "&#xA; start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100&#xA; ") }) }), 2)/json/data/results/content >> > > >> > > >> > > >> > > Request send: >> > > >> > > POST / HTTP/1.1 >> > > >> > > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) >> > > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 >> > > Safari/537.36 >> > > >> > > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 >> > > >> > > Host: copart.com:3333 >> > > >> > > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 >> > > >> > > Connection: keep-alive >> > > >> > > Content-Length: 75 >> > > >> > > >> > > >> > > >> > > >> > > >> > > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 >> > > >> > > >> > > >> > > Payload is not correctly encoded it should be: >> > > >> > > start=0&length=100&sort=date_type&page=0&size=100 >> > > >> > > >> > > >> > > How can I force function http:send-request to serialize CDATA without escaping ? >> > > >> > > >> > > >> > > >> > > >> > > Best Regards >> > > >> > > Bogdan Bogucki >> > > >> > > >> > >>

On Wed, Jul 4, 2018 at 10:05 PM Christian Grün <christian.gruen at gmail.com> wrote: > > And thanks for the confirmation. I will try to find out why the two alternatives make a difference. > > > > Bogdan Bogucki <bbogucki at hussar.pl> schrieb am Mi., 4. Juli 2018, 21:56: >> >> Hello, >> >> It works with body as a third parameter. Body has to be defined in http request because without that third parameter is ignored. >> >> Working example: >> >> for $x in http:send-request( >> <http:request method='post'> >> <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"/> >> <http:body media-type='application/x-www-form-urlencoded; charset=UTF-8'/> >> </http:request> >> ,'http:localhost:3333', 'start=0&amp;length=100&amp; >> ')[2]/data/results/content return $x >> >> Thank you Christian for help. >> >> Best Regards >> Bogdan Bogucki >> >> >> -----Wiadomość oryginalna----- >> Od: Christian Grün [mailto:christian.gruen at gmail.com] >> Wysłano: 4 lipca 2018 18:31 >> Do: bbogucki at hussar.pl >> DW: BaseX >> Temat: Re: [basex-talk] http:send-request - problem with form data >> >> Hi Bogdan, >> >> I didn’t try it, but what happens if you specify the body as third argument? >> >> Cheers, >> Christian >> >> >> On Tue, Jul 3, 2018 at 10:14 PM Bogdan Bogucki <bbogucki at hussar.pl> wrote: >> > >> > Hello, >> > >> > I don't want pass values in query string. Server side expect parameters in payload. >> > HTML Form parameters in POST method are passed in payload not in URI. >> > This is the reason why I put parameters in body. Problem is >> > escaping & character with &amp; How can I force BaseX to not escape characters in CDATA element ? >> > >> > This is my code: >> > <http:request method='post' href='http://localhost:3333'> >> > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"/> >> > <http:body media-type='application/x-www-form-urlencoded; charset=UTF-8'> >> > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> >> > </http:body> >> > </http:request> >> > >> > On server side I received following request: >> > >> > POST / HTTP/1.1 >> > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) >> > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 >> > Safari/537.36 >> > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 >> > >> > Host: copart.com:3333 >> > >> > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 >> > >> > Connection: keep-alive >> > Content-Length: 75 >> > >> > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 >> > >> > I expect that CDATA will be not escaped, or I am wrong ? >> > >> > >> > Pozdrawiam >> > Bogdan Bogucki >> > >> > HUSSAR SYSTEMS >> > >> > NIP: 532-170-12-00 >> > Regon: 142624070 >> > >> > e-mail: bbogucki at hussar.pl >> > tel. kom.: +48 607 409 301 >> > www: www.hussar.pl >> > >> > -----Wiadomość oryginalna----- >> > Od: Christian Grün [mailto:christian.gruen at gmail.com] >> > Wysłano: 3 lipca 2018 12:11 >> > Do: bbogucki at hussar.pl >> > DW: BaseX >> > Temat: Re: [basex-talk] http:send-request - problem with form data >> > >> > Hi Bogdan, >> > >> > If you want your arguments parsed as HTTP query parameters, it should suffice to attach them to the URL: >> > >> > let $url := 'http://localhost:3333' >> > let $params := ``[start=0&length=100&sort=date_type&page=0&size=100]`` >> > return http:send-request( >> > <http:request method='post' href='{ $url || '?' || $params }'> ... >> > >> > If this is not what you need, you might need to give us more information on how the request will be handled server-side (for example, do you use RESTXQ?). >> > >> > Best, >> > Christian >> > >> > >> > >> > >> > On Mon, Jul 2, 2018 at 11:11 PM Bogdan Bogucki <bbogucki at hussar.pl> wrote: >> > > >> > > Hello, >> > > >> > > >> > > >> > > I have problem with http:send-request. I want to send post method request with form data but I encountered problem with characters encoding. >> > > >> > > Example: >> > > >> > > >> > > >> > > Request: >> > > >> > > for $x in http:send-request( >> > > >> > > <http:request method='post' href= >> > > >> > > 'http://localhost:3333'> >> > > >> > > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT >> > > 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) >> > > Chrome/67.0.3396.99 Safari/537.36"/> >> > > >> > > <http:body media-type='application/x-www-form-urlencoded; >> > > charset=UTF-8'> >> > > >> > > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> >> > > >> > > </http:body> >> > > >> > > </http:request> >> > > >> > > )[2]/json/data/results/content >> > > >> > > return $x >> > > >> > > >> > > >> > > Optimized Query: >> > > >> > > util:item-at(http:send-request(element Q{http://expath.org/ns/http-client}request { (attribute method { ("post") }, attribute href { ("http://localhost:3333") }, element Q{http://expath.org/ns/http-client}header { (attribute name { ("User-Agent") }, attribute value { ("Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36") }) }, element Q{http://expath.org/ns/http-client}body { (attribute media-type { ("application/x-www-form-urlencoded; charset=UTF-8") }, "&#xA; start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100&#xA; ") }) }), 2)/json/data/results/content >> > > >> > > >> > > >> > > Request send: >> > > >> > > POST / HTTP/1.1 >> > > >> > > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) >> > > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 >> > > Safari/537.36 >> > > >> > > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 >> > > >> > > Host: copart.com:3333 >> > > >> > > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 >> > > >> > > Connection: keep-alive >> > > >> > > Content-Length: 75 >> > > >> > > >> > > >> > > >> > > >> > > >> > > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 >> > > >> > > >> > > >> > > Payload is not correctly encoded it should be: >> > > >> > > start=0&length=100&sort=date_type&page=0&size=100 >> > > >> > > >> > > >> > > How can I force function http:send-request to serialize CDATA without escaping ? >> > > >> > > >> > > >> > > >> > > >> > > Best Regards >> > > >> > > Bogdan Bogucki >> > > >> > > >> > >>

> I have one doubt why when we don't define body in http request third parameter is ignored ?

> Only when body element is provided parseBody with bodies is invoked. > I have to check source code to figure out how to pass body payload via third argument. > > if(body != null) { > final QNm pl = body.qname(); > // single part request > if(pl.eq(Q_BODY)) { > parseBody(body, bodies, hr.payloadAtts, hr.payload); > hr.isMultipart = false; > // multipart request > } else if(pl.eq(Q_MULTIPART)) { > parseMultipart(body, bodies.iter(), hr.payloadAtts, hr.parts); > hr.isMultipart = true; > } else { > throw HC_REQ_X.get(info, "Unknown payload element: " + body.qname()); > } > } > > Best Regards > Bogdan Bogucki > > > -----Wiadomość oryginalna----- > Od: Christian Grün [mailto:christian.gruen at gmail.com] > Wysłano: 12 lipca 2018 15:36 > Do: Bogdan Bogucki > DW: BaseX > Temat: Re: [basex-talk] http:send-request - problem with form data > > Hi Bogdan, > > I finally had some time to get to the bottom of the conversion inconsistencies: > > If the value is supplied as child of the http:body node, it will be serialized as text node. If it’s supplied as extra parameter, it will be serialized as string. If text nodes are serialized, entities will be encoded. Strings will be returned without encoded entities: > > <a><![CDATA[&]]></a> → <a>&amp;</a> > ``[&]`` → & > > I agree this is not very intuitive, and I have now rewritten this as > follows: If the supplied output media-type does not result in one of the default serialization methods of the spec (xml, xhtml, html, json, text, adaptive), the text node will be atomized (i.e., converted to an item of type xs:untypedAtomic), which will then be serialized as plain string. With the new snapshot [1], it should make no difference if you supply your value as child of http:body or as extra argument. > > As a general guide, I would recommended everyone to use the third argument for body items. It will be both safer and faster (in particular if binary data can be streamed). As the rich syntax of http:send-request is regularly confusing users on the list, I have reopened an old Github issue, which is about providing additional and simpler functions for the most widely used HTTP methods [2]. > > Best, > Christian > > [1] http://files.basex.org/releases/latest/ > [2] https://github.com/BaseXdb/basex/issues/914 > > > > On Wed, Jul 4, 2018 at 10:05 PM Christian Grün <christian.gruen at gmail.com> wrote: > > > > And thanks for the confirmation. I will try to find out why the two alternatives make a difference. > > > > > > > > Bogdan Bogucki <bbogucki at hussar.pl> schrieb am Mi., 4. Juli 2018, 21:56: > >> > >> Hello, > >> > >> It works with body as a third parameter. Body has to be defined in http request because without that third parameter is ignored. > >> > >> Working example: > >> > >> for $x in http:send-request( > >> <http:request method='post'> > >> <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"/> > >> <http:body media-type='application/x-www-form-urlencoded; charset=UTF-8'/> > >> </http:request> > >> ,'http:localhost:3333', 'start=0&amp;length=100&amp; > >> ')[2]/data/results/content return $x > >> > >> Thank you Christian for help. > >> > >> Best Regards > >> Bogdan Bogucki > >> > >> > >> -----Wiadomość oryginalna----- > >> Od: Christian Grün [mailto:christian.gruen at gmail.com] > >> Wysłano: 4 lipca 2018 18:31 > >> Do: bbogucki at hussar.pl > >> DW: BaseX > >> Temat: Re: [basex-talk] http:send-request - problem with form data > >> > >> Hi Bogdan, > >> > >> I didn’t try it, but what happens if you specify the body as third argument? > >> > >> Cheers, > >> Christian > >> > >> > >> On Tue, Jul 3, 2018 at 10:14 PM Bogdan Bogucki <bbogucki at hussar.pl> wrote: > >> > > >> > Hello, > >> > > >> > I don't want pass values in query string. Server side expect parameters in payload. > >> > HTML Form parameters in POST method are passed in payload not in URI. > >> > This is the reason why I put parameters in body. Problem is > >> > escaping & character with &amp; How can I force BaseX to not escape characters in CDATA element ? > >> > > >> > This is my code: > >> > <http:request method='post' href='http://localhost:3333'> > >> > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"/> > >> > <http:body media-type='application/x-www-form-urlencoded; charset=UTF-8'> > >> > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> > >> > </http:body> > >> > </http:request> > >> > > >> > On server side I received following request: > >> > > >> > POST / HTTP/1.1 > >> > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) > >> > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > >> > Safari/537.36 > >> > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 > >> > > >> > Host: copart.com:3333 > >> > > >> > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 > >> > > >> > Connection: keep-alive > >> > Content-Length: 75 > >> > > >> > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 > >> > > >> > I expect that CDATA will be not escaped, or I am wrong ? > >> > > >> > > >> > Pozdrawiam > >> > Bogdan Bogucki > >> > > >> > HUSSAR SYSTEMS > >> > > >> > NIP: 532-170-12-00 > >> > Regon: 142624070 > >> > > >> > e-mail: bbogucki at hussar.pl > >> > tel. kom.: +48 607 409 301 > >> > www: www.hussar.pl > >> > > >> > -----Wiadomość oryginalna----- > >> > Od: Christian Grün [mailto:christian.gruen at gmail.com] > >> > Wysłano: 3 lipca 2018 12:11 > >> > Do: bbogucki at hussar.pl > >> > DW: BaseX > >> > Temat: Re: [basex-talk] http:send-request - problem with form data > >> > > >> > Hi Bogdan, > >> > > >> > If you want your arguments parsed as HTTP query parameters, it should suffice to attach them to the URL: > >> > > >> > let $url := 'http://localhost:3333' > >> > let $params := ``[start=0&length=100&sort=date_type&page=0&size=100]`` > >> > return http:send-request( > >> > <http:request method='post' href='{ $url || '?' || $params }'> ... > >> > > >> > If this is not what you need, you might need to give us more information on how the request will be handled server-side (for example, do you use RESTXQ?). > >> > > >> > Best, > >> > Christian > >> > > >> > > >> > > >> > > >> > On Mon, Jul 2, 2018 at 11:11 PM Bogdan Bogucki <bbogucki at hussar.pl> wrote: > >> > > > >> > > Hello, > >> > > > >> > > > >> > > > >> > > I have problem with http:send-request. I want to send post method request with form data but I encountered problem with characters encoding. > >> > > > >> > > Example: > >> > > > >> > > > >> > > > >> > > Request: > >> > > > >> > > for $x in http:send-request( > >> > > > >> > > <http:request method='post' href= > >> > > > >> > > 'http://localhost:3333'> > >> > > > >> > > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT > >> > > 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) > >> > > Chrome/67.0.3396.99 Safari/537.36"/> > >> > > > >> > > <http:body media-type='application/x-www-form-urlencoded; > >> > > charset=UTF-8'> > >> > > > >> > > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> > >> > > > >> > > </http:body> > >> > > > >> > > </http:request> > >> > > > >> > > )[2]/json/data/results/content > >> > > > >> > > return $x > >> > > > >> > > > >> > > > >> > > Optimized Query: > >> > > > >> > > util:item-at(http:send-request(element Q{http://expath.org/ns/http-client}request { (attribute method { ("post") }, attribute href { ("http://localhost:3333") }, element Q{http://expath.org/ns/http-client}header { (attribute name { ("User-Agent") }, attribute value { ("Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36") }) }, element Q{http://expath.org/ns/http-client}body { (attribute media-type { ("application/x-www-form-urlencoded; charset=UTF-8") }, "&#xA; start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100&#xA; ") }) }), 2)/json/data/results/content > >> > > > >> > > > >> > > > >> > > Request send: > >> > > > >> > > POST / HTTP/1.1 > >> > > > >> > > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) > >> > > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > >> > > Safari/537.36 > >> > > > >> > > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 > >> > > > >> > > Host: copart.com:3333 > >> > > > >> > > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 > >> > > > >> > > Connection: keep-alive > >> > > > >> > > Content-Length: 75 > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 > >> > > > >> > > > >> > > > >> > > Payload is not correctly encoded it should be: > >> > > > >> > > start=0&length=100&sort=date_type&page=0&size=100 > >> > > > >> > > > >> > > > >> > > How can I force function http:send-request to serialize CDATA without escaping ? > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > Best Regards > >> > > > >> > > Bogdan Bogucki > >> > > > >> > > > >> > > >> >

On Thu, Jul 12, 2018 at 4:33 PM Christian Grün <christian.gruen at gmail.com> wrote: > Hi Bogdan, > > > I have one doubt why when we don't define body in http request third > parameter is ignored ? > > It was defined like this in the EXPath spec [1]: > > “When a body element has an empty content (i.e. it has no child node > at all) its content is given by the parameter $bodies. In a single > part request, this param must have at most one item. If the body is > empty, the param cannot be the empty sequence. In a multipart request, > $bodies must have as many items as there are empty body elements. If > there are three empty body elements, the content of the first of them > is $bodies[1], and so on. The number of empty body elements must be > equal to the number of items in $bodies.” > > I guess it would have been more user-friendly if an error was raised > if these requirements are not met by the function call. > > Anyway, as indicated, I’d be happy if we had functions with > unambiguous function arguments in the future. The Zorba HTTP Client > Module could serve as inspiration [2]. I have never tried it in > practice, but it looks pretty well-designed (an additional options > argument should probably added to the simple functions). > > Best, > Christian > > [1] http://expath.org/spec/http-client#d2e362 > [2] http://www.zorba.io/documentation/3.0/modules/zorba/io/http-client/ > > > > > Only when body element is provided parseBody with bodies is invoked. > > I have to check source code to figure out how to pass body payload via > third argument. > > > > if(body != null) { > > final QNm pl = body.qname(); > > // single part request > > if(pl.eq(Q_BODY)) { > > parseBody(body, bodies, hr.payloadAtts, hr.payload); > > hr.isMultipart = false; > > // multipart request > > } else if(pl.eq(Q_MULTIPART)) { > > parseMultipart(body, bodies.iter(), hr.payloadAtts, hr.parts); > > hr.isMultipart = true; > > } else { > > throw HC_REQ_X.get(info, "Unknown payload element: " + > body.qname()); > > } > > } > > > > Best Regards > > Bogdan Bogucki > > > > > > -----Wiadomość oryginalna----- > > Od: Christian Grün [mailto:christian.gruen at gmail.com] > > Wysłano: 12 lipca 2018 15:36 > > Do: Bogdan Bogucki > > DW: BaseX > > Temat: Re: [basex-talk] http:send-request - problem with form data > > > > Hi Bogdan, > > > > I finally had some time to get to the bottom of the conversion > inconsistencies: > > > > If the value is supplied as child of the http:body node, it will be > serialized as text node. If it’s supplied as extra parameter, it will be > serialized as string. If text nodes are serialized, entities will be > encoded. Strings will be returned without encoded entities: > > > > <a><![CDATA[&]]></a> → <a>&amp;</a> > > ``[&]`` → & > > > > I agree this is not very intuitive, and I have now rewritten this as > > follows: If the supplied output media-type does not result in one of the > default serialization methods of the spec (xml, xhtml, html, json, text, > adaptive), the text node will be atomized (i.e., converted to an item of > type xs:untypedAtomic), which will then be serialized as plain string. With > the new snapshot [1], it should make no difference if you supply your value > as child of http:body or as extra argument. > > > > As a general guide, I would recommended everyone to use the third > argument for body items. It will be both safer and faster (in particular if > binary data can be streamed). As the rich syntax of http:send-request is > regularly confusing users on the list, I have reopened an old Github issue, > which is about providing additional and simpler functions for the most > widely used HTTP methods [2]. > > > > Best, > > Christian > > > > [1] http://files.basex.org/releases/latest/ > > [2] https://github.com/BaseXdb/basex/issues/914 > > > > > > > > On Wed, Jul 4, 2018 at 10:05 PM Christian Grün < > christian.gruen at gmail.com> wrote: > > > > > > And thanks for the confirmation. I will try to find out why the two > alternatives make a difference. > > > > > > > > > > > > Bogdan Bogucki <bbogucki at hussar.pl> schrieb am Mi., 4. Juli 2018, > 21:56: > > >> > > >> Hello, > > >> > > >> It works with body as a third parameter. Body has to be defined in > http request because without that third parameter is ignored. > > >> > > >> Working example: > > >> > > >> for $x in http:send-request( > > >> <http:request method='post'> > > >> <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT > 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > Safari/537.36"/> > > >> <http:body media-type='application/x-www-form-urlencoded; > charset=UTF-8'/> > > >> </http:request> > > >> ,'http:localhost:3333', 'start=0&amp;length=100&amp; > > >> ')[2]/data/results/content return $x > > >> > > >> Thank you Christian for help. > > >> > > >> Best Regards > > >> Bogdan Bogucki > > >> > > >> > > >> -----Wiadomość oryginalna----- > > >> Od: Christian Grün [mailto:christian.gruen at gmail.com] > > >> Wysłano: 4 lipca 2018 18:31 > > >> Do: bbogucki at hussar.pl > > >> DW: BaseX > > >> Temat: Re: [basex-talk] http:send-request - problem with form data > > >> > > >> Hi Bogdan, > > >> > > >> I didn’t try it, but what happens if you specify the body as third > argument? > > >> > > >> Cheers, > > >> Christian > > >> > > >> > > >> On Tue, Jul 3, 2018 at 10:14 PM Bogdan Bogucki <bbogucki at hussar.pl> > wrote: > > >> > > > >> > Hello, > > >> > > > >> > I don't want pass values in query string. Server side expect > parameters in payload. > > >> > HTML Form parameters in POST method are passed in payload not in > URI. > > >> > This is the reason why I put parameters in body. Problem is > > >> > escaping & character with &amp; How can I force BaseX to not escape > characters in CDATA element ? > > >> > > > >> > This is my code: > > >> > <http:request method='post' href='http://localhost:3333'> > > >> > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT > 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > Safari/537.36"/> > > >> > <http:body media-type='application/x-www-form-urlencoded; > charset=UTF-8'> > > >> > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> > > >> > </http:body> > > >> > </http:request> > > >> > > > >> > On server side I received following request: > > >> > > > >> > POST / HTTP/1.1 > > >> > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) > > >> > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > > >> > Safari/537.36 > > >> > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 > > >> > > > >> > Host: copart.com:3333 > > >> > > > >> > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 > > >> > > > >> > Connection: keep-alive > > >> > Content-Length: 75 > > >> > > > >> > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 > > >> > > > >> > I expect that CDATA will be not escaped, or I am wrong ? > > >> > > > >> > > > >> > Pozdrawiam > > >> > Bogdan Bogucki > > >> > > > >> > HUSSAR SYSTEMS > > >> > > > >> > NIP: 532-170-12-00 > > >> > Regon: 142624070 > > >> > > > >> > e-mail: bbogucki at hussar.pl > > >> > tel. kom.: +48 607 409 301 > > >> > www: www.hussar.pl > > >> > > > >> > -----Wiadomość oryginalna----- > > >> > Od: Christian Grün [mailto:christian.gruen at gmail.com] > > >> > Wysłano: 3 lipca 2018 12:11 > > >> > Do: bbogucki at hussar.pl > > >> > DW: BaseX > > >> > Temat: Re: [basex-talk] http:send-request - problem with form data > > >> > > > >> > Hi Bogdan, > > >> > > > >> > If you want your arguments parsed as HTTP query parameters, it > should suffice to attach them to the URL: > > >> > > > >> > let $url := 'http://localhost:3333' > > >> > let $params := > ``[start=0&length=100&sort=date_type&page=0&size=100]`` > > >> > return http:send-request( > > >> > <http:request method='post' href='{ $url || '?' || $params }'> > ... > > >> > > > >> > If this is not what you need, you might need to give us more > information on how the request will be handled server-side (for example, do > you use RESTXQ?). > > >> > > > >> > Best, > > >> > Christian > > >> > > > >> > > > >> > > > >> > > > >> > On Mon, Jul 2, 2018 at 11:11 PM Bogdan Bogucki <bbogucki at hussar.pl> > wrote: > > >> > > > > >> > > Hello, > > >> > > > > >> > > > > >> > > > > >> > > I have problem with http:send-request. I want to send post method > request with form data but I encountered problem with characters encoding. > > >> > > > > >> > > Example: > > >> > > > > >> > > > > >> > > > > >> > > Request: > > >> > > > > >> > > for $x in http:send-request( > > >> > > > > >> > > <http:request method='post' href= > > >> > > > > >> > > 'http://localhost:3333'> > > >> > > > > >> > > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT > > >> > > 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) > > >> > > Chrome/67.0.3396.99 Safari/537.36"/> > > >> > > > > >> > > <http:body media-type='application/x-www-form-urlencoded; > > >> > > charset=UTF-8'> > > >> > > > > >> > > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> > > >> > > > > >> > > </http:body> > > >> > > > > >> > > </http:request> > > >> > > > > >> > > )[2]/json/data/results/content > > >> > > > > >> > > return $x > > >> > > > > >> > > > > >> > > > > >> > > Optimized Query: > > >> > > > > >> > > util:item-at(http:send-request(element Q{ > http://expath.org/ns/http-client}request { (attribute method { ("post") > }, attribute href { ("http://localhost:3333") }, element Q{ > http://expath.org/ns/http-client}header { (attribute name { > ("User-Agent") }, attribute value { ("Mozilla/5.0 (Windows NT 6.1; Win64; > x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > Safari/537.36") }) }, element Q{http://expath.org/ns/http-client}body { > (attribute media-type { ("application/x-www-form-urlencoded; > charset=UTF-8") }, "&#xA; > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100&#xA; > ") }) }), 2)/json/data/results/content > > >> > > > > >> > > > > >> > > > > >> > > Request send: > > >> > > > > >> > > POST / HTTP/1.1 > > >> > > > > >> > > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) > > >> > > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 > > >> > > Safari/537.36 > > >> > > > > >> > > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 > > >> > > > > >> > > Host: copart.com:3333 > > >> > > > > >> > > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 > > >> > > > > >> > > Connection: keep-alive > > >> > > > > >> > > Content-Length: 75 > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 > > >> > > > > >> > > > > >> > > > > >> > > Payload is not correctly encoded it should be: > > >> > > > > >> > > start=0&length=100&sort=date_type&page=0&size=100 > > >> > > > > >> > > > > >> > > > > >> > > How can I force function http:send-request to serialize CDATA > without escaping ? > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > Best Regards > > >> > > > > >> > > Bogdan Bogucki > > >> > > > > >> > > > > >> > > > >> > > >

Joe Wicentowski <joewiz at gmail.com> schrieb am Fr., 13. Juli 2018, 03:41: > The Zorba HTTP spec is very interesting. I think the original EXPath HTTP > spec (and others) may have taken a more map-and-array form had those data > structures been available in XQuery at the time. It would be interesting to > reconsider some of these specs in a future major revision to them. > > On Thu, Jul 12, 2018 at 4:33 PM Christian Grün <christian.gruen at gmail.com> > wrote: > >> Hi Bogdan, >> >> > I have one doubt why when we don't define body in http request third >> parameter is ignored ? >> >> It was defined like this in the EXPath spec [1]: >> >> “When a body element has an empty content (i.e. it has no child node >> at all) its content is given by the parameter $bodies. In a single >> part request, this param must have at most one item. If the body is >> empty, the param cannot be the empty sequence. In a multipart request, >> $bodies must have as many items as there are empty body elements. If >> there are three empty body elements, the content of the first of them >> is $bodies[1], and so on. The number of empty body elements must be >> equal to the number of items in $bodies.” >> >> I guess it would have been more user-friendly if an error was raised >> if these requirements are not met by the function call. >> >> Anyway, as indicated, I’d be happy if we had functions with >> unambiguous function arguments in the future. The Zorba HTTP Client >> Module could serve as inspiration [2]. I have never tried it in >> practice, but it looks pretty well-designed (an additional options >> argument should probably added to the simple functions). >> >> Best, >> Christian >> >> [1] http://expath.org/spec/http-client#d2e362 >> [2] http://www.zorba.io/documentation/3.0/modules/zorba/io/http-client/ >> >> >> >> > Only when body element is provided parseBody with bodies is invoked. >> > I have to check source code to figure out how to pass body payload via >> third argument. >> > >> > if(body != null) { >> > final QNm pl = body.qname(); >> > // single part request >> > if(pl.eq(Q_BODY)) { >> > parseBody(body, bodies, hr.payloadAtts, hr.payload); >> > hr.isMultipart = false; >> > // multipart request >> > } else if(pl.eq(Q_MULTIPART)) { >> > parseMultipart(body, bodies.iter(), hr.payloadAtts, hr.parts); >> > hr.isMultipart = true; >> > } else { >> > throw HC_REQ_X.get(info, "Unknown payload element: " + >> body.qname()); >> > } >> > } >> > >> > Best Regards >> > Bogdan Bogucki >> > >> > >> > -----Wiadomość oryginalna----- >> > Od: Christian Grün [mailto:christian.gruen at gmail.com] >> > Wysłano: 12 lipca 2018 15:36 >> > Do: Bogdan Bogucki >> > DW: BaseX >> > Temat: Re: [basex-talk] http:send-request - problem with form data >> > >> > Hi Bogdan, >> > >> > I finally had some time to get to the bottom of the conversion >> inconsistencies: >> > >> > If the value is supplied as child of the http:body node, it will be >> serialized as text node. If it’s supplied as extra parameter, it will be >> serialized as string. If text nodes are serialized, entities will be >> encoded. Strings will be returned without encoded entities: >> > >> > <a><![CDATA[&]]></a> → <a>&amp;</a> >> > ``[&]`` → & >> > >> > I agree this is not very intuitive, and I have now rewritten this as >> > follows: If the supplied output media-type does not result in one of >> the default serialization methods of the spec (xml, xhtml, html, json, >> text, adaptive), the text node will be atomized (i.e., converted to an item >> of type xs:untypedAtomic), which will then be serialized as plain string. >> With the new snapshot [1], it should make no difference if you supply your >> value as child of http:body or as extra argument. >> > >> > As a general guide, I would recommended everyone to use the third >> argument for body items. It will be both safer and faster (in particular if >> binary data can be streamed). As the rich syntax of http:send-request is >> regularly confusing users on the list, I have reopened an old Github issue, >> which is about providing additional and simpler functions for the most >> widely used HTTP methods [2]. >> > >> > Best, >> > Christian >> > >> > [1] http://files.basex.org/releases/latest/ >> > [2] https://github.com/BaseXdb/basex/issues/914 >> > >> > >> > >> > On Wed, Jul 4, 2018 at 10:05 PM Christian Grün < >> christian.gruen at gmail.com> wrote: >> > > >> > > And thanks for the confirmation. I will try to find out why the two >> alternatives make a difference. >> > > >> > > >> > > >> > > Bogdan Bogucki <bbogucki at hussar.pl> schrieb am Mi., 4. Juli 2018, >> 21:56: >> > >> >> > >> Hello, >> > >> >> > >> It works with body as a third parameter. Body has to be defined in >> http request because without that third parameter is ignored. >> > >> >> > >> Working example: >> > >> >> > >> for $x in http:send-request( >> > >> <http:request method='post'> >> > >> <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT >> 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 >> Safari/537.36"/> >> > >> <http:body media-type='application/x-www-form-urlencoded; >> charset=UTF-8'/> >> > >> </http:request> >> > >> ,'http:localhost:3333', 'start=0&amp;length=100&amp; >> > >> ')[2]/data/results/content return $x >> > >> >> > >> Thank you Christian for help. >> > >> >> > >> Best Regards >> > >> Bogdan Bogucki >> > >> >> > >> >> > >> -----Wiadomość oryginalna----- >> > >> Od: Christian Grün [mailto:christian.gruen at gmail.com] >> > >> Wysłano: 4 lipca 2018 18:31 >> > >> Do: bbogucki at hussar.pl >> > >> DW: BaseX >> > >> Temat: Re: [basex-talk] http:send-request - problem with form data >> > >> >> > >> Hi Bogdan, >> > >> >> > >> I didn’t try it, but what happens if you specify the body as third >> argument? >> > >> >> > >> Cheers, >> > >> Christian >> > >> >> > >> >> > >> On Tue, Jul 3, 2018 at 10:14 PM Bogdan Bogucki <bbogucki at hussar.pl> >> wrote: >> > >> > >> > >> > Hello, >> > >> > >> > >> > I don't want pass values in query string. Server side expect >> parameters in payload. >> > >> > HTML Form parameters in POST method are passed in payload not in >> URI. >> > >> > This is the reason why I put parameters in body. Problem is >> > >> > escaping & character with &amp; How can I force BaseX to not >> escape characters in CDATA element ? >> > >> > >> > >> > This is my code: >> > >> > <http:request method='post' href='http://localhost:3333'> >> > >> > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT >> 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 >> Safari/537.36"/> >> > >> > <http:body media-type='application/x-www-form-urlencoded; >> charset=UTF-8'> >> > >> > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> >> > >> > </http:body> >> > >> > </http:request> >> > >> > >> > >> > On server side I received following request: >> > >> > >> > >> > POST / HTTP/1.1 >> > >> > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) >> > >> > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 >> > >> > Safari/537.36 >> > >> > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 >> > >> > >> > >> > Host: copart.com:3333 >> > >> > >> > >> > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 >> > >> > >> > >> > Connection: keep-alive >> > >> > Content-Length: 75 >> > >> > >> > >> > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 >> > >> > >> > >> > I expect that CDATA will be not escaped, or I am wrong ? >> > >> > >> > >> > >> > >> > Pozdrawiam >> > >> > Bogdan Bogucki >> > >> > >> > >> > HUSSAR SYSTEMS >> > >> > >> > >> > NIP: 532-170-12-00 >> > >> > Regon: 142624070 >> > >> > >> > >> > e-mail: bbogucki at hussar.pl >> > >> > tel. kom.: +48 607 409 301 >> > >> > www: www.hussar.pl >> > >> > >> > >> > -----Wiadomość oryginalna----- >> > >> > Od: Christian Grün [mailto:christian.gruen at gmail.com] >> > >> > Wysłano: 3 lipca 2018 12:11 >> > >> > Do: bbogucki at hussar.pl >> > >> > DW: BaseX >> > >> > Temat: Re: [basex-talk] http:send-request - problem with form data >> > >> > >> > >> > Hi Bogdan, >> > >> > >> > >> > If you want your arguments parsed as HTTP query parameters, it >> should suffice to attach them to the URL: >> > >> > >> > >> > let $url := 'http://localhost:3333' >> > >> > let $params := >> ``[start=0&length=100&sort=date_type&page=0&size=100]`` >> > >> > return http:send-request( >> > >> > <http:request method='post' href='{ $url || '?' || $params }'> >> ... >> > >> > >> > >> > If this is not what you need, you might need to give us more >> information on how the request will be handled server-side (for example, do >> you use RESTXQ?). >> > >> > >> > >> > Best, >> > >> > Christian >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > On Mon, Jul 2, 2018 at 11:11 PM Bogdan Bogucki <bbogucki at hussar.pl> >> wrote: >> > >> > > >> > >> > > Hello, >> > >> > > >> > >> > > >> > >> > > >> > >> > > I have problem with http:send-request. I want to send post >> method request with form data but I encountered problem with characters >> encoding. >> > >> > > >> > >> > > Example: >> > >> > > >> > >> > > >> > >> > > >> > >> > > Request: >> > >> > > >> > >> > > for $x in http:send-request( >> > >> > > >> > >> > > <http:request method='post' href= >> > >> > > >> > >> > > 'http://localhost:3333'> >> > >> > > >> > >> > > <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT >> > >> > > 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) >> > >> > > Chrome/67.0.3396.99 Safari/537.36"/> >> > >> > > >> > >> > > <http:body media-type='application/x-www-form-urlencoded; >> > >> > > charset=UTF-8'> >> > >> > > >> > >> > > <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]> >> > >> > > >> > >> > > </http:body> >> > >> > > >> > >> > > </http:request> >> > >> > > >> > >> > > )[2]/json/data/results/content >> > >> > > >> > >> > > return $x >> > >> > > >> > >> > > >> > >> > > >> > >> > > Optimized Query: >> > >> > > >> > >> > > util:item-at(http:send-request(element Q{ >> http://expath.org/ns/http-client}request { (attribute method { ("post") >> }, attribute href { ("http://localhost:3333") }, element Q{ >> http://expath.org/ns/http-client}header { (attribute name { >> ("User-Agent") }, attribute value { ("Mozilla/5.0 (Windows NT 6.1; Win64; >> x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 >> Safari/537.36") }) }, element Q{http://expath.org/ns/http-client}body { >> (attribute media-type { ("application/x-www-form-urlencoded; >> charset=UTF-8") }, "&#xA; >> start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100&#xA; >> ") }) }), 2)/json/data/results/content >> > >> > > >> > >> > > >> > >> > > >> > >> > > Request send: >> > >> > > >> > >> > > POST / HTTP/1.1 >> > >> > > >> > >> > > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) >> > >> > > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 >> > >> > > Safari/537.36 >> > >> > > >> > >> > > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 >> > >> > > >> > >> > > Host: copart.com:3333 >> > >> > > >> > >> > > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 >> > >> > > >> > >> > > Connection: keep-alive >> > >> > > >> > >> > > Content-Length: 75 >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100 >> > >> > > >> > >> > > >> > >> > > >> > >> > > Payload is not correctly encoded it should be: >> > >> > > >> > >> > > start=0&length=100&sort=date_type&page=0&size=100 >> > >> > > >> > >> > > >> > >> > > >> > >> > > How can I force function http:send-request to serialize CDATA >> without escaping ? >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > Best Regards >> > >> > > >> > >> > > Bogdan Bogucki >> > >> > > >> > >> > > >> > >> > >> > >> >> > >> > -- > Sent from my iPhone >