Setting the context item via the native protocol
Thursday, 15 March 2012 · 9 messages · last reply 17 Mar 2012
CG
Quoted text · Charles Duffy · 2 lines
> Right now, I'm aware of two ways to set the context item via the native > protocol:
Here are some other alternatives (...not sure what you mean with the "native protocol"?):
-- If you are using the command-line, you can also use the "-i" flag to bind documents or databases to the context.
-- If you are working more low level, you can directly bind items to the QueryProcessor [1]
-- If you are using REST, you can use the Context parameter (e.g., the <rest:context> element) to bind items to the context [2]
Quoted text · Charles Duffy · 2 lines
> (2) Use the "CS" ["Context Set"] command > - Disadvantage: Requires a database to be open (why?)
This command works similar to other database commands, which refer to the currently opened database instance.
What's the way you would prefer?
Christian
[1] https://github.com/BaseXdb/basex-examples/blob/master/src/main/java/org/basex/examples/query/BindContext.java [2] http://docs.basex.org/wiki/REST
CD
Charles Duffy15 Mar 2012 02:44 #
Howdy --
Right now, I'm aware of two ways to set the context item via the native protocol:
(1) Insert a "declare context item" header into the document
- Disadvantage: Should only work when XQUERY3 is true
- Disadvantage: Modifies line/column references for errors received
- Disadvantage: Hard to ensure that user-provided XML document will evaluate to itself in XQuery (security risk)
(2) Use the "CS" ["Context Set"] command
- Disadvantage: Requires a database to be open (why?)
- Disadvantage: Hard to ensure that user-provided XML document will evaluate to itself in XQuery (security risk)
Is there a better approach?
CD
Quoted text · Christian Grün · 6 lines
On 03/14/2012 08:08 PM, Christian Grün wrote: >> Right now, I'm aware of two ways to set the context item via the native >> protocol: > > Here are some other alternatives (...not sure what you mean with the > "native protocol"?):
By "native protocol", I was referring to the protocol documented at http://docs.basex.org/wiki/Server_Protocol#Transfer_Protocol, ie. operating with a server over-the-wire.
Quoted text · Christian Grün · 2 lines
> -- If you are using the command-line, you can also use the "-i" flag > to bind documents or databases to the context.
I see that basexclient's -i uses the CHECK command over-the-wire to ask the server to open the named file as a database. Unfortunately, I'm looking to provide content over-the-wire, rather than being able to simply pass a location.
Quoted text · Christian Grün · 2 lines
> -- If you are working more low level, you can directly bind items to > the QueryProcessor [1]
...which would be perfect if my usage were embedded. That said, this is actually very compelling -- if only I knew the over-the-wire equivalent to proc.context().
Quoted text · Christian Grün · 2 lines
> -- If you are using REST, you can use the Context parameter (e.g., the > <rest:context> element) to bind items to the context [2]
The REST interface is what I've been using historically; I'm currently in the process of porting my code to work without it.
Quoted text · Christian Grün · 5 lines
>> (2) Use the "CS" ["Context Set"] command >> - Disadvantage: Requires a database to be open (why?) > > This command works similar to other database commands, which refer to > the currently opened database instance.
The "xquery" command, for instance, can be run without a currently opened database instance -- if run with a query which doesn't refer to the context, it succeeds without such a context.
Quoted text · Christian Grün · 1 line
> What's the way you would prefer?
Hopefully something exists already. What I would consider ideal would be an equivalent to ADD (including respecting the
PARSER option) which sets the context item rather than storing to the database.Quoted text · Christian Grün · 4 lines
> Christian > > [1] https://github.com/BaseXdb/basex-examples/blob/master/src/main/java/org/basex/examples/query/BindContext.java > [2] http://docs.basex.org/wiki/REST
CD
Charles Duffy15 Mar 2012 18:51 #
Quoted text · Christian Grün · 8 lines
On 03/14/2012 08:39 PM, Charles Duffy wrote: > On 03/14/2012 08:08 PM, Christian Grün wrote: >> -- If you are working more low level, you can directly bind items to >> the QueryProcessor [1] > > ...which would be perfect if my usage were embedded. That said, this is > actually very compelling -- if only I knew the over-the-wire equivalent > to proc.context().
This still remains the first-resort approach, if possible. However...
Quoted text · Charles Duffy · 5 lines
>> What's the way you would prefer? > > Hopefully something exists already. What I would consider ideal would be > an equivalent to ADD (including respecting the PARSER option) which sets > the context item rather than storing to the database.
...actually -- if one can bind parameters to a CS command, doing this (with CS able to work without a database) would work as well as a new ADD-equivalent command, while being more flexible -- as all the parsers are available from the XQuery language, they could be invoked from the query given as argument to CS.
That is:
SET BINDINGS $json-string={'foo':'bar'}
CS declare $json-string external; json:parse($json-string)That said, it'd be nice if queries created via the QUERY command could be executed with the context set as their destination rather than having results returned as a string -- an equivalent to EXECUTE which instead puts the output in the context set.
CD
Quoted text · Charles Duffy · 12 lines
On 03/14/2012 07:44 PM, Charles Duffy wrote: > Howdy -- > > Right now, I'm aware of two ways to set the context item via the native > protocol: > > (1) Insert a "declare context item" header into the document > > - Disadvantage: Should only work when XQUERY3 is true > - Disadvantage: Modifies line/column references for errors received > - Disadvantage: Hard to ensure that user-provided XML document will > evaluate to itself in XQuery (security risk)
Bigger disadvantage: ContextItemDecl is allowed only in the second portion of the prolog, so inserting this unambiguously would require parsing the 3rd-party document. Ugh.
Quoted text · Charles Duffy · 5 lines
> (2) Use the "CS" ["Context Set"] command > > - Disadvantage: Requires a database to be open (why?) > - Disadvantage: Hard to ensure that user-provided XML document will > evaluate to itself in XQuery (security risk)
I really should have tested before considering this feasible:
Quoted text · Charles Duffy · 1 line
> CS <root/>
[BASX0007] Query must yield database nodes.
CD
Charles Duffy16 Mar 2012 19:01 #
Quoted text · Charles Duffy · 28 lines
On 03/16/2012 11:06 AM, Charles Duffy wrote: > On 03/14/2012 07:44 PM, Charles Duffy wrote: >> Howdy -- >> >> Right now, I'm aware of two ways to set the context item via the native >> protocol: >> >> (1) Insert a "declare context item" header into the document >> >> - Disadvantage: Should only work when XQUERY3 is true >> - Disadvantage: Modifies line/column references for errors received >> - Disadvantage: Hard to ensure that user-provided XML document will >> evaluate to itself in XQuery (security risk) > > Bigger disadvantage: ContextItemDecl is allowed only in the second > portion of the prolog, so inserting this unambiguously would require > parsing the 3rd-party document. Ugh. > >> (2) Use the "CS" ["Context Set"] command >> >> - Disadvantage: Requires a database to be open (why?) >> - Disadvantage: Hard to ensure that user-provided XML document will >> evaluate to itself in XQuery (security risk) > > I really should have tested before considering this feasible: > > > CS <root/> > [BASX0007] Query must yield database nodes.
So:
Context.current() accepts only Nodes, which are database-backed; QueryProcessor.context(), by contrast, can accept the ItemCache returned by a query which doesn't return a database reference.
As such -- it seems to me that what is needed is an extension to the query command protocol (as implemented by ClientListener.query()), since a QueryProcessor is available when query commands are being handled but not otherwise.
Perhaps we could define a constant "name" value, something not legal as a string, used to refer to the context as a target for the existing BIND command? Let me propose the byte \1, as in:
\3 query_id\0 \1\0 <root/>\0 \0
...the "type" field in this case being empty simply because I'm not quite sure what to do with it.
It might also be worthwhile to have an additional query command to the effect of "execute query with id1, bind result to query id2", as follows:
\8 source_query_id\0 dest_query_id\0 bind_variable_name\0
...with the byte \1 remaining a valid bind_variable_name value indicating that the target is the context item.
Allowing this would permit clients to ask the server to do parsing operations remotely and set the results as a context variable for a future operation, without needing to transfer the parse results to the client between operations.
My goal here, by the way, is to have feature-parity between the REST API and the server protocol -- the REST API allows the context item to be set when running an arbitrary, unmodified query; the server protocol, at present, does not. One of my side projects is providing a
very-high-level, stateless Python API able to work through both backends (my employer has indicated intent to allow this to be eventually released), and this limitation in the protocol prevents parity between the backends.
Thanks!
CG
Christian Grün16 Mar 2012 20:01 #
Quoted text · Charles Duffy · 8 lines
> Context.current() accepts only Nodes, which are database-backed; > QueryProcessor.context(), by contrast, can accept the ItemCache returned by > a query which doesn't return a database reference. > > As such -- it seems to me that what is needed is an extension to the query > command protocol (as implemented by ClientListener.query()), since a > QueryProcessor is available when query commands are being handled but not > otherwise.
Helpful ideas. I thought about sth. similar, which is why I'll now add a new CONTEXT command with a value (using code \14) and optional type as arguments.
Quoted text · Charles Duffy · 2 lines
> It might also be worthwhile to have an additional query command to the > effect of "execute query with id1, bind result to query id2", as follows:
While I clearly see the advantages of that approach, I'm a little bit hesitant to introduce this feature too quickly, as I think this will introduce a number of open issues (some of the queries may be updating; it may be generally desirable to introduce the concept of transactions in order to perform multiple commands and queries with no other processes interfering; etc.).
Quoted text · Charles Duffy · 4 lines
> My goal here, by the way, is to have feature-parity between the REST API and > the server protocol -- the REST API allows the context item to be set when > running an arbitrary, unmodified query; the server protocol, at present, > does not.
Yes, that's absolutely sensible. Your feedback is welcome,
Christian
CG
Christian Grün16 Mar 2012 21:34 #
The context() method has been added. Moreover, I have added new bind() and context() function signatures to the client APIs with an additional "type" argument.
Looking forward to your first tests,
Christian
PS: the "latest snapshots" are currently uploaded; it usually takes around 5 minutes until all files have been updated.
Quoted text · Christian Grün · 34 lines
On Fri, Mar 16, 2012 at 8:01 PM, Christian Grün <christian.gruen at gmail.com> wrote: >> Context.current() accepts only Nodes, which are database-backed; >> QueryProcessor.context(), by contrast, can accept the ItemCache returned by >> a query which doesn't return a database reference. >> >> As such -- it seems to me that what is needed is an extension to the query >> command protocol (as implemented by ClientListener.query()), since a >> QueryProcessor is available when query commands are being handled but not >> otherwise. > > Helpful ideas. I thought about sth. similar, which is why I'll now add > a new CONTEXT command with a value (using code \14) and optional type > as arguments. > > >> It might also be worthwhile to have an additional query command to the >> effect of "execute query with id1, bind result to query id2", as follows: > > While I clearly see the advantages of that approach, I'm a little bit > hesitant to introduce this feature too quickly, as I think this will > introduce a number of open issues (some of the queries may be > updating; it may be generally desirable to introduce the concept of > transactions in order to perform multiple commands and queries with no > other processes interfering; etc.). > > >> My goal here, by the way, is to have feature-parity between the REST API and >> the server protocol -- the REST API allows the context item to be set when >> running an arbitrary, unmodified query; the server protocol, at present, >> does not. > > Yes, that's absolutely sensible. Your feedback is welcome, > Christian
CD
Charles Duffy17 Mar 2012 23:46 #
Quoted text · Christian Grün · 4 lines
On 03/16/2012 03:34 PM, Christian Grün wrote: > The context() method has been added. Moreover, I have added new bind() > and context() function signatures to the client APIs with an > additional "type" argument.
Most excellent; thanks!
I'm a little curious how the default behavior when no type value is passed would be used -- as it stands, I always get "[XPDY0050] Root of the context item must be a document node" when not explicitly passing "XML" in the type field.
Perhaps it might make more sense for the client libraries to pass in 'XML' as the default value when .context() is called with only two arguments?
Written for BaseX 7.1. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery