Programmatically extracting function signatures
Monday, 17 September 2018 · 8 messages · last reply 7 Nov 2018
AB
If you look in lib/ folder the file modules.zip contains the BaseX modules as .xqm stub files with functions marked as external.
The entry for
archive:delete is shown below. At one point I am sure these *.xqm could be parsed by functions like inspect:module and inspect:xqdoc Currently they raise errors like [XQST0045] Function '...' is in reserved namespace.
I believe were generated by scraping the wiki :)
However:
* some of these are a little out of date (at least in 9.02)
* modules.zip is not present in recent 9.1 betas
This is a shame as I hoped to use to one day use these in some kind of IDE for content assist.
(although I have hoped that for a long time without actually doing it.)
Regards
/Andy
(:~
: Deletes entries from an <code>$archive</code> .
: The format of <code>$entries</code> is the same as for <a href="#archive:create">archive:create</a> .
:
: @error bxerr:ARCH0005 the entries of the given archive cannot be modified.
: @error bxerr:ARCH9999 archive creation failed for some other reason. :)
declare function archive:delete($archive as xs:base64Binary, $entries as item()*) as xs:base64Binary external;Quoted text · Bridger Dyson-Smith · 23 lines
On 17 September 2018 at 16:43, Bridger Dyson-Smith <bdysonsmith at gmail.com> wrote: > Hi all - > > I was curious if there was a way to extract function signatures from > BaseX. I'm currently reading through the (excellent) wiki but was curious > if there was a more automated process I might be able to use. I've been > trying to think of something clever with function-lookup, but I can't seem > to make it dynamic based on module namespace; e.g. > > ``` > for $fun in function-lookup(xs:QName("archive:delete"), 1) > return > $fun > ``` > > I may just grab the wiki and parse it :), but I'm also interested in > changes between versions of the processor. > > Any thoughts or suggestions would be greatly appreciated! > Best, > Bridger >
AB
Sounds good. Of course I meant the etc/ folder not lib/.
/Andy
Quoted text · Bridger Dyson-Smith · 74 lines
On 17 September 2018 at 17:28, Bridger Dyson-Smith <bdysonsmith at gmail.com> wrote: > Andy - > thank you for that suggestion! > > On Mon, Sep 17, 2018 at 12:13 PM Andy Bunce <bunce.andy at gmail.com> wrote: > >> If you look in lib/ folder the file modules.zip contains the BaseX >> modules as .xqm stub files with functions marked as external. >> The entry for archive:delete is shown below. At one point I am sure these >> *.xqm could be parsed by functions like inspect:module and inspect:xqdoc >> Currently they raise errors like [XQST0045] Function '...' is in reserved >> namespace. >> >> I believe were generated by scraping the wiki :) >> >> However: >> * some of these are a little out of date (at least in 9.02) >> * modules.zip is not present in recent 9.1 betas >> >> This is a shame as I hoped to use to one day use these in some kind of >> IDE for content assist. >> (although I have hoped that for a long time without actually doing it.) >> >> This is exactly the use case I have. I'm attempting to help work on an > IntelliJ plugin that will leverage the signatures (for things I don't quite > understand :). > > Regards >> /Andy >> >> (:~ >> : Deletes entries from an <code>$archive</code> . >> : The format of <code>$entries</code> is the same as for <a >> href="#archive:create">archive:create</a> . >> : >> : @error bxerr:ARCH0005 the entries of the given archive cannot be >> modified. >> : @error bxerr:ARCH9999 archive creation failed for some other reason. >> :) >> declare function archive:delete($archive as xs:base64Binary, $entries as >> item()*) as xs:base64Binary external; >> >> Thanks again for the suggestion. > Best, > Bridger > > >> On 17 September 2018 at 16:43, Bridger Dyson-Smith <bdysonsmith at gmail.com >> > wrote: >> >>> Hi all - >>> >>> I was curious if there was a way to extract function signatures from >>> BaseX. I'm currently reading through the (excellent) wiki but was curious >>> if there was a more automated process I might be able to use. I've been >>> trying to think of something clever with function-lookup, but I can't seem >>> to make it dynamic based on module namespace; e.g. >>> >>> ``` >>> for $fun in function-lookup(xs:QName("archive:delete"), 1) >>> return >>> $fun >>> ``` >>> >>> I may just grab the wiki and parse it :), but I'm also interested in >>> changes between versions of the processor. >>> >>> Any thoughts or suggestions would be greatly appreciated! >>> Best, >>> Bridger >>> >> >>
BD
Bridger Dyson-Smith17 Sep 2018 19:38 #
Hi all -
I was curious if there was a way to extract function signatures from BaseX. I'm currently reading through the (excellent) wiki but was curious if there was a more automated process I might be able to use. I've been trying to think of something clever with function-lookup, but I can't seem to make it dynamic based on module namespace; e.g.
for $fun in function-lookup(xs:QName("archive:delete"), 1)
return
$funI may just grab the wiki and parse it :), but I'm also interested in changes between versions of the processor.
Any thoughts or suggestions would be greatly appreciated!
Best,
Bridger
BD
Andy -
thank you for that suggestion!
Quoted text · Andy Bunce · 19 lines
On Mon, Sep 17, 2018 at 12:13 PM Andy Bunce <bunce.andy at gmail.com> wrote: > If you look in lib/ folder the file modules.zip contains the BaseX modules > as .xqm stub files with functions marked as external. > The entry for archive:delete is shown below. At one point I am sure these > *.xqm could be parsed by functions like inspect:module and inspect:xqdoc > Currently they raise errors like [XQST0045] Function '...' is in reserved > namespace. > > I believe were generated by scraping the wiki :) > > However: > * some of these are a little out of date (at least in 9.02) > * modules.zip is not present in recent 9.1 betas > > This is a shame as I hoped to use to one day use these in some kind of IDE > for content assist. > (although I have hoped that for a long time without actually doing it.) > > This is exactly the use case I have. I'm attempting to help work on an
IntelliJ plugin that will leverage the signatures (for things I don't quite understand :).
Regards
Quoted text · Andy Bunce · 15 lines
> /Andy > > (:~ > : Deletes entries from an <code>$archive</code> . > : The format of <code>$entries</code> is the same as for <a > href="#archive:create">archive:create</a> . > : > : @error bxerr:ARCH0005 the entries of the given archive cannot be > modified. > : @error bxerr:ARCH9999 archive creation failed for some other reason. > :) > declare function archive:delete($archive as xs:base64Binary, $entries as > item()*) as xs:base64Binary external; > > Thanks again for the suggestion.
Best,
Bridger
Quoted text · Bridger Dyson-Smith · 26 lines
> On 17 September 2018 at 16:43, Bridger Dyson-Smith <bdysonsmith at gmail.com> > wrote: > >> Hi all - >> >> I was curious if there was a way to extract function signatures from >> BaseX. I'm currently reading through the (excellent) wiki but was curious >> if there was a more automated process I might be able to use. I've been >> trying to think of something clever with function-lookup, but I can't seem >> to make it dynamic based on module namespace; e.g. >> >> ``` >> for $fun in function-lookup(xs:QName("archive:delete"), 1) >> return >> $fun >> ``` >> >> I may just grab the wiki and parse it :), but I'm also interested in >> changes between versions of the processor. >> >> Any thoughts or suggestions would be greatly appreciated! >> Best, >> Bridger >> > >
CG
Christian Grün24 Sep 2018 18:31 #
Hi Bridger,
I’m excited to hear about your contributions to the IntelliJ XQuery plugin. This will be a good reason indeed for updating the stub files, which (as Andy has correctly guessed) are based on our Wiki pages.
It’s been a while ago since I looked at this, but I’ll be glad to create updated versions of the files. I’ve created a new issue for this step [1]. For a start, the files of the 9.0.2 release might do the job.
Cheers,
Christian
[1] https://github.com/BaseXdb/basex/issues/1623
Quoted text · Bridger Dyson-Smith · 59 lines
On Mon, Sep 17, 2018 at 6:23 PM Bridger Dyson-Smith <bdysonsmith at gmail.com> wrote: > > Andy - > thank you for that suggestion! > > On Mon, Sep 17, 2018 at 12:13 PM Andy Bunce <bunce.andy at gmail.com> wrote: >> >> If you look in lib/ folder the file modules.zip contains the BaseX modules as .xqm stub files with functions marked as external. >> The entry for archive:delete is shown below. At one point I am sure these *.xqm could be parsed by functions like inspect:module and inspect:xqdoc >> Currently they raise errors like [XQST0045] Function '...' is in reserved namespace. >> >> I believe were generated by scraping the wiki :) >> >> However: >> * some of these are a little out of date (at least in 9.02) >> * modules.zip is not present in recent 9.1 betas >> >> This is a shame as I hoped to use to one day use these in some kind of IDE for content assist. >> (although I have hoped that for a long time without actually doing it.) >> > This is exactly the use case I have. I'm attempting to help work on an IntelliJ plugin that will leverage the signatures (for things I don't quite understand :). > >> Regards >> /Andy >> >> (:~ >> : Deletes entries from an <code>$archive</code> . >> : The format of <code>$entries</code> is the same as for <a href="#archive:create">archive:create</a> . >> : >> : @error bxerr:ARCH0005 the entries of the given archive cannot be modified. >> : @error bxerr:ARCH9999 archive creation failed for some other reason. >> :) >> declare function archive:delete($archive as xs:base64Binary, $entries as item()*) as xs:base64Binary external; >> > Thanks again for the suggestion. > Best, > Bridger > >> >> On 17 September 2018 at 16:43, Bridger Dyson-Smith <bdysonsmith at gmail.com> wrote: >>> >>> Hi all - >>> >>> I was curious if there was a way to extract function signatures from BaseX. I'm currently reading through the (excellent) wiki but was curious if there was a more automated process I might be able to use. I've been trying to think of something clever with function-lookup, but I can't seem to make it dynamic based on module namespace; e.g. >>> >>> ``` >>> for $fun in function-lookup(xs:QName("archive:delete"), 1) >>> return >>> $fun >>> ``` >>> >>> I may just grab the wiki and parse it :), but I'm also interested in changes between versions of the processor. >>> >>> Any thoughts or suggestions would be greatly appreciated! >>> Best, >>> Bridger >> >>
CG
Hi Bridger,
I am glad to report I have created new stub files for the BaseX XQuery Modules [1]. They’ll now be included in the official releases again [2]. I have also uploaded the script that I wrote for generating the xqdoc output [3]. It’s far from perfect, but definitely more complete than the old version. If you encounter any errors, please don’t hesitate to tell me.
Could you give us a little update on your contribution to the IntelliJ XQuery plugin?
Cheers,
Christian
[1] https://github.com/BaseXdb/basex/issues/1623
[2] http://files.basex.org/releases/latest/
[3] https://github.com/BaseXdb/basex-dist/blob/master/wiki2xqdoc.xq
Quoted text · Bridger Dyson-Smith · 18 lines
On Mon, Sep 17, 2018 at 5:38 PM Bridger Dyson-Smith <bdysonsmith at gmail.com> wrote: > > Hi all - > > I was curious if there was a way to extract function signatures from BaseX. I'm currently reading through the (excellent) wiki but was curious if there was a more automated process I might be able to use. I've been trying to think of something clever with function-lookup, but I can't seem to make it dynamic based on module namespace; e.g. > > ``` > for $fun in function-lookup(xs:QName("archive:delete"), 1) > return > $fun > ``` > > I may just grab the wiki and parse it :), but I'm also interested in changes between versions of the processor. > > Any thoughts or suggestions would be greatly appreciated! > Best, > Bridger
BD
Bridger Dyson-Smith7 Nov 2018 03:27 #
Hi Christian -
thank you for your work on the xqdoc files (and the link to the xquery used for generating them)!
The current state of XQuery plugins for the JetBrains/IntelliJ IDEs has changed a bit since I added the IDEA documentation to the wiki. Grzegorz Ligas' XQuery Support plugin has been forked by the talented developers at OverStory, with lots of MarkLogic-specific features added. Reece Dunn also has been working on an XQuery plugin that takes a slightly different approach: rather than focusing on one implementation, Reece is working on a plugin that provides broader language support. The very minor work that I've done has been for Reece - I'm (slowly) adding implementation builtin function signatures that will be used by his plugin to provide improved static analysis.
I hope that provides a bit of information regarding news with XQuery/JetBrains plugins.
Best,
Bridger
Quoted text · Christian Grün · 47 lines
On Tue, Oct 30, 2018 at 9:13 AM Christian Grün <christian.gruen at gmail.com> wrote: > Hi Bridger, > > I am glad to report I have created new stub files for the BaseX XQuery > Modules [1]. They’ll now be included in the official releases again > [2]. I have also uploaded the script that I wrote for generating the > xqdoc output [3]. It’s far from perfect, but definitely more complete > than the old version. If you encounter any errors, please don’t > hesitate to tell me. > > Could you give us a little update on your contribution to the IntelliJ > XQuery plugin? > > Cheers, > Christian > > [1] https://github.com/BaseXdb/basex/issues/1623 > [2] http://files.basex.org/releases/latest/ > [3] https://github.com/BaseXdb/basex-dist/blob/master/wiki2xqdoc.xq > > > > On Mon, Sep 17, 2018 at 5:38 PM Bridger Dyson-Smith > <bdysonsmith at gmail.com> wrote: > > > > Hi all - > > > > I was curious if there was a way to extract function signatures from > BaseX. I'm currently reading through the (excellent) wiki but was curious > if there was a more automated process I might be able to use. I've been > trying to think of something clever with function-lookup, but I can't seem > to make it dynamic based on module namespace; e.g. > > > > ``` > > for $fun in function-lookup(xs:QName("archive:delete"), 1) > > return > > $fun > > ``` > > > > I may just grab the wiki and parse it :), but I'm also interested in > changes between versions of the processor. > > > > Any thoughts or suggestions would be greatly appreciated! > > Best, > > Bridger >
RD
Reece Dunn7 Nov 2018 08:11 #
Quoted text · Bridger Dyson-Smith · 15 lines
On Wed, 7 Nov 2018 at 01:27, Bridger Dyson-Smith <bdysonsmith at gmail.com> wrote: > Hi Christian - > > thank you for your work on the xqdoc files (and the link to the xquery > used for generating them)! > > The current state of XQuery plugins for the JetBrains/IntelliJ IDEs has > changed a bit since I added the IDEA documentation to the wiki. Grzegorz > Ligas' XQuery Support plugin has been forked by the talented developers at > OverStory, with lots of MarkLogic-specific features added. Reece Dunn also > has been working on an XQuery plugin that takes a slightly different > approach: rather than focusing on one implementation, Reece is working on a > plugin that provides broader language support. >
To add some background... My plugin started as an attempt to address various issues in the XQuery parser in Grzegorz' plugin. [1] was the point where I decided to attempt creating my own plugin, after reporting various issues to Grzegorz. Specifically, my plugin aimed to:
1. Address the keyword vs identifier issue in various places [2].
2. Provide a robust lexer and parser with error recovery.
3. Provide full support for the MarkLogic syntax in addition to XQuery 3 support -- this later evolved into adding support for other XQuery extensions (updating, full text, scripting), XQuery 3.1 support, and BaseX and Saxon vendor extension support.
Version 1.2 of my plugin has support for the BaseX update, fuzzy, and non-deterministic extensions. Version 1.3 (in development) adds support for the BaseX 9.1 ternary if, elvis operator, and if without else syntax extensions. My plan is to publish 1.3 when the IntelliJ 2018.3 release candidate is released, so the plugin is available for the full release.
My plugin does not currently support running queries, debugging, code reformatting, auto-completion, and various other features that Grzegorz' plugin supports. These are planned (I am currently working on running queries), but these things take time. I've also been improving function lookup in 1.3 to be standards conformant -- this is complex, and is not 100% complete. The technical challenges in implementing XQuery support in an IDE are different to those implementing it in a query processor as it needs to support the functionality and capabiilties of both the IDE and XQuery.
My long-term plans are to implement as many of the XPath and XQuery static errors from the error condition list as possible to spot all statically determinable errors in the IDE, as well as various other inspections.
I'm also considering adding specific support for the XPath subset of XQuery and integrating that with XSLT for better XPath support in IntelliJ.
Quoted text · Bridger Dyson-Smith · 4 lines
The very minor work that I've done has been for Reece - I'm (slowly) adding > implementation builtin function signatures that will be used by his plugin > to provide improved static analysis. >
My plans have always been to describe the built-in functions and static context as XQuery files. I completed this for the XQuery and MarkLogic functions, and Bridger has added BaseX, EXPath/EXQuery, and Saxon built-in function definitions. In the future, I want to add complete API descriptions using xqdoc comments so I can integrate them into the IDE when my plugin supports displaying the xqdoc information in place. I also want to use the information in the function annotations to report functions that require a different version of the XQuery processor.
[1] https://github.com/ligasgr/intellij-xquery/issues/199
[2] I have solved this by making the NCName and keyword tokens implement a common interface, then check for that interface when looking for NCNames. I then remove the keyword styling on keywords in NCName positions after the code has been parsed. I also use this to support the reserved function name functionality.
Kind regards,
Reece
I hope that provides a bit of information regarding news with
Quoted text · Bridger Dyson-Smith · 55 lines
> XQuery/JetBrains plugins. > Best, > Bridger > > > > On Tue, Oct 30, 2018 at 9:13 AM Christian Grün <christian.gruen at gmail.com> > wrote: > >> Hi Bridger, >> >> I am glad to report I have created new stub files for the BaseX XQuery >> Modules [1]. They’ll now be included in the official releases again >> [2]. I have also uploaded the script that I wrote for generating the >> xqdoc output [3]. It’s far from perfect, but definitely more complete >> than the old version. If you encounter any errors, please don’t >> hesitate to tell me. >> >> Could you give us a little update on your contribution to the IntelliJ >> XQuery plugin? >> >> Cheers, >> Christian >> >> [1] https://github.com/BaseXdb/basex/issues/1623 >> [2] http://files.basex.org/releases/latest/ >> [3] https://github.com/BaseXdb/basex-dist/blob/master/wiki2xqdoc.xq >> >> >> >> On Mon, Sep 17, 2018 at 5:38 PM Bridger Dyson-Smith >> <bdysonsmith at gmail.com> wrote: >> > >> > Hi all - >> > >> > I was curious if there was a way to extract function signatures from >> BaseX. I'm currently reading through the (excellent) wiki but was curious >> if there was a more automated process I might be able to use. I've been >> trying to think of something clever with function-lookup, but I can't seem >> to make it dynamic based on module namespace; e.g. >> > >> > ``` >> > for $fun in function-lookup(xs:QName("archive:delete"), 1) >> > return >> > $fun >> > ``` >> > >> > I may just grab the wiki and parse it :), but I'm also interested in >> changes between versions of the processor. >> > >> > Any thoughts or suggestions would be greatly appreciated! >> > Best, >> > Bridger >> >
Written for BaseX 9.0, 9.1. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery