BaseXdb/basex] Packaging: Java and XQuery code (#1413)

> but I wanted to be sure I was going about it correctly because it still > seems a little like magic to me. I am also wondering whether this new > feature is something I might want to take advantage of.

> The module (https://github.com/timathom/basex-rdf) uses Günther Rademacher's > REx Parser Generator[1] to create a Java parser for the RDF TriG/Turtle > syntax[2], using the BaseX extension function option. > > Following the BaseX documentation[3], I have created a XAR package that > includes the JAR file and an XQuery wrapper. The wrapper calls the function > exposed by the Java module (import module namespace trig = > "org.basex.modules.rdf.TriG";) and then processes the raw parse tree, > overlaying some additional abstractions (I confess that I cheated and used > XSLT for most of the processing functionality). > > Because of the built-in BaseX extension function, this approach seems > different from a straightforward "combined" module approach, but, again, I > wanted to ask whether the way I'm doing this seems correct. > > Thanks in advance. > > Tim > > [1] http://www.bottlecaps.de/rex/ > [2] https://www.w3.org/TR/trig/ > [3] docs.basex.org/wiki/Repository > > -- > Tim A. Thompson > Discovery Metadata Librarian > Yale University Library >

On Fri, Oct 20, 2017 at 2:25 PM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Tim, > > Thanks for the link to your RDF library. > > I’m glad to hear that the new packaging mechanism has already found > watchers. I have just uploaded a new snapshot, and I would be glad to > have your feedback as first tester ;) > > > but I wanted to be sure I was going about it correctly because it still > > seems a little like magic to me. I am also wondering whether this new > > feature is something I might want to take advantage of. > > I haven’t run any queries yet with your module, but the package > structure looks fine. > > Our experience over the years is that the EXPath packaging mechanism > is a bit difficult to understand, and not as portable as we believe it > could be. It would be nice, for example, if a package could be > distributed without implementation-specific details inside. But it’s > always easy to complain, and it takes a lot of time to get things > going. > > For our own use cases, the simple packaging mechanism is just good > enough. It would be nice if we could add some versioning support in > the future, though. > > Cheers > Christian > > > > > The module (https://github.com/timathom/basex-rdf) uses Günther > Rademacher's > > REx Parser Generator[1] to create a Java parser for the RDF TriG/Turtle > > syntax[2], using the BaseX extension function option. > > > > Following the BaseX documentation[3], I have created a XAR package that > > includes the JAR file and an XQuery wrapper. The wrapper calls the > function > > exposed by the Java module (import module namespace trig = > > "org.basex.modules.rdf.TriG";) and then processes the raw parse tree, > > overlaying some additional abstractions (I confess that I cheated and > used > > XSLT for most of the processing functionality). > > > > Because of the built-in BaseX extension function, this approach seems > > different from a straightforward "combined" module approach, but, again, > I > > wanted to ask whether the way I'm doing this seems correct. > > > > Thanks in advance. > > > > Tim > > > > [1] http://www.bottlecaps.de/rex/ > > [2] https://www.w3.org/TR/trig/ > > [3] docs.basex.org/wiki/Repository > > > > -- > > Tim A. Thompson > > Discovery Metadata Librarian > > Yale University Library > > >

On Tue, Oct 24, 2017 at 4:16 PM, Tim Thompson <timathom at gmail.com> wrote: > Thanks, Christian. I had a quick question about the expath-pkg.xml file > (attached). I have two XSLT files that I want to list separately, along with > listing Saxon-HE (version 9.8.x) as a dependency. However, when I add a > second dependency element and try to import the package, I get an error: > > [bxerr:BXRE0009] Package version is not supported. > > Looking at the EXPath package schema[1], it seems as though this should be > valid (the dependency element should be repeatable). > > Tim > > [1] http://expath.org/spec/pkg#structure > > -- > Tim A. Thompson > Discovery Metadata Librarian > Yale University Library > > > On Fri, Oct 20, 2017 at 2:25 PM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> Hi Tim, >> >> Thanks for the link to your RDF library. >> >> I’m glad to hear that the new packaging mechanism has already found >> watchers. I have just uploaded a new snapshot, and I would be glad to >> have your feedback as first tester ;) >> >> > but I wanted to be sure I was going about it correctly because it still >> > seems a little like magic to me. I am also wondering whether this new >> > feature is something I might want to take advantage of. >> >> I haven’t run any queries yet with your module, but the package >> structure looks fine. >> >> Our experience over the years is that the EXPath packaging mechanism >> is a bit difficult to understand, and not as portable as we believe it >> could be. It would be nice, for example, if a package could be >> distributed without implementation-specific details inside. But it’s >> always easy to complain, and it takes a lot of time to get things >> going. >> >> For our own use cases, the simple packaging mechanism is just good >> enough. It would be nice if we could add some versioning support in >> the future, though. >> >> Cheers >> Christian >> >> >> >> > The module (https://github.com/timathom/basex-rdf) uses Günther >> > Rademacher's >> > REx Parser Generator[1] to create a Java parser for the RDF TriG/Turtle >> > syntax[2], using the BaseX extension function option. >> > >> > Following the BaseX documentation[3], I have created a XAR package that >> > includes the JAR file and an XQuery wrapper. The wrapper calls the >> > function >> > exposed by the Java module (import module namespace trig = >> > "org.basex.modules.rdf.TriG";) and then processes the raw parse tree, >> > overlaying some additional abstractions (I confess that I cheated and >> > used >> > XSLT for most of the processing functionality). >> > >> > Because of the built-in BaseX extension function, this approach seems >> > different from a straightforward "combined" module approach, but, again, >> > I >> > wanted to ask whether the way I'm doing this seems correct. >> > >> > Thanks in advance. >> > >> > Tim >> > >> > [1] http://www.bottlecaps.de/rex/ >> > [2] https://www.w3.org/TR/trig/ >> > [3] docs.basex.org/wiki/Repository >> > >> > -- >> > Tim A. Thompson >> > Discovery Metadata Librarian >> > Yale University Library >> > > >

On 24 October 2017 at 15:16, Tim Thompson <timathom at gmail.com> wrote: > Thanks, Christian. I had a quick question about the expath-pkg.xml file > (attached). I have two XSLT files that I want to list separately, along > with listing Saxon-HE (version 9.8.x) as a dependency. However, when I add > a second dependency element and try to import the package, I get an error: > > [bxerr:BXRE0009] Package version is not supported. > > Looking at the EXPath package schema[1], it seems as though this should be > valid (the dependency element should be repeatable). > > Tim > > [1] http://expath.org/spec/pkg#structure > > -- > Tim A. Thompson > Discovery Metadata Librarian > Yale University Library > > > On Fri, Oct 20, 2017 at 2:25 PM, Christian Grün <christian.gruen at gmail.com > > wrote: > >> Hi Tim, >> >> Thanks for the link to your RDF library. >> >> I’m glad to hear that the new packaging mechanism has already found >> watchers. I have just uploaded a new snapshot, and I would be glad to >> have your feedback as first tester ;) >> >> > but I wanted to be sure I was going about it correctly because it still >> > seems a little like magic to me. I am also wondering whether this new >> > feature is something I might want to take advantage of. >> >> I haven’t run any queries yet with your module, but the package >> structure looks fine. >> >> Our experience over the years is that the EXPath packaging mechanism >> is a bit difficult to understand, and not as portable as we believe it >> could be. It would be nice, for example, if a package could be >> distributed without implementation-specific details inside. But it’s >> always easy to complain, and it takes a lot of time to get things >> going. >> >> For our own use cases, the simple packaging mechanism is just good >> enough. It would be nice if we could add some versioning support in >> the future, though. >> >> Cheers >> Christian >> >> >> >> > The module (https://github.com/timathom/basex-rdf) uses Günther >> Rademacher's >> > REx Parser Generator[1] to create a Java parser for the RDF TriG/Turtle >> > syntax[2], using the BaseX extension function option. >> > >> > Following the BaseX documentation[3], I have created a XAR package that >> > includes the JAR file and an XQuery wrapper. The wrapper calls the >> function >> > exposed by the Java module (import module namespace trig = >> > "org.basex.modules.rdf.TriG";) and then processes the raw parse tree, >> > overlaying some additional abstractions (I confess that I cheated and >> used >> > XSLT for most of the processing functionality). >> > >> > Because of the built-in BaseX extension function, this approach seems >> > different from a straightforward "combined" module approach, but, >> again, I >> > wanted to ask whether the way I'm doing this seems correct. >> > >> > Thanks in advance. >> > >> > Tim >> > >> > [1] http://www.bottlecaps.de/rex/ >> > [2] https://www.w3.org/TR/trig/ >> > [3] docs.basex.org/wiki/Repository >> > >> > -- >> > Tim A. Thompson >> > Discovery Metadata Librarian >> > Yale University Library >> > >> > >

On Tue, Oct 24, 2017 at 12:36 PM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Tim, > > I’ll have a look soon, probably tomorrow. Out of interest: Have you > been successful using our combined approach? > > Best, > Christian > > > > On Tue, Oct 24, 2017 at 4:16 PM, Tim Thompson <timathom at gmail.com> wrote: > > Thanks, Christian. I had a quick question about the expath-pkg.xml file > > (attached). I have two XSLT files that I want to list separately, along > with > > listing Saxon-HE (version 9.8.x) as a dependency. However, when I add a > > second dependency element and try to import the package, I get an error: > > > > [bxerr:BXRE0009] Package version is not supported. > > > > Looking at the EXPath package schema[1], it seems as though this should > be > > valid (the dependency element should be repeatable). > > > > Tim > > > > [1] http://expath.org/spec/pkg#structure > > > > -- > > Tim A. Thompson > > Discovery Metadata Librarian > > Yale University Library > > > > > > On Fri, Oct 20, 2017 at 2:25 PM, Christian Grün < > christian.gruen at gmail.com> > > wrote: > >> > >> Hi Tim, > >> > >> Thanks for the link to your RDF library. > >> > >> I’m glad to hear that the new packaging mechanism has already found > >> watchers. I have just uploaded a new snapshot, and I would be glad to > >> have your feedback as first tester ;) > >> > >> > but I wanted to be sure I was going about it correctly because it > still > >> > seems a little like magic to me. I am also wondering whether this new > >> > feature is something I might want to take advantage of. > >> > >> I haven’t run any queries yet with your module, but the package > >> structure looks fine. > >> > >> Our experience over the years is that the EXPath packaging mechanism > >> is a bit difficult to understand, and not as portable as we believe it > >> could be. It would be nice, for example, if a package could be > >> distributed without implementation-specific details inside. But it’s > >> always easy to complain, and it takes a lot of time to get things > >> going. > >> > >> For our own use cases, the simple packaging mechanism is just good > >> enough. It would be nice if we could add some versioning support in > >> the future, though. > >> > >> Cheers > >> Christian > >> > >> > >> > >> > The module (https://github.com/timathom/basex-rdf) uses Günther > >> > Rademacher's > >> > REx Parser Generator[1] to create a Java parser for the RDF > TriG/Turtle > >> > syntax[2], using the BaseX extension function option. > >> > > >> > Following the BaseX documentation[3], I have created a XAR package > that > >> > includes the JAR file and an XQuery wrapper. The wrapper calls the > >> > function > >> > exposed by the Java module (import module namespace trig = > >> > "org.basex.modules.rdf.TriG";) and then processes the raw parse tree, > >> > overlaying some additional abstractions (I confess that I cheated and > >> > used > >> > XSLT for most of the processing functionality). > >> > > >> > Because of the built-in BaseX extension function, this approach seems > >> > different from a straightforward "combined" module approach, but, > again, > >> > I > >> > wanted to ask whether the way I'm doing this seems correct. > >> > > >> > Thanks in advance. > >> > > >> > Tim > >> > > >> > [1] http://www.bottlecaps.de/rex/ > >> > [2] https://www.w3.org/TR/trig/ > >> > [3] docs.basex.org/wiki/Repository > >> > > >> > -- > >> > Tim A. Thompson > >> > Discovery Metadata Librarian > >> > Yale University Library > >> > > > > > >

> When I try to use the BaseX fetch:text function > to retrieve an RDF text file and pass it to my combined module parser > function, I get an error (in the the latest BaseX snapshot: beta a48337f): > Function parseTrigDoc(xs:string) cannot be called with (xs:string).

On Tue, Oct 24, 2017 at 5:25 PM, Tim Thompson <timathom at gmail.com> wrote: > Hi, Christian, > > I had a chance to try out the combined module feature this afternoon. > Copies of the combined JAR file and related files are here: > https://github.com/timathom/basex-rdf. > > I was able to create a basic module, but I did run into a few hiccups > along the way: > > 1. From my XQuery module, I want to call a pair of XSLT stylesheets, > so I tried adding these to the JAR as well. However, when I import the > combined JAR file, the XSLT files are not copied over. If I copy them > manually into the BaseXRepo folder, then it works. > 2. In my previous experiments with the EXPath repo approach, I was > able to import the Java module while also defining a separate namespace for > the XQuery wrapper module itself. Thus, I was able to define additional > XQuery functions in the namespace of the wrapper module. With the new > combined approach, however, it seems that each XQuery function must > correspond to a Java function, and the namespaces must match (e.g., " > http://basex.org/modules/rdf/Triples > <http://basex.org/modules/rdf/Triples>" and "java: > org.basex.modules.rdf.Triples"). I had to remove my additional XQuery > functions from the wrapper because BaseX was throwing an error that the > functions were not Java functions. > > Just let me know if I can provide any other feedback that would be helpful. > > Best, > Tim > > > -- > Tim A. Thompson > Discovery Metadata Librarian > Yale University Library > > > On Tue, Oct 24, 2017 at 12:36 PM, Christian Grün < > christian.gruen at gmail.com> wrote: > >> Hi Tim, >> >> I’ll have a look soon, probably tomorrow. Out of interest: Have you >> been successful using our combined approach? >> >> Best, >> Christian >> >> >> >> On Tue, Oct 24, 2017 at 4:16 PM, Tim Thompson <timathom at gmail.com> wrote: >> > Thanks, Christian. I had a quick question about the expath-pkg.xml file >> > (attached). I have two XSLT files that I want to list separately, along >> with >> > listing Saxon-HE (version 9.8.x) as a dependency. However, when I add a >> > second dependency element and try to import the package, I get an error: >> > >> > [bxerr:BXRE0009] Package version is not supported. >> > >> > Looking at the EXPath package schema[1], it seems as though this should >> be >> > valid (the dependency element should be repeatable). >> > >> > Tim >> > >> > [1] http://expath.org/spec/pkg#structure >> > >> > -- >> > Tim A. Thompson >> > Discovery Metadata Librarian >> > Yale University Library >> > >> > >> > On Fri, Oct 20, 2017 at 2:25 PM, Christian Grün < >> christian.gruen at gmail.com> >> > wrote: >> >> >> >> Hi Tim, >> >> >> >> Thanks for the link to your RDF library. >> >> >> >> I’m glad to hear that the new packaging mechanism has already found >> >> watchers. I have just uploaded a new snapshot, and I would be glad to >> >> have your feedback as first tester ;) >> >> >> >> > but I wanted to be sure I was going about it correctly because it >> still >> >> > seems a little like magic to me. I am also wondering whether this new >> >> > feature is something I might want to take advantage of. >> >> >> >> I haven’t run any queries yet with your module, but the package >> >> structure looks fine. >> >> >> >> Our experience over the years is that the EXPath packaging mechanism >> >> is a bit difficult to understand, and not as portable as we believe it >> >> could be. It would be nice, for example, if a package could be >> >> distributed without implementation-specific details inside. But it’s >> >> always easy to complain, and it takes a lot of time to get things >> >> going. >> >> >> >> For our own use cases, the simple packaging mechanism is just good >> >> enough. It would be nice if we could add some versioning support in >> >> the future, though. >> >> >> >> Cheers >> >> Christian >> >> >> >> >> >> >> >> > The module (https://github.com/timathom/basex-rdf) uses Günther >> >> > Rademacher's >> >> > REx Parser Generator[1] to create a Java parser for the RDF >> TriG/Turtle >> >> > syntax[2], using the BaseX extension function option. >> >> > >> >> > Following the BaseX documentation[3], I have created a XAR package >> that >> >> > includes the JAR file and an XQuery wrapper. The wrapper calls the >> >> > function >> >> > exposed by the Java module (import module namespace trig = >> >> > "org.basex.modules.rdf.TriG";) and then processes the raw parse tree, >> >> > overlaying some additional abstractions (I confess that I cheated and >> >> > used >> >> > XSLT for most of the processing functionality). >> >> > >> >> > Because of the built-in BaseX extension function, this approach seems >> >> > different from a straightforward "combined" module approach, but, >> again, >> >> > I >> >> > wanted to ask whether the way I'm doing this seems correct. >> >> > >> >> > Thanks in advance. >> >> > >> >> > Tim >> >> > >> >> > [1] http://www.bottlecaps.de/rex/ >> >> > [2] https://www.w3.org/TR/trig/ >> >> > [3] docs.basex.org/wiki/Repository >> >> > >> >> > -- >> >> > Tim A. Thompson >> >> > Discovery Metadata Librarian >> >> > Yale University Library >> >> > >> > >> > >> > >

On Wed, Oct 25, 2017 at 11:25 AM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Tim, > > Thanks for the hint, and the intent to create a combined package for > your code (our approach seems to be too limited indeed). > > Once again my day was shorter than I expected, so I’ll do my best to > check out your project tomorrow. > > > When I try to use the BaseX fetch:text function > > to retrieve an RDF text file and pass it to my combined module parser > > function, I get an error (in the the latest BaseX snapshot: beta > a48337f): > > Function parseTrigDoc(xs:string) cannot be called with (xs:string). > > My spontaneous guess is that fetch:text returns a “streamable” string > (its contents will only be generated if they are requested), and our > Java conversion mapping seem not be prepared to handle these types of > strings. Actually they should be, so I’ll check how this can be fixed. > In the meanwhile, you should be able to work around the restriction by > explicitly requesting the string via > stream:materialize(fetch:text(...)). > > Hope this helps, > Christian > > [1] http://docs.basex.org/wiki/Fetch_Module#fetch:text > [2] http://docs.basex.org/wiki/Streaming_Module >

On Thu, Oct 26, 2017 at 1:16 AM, Tim Thompson <timathom at gmail.com> wrote: > Thanks, I did try stream:materialize, but that didn't seem to help. > Regarding the "limitation" of the new combined approach, I think that may > actually be a plus. In my case, it forced me to make my code a bit more > modular. > > Tim > > -- > Tim A. Thompson > Discovery Metadata Librarian > Yale University Library > > On Wed, Oct 25, 2017 at 11:25 AM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> Hi Tim, >> >> Thanks for the hint, and the intent to create a combined package for >> your code (our approach seems to be too limited indeed). >> >> Once again my day was shorter than I expected, so I’ll do my best to >> check out your project tomorrow. >> >> > When I try to use the BaseX fetch:text function >> > to retrieve an RDF text file and pass it to my combined module parser >> > function, I get an error (in the the latest BaseX snapshot: beta >> > a48337f): >> > Function parseTrigDoc(xs:string) cannot be called with (xs:string). >> >> My spontaneous guess is that fetch:text returns a “streamable” string >> (its contents will only be generated if they are requested), and our >> Java conversion mapping seem not be prepared to handle these types of >> strings. Actually they should be, so I’ll check how this can be fixed. >> In the meanwhile, you should be able to work around the restriction by >> explicitly requesting the string via >> stream:materialize(fetch:text(...)). >> >> Hope this helps, >> Christian >> >> [1] http://docs.basex.org/wiki/Fetch_Module#fetch:text >> [2] http://docs.basex.org/wiki/Streaming_Module > >