Importing XML (DITA) validated with Relax NG schemas?

From: Robert Crews <rcrews at hortonworks.com> Date: Wednesday, September 12, 2018 at 12:59 PM To: "basex-talk at mailman.uni-konstanz.de" <basex-talk at mailman.uni-konstanz.de> Subject: Importing XML (DITA) validated with Relax NG schemas? I’m trying to set up BaseX to manage a repository of DITA content. The DITA content has been specialized to validate against Relax NG schemas instead of DTDs. While importing content that validate against DTDs is straight forward, I have not yet figured out the best way to import Relax NG based content. So, I have a few questions: * I see there is the Validation module [1] and that’s good because I think XQuery is my best option for managing the imports. However, I need to pass a specific schema to the functions. DITA content utilizes multiple schemas. Is there some way I can pass a directory to these functions instead of just a single file reference? * I currently have the CATFILE option set [2]. Does this help or hinder importing when choosing to validate against Relax NG? Is there some way I can set the global options to make validating against Relax NG easier? * I’ve seen multiple examples of people actually storing the schema(s) in the database. Is that necessary or help for using the validation module? Thanks for your help! [1] http://docs.basex.org/wiki/Validation_Module#RelaxNG_Validation [2] http://docs.basex.org/wiki/Catalog_Resolver http://docs.basex.org/wiki/Options#CATFILE For reference, here is how Oxygen handles catalog resolution<https://www.oxygenxml.com/doc/versions/20.1/ug-editor/topics/using-XML-Catalogs.html>. See the bottom of Table 1. Catalog Mappings for a summary of how Relax NG is resolved.

From: Jason Davis <jason.davis at hortonworks.com> Date: Monday, September 17, 2018 at 10:36 AM To: Robert Crews <rcrews at hortonworks.com>, "basex-talk at mailman.uni-konstanz.de" <basex-talk at mailman.uni-konstanz.de> Subject: Re: Importing XML (DITA) validated with Relax NG schemas? Robert sent this email to the list on my behalf last week. However, it’s not getting any traction. So, I am bumping it. While I think BaseX is great and shows a lot of promise, my implementation of it is currently dysfunctional and I’m stumped on how to get my project moving. There is very little in the way of documentation regarding managing XML that validates against Relax NG. I’ve made some progress in terms of using XQuery to handle the import (which appears to be the only way of validating against Relax NG) but I keep running into odd errors such as “Resource ‘…’ does not exist” when it in fact does exist. Can someone on the BaseX team please respond to this thread and provide a bit of insight? Thanks, Jason From: Robert Crews <rcrews at hortonworks.com> Date: Wednesday, September 12, 2018 at 12:59 PM To: "basex-talk at mailman.uni-konstanz.de" <basex-talk at mailman.uni-konstanz.de> Subject: Importing XML (DITA) validated with Relax NG schemas? I’m trying to set up BaseX to manage a repository of DITA content. The DITA content has been specialized to validate against Relax NG schemas instead of DTDs. While importing content that validate against DTDs is straight forward, I have not yet figured out the best way to import Relax NG based content. So, I have a few questions: * I see there is the Validation module [1] and that’s good because I think XQuery is my best option for managing the imports. However, I need to pass a specific schema to the functions. DITA content utilizes multiple schemas. Is there some way I can pass a directory to these functions instead of just a single file reference? * I currently have the CATFILE option set [2]. Does this help or hinder importing when choosing to validate against Relax NG? Is there some way I can set the global options to make validating against Relax NG easier? * I’ve seen multiple examples of people actually storing the schema(s) in the database. Is that necessary or help for using the validation module? Thanks for your help! [1] http://docs.basex.org/wiki/Validation_Module#RelaxNG_Validation [2] http://docs.basex.org/wiki/Catalog_Resolver http://docs.basex.org/wiki/Options#CATFILE For reference, here is how Oxygen handles catalog resolution<https://www.oxygenxml.com/doc/versions/20.1/ug-editor/topics/using-XML-Catalogs.html>. See the bottom of Table 1. Catalog Mappings for a summary of how Relax NG is resolved.

On Mon, Sep 17, 2018 at 9:55 PM Jason Davis <jason.davis at hortonworks.com> wrote: > > My mistake. It appears I may have misstated my problem. The query I used worked fine for importing resources. The problem appears to be related to how BaseX parses the resources. It doesn’t appear to take into account default attribute values. Given that the Relax NG spec clearly states default attributes are not part of the design, this doesn’t come across as a surprise. However, default attributes are meaningful with regard to DITA and there is a Java framework you can install as a plugin to the DITA Open Toolkit that supports parsing of Relax NG DITA content with default attributes [1]. > > Is it possible to setup databases created in BaseX with Relax NG parsing of default attributes? Is it just a matter of including the dita-ng.jar from the DITA OT plugin in my classpath? > > Thanks, > Jason > > [1] https://github.com/oxygenxml/dita-relaxng-defaults

On 9/25/18, 12:38 AM, "Christian Grün" <christian.gruen at gmail.com> wrote:

On Mon, Sep 17, 2018 at 9:55 PM Jason Davis <jason.davis at hortonworks.com> wrote: > > My mistake. It appears I may have misstated my problem. The query I used worked fine for importing resources. The problem appears to be related to how BaseX parses the resources. It doesn’t appear to take into account default attribute values. Given that the Relax NG spec clearly states default attributes are not part of the design, this doesn’t come across as a surprise. However, default attributes are meaningful with regard to DITA and there is a Java framework you can install as a plugin to the DITA Open Toolkit that supports parsing of Relax NG DITA content with default attributes [1]. > > Is it possible to setup databases created in BaseX with Relax NG parsing of default attributes? Is it just a matter of including the dita-ng.jar from the DITA OT plugin in my classpath? > > Thanks, > Jason > > [1] https://github.com/oxygenxml/dita-relaxng-defaults

> Hi Christian, > > Thanks for your response. Understood with regard to Jing. The plugin I linked > also uses Jing. I'm just not clear which version. I believe the source for at > least the dita-ng.jar is linked from the repository [1]. > > > With regard to the questions asked in this post, after investigation, I > believe I have a better understanding of the problem I'm trying to solve. > Importing Relzx NG XML into the database was not the problem. The problem > exists between the database and the application layer that interfaces with it. > I opened a feature request that summarizes it [2]. > > Thanks, > Jason > > [1] https://github.com/georgebina/dita-ng > [2] https://github.com/BaseXdb/basex/issues/1624 > > On 9/25/18, 12:38 AM, "Christian Grün" <christian.gr... at gmail.com> wrote: > > Hi Jason, > > Please note that BaseX uses the Jing RELAX NG validator. It does not > embed the DITA Open Toolkit. > > I didn’t have time to look at the DITA OT plugin so far. It seems that > the repository of the oXygen plugin you referenced only contains the > JAR file. Do you possibly know where we can find the source files? > > Cheers > Christian > > PS: It seems that you and your colleague asked different questions in > your previous posts. Have these questions been solved? > > > > On Mon, Sep 17, 2018 at 9:55 PM Jason Davis <jason.da... at hortonworks.com> > wrote: > > > > My mistake. It appears I may have misstated my problem. The query I used > worked fine for importing resources. The problem appears to be related to how > BaseX parses the resources. It doesn’t appear to take into account default > attribute values. Given that the Relax NG spec clearly states default > attributes are not part of the design, this doesn’t come across as a surprise. > However, default attributes are meaningful with regard to DITA and there is a > Java framework you can install as a plugin to the DITA Open Toolkit that > supports parsing of Relax NG DITA content with default attributes [1]. > > > > Is it possible to setup databases created in BaseX with Relax NG parsing > of default attributes? Is it just a matter of including the dita-ng.jar from > the DITA OT plugin in my classpath? > > > > Thanks, > > Jason > > > > [1] https://github.com/oxygenxml/dita-relaxng-defaults > > >

> "javax.xml.parsers.SAXParserFactory" factory? >

So in order to test if RNG-based XML parsing works with BaseX we would > somehow need a way to add the "dita-ng.jar" to the baseX server's class > path but it should also come in the classpath before the "xercesImpl.jar". >

On 10/1/2018 9:04 PM, Christian Grün wrote: > Hi Radu, > > How does baseX parse XML document, does it do so using the regular > "javax.xml.parsers.SAXParserFactory" factory? > > > Yes, it does [1]! > > So in order to test if RNG-based XML parsing works with BaseX we would > somehow need a way to add the "dita-ng.jar" to the baseX server's class > path but it should also come in the classpath before the > "xercesImpl.jar". > > > Currently, there is no particular order in which libraries are imported > [2], but if the chosen order is wrong, the dita-ng.jar could be added > manually to the start script. > > Hope this helps, > Christian > > [1] https://github.com/BaseXdb/basex/blob/master/basex-core/src/main/java/org/basex/io/parse/xml/XmlParser.java > [2] https://github.com/BaseXdb/basex/blob/master/basex-core/etc/basex > >

> I set the CP variable like so: > CP=$MAIN/BaseX.jar:$MAIN/lib/custom/dita-ng.jar:$MAIN/lib/*:$MAIN/lib/custom/*:$CLASSPATH > > This appears to be slightly different than the example you linked Christian. I’m using BaseX 9.0.2. Does this make a difference?

> I added an echo $CLASSPATH line under the CP variable. When I run the script, the echo statement is blank.

> Is there a way to see how the classpath is set when running this script?

On 10/3/18, 3:24 AM, "Christian Grün" <christian.gruen at gmail.com> wrote:

> I set the CP variable like so: > CP=$MAIN/BaseX.jar:$MAIN/lib/custom/dita-ng.jar:$MAIN/lib/*:$MAIN/lib/custom/*:$CLASSPATH > > This appears to be slightly different than the example you linked Christian. I’m using BaseX 9.0.2. Does this make a difference?

> I added an echo $CLASSPATH line under the CP variable. When I run the script, the echo statement is blank.

> Is there a way to see how the classpath is set when running this script?

> Hi Christian, Radu, > > I’ve tried adding the dita-ng.jar to the lib/custom dir of basex and then > manually modified the startup script to load it first. I can even confirm > that it’s the first jar on the path using: > > proc:property('java.class.path') > > However, the database still fails to parse the XML with default attributes > applied. I find myself having to cobble together an undesirable workaround > whereby I manually supply the default attribute values myself in order to > get my project to work with BaseX. Do you have any further suggestions for > how I might get this to work? > > Thanks, > Jason > > On 10/3/18, 3:24 AM, "Christian Grün" <christian.gruen at gmail.com> wrote: > > Hi Jason (cc to the list), > > > I set the CP variable like so: > > > CP=$MAIN/BaseX.jar:$MAIN/lib/custom/dita-ng.jar:$MAIN/lib/*:$MAIN/lib/custom/*:$CLASSPATH > > > > This appears to be slightly different than the example you linked > Christian. I’m using BaseX 9.0.2. Does this make a difference? > > The start scripts in the official distributions are created from the > GitHub examples I linked, so they are slightly different. > > > I added an echo $CLASSPATH line under the CP variable. When I run > the script, the echo statement is blank. > > In the script, no value will be bound to the $CLASSPATH variable. > Instead, you can assign values to this variable by yourself, which > will then be appended to the $CP variable. If you didn’t do so, and if > your Linux environment does not have any other values assigned to this > variable (which is the default), the output will necessarily be empty. > > > Is there a way to see how the classpath is set when running this > script? > > To answer the "how": It will be set via the line that you will find > some lines below in the script, and the -cp Java argument: > > exec java -cp "$CP" $BASEX_JVM org.basex.BaseX "$@" > > If you want to know which value is bound to $CP, try "echo $CP". In > Java, the full user class path at runtime will be bound to the > "java.class.path" system property. It can e.g. be retrieved via > proc:property('java.class.path') [1]. > > Christian > > [1] http://docs.basex.org/wiki/Process_Module#proc:property > > > >

> CATFILE=file:/D:/projects/eXml/frameworks/dita/DITA-OT2.x/catalog-dita.xml

On 10/18/2018 12:16 AM, Christian Grün wrote: > Hi Radu, > > Do you think the DITA parsing should work if dita-ng.jar is placed first > in the classpath? > > Cheers, > Christian > > > > > Jason Davis <jason.davis at hortonworks.com > <mailto:jason.davis at hortonworks.com>> schrieb am Mo., 15. Okt. 2018, 17:46: > > Hi Christian, Radu, > > I’ve tried adding the dita-ng.jar to the lib/custom dir of basex and > then manually modified the startup script to load it first. I can > even confirm that it’s the first jar on the path using: > > proc:property('java.class.path') > > However, the database still fails to parse the XML with default > attributes applied. I find myself having to cobble together an > undesirable workaround whereby I manually supply the default > attribute values myself in order to get my project to work with > BaseX. Do you have any further suggestions for how I might get this > to work? > > Thanks, > Jason > > On 10/3/18, 3:24 AM, "Christian Grün" <christian.gruen at gmail.com > <mailto:christian.gruen at gmail.com>> wrote: > > Hi Jason (cc to the list), > > > I set the CP variable like so: > > > CP=$MAIN/BaseX.jar:$MAIN/lib/custom/dita-ng.jar:$MAIN/lib/*:$MAIN/lib/custom/*:$CLASSPATH > > > > This appears to be slightly different than the example you > linked Christian. I’m using BaseX 9.0.2. Does this make a difference? > > The start scripts in the official distributions are created from the > GitHub examples I linked, so they are slightly different. > > > I added an echo $CLASSPATH line under the CP variable. When I > run the script, the echo statement is blank. > > In the script, no value will be bound to the $CLASSPATH variable. > Instead, you can assign values to this variable by yourself, which > will then be appended to the $CP variable. If you didn’t do so, > and if > your Linux environment does not have any other values assigned > to this > variable (which is the default), the output will necessarily be > empty. > > > Is there a way to see how the classpath is set when running > this script? > > To answer the "how": It will be set via the line that you will find > some lines below in the script, and the -cp Java argument: > > exec java -cp "$CP" $BASEX_JVM org.basex.BaseX "$@" > > If you want to know which value is bound to $CP, try "echo $CP". In > Java, the full user class path at runtime will be bound to the > "java.class.path" system property. It can e.g. be retrieved via > proc:property('java.class.path') [1]. > > Christian > > [1] http://docs.basex.org/wiki/Process_Module#proc:property > > >

> - I created a new database. > - Added an XML document to the database, the XML document containing a > reference to the RNG schema.

> Will I find that extra logging in the console I used to start > "basexhttp.bat"? Or do I need to first configure the logging?

On 10/19/2018 7:26 PM, Christian Grün wrote: > Hi Radu, > > Thanks for trying to get this working. > >> - I created a new database. >> - Added an XML document to the database, the XML document containing a >> reference to the RNG schema. > > Just a guess: You may need to supply the catalog before adding the > document to the database. In the GUI, if you open the Database → New… > dialog, you can specify the file via the parser dialog. Adding the > path in the .basex file should work as well (it doesn’t matter if you > specify a local file path or a file:// URI). > >> Will I find that extra logging in the console I used to start >> "basexhttp.bat"? Or do I need to first configure the logging? > > If you start basex instances with the -d flag, you’ll get some more feedback. > > Hope this helps, further questions are welcome, > Christian >