11.9 doc() not working with DOCTYPE declaration

Von: "Graydon Saunders" <graydonish at fastmail.com> An: BaseX <BaseX-Talk at mailman.uni-konstanz.de> Betreff: [basex-talk] 11.9 doc() not working with DOCTYPE declaration Hello -- I'm on linux, using 11.9 and using 20:59 graydon % java -version openjdk version "21.0.7" 2025-04-15 OpenJDK Runtime Environment (Red_Hat-21.0.7.0.6-1) (build 21.0.7+6) OpenJDK 64-Bit Server VM (Red_Hat-21.0.7.0.6-1) (build 21.0.7+6, mixed mode, sharing) If I use the basexgui "Create DB" dialog and the internal parser, I can load the whole folder without error; I get the expected DB and the named entities in the content have been resolved. (These are effectively HTML 4 entity names, so that's not completely guaranteeing that the DTD in the DOCTYPE is being found and parsed somewhere.) If I use the doc() function with an absolute file system path to one of the same files that loaded into the DB just fine, I get an error. I get the same error if I try to switch off the internal parser and set a catalog. It's at least several files in the folder that loads fine as a DB. All of this works fine with 11.8; whatever changed changed in 11.9. The error is: Could not execute XQUERY doc('/home/graydon/GeneralCode/git/cpc-conversion/Chapter/BU4372-14A.xml'): Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk at mailman.uni-konstanz.de Version: BaseX 11.9 Java: Red Hat, Inc., 21.0.7 OS: Linux, amd64 Stack Trace: javax.xml.catalog.CatalogException: JAXP09040002: No match found for href '/home/graydon/GeneralCode/git/cpc-conversion/Chapter/BU4372-14A.xml' and base '/home/graydon/GeneralCode/git/cpc-conversion/xquery/file'. at java.xml/javax.xml.catalog.CatalogMessages.reportError(CatalogMessages.java:74) at java.xml/javax.xml.catalog.CatalogResolverImpl.resolve(CatalogResolverImpl.java:153) at org.basex.query.StaticContext.resolve(StaticContext.java:156) at org.basex.query.StaticContext.resolve(StaticContext.java:137) at org.basex.query.QueryInput.<init>(QueryInput.java:31) at org.basex.query.func.fn.Docs.queryInput(Docs.java:37) at org.basex.query.func.fn.Docs.lambda$accept$0(Docs.java:67) at org.basex.query.scope.LockVisitor.lock(LockVisitor.java:48) at org.basex.query.func.fn.Docs.accept(Docs.java:56) at org.basex.query.scope.MainModule.databases(MainModule.java:93) at org.basex.query.QueryContext.addLocks(QueryContext.java:412) at org.basex.query.QueryProcessor.addLocks(QueryProcessor.java:230) at org.basex.core.cmd.AQuery.addLocks(AQuery.java:170) at org.basex.core.locks.Locking.acquire(Locking.java:86) at org.basex.core.jobs.Job.register(Job.java:48) at org.basex.core.Command.execute(Command.java:90) at org.basex.gui.GUI.execute(GUI.java:430) at org.basex.gui.GUI.lambda$execute$5(GUI.java:375) at java.base/java.lang.Thread.run(Thread.java:1583) I can go on using 11.8 for now, so this is only somewhat alarming rather than completely alarming. -- Graydon

On Sun, May 11, 2025, at 06:01, Gunther Rademacher wrote: > Hi Graydon, > > thank you for reporting this. I was able to reproduce the behaviour that you are seeing - not only with BaseX 11.9 and Java 21, but also with BaseX 11.8 and Java 11. > > The root cause appears to be the standard JDK catalog resolver, javax.xml.catalog.CatalogResolverImpl, which tends to throw a CatalogException when a URI cannot be resolved via the catalog. Unfortunately, BaseX does not currently handle this exception type. It expects a TransformerException, which is why the process fails in your case. > > In my own environment, I’ve never encountered this issue because I always have XML Resolver, i.e. org.xmlresolver.Resolver, on the classpath. This resolver replaces the standard implementation and simply returns null when a match isn’t found, rather than throwing an exception. > > I’ll open a GitHub issue to ensure that BaseX is updated to handle CatalogException properly. In the meantime, a practical workaround is to add XML Resolver to your classpath. I suspect you may already have it in place in the environment where you tested BaseX 11.8, which could explain why the issue didn’t surface there. > > Also, as you mentioned, fn:doc was recently updated. You can find more about those changes in this discussion: > https://www.mail-archive.com/basex-talk at mailman.uni-konstanz.de/msg16163.html > > However, I don’t believe those updates are directly related to the issue you're seeing. > > Best regards, > Gunther > > *Gesendet: *Sonntag, 11. Mai 2025 um 03:16 > *Von: *"Graydon Saunders" <graydonish at fastmail.com> > *An: *BaseX <BaseX-Talk at mailman.uni-konstanz.de> > *Betreff: *[basex-talk] 11.9 doc() not working with DOCTYPE declaration > Hello -- > > I'm on linux, using 11.9 and using > > 20:59 graydon % java -version > openjdk version "21.0.7" 2025-04-15 > OpenJDK Runtime Environment (Red_Hat-21.0.7.0.6-1) (build 21.0.7+6) > OpenJDK 64-Bit Server VM (Red_Hat-21.0.7.0.6-1) (build 21.0.7+6, mixed mode, sharing) > > If I use the basexgui "Create DB" dialog and the internal parser, I can load the whole folder without error; I get the expected DB and the named entities in the content have been resolved. (These are effectively HTML 4 entity names, so that's not completely guaranteeing that the DTD in the DOCTYPE is being found and parsed somewhere.) > > If I use the doc() function with an absolute file system path to one of the same files that loaded into the DB just fine, I get an error. I get the same error if I try to switch off the internal parser and set a catalog. It's at least several files in the folder that loads fine as a DB. > > All of this works fine with 11.8; whatever changed changed in 11.9. > > The error is: > > Could not execute XQUERY doc('/home/graydon/GeneralCode/git/cpc-conversion/Chapter/BU4372-14A.xml'): > Improper use? Potential bug? Your feedback is welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 11.9 > Java: Red Hat, Inc., 21.0.7 > OS: Linux, amd64 > Stack Trace: > javax.xml.catalog.CatalogException: JAXP09040002: No match found for href '/home/graydon/GeneralCode/git/cpc-conversion/Chapter/BU4372-14A.xml' and base '/home/graydon/GeneralCode/git/cpc-conversion/xquery/file'. > at java.xml/javax.xml.catalog.CatalogMessages.reportError(CatalogMessages.java:74) > at java.xml/javax.xml.catalog.CatalogResolverImpl.resolve(CatalogResolverImpl.java:153) > at org.basex.query.StaticContext.resolve(StaticContext.java:156) > at org.basex.query.StaticContext.resolve(StaticContext.java:137) > at org.basex.query.QueryInput.<init>(QueryInput.java:31) > at org.basex.query.func.fn.Docs.queryInput(Docs.java:37) > at org.basex.query.func.fn.Docs.lambda$accept$0(Docs.java:67) > at org.basex.query.scope.LockVisitor.lock(LockVisitor.java:48) > at org.basex.query.func.fn.Docs.accept(Docs.java:56) > at org.basex.query.scope.MainModule.databases(MainModule.java:93) > at org.basex.query.QueryContext.addLocks(QueryContext.java:412) > at org.basex.query.QueryProcessor.addLocks(QueryProcessor.java:230) > at org.basex.core.cmd.AQuery.addLocks(AQuery.java:170) > at org.basex.core.locks.Locking.acquire(Locking.java:86) > at org.basex.core.jobs.Job.register(Job.java:48) > at org.basex.core.Command.execute(Command.java:90) > at org.basex.gui.GUI.execute(GUI.java:430) > at org.basex.gui.GUI.lambda$execute$5(GUI.java:375) > at java.base/java.lang.Thread.run(Thread.java:1583) > > I can go on using 11.8 for now, so this is only somewhat alarming rather than completely alarming. > > -- Graydon

On Sun, May 11, 2025 at 5:13 PM Graydon Saunders <graydonish at fastmail.com> wrote: > Hi Gunther, > > Thank you! > > Dropping xmlresolver-6.0.14-data.jar and xmlresolver-6.0.14.jar from the > zip release via https://github.com/xmlresolver/xmlresolver/releases into > basex/lib/custom works. The document parses and the entities are resolved; > I have the catalog set via "local options" in both .basex and .basexgui, so > I'm going to think this is expected. And I can confirm that I don't need to > explicitly set any options in doc() for things to work. > > I'm sure this is a much more complex issue than it seems to me, but > perhaps it would be a useful thing to ship the open resolver with BaseX? > > Much appreciated, > Graydon > > On Sun, May 11, 2025, at 06:01, Gunther Rademacher wrote: > > Hi Graydon, > > thank you for reporting this. I was able to reproduce the behaviour that > you are seeing - not only with BaseX 11.9 and Java 21, but also with BaseX > 11.8 and Java 11. > > The root cause appears to be the standard JDK catalog resolver, > javax.xml.catalog.CatalogResolverImpl, which tends to throw a > CatalogException when a URI cannot be resolved via the catalog. > Unfortunately, BaseX does not currently handle this exception type. It > expects a TransformerException, which is why the process fails in your case. > > In my own environment, I’ve never encountered this issue because I always > have XML Resolver, i.e. org.xmlresolver.Resolver, on the classpath. This > resolver replaces the standard implementation and simply returns null when > a match isn’t found, rather than throwing an exception. > > I’ll open a GitHub issue to ensure that BaseX is updated to handle > CatalogException properly. In the meantime, a practical workaround is to > add XML Resolver to your classpath. I suspect you may already have it in > place in the environment where you tested BaseX 11.8, which could explain > why the issue didn’t surface there. > > Also, as you mentioned, fn:doc was recently updated. You can find more > about those changes in this discussion: > > https://www.mail-archive.com/basex-talk at mailman.uni-konstanz.de/msg16163.html > > However, I don’t believe those updates are directly related to the issue > you're seeing. > > Best regards, > Gunther > > *Gesendet: *Sonntag, 11. Mai 2025 um 03:16 > *Von: *"Graydon Saunders" <graydonish at fastmail.com> > *An: *BaseX <BaseX-Talk at mailman.uni-konstanz.de> > *Betreff: *[basex-talk] 11.9 doc() not working with DOCTYPE declaration > Hello -- > > I'm on linux, using 11.9 and using > > 20:59 graydon % java -version > openjdk version "21.0.7" 2025-04-15 > OpenJDK Runtime Environment (Red_Hat-21.0.7.0.6-1) (build 21.0.7+6) > OpenJDK 64-Bit Server VM (Red_Hat-21.0.7.0.6-1) (build 21.0.7+6, mixed > mode, sharing) > > If I use the basexgui "Create DB" dialog and the internal parser, I can > load the whole folder without error; I get the expected DB and the named > entities in the content have been resolved. (These are effectively HTML 4 > entity names, so that's not completely guaranteeing that the DTD in the > DOCTYPE is being found and parsed somewhere.) > > If I use the doc() function with an absolute file system path to one of > the same files that loaded into the DB just fine, I get an error. I get the > same error if I try to switch off the internal parser and set a catalog. > It's at least several files in the folder that loads fine as a DB. > > All of this works fine with 11.8; whatever changed changed in 11.9. > > The error is: > > Could not execute XQUERY > doc('/home/graydon/GeneralCode/git/cpc-conversion/Chapter/BU4372-14A.xml'): > Improper use? Potential bug? Your feedback is welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 11.9 > Java: Red Hat, Inc., 21.0.7 > OS: Linux, amd64 > Stack Trace: > javax.xml.catalog.CatalogException: JAXP09040002: No match found for href > '/home/graydon/GeneralCode/git/cpc-conversion/Chapter/BU4372-14A.xml' and > base '/home/graydon/GeneralCode/git/cpc-conversion/xquery/file'. > at > java.xml/javax.xml.catalog.CatalogMessages.reportError(CatalogMessages.java:74) > at > java.xml/javax.xml.catalog.CatalogResolverImpl.resolve(CatalogResolverImpl.java:153) > at org.basex.query.StaticContext.resolve(StaticContext.java:156) > at org.basex.query.StaticContext.resolve(StaticContext.java:137) > at org.basex.query.QueryInput.<init>(QueryInput.java:31) > at org.basex.query.func.fn.Docs.queryInput(Docs.java:37) > at org.basex.query.func.fn.Docs.lambda$accept$0(Docs.java:67) > at org.basex.query.scope.LockVisitor.lock(LockVisitor.java:48) > at org.basex.query.func.fn.Docs.accept(Docs.java:56) > at org.basex.query.scope.MainModule.databases(MainModule.java:93) > at org.basex.query.QueryContext.addLocks(QueryContext.java:412) > at org.basex.query.QueryProcessor.addLocks(QueryProcessor.java:230) > at org.basex.core.cmd.AQuery.addLocks(AQuery.java:170) > at org.basex.core.locks.Locking.acquire(Locking.java:86) > at org.basex.core.jobs.Job.register(Job.java:48) > at org.basex.core.Command.execute(Command.java:90) > at org.basex.gui.GUI.execute(GUI.java:430) > at org.basex.gui.GUI.lambda$execute$5(GUI.java:375) > at java.base/java.lang.Thread.run(Thread.java:1583) > > I can go on using 11.8 for now, so this is only somewhat alarming rather > than completely alarming. > > -- Graydon > > >

On Thu, May 15, 2025, at 03:21, Christian Grün wrote: > Hi Graydon, > > With the latest snapshot, the unexpected stack trace will be correctly caught [1]. The default behavior of the standard resolver can be changed with a system property [2]. We’ll think about adding Norm’s resolver to the distributions of BaseX. > > Cheers, > Christian > > [1] https://files.basex.org/releases/latest/ > [2] https://docs.basex.org/12/Catalog_Resolver#additional_notes > > > On Sun, May 11, 2025 at 5:13 PM Graydon Saunders <graydonish at fastmail.com> wrote: >> __ >> Hi Gunther, >> >> Thank you! >> >> Dropping xmlresolver-6.0.14-data.jar and xmlresolver-6.0.14.jar from the zip release via https://github.com/xmlresolver/xmlresolver/releases into basex/lib/custom works. The document parses and the entities are resolved; I have the catalog set via "local options" in both .basex and .basexgui, so I'm going to think this is expected. And I can confirm that I don't need to explicitly set any options in doc() for things to work. >> >> I'm sure this is a much more complex issue than it seems to me, but perhaps it would be a useful thing to ship the open resolver with BaseX? >> >> Much appreciated, >> Graydon >> >> On Sun, May 11, 2025, at 06:01, Gunther Rademacher wrote: >>> Hi Graydon, >>> >>> thank you for reporting this. I was able to reproduce the behaviour that you are seeing - not only with BaseX 11.9 and Java 21, but also with BaseX 11.8 and Java 11. >>> >>> The root cause appears to be the standard JDK catalog resolver, javax.xml.catalog.CatalogResolverImpl, which tends to throw a CatalogException when a URI cannot be resolved via the catalog. Unfortunately, BaseX does not currently handle this exception type. It expects a TransformerException, which is why the process fails in your case. >>> >>> In my own environment, I’ve never encountered this issue because I always have XML Resolver, i.e. org.xmlresolver.Resolver, on the classpath. This resolver replaces the standard implementation and simply returns null when a match isn’t found, rather than throwing an exception. >>> >>> I’ll open a GitHub issue to ensure that BaseX is updated to handle CatalogException properly. In the meantime, a practical workaround is to add XML Resolver to your classpath. I suspect you may already have it in place in the environment where you tested BaseX 11.8, which could explain why the issue didn’t surface there. >>> >>> Also, as you mentioned, fn:doc was recently updated. You can find more about those changes in this discussion: >>> https://www.mail-archive.com/basex-talk at mailman.uni-konstanz.de/msg16163.html >>> >>> However, I don’t believe those updates are directly related to the issue you're seeing. >>> >>> Best regards, >>> Gunther >>> >>> *Gesendet: *Sonntag, 11. Mai 2025 um 03:16 >>> *Von: *"Graydon Saunders" <graydonish at fastmail.com> >>> *An: *BaseX <BaseX-Talk at mailman.uni-konstanz.de> >>> *Betreff: *[basex-talk] 11.9 doc() not working with DOCTYPE declaration >>> Hello -- >>> >>> I'm on linux, using 11.9 and using >>> >>> 20:59 graydon % java -version >>> openjdk version "21.0.7" 2025-04-15 >>> OpenJDK Runtime Environment (Red_Hat-21.0.7.0.6-1) (build 21.0.7+6) >>> OpenJDK 64-Bit Server VM (Red_Hat-21.0.7.0.6-1) (build 21.0.7+6, mixed mode, sharing) >>> >>> If I use the basexgui "Create DB" dialog and the internal parser, I can load the whole folder without error; I get the expected DB and the named entities in the content have been resolved. (These are effectively HTML 4 entity names, so that's not completely guaranteeing that the DTD in the DOCTYPE is being found and parsed somewhere.) >>> >>> If I use the doc() function with an absolute file system path to one of the same files that loaded into the DB just fine, I get an error. I get the same error if I try to switch off the internal parser and set a catalog. It's at least several files in the folder that loads fine as a DB. >>> >>> All of this works fine with 11.8; whatever changed changed in 11.9. >>> >>> The error is: >>> >>> Could not execute XQUERY doc('/home/graydon/GeneralCode/git/cpc-conversion/Chapter/BU4372-14A.xml'): >>> Improper use? Potential bug? Your feedback is welcome: >>> Contact: basex-talk at mailman.uni-konstanz.de >>> Version: BaseX 11.9 >>> Java: Red Hat, Inc., 21.0.7 >>> OS: Linux, amd64 >>> Stack Trace: >>> javax.xml.catalog.CatalogException: JAXP09040002: No match found for href '/home/graydon/GeneralCode/git/cpc-conversion/Chapter/BU4372-14A.xml' and base '/home/graydon/GeneralCode/git/cpc-conversion/xquery/file'. >>> at java.xml/javax.xml.catalog.CatalogMessages.reportError(CatalogMessages.java:74) >>> at java.xml/javax.xml.catalog.CatalogResolverImpl.resolve(CatalogResolverImpl.java:153) >>> at org.basex.query.StaticContext.resolve(StaticContext.java:156) >>> at org.basex.query.StaticContext.resolve(StaticContext.java:137) >>> at org.basex.query.QueryInput.<init>(QueryInput.java:31) >>> at org.basex.query.func.fn.Docs.queryInput(Docs.java:37) >>> at org.basex.query.func.fn.Docs.lambda$accept$0(Docs.java:67) >>> at org.basex.query.scope.LockVisitor.lock(LockVisitor.java:48) >>> at org.basex.query.func.fn.Docs.accept(Docs.java:56) >>> at org.basex.query.scope.MainModule.databases(MainModule.java:93) >>> at org.basex.query.QueryContext.addLocks(QueryContext.java:412) >>> at org.basex.query.QueryProcessor.addLocks(QueryProcessor.java:230) >>> at org.basex.core.cmd.AQuery.addLocks(AQuery.java:170) >>> at org.basex.core.locks.Locking.acquire(Locking.java:86) >>> at org.basex.core.jobs.Job.register(Job.java:48) >>> at org.basex.core.Command.execute(Command.java:90) >>> at org.basex.gui.GUI.execute(GUI.java:430) >>> at org.basex.gui.GUI.lambda$execute$5(GUI.java:375) >>> at java.base/java.lang.Thread.run(Thread.java:1583) >>> >>> I can go on using 11.8 for now, so this is only somewhat alarming rather than completely alarming. >>> >>> -- Graydon >>