Main Page » Mailing List » September 2025

Multithreading issues after upgrade to BaseX 12

Monday, 22 September 2025 · 2 messages
JB
Johannes Bauer22 Sep 2025 09:26 #
Hi, we experience some problems after the upgrade to version 12 with the fulltext search. We are using xquery:fork-join() to run the fulltext search on multiple databases at the same time (see attachment). This worked fine with version 11 but has strange effects after the upgrade to version 12. With BaseX 11 the log output is like this: 09:03:51.054      [0:0:0:0:0:0:0:1]:55079 admin REQUEST     [POST] /dba/query?file=ftsearch.xq 09:03:51.203      SERVER      admin TRACE local:search-db(ALL) 100.4 ms 09:03:51.204      SERVER      admin TRACE local:search-db(ALL) 1721 09:03:51.243      SERVER      admin TRACE local:matches() 38.84 ms 09:03:51.270      SERVER      admin TRACE local:search-db(MD) 166.69 ms 09:03:51.270      SERVER      admin TRACE local:search-db(MD) 1883 09:03:51.282      SERVER      admin TRACE local:matches() 11.9 ms ... 09:03:52.284      SERVER      admin TRACE local:search-db(XP) 307.84 ms 09:03:52.284      SERVER      admin TRACE local:search-db(XP) 74128 09:03:52.525      SERVER      admin TRACE local:matches() 241.64 ms 09:03:52.796      SERVER      admin TRACE local:matches() 581.56 ms 09:03:52.796      SERVER      admin TRACE local:search-distinct() search "test" 1710.79 ms 09:03:53.200      [0:0:0:0:0:0:0:1]:55079 admin 200         2160.7 ms After about 2 seconds the search is finished. With BaseX 12 two things might happen. ConcurrentModificationException This happens almost immediatelly. 500   Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk at mailman.uni-konstanz.de Version: BaseX 12.0 Java: Eclipse Adoptium, 21 OS: Windows 10, amd64 Stack Trace: java.util.ConcurrentModificationException at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1095) at java.base/java.util.ArrayList$Itr.next(ArrayList.java:1049) at org.basex.query.QueryResources.close(QueryResources.java:94) at org.basex.query.QueryContext.close(QueryContext.java:543) at org.basex.http.restxq.RestXqResponse.finish(RestXqResponse.java:152) at org.basex.http.web.WebResponse.create(WebResponse.java:63) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:70) at org.basex.http.BaseXServlet.service(BaseXServlet.java:68) at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:587) at org.eclipse.jetty.ee9.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1397) at org.eclipse.jetty.ee9.servlet.ServletHolder.handle(ServletHolder.java:765) at org.eclipse.jetty.ee9.servlet.ServletHandler.doHandle(ServletHandler.java:528) at org.eclipse.jetty.ee9.nested.ScopedHandler.handle(ScopedHandler.java:127) at org.eclipse.jetty.ee9.security.SecurityHandler.handle(SecurityHandler.java:603) at org.eclipse.jetty.ee9.nested.HandlerWrapper.handle(HandlerWrapper.java:124) at org.eclipse.jetty.ee9.nested.ScopedHandler.nextHandle(ScopedHandler.java:197) at org.eclipse.jetty.ee9.nested.SessionHandler.doHandle(SessionHandler.java:612) at org.eclipse.jetty.ee9.nested.ScopedHandler.nextHandle(ScopedHandler.java:195) at org.eclipse.jetty.ee9.nested.ContextHandler.doHandle(ContextHandler.java:1045) at org.eclipse.jetty.ee9.nested.ScopedHandler.nextScope(ScopedHandler.java:164) at org.eclipse.jetty.ee9.servlet.ServletHandler.doScope(ServletHandler.java:483) at org.eclipse.jetty.ee9.nested.ScopedHandler.nextScope(ScopedHandler.java:162) at org.eclipse.jetty.ee9.nested.SessionHandler.doScope(SessionHandler.java:589) at org.eclipse.jetty.ee9.nested.ScopedHandler.nextScope(ScopedHandler.java:162) at org.eclipse.jetty.ee9.nested.ContextHandler.doScope(ContextHandler.java:966) at org.eclipse.jetty.ee9.nested.ScopedHandler.handle(ScopedHandler.java:125) at org.eclipse.jetty.ee9.nested.ContextHandler.handle(ContextHandler.java:1719) at org.eclipse.jetty.ee9.nested.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1565) at org.eclipse.jetty.ee9.nested.HttpChannel.dispatch(HttpChannel.java:724) at org.eclipse.jetty.ee9.nested.HttpChannel.handle(HttpChannel.java:512) at org.eclipse.jetty.ee9.nested.ContextHandler$CoreContextHandler$CoreToNestedHandler.handle(ContextHandler.java:3029) at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1071) at org.eclipse.jetty.server.Server.handle(Server.java:182) at org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:678) at org.eclipse.jetty.server.internal.HttpConnection.onFillable(HttpConnection.java:416) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:322) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:99) at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:480) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:443) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:981) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1211) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1166) at java.base/java.lang.Thread.run(Thread.java:1583)   189.44 ms    Full text search never finishes 09:00:12.422      [0:0:0:0:0:0:0:1]:55041 admin REQUEST     [POST] /dba/query?file=ftsearch.xq         09:00:12.633      SERVER      admin TRACE local:search-db(ALL) 115.46 ms             09:00:12.635      SERVER      admin TRACE local:search-db(ALL) 1721            09:00:12.672      SERVER      admin TRACE local:matches() 37.09 ms             09:00:12.711      SERVER      admin TRACE local:search-db(MD) 194.24 ms        09:00:12.711      SERVER      admin TRACE local:search-db(MD) 1883             09:00:12.725      SERVER      admin TRACE local:matches() 13.66 ms             09:00:12.851      SERVER      admin TRACE local:search-db(LS) 334.48 ms        09:00:12.851      SERVER      admin TRACE local:search-db(LS) 169524           09:00:12.923      SERVER      admin TRACE local:search-db(CT) 406.58 ms        09:00:12.923      SERVER      admin TRACE local:search-db(CT) 80086            ... 09:00:13.483      SERVER      admin TRACE local:search-db(US) 47012            09:00:13.553      SERVER      admin TRACE local:search-db(POC) 828.06 ms             09:00:13.554      SERVER      admin TRACE local:search-db(POC) 10147           09:00:13.693      SERVER      admin TRACE local:search-db(MR) 1020.89 ms             09:00:13.693      SERVER      admin TRACE local:search-db(MR) 223155           The process "hangs" and never finishes and CPU is running on full load. The current database where it hangs is always a different one. Does this have anything to do with the Jetty update? It seems to be related to the multithreading. Best Regards Johannes
CG
Christian Grün22 Sep 2025 09:33 #
Hi Johannes, This has been triggered by JDK 17, which seems to apply other, probably more advanced parallelization strategies than previous versions. We know how to fix this in BaseX, but it may take a while [1]. You may get rid of the problem if you only do what’s really necessary inside the xquery:fork-join() function arguments. Best, Christian [1] https://github.com/BaseXdb/basex/issues/2479
Quoted text · Johannes Bauer · 49 lines
Von: Johannes Bauer via BaseX-Talk <basex-talk at mailman.uni-konstanz.de> Gesendet: Montag, 22. September 2025 09:26 An: BaseX <basex-talk at mailman.uni-konstanz.de> Betreff: [basex-talk] Multithreading issues after upgrade to BaseX 12 Hi, we experience some problems after the upgrade to version 12 with the fulltext search. We are using xquery:fork-join() to run the fulltext search on multiple databases at the same time (see attachment). This worked fine with version 11 but has strange effects after the upgrade to version 12. With BaseX 11 the log output is like this: 09:03:51.054      [0:0:0:0:0:0:0:1]:55079 admin REQUEST     [POST] /dba/query?file=ftsearch.xq 09:03:51.203      SERVER      admin TRACE local:search-db(ALL) 100.4 ms 09:03:51.204      SERVER      admin TRACE local:search-db(ALL) 1721 09:03:51.243      SERVER      admin TRACE local:matches() 38.84 ms 09:03:51.270      SERVER      admin TRACE local:search-db(MD) 166.69 ms 09:03:51.270      SERVER      admin TRACE local:search-db(MD) 1883 09:03:51.282      SERVER      admin TRACE local:matches() 11.9 ms ... 09:03:52.284      SERVER      admin TRACE local:search-db(XP) 307.84 ms 09:03:52.284      SERVER      admin TRACE local:search-db(XP) 74128 09:03:52.525      SERVER      admin TRACE local:matches() 241.64 ms 09:03:52.796      SERVER      admin TRACE local:matches() 581.56 ms 09:03:52.796      SERVER      admin TRACE local:search-distinct() search "test" 1710.79 ms 09:03:53.200      [0:0:0:0:0:0:0:1]:55079 admin 200         2160.7 ms After about 2 seconds the search is finished. With BaseX 12 two things might happen. ConcurrentModificationException This happens almost immediatelly. 500   Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk at mailman.uni-konstanz.de Version: BaseX 12.0 Java: Eclipse Adoptium, 21 OS: Windows 10, amd64 Stack Trace: java.util.ConcurrentModificationException at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1095) at java.base/java.util.ArrayList$Itr.next(ArrayList.java:1049) at org.basex.query.QueryResources.close(QueryResources.java:94) at org.basex.query.QueryContext.close(QueryContext.java:543) at org.basex.http.restxq.RestXqResponse.finish(RestXqResponse.java:152) at org.basex.http.web.WebResponse.create(WebResponse.java:63) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:70) at org.basex.http.BaseXServlet.service(BaseXServlet.java:68) at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:587) at org.eclipse.jetty.ee9.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1397) at org.eclipse.jetty.ee9.servlet.ServletHolder.handle(ServletHolder.java:765) at org.eclipse.jetty.ee9.servlet.ServletHandler.doHandle(ServletHandler.java:528) at org.eclipse.jetty.ee9.nested.ScopedHandler.handle(ScopedHandler.java:127) at org.eclipse.jetty.ee9.security.SecurityHandler.handle(SecurityHandler.java:603) at org.eclipse.jetty.ee9.nested.HandlerWrapper.handle(HandlerWrapper.java:124) at org.eclipse.jetty.ee9.nested.ScopedHandler.nextHandle(ScopedHandler.java:197) at org.eclipse.jetty.ee9.nested.SessionHandler.doHandle(SessionHandler.java:612) at org.eclipse.jetty.ee9.nested.ScopedHandler.nextHandle(ScopedHandler.java:195) at org.eclipse.jetty.ee9.nested.ContextHandler.doHandle(ContextHandler.java:1045) at org.eclipse.jetty.ee9.nested.ScopedHandler.nextScope(ScopedHandler.java:164) at org.eclipse.jetty.ee9.servlet.ServletHandler.doScope(ServletHandler.java:483) at org.eclipse.jetty.ee9.nested.ScopedHandler.nextScope(ScopedHandler.java:162) at org.eclipse.jetty.ee9.nested.SessionHandler.doScope(SessionHandler.java:589) at org.eclipse.jetty.ee9.nested.ScopedHandler.nextScope(ScopedHandler.java:162) at org.eclipse.jetty.ee9.nested.ContextHandler.doScope(ContextHandler.java:966) at org.eclipse.jetty.ee9.nested.ScopedHandler.handle(ScopedHandler.java:125) at org.eclipse.jetty.ee9.nested.ContextHandler.handle(ContextHandler.java:1719) at org.eclipse.jetty.ee9.nested.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1565) at org.eclipse.jetty.ee9.nested.HttpChannel.dispatch(HttpChannel.java:724) at org.eclipse.jetty.ee9.nested.HttpChannel.handle(HttpChannel.java:512) at org.eclipse.jetty.ee9.nested.ContextHandler$CoreContextHandler$CoreToNestedHandler.handle(ContextHandler.java:3029) at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1071) at org.eclipse.jetty.server.Server.handle(Server.java:182) at org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:678) at org.eclipse.jetty.server.internal.HttpConnection.onFillable(HttpConnection.java:416) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:322) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:99) at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:480) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:443) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:981) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1211) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1166) at java.base/java.lang.Thread.run(Thread.java:1583)   189.44 ms    Full text search never finishes 09:00:12.422      [0:0:0:0:0:0:0:1]:55041 admin REQUEST     [POST] /dba/query?file=ftsearch.xq         09:00:12.633      SERVER      admin TRACE local:search-db(ALL) 115.46 ms             09:00:12.635      SERVER      admin TRACE local:search-db(ALL) 1721            09:00:12.672      SERVER      admin TRACE local:matches() 37.09 ms             09:00:12.711      SERVER      admin TRACE local:search-db(MD) 194.24 ms        09:00:12.711      SERVER      admin TRACE local:search-db(MD) 1883             09:00:12.725      SERVER      admin TRACE local:matches() 13.66 ms             09:00:12.851      SERVER      admin TRACE local:search-db(LS) 334.48 ms        09:00:12.851      SERVER      admin TRACE local:search-db(LS) 169524           09:00:12.923      SERVER      admin TRACE local:search-db(CT) 406.58 ms        09:00:12.923      SERVER      admin TRACE local:search-db(CT) 80086            ... 09:00:13.483      SERVER      admin TRACE local:search-db(US) 47012            09:00:13.553      SERVER      admin TRACE local:search-db(POC) 828.06 ms             09:00:13.554      SERVER      admin TRACE local:search-db(POC) 10147           09:00:13.693      SERVER      admin TRACE local:search-db(MR) 1020.89 ms             09:00:13.693      SERVER      admin TRACE local:search-db(MR) 223155           The process "hangs" and never finishes and CPU is running on full load. The current database where it hangs is always a different one. Does this have anything to do with the Jetty update? It seems to be related to the multithreading. Best Regards Johannes

Written for BaseX 12.0. BaseX may have changed since: please check the current documentation.

« Who wants to take over the management of my RBaseX package?db:drop error not being caught »
⚡Generated with XQuery