Optimize database never returns, leaves database in "opened by another process" state
Monday, 26 February 2024 · 11 messages · last reply 4 Mar 2024
EK
Eliot Kimber26 Feb 2024 17:25 #
Using Basex 10.7 on Linux.
I’m running a sequence of jobs to update and optimize a set of databases following loading a number of documents created dynamically (as opposed to being read from disk).
I’m seeing a new behavior, which is that the optimization step never completes but also doesn’t show any error in the log. The database shows no items and is in the locked by another process state if I try to drop it. This behavior seems to be consistently repeatable with my current code base (I’m working on some code updates, so it’s possible I’ve introduced something that would cause this behavior but I haven’t changed the code that leads up to the failing optimize). The server has plenty of disk space, etc.
Optimization code is:
try {
if (db:exists($database))
then
(
util:logToConsole('dbadmin:optimizeDatabase', ``[Optimizing database `{$database}`]``),
db:optimize($database, true(), $dbadmin:dbOptimizeOptions)
)
else util:logToConsole('dbadmin:optimizeDatabase', ``[Database '`{$database}`' does not exist. Nothing to optimize.]``)
} catch * {
util:logToConsole(
'dbadmin:optimizeDatabase',
``[Exception optimizing database '`{$database}`': `{$err:code}` - `{$err:description}`]``,
'error')
}And the optimization options are:
declare variable $dbadmin:dbOptimizeOptions as map(*) :=
(: Turn on all the indexes :)
map {
'attrindex' : true(),
'tokenindex' : true(),
'textindex' : true(),
'ftindex' : true()
};
This code has been working fine for a long time and I’ve been running 10.7 for a least a couple of months, so I’m wondering:
A) Would would cause this behavior?
B) How can I diagnose it short of debugging the Java code (which I can do but it’s non-trivial for me to set up).
Thanks,
Eliot
_____________________________________________
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>
CG
Christian Grün28 Feb 2024 16:36 #
…this one could be related to a bug that was recently fixed in the latest snapshot [1]. About time to get BaseX 11 finished…
[1]
https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496
Quoted text · Eliot Kimber · 109 lines
On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber <eliot.kimber at servicenow.com> wrote: > Using Basex 10.7 on Linux. > > > I’m running a sequence of jobs to update and optimize a set of databases > following loading a number of documents created dynamically (as opposed to > being read from disk). > > > > I’m seeing a new behavior, which is that the optimization step never > completes but also doesn’t show any error in the log. The database shows no > items and is in the locked by another process state if I try to drop it. > This behavior seems to be consistently repeatable with my current code base > (I’m working on some code updates, so it’s possible I’ve introduced > something that would cause this behavior but I haven’t changed the code > that leads up to the failing optimize). The server has plenty of disk > space, etc. > > > > Optimization code is: > > try { > > if (db:exists($database)) > > then > > ( > > util:logToConsole('dbadmin:optimizeDatabase', ``[Optimizing > database `{$database}`]``), > > db:optimize($database, true(), $dbadmin:dbOptimizeOptions) > > ) > > else util:logToConsole('dbadmin:optimizeDatabase', ``[Database > '`{$database}`' does not exist. Nothing to optimize.]``) > > } catch * { > > util:logToConsole( > > 'dbadmin:optimizeDatabase', > > ``[Exception optimizing database '`{$database}`': `{$err:code}` - > `{$err:description}`]``, > > 'error') > > } > > > > And the optimization options are: > > declare variable $dbadmin:dbOptimizeOptions as map(*) := > > (: Turn on all the indexes :) > > map { > > 'attrindex' : true(), > > 'tokenindex' : true(), > > 'textindex' : true(), > > 'ftindex' : true() > > }; > > This code has been working fine for a long time and I’ve been running 10.7 > for a least a couple of months, so I’m wondering: > > A) Would would cause this behavior? > B) How can I diagnose it short of debugging the Java code (which I can do > but it’s non-trivial for me to set up). > > > > Thanks, > > > > Eliot > > > > _____________________________________________ > > *Eliot Kimber* > > Sr Staff Content Engineer > > O: 512 554 9368 > > M: 512 554 9368 > > servicenow.com <https://www.servicenow.com> > > LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter > <https://twitter.com/servicenow> | YouTube > <https://www.youtube.com/user/servicenowinc> | Facebook > <https://www.facebook.com/servicenow> >
EK
Eliot Kimber28 Feb 2024 17:01 #
I’ll check out the snapshot.
I was starting to dig into the Java code yesterday but was first testing if I needed the optimization at all in this case, which it looks like I don’t (the consumer of the stored docs doesn’t depend on indexing to find things).
Cheers,
E.
_____________________________________________
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>
Quoted text · Christian Grün · 52 lines
From: Christian Grün <christian.gruen at gmail.com> Date: Wednesday, February 28, 2024 at 9:36 AM To: Eliot Kimber <eliot.kimber at servicenow.com> Cc: basex-talk at mailman.uni-konstanz.de <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state [External Email] …this one could be related to a bug that was recently fixed in the latest snapshot [1]. About time to get BaseX 11 finished…[1] https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber <eliot.kimber@servicenow i This message needs your attention • Someone new is on this email. Provided by ServiceNow DT (Employee Portal KB0077950) - This banner is visible only to ServiceNow employees.<https://mimecast.com> …this one could be related to a bug that was recently fixed in the latest snapshot [1]. About time to get BaseX 11 finished… [1] https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496 On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber <eliot.kimber at servicenow.com<mailto:eliot.kimber at servicenow.com>> wrote: Using Basex 10.7 on Linux. I’m running a sequence of jobs to update and optimize a set of databases following loading a number of documents created dynamically (as opposed to being read from disk). I’m seeing a new behavior, which is that the optimization step never completes but also doesn’t show any error in the log. The database shows no items and is in the locked by another process state if I try to drop it. This behavior seems to be consistently repeatable with my current code base (I’m working on some code updates, so it’s possible I’ve introduced something that would cause this behavior but I haven’t changed the code that leads up to the failing optimize). The server has plenty of disk space, etc. Optimization code is: try { if (db:exists($database)) then ( util:logToConsole('dbadmin:optimizeDatabase', ``[Optimizing database `{$database}`]``), db:optimize($database, true(), $dbadmin:dbOptimizeOptions) ) else util:logToConsole('dbadmin:optimizeDatabase', ``[Database '`{$database}`' does not exist. Nothing to optimize.]``) } catch * { util:logToConsole( 'dbadmin:optimizeDatabase', ``[Exception optimizing database '`{$database}`': `{$err:code}` - `{$err:description}`]``, 'error') } And the optimization options are: declare variable $dbadmin:dbOptimizeOptions as map(*) := (: Turn on all the indexes :) map { 'attrindex' : true(), 'tokenindex' : true(), 'textindex' : true(), 'ftindex' : true() }; This code has been working fine for a long time and I’ve been running 10.7 for a least a couple of months, so I’m wondering: A) Would would cause this behavior? B) How can I diagnose it short of debugging the Java code (which I can do but it’s non-trivial for me to set up). Thanks, Eliot Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>
EK
Eliot Kimber29 Feb 2024 09:45 #
I’m trying to compile the latest code but “mvn clean install” fails on failure to download some dependencies:
[INFO] --------------------------< org.basex:basex >--------------------------- [INFO] Building BaseX Core 11.0-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]--------------------------------- Downloading from devsnc-mirror: http://nexus.proxy.devsnc.com/content/groups/public/jp/sourceforge/igo/igo/0.4.3/igo-0.4.3.pom [WARNING] The POM for jp.sourceforge.igo:igo:jar:0.4.3 is missing, no dependency information available Downloading from devsnc-mirror: http://nexus.proxy.devsnc.com/content/groups/public/org/apache/lucene-stemmers/3.4.0/lucene-stemmers-3.4.0.pom [WARNING] The POM for org.apache:lucene-stemmers:jar:3.4.0 is missing, no dependency information available Downloading from devsnc-mirror: http://nexus.proxy.devsnc.com/content/groups/public/jp/sourceforge/igo/igo/0.4.3/igo-0.4.3.jar Downloading from devsnc-mirror: http://nexus.proxy.devsnc.com/content/groups/public/org/apache/lucene-stemmers/3.4.0/lucene-stemmers-3.4.0.jar [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------ [INFO] Total time: 5.589 s
[INFO] Finished at: 2024-02-29T02:34:11-06:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project basex: Could not resolve dependencies for project org.basex:basex:jar:11.0-SNAPSHOT: The following artifacts could not be resolved: jp.sourceforge.igo:igo:jar:0.4.3 (absent), org.apache:lucene-stemmers:jar:3.4.0 (absent): Could not find artifact jp.sourceforge.igo:igo:jar:0.4.3 in devsnc-mirror (http://nexus.proxy.devsnc.com/content/groups/public) -> [Help 1]
And then:
[ERROR] Failed to execute goal on project basex: Could not resolve dependencies for project org.basex:basex:jar:11.0-SNAPSHOT: The following artifacts could not be resolved: jp.sourceforge.igo:igo:jar:0.4.3 (absent), org.apache:lucene-stemmers:jar:3.4.0 (absent): jp.sourceforge.igo:igo:jar:0.4.3 was not found in http://nexus.proxy.devsnc.com/content/groups/public during a previous attempt. This failure was cached in the local repository and resolution is not reattempted until the update interval of devsnc-mirror has elapsed or updates are forced -> [Help 1]
Where ‘nexus.proxy.devsnc.com’ is our internal Maven proxy.
This is after removing my local maven repository cache.
Before doing that, I got the “present but not available” failure on these libraries. The jars are present in my original .m2/repository but not in a newly-created repository (having moved the original one out of the way).
I’m using maven 3.9.6.
As far as I can tell from my research, this is a change in behavior from Maven 2 to Maven 3 where it refuses to use local dependencies that were fetched as a side effect of some other dependency that cannot now be fetched. But I’m not finding any maven options that will work around this failure.
Any idea how I can move past this? Or is there a nightly build somewhere I can download?
Thanks,
Eliot
_____________________________________________
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>
Quoted text · Eliot Kimber · 67 lines
From: Eliot Kimber <eliot.kimber at servicenow.com> Date: Wednesday, February 28, 2024 at 10:01 AM To: Christian Grün <christian.gruen at gmail.com> Cc: basex-talk at mailman.uni-konstanz.de <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state I’ll check out the snapshot. I was starting to dig into the Java code yesterday but was first testing if I needed the optimization at all in this case, which it looks like I don’t (the consumer of the stored docs doesn’t depend on indexing to find things). Cheers, E. Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow> From: Christian Grün <christian.gruen at gmail.com> Date: Wednesday, February 28, 2024 at 9:36 AM To: Eliot Kimber <eliot.kimber at servicenow.com> Cc: basex-talk at mailman.uni-konstanz.de <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state [External Email] …this one could be related to a bug that was recently fixed in the latest snapshot [1]. About time to get BaseX 11 finished…[1] https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber <eliot.kimber@servicenow i This message needs your attention • Someone new is on this email. Provided by ServiceNow DT (Employee Portal KB0077950) - This banner is visible only to ServiceNow employees.<https://mimecast.com> …this one could be related to a bug that was recently fixed in the latest snapshot [1]. About time to get BaseX 11 finished… [1] https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496 On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber <eliot.kimber at servicenow.com<mailto:eliot.kimber at servicenow.com>> wrote: Using Basex 10.7 on Linux. I’m running a sequence of jobs to update and optimize a set of databases following loading a number of documents created dynamically (as opposed to being read from disk). I’m seeing a new behavior, which is that the optimization step never completes but also doesn’t show any error in the log. The database shows no items and is in the locked by another process state if I try to drop it. This behavior seems to be consistently repeatable with my current code base (I’m working on some code updates, so it’s possible I’ve introduced something that would cause this behavior but I haven’t changed the code that leads up to the failing optimize). The server has plenty of disk space, etc. Optimization code is: try { if (db:exists($database)) then ( util:logToConsole('dbadmin:optimizeDatabase', ``[Optimizing database `{$database}`]``), db:optimize($database, true(), $dbadmin:dbOptimizeOptions) ) else util:logToConsole('dbadmin:optimizeDatabase', ``[Database '`{$database}`' does not exist. Nothing to optimize.]``) } catch * { util:logToConsole( 'dbadmin:optimizeDatabase', ``[Exception optimizing database '`{$database}`': `{$err:code}` - `{$err:description}`]``, 'error') } And the optimization options are: declare variable $dbadmin:dbOptimizeOptions as map(*) := (: Turn on all the indexes :) map { 'attrindex' : true(), 'tokenindex' : true(), 'textindex' : true(), 'ftindex' : true() }; This code has been working fine for a long time and I’ve been running 10.7 for a least a couple of months, so I’m wondering: A) Would would cause this behavior? B) How can I diagnose it short of debugging the Java code (which I can do but it’s non-trivial for me to set up). Thanks, Eliot Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>
EK
Eliot Kimber29 Feb 2024 10:14 #
Found the latest build at https://files.basex.org/releases/latest/
Cheers,
E.
_____________________________________________
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>
Quoted text · Eliot Kimber · 106 lines
From: Eliot Kimber <eliot.kimber at servicenow.com> Date: Thursday, February 29, 2024 at 2:45 AM To: Christian Grün <christian.gruen at gmail.com> Cc: basex-talk at mailman.uni-konstanz.de <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state I’m trying to compile the latest code but “mvn clean install” fails on failure to download some dependencies: [INFO] --------------------------< org.basex:basex >--------------------------- [INFO] Building BaseX Core 11.0-SNAPSHOT [INFO] from pom.xml [INFO] --------------------------------[ jar ]--------------------------------- Downloading from devsnc-mirror: http://nexus.proxy.devsnc.com/content/groups/public/jp/sourceforge/igo/igo/0.4.3/igo-0.4.3.pom [WARNING] The POM for jp.sourceforge.igo:igo:jar:0.4.3 is missing, no dependency information available Downloading from devsnc-mirror: http://nexus.proxy.devsnc.com/content/groups/public/org/apache/lucene-stemmers/3.4.0/lucene-stemmers-3.4.0.pom [WARNING] The POM for org.apache:lucene-stemmers:jar:3.4.0 is missing, no dependency information available Downloading from devsnc-mirror: http://nexus.proxy.devsnc.com/content/groups/public/jp/sourceforge/igo/igo/0.4.3/igo-0.4.3.jar Downloading from devsnc-mirror: http://nexus.proxy.devsnc.com/content/groups/public/org/apache/lucene-stemmers/3.4.0/lucene-stemmers-3.4.0.jar [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 5.589 s [INFO] Finished at: 2024-02-29T02:34:11-06:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal on project basex: Could not resolve dependencies for project org.basex:basex:jar:11.0-SNAPSHOT: The following artifacts could not be resolved: jp.sourceforge.igo:igo:jar:0.4.3 (absent), org.apache:lucene-stemmers:jar:3.4.0 (absent): Could not find artifact jp.sourceforge.igo:igo:jar:0.4.3 in devsnc-mirror (http://nexus.proxy.devsnc.com/content/groups/public) -> [Help 1] And then: [ERROR] Failed to execute goal on project basex: Could not resolve dependencies for project org.basex:basex:jar:11.0-SNAPSHOT: The following artifacts could not be resolved: jp.sourceforge.igo:igo:jar:0.4.3 (absent), org.apache:lucene-stemmers:jar:3.4.0 (absent): jp.sourceforge.igo:igo:jar:0.4.3 was not found in http://nexus.proxy.devsnc.com/content/groups/public during a previous attempt. This failure was cached in the local repository and resolution is not reattempted until the update interval of devsnc-mirror has elapsed or updates are forced -> [Help 1] Where ‘nexus.proxy.devsnc.com’ is our internal Maven proxy. This is after removing my local maven repository cache. Before doing that, I got the “present but not available” failure on these libraries. The jars are present in my original .m2/repository but not in a newly-created repository (having moved the original one out of the way). I’m using maven 3.9.6. As far as I can tell from my research, this is a change in behavior from Maven 2 to Maven 3 where it refuses to use local dependencies that were fetched as a side effect of some other dependency that cannot now be fetched. But I’m not finding any maven options that will work around this failure. Any idea how I can move past this? Or is there a nightly build somewhere I can download? Thanks, Eliot Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow> From: Eliot Kimber <eliot.kimber at servicenow.com> Date: Wednesday, February 28, 2024 at 10:01 AM To: Christian Grün <christian.gruen at gmail.com> Cc: basex-talk at mailman.uni-konstanz.de <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state I’ll check out the snapshot. I was starting to dig into the Java code yesterday but was first testing if I needed the optimization at all in this case, which it looks like I don’t (the consumer of the stored docs doesn’t depend on indexing to find things). Cheers, E. Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow> From: Christian Grün <christian.gruen at gmail.com> Date: Wednesday, February 28, 2024 at 9:36 AM To: Eliot Kimber <eliot.kimber at servicenow.com> Cc: basex-talk at mailman.uni-konstanz.de <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state [External Email] …this one could be related to a bug that was recently fixed in the latest snapshot [1]. About time to get BaseX 11 finished…[1] https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber <eliot.kimber@servicenow i This message needs your attention • Someone new is on this email. Provided by ServiceNow DT (Employee Portal KB0077950) - This banner is visible only to ServiceNow employees.<https://mimecast.com> …this one could be related to a bug that was recently fixed in the latest snapshot [1]. About time to get BaseX 11 finished… [1] https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496 On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber <eliot.kimber at servicenow.com<mailto:eliot.kimber at servicenow.com>> wrote: Using Basex 10.7 on Linux. I’m running a sequence of jobs to update and optimize a set of databases following loading a number of documents created dynamically (as opposed to being read from disk). I’m seeing a new behavior, which is that the optimization step never completes but also doesn’t show any error in the log. The database shows no items and is in the locked by another process state if I try to drop it. This behavior seems to be consistently repeatable with my current code base (I’m working on some code updates, so it’s possible I’ve introduced something that would cause this behavior but I haven’t changed the code that leads up to the failing optimize). The server has plenty of disk space, etc. Optimization code is: try { if (db:exists($database)) then ( util:logToConsole('dbadmin:optimizeDatabase', ``[Optimizing database `{$database}`]``), db:optimize($database, true(), $dbadmin:dbOptimizeOptions) ) else util:logToConsole('dbadmin:optimizeDatabase', ``[Database '`{$database}`' does not exist. Nothing to optimize.]``) } catch * { util:logToConsole( 'dbadmin:optimizeDatabase', ``[Exception optimizing database '`{$database}`': `{$err:code}` - `{$err:description}`]``, 'error') } And the optimization options are: declare variable $dbadmin:dbOptimizeOptions as map(*) := (: Turn on all the indexes :) map { 'attrindex' : true(), 'tokenindex' : true(), 'textindex' : true(), 'ftindex' : true() }; This code has been working fine for a long time and I’ve been running 10.7 for a least a couple of months, so I’m wondering: A) Would would cause this behavior? B) How can I diagnose it short of debugging the Java code (which I can do but it’s non-trivial for me to set up). Thanks, Eliot Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>
EK
Eliot Kimber29 Feb 2024 10:40 #
Using the latest build, 11.0 beta 17d8426, the prof:dump() function is reported as an unknown function.
What replaces it (or where can I find the V11 docs)?
Thanks,
E.
_____________________________________________
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>
Quoted text · Eliot Kimber · 120 lines
From: Eliot Kimber <eliot.kimber at servicenow.com> Date: Thursday, February 29, 2024 at 3:15 AM To: Christian Grün <christian.gruen at gmail.com> Cc: basex-talk at mailman.uni-konstanz.de <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state Found the latest build at https://files.basex.org/releases/latest/ Cheers, E. Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow> From: Eliot Kimber <eliot.kimber at servicenow.com> Date: Thursday, February 29, 2024 at 2:45 AM To: Christian Grün <christian.gruen at gmail.com> Cc: basex-talk at mailman.uni-konstanz.de <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state I’m trying to compile the latest code but “mvn clean install” fails on failure to download some dependencies: [INFO] --------------------------< org.basex:basex >--------------------------- [INFO] Building BaseX Core 11.0-SNAPSHOT [INFO] from pom.xml [INFO] --------------------------------[ jar ]--------------------------------- Downloading from devsnc-mirror: http://nexus.proxy.devsnc.com/content/groups/public/jp/sourceforge/igo/igo/0.4.3/igo-0.4.3.pom [WARNING] The POM for jp.sourceforge.igo:igo:jar:0.4.3 is missing, no dependency information available Downloading from devsnc-mirror: http://nexus.proxy.devsnc.com/content/groups/public/org/apache/lucene-stemmers/3.4.0/lucene-stemmers-3.4.0.pom [WARNING] The POM for org.apache:lucene-stemmers:jar:3.4.0 is missing, no dependency information available Downloading from devsnc-mirror: http://nexus.proxy.devsnc.com/content/groups/public/jp/sourceforge/igo/igo/0.4.3/igo-0.4.3.jar Downloading from devsnc-mirror: http://nexus.proxy.devsnc.com/content/groups/public/org/apache/lucene-stemmers/3.4.0/lucene-stemmers-3.4.0.jar [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 5.589 s [INFO] Finished at: 2024-02-29T02:34:11-06:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal on project basex: Could not resolve dependencies for project org.basex:basex:jar:11.0-SNAPSHOT: The following artifacts could not be resolved: jp.sourceforge.igo:igo:jar:0.4.3 (absent), org.apache:lucene-stemmers:jar:3.4.0 (absent): Could not find artifact jp.sourceforge.igo:igo:jar:0.4.3 in devsnc-mirror (http://nexus.proxy.devsnc.com/content/groups/public) -> [Help 1] And then: [ERROR] Failed to execute goal on project basex: Could not resolve dependencies for project org.basex:basex:jar:11.0-SNAPSHOT: The following artifacts could not be resolved: jp.sourceforge.igo:igo:jar:0.4.3 (absent), org.apache:lucene-stemmers:jar:3.4.0 (absent): jp.sourceforge.igo:igo:jar:0.4.3 was not found in http://nexus.proxy.devsnc.com/content/groups/public during a previous attempt. This failure was cached in the local repository and resolution is not reattempted until the update interval of devsnc-mirror has elapsed or updates are forced -> [Help 1] Where ‘nexus.proxy.devsnc.com’ is our internal Maven proxy. This is after removing my local maven repository cache. Before doing that, I got the “present but not available” failure on these libraries. The jars are present in my original .m2/repository but not in a newly-created repository (having moved the original one out of the way). I’m using maven 3.9.6. As far as I can tell from my research, this is a change in behavior from Maven 2 to Maven 3 where it refuses to use local dependencies that were fetched as a side effect of some other dependency that cannot now be fetched. But I’m not finding any maven options that will work around this failure. Any idea how I can move past this? Or is there a nightly build somewhere I can download? Thanks, Eliot Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow> From: Eliot Kimber <eliot.kimber at servicenow.com> Date: Wednesday, February 28, 2024 at 10:01 AM To: Christian Grün <christian.gruen at gmail.com> Cc: basex-talk at mailman.uni-konstanz.de <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state I’ll check out the snapshot. I was starting to dig into the Java code yesterday but was first testing if I needed the optimization at all in this case, which it looks like I don’t (the consumer of the stored docs doesn’t depend on indexing to find things). Cheers, E. Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow> From: Christian Grün <christian.gruen at gmail.com> Date: Wednesday, February 28, 2024 at 9:36 AM To: Eliot Kimber <eliot.kimber at servicenow.com> Cc: basex-talk at mailman.uni-konstanz.de <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state [External Email] …this one could be related to a bug that was recently fixed in the latest snapshot [1]. About time to get BaseX 11 finished…[1] https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber <eliot.kimber@servicenow i This message needs your attention • Someone new is on this email. Provided by ServiceNow DT (Employee Portal KB0077950) - This banner is visible only to ServiceNow employees.<https://mimecast.com> …this one could be related to a bug that was recently fixed in the latest snapshot [1]. About time to get BaseX 11 finished… [1] https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496 On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber <eliot.kimber at servicenow.com<mailto:eliot.kimber at servicenow.com>> wrote: Using Basex 10.7 on Linux. I’m running a sequence of jobs to update and optimize a set of databases following loading a number of documents created dynamically (as opposed to being read from disk). I’m seeing a new behavior, which is that the optimization step never completes but also doesn’t show any error in the log. The database shows no items and is in the locked by another process state if I try to drop it. This behavior seems to be consistently repeatable with my current code base (I’m working on some code updates, so it’s possible I’ve introduced something that would cause this behavior but I haven’t changed the code that leads up to the failing optimize). The server has plenty of disk space, etc. Optimization code is: try { if (db:exists($database)) then ( util:logToConsole('dbadmin:optimizeDatabase', ``[Optimizing database `{$database}`]``), db:optimize($database, true(), $dbadmin:dbOptimizeOptions) ) else util:logToConsole('dbadmin:optimizeDatabase', ``[Database '`{$database}`' does not exist. Nothing to optimize.]``) } catch * { util:logToConsole( 'dbadmin:optimizeDatabase', ``[Exception optimizing database '`{$database}`': `{$err:code}` - `{$err:description}`]``, 'error') } And the optimization options are: declare variable $dbadmin:dbOptimizeOptions as map(*) := (: Turn on all the indexes :) map { 'attrindex' : true(), 'tokenindex' : true(), 'textindex' : true(), 'ftindex' : true() }; This code has been working fine for a long time and I’ve been running 10.7 for a least a couple of months, so I’m wondering: A) Would would cause this behavior? B) How can I diagnose it short of debugging the Java code (which I can do but it’s non-trivial for me to set up). Thanks, Eliot Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>
AB
Andy Bunce29 Feb 2024 11:15 #
Maybe: https://help.basex.org/main/Profiling_Module
/Andy
Quoted text · Eliot Kimber · 391 lines
On Thu, 29 Feb 2024 at 09:40, Eliot Kimber <eliot.kimber at servicenow.com> wrote: > Using the latest build, 11.0 beta 17d8426, the prof:dump() function is > reported as an unknown function. > > > > What replaces it (or where can I find the V11 docs)? > > > > Thanks, > > > > E. > > > > > > _____________________________________________ > > *Eliot Kimber* > > Sr Staff Content Engineer > > O: 512 554 9368 > > M: 512 554 9368 > > servicenow.com <https://www.servicenow.com> > > LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter > <https://twitter.com/servicenow> | YouTube > <https://www.youtube.com/user/servicenowinc> | Facebook > <https://www.facebook.com/servicenow> > > > > *From: *Eliot Kimber <eliot.kimber at servicenow.com> > *Date: *Thursday, February 29, 2024 at 3:15 AM > *To: *Christian Grün <christian.gruen at gmail.com> > *Cc: *basex-talk at mailman.uni-konstanz.de < > basex-talk at mailman.uni-konstanz.de> > *Subject: *Re: [basex-talk] Optimize database never returns, leaves > database in "opened by another process" state > > Found the latest build at https://files.basex.org/releases/latest/ > > > > Cheers, > > > > E. > > > > _____________________________________________ > > *Eliot Kimber* > > Sr Staff Content Engineer > > O: 512 554 9368 > > M: 512 554 9368 > > servicenow.com <https://www.servicenow.com> > > LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter > <https://twitter.com/servicenow> | YouTube > <https://www.youtube.com/user/servicenowinc> | Facebook > <https://www.facebook.com/servicenow> > > > > *From: *Eliot Kimber <eliot.kimber at servicenow.com> > *Date: *Thursday, February 29, 2024 at 2:45 AM > *To: *Christian Grün <christian.gruen at gmail.com> > *Cc: *basex-talk at mailman.uni-konstanz.de < > basex-talk at mailman.uni-konstanz.de> > *Subject: *Re: [basex-talk] Optimize database never returns, leaves > database in "opened by another process" state > > I’m trying to compile the latest code but “mvn clean install” fails on > failure to download some dependencies: > > [*INFO*] *--------------------------< *org.basex:basex* > >---------------------------* > > [*INFO*] *Building BaseX Core 11.0-SNAPSHOT* > > [*INFO*] from pom.xml > > [*INFO*] *--------------------------------[ jar > ]---------------------------------* > > Downloading from devsnc-mirror: > http://nexus.proxy.devsnc.com/content/groups/public/jp/sourceforge/igo/igo/0.4.3/igo-0.4.3.pom > > [*WARNING*] The POM for jp.sourceforge.igo:igo:jar:0.4.3 is missing, no > dependency information available > > Downloading from devsnc-mirror: > http://nexus.proxy.devsnc.com/content/groups/public/org/apache/lucene-stemmers/3.4.0/lucene-stemmers-3.4.0.pom > > [*WARNING*] The POM for org.apache:lucene-stemmers:jar:3.4.0 is missing, > no dependency information available > > Downloading from devsnc-mirror: > http://nexus.proxy.devsnc.com/content/groups/public/jp/sourceforge/igo/igo/0.4.3/igo-0.4.3.jar > > Downloading from devsnc-mirror: > http://nexus.proxy.devsnc.com/content/groups/public/org/apache/lucene-stemmers/3.4.0/lucene-stemmers-3.4.0.jar > > [*INFO*] > *------------------------------------------------------------------------* > > [*INFO*] *BUILD FAILURE* > > [*INFO*] > *------------------------------------------------------------------------* > > [*INFO*] Total time: 5.589 s > > [*INFO*] Finished at: 2024-02-29T02:34:11-06:00 > > [*INFO*] > *------------------------------------------------------------------------* > > [*ERROR*] Failed to execute goal on project basex: *Could not resolve > dependencies for project org.basex:basex:jar:11.0-SNAPSHOT: The following > artifacts could not be resolved: jp.sourceforge.igo:igo:jar:0.4.3 (absent), > org.apache:lucene-stemmers:jar:3.4.0 (absent): Could not find artifact > jp.sourceforge.igo:igo:jar:0.4.3 in devsnc-mirror > (http://nexus.proxy.devsnc.com/content/groups/public > <http://nexus.proxy.devsnc.com/content/groups/public>)* -> *[Help 1]* > > > > And then: > > [*ERROR*] Failed to execute goal on project basex: *Could not resolve > dependencies for project org.basex:basex:jar:11.0-SNAPSHOT: The following > artifacts could not be resolved: jp.sourceforge.igo:igo:jar:0.4.3 (absent), > org.apache:lucene-stemmers:jar:3.4.0 (absent): > jp.sourceforge.igo:igo:jar:0.4.3 was not found in > http://nexus.proxy.devsnc.com/content/groups/public > <http://nexus.proxy.devsnc.com/content/groups/public> during a previous > attempt. This failure was cached in the local repository and resolution is > not reattempted until the update interval of devsnc-mirror has elapsed or > updates are forced* -> *[Help 1]* > > > > Where ‘nexus.proxy.devsnc.com’ is our internal Maven proxy. > > This is after removing my local maven repository cache. > > Before doing that, I got the “present but not available” failure on these > libraries. The jars are present in my original .m2/repository but not in a > newly-created repository (having moved the original one out of the way). > > > > I’m using maven 3.9.6. > > > > As far as I can tell from my research, this is a change in behavior from > Maven 2 to Maven 3 where it refuses to use local dependencies that were > fetched as a side effect of some other dependency that cannot now be > fetched. But I’m not finding any maven options that will work around this > failure. > > > > Any idea how I can move past this? Or is there a nightly build somewhere I > can download? > > > > Thanks, > > > > Eliot > > _____________________________________________ > > *Eliot Kimber* > > Sr Staff Content Engineer > > O: 512 554 9368 > > M: 512 554 9368 > > servicenow.com <https://www.servicenow.com> > > LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter > <https://twitter.com/servicenow> | YouTube > <https://www.youtube.com/user/servicenowinc> | Facebook > <https://www.facebook.com/servicenow> > > > > *From: *Eliot Kimber <eliot.kimber at servicenow.com> > *Date: *Wednesday, February 28, 2024 at 10:01 AM > *To: *Christian Grün <christian.gruen at gmail.com> > *Cc: *basex-talk at mailman.uni-konstanz.de < > basex-talk at mailman.uni-konstanz.de> > *Subject: *Re: [basex-talk] Optimize database never returns, leaves > database in "opened by another process" state > > I’ll check out the snapshot. > > I was starting to dig into the Java code yesterday but was first testing > if I needed the optimization at all in this case, which it looks like I > don’t (the consumer of the stored docs doesn’t depend on indexing to find > things). > > > > Cheers, > > > > E. > > > > _____________________________________________ > > *Eliot Kimber* > > Sr Staff Content Engineer > > O: 512 554 9368 > > M: 512 554 9368 > > servicenow.com <https://www.servicenow.com> > > LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter > <https://twitter.com/servicenow> | YouTube > <https://www.youtube.com/user/servicenowinc> | Facebook > <https://www.facebook.com/servicenow> > > > > *From: *Christian Grün <christian.gruen at gmail.com> > *Date: *Wednesday, February 28, 2024 at 9:36 AM > *To: *Eliot Kimber <eliot.kimber at servicenow.com> > *Cc: *basex-talk at mailman.uni-konstanz.de < > basex-talk at mailman.uni-konstanz.de> > *Subject: *Re: [basex-talk] Optimize database never returns, leaves > database in "opened by another process" state > *[External Email]* > > > ------------------------------ > > …this one could be related to a bug that was recently fixed in the latest > snapshot [1]. About time to get BaseX 11 finished… > > > > [1] > https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496 > > > > > > > > On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber <eliot.kimber at servicenow.com> > wrote: > > Using Basex 10.7 on Linux. > > > I’m running a sequence of jobs to update and optimize a set of databases > following loading a number of documents created dynamically (as opposed to > being read from disk). > > > > I’m seeing a new behavior, which is that the optimization step never > completes but also doesn’t show any error in the log. The database shows no > items and is in the locked by another process state if I try to drop it. > This behavior seems to be consistently repeatable with my current code base > (I’m working on some code updates, so it’s possible I’ve introduced > something that would cause this behavior but I haven’t changed the code > that leads up to the failing optimize). The server has plenty of disk > space, etc. > > > > Optimization code is: > > try { > > if (db:exists($database)) > > then > > ( > > util:logToConsole('dbadmin:optimizeDatabase', ``[Optimizing > database `{$database}`]``), > > db:optimize($database, true(), $dbadmin:dbOptimizeOptions) > > ) > > else util:logToConsole('dbadmin:optimizeDatabase', ``[Database > '`{$database}`' does not exist. Nothing to optimize.]``) > > } catch * { > > util:logToConsole( > > 'dbadmin:optimizeDatabase', > > ``[Exception optimizing database '`{$database}`': `{$err:code}` - > `{$err:description}`]``, > > 'error') > > } > > > > And the optimization options are: > > declare variable $dbadmin:dbOptimizeOptions as map(*) := > > (: Turn on all the indexes :) > > map { > > 'attrindex' : true(), > > 'tokenindex' : true(), > > 'textindex' : true(), > > 'ftindex' : true() > > }; > > This code has been working fine for a long time and I’ve been running 10.7 > for a least a couple of months, so I’m wondering: > > A) Would would cause this behavior? > B) How can I diagnose it short of debugging the Java code (which I can do > but it’s non-trivial for me to set up). > > > > Thanks, > > > > Eliot > > > > _____________________________________________ > > *Eliot Kimber* > > Sr Staff Content Engineer > > O: 512 554 9368 > > M: 512 554 9368 > > servicenow.com <https://www.servicenow.com> > > LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter > <https://twitter.com/servicenow> | YouTube > <https://www.youtube.com/user/servicenowinc> | Facebook > <https://www.facebook.com/servicenow> > >
CG
Christian Grün1 Mar 2024 10:15 #
…exactly; thanks, Andy. Time works against us, so we’ve added a link to the new documentation on the start page of docs.basex.org.
Quoted text · Andy Bunce · 397 lines
On Thu, Feb 29, 2024 at 11:15 AM Andy Bunce <bunce.andy at gmail.com> wrote: > Maybe: https://help.basex.org/main/Profiling_Module > > /Andy > > On Thu, 29 Feb 2024 at 09:40, Eliot Kimber <eliot.kimber at servicenow.com> > wrote: > >> Using the latest build, 11.0 beta 17d8426, the prof:dump() function is >> reported as an unknown function. >> >> >> >> What replaces it (or where can I find the V11 docs)? >> >> >> >> Thanks, >> >> >> >> E. >> >> >> >> >> >> _____________________________________________ >> >> *Eliot Kimber* >> >> Sr Staff Content Engineer >> >> O: 512 554 9368 >> >> M: 512 554 9368 >> >> servicenow.com <https://www.servicenow.com> >> >> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter >> <https://twitter.com/servicenow> | YouTube >> <https://www.youtube.com/user/servicenowinc> | Facebook >> <https://www.facebook.com/servicenow> >> >> >> >> *From: *Eliot Kimber <eliot.kimber at servicenow.com> >> *Date: *Thursday, February 29, 2024 at 3:15 AM >> *To: *Christian Grün <christian.gruen at gmail.com> >> *Cc: *basex-talk at mailman.uni-konstanz.de < >> basex-talk at mailman.uni-konstanz.de> >> *Subject: *Re: [basex-talk] Optimize database never returns, leaves >> database in "opened by another process" state >> >> Found the latest build at https://files.basex.org/releases/latest/ >> >> >> >> Cheers, >> >> >> >> E. >> >> >> >> _____________________________________________ >> >> *Eliot Kimber* >> >> Sr Staff Content Engineer >> >> O: 512 554 9368 >> >> M: 512 554 9368 >> >> servicenow.com <https://www.servicenow.com> >> >> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter >> <https://twitter.com/servicenow> | YouTube >> <https://www.youtube.com/user/servicenowinc> | Facebook >> <https://www.facebook.com/servicenow> >> >> >> >> *From: *Eliot Kimber <eliot.kimber at servicenow.com> >> *Date: *Thursday, February 29, 2024 at 2:45 AM >> *To: *Christian Grün <christian.gruen at gmail.com> >> *Cc: *basex-talk at mailman.uni-konstanz.de < >> basex-talk at mailman.uni-konstanz.de> >> *Subject: *Re: [basex-talk] Optimize database never returns, leaves >> database in "opened by another process" state >> >> I’m trying to compile the latest code but “mvn clean install” fails on >> failure to download some dependencies: >> >> [*INFO*] *--------------------------< *org.basex:basex* >> >---------------------------* >> >> [*INFO*] *Building BaseX Core 11.0-SNAPSHOT* >> >> [*INFO*] from pom.xml >> >> [*INFO*] *--------------------------------[ jar >> ]---------------------------------* >> >> Downloading from devsnc-mirror: >> http://nexus.proxy.devsnc.com/content/groups/public/jp/sourceforge/igo/igo/0.4.3/igo-0.4.3.pom >> >> [*WARNING*] The POM for jp.sourceforge.igo:igo:jar:0.4.3 is missing, no >> dependency information available >> >> Downloading from devsnc-mirror: >> http://nexus.proxy.devsnc.com/content/groups/public/org/apache/lucene-stemmers/3.4.0/lucene-stemmers-3.4.0.pom >> >> [*WARNING*] The POM for org.apache:lucene-stemmers:jar:3.4.0 is missing, >> no dependency information available >> >> Downloading from devsnc-mirror: >> http://nexus.proxy.devsnc.com/content/groups/public/jp/sourceforge/igo/igo/0.4.3/igo-0.4.3.jar >> >> Downloading from devsnc-mirror: >> http://nexus.proxy.devsnc.com/content/groups/public/org/apache/lucene-stemmers/3.4.0/lucene-stemmers-3.4.0.jar >> >> [*INFO*] >> *------------------------------------------------------------------------* >> >> [*INFO*] *BUILD FAILURE* >> >> [*INFO*] >> *------------------------------------------------------------------------* >> >> [*INFO*] Total time: 5.589 s >> >> [*INFO*] Finished at: 2024-02-29T02:34:11-06:00 >> >> [*INFO*] >> *------------------------------------------------------------------------* >> >> [*ERROR*] Failed to execute goal on project basex: *Could not resolve >> dependencies for project org.basex:basex:jar:11.0-SNAPSHOT: The following >> artifacts could not be resolved: jp.sourceforge.igo:igo:jar:0.4.3 (absent), >> org.apache:lucene-stemmers:jar:3.4.0 (absent): Could not find artifact >> jp.sourceforge.igo:igo:jar:0.4.3 in devsnc-mirror >> (http://nexus.proxy.devsnc.com/content/groups/public >> <http://nexus.proxy.devsnc.com/content/groups/public>)* -> *[Help 1]* >> >> >> >> And then: >> >> [*ERROR*] Failed to execute goal on project basex: *Could not resolve >> dependencies for project org.basex:basex:jar:11.0-SNAPSHOT: The following >> artifacts could not be resolved: jp.sourceforge.igo:igo:jar:0.4.3 (absent), >> org.apache:lucene-stemmers:jar:3.4.0 (absent): >> jp.sourceforge.igo:igo:jar:0.4.3 was not found in >> http://nexus.proxy.devsnc.com/content/groups/public >> <http://nexus.proxy.devsnc.com/content/groups/public> during a previous >> attempt. This failure was cached in the local repository and resolution is >> not reattempted until the update interval of devsnc-mirror has elapsed or >> updates are forced* -> *[Help 1]* >> >> >> >> Where ‘nexus.proxy.devsnc.com’ is our internal Maven proxy. >> >> This is after removing my local maven repository cache. >> >> Before doing that, I got the “present but not available” failure on these >> libraries. The jars are present in my original .m2/repository but not in a >> newly-created repository (having moved the original one out of the way). >> >> >> >> I’m using maven 3.9.6. >> >> >> >> As far as I can tell from my research, this is a change in behavior from >> Maven 2 to Maven 3 where it refuses to use local dependencies that were >> fetched as a side effect of some other dependency that cannot now be >> fetched. But I’m not finding any maven options that will work around this >> failure. >> >> >> >> Any idea how I can move past this? Or is there a nightly build somewhere >> I can download? >> >> >> >> Thanks, >> >> >> >> Eliot >> >> _____________________________________________ >> >> *Eliot Kimber* >> >> Sr Staff Content Engineer >> >> O: 512 554 9368 >> >> M: 512 554 9368 >> >> servicenow.com <https://www.servicenow.com> >> >> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter >> <https://twitter.com/servicenow> | YouTube >> <https://www.youtube.com/user/servicenowinc> | Facebook >> <https://www.facebook.com/servicenow> >> >> >> >> *From: *Eliot Kimber <eliot.kimber at servicenow.com> >> *Date: *Wednesday, February 28, 2024 at 10:01 AM >> *To: *Christian Grün <christian.gruen at gmail.com> >> *Cc: *basex-talk at mailman.uni-konstanz.de < >> basex-talk at mailman.uni-konstanz.de> >> *Subject: *Re: [basex-talk] Optimize database never returns, leaves >> database in "opened by another process" state >> >> I’ll check out the snapshot. >> >> I was starting to dig into the Java code yesterday but was first testing >> if I needed the optimization at all in this case, which it looks like I >> don’t (the consumer of the stored docs doesn’t depend on indexing to find >> things). >> >> >> >> Cheers, >> >> >> >> E. >> >> >> >> _____________________________________________ >> >> *Eliot Kimber* >> >> Sr Staff Content Engineer >> >> O: 512 554 9368 >> >> M: 512 554 9368 >> >> servicenow.com <https://www.servicenow.com> >> >> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter >> <https://twitter.com/servicenow> | YouTube >> <https://www.youtube.com/user/servicenowinc> | Facebook >> <https://www.facebook.com/servicenow> >> >> >> >> *From: *Christian Grün <christian.gruen at gmail.com> >> *Date: *Wednesday, February 28, 2024 at 9:36 AM >> *To: *Eliot Kimber <eliot.kimber at servicenow.com> >> *Cc: *basex-talk at mailman.uni-konstanz.de < >> basex-talk at mailman.uni-konstanz.de> >> *Subject: *Re: [basex-talk] Optimize database never returns, leaves >> database in "opened by another process" state >> *[External Email]* >> >> >> ------------------------------ >> >> …this one could be related to a bug that was recently fixed in the latest >> snapshot [1]. About time to get BaseX 11 finished… >> >> >> >> [1] >> https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496 >> >> >> >> >> >> >> >> On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber <eliot.kimber at servicenow.com> >> wrote: >> >> Using Basex 10.7 on Linux. >> >> >> I’m running a sequence of jobs to update and optimize a set of databases >> following loading a number of documents created dynamically (as opposed to >> being read from disk). >> >> >> >> I’m seeing a new behavior, which is that the optimization step never >> completes but also doesn’t show any error in the log. The database shows no >> items and is in the locked by another process state if I try to drop it. >> This behavior seems to be consistently repeatable with my current code base >> (I’m working on some code updates, so it’s possible I’ve introduced >> something that would cause this behavior but I haven’t changed the code >> that leads up to the failing optimize). The server has plenty of disk >> space, etc. >> >> >> >> Optimization code is: >> >> try { >> >> if (db:exists($database)) >> >> then >> >> ( >> >> util:logToConsole('dbadmin:optimizeDatabase', ``[Optimizing >> database `{$database}`]``), >> >> db:optimize($database, true(), $dbadmin:dbOptimizeOptions) >> >> ) >> >> else util:logToConsole('dbadmin:optimizeDatabase', ``[Database >> '`{$database}`' does not exist. Nothing to optimize.]``) >> >> } catch * { >> >> util:logToConsole( >> >> 'dbadmin:optimizeDatabase', >> >> ``[Exception optimizing database '`{$database}`': `{$err:code}` - >> `{$err:description}`]``, >> >> 'error') >> >> } >> >> >> >> And the optimization options are: >> >> declare variable $dbadmin:dbOptimizeOptions as map(*) := >> >> (: Turn on all the indexes :) >> >> map { >> >> 'attrindex' : true(), >> >> 'tokenindex' : true(), >> >> 'textindex' : true(), >> >> 'ftindex' : true() >> >> }; >> >> This code has been working fine for a long time and I’ve been running >> 10.7 for a least a couple of months, so I’m wondering: >> >> A) Would would cause this behavior? >> B) How can I diagnose it short of debugging the Java code (which I can do >> but it’s non-trivial for me to set up). >> >> >> >> Thanks, >> >> >> >> Eliot >> >> >> >> _____________________________________________ >> >> *Eliot Kimber* >> >> Sr Staff Content Engineer >> >> O: 512 554 9368 >> >> M: 512 554 9368 >> >> servicenow.com <https://www.servicenow.com> >> >> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter >> <https://twitter.com/servicenow> | YouTube >> <https://www.youtube.com/user/servicenowinc> | Facebook >> <https://www.facebook.com/servicenow> >> >>
EK
Using the 27-02-2024 build I have confirmed that the optimize database deadlock seems to be resolved.
I was able to easily upgrade my code to replace prof:dump() with message() and db:open() with
db:get() and everything else seems to be working as it should.
I like the new Editor replacement for the old Query feature in the DBA app.
Cheers,
E.
_____________________________________________
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>Quoted text · Christian Grün · 52 lines
From: Christian Grün <christian.gruen at gmail.com> Date: Wednesday, February 28, 2024 at 9:36 AM To: Eliot Kimber <eliot.kimber at servicenow.com> Cc: basex-talk at mailman.uni-konstanz.de <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state [External Email] …this one could be related to a bug that was recently fixed in the latest snapshot [1]. About time to get BaseX 11 finished…[1] https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber <eliot.kimber@servicenow i This message needs your attention • Someone new is on this email. Provided by ServiceNow DT (Employee Portal KB0077950) - This banner is visible only to ServiceNow employees.<https://mimecast.com> …this one could be related to a bug that was recently fixed in the latest snapshot [1]. About time to get BaseX 11 finished… [1] https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496 On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber <eliot.kimber at servicenow.com<mailto:eliot.kimber at servicenow.com>> wrote: Using Basex 10.7 on Linux. I’m running a sequence of jobs to update and optimize a set of databases following loading a number of documents created dynamically (as opposed to being read from disk). I’m seeing a new behavior, which is that the optimization step never completes but also doesn’t show any error in the log. The database shows no items and is in the locked by another process state if I try to drop it. This behavior seems to be consistently repeatable with my current code base (I’m working on some code updates, so it’s possible I’ve introduced something that would cause this behavior but I haven’t changed the code that leads up to the failing optimize). The server has plenty of disk space, etc. Optimization code is: try { if (db:exists($database)) then ( util:logToConsole('dbadmin:optimizeDatabase', ``[Optimizing database `{$database}`]``), db:optimize($database, true(), $dbadmin:dbOptimizeOptions) ) else util:logToConsole('dbadmin:optimizeDatabase', ``[Database '`{$database}`' does not exist. Nothing to optimize.]``) } catch * { util:logToConsole( 'dbadmin:optimizeDatabase', ``[Exception optimizing database '`{$database}`': `{$err:code}` - `{$err:description}`]``, 'error') } And the optimization options are: declare variable $dbadmin:dbOptimizeOptions as map(*) := (: Turn on all the indexes :) map { 'attrindex' : true(), 'tokenindex' : true(), 'textindex' : true(), 'ftindex' : true() }; This code has been working fine for a long time and I’ve been running 10.7 for a least a couple of months, so I’m wondering: A) Would would cause this behavior? B) How can I diagnose it short of debugging the Java code (which I can do but it’s non-trivial for me to set up). Thanks, Eliot Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>
CG
Christian Grün1 Mar 2024 16:30 #
Glad to hear it, thanks Eliot.
Eliot Kimber <eliot.kimber at servicenow.com> schrieb am Fr., 1. März 2024, 16:21:
Quoted text · Eliot Kimber · 179 lines
> Using the 27-02-2024 build I have confirmed that the optimize database > deadlock seems to be resolved. > > > > I was able to easily upgrade my code to replace prof:dump() with message() > and db:open() with db:get() and everything else seems to be working as it > should. > > > > I like the new Editor replacement for the old Query feature in the DBA app. > > > > Cheers, > > > > E. > > > > _____________________________________________ > > *Eliot Kimber* > > Sr Staff Content Engineer > > O: 512 554 9368 > > M: 512 554 9368 > > servicenow.com <https://www.servicenow.com> > > LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter > <https://twitter.com/servicenow> | YouTube > <https://www.youtube.com/user/servicenowinc> | Facebook > <https://www.facebook.com/servicenow> > > > > *From: *Christian Grün <christian.gruen at gmail.com> > *Date: *Wednesday, February 28, 2024 at 9:36 AM > *To: *Eliot Kimber <eliot.kimber at servicenow.com> > *Cc: *basex-talk at mailman.uni-konstanz.de < > basex-talk at mailman.uni-konstanz.de> > *Subject: *Re: [basex-talk] Optimize database never returns, leaves > database in "opened by another process" state > *[External Email]* > > > ------------------------------ > > …this one could be related to a bug that was recently fixed in the latest > snapshot [1]. About time to get BaseX 11 finished… > > > > [1] > https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496 > > > > > > > > On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber <eliot.kimber at servicenow.com> > wrote: > > Using Basex 10.7 on Linux. > > > I’m running a sequence of jobs to update and optimize a set of databases > following loading a number of documents created dynamically (as opposed to > being read from disk). > > > > I’m seeing a new behavior, which is that the optimization step never > completes but also doesn’t show any error in the log. The database shows no > items and is in the locked by another process state if I try to drop it. > This behavior seems to be consistently repeatable with my current code base > (I’m working on some code updates, so it’s possible I’ve introduced > something that would cause this behavior but I haven’t changed the code > that leads up to the failing optimize). The server has plenty of disk > space, etc. > > > > Optimization code is: > > try { > > if (db:exists($database)) > > then > > ( > > util:logToConsole('dbadmin:optimizeDatabase', ``[Optimizing > database `{$database}`]``), > > db:optimize($database, true(), $dbadmin:dbOptimizeOptions) > > ) > > else util:logToConsole('dbadmin:optimizeDatabase', ``[Database > '`{$database}`' does not exist. Nothing to optimize.]``) > > } catch * { > > util:logToConsole( > > 'dbadmin:optimizeDatabase', > > ``[Exception optimizing database '`{$database}`': `{$err:code}` - > `{$err:description}`]``, > > 'error') > > } > > > > And the optimization options are: > > declare variable $dbadmin:dbOptimizeOptions as map(*) := > > (: Turn on all the indexes :) > > map { > > 'attrindex' : true(), > > 'tokenindex' : true(), > > 'textindex' : true(), > > 'ftindex' : true() > > }; > > This code has been working fine for a long time and I’ve been running 10.7 > for a least a couple of months, so I’m wondering: > > A) Would would cause this behavior? > B) How can I diagnose it short of debugging the Java code (which I can do > but it’s non-trivial for me to set up). > > > > Thanks, > > > > Eliot > > > > _____________________________________________ > > *Eliot Kimber* > > Sr Staff Content Engineer > > O: 512 554 9368 > > M: 512 554 9368 > > servicenow.com <https://www.servicenow.com> > > LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter > <https://twitter.com/servicenow> | YouTube > <https://www.youtube.com/user/servicenowinc> | Facebook > <https://www.facebook.com/servicenow> > >
EK
Eliot Kimber4 Mar 2024 10:36 #
As reported in a separate thread, I’m still getting the optimization failure for large databases but not smaller ones. This behavior is consistent in my tests.
Cheers,
E.
_____________________________________________
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>
Quoted text · Christian Grün · 66 lines
From: Christian Grün <christian.gruen at gmail.com> Date: Friday, March 1, 2024 at 9:30 AM To: Eliot Kimber <eliot.kimber at servicenow.com> Cc: BaseX <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state [External Email] Glad to hear it, thanks Eliot. Eliot Kimber <eliot.kimber at servicenow.com<mailto:eliot.kimber at servicenow.com>> schrieb am Fr., 1. März 2024, 16:21: Using the 27-02-2024 build I have confirmed that the optimize database deadlock seems to be resolved. I was able to easily upgrade my code to replace prof:dump() with message() and db:open() with db:get() and everything else seems to be working as it should. I like the new Editor replacement for the old Query feature in the DBA app. Cheers, E. Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow> From: Christian Grün <christian.gruen at gmail.com<mailto:christian.gruen at gmail.com>> Date: Wednesday, February 28, 2024 at 9:36 AM To: Eliot Kimber <eliot.kimber at servicenow.com<mailto:eliot.kimber at servicenow.com>> Cc: basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> <basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de>> Subject: Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state [External Email] …this one could be related to a bug that was recently fixed in the latest snapshot [1]. About time to get BaseX 11 finished… [1] https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496 On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber <eliot.kimber at servicenow.com<mailto:eliot.kimber at servicenow.com>> wrote: Using Basex 10.7 on Linux. I’m running a sequence of jobs to update and optimize a set of databases following loading a number of documents created dynamically (as opposed to being read from disk). I’m seeing a new behavior, which is that the optimization step never completes but also doesn’t show any error in the log. The database shows no items and is in the locked by another process state if I try to drop it. This behavior seems to be consistently repeatable with my current code base (I’m working on some code updates, so it’s possible I’ve introduced something that would cause this behavior but I haven’t changed the code that leads up to the failing optimize). The server has plenty of disk space, etc. Optimization code is: try { if (db:exists($database)) then ( util:logToConsole('dbadmin:optimizeDatabase', ``[Optimizing database `{$database}`]``), db:optimize($database, true(), $dbadmin:dbOptimizeOptions) ) else util:logToConsole('dbadmin:optimizeDatabase', ``[Database '`{$database}`' does not exist. Nothing to optimize.]``) } catch * { util:logToConsole( 'dbadmin:optimizeDatabase', ``[Exception optimizing database '`{$database}`': `{$err:code}` - `{$err:description}`]``, 'error') } And the optimization options are: declare variable $dbadmin:dbOptimizeOptions as map(*) := (: Turn on all the indexes :) map { 'attrindex' : true(), 'tokenindex' : true(), 'textindex' : true(), 'ftindex' : true() }; This code has been working fine for a long time and I’ve been running 10.7 for a least a couple of months, so I’m wondering: A) Would would cause this behavior? B) How can I diagnose it short of debugging the Java code (which I can do but it’s non-trivial for me to set up). Thanks, Eliot Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>
Written for BaseX 10.7. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery