IllegalMonitorStateException
Tuesday, 23 September 2014 · 13 messages · last reply 7 Oct 2014
GJ
Gerald de Jong23 Sep 2014 12:10 #
Dear BaseX experts,
I'm wondering if anybody can see something that I'm doing wrong. I would expect it to be possible to add documents from multiple threads. Am I wrong about that??
When I set two of my datasets saving documents to BaseX 7.9 at the same time, I get this exception:
$ basexserver
BaseX 7.9 [Server]
Server was started (port: 1984)
Exception in thread "Thread-2295" java.lang.IllegalMonitorStateException at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) at
java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) at
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) at org.basex.core.DBLocking.release(DBLocking.java:219)
at org.basex.core.Context.unregister(Context.java:249)
at org.basex.core.Command.execute(Command.java:104)
at org.basex.core.Command.execute(Command.java:117)
at org.basex.server.ClientListener.execute(ClientListener.java:384) at org.basex.server.ClientListener.add(ClientListener.java:355) at org.basex.server.ClientListener.run(ClientListener.java:100)
All I am doing is making concurrent calls in two parallel sessions:
session = new ClientSession(...)
session.execute(new Open(database))
session.execute(new Set("autoflush", "false"))
session.add([path string], [document-bytes]) // a zillion times session.execute(new Flush())
During the execution of the parallel add commands, it crashes.
The relevant code:
def withSession[T](block: ClientSession => T): T = {
val session = new ClientSession(host, port, user, pass)
try {
block(session)
}
finally {
session.close()
}
} def withDbSession[T](database: String)(block: ClientSession => T): T = { withSession {
session =>
session.execute(new Open(database))
session.execute(new Set("autoflush", "false"))
val result = block(session)
session.execute(new Flush())
result
}
}
then:
withDbSession {
session =>
// many calls to session.add(...)
}--
Delving BV, Vasteland 8, Rotterdam
http://www.delving.eu
http://twitter.com/fluxe
skype: beautifulcode
+31629339805
CG
Christian Grün23 Sep 2014 12:46 #
...difficult to tell, as we don't have too much experience with Scala. Could you provide this example as Java code?
Quoted text · Gerald de Jong · 74 lines
On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong <gerald at delving.eu> wrote: > Dear BaseX experts, > > I'm wondering if anybody can see something that I'm doing wrong. I would > expect it to be possible to add documents from multiple threads. Am I wrong > about that?? > > When I set two of my datasets saving documents to BaseX 7.9 at the same > time, I get this exception: > > $ basexserver > BaseX 7.9 [Server] > Server was started (port: 1984) > Exception in thread "Thread-2295" java.lang.IllegalMonitorStateException > at > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) > at > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) > at org.basex.core.DBLocking.release(DBLocking.java:219) > at org.basex.core.Context.unregister(Context.java:249) > at org.basex.core.Command.execute(Command.java:104) > at org.basex.core.Command.execute(Command.java:117) > at org.basex.server.ClientListener.execute(ClientListener.java:384) > at org.basex.server.ClientListener.add(ClientListener.java:355) > at org.basex.server.ClientListener.run(ClientListener.java:100) > > All I am doing is making concurrent calls in two parallel sessions: > > session = new ClientSession(...) > session.execute(new Open(database)) > session.execute(new Set("autoflush", "false")) > session.add([path string], [document-bytes]) // a zillion times > session.execute(new Flush()) > > During the execution of the parallel add commands, it crashes. > > The relevant code: > > def withSession[T](block: ClientSession => T): T = { > val session = new ClientSession(host, port, user, pass) > try { > block(session) > } > finally { > session.close() > } > } > > def withDbSession[T](database: String)(block: ClientSession => T): T = { > withSession { > session => > session.execute(new Open(database)) > session.execute(new Set("autoflush", "false")) > val result = block(session) > session.execute(new Flush()) > result > } > } > > then: > > withDbSession { > session => > // many calls to session.add(...) > } > > -- > Delving BV, Vasteland 8, Rotterdam > http://www.delving.eu > http://twitter.com/fluxe > skype: beautifulcode > +31629339805
GJ
Gerald de Jong23 Sep 2014 13:17 #
The fact that it's scala should be insignificant, since I'm using the Java API.
What's happening is that two (or more) threads are doing this at the same time:
ClientSession session = new ClientSession(...);
session.execute(new Open(database));
session.execute(new Set("autoflush", "false"));
session.add([path string], [document-bytes]); // a zillion times
session.execute(new Flush());
session.close();My first question is of course if this should work parallel with no problems "in theory".
Quoted text · Christian Grün · 88 lines
On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün <christian.gruen at gmail.com> wrote: > ...difficult to tell, as we don't have too much experience with Scala. > Could you provide this example as Java code? > > > > On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong <gerald at delving.eu> > wrote: > > Dear BaseX experts, > > > > I'm wondering if anybody can see something that I'm doing wrong. I would > > expect it to be possible to add documents from multiple threads. Am I > wrong > > about that?? > > > > When I set two of my datasets saving documents to BaseX 7.9 at the same > > time, I get this exception: > > > > $ basexserver > > BaseX 7.9 [Server] > > Server was started (port: 1984) > > Exception in thread "Thread-2295" java.lang.IllegalMonitorStateException > > at > > > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) > > at > > > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) > > at > > > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) > > at org.basex.core.DBLocking.release(DBLocking.java:219) > > at org.basex.core.Context.unregister(Context.java:249) > > at org.basex.core.Command.execute(Command.java:104) > > at org.basex.core.Command.execute(Command.java:117) > > at org.basex.server.ClientListener.execute(ClientListener.java:384) > > at org.basex.server.ClientListener.add(ClientListener.java:355) > > at org.basex.server.ClientListener.run(ClientListener.java:100) > > > > All I am doing is making concurrent calls in two parallel sessions: > > > > session = new ClientSession(...) > > session.execute(new Open(database)) > > session.execute(new Set("autoflush", "false")) > > session.add([path string], [document-bytes]) // a zillion times > > session.execute(new Flush()) > > > > During the execution of the parallel add commands, it crashes. > > > > The relevant code: > > > > def withSession[T](block: ClientSession => T): T = { > > val session = new ClientSession(host, port, user, pass) > > try { > > block(session) > > } > > finally { > > session.close() > > } > > } > > > > def withDbSession[T](database: String)(block: ClientSession => T): T = > { > > withSession { > > session => > > session.execute(new Open(database)) > > session.execute(new Set("autoflush", "false")) > > val result = block(session) > > session.execute(new Flush()) > > result > > } > > } > > > > then: > > > > withDbSession { > > session => > > // many calls to session.add(...) > > } > > > > -- > > Delving BV, Vasteland 8, Rotterdam > > http://www.delving.eu > > http://twitter.com/fluxe > > skype: beautifulcode > > +31629339805 >
--
Delving BV, Vasteland 8, Rotterdam
http://www.delving.eu
http://twitter.com/fluxe
skype: beautifulcode
+31629339805
CG
Christian Grün23 Sep 2014 13:34 #
Quoted text · Gerald de Jong · 2 lines
> The fact that it's scala should be insignificant, since I'm using the Java > API.
Well, it might be theoretically insignificant, but we cannot give a quick answer on what's your problem. In general, parallel client requests are no problem, we have various JUnit tests doing exactly that (see e.g. [1], [2] or [3]).
[1] https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/basex/server/ConcurrencyTest.java [2] https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/RESTConcurrencyTest.java [3] https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/ServerStressTest.java
Quoted text · Gerald de Jong · 112 lines
> > What's happening is that two (or more) threads are doing this at the same > time: > > ClientSession session = new ClientSession(...); > session.execute(new Open(database)); > session.execute(new Set("autoflush", "false")); > session.add([path string], [document-bytes]); // a zillion times > session.execute(new Flush()); > session.close(); > > My first question is of course if this should work parallel with no problems > "in theory". > > On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> ...difficult to tell, as we don't have too much experience with Scala. >> Could you provide this example as Java code? >> >> >> >> On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong <gerald at delving.eu> >> wrote: >> > Dear BaseX experts, >> > >> > I'm wondering if anybody can see something that I'm doing wrong. I would >> > expect it to be possible to add documents from multiple threads. Am I >> > wrong >> > about that?? >> > >> > When I set two of my datasets saving documents to BaseX 7.9 at the same >> > time, I get this exception: >> > >> > $ basexserver >> > BaseX 7.9 [Server] >> > Server was started (port: 1984) >> > Exception in thread "Thread-2295" java.lang.IllegalMonitorStateException >> > at >> > >> > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) >> > at >> > >> > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) >> > at >> > >> > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) >> > at org.basex.core.DBLocking.release(DBLocking.java:219) >> > at org.basex.core.Context.unregister(Context.java:249) >> > at org.basex.core.Command.execute(Command.java:104) >> > at org.basex.core.Command.execute(Command.java:117) >> > at org.basex.server.ClientListener.execute(ClientListener.java:384) >> > at org.basex.server.ClientListener.add(ClientListener.java:355) >> > at org.basex.server.ClientListener.run(ClientListener.java:100) >> > >> > All I am doing is making concurrent calls in two parallel sessions: >> > >> > session = new ClientSession(...) >> > session.execute(new Open(database)) >> > session.execute(new Set("autoflush", "false")) >> > session.add([path string], [document-bytes]) // a zillion times >> > session.execute(new Flush()) >> > >> > During the execution of the parallel add commands, it crashes. >> > >> > The relevant code: >> > >> > def withSession[T](block: ClientSession => T): T = { >> > val session = new ClientSession(host, port, user, pass) >> > try { >> > block(session) >> > } >> > finally { >> > session.close() >> > } >> > } >> > >> > def withDbSession[T](database: String)(block: ClientSession => T): T = >> > { >> > withSession { >> > session => >> > session.execute(new Open(database)) >> > session.execute(new Set("autoflush", "false")) >> > val result = block(session) >> > session.execute(new Flush()) >> > result >> > } >> > } >> > >> > then: >> > >> > withDbSession { >> > session => >> > // many calls to session.add(...) >> > } >> > >> > -- >> > Delving BV, Vasteland 8, Rotterdam >> > http://www.delving.eu >> > http://twitter.com/fluxe >> > skype: beautifulcode >> > +31629339805 > > > > > -- > Delving BV, Vasteland 8, Rotterdam > http://www.delving.eu > http://twitter.com/fluxe > skype: beautifulcode > +31629339805
GJ
Gerald de Jong23 Sep 2014 13:46 #
Thanks Christian. I will try to find out how I can possibly cause an IllegalMonitorStateException.
Looking at these tests, it appears that none of them do large numbers of ADD commands. Did I miss something?
Quoted text · Christian Grün · 141 lines
On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün <christian.gruen at gmail.com> wrote: > > The fact that it's scala should be insignificant, since I'm using the > Java > > API. > > Well, it might be theoretically insignificant, but we cannot give a > quick answer on what's your problem. In general, parallel client > requests are no problem, we have various JUnit tests doing exactly > that (see e.g. [1], [2] or [3]). > > [1] > https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/basex/server/ConcurrencyTest.java > [2] > https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/RESTConcurrencyTest.java > [3] > https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/ServerStressTest.java > > > > > > What's happening is that two (or more) threads are doing this at the same > > time: > > > > ClientSession session = new ClientSession(...); > > session.execute(new Open(database)); > > session.execute(new Set("autoflush", "false")); > > session.add([path string], [document-bytes]); // a zillion times > > session.execute(new Flush()); > > session.close(); > > > > My first question is of course if this should work parallel with no > problems > > "in theory". > > > > On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün < > christian.gruen at gmail.com> > > wrote: > >> > >> ...difficult to tell, as we don't have too much experience with Scala. > >> Could you provide this example as Java code? > >> > >> > >> > >> On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong <gerald at delving.eu> > >> wrote: > >> > Dear BaseX experts, > >> > > >> > I'm wondering if anybody can see something that I'm doing wrong. I > would > >> > expect it to be possible to add documents from multiple threads. Am I > >> > wrong > >> > about that?? > >> > > >> > When I set two of my datasets saving documents to BaseX 7.9 at the > same > >> > time, I get this exception: > >> > > >> > $ basexserver > >> > BaseX 7.9 [Server] > >> > Server was started (port: 1984) > >> > Exception in thread "Thread-2295" > java.lang.IllegalMonitorStateException > >> > at > >> > > >> > > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) > >> > at > >> > > >> > > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) > >> > at > >> > > >> > > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) > >> > at org.basex.core.DBLocking.release(DBLocking.java:219) > >> > at org.basex.core.Context.unregister(Context.java:249) > >> > at org.basex.core.Command.execute(Command.java:104) > >> > at org.basex.core.Command.execute(Command.java:117) > >> > at org.basex.server.ClientListener.execute(ClientListener.java:384) > >> > at org.basex.server.ClientListener.add(ClientListener.java:355) > >> > at org.basex.server.ClientListener.run(ClientListener.java:100) > >> > > >> > All I am doing is making concurrent calls in two parallel sessions: > >> > > >> > session = new ClientSession(...) > >> > session.execute(new Open(database)) > >> > session.execute(new Set("autoflush", "false")) > >> > session.add([path string], [document-bytes]) // a zillion times > >> > session.execute(new Flush()) > >> > > >> > During the execution of the parallel add commands, it crashes. > >> > > >> > The relevant code: > >> > > >> > def withSession[T](block: ClientSession => T): T = { > >> > val session = new ClientSession(host, port, user, pass) > >> > try { > >> > block(session) > >> > } > >> > finally { > >> > session.close() > >> > } > >> > } > >> > > >> > def withDbSession[T](database: String)(block: ClientSession => T): > T = > >> > { > >> > withSession { > >> > session => > >> > session.execute(new Open(database)) > >> > session.execute(new Set("autoflush", "false")) > >> > val result = block(session) > >> > session.execute(new Flush()) > >> > result > >> > } > >> > } > >> > > >> > then: > >> > > >> > withDbSession { > >> > session => > >> > // many calls to session.add(...) > >> > } > >> > > >> > -- > >> > Delving BV, Vasteland 8, Rotterdam > >> > http://www.delving.eu > >> > http://twitter.com/fluxe > >> > skype: beautifulcode > >> > +31629339805 > > > > > > > > > > -- > > Delving BV, Vasteland 8, Rotterdam > > http://www.delving.eu > > http://twitter.com/fluxe > > skype: beautifulcode > > +31629339805 >
--
Delving BV, Vasteland 8, Rotterdam
http://www.delving.eu
http://twitter.com/fluxe
skype: beautifulcode
+31629339805
CG
Christian Grün23 Sep 2014 13:53 #
Quoted text · Gerald de Jong · 2 lines
> Looking at these tests, it appears that none of them do large numbers of ADD > commands. Did I miss something?
You may need to check the other tests. But as the (updating) add command will never be executed in parallel, it shouldn't cause any problems with parallelization (feel free to tell me if I'm wrong).
Quoted text · Gerald de Jong · 154 lines
> > On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> > The fact that it's scala should be insignificant, since I'm using the >> > Java >> > API. >> >> Well, it might be theoretically insignificant, but we cannot give a >> quick answer on what's your problem. In general, parallel client >> requests are no problem, we have various JUnit tests doing exactly >> that (see e.g. [1], [2] or [3]). >> >> [1] >> https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/basex/server/ConcurrencyTest.java >> [2] >> https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/RESTConcurrencyTest.java >> [3] >> https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/ServerStressTest.java >> >> >> > >> > What's happening is that two (or more) threads are doing this at the >> > same >> > time: >> > >> > ClientSession session = new ClientSession(...); >> > session.execute(new Open(database)); >> > session.execute(new Set("autoflush", "false")); >> > session.add([path string], [document-bytes]); // a zillion times >> > session.execute(new Flush()); >> > session.close(); >> > >> > My first question is of course if this should work parallel with no >> > problems >> > "in theory". >> > >> > On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün >> > <christian.gruen at gmail.com> >> > wrote: >> >> >> >> ...difficult to tell, as we don't have too much experience with Scala. >> >> Could you provide this example as Java code? >> >> >> >> >> >> >> >> On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong <gerald at delving.eu> >> >> wrote: >> >> > Dear BaseX experts, >> >> > >> >> > I'm wondering if anybody can see something that I'm doing wrong. I >> >> > would >> >> > expect it to be possible to add documents from multiple threads. Am >> >> > I >> >> > wrong >> >> > about that?? >> >> > >> >> > When I set two of my datasets saving documents to BaseX 7.9 at the >> >> > same >> >> > time, I get this exception: >> >> > >> >> > $ basexserver >> >> > BaseX 7.9 [Server] >> >> > Server was started (port: 1984) >> >> > Exception in thread "Thread-2295" >> >> > java.lang.IllegalMonitorStateException >> >> > at >> >> > >> >> > >> >> > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) >> >> > at >> >> > >> >> > >> >> > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) >> >> > at >> >> > >> >> > >> >> > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) >> >> > at org.basex.core.DBLocking.release(DBLocking.java:219) >> >> > at org.basex.core.Context.unregister(Context.java:249) >> >> > at org.basex.core.Command.execute(Command.java:104) >> >> > at org.basex.core.Command.execute(Command.java:117) >> >> > at org.basex.server.ClientListener.execute(ClientListener.java:384) >> >> > at org.basex.server.ClientListener.add(ClientListener.java:355) >> >> > at org.basex.server.ClientListener.run(ClientListener.java:100) >> >> > >> >> > All I am doing is making concurrent calls in two parallel sessions: >> >> > >> >> > session = new ClientSession(...) >> >> > session.execute(new Open(database)) >> >> > session.execute(new Set("autoflush", "false")) >> >> > session.add([path string], [document-bytes]) // a zillion times >> >> > session.execute(new Flush()) >> >> > >> >> > During the execution of the parallel add commands, it crashes. >> >> > >> >> > The relevant code: >> >> > >> >> > def withSession[T](block: ClientSession => T): T = { >> >> > val session = new ClientSession(host, port, user, pass) >> >> > try { >> >> > block(session) >> >> > } >> >> > finally { >> >> > session.close() >> >> > } >> >> > } >> >> > >> >> > def withDbSession[T](database: String)(block: ClientSession => T): >> >> > T = >> >> > { >> >> > withSession { >> >> > session => >> >> > session.execute(new Open(database)) >> >> > session.execute(new Set("autoflush", "false")) >> >> > val result = block(session) >> >> > session.execute(new Flush()) >> >> > result >> >> > } >> >> > } >> >> > >> >> > then: >> >> > >> >> > withDbSession { >> >> > session => >> >> > // many calls to session.add(...) >> >> > } >> >> > >> >> > -- >> >> > Delving BV, Vasteland 8, Rotterdam >> >> > http://www.delving.eu >> >> > http://twitter.com/fluxe >> >> > skype: beautifulcode >> >> > +31629339805 >> > >> > >> > >> > >> > -- >> > Delving BV, Vasteland 8, Rotterdam >> > http://www.delving.eu >> > http://twitter.com/fluxe >> > skype: beautifulcode >> > +31629339805 > > > > > -- > Delving BV, Vasteland 8, Rotterdam > http://www.delving.eu > http://twitter.com/fluxe > skype: beautifulcode > +31629339805
GJ
Gerald de Jong23 Sep 2014 14:15 #
Can I start up the basexserver with some kind of logging so I can see what leads to the IllegalMonitorStateException?
Quoted text · Christian Grün · 177 lines
On Tue, Sep 23, 2014 at 1:53 PM, Christian Grün <christian.gruen at gmail.com> wrote: > > Looking at these tests, it appears that none of them do large numbers of > ADD > > commands. Did I miss something? > > You may need to check the other tests. But as the (updating) add > command will never be executed in parallel, it shouldn't cause any > problems with parallelization (feel free to tell me if I'm wrong). > > > > > > On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün < > christian.gruen at gmail.com> > > wrote: > >> > >> > The fact that it's scala should be insignificant, since I'm using the > >> > Java > >> > API. > >> > >> Well, it might be theoretically insignificant, but we cannot give a > >> quick answer on what's your problem. In general, parallel client > >> requests are no problem, we have various JUnit tests doing exactly > >> that (see e.g. [1], [2] or [3]). > >> > >> [1] > >> > https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/basex/server/ConcurrencyTest.java > >> [2] > >> > https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/RESTConcurrencyTest.java > >> [3] > >> > https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/ServerStressTest.java > >> > >> > >> > > >> > What's happening is that two (or more) threads are doing this at the > >> > same > >> > time: > >> > > >> > ClientSession session = new ClientSession(...); > >> > session.execute(new Open(database)); > >> > session.execute(new Set("autoflush", "false")); > >> > session.add([path string], [document-bytes]); // a zillion times > >> > session.execute(new Flush()); > >> > session.close(); > >> > > >> > My first question is of course if this should work parallel with no > >> > problems > >> > "in theory". > >> > > >> > On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün > >> > <christian.gruen at gmail.com> > >> > wrote: > >> >> > >> >> ...difficult to tell, as we don't have too much experience with > Scala. > >> >> Could you provide this example as Java code? > >> >> > >> >> > >> >> > >> >> On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong <gerald at delving.eu> > >> >> wrote: > >> >> > Dear BaseX experts, > >> >> > > >> >> > I'm wondering if anybody can see something that I'm doing wrong. I > >> >> > would > >> >> > expect it to be possible to add documents from multiple threads. > Am > >> >> > I > >> >> > wrong > >> >> > about that?? > >> >> > > >> >> > When I set two of my datasets saving documents to BaseX 7.9 at the > >> >> > same > >> >> > time, I get this exception: > >> >> > > >> >> > $ basexserver > >> >> > BaseX 7.9 [Server] > >> >> > Server was started (port: 1984) > >> >> > Exception in thread "Thread-2295" > >> >> > java.lang.IllegalMonitorStateException > >> >> > at > >> >> > > >> >> > > >> >> > > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) > >> >> > at > >> >> > > >> >> > > >> >> > > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) > >> >> > at > >> >> > > >> >> > > >> >> > > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) > >> >> > at org.basex.core.DBLocking.release(DBLocking.java:219) > >> >> > at org.basex.core.Context.unregister(Context.java:249) > >> >> > at org.basex.core.Command.execute(Command.java:104) > >> >> > at org.basex.core.Command.execute(Command.java:117) > >> >> > at org.basex.server.ClientListener.execute(ClientListener.java:384) > >> >> > at org.basex.server.ClientListener.add(ClientListener.java:355) > >> >> > at org.basex.server.ClientListener.run(ClientListener.java:100) > >> >> > > >> >> > All I am doing is making concurrent calls in two parallel sessions: > >> >> > > >> >> > session = new ClientSession(...) > >> >> > session.execute(new Open(database)) > >> >> > session.execute(new Set("autoflush", "false")) > >> >> > session.add([path string], [document-bytes]) // a zillion > times > >> >> > session.execute(new Flush()) > >> >> > > >> >> > During the execution of the parallel add commands, it crashes. > >> >> > > >> >> > The relevant code: > >> >> > > >> >> > def withSession[T](block: ClientSession => T): T = { > >> >> > val session = new ClientSession(host, port, user, pass) > >> >> > try { > >> >> > block(session) > >> >> > } > >> >> > finally { > >> >> > session.close() > >> >> > } > >> >> > } > >> >> > > >> >> > def withDbSession[T](database: String)(block: ClientSession => > T): > >> >> > T = > >> >> > { > >> >> > withSession { > >> >> > session => > >> >> > session.execute(new Open(database)) > >> >> > session.execute(new Set("autoflush", "false")) > >> >> > val result = block(session) > >> >> > session.execute(new Flush()) > >> >> > result > >> >> > } > >> >> > } > >> >> > > >> >> > then: > >> >> > > >> >> > withDbSession { > >> >> > session => > >> >> > // many calls to session.add(...) > >> >> > } > >> >> > > >> >> > -- > >> >> > Delving BV, Vasteland 8, Rotterdam > >> >> > http://www.delving.eu > >> >> > http://twitter.com/fluxe > >> >> > skype: beautifulcode > >> >> > +31629339805 > >> > > >> > > >> > > >> > > >> > -- > >> > Delving BV, Vasteland 8, Rotterdam > >> > http://www.delving.eu > >> > http://twitter.com/fluxe > >> > skype: beautifulcode > >> > +31629339805 > > > > > > > > > > -- > > Delving BV, Vasteland 8, Rotterdam > > http://www.delving.eu > > http://twitter.com/fluxe > > skype: beautifulcode > > +31629339805 >
--
Delving BV, Vasteland 8, Rotterdam
http://www.delving.eu
http://twitter.com/fluxe
skype: beautifulcode
+31629339805
CG
Christian Grün23 Sep 2014 14:17 #
You can set the debugging flag and check out the database log files to get more insight into what's going on.
Quoted text · Gerald de Jong · 193 lines
On Tue, Sep 23, 2014 at 2:15 PM, Gerald de Jong <gerald at delving.eu> wrote: > Can I start up the basexserver with some kind of logging so I can see what > leads to the IllegalMonitorStateException? > > On Tue, Sep 23, 2014 at 1:53 PM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> > Looking at these tests, it appears that none of them do large numbers of >> > ADD >> > commands. Did I miss something? >> >> You may need to check the other tests. But as the (updating) add >> command will never be executed in parallel, it shouldn't cause any >> problems with parallelization (feel free to tell me if I'm wrong). >> >> >> > >> > On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün >> > <christian.gruen at gmail.com> >> > wrote: >> >> >> >> > The fact that it's scala should be insignificant, since I'm using the >> >> > Java >> >> > API. >> >> >> >> Well, it might be theoretically insignificant, but we cannot give a >> >> quick answer on what's your problem. In general, parallel client >> >> requests are no problem, we have various JUnit tests doing exactly >> >> that (see e.g. [1], [2] or [3]). >> >> >> >> [1] >> >> >> >> https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/basex/server/ConcurrencyTest.java >> >> [2] >> >> >> >> https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/RESTConcurrencyTest.java >> >> [3] >> >> >> >> https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/ServerStressTest.java >> >> >> >> >> >> > >> >> > What's happening is that two (or more) threads are doing this at the >> >> > same >> >> > time: >> >> > >> >> > ClientSession session = new ClientSession(...); >> >> > session.execute(new Open(database)); >> >> > session.execute(new Set("autoflush", "false")); >> >> > session.add([path string], [document-bytes]); // a zillion times >> >> > session.execute(new Flush()); >> >> > session.close(); >> >> > >> >> > My first question is of course if this should work parallel with no >> >> > problems >> >> > "in theory". >> >> > >> >> > On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün >> >> > <christian.gruen at gmail.com> >> >> > wrote: >> >> >> >> >> >> ...difficult to tell, as we don't have too much experience with >> >> >> Scala. >> >> >> Could you provide this example as Java code? >> >> >> >> >> >> >> >> >> >> >> >> On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong <gerald at delving.eu> >> >> >> wrote: >> >> >> > Dear BaseX experts, >> >> >> > >> >> >> > I'm wondering if anybody can see something that I'm doing wrong. I >> >> >> > would >> >> >> > expect it to be possible to add documents from multiple threads. >> >> >> > Am >> >> >> > I >> >> >> > wrong >> >> >> > about that?? >> >> >> > >> >> >> > When I set two of my datasets saving documents to BaseX 7.9 at the >> >> >> > same >> >> >> > time, I get this exception: >> >> >> > >> >> >> > $ basexserver >> >> >> > BaseX 7.9 [Server] >> >> >> > Server was started (port: 1984) >> >> >> > Exception in thread "Thread-2295" >> >> >> > java.lang.IllegalMonitorStateException >> >> >> > at >> >> >> > >> >> >> > >> >> >> > >> >> >> > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) >> >> >> > at >> >> >> > >> >> >> > >> >> >> > >> >> >> > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) >> >> >> > at >> >> >> > >> >> >> > >> >> >> > >> >> >> > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) >> >> >> > at org.basex.core.DBLocking.release(DBLocking.java:219) >> >> >> > at org.basex.core.Context.unregister(Context.java:249) >> >> >> > at org.basex.core.Command.execute(Command.java:104) >> >> >> > at org.basex.core.Command.execute(Command.java:117) >> >> >> > at >> >> >> > org.basex.server.ClientListener.execute(ClientListener.java:384) >> >> >> > at org.basex.server.ClientListener.add(ClientListener.java:355) >> >> >> > at org.basex.server.ClientListener.run(ClientListener.java:100) >> >> >> > >> >> >> > All I am doing is making concurrent calls in two parallel >> >> >> > sessions: >> >> >> > >> >> >> > session = new ClientSession(...) >> >> >> > session.execute(new Open(database)) >> >> >> > session.execute(new Set("autoflush", "false")) >> >> >> > session.add([path string], [document-bytes]) // a zillion >> >> >> > times >> >> >> > session.execute(new Flush()) >> >> >> > >> >> >> > During the execution of the parallel add commands, it crashes. >> >> >> > >> >> >> > The relevant code: >> >> >> > >> >> >> > def withSession[T](block: ClientSession => T): T = { >> >> >> > val session = new ClientSession(host, port, user, pass) >> >> >> > try { >> >> >> > block(session) >> >> >> > } >> >> >> > finally { >> >> >> > session.close() >> >> >> > } >> >> >> > } >> >> >> > >> >> >> > def withDbSession[T](database: String)(block: ClientSession => >> >> >> > T): >> >> >> > T = >> >> >> > { >> >> >> > withSession { >> >> >> > session => >> >> >> > session.execute(new Open(database)) >> >> >> > session.execute(new Set("autoflush", "false")) >> >> >> > val result = block(session) >> >> >> > session.execute(new Flush()) >> >> >> > result >> >> >> > } >> >> >> > } >> >> >> > >> >> >> > then: >> >> >> > >> >> >> > withDbSession { >> >> >> > session => >> >> >> > // many calls to session.add(...) >> >> >> > } >> >> >> > >> >> >> > -- >> >> >> > Delving BV, Vasteland 8, Rotterdam >> >> >> > http://www.delving.eu >> >> >> > http://twitter.com/fluxe >> >> >> > skype: beautifulcode >> >> >> > +31629339805 >> >> > >> >> > >> >> > >> >> > >> >> > -- >> >> > Delving BV, Vasteland 8, Rotterdam >> >> > http://www.delving.eu >> >> > http://twitter.com/fluxe >> >> > skype: beautifulcode >> >> > +31629339805 >> > >> > >> > >> > >> > -- >> > Delving BV, Vasteland 8, Rotterdam >> > http://www.delving.eu >> > http://twitter.com/fluxe >> > skype: beautifulcode >> > +31629339805 > > > > > -- > Delving BV, Vasteland 8, Rotterdam > http://www.delving.eu > http://twitter.com/fluxe > skype: beautifulcode > +31629339805
GJ
Gerald de Jong24 Sep 2014 17:41 #
It happened again. Could I not argue that nothing I can do as a client to BaseX should be able to freeze it? I mean, even if, for some reason, things are getting muddled up.
Exception in thread "Thread-56391" java.lang.IllegalMonitorStateException at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) at
java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) at
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) at org.basex.core.DBLocking.release(DBLocking.java:219)
at org.basex.core.Context.unregister(Context.java:249)
at org.basex.core.Command.execute(Command.java:104)
Quoted text · Christian Grün · 215 lines
On Tue, Sep 23, 2014 at 2:17 PM, Christian Grün <christian.gruen at gmail.com> wrote: > You can set the debugging flag and check out the database log files to > get more insight into what's going on. > > > On Tue, Sep 23, 2014 at 2:15 PM, Gerald de Jong <gerald at delving.eu> wrote: > > Can I start up the basexserver with some kind of logging so I can see > what > > leads to the IllegalMonitorStateException? > > > > On Tue, Sep 23, 2014 at 1:53 PM, Christian Grün < > christian.gruen at gmail.com> > > wrote: > >> > >> > Looking at these tests, it appears that none of them do large numbers > of > >> > ADD > >> > commands. Did I miss something? > >> > >> You may need to check the other tests. But as the (updating) add > >> command will never be executed in parallel, it shouldn't cause any > >> problems with parallelization (feel free to tell me if I'm wrong). > >> > >> > >> > > >> > On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün > >> > <christian.gruen at gmail.com> > >> > wrote: > >> >> > >> >> > The fact that it's scala should be insignificant, since I'm using > the > >> >> > Java > >> >> > API. > >> >> > >> >> Well, it might be theoretically insignificant, but we cannot give a > >> >> quick answer on what's your problem. In general, parallel client > >> >> requests are no problem, we have various JUnit tests doing exactly > >> >> that (see e.g. [1], [2] or [3]). > >> >> > >> >> [1] > >> >> > >> >> > https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/basex/server/ConcurrencyTest.java > >> >> [2] > >> >> > >> >> > https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/RESTConcurrencyTest.java > >> >> [3] > >> >> > >> >> > https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/ServerStressTest.java > >> >> > >> >> > >> >> > > >> >> > What's happening is that two (or more) threads are doing this at > the > >> >> > same > >> >> > time: > >> >> > > >> >> > ClientSession session = new ClientSession(...); > >> >> > session.execute(new Open(database)); > >> >> > session.execute(new Set("autoflush", "false")); > >> >> > session.add([path string], [document-bytes]); // a zillion > times > >> >> > session.execute(new Flush()); > >> >> > session.close(); > >> >> > > >> >> > My first question is of course if this should work parallel with no > >> >> > problems > >> >> > "in theory". > >> >> > > >> >> > On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün > >> >> > <christian.gruen at gmail.com> > >> >> > wrote: > >> >> >> > >> >> >> ...difficult to tell, as we don't have too much experience with > >> >> >> Scala. > >> >> >> Could you provide this example as Java code? > >> >> >> > >> >> >> > >> >> >> > >> >> >> On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong < > gerald at delving.eu> > >> >> >> wrote: > >> >> >> > Dear BaseX experts, > >> >> >> > > >> >> >> > I'm wondering if anybody can see something that I'm doing > wrong. I > >> >> >> > would > >> >> >> > expect it to be possible to add documents from multiple threads. > >> >> >> > Am > >> >> >> > I > >> >> >> > wrong > >> >> >> > about that?? > >> >> >> > > >> >> >> > When I set two of my datasets saving documents to BaseX 7.9 at > the > >> >> >> > same > >> >> >> > time, I get this exception: > >> >> >> > > >> >> >> > $ basexserver > >> >> >> > BaseX 7.9 [Server] > >> >> >> > Server was started (port: 1984) > >> >> >> > Exception in thread "Thread-2295" > >> >> >> > java.lang.IllegalMonitorStateException > >> >> >> > at > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) > >> >> >> > at > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) > >> >> >> > at > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) > >> >> >> > at org.basex.core.DBLocking.release(DBLocking.java:219) > >> >> >> > at org.basex.core.Context.unregister(Context.java:249) > >> >> >> > at org.basex.core.Command.execute(Command.java:104) > >> >> >> > at org.basex.core.Command.execute(Command.java:117) > >> >> >> > at > >> >> >> > org.basex.server.ClientListener.execute(ClientListener.java:384) > >> >> >> > at org.basex.server.ClientListener.add(ClientListener.java:355) > >> >> >> > at org.basex.server.ClientListener.run(ClientListener.java:100) > >> >> >> > > >> >> >> > All I am doing is making concurrent calls in two parallel > >> >> >> > sessions: > >> >> >> > > >> >> >> > session = new ClientSession(...) > >> >> >> > session.execute(new Open(database)) > >> >> >> > session.execute(new Set("autoflush", "false")) > >> >> >> > session.add([path string], [document-bytes]) // a zillion > >> >> >> > times > >> >> >> > session.execute(new Flush()) > >> >> >> > > >> >> >> > During the execution of the parallel add commands, it crashes. > >> >> >> > > >> >> >> > The relevant code: > >> >> >> > > >> >> >> > def withSession[T](block: ClientSession => T): T = { > >> >> >> > val session = new ClientSession(host, port, user, pass) > >> >> >> > try { > >> >> >> > block(session) > >> >> >> > } > >> >> >> > finally { > >> >> >> > session.close() > >> >> >> > } > >> >> >> > } > >> >> >> > > >> >> >> > def withDbSession[T](database: String)(block: ClientSession => > >> >> >> > T): > >> >> >> > T = > >> >> >> > { > >> >> >> > withSession { > >> >> >> > session => > >> >> >> > session.execute(new Open(database)) > >> >> >> > session.execute(new Set("autoflush", "false")) > >> >> >> > val result = block(session) > >> >> >> > session.execute(new Flush()) > >> >> >> > result > >> >> >> > } > >> >> >> > } > >> >> >> > > >> >> >> > then: > >> >> >> > > >> >> >> > withDbSession { > >> >> >> > session => > >> >> >> > // many calls to session.add(...) > >> >> >> > } > >> >> >> > > >> >> >> > -- > >> >> >> > Delving BV, Vasteland 8, Rotterdam > >> >> >> > http://www.delving.eu > >> >> >> > http://twitter.com/fluxe > >> >> >> > skype: beautifulcode > >> >> >> > +31629339805 > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > -- > >> >> > Delving BV, Vasteland 8, Rotterdam > >> >> > http://www.delving.eu > >> >> > http://twitter.com/fluxe > >> >> > skype: beautifulcode > >> >> > +31629339805 > >> > > >> > > >> > > >> > > >> > -- > >> > Delving BV, Vasteland 8, Rotterdam > >> > http://www.delving.eu > >> > http://twitter.com/fluxe > >> > skype: beautifulcode > >> > +31629339805 > > > > > > > > > > -- > > Delving BV, Vasteland 8, Rotterdam > > http://www.delving.eu > > http://twitter.com/fluxe > > skype: beautifulcode > > +31629339805 >
--
Delving BV, Vasteland 8, Rotterdam
http://www.delving.eu
http://twitter.com/fluxe
skype: beautifulcode
+31629339805
DK
Dirk Kirsten26 Sep 2014 11:59 #
Hello Gerald,
I completely agree. I guess, one could consider this a bug (but of course, by definition, BaseX does not has bugs; it's hidden features). Seems like something is going wrong when releasing the write lock. Could you please test it with our latest snapshot, to see if the issue was already fixed?
Christian is currently on vacation, so an actual bug fix could take some more time than usual. If I have time, I'll try to look into it.
Cheers,
Dirk
Quoted text · Gerald de Jong · 236 lines
On 09/24/2014 05:41 PM, Gerald de Jong wrote: > It happened again. Could I not argue that nothing I can do as a client to > BaseX should be able to freeze it? I mean, even if, for some reason, > things are getting muddled up. > > Exception in thread "Thread-56391" java.lang.IllegalMonitorStateException > at > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) > at > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) > at org.basex.core.DBLocking.release(DBLocking.java:219) > at org.basex.core.Context.unregister(Context.java:249) > at org.basex.core.Command.execute(Command.java:104) > > > On Tue, Sep 23, 2014 at 2:17 PM, Christian Grün <christian.gruen at gmail.com> > wrote: > >> You can set the debugging flag and check out the database log files to >> get more insight into what's going on. >> >> >> On Tue, Sep 23, 2014 at 2:15 PM, Gerald de Jong <gerald at delving.eu> wrote: >>> Can I start up the basexserver with some kind of logging so I can see >> what >>> leads to the IllegalMonitorStateException? >>> >>> On Tue, Sep 23, 2014 at 1:53 PM, Christian Grün < >> christian.gruen at gmail.com> >>> wrote: >>>> >>>>> Looking at these tests, it appears that none of them do large numbers >> of >>>>> ADD >>>>> commands. Did I miss something? >>>> >>>> You may need to check the other tests. But as the (updating) add >>>> command will never be executed in parallel, it shouldn't cause any >>>> problems with parallelization (feel free to tell me if I'm wrong). >>>> >>>> >>>>> >>>>> On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün >>>>> <christian.gruen at gmail.com> >>>>> wrote: >>>>>> >>>>>>> The fact that it's scala should be insignificant, since I'm using >> the >>>>>>> Java >>>>>>> API. >>>>>> >>>>>> Well, it might be theoretically insignificant, but we cannot give a >>>>>> quick answer on what's your problem. In general, parallel client >>>>>> requests are no problem, we have various JUnit tests doing exactly >>>>>> that (see e.g. [1], [2] or [3]). >>>>>> >>>>>> [1] >>>>>> >>>>>> >> https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/basex/server/ConcurrencyTest.java >>>>>> [2] >>>>>> >>>>>> >> https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/RESTConcurrencyTest.java >>>>>> [3] >>>>>> >>>>>> >> https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/ServerStressTest.java >>>>>> >>>>>> >>>>>>> >>>>>>> What's happening is that two (or more) threads are doing this at >> the >>>>>>> same >>>>>>> time: >>>>>>> >>>>>>> ClientSession session = new ClientSession(...); >>>>>>> session.execute(new Open(database)); >>>>>>> session.execute(new Set("autoflush", "false")); >>>>>>> session.add([path string], [document-bytes]); // a zillion >> times >>>>>>> session.execute(new Flush()); >>>>>>> session.close(); >>>>>>> >>>>>>> My first question is of course if this should work parallel with no >>>>>>> problems >>>>>>> "in theory". >>>>>>> >>>>>>> On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün >>>>>>> <christian.gruen at gmail.com> >>>>>>> wrote: >>>>>>>> >>>>>>>> ...difficult to tell, as we don't have too much experience with >>>>>>>> Scala. >>>>>>>> Could you provide this example as Java code? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong < >> gerald at delving.eu> >>>>>>>> wrote: >>>>>>>>> Dear BaseX experts, >>>>>>>>> >>>>>>>>> I'm wondering if anybody can see something that I'm doing >> wrong. I >>>>>>>>> would >>>>>>>>> expect it to be possible to add documents from multiple threads. >>>>>>>>> Am >>>>>>>>> I >>>>>>>>> wrong >>>>>>>>> about that?? >>>>>>>>> >>>>>>>>> When I set two of my datasets saving documents to BaseX 7.9 at >> the >>>>>>>>> same >>>>>>>>> time, I get this exception: >>>>>>>>> >>>>>>>>> $ basexserver >>>>>>>>> BaseX 7.9 [Server] >>>>>>>>> Server was started (port: 1984) >>>>>>>>> Exception in thread "Thread-2295" >>>>>>>>> java.lang.IllegalMonitorStateException >>>>>>>>> at >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) >>>>>>>>> at >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >> java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) >>>>>>>>> at >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) >>>>>>>>> at org.basex.core.DBLocking.release(DBLocking.java:219) >>>>>>>>> at org.basex.core.Context.unregister(Context.java:249) >>>>>>>>> at org.basex.core.Command.execute(Command.java:104) >>>>>>>>> at org.basex.core.Command.execute(Command.java:117) >>>>>>>>> at >>>>>>>>> org.basex.server.ClientListener.execute(ClientListener.java:384) >>>>>>>>> at org.basex.server.ClientListener.add(ClientListener.java:355) >>>>>>>>> at org.basex.server.ClientListener.run(ClientListener.java:100) >>>>>>>>> >>>>>>>>> All I am doing is making concurrent calls in two parallel >>>>>>>>> sessions: >>>>>>>>> >>>>>>>>> session = new ClientSession(...) >>>>>>>>> session.execute(new Open(database)) >>>>>>>>> session.execute(new Set("autoflush", "false")) >>>>>>>>> session.add([path string], [document-bytes]) // a zillion >>>>>>>>> times >>>>>>>>> session.execute(new Flush()) >>>>>>>>> >>>>>>>>> During the execution of the parallel add commands, it crashes. >>>>>>>>> >>>>>>>>> The relevant code: >>>>>>>>> >>>>>>>>> def withSession[T](block: ClientSession => T): T = { >>>>>>>>> val session = new ClientSession(host, port, user, pass) >>>>>>>>> try { >>>>>>>>> block(session) >>>>>>>>> } >>>>>>>>> finally { >>>>>>>>> session.close() >>>>>>>>> } >>>>>>>>> } >>>>>>>>> >>>>>>>>> def withDbSession[T](database: String)(block: ClientSession => >>>>>>>>> T): >>>>>>>>> T = >>>>>>>>> { >>>>>>>>> withSession { >>>>>>>>> session => >>>>>>>>> session.execute(new Open(database)) >>>>>>>>> session.execute(new Set("autoflush", "false")) >>>>>>>>> val result = block(session) >>>>>>>>> session.execute(new Flush()) >>>>>>>>> result >>>>>>>>> } >>>>>>>>> } >>>>>>>>> >>>>>>>>> then: >>>>>>>>> >>>>>>>>> withDbSession { >>>>>>>>> session => >>>>>>>>> // many calls to session.add(...) >>>>>>>>> } >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Delving BV, Vasteland 8, Rotterdam >>>>>>>>> http://www.delving.eu >>>>>>>>> http://twitter.com/fluxe >>>>>>>>> skype: beautifulcode >>>>>>>>> +31629339805 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Delving BV, Vasteland 8, Rotterdam >>>>>>> http://www.delving.eu >>>>>>> http://twitter.com/fluxe >>>>>>> skype: beautifulcode >>>>>>> +31629339805 >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Delving BV, Vasteland 8, Rotterdam >>>>> http://www.delving.eu >>>>> http://twitter.com/fluxe >>>>> skype: beautifulcode >>>>> +31629339805 >>> >>> >>> >>> >>> -- >>> Delving BV, Vasteland 8, Rotterdam >>> http://www.delving.eu >>> http://twitter.com/fluxe >>> skype: beautifulcode >>> +31629339805 >> > > >
--
Dirk Kirsten, BaseX GmbH, http://basex.org
|-- Firmensitz: Blarerstrasse 56, 78462 Konstanz
|-- Registergericht Freiburg, HRB: 708285, Geschäftsführer:
| Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
GJ
Gerald de Jong29 Sep 2014 13:35 #
I'm glad you agree! I am not able to reproduce it consistently, and I will certainly try out the latest snapshot as much as possible.
Quoted text · Dirk Kirsten · 271 lines
On Fri, Sep 26, 2014 at 12:59 PM, Dirk Kirsten <dk at basex.org> wrote: > Hello Gerald, > > I completely agree. I guess, one could consider this a bug (but of > course, by definition, BaseX does not has bugs; it's hidden features). > Seems like something is going wrong when releasing the write lock. Could > you please test it with our latest snapshot, to see if the issue was > already fixed? > > Christian is currently on vacation, so an actual bug fix could take some > more time than usual. If I have time, I'll try to look into it. > > Cheers, > Dirk > > On 09/24/2014 05:41 PM, Gerald de Jong wrote: > > It happened again. Could I not argue that nothing I can do as a client > to > > BaseX should be able to freeze it? I mean, even if, for some reason, > > things are getting muddled up. > > > > Exception in thread "Thread-56391" java.lang.IllegalMonitorStateException > > at > > > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) > > at > > > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) > > at > > > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) > > at org.basex.core.DBLocking.release(DBLocking.java:219) > > at org.basex.core.Context.unregister(Context.java:249) > > at org.basex.core.Command.execute(Command.java:104) > > > > > > On Tue, Sep 23, 2014 at 2:17 PM, Christian Grün < > christian.gruen at gmail.com> > > wrote: > > > >> You can set the debugging flag and check out the database log files to > >> get more insight into what's going on. > >> > >> > >> On Tue, Sep 23, 2014 at 2:15 PM, Gerald de Jong <gerald at delving.eu> > wrote: > >>> Can I start up the basexserver with some kind of logging so I can see > >> what > >>> leads to the IllegalMonitorStateException? > >>> > >>> On Tue, Sep 23, 2014 at 1:53 PM, Christian Grün < > >> christian.gruen at gmail.com> > >>> wrote: > >>>> > >>>>> Looking at these tests, it appears that none of them do large numbers > >> of > >>>>> ADD > >>>>> commands. Did I miss something? > >>>> > >>>> You may need to check the other tests. But as the (updating) add > >>>> command will never be executed in parallel, it shouldn't cause any > >>>> problems with parallelization (feel free to tell me if I'm wrong). > >>>> > >>>> > >>>>> > >>>>> On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün > >>>>> <christian.gruen at gmail.com> > >>>>> wrote: > >>>>>> > >>>>>>> The fact that it's scala should be insignificant, since I'm using > >> the > >>>>>>> Java > >>>>>>> API. > >>>>>> > >>>>>> Well, it might be theoretically insignificant, but we cannot give a > >>>>>> quick answer on what's your problem. In general, parallel client > >>>>>> requests are no problem, we have various JUnit tests doing exactly > >>>>>> that (see e.g. [1], [2] or [3]). > >>>>>> > >>>>>> [1] > >>>>>> > >>>>>> > >> > https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/basex/server/ConcurrencyTest.java > >>>>>> [2] > >>>>>> > >>>>>> > >> > https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/RESTConcurrencyTest.java > >>>>>> [3] > >>>>>> > >>>>>> > >> > https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/ServerStressTest.java > >>>>>> > >>>>>> > >>>>>>> > >>>>>>> What's happening is that two (or more) threads are doing this at > >> the > >>>>>>> same > >>>>>>> time: > >>>>>>> > >>>>>>> ClientSession session = new ClientSession(...); > >>>>>>> session.execute(new Open(database)); > >>>>>>> session.execute(new Set("autoflush", "false")); > >>>>>>> session.add([path string], [document-bytes]); // a zillion > >> times > >>>>>>> session.execute(new Flush()); > >>>>>>> session.close(); > >>>>>>> > >>>>>>> My first question is of course if this should work parallel with no > >>>>>>> problems > >>>>>>> "in theory". > >>>>>>> > >>>>>>> On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün > >>>>>>> <christian.gruen at gmail.com> > >>>>>>> wrote: > >>>>>>>> > >>>>>>>> ...difficult to tell, as we don't have too much experience with > >>>>>>>> Scala. > >>>>>>>> Could you provide this example as Java code? > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong < > >> gerald at delving.eu> > >>>>>>>> wrote: > >>>>>>>>> Dear BaseX experts, > >>>>>>>>> > >>>>>>>>> I'm wondering if anybody can see something that I'm doing > >> wrong. I > >>>>>>>>> would > >>>>>>>>> expect it to be possible to add documents from multiple threads. > >>>>>>>>> Am > >>>>>>>>> I > >>>>>>>>> wrong > >>>>>>>>> about that?? > >>>>>>>>> > >>>>>>>>> When I set two of my datasets saving documents to BaseX 7.9 at > >> the > >>>>>>>>> same > >>>>>>>>> time, I get this exception: > >>>>>>>>> > >>>>>>>>> $ basexserver > >>>>>>>>> BaseX 7.9 [Server] > >>>>>>>>> Server was started (port: 1984) > >>>>>>>>> Exception in thread "Thread-2295" > >>>>>>>>> java.lang.IllegalMonitorStateException > >>>>>>>>> at > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >> > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) > >>>>>>>>> at > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >> > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) > >>>>>>>>> at > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >> > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) > >>>>>>>>> at org.basex.core.DBLocking.release(DBLocking.java:219) > >>>>>>>>> at org.basex.core.Context.unregister(Context.java:249) > >>>>>>>>> at org.basex.core.Command.execute(Command.java:104) > >>>>>>>>> at org.basex.core.Command.execute(Command.java:117) > >>>>>>>>> at > >>>>>>>>> org.basex.server.ClientListener.execute(ClientListener.java:384) > >>>>>>>>> at org.basex.server.ClientListener.add(ClientListener.java:355) > >>>>>>>>> at org.basex.server.ClientListener.run(ClientListener.java:100) > >>>>>>>>> > >>>>>>>>> All I am doing is making concurrent calls in two parallel > >>>>>>>>> sessions: > >>>>>>>>> > >>>>>>>>> session = new ClientSession(...) > >>>>>>>>> session.execute(new Open(database)) > >>>>>>>>> session.execute(new Set("autoflush", "false")) > >>>>>>>>> session.add([path string], [document-bytes]) // a zillion > >>>>>>>>> times > >>>>>>>>> session.execute(new Flush()) > >>>>>>>>> > >>>>>>>>> During the execution of the parallel add commands, it crashes. > >>>>>>>>> > >>>>>>>>> The relevant code: > >>>>>>>>> > >>>>>>>>> def withSession[T](block: ClientSession => T): T = { > >>>>>>>>> val session = new ClientSession(host, port, user, pass) > >>>>>>>>> try { > >>>>>>>>> block(session) > >>>>>>>>> } > >>>>>>>>> finally { > >>>>>>>>> session.close() > >>>>>>>>> } > >>>>>>>>> } > >>>>>>>>> > >>>>>>>>> def withDbSession[T](database: String)(block: ClientSession => > >>>>>>>>> T): > >>>>>>>>> T = > >>>>>>>>> { > >>>>>>>>> withSession { > >>>>>>>>> session => > >>>>>>>>> session.execute(new Open(database)) > >>>>>>>>> session.execute(new Set("autoflush", "false")) > >>>>>>>>> val result = block(session) > >>>>>>>>> session.execute(new Flush()) > >>>>>>>>> result > >>>>>>>>> } > >>>>>>>>> } > >>>>>>>>> > >>>>>>>>> then: > >>>>>>>>> > >>>>>>>>> withDbSession { > >>>>>>>>> session => > >>>>>>>>> // many calls to session.add(...) > >>>>>>>>> } > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> Delving BV, Vasteland 8, Rotterdam > >>>>>>>>> http://www.delving.eu > >>>>>>>>> http://twitter.com/fluxe > >>>>>>>>> skype: beautifulcode > >>>>>>>>> +31629339805 > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> Delving BV, Vasteland 8, Rotterdam > >>>>>>> http://www.delving.eu > >>>>>>> http://twitter.com/fluxe > >>>>>>> skype: beautifulcode > >>>>>>> +31629339805 > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Delving BV, Vasteland 8, Rotterdam > >>>>> http://www.delving.eu > >>>>> http://twitter.com/fluxe > >>>>> skype: beautifulcode > >>>>> +31629339805 > >>> > >>> > >>> > >>> > >>> -- > >>> Delving BV, Vasteland 8, Rotterdam > >>> http://www.delving.eu > >>> http://twitter.com/fluxe > >>> skype: beautifulcode > >>> +31629339805 > >> > > > > > > > > -- > Dirk Kirsten, BaseX GmbH, http://basex.org > |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 >
--
Delving BV, Vasteland 8, Rotterdam
http://www.delving.eu
http://twitter.com/fluxe
skype: beautifulcode
+31629339805
CG
Hi Gerald,
did you manage to write some Java code that causes the same problems? I am afraid that this is currently all I can offer.
Christian
Quoted text · Gerald de Jong · 246 lines
On Wed, Sep 24, 2014 at 5:41 PM, Gerald de Jong <gerald at delving.eu> wrote: > It happened again. Could I not argue that nothing I can do as a client to > BaseX should be able to freeze it? I mean, even if, for some reason, things > are getting muddled up. > > Exception in thread "Thread-56391" java.lang.IllegalMonitorStateException > at > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) > at > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) > at org.basex.core.DBLocking.release(DBLocking.java:219) > at org.basex.core.Context.unregister(Context.java:249) > at org.basex.core.Command.execute(Command.java:104) > > > On Tue, Sep 23, 2014 at 2:17 PM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> You can set the debugging flag and check out the database log files to >> get more insight into what's going on. >> >> >> On Tue, Sep 23, 2014 at 2:15 PM, Gerald de Jong <gerald at delving.eu> wrote: >> > Can I start up the basexserver with some kind of logging so I can see >> > what >> > leads to the IllegalMonitorStateException? >> > >> > On Tue, Sep 23, 2014 at 1:53 PM, Christian Grün >> > <christian.gruen at gmail.com> >> > wrote: >> >> >> >> > Looking at these tests, it appears that none of them do large numbers >> >> > of >> >> > ADD >> >> > commands. Did I miss something? >> >> >> >> You may need to check the other tests. But as the (updating) add >> >> command will never be executed in parallel, it shouldn't cause any >> >> problems with parallelization (feel free to tell me if I'm wrong). >> >> >> >> >> >> > >> >> > On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün >> >> > <christian.gruen at gmail.com> >> >> > wrote: >> >> >> >> >> >> > The fact that it's scala should be insignificant, since I'm using >> >> >> > the >> >> >> > Java >> >> >> > API. >> >> >> >> >> >> Well, it might be theoretically insignificant, but we cannot give a >> >> >> quick answer on what's your problem. In general, parallel client >> >> >> requests are no problem, we have various JUnit tests doing exactly >> >> >> that (see e.g. [1], [2] or [3]). >> >> >> >> >> >> [1] >> >> >> >> >> >> >> >> >> https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/basex/server/ConcurrencyTest.java >> >> >> [2] >> >> >> >> >> >> >> >> >> https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/RESTConcurrencyTest.java >> >> >> [3] >> >> >> >> >> >> >> >> >> https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/ServerStressTest.java >> >> >> >> >> >> >> >> >> > >> >> >> > What's happening is that two (or more) threads are doing this at >> >> >> > the >> >> >> > same >> >> >> > time: >> >> >> > >> >> >> > ClientSession session = new ClientSession(...); >> >> >> > session.execute(new Open(database)); >> >> >> > session.execute(new Set("autoflush", "false")); >> >> >> > session.add([path string], [document-bytes]); // a zillion >> >> >> > times >> >> >> > session.execute(new Flush()); >> >> >> > session.close(); >> >> >> > >> >> >> > My first question is of course if this should work parallel with >> >> >> > no >> >> >> > problems >> >> >> > "in theory". >> >> >> > >> >> >> > On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün >> >> >> > <christian.gruen at gmail.com> >> >> >> > wrote: >> >> >> >> >> >> >> >> ...difficult to tell, as we don't have too much experience with >> >> >> >> Scala. >> >> >> >> Could you provide this example as Java code? >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong >> >> >> >> <gerald at delving.eu> >> >> >> >> wrote: >> >> >> >> > Dear BaseX experts, >> >> >> >> > >> >> >> >> > I'm wondering if anybody can see something that I'm doing >> >> >> >> > wrong. I >> >> >> >> > would >> >> >> >> > expect it to be possible to add documents from multiple >> >> >> >> > threads. >> >> >> >> > Am >> >> >> >> > I >> >> >> >> > wrong >> >> >> >> > about that?? >> >> >> >> > >> >> >> >> > When I set two of my datasets saving documents to BaseX 7.9 at >> >> >> >> > the >> >> >> >> > same >> >> >> >> > time, I get this exception: >> >> >> >> > >> >> >> >> > $ basexserver >> >> >> >> > BaseX 7.9 [Server] >> >> >> >> > Server was started (port: 1984) >> >> >> >> > Exception in thread "Thread-2295" >> >> >> >> > java.lang.IllegalMonitorStateException >> >> >> >> > at >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) >> >> >> >> > at >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) >> >> >> >> > at >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) >> >> >> >> > at org.basex.core.DBLocking.release(DBLocking.java:219) >> >> >> >> > at org.basex.core.Context.unregister(Context.java:249) >> >> >> >> > at org.basex.core.Command.execute(Command.java:104) >> >> >> >> > at org.basex.core.Command.execute(Command.java:117) >> >> >> >> > at >> >> >> >> > >> >> >> >> > org.basex.server.ClientListener.execute(ClientListener.java:384) >> >> >> >> > at org.basex.server.ClientListener.add(ClientListener.java:355) >> >> >> >> > at org.basex.server.ClientListener.run(ClientListener.java:100) >> >> >> >> > >> >> >> >> > All I am doing is making concurrent calls in two parallel >> >> >> >> > sessions: >> >> >> >> > >> >> >> >> > session = new ClientSession(...) >> >> >> >> > session.execute(new Open(database)) >> >> >> >> > session.execute(new Set("autoflush", "false")) >> >> >> >> > session.add([path string], [document-bytes]) // a zillion >> >> >> >> > times >> >> >> >> > session.execute(new Flush()) >> >> >> >> > >> >> >> >> > During the execution of the parallel add commands, it crashes. >> >> >> >> > >> >> >> >> > The relevant code: >> >> >> >> > >> >> >> >> > def withSession[T](block: ClientSession => T): T = { >> >> >> >> > val session = new ClientSession(host, port, user, pass) >> >> >> >> > try { >> >> >> >> > block(session) >> >> >> >> > } >> >> >> >> > finally { >> >> >> >> > session.close() >> >> >> >> > } >> >> >> >> > } >> >> >> >> > >> >> >> >> > def withDbSession[T](database: String)(block: ClientSession >> >> >> >> > => >> >> >> >> > T): >> >> >> >> > T = >> >> >> >> > { >> >> >> >> > withSession { >> >> >> >> > session => >> >> >> >> > session.execute(new Open(database)) >> >> >> >> > session.execute(new Set("autoflush", "false")) >> >> >> >> > val result = block(session) >> >> >> >> > session.execute(new Flush()) >> >> >> >> > result >> >> >> >> > } >> >> >> >> > } >> >> >> >> > >> >> >> >> > then: >> >> >> >> > >> >> >> >> > withDbSession { >> >> >> >> > session => >> >> >> >> > // many calls to session.add(...) >> >> >> >> > } >> >> >> >> > >> >> >> >> > -- >> >> >> >> > Delving BV, Vasteland 8, Rotterdam >> >> >> >> > http://www.delving.eu >> >> >> >> > http://twitter.com/fluxe >> >> >> >> > skype: beautifulcode >> >> >> >> > +31629339805 >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > -- >> >> >> > Delving BV, Vasteland 8, Rotterdam >> >> >> > http://www.delving.eu >> >> >> > http://twitter.com/fluxe >> >> >> > skype: beautifulcode >> >> >> > +31629339805 >> >> > >> >> > >> >> > >> >> > >> >> > -- >> >> > Delving BV, Vasteland 8, Rotterdam >> >> > http://www.delving.eu >> >> > http://twitter.com/fluxe >> >> > skype: beautifulcode >> >> > +31629339805 >> > >> > >> > >> > >> > -- >> > Delving BV, Vasteland 8, Rotterdam >> > http://www.delving.eu >> > http://twitter.com/fluxe >> > skype: beautifulcode >> > +31629339805 > > > > > -- > Delving BV, Vasteland 8, Rotterdam > http://www.delving.eu > http://twitter.com/fluxe > skype: beautifulcode > +31629339805
GJ
Gerald de Jong7 Oct 2014 15:36 #
Thanks, Christian. I haven't seen it crash recently, but I'm watching closely.
I think I will also take another approach to getting the bulk of the data into BaseX. The slowdown that I was talking about earlier was a little annoying (haven't tried yet with the latest snapshot, sorry), and my approach of storing one document per "record" (there are millions) might not be the smartest.
So now I've built a module which builds a single big text file of those million records, and when I load that with a single BaseX command it's much much faster. The only thing I'm missing is a nice progress bar, but I can probably live without that.
Quoted text · Christian Grün · 280 lines
On Mon, Oct 6, 2014 at 2:34 PM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Gerald, > > did you manage to write some Java code that causes the same problems? > I am afraid that this is currently all I can offer. > > Christian > > > > On Wed, Sep 24, 2014 at 5:41 PM, Gerald de Jong <gerald at delving.eu> wrote: > > It happened again. Could I not argue that nothing I can do as a client > to > > BaseX should be able to freeze it? I mean, even if, for some reason, > things > > are getting muddled up. > > > > Exception in thread "Thread-56391" java.lang.IllegalMonitorStateException > > at > > > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) > > at > > > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) > > at > > > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) > > at org.basex.core.DBLocking.release(DBLocking.java:219) > > at org.basex.core.Context.unregister(Context.java:249) > > at org.basex.core.Command.execute(Command.java:104) > > > > > > On Tue, Sep 23, 2014 at 2:17 PM, Christian Grün < > christian.gruen at gmail.com> > > wrote: > >> > >> You can set the debugging flag and check out the database log files to > >> get more insight into what's going on. > >> > >> > >> On Tue, Sep 23, 2014 at 2:15 PM, Gerald de Jong <gerald at delving.eu> > wrote: > >> > Can I start up the basexserver with some kind of logging so I can see > >> > what > >> > leads to the IllegalMonitorStateException? > >> > > >> > On Tue, Sep 23, 2014 at 1:53 PM, Christian Grün > >> > <christian.gruen at gmail.com> > >> > wrote: > >> >> > >> >> > Looking at these tests, it appears that none of them do large > numbers > >> >> > of > >> >> > ADD > >> >> > commands. Did I miss something? > >> >> > >> >> You may need to check the other tests. But as the (updating) add > >> >> command will never be executed in parallel, it shouldn't cause any > >> >> problems with parallelization (feel free to tell me if I'm wrong). > >> >> > >> >> > >> >> > > >> >> > On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün > >> >> > <christian.gruen at gmail.com> > >> >> > wrote: > >> >> >> > >> >> >> > The fact that it's scala should be insignificant, since I'm > using > >> >> >> > the > >> >> >> > Java > >> >> >> > API. > >> >> >> > >> >> >> Well, it might be theoretically insignificant, but we cannot give > a > >> >> >> quick answer on what's your problem. In general, parallel client > >> >> >> requests are no problem, we have various JUnit tests doing exactly > >> >> >> that (see e.g. [1], [2] or [3]). > >> >> >> > >> >> >> [1] > >> >> >> > >> >> >> > >> >> >> > https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/basex/server/ConcurrencyTest.java > >> >> >> [2] > >> >> >> > >> >> >> > >> >> >> > https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/RESTConcurrencyTest.java > >> >> >> [3] > >> >> >> > >> >> >> > >> >> >> > https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/performance/ServerStressTest.java > >> >> >> > >> >> >> > >> >> >> > > >> >> >> > What's happening is that two (or more) threads are doing this at > >> >> >> > the > >> >> >> > same > >> >> >> > time: > >> >> >> > > >> >> >> > ClientSession session = new ClientSession(...); > >> >> >> > session.execute(new Open(database)); > >> >> >> > session.execute(new Set("autoflush", "false")); > >> >> >> > session.add([path string], [document-bytes]); // a zillion > >> >> >> > times > >> >> >> > session.execute(new Flush()); > >> >> >> > session.close(); > >> >> >> > > >> >> >> > My first question is of course if this should work parallel with > >> >> >> > no > >> >> >> > problems > >> >> >> > "in theory". > >> >> >> > > >> >> >> > On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün > >> >> >> > <christian.gruen at gmail.com> > >> >> >> > wrote: > >> >> >> >> > >> >> >> >> ...difficult to tell, as we don't have too much experience with > >> >> >> >> Scala. > >> >> >> >> Could you provide this example as Java code? > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong > >> >> >> >> <gerald at delving.eu> > >> >> >> >> wrote: > >> >> >> >> > Dear BaseX experts, > >> >> >> >> > > >> >> >> >> > I'm wondering if anybody can see something that I'm doing > >> >> >> >> > wrong. I > >> >> >> >> > would > >> >> >> >> > expect it to be possible to add documents from multiple > >> >> >> >> > threads. > >> >> >> >> > Am > >> >> >> >> > I > >> >> >> >> > wrong > >> >> >> >> > about that?? > >> >> >> >> > > >> >> >> >> > When I set two of my datasets saving documents to BaseX 7.9 > at > >> >> >> >> > the > >> >> >> >> > same > >> >> >> >> > time, I get this exception: > >> >> >> >> > > >> >> >> >> > $ basexserver > >> >> >> >> > BaseX 7.9 [Server] > >> >> >> >> > Server was started (port: 1984) > >> >> >> >> > Exception in thread "Thread-2295" > >> >> >> >> > java.lang.IllegalMonitorStateException > >> >> >> >> > at > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) > >> >> >> >> > at > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) > >> >> >> >> > at > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) > >> >> >> >> > at org.basex.core.DBLocking.release(DBLocking.java:219) > >> >> >> >> > at org.basex.core.Context.unregister(Context.java:249) > >> >> >> >> > at org.basex.core.Command.execute(Command.java:104) > >> >> >> >> > at org.basex.core.Command.execute(Command.java:117) > >> >> >> >> > at > >> >> >> >> > > >> >> >> >> > > org.basex.server.ClientListener.execute(ClientListener.java:384) > >> >> >> >> > at > org.basex.server.ClientListener.add(ClientListener.java:355) > >> >> >> >> > at > org.basex.server.ClientListener.run(ClientListener.java:100) > >> >> >> >> > > >> >> >> >> > All I am doing is making concurrent calls in two parallel > >> >> >> >> > sessions: > >> >> >> >> > > >> >> >> >> > session = new ClientSession(...) > >> >> >> >> > session.execute(new Open(database)) > >> >> >> >> > session.execute(new Set("autoflush", "false")) > >> >> >> >> > session.add([path string], [document-bytes]) // a > zillion > >> >> >> >> > times > >> >> >> >> > session.execute(new Flush()) > >> >> >> >> > > >> >> >> >> > During the execution of the parallel add commands, it > crashes. > >> >> >> >> > > >> >> >> >> > The relevant code: > >> >> >> >> > > >> >> >> >> > def withSession[T](block: ClientSession => T): T = { > >> >> >> >> > val session = new ClientSession(host, port, user, pass) > >> >> >> >> > try { > >> >> >> >> > block(session) > >> >> >> >> > } > >> >> >> >> > finally { > >> >> >> >> > session.close() > >> >> >> >> > } > >> >> >> >> > } > >> >> >> >> > > >> >> >> >> > def withDbSession[T](database: String)(block: ClientSession > >> >> >> >> > => > >> >> >> >> > T): > >> >> >> >> > T = > >> >> >> >> > { > >> >> >> >> > withSession { > >> >> >> >> > session => > >> >> >> >> > session.execute(new Open(database)) > >> >> >> >> > session.execute(new Set("autoflush", "false")) > >> >> >> >> > val result = block(session) > >> >> >> >> > session.execute(new Flush()) > >> >> >> >> > result > >> >> >> >> > } > >> >> >> >> > } > >> >> >> >> > > >> >> >> >> > then: > >> >> >> >> > > >> >> >> >> > withDbSession { > >> >> >> >> > session => > >> >> >> >> > // many calls to session.add(...) > >> >> >> >> > } > >> >> >> >> > > >> >> >> >> > -- > >> >> >> >> > Delving BV, Vasteland 8, Rotterdam > >> >> >> >> > http://www.delving.eu > >> >> >> >> > http://twitter.com/fluxe > >> >> >> >> > skype: beautifulcode > >> >> >> >> > +31629339805 > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > -- > >> >> >> > Delving BV, Vasteland 8, Rotterdam > >> >> >> > http://www.delving.eu > >> >> >> > http://twitter.com/fluxe > >> >> >> > skype: beautifulcode > >> >> >> > +31629339805 > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > -- > >> >> > Delving BV, Vasteland 8, Rotterdam > >> >> > http://www.delving.eu > >> >> > http://twitter.com/fluxe > >> >> > skype: beautifulcode > >> >> > +31629339805 > >> > > >> > > >> > > >> > > >> > -- > >> > Delving BV, Vasteland 8, Rotterdam > >> > http://www.delving.eu > >> > http://twitter.com/fluxe > >> > skype: beautifulcode > >> > +31629339805 > > > > > > > > > > -- > > Delving BV, Vasteland 8, Rotterdam > > http://www.delving.eu > > http://twitter.com/fluxe > > skype: beautifulcode > > +31629339805 >
--
Delving BV, Vasteland 8, Rotterdam
http://www.delving.eu
http://twitter.com/fluxe
skype: beautifulcode
+31629339805
Written for BaseX 7.9. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery