Concurrency Issue while loading documents.

Friday, 14 August 2015 · 3 messages · last reply 17 Aug 2015
AK
ankit kumar14 Aug 2015 14:05 #
Hi, We are adding number of documents concurrently to the Basex database using basex java api. Our application runs three threads in which each thread is adding documents to it by executing following instructions in java as given below. new Add("",new File(source).getAbsolutePath()).execute(context); while running our application sometimes throws the following error Java: Oracle Corporation, 1.7.0_71 OS: Windows 7, amd64 Stack Trace:
java.lang.ArrayIndexOutOfBoundsException: 8
at org.basex.util.hash.TokenSet.index(TokenSet.java:161)
at org.basex.util.hash.TokenSet.put(TokenSet.java:97)
at org.basex.data.Namespaces.open(Namespaces.java:319)
at org.basex.data.NSScope.open(NSScope.java:72)
at org.basex.data.Data.insert(Data.java:775)
at org.basex.query.up.atomic.Insert.apply(Insert.java:44)
at
org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(AtomicUpdateCache.java:297)
at
org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUpdateCache.java:281)
at org.basex.core.cmd.Add.run(Add.java:66)
at org.basex.core.Command.run(Command.java:398)
at org.basex.core.Command.execute(Command.java:100)
at org.basex.core.Command.execute(Command.java:123)
at com.myapp.MyDataBase.addSource(MyDataBase.java:85)

at com.myapp.ProcessorLoader$1.run(ProcessorLoader.java:132)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
thanks, Anky
CG
Christian Grün14 Aug 2015 14:58 #
Hi Ankit, I guess you are working with the latest version of BaseX? Could you please provide us with some code that allows us to reproduce the bug? Thank in advance, Christian
Quoted text · ankit kumar · 43 lines
On Fri, Aug 14, 2015 at 2:05 PM, ankit kumar <anky4bugs at gmail.com> wrote: > Hi, > > We are adding number of documents concurrently to the Basex database using > basex java api. Our application runs three threads in which each thread is > adding documents to it by executing following instructions in java as given > below. > > new Add("",new File(source).getAbsolutePath()).execute(context); > > while running our application sometimes throws the following error > > Java: Oracle Corporation, 1.7.0_71 > OS: Windows 7, amd64 > Stack Trace: > java.lang.ArrayIndexOutOfBoundsException: 8 > at org.basex.util.hash.TokenSet.index(TokenSet.java:161) > at org.basex.util.hash.TokenSet.put(TokenSet.java:97) > at org.basex.data.Namespaces.open(Namespaces.java:319) > at org.basex.data.NSScope.open(NSScope.java:72) > at org.basex.data.Data.insert(Data.java:775) > at org.basex.query.up.atomic.Insert.apply(Insert.java:44) > at > org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(AtomicUpdateCache.java:297) > at > org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUpdateCache.java:281) > at org.basex.core.cmd.Add.run(Add.java:66) > at org.basex.core.Command.run(Command.java:398) > at org.basex.core.Command.execute(Command.java:100) > at org.basex.core.Command.execute(Command.java:123) > at com.myapp.MyDataBase.addSource(MyDataBase.java:85) > > at com.myapp.ProcessorLoader$1.run(ProcessorLoader.java:132) > at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) > at java.util.concurrent.FutureTask.run(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > at java.lang.Thread.run(Unknown Source) > > > thanks, > Anky >
CG
Christian Grün17 Aug 2015 13:29 #
Hi Ankit, Have you been successful so far? You can have a look at two stress tests that I've just added [1,2]. In the tests, up to 10'000 documents are added to a database by multiple threads/clients Best, Christian [1] https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/server/ServerAddTest.java [2] https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/basex/local/multiple/MultipleAddTest.java
Quoted text · Christian Grün · 54 lines
On Fri, Aug 14, 2015 at 2:58 PM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Ankit, > > I guess you are working with the latest version of BaseX? Could you > please provide us with some code that allows us to reproduce the bug? > > Thank in advance, > Christian > > > On Fri, Aug 14, 2015 at 2:05 PM, ankit kumar <anky4bugs at gmail.com> wrote: >> Hi, >> >> We are adding number of documents concurrently to the Basex database using >> basex java api. Our application runs three threads in which each thread is >> adding documents to it by executing following instructions in java as given >> below. >> >> new Add("",new File(source).getAbsolutePath()).execute(context); >> >> while running our application sometimes throws the following error >> >> Java: Oracle Corporation, 1.7.0_71 >> OS: Windows 7, amd64 >> Stack Trace: >> java.lang.ArrayIndexOutOfBoundsException: 8 >> at org.basex.util.hash.TokenSet.index(TokenSet.java:161) >> at org.basex.util.hash.TokenSet.put(TokenSet.java:97) >> at org.basex.data.Namespaces.open(Namespaces.java:319) >> at org.basex.data.NSScope.open(NSScope.java:72) >> at org.basex.data.Data.insert(Data.java:775) >> at org.basex.query.up.atomic.Insert.apply(Insert.java:44) >> at >> org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(AtomicUpdateCache.java:297) >> at >> org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUpdateCache.java:281) >> at org.basex.core.cmd.Add.run(Add.java:66) >> at org.basex.core.Command.run(Command.java:398) >> at org.basex.core.Command.execute(Command.java:100) >> at org.basex.core.Command.execute(Command.java:123) >> at com.myapp.MyDataBase.addSource(MyDataBase.java:85) >> >> at com.myapp.ProcessorLoader$1.run(ProcessorLoader.java:132) >> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) >> at java.util.concurrent.FutureTask.run(Unknown Source) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) >> at java.lang.Thread.run(Unknown Source) >> >> >> thanks, >> Anky >>

⚡Generated with XQuery