BaseXClient

> I builded a kind of connection pool using BaseXClient.java class, I > simulated several users trying insert documents in BaseX, > the result was a disaster. > ERROR: data.BaseXDataAccess - Error closing db > java.io.IOException: Stopped at line 1, column 19: > Unknown command: 1340477810253095000. Try HELP. > at data.BaseXClient.execute(BaseXClient.java:99) > at data.BaseXClient.execute(BaseXClient.java:110) > at data.BaseXDataAccess.save(BaseXDataAccess.java:169) > at data.TestInsert$Insert.run(TestInsert.java:88) > at java.lang.Thread.run(Thread.java:680) > Exception in thread "Thread-1" java.lang.RuntimeException: > java.io.IOException: > at data.BaseXDataAccess.save(BaseXDataAccess.java:164) > at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: > com.xftec.hyperion.data.BaseXDataAccess - Error closing db > > I started several threads each one takes a BaseXClient connection and as > soon as it finishs its job BaseXClient is put in a > queue again to be took by other thread. > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

> There was an error in my code, I fixed it. > Now I am not getting errors on client side, but BaseX is not persisting all > documents that I inserted. > I created 20 BaseXClient's, 10 threads to take BasXClient from a > LinkedBlockingQueue, each thread was > responsible to insert 1000 documents, total of documents 10 x 1000 = 10K, > however was inserted 9801. > > Regards > > > On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> Hi Willian, >> >> thanks for your mail. I would assume that one single BaseXClient >> object was used for several concurrent tasks. As BaseXClient objects >> are very light-weight, and as the registration of new clients is >> usually more than fast enough, the usual approach is to create a new >> BaseXClient instance for a dedicated action. >> >> Hope this helps, >> Christian >> _______________________________ >> >> > I builded a kind of connection pool using BaseXClient.java class, I >> > simulated several users trying insert documents in BaseX, >> > the result was a disaster. >> > ERROR: data.BaseXDataAccess - Error closing db >> > java.io.IOException: Stopped at line 1, column 19: >> > Unknown command: 1340477810253095000. Try HELP. >> > at data.BaseXClient.execute(BaseXClient.java:99) >> > at data.BaseXClient.execute(BaseXClient.java:110) >> > at data.BaseXDataAccess.save(BaseXDataAccess.java:169) >> > at data.TestInsert$Insert.run(TestInsert.java:88) >> > at java.lang.Thread.run(Thread.java:680) >> > Exception in thread "Thread-1" java.lang.RuntimeException: >> > java.io.IOException: >> > at data.BaseXDataAccess.save(BaseXDataAccess.java:164) >> > at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: >> > com.xftec.hyperion.data.BaseXDataAccess - Error closing db >> > >> > I started several threads each one takes a BaseXClient connection and as >> > soon as it finishs its job BaseXClient is put in a >> > queue again to be took by other thread. >> > >> > _______________________________________________ >> > BaseX-Talk mailing list >> > BaseX-Talk at mailman.uni-konstanz.de >> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > > >

On Sunday 24 June 2012 11:24:00 Willian Barcella wrote: > There was an error in my code, I fixed it. > Now I am not getting errors on client side, but BaseX is not persisting all > documents that I inserted. > I created 20 BaseXClient's, 10 threads to take BasXClient from a > LinkedBlockingQueue, each thread was > responsible to insert 1000 documents, total of documents 10 x 1000 = 10K, > however was inserted 9801. > > Regards > > On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün > > <christian.gruen at gmail.com>wrote: > > Hi Willian, > > > > thanks for your mail. I would assume that one single BaseXClient > > object was used for several concurrent tasks. As BaseXClient objects > > are very light-weight, and as the registration of new clients is > > usually more than fast enough, the usual approach is to create a new > > BaseXClient instance for a dedicated action. > > > > Hope this helps, > > Christian > > _______________________________ > > > > > I builded a kind of connection pool using BaseXClient.java class, I > > > simulated several users trying insert documents in BaseX, > > > the result was a disaster. > > > ERROR: data.BaseXDataAccess - Error closing db > > > java.io.IOException: Stopped at line 1, column 19: > > > Unknown command: 1340477810253095000. Try HELP. > > > > > > at data.BaseXClient.execute(BaseXClient.java:99) > > > at data.BaseXClient.execute(BaseXClient.java:110) > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:169) > > > at data.TestInsert$Insert.run(TestInsert.java:88) > > > at java.lang.Thread.run(Thread.java:680) > > > > > > Exception in thread "Thread-1" java.lang.RuntimeException: > > > > > > java.io.IOException: > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:164) > > > > > > at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: > > > com.xftec.hyperion.data.BaseXDataAccess - Error closing db > > > > > > I started several threads each one takes a BaseXClient connection and as > > > soon as it finishs its job BaseXClient is put in a > > > queue again to be took by other thread. > > > > > > _______________________________________________ > > > BaseX-Talk mailing list > > > BaseX-Talk at mailman.uni-konstanz.de > > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

On Sun, Jun 24, 2012 at 4:47 PM, Willian Barcella <wbarcella at gmail.com> wrote: > I close DB each insert, this is my code: > ... > public void save(String key, DocumentWrapper value) > { > BaseXClient baseXClient = null; > try > { > //Open or Create database > baseXClient = this.takeConnection(); > baseXClient.execute(format(CHECK, value.getCollectionName())); > InputStream is = > XMLUtils.nodeToInputStream(value.getDocument()); > baseXClient.replace(value.getId(), is); > } catch (Exception e) > { > throw new RuntimeException(e); > } finally > { > try > { > baseXClient.execute(CLOSE); > if (baseXClient != null) > this.giveBack(baseXClient); > } catch (IOException e) > { > logger.error("Error closing db", e); > } > } > } > ... > > What you mean "single writer"? > > > On Sun, Jun 24, 2012 at 11:38 AM, Dimitar Popov > <dimitar.popov at uni-konstanz.de> wrote: >> >> check if you close the database after each insert operation. you should >> also >> know that currently, basex allows only a single writer, i.e. if you use >> multiple threads just to insert of the documents faster, that won't >> happen. >> >> regards, >> dimitar >> >> On Sunday 24 June 2012 11:24:00 Willian Barcella wrote: >> > There was an error in my code, I fixed it. >> > Now I am not getting errors on client side, but BaseX is not persisting >> > all >> > documents that I inserted. >> > I created 20 BaseXClient's, 10 threads to take BasXClient from a >> > LinkedBlockingQueue, each thread was >> > responsible to insert 1000 documents, total of documents 10 x 1000 = >> > 10K, >> > however was inserted 9801. >> > >> > Regards >> > >> > On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün >> > >> > <christian.gruen at gmail.com>wrote: >> > > Hi Willian, >> > > >> > > thanks for your mail. I would assume that one single BaseXClient >> > > object was used for several concurrent tasks. As BaseXClient objects >> > > are very light-weight, and as the registration of new clients is >> > > usually more than fast enough, the usual approach is to create a new >> > > BaseXClient instance for a dedicated action. >> > > >> > > Hope this helps, >> > > Christian >> > > _______________________________ >> > > >> > > > I builded a kind of connection pool using BaseXClient.java class, I >> > > > simulated several users trying insert documents in BaseX, >> > > > the result was a disaster. >> > > > ERROR: data.BaseXDataAccess - Error closing db >> > > > java.io.IOException: Stopped at line 1, column 19: >> > > > Unknown command: 1340477810253095000. Try HELP. >> > > > >> > > > at data.BaseXClient.execute(BaseXClient.java:99) >> > > > at data.BaseXClient.execute(BaseXClient.java:110) >> > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:169) >> > > > at data.TestInsert$Insert.run(TestInsert.java:88) >> > > > at java.lang.Thread.run(Thread.java:680) >> > > > >> > > > Exception in thread "Thread-1" java.lang.RuntimeException: >> > > > >> > > > java.io.IOException: >> > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:164) >> > > > >> > > > at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: >> > > > com.xftec.hyperion.data.BaseXDataAccess - Error closing db >> > > > >> > > > I started several threads each one takes a BaseXClient connection >> > > > and as >> > > > soon as it finishs its job BaseXClient is put in a >> > > > queue again to be took by other thread. >> > > > >> > > > _______________________________________________ >> > > > BaseX-Talk mailing list >> > > > BaseX-Talk at mailman.uni-konstanz.de >> > > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

On Sunday 24 June 2012 11:47:15 Willian Barcella wrote: > I close DB each insert, this is my code: > ... > public void save(String key, DocumentWrapper value) > { > BaseXClient baseXClient = null; > try > { > //Open or Create database > baseXClient = this.takeConnection(); > baseXClient.execute(format(CHECK, value.getCollectionName())); > InputStream is = > XMLUtils.nodeToInputStream(value.getDocument());

> baseXClient.replace(value.getId(), is); > } catch (Exception e) > {

> throw new RuntimeException(e); > } finally > { > try > { > baseXClient.execute(CLOSE); > if (baseXClient != null) > this.giveBack(baseXClient); > } catch (IOException e) > { > logger.error("Error closing db", e); > } > } > } > ... > > What you mean "single writer"?

> On Sun, Jun 24, 2012 at 11:38 AM, Dimitar Popov < > > dimitar.popov at uni-konstanz.de> wrote: > > check if you close the database after each insert operation. you should > > also > > know that currently, basex allows only a single writer, i.e. if you use > > multiple threads just to insert of the documents faster, that won't > > happen. > > > > regards, > > dimitar > > > > On Sunday 24 June 2012 11:24:00 Willian Barcella wrote: > > > There was an error in my code, I fixed it. > > > Now I am not getting errors on client side, but BaseX is not persisting > > > > all > > > > > documents that I inserted. > > > I created 20 BaseXClient's, 10 threads to take BasXClient from a > > > LinkedBlockingQueue, each thread was > > > responsible to insert 1000 documents, total of documents 10 x 1000 = > > > 10K, > > > however was inserted 9801. > > > > > > Regards > > > > > > On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün > > > > > > <christian.gruen at gmail.com>wrote: > > > > Hi Willian, > > > > > > > > thanks for your mail. I would assume that one single BaseXClient > > > > object was used for several concurrent tasks. As BaseXClient objects > > > > are very light-weight, and as the registration of new clients is > > > > usually more than fast enough, the usual approach is to create a new > > > > BaseXClient instance for a dedicated action. > > > > > > > > Hope this helps, > > > > Christian > > > > _______________________________ > > > > > > > > > I builded a kind of connection pool using BaseXClient.java class, I > > > > > simulated several users trying insert documents in BaseX, > > > > > the result was a disaster. > > > > > ERROR: data.BaseXDataAccess - Error closing db > > > > > java.io.IOException: Stopped at line 1, column 19: > > > > > Unknown command: 1340477810253095000. Try HELP. > > > > > > > > > > at data.BaseXClient.execute(BaseXClient.java:99) > > > > > at data.BaseXClient.execute(BaseXClient.java:110) > > > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:169) > > > > > at data.TestInsert$Insert.run(TestInsert.java:88) > > > > > at java.lang.Thread.run(Thread.java:680) > > > > > > > > > > Exception in thread "Thread-1" java.lang.RuntimeException: > > > > > > > > > > java.io.IOException: > > > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:164) > > > > > > > > > > at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: > > > > > com.xftec.hyperion.data.BaseXDataAccess - Error closing db > > > > > > > > > > I started several threads each one takes a BaseXClient connection > > > > and as > > > > > > > soon as it finishs its job BaseXClient is put in a > > > > > queue again to be took by other thread. > > > > > > > > > > _______________________________________________ > > > > > BaseX-Talk mailing list > > > > > BaseX-Talk at mailman.uni-konstanz.de > > > > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > > _______________________________________________ > > BaseX-Talk mailing list > > BaseX-Talk at mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

> baseXClient can be null because can happen an exception taking > this from queue.

> For every call to method save() should I create a new BaseXClient?

>> ___________________________ >> >> On Sun, Jun 24, 2012 at 4:47 PM, Willian Barcella <wbarcella at gmail.com> >> wrote: >> > I close DB each insert, this is my code: >> > ... >> > public void save(String key, DocumentWrapper value) >> > { >> > BaseXClient baseXClient = null; >> > try >> > { >> > //Open or Create database >> > baseXClient = this.takeConnection(); >> > baseXClient.execute(format(CHECK, >> > value.getCollectionName())); >> > InputStream is = >> > XMLUtils.nodeToInputStream(value.getDocument()); >> > baseXClient.replace(value.getId(), is); >> > } catch (Exception e) >> > { >> > throw new RuntimeException(e); >> > } finally >> > { >> > try >> > { >> > baseXClient.execute(CLOSE); >> > if (baseXClient != null) >> > this.giveBack(baseXClient); >> > } catch (IOException e) >> > { >> > logger.error("Error closing db", e); >> > } >> > } >> > } >> > ... >> > >> > What you mean "single writer"? >> > >> > >> > On Sun, Jun 24, 2012 at 11:38 AM, Dimitar Popov >> > <dimitar.popov at uni-konstanz.de> wrote: >> >> >> >> check if you close the database after each insert operation. you should >> >> also >> >> know that currently, basex allows only a single writer, i.e. if you use >> >> multiple threads just to insert of the documents faster, that won't >> >> happen. >> >> >> >> regards, >> >> dimitar >> >> >> >> On Sunday 24 June 2012 11:24:00 Willian Barcella wrote: >> >> > There was an error in my code, I fixed it. >> >> > Now I am not getting errors on client side, but BaseX is not >> >> > persisting >> >> > all >> >> > documents that I inserted. >> >> > I created 20 BaseXClient's, 10 threads to take BasXClient from a >> >> > LinkedBlockingQueue, each thread was >> >> > responsible to insert 1000 documents, total of documents 10 x 1000 = >> >> > 10K, >> >> > however was inserted 9801. >> >> > >> >> > Regards >> >> > >> >> > On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün >> >> > >> >> > <christian.gruen at gmail.com>wrote: >> >> > > Hi Willian, >> >> > > >> >> > > thanks for your mail. I would assume that one single BaseXClient >> >> > > object was used for several concurrent tasks. As BaseXClient >> >> > > objects >> >> > > are very light-weight, and as the registration of new clients is >> >> > > usually more than fast enough, the usual approach is to create a >> >> > > new >> >> > > BaseXClient instance for a dedicated action. >> >> > > >> >> > > Hope this helps, >> >> > > Christian >> >> > > _______________________________ >> >> > > >> >> > > > I builded a kind of connection pool using BaseXClient.java class, >> >> > > > I >> >> > > > simulated several users trying insert documents in BaseX, >> >> > > > the result was a disaster. >> >> > > > ERROR: data.BaseXDataAccess - Error closing db >> >> > > > java.io.IOException: Stopped at line 1, column 19: >> >> > > > Unknown command: 1340477810253095000. Try HELP. >> >> > > > >> >> > > > at data.BaseXClient.execute(BaseXClient.java:99) >> >> > > > at data.BaseXClient.execute(BaseXClient.java:110) >> >> > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:169) >> >> > > > at data.TestInsert$Insert.run(TestInsert.java:88) >> >> > > > at java.lang.Thread.run(Thread.java:680) >> >> > > > >> >> > > > Exception in thread "Thread-1" java.lang.RuntimeException: >> >> > > > >> >> > > > java.io.IOException: >> >> > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:164) >> >> > > > >> >> > > > at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: >> >> > > > com.xftec.hyperion.data.BaseXDataAccess - Error closing db >> >> > > > >> >> > > > I started several threads each one takes a BaseXClient connection >> >> > > > and as >> >> > > > soon as it finishs its job BaseXClient is put in a >> >> > > > queue again to be took by other thread. >> >> > > > >> >> > > > _______________________________________________ >> >> > > > BaseX-Talk mailing list >> >> > > > BaseX-Talk at mailman.uni-konstanz.de >> >> > > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> >> _______________________________________________ >> >> BaseX-Talk mailing list >> >> BaseX-Talk at mailman.uni-konstanz.de >> >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > >> > >> > >> > _______________________________________________ >> > BaseX-Talk mailing list >> > BaseX-Talk at mailman.uni-konstanz.de >> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > > >

> Hi Willian, > > thanks for your mail. I would assume that one single BaseXClient > object was used for several concurrent tasks. As BaseXClient objects > are very light-weight, and as the registration of new clients is > usually more than fast enough, the usual approach is to create a new > BaseXClient instance for a dedicated action. > > Hope this helps, > Christian > _______________________________ > > > I builded a kind of connection pool using BaseXClient.java class, I > > simulated several users trying insert documents in BaseX, > > the result was a disaster. > > ERROR: data.BaseXDataAccess - Error closing db > > java.io.IOException: Stopped at line 1, column 19: > > Unknown command: 1340477810253095000. Try HELP. > > at data.BaseXClient.execute(BaseXClient.java:99) > > at data.BaseXClient.execute(BaseXClient.java:110) > > at data.BaseXDataAccess.save(BaseXDataAccess.java:169) > > at data.TestInsert$Insert.run(TestInsert.java:88) > > at java.lang.Thread.run(Thread.java:680) > > Exception in thread "Thread-1" java.lang.RuntimeException: > > java.io.IOException: > > at data.BaseXDataAccess.save(BaseXDataAccess.java:164) > > at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: > > com.xftec.hyperion.data.BaseXDataAccess - Error closing db > > > > I started several threads each one takes a BaseXClient connection and as > > soon as it finishs its job BaseXClient is put in a > > queue again to be took by other thread. > > > > _______________________________________________ > > BaseX-Talk mailing list > > BaseX-Talk at mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > >

On Sun, Jun 24, 2012 at 11:38 AM, Dimitar Popov < dimitar.popov at uni-konstanz.de> wrote: > check if you close the database after each insert operation. you should > also > know that currently, basex allows only a single writer, i.e. if you use > multiple threads just to insert of the documents faster, that won't happen. > > regards, > dimitar > > On Sunday 24 June 2012 11:24:00 Willian Barcella wrote: > > There was an error in my code, I fixed it. > > Now I am not getting errors on client side, but BaseX is not persisting > all > > documents that I inserted. > > I created 20 BaseXClient's, 10 threads to take BasXClient from a > > LinkedBlockingQueue, each thread was > > responsible to insert 1000 documents, total of documents 10 x 1000 = 10K, > > however was inserted 9801. > > > > Regards > > > > On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün > > > > <christian.gruen at gmail.com>wrote: > > > Hi Willian, > > > > > > thanks for your mail. I would assume that one single BaseXClient > > > object was used for several concurrent tasks. As BaseXClient objects > > > are very light-weight, and as the registration of new clients is > > > usually more than fast enough, the usual approach is to create a new > > > BaseXClient instance for a dedicated action. > > > > > > Hope this helps, > > > Christian > > > _______________________________ > > > > > > > I builded a kind of connection pool using BaseXClient.java class, I > > > > simulated several users trying insert documents in BaseX, > > > > the result was a disaster. > > > > ERROR: data.BaseXDataAccess - Error closing db > > > > java.io.IOException: Stopped at line 1, column 19: > > > > Unknown command: 1340477810253095000. Try HELP. > > > > > > > > at data.BaseXClient.execute(BaseXClient.java:99) > > > > at data.BaseXClient.execute(BaseXClient.java:110) > > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:169) > > > > at data.TestInsert$Insert.run(TestInsert.java:88) > > > > at java.lang.Thread.run(Thread.java:680) > > > > > > > > Exception in thread "Thread-1" java.lang.RuntimeException: > > > > > > > > java.io.IOException: > > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:164) > > > > > > > > at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: > > > > com.xftec.hyperion.data.BaseXDataAccess - Error closing db > > > > > > > > I started several threads each one takes a BaseXClient connection > and as > > > > soon as it finishs its job BaseXClient is put in a > > > > queue again to be took by other thread. > > > > > > > > _______________________________________________ > > > > BaseX-Talk mailing list > > > > BaseX-Talk at mailman.uni-konstanz.de > > > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

> I close DB each insert, this is my code: > ... > public void save(String key, DocumentWrapper value) > { > BaseXClient baseXClient = null; > try > { > //Open or Create database > baseXClient = this.takeConnection(); > baseXClient.execute(format(CHECK, value.getCollectionName())); > InputStream is = > XMLUtils.nodeToInputStream(value.getDocument()); > baseXClient.replace(value.getId(), is); > } catch (Exception e) > { > throw new RuntimeException(e); > } finally > { > try > { > baseXClient.execute(CLOSE); > if (baseXClient != null) > this.giveBack(baseXClient); > } catch (IOException e) > { > logger.error("Error closing db", e); > } > } > } > ... > > What you mean "single writer"? > > > On Sun, Jun 24, 2012 at 11:38 AM, Dimitar Popov < > dimitar.popov at uni-konstanz.de> wrote: > >> check if you close the database after each insert operation. you should >> also >> know that currently, basex allows only a single writer, i.e. if you use >> multiple threads just to insert of the documents faster, that won't >> happen. >> >> regards, >> dimitar >> >> On Sunday 24 June 2012 11:24:00 Willian Barcella wrote: >> > There was an error in my code, I fixed it. >> > Now I am not getting errors on client side, but BaseX is not persisting >> all >> > documents that I inserted. >> > I created 20 BaseXClient's, 10 threads to take BasXClient from a >> > LinkedBlockingQueue, each thread was >> > responsible to insert 1000 documents, total of documents 10 x 1000 = >> 10K, >> > however was inserted 9801. >> > >> > Regards >> > >> > On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün >> > >> > <christian.gruen at gmail.com>wrote: >> > > Hi Willian, >> > > >> > > thanks for your mail. I would assume that one single BaseXClient >> > > object was used for several concurrent tasks. As BaseXClient objects >> > > are very light-weight, and as the registration of new clients is >> > > usually more than fast enough, the usual approach is to create a new >> > > BaseXClient instance for a dedicated action. >> > > >> > > Hope this helps, >> > > Christian >> > > _______________________________ >> > > >> > > > I builded a kind of connection pool using BaseXClient.java class, I >> > > > simulated several users trying insert documents in BaseX, >> > > > the result was a disaster. >> > > > ERROR: data.BaseXDataAccess - Error closing db >> > > > java.io.IOException: Stopped at line 1, column 19: >> > > > Unknown command: 1340477810253095000. Try HELP. >> > > > >> > > > at data.BaseXClient.execute(BaseXClient.java:99) >> > > > at data.BaseXClient.execute(BaseXClient.java:110) >> > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:169) >> > > > at data.TestInsert$Insert.run(TestInsert.java:88) >> > > > at java.lang.Thread.run(Thread.java:680) >> > > > >> > > > Exception in thread "Thread-1" java.lang.RuntimeException: >> > > > >> > > > java.io.IOException: >> > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:164) >> > > > >> > > > at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: >> > > > com.xftec.hyperion.data.BaseXDataAccess - Error closing db >> > > > >> > > > I started several threads each one takes a BaseXClient connection >> and as >> > > > soon as it finishs its job BaseXClient is put in a >> > > > queue again to be took by other thread. >> > > > >> > > > _______________________________________________ >> > > > BaseX-Talk mailing list >> > > > BaseX-Talk at mailman.uni-konstanz.de >> > > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > >

On Sunday 24 June 2012 12:30:08 Willian Barcella wrote: > > > What you mean "single writer"? > > > > I mean that, the basex server has internal lock mechanism, which blocks > > all sessions willing to execute an updating query, and effectively, it > > executes them sequentially, i.e. one after another. this means that basex > > does not support parallel execution of update queries. > > *Even BaseX does not support parallel updates, in server side it shouldn't > lose documents or it should throw an exception to client side saying that > document was not inserted. *

> I'm a little bit confused about your code. Some examples: > > - after the execution of "baseXClient.execute(CLOSE)", baseXClient can > never be null; however, it may actually be null before the call of > execute() >

> - what is hidden behind the CHECK and CLOSE variables? >

> > At least for debugging purposes, I would recommend to create new > client instead of getting them from the pool. Your connection should > then be terminated via baseXClient.close(). >

> > Christian > ___________________________ > > On Sun, Jun 24, 2012 at 4:47 PM, Willian Barcella <wbarcella at gmail.com> > wrote: > > I close DB each insert, this is my code: > > ... > > public void save(String key, DocumentWrapper value) > > { > > BaseXClient baseXClient = null; > > try > > { > > //Open or Create database > > baseXClient = this.takeConnection(); > > baseXClient.execute(format(CHECK, > value.getCollectionName())); > > InputStream is = > > XMLUtils.nodeToInputStream(value.getDocument()); > > baseXClient.replace(value.getId(), is); > > } catch (Exception e) > > { > > throw new RuntimeException(e); > > } finally > > { > > try > > { > > baseXClient.execute(CLOSE); > > if (baseXClient != null) > > this.giveBack(baseXClient); > > } catch (IOException e) > > { > > logger.error("Error closing db", e); > > } > > } > > } > > ... > > > > What you mean "single writer"? > > > > > > On Sun, Jun 24, 2012 at 11:38 AM, Dimitar Popov > > <dimitar.popov at uni-konstanz.de> wrote: > >> > >> check if you close the database after each insert operation. you should > >> also > >> know that currently, basex allows only a single writer, i.e. if you use > >> multiple threads just to insert of the documents faster, that won't > >> happen. > >> > >> regards, > >> dimitar > >> > >> On Sunday 24 June 2012 11:24:00 Willian Barcella wrote: > >> > There was an error in my code, I fixed it. > >> > Now I am not getting errors on client side, but BaseX is not > persisting > >> > all > >> > documents that I inserted. > >> > I created 20 BaseXClient's, 10 threads to take BasXClient from a > >> > LinkedBlockingQueue, each thread was > >> > responsible to insert 1000 documents, total of documents 10 x 1000 = > >> > 10K, > >> > however was inserted 9801. > >> > > >> > Regards > >> > > >> > On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün > >> > > >> > <christian.gruen at gmail.com>wrote: > >> > > Hi Willian, > >> > > > >> > > thanks for your mail. I would assume that one single BaseXClient > >> > > object was used for several concurrent tasks. As BaseXClient objects > >> > > are very light-weight, and as the registration of new clients is > >> > > usually more than fast enough, the usual approach is to create a new > >> > > BaseXClient instance for a dedicated action. > >> > > > >> > > Hope this helps, > >> > > Christian > >> > > _______________________________ > >> > > > >> > > > I builded a kind of connection pool using BaseXClient.java class, > I > >> > > > simulated several users trying insert documents in BaseX, > >> > > > the result was a disaster. > >> > > > ERROR: data.BaseXDataAccess - Error closing db > >> > > > java.io.IOException: Stopped at line 1, column 19: > >> > > > Unknown command: 1340477810253095000. Try HELP. > >> > > > > >> > > > at data.BaseXClient.execute(BaseXClient.java:99) > >> > > > at data.BaseXClient.execute(BaseXClient.java:110) > >> > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:169) > >> > > > at data.TestInsert$Insert.run(TestInsert.java:88) > >> > > > at java.lang.Thread.run(Thread.java:680) > >> > > > > >> > > > Exception in thread "Thread-1" java.lang.RuntimeException: > >> > > > > >> > > > java.io.IOException: > >> > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:164) > >> > > > > >> > > > at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: > >> > > > com.xftec.hyperion.data.BaseXDataAccess - Error closing db > >> > > > > >> > > > I started several threads each one takes a BaseXClient connection > >> > > > and as > >> > > > soon as it finishs its job BaseXClient is put in a > >> > > > queue again to be took by other thread. > >> > > > > >> > > > _______________________________________________ > >> > > > BaseX-Talk mailing list > >> > > > BaseX-Talk at mailman.uni-konstanz.de > >> > > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >> _______________________________________________ > >> BaseX-Talk mailing list > >> BaseX-Talk at mailman.uni-konstanz.de > >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > > > > > > _______________________________________________ > > BaseX-Talk mailing list > > BaseX-Talk at mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > >

On Sun, Jun 24, 2012 at 12:17 PM, Dimitar Popov < dimitar.popov at uni-konstanz.de> wrote: > ** > > some hints, which may help reveal the problem: > > > > On Sunday 24 June 2012 11:47:15 Willian Barcella wrote: > > > I close DB each insert, this is my code: > > > ... > > > public void save(String key, DocumentWrapper value) > > > { > > > BaseXClient baseXClient = null; > > > try > > > { > > > //Open or Create database > > > baseXClient = this.takeConnection(); > > > baseXClient.execute(format(CHECK, value.getCollectionName())); > > > InputStream is = > > > XMLUtils.nodeToInputStream(value.getDocument()); > > > > is DocumentWrapper.getCollectionName() thread-safe? >

> > > > baseXClient.replace(value.getId(), is); > > > } catch (Exception e) > > > { > > > > you may want to log a possible exception here - otherwise, it will be > silently ignored, since it occurs in a thread other than the main one. >

> > > > throw new RuntimeException(e); > > > } finally > > > { > > > try > > > { > > > baseXClient.execute(CLOSE); > > > if (baseXClient != null) > > > this.giveBack(baseXClient); > > > } catch (IOException e) > > > { > > > logger.error("Error closing db", e); > > > } > > > } > > > } > > > ... > > > > > > What you mean "single writer"? > > > > I mean that, the basex server has internal lock mechanism, which blocks > all sessions willing to execute an updating query, and effectively, it > executes them sequentially, i.e. one after another. this means that basex > does not support parallel execution of update queries. >

> > > regards, > > dimitar > > > On Sun, Jun 24, 2012 at 11:38 AM, Dimitar Popov < > > > > > > dimitar.popov at uni-konstanz.de> wrote: > > > > check if you close the database after each insert operation. you should > > > > also > > > > know that currently, basex allows only a single writer, i.e. if you use > > > > multiple threads just to insert of the documents faster, that won't > > > > happen. > > > > > > > > regards, > > > > dimitar > > > > > > > > On Sunday 24 June 2012 11:24:00 Willian Barcella wrote: > > > > > There was an error in my code, I fixed it. > > > > > Now I am not getting errors on client side, but BaseX is not > persisting > > > > > > > > all > > > > > > > > > documents that I inserted. > > > > > I created 20 BaseXClient's, 10 threads to take BasXClient from a > > > > > LinkedBlockingQueue, each thread was > > > > > responsible to insert 1000 documents, total of documents 10 x 1000 = > > > > > 10K, > > > > > however was inserted 9801. > > > > > > > > > > Regards > > > > > > > > > > On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün > > > > > > > > > > <christian.gruen at gmail.com>wrote: > > > > > > Hi Willian, > > > > > > > > > > > > thanks for your mail. I would assume that one single BaseXClient > > > > > > object was used for several concurrent tasks. As BaseXClient > objects > > > > > > are very light-weight, and as the registration of new clients is > > > > > > usually more than fast enough, the usual approach is to create a > new > > > > > > BaseXClient instance for a dedicated action. > > > > > > > > > > > > Hope this helps, > > > > > > Christian > > > > > > _______________________________ > > > > > > > > > > > > > I builded a kind of connection pool using BaseXClient.java > class, I > > > > > > > simulated several users trying insert documents in BaseX, > > > > > > > the result was a disaster. > > > > > > > ERROR: data.BaseXDataAccess - Error closing db > > > > > > > java.io.IOException: Stopped at line 1, column 19: > > > > > > > Unknown command: 1340477810253095000. Try HELP. > > > > > > > > > > > > > > at data.BaseXClient.execute(BaseXClient.java:99) > > > > > > > at data.BaseXClient.execute(BaseXClient.java:110) > > > > > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:169) > > > > > > > at data.TestInsert$Insert.run(TestInsert.java:88) > > > > > > > at java.lang.Thread.run(Thread.java:680) > > > > > > > > > > > > > > Exception in thread "Thread-1" java.lang.RuntimeException: > > > > > > > > > > > > > > java.io.IOException: > > > > > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:164) > > > > > > > > > > > > > > at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: > > > > > > > com.xftec.hyperion.data.BaseXDataAccess - Error closing db > > > > > > > > > > > > > > I started several threads each one takes a BaseXClient connection > > > > > > > > and as > > > > > > > > > > > soon as it finishs its job BaseXClient is put in a > > > > > > > queue again to be took by other thread. > > > > > > > > > > > > > > _______________________________________________ > > > > > > > BaseX-Talk mailing list > > > > > > > BaseX-Talk at mailman.uni-konstanz.de > > > > > > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > > > > > > _______________________________________________ > > > > BaseX-Talk mailing list > > > > BaseX-Talk at mailman.uni-konstanz.de > > > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

> > baseXClient can be null because can happen an exception taking > > this from queue. > > If that’s true, I’d call this somewhat spectacular ;) Note that > baseXClient is a local variable, which cannot be invalidated by any > other thread. But it may well be that takeConnection() returns a null > reference, in which case it won't make sense to call > baseXClient.execute() just after that. > > > For every call to method save() should I create a new BaseXClient? > > Exactly. Either way, please check your code once more, or try to > provide us with an sscce [1]. I have doubts that the erroneous > inserting behavior is really rooted in the BaseX core classes. > > Christian > > [1] http://sscce.org/ > > > >> ___________________________ > >> > >> On Sun, Jun 24, 2012 at 4:47 PM, Willian Barcella <wbarcella at gmail.com> > >> wrote: > >> > I close DB each insert, this is my code: > >> > ... > >> > public void save(String key, DocumentWrapper value) > >> > { > >> > BaseXClient baseXClient = null; > >> > try > >> > { > >> > //Open or Create database > >> > baseXClient = this.takeConnection(); > >> > baseXClient.execute(format(CHECK, > >> > value.getCollectionName())); > >> > InputStream is = > >> > XMLUtils.nodeToInputStream(value.getDocument()); > >> > baseXClient.replace(value.getId(), is); > >> > } catch (Exception e) > >> > { > >> > throw new RuntimeException(e); > >> > } finally > >> > { > >> > try > >> > { > >> > baseXClient.execute(CLOSE); > >> > if (baseXClient != null) > >> > this.giveBack(baseXClient); > >> > } catch (IOException e) > >> > { > >> > logger.error("Error closing db", e); > >> > } > >> > } > >> > } > >> > ... > >> > > >> > What you mean "single writer"? > >> > > >> > > >> > On Sun, Jun 24, 2012 at 11:38 AM, Dimitar Popov > >> > <dimitar.popov at uni-konstanz.de> wrote: > >> >> > >> >> check if you close the database after each insert operation. you > should > >> >> also > >> >> know that currently, basex allows only a single writer, i.e. if you > use > >> >> multiple threads just to insert of the documents faster, that won't > >> >> happen. > >> >> > >> >> regards, > >> >> dimitar > >> >> > >> >> On Sunday 24 June 2012 11:24:00 Willian Barcella wrote: > >> >> > There was an error in my code, I fixed it. > >> >> > Now I am not getting errors on client side, but BaseX is not > >> >> > persisting > >> >> > all > >> >> > documents that I inserted. > >> >> > I created 20 BaseXClient's, 10 threads to take BasXClient from a > >> >> > LinkedBlockingQueue, each thread was > >> >> > responsible to insert 1000 documents, total of documents 10 x 1000 > = > >> >> > 10K, > >> >> > however was inserted 9801. > >> >> > > >> >> > Regards > >> >> > > >> >> > On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün > >> >> > > >> >> > <christian.gruen at gmail.com>wrote: > >> >> > > Hi Willian, > >> >> > > > >> >> > > thanks for your mail. I would assume that one single BaseXClient > >> >> > > object was used for several concurrent tasks. As BaseXClient > >> >> > > objects > >> >> > > are very light-weight, and as the registration of new clients is > >> >> > > usually more than fast enough, the usual approach is to create a > >> >> > > new > >> >> > > BaseXClient instance for a dedicated action. > >> >> > > > >> >> > > Hope this helps, > >> >> > > Christian > >> >> > > _______________________________ > >> >> > > > >> >> > > > I builded a kind of connection pool using BaseXClient.java > class, > >> >> > > > I > >> >> > > > simulated several users trying insert documents in BaseX, > >> >> > > > the result was a disaster. > >> >> > > > ERROR: data.BaseXDataAccess - Error closing db > >> >> > > > java.io.IOException: Stopped at line 1, column 19: > >> >> > > > Unknown command: 1340477810253095000. Try HELP. > >> >> > > > > >> >> > > > at data.BaseXClient.execute(BaseXClient.java:99) > >> >> > > > at data.BaseXClient.execute(BaseXClient.java:110) > >> >> > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:169) > >> >> > > > at data.TestInsert$Insert.run(TestInsert.java:88) > >> >> > > > at java.lang.Thread.run(Thread.java:680) > >> >> > > > > >> >> > > > Exception in thread "Thread-1" java.lang.RuntimeException: > >> >> > > > > >> >> > > > java.io.IOException: > >> >> > > > at data.BaseXDataAccess.save(BaseXDataAccess.java:164) > >> >> > > > > >> >> > > > at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: > >> >> > > > com.xftec.hyperion.data.BaseXDataAccess - Error closing db > >> >> > > > > >> >> > > > I started several threads each one takes a BaseXClient > connection > >> >> > > > and as > >> >> > > > soon as it finishs its job BaseXClient is put in a > >> >> > > > queue again to be took by other thread. > >> >> > > > > >> >> > > > _______________________________________________ > >> >> > > > BaseX-Talk mailing list > >> >> > > > BaseX-Talk at mailman.uni-konstanz.de > >> >> > > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >> >> _______________________________________________ > >> >> BaseX-Talk mailing list > >> >> BaseX-Talk at mailman.uni-konstanz.de > >> >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >> > > >> > > >> > > >> > _______________________________________________ > >> > BaseX-Talk mailing list > >> > BaseX-Talk at mailman.uni-konstanz.de > >> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >> > > > > > >

On Sun, Jun 24, 2012 at 2:12 PM, Dimitar Popov < dimitar.popov at uni-konstanz.de> wrote: > On Sunday 24 June 2012 12:30:08 Willian Barcella wrote: > > > > What you mean "single writer"? > > > > > > I mean that, the basex server has internal lock mechanism, which blocks > > > all sessions willing to execute an updating query, and effectively, it > > > executes them sequentially, i.e. one after another. this means that > basex > > > does not support parallel execution of update queries. > > > > *Even BaseX does not support parallel updates, in server side it > shouldn't > > lose documents or it should throw an exception to client side saying that > > document was not inserted. * > > agreed. can you put a static counter in your class, which is incremented > each > time the save method is called. this can help isolate the cause: if the > value > of the counter is different from 10000, then the problem is somewhere in > the > thread management code; otherwise, it has to be somewhere in the baseX > code. > > regards, > dimitar >