Documents added by db:add(...) not visible for other existing database connections?

On Sun, Sep 30, 2012 at 12:24 AM, Arjan van Bentem <arjan.vanbentem at bidnetwork.org> wrote: > Hi all, > > I noticed that TCP/IP clients are not seeing new documents that are created > using db:add(...) in another client: > > Once a query has been executed, subsequent queries using the very same > database connection do not see/return new documents that were meanwhile > created using another connection. When creating some (different) document > using one's own existing connection, then everything seems to be in-sync for > a moment. But right after that, any new documents are again not seen. > > Updates made in other clients are visible just fine. So, the documents that > are returned are up-to-date. It's just that not all documents are retrieved. > > I can see that in the query plans, the size in <DBNodeSeq size="..."> does > not change, and is lower than the size that clients with a newer connection > would show. > > Should it be okay to keep a connection open? Or is the expected behaviour to > need a new connection to see new documents? > > Details below. > > Thanks, > Arjan. > > > 1. Start the server and a client in debugging mode: > > ./basexserver -d -S > ./basexclient -U admin -P admin -d -x > > 2. Create and open a database: > > create database db1 > open db1 > > 3. Keep the client open, and make another client insert some documents. Like > every second for about 2 minutes, each time using a new connection: > > for i in {1..120}; \ > do doc="<test><a>`uuidgen`</a></test>"; \ > echo $i, $doc; \ > ./basexclient -U admin -P admin "db:add('db1', '$doc', 'dummy')"; \ > sleep 1; \ > done > > 4. While Step 3 is running, back in the first client, execute: > > xquery count(/test) > > The log in data/.logs shows: > > REQUEST XQUERY count(/test) > OK Query executed in 25.74 ms. 223.77 ms > > So far, so good. > > 5. Repeat Step 4 to see the result NOT change, while the loop from Step 3 is > still adding new documents. Logs show a faster result, but otherwise are the > same. Also the query plans are always the same. (Typically: the size in > <DBNodeSeq size="..."> does not change.) > > REQUEST XQUERY count(/test) > OK Query executed in 5.48 ms. 7.29 ms > > 6. Using "close" followed by "open db1" gets one a more up-to-date value for > "count". Also, using the connection to add some random other type of > document, updates the value for "count(/test)" as well: > > xquery db:add('db1', '<another-test><xyz>123</xyz></another-test>', > 'dummy') > > 7. It's not just "count" that does not change its value. Also, one can only > query documents up to that count as returned by Step 4. Like if Step 4 > returned "15", then the following would return 5 results, not 10: > > xquery (/test)[position()=10 to 20] > > Also, the following would only return the count as returned earlier: > > xquery /test > > Like noted, UPDATES made in other clients are visible just fine: > > xquery replace value of node /another-test/xyz with '456' > xquery for $n in /test/a return replace value of node $n with > fn:generate-id($n) > > > Tested in both 7.3 and 7.5-20120929.201251 from > http://files.basex.org/releases/latest/, in Oracle Java 7 on Mac OS X > 10.8.2. I first noticed that my running application did not see new > documents created by a test script. But the command line tools show the same > issue. > > Any clue? > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

On Sun, Sep 30, 2012 at 12:24 AM, Arjan van Bentem <arjan.vanbentem at bidnetwork.org> wrote: > Hi all, > > I noticed that TCP/IP clients are not seeing new documents that are created > using db:add(...) in another client: > > Once a query has been executed, subsequent queries using the very same > database connection do not see/return new documents that were meanwhile > created using another connection. When creating some (different) document > using one's own existing connection, then everything seems to be in-sync for > a moment. But right after that, any new documents are again not seen. > > Updates made in other clients are visible just fine. So, the documents that > are returned are up-to-date. It's just that not all documents are retrieved. > > I can see that in the query plans, the size in <DBNodeSeq size="..."> does > not change, and is lower than the size that clients with a newer connection > would show. > > Should it be okay to keep a connection open? Or is the expected behaviour to > need a new connection to see new documents? > > Details below. > > Thanks, > Arjan. > > > 1. Start the server and a client in debugging mode: > > ./basexserver -d -S > ./basexclient -U admin -P admin -d -x > > 2. Create and open a database: > > create database db1 > open db1 > > 3. Keep the client open, and make another client insert some documents. Like > every second for about 2 minutes, each time using a new connection: > > for i in {1..120}; \ > do doc="<test><a>`uuidgen`</a></test>"; \ > echo $i, $doc; \ > ./basexclient -U admin -P admin "db:add('db1', '$doc', 'dummy')"; \ > sleep 1; \ > done > > 4. While Step 3 is running, back in the first client, execute: > > xquery count(/test) > > The log in data/.logs shows: > > REQUEST XQUERY count(/test) > OK Query executed in 25.74 ms. 223.77 ms > > So far, so good. > > 5. Repeat Step 4 to see the result NOT change, while the loop from Step 3 is > still adding new documents. Logs show a faster result, but otherwise are the > same. Also the query plans are always the same. (Typically: the size in > <DBNodeSeq size="..."> does not change.) > > REQUEST XQUERY count(/test) > OK Query executed in 5.48 ms. 7.29 ms > > 6. Using "close" followed by "open db1" gets one a more up-to-date value for > "count". Also, using the connection to add some random other type of > document, updates the value for "count(/test)" as well: > > xquery db:add('db1', '<another-test><xyz>123</xyz></another-test>', > 'dummy') > > 7. It's not just "count" that does not change its value. Also, one can only > query documents up to that count as returned by Step 4. Like if Step 4 > returned "15", then the following would return 5 results, not 10: > > xquery (/test)[position()=10 to 20] > > Also, the following would only return the count as returned earlier: > > xquery /test > > Like noted, UPDATES made in other clients are visible just fine: > > xquery replace value of node /another-test/xyz with '456' > xquery for $n in /test/a return replace value of node $n with > fn:generate-id($n) > > > Tested in both 7.3 and 7.5-20120929.201251 from > http://files.basex.org/releases/latest/, in Oracle Java 7 on Mac OS X > 10.8.2. I first noticed that my running application did not see new > documents created by a test script. But the command line tools show the same > issue. > > Any clue? > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >