Read block? Kill me now.

Am 06.01.11 05:20, schrieb Charles F. Munat: > So I went live with my embedded Base-X website and that has been a > very painful learning experience. > > I had been opening the DB on each request and running various queries, > but assumed that the process would close it when it exited. That was > bad, bad, bad. About an hour after I uploaded the new site (and after > I'd gone to bed for a nine hour snooze), the site started serving a > 500 error code because too many files were open. > > OK, figured that out. So I created a singleton and with a method that > opens the database in a try/catch block, and closes it in the finally > block. Problem solved. > > Sort of. > > Apologies for the Scala, but here is the singleton code: > > object DB { > private val context = new Context() > new org.basex.core.cmd.Set("dbpath", "/var/www/db").execute(context) > > def executeQuery(query: String): String = { > try { > new Open("data").execute(context) > new XQuery(query).execute(context) > } catch { > case _ => > new CreateDB("data", "<data/>").execute(context) > new XQuery(query).execute(context) > } finally { > new Close().execute(context) > } > } > } > > Try blocks in Scala return a value, so this method returns the output > of the passed-in XQuery. Works like a charm. > > When a request is received, my servlet (using Circumflex, a Scala web > framework) creates a new RequestRouter class. A "get" method is called > and passed the URL. Here is that class and method, which shows how I > am calling the above DB object: > > class Main extends RequestRouter { > get(url) = { > val query = Rx.getQuery(url) > val output = DB.executeQuery(qry) > > "<!DOCTYPE html>\n" + output > } > } > > This passes the URL (url) to a method on another singleton > (Rx.getQuery) which returns an XQuery based on the URL. That query is > passed to DB.executeQuery which stores the output in the immutable > variable "output." I then prepend a doctype and serve it as text/html. > > This runs great... for about an hour. Then the server stops responding > altogether. I've searched the logs, and I found this error: > > java.io.IOException: Stream Closed > java.io.RandomAccessFile.seek(Native Method) > org.basex.io.TableDiskAccess.readBlock(TableDiskAccess.java:349) > org.basex.io.TableDiskAccess.cursor(TableDiskAccess.java:326) > org.basex.io.TableDiskAccess.read1(TableDiskAccess.java:92) > org.basex.data.Data.kind(Data.java:305) > org.basex.query.item.DBNode$4.next(DBNode.java:273) > org.basex.query.path.IterStep$1.next(IterStep.java:45) > org.basex.query.path.AxisPath.iter(AxisPath.java:437) > org.basex.query.path.AxisPath.iter(AxisPath.java:406) > org.basex.query.QueryContext.iter(QueryContext.java:306) > org.basex.query.expr.For$1.init(For.java:127) > org.basex.query.expr.For$1.next(For.java:92) > org.basex.query.expr.FLWR$1.next(FLWR.java:63) > org.basex.query.expr.Constr.<init>(Constr.java:53) > org.basex.query.expr.CElem.item(CElem.java:112) > org.basex.query.expr.CElem.item(CElem.java:1) > org.basex.query.expr.CFrag.item(CFrag.java:1) > org.basex.query.expr.ParseExpr.iter(ParseExpr.java:49) > org.basex.query.QueryContext.iter(QueryContext.java:306) > org.basex.query.expr.FLWR$1.next(FLWR.java:67) > org.basex.query.iter.Iter.finish(Iter.java:65) > org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) > org.basex.query.expr.Func.iter(Func.java:80) > org.basex.query.QueryContext.iter(QueryContext.java:306) > org.basex.query.expr.FuncCall.iter(FuncCall.java:76) > org.basex.query.QueryContext.iter(QueryContext.java:306) > org.basex.query.expr.TypeCase.iter(TypeCase.java:95) > org.basex.query.expr.TypeSwitch.iter(TypeSwitch.java:74) > org.basex.query.QueryContext.iter(QueryContext.java:306) > org.basex.query.expr.Constr.<init>(Constr.java:52) > org.basex.query.up.Replace.item(Replace.java:50) > org.basex.query.expr.ParseExpr.iter(ParseExpr.java:49) > org.basex.query.QueryContext.iter(QueryContext.java:306) > org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) > org.basex.query.expr.GFLWOR.iter(GFLWOR.java:298) > org.basex.query.expr.GFLWOR.iter(GFLWOR.java:291) > org.basex.query.expr.GFLWOR.iter(GFLWOR.java:266) > org.basex.query.QueryContext.iter(QueryContext.java:306) > org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) > org.basex.query.up.Transform.iter(Transform.java:86) > org.basex.query.QueryContext.iter(QueryContext.java:306) > org.basex.query.expr.FLWR$1.next(FLWR.java:67) > org.basex.query.iter.Iter.finish(Iter.java:65) > org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) > org.basex.query.expr.Func.iter(Func.java:80) > org.basex.query.QueryContext.iter(QueryContext.java:306) > org.basex.query.expr.FuncCall.iter(FuncCall.java:76) > org.basex.query.QueryContext.iter(QueryContext.java:306) > > I'm guessing that this is some sort of race condition? > > What is the trick to using Base-X in embedded mode? Do I need to be > doing some sort of threading here? (I have managed to avoid learning > anything about threads for 15 years now. Hate to ruin that record.) > > Suggestions? Advice? At this point I'm seriously thinking of just > running Base-X as a server and connecting through a connection pool, > but I have to figure that out tonight, so if there's an easier answer, > I'd sure love to know it. > > Thanks again for all your help! > > Chas. > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Am 06.01.11 15:46, schrieb Charles F. Munat: > The latest version. > > That's good to know. I haven't changed the query yet because I've been > struggling to fix other bugs, but I should be fixing that today. > > Do you think that performance would be better running client/server > instead? If so, do you know any example code that is up-to-date with > the latest version. The sample on the website doesn't work as it's > currently written (at least it didn't for me). > > Thanks, > > Chas. > > On 01/06/2011 8:08 AM, Andreas Weiler wrote: >> Hi Chas, >> >> what BaseX version are you running? >> >> As a first hint, you don't have to open the database, when you are using >> the doc('database') function in your queries. So you don't have to close >> it, too. >> >> Kind regards, >> Andreas >> >> Am 06.01.11 05:20, schrieb Charles F. Munat: >>> So I went live with my embedded Base-X website and that has been a >>> very painful learning experience. >>> >>> I had been opening the DB on each request and running various queries, >>> but assumed that the process would close it when it exited. That was >>> bad, bad, bad. About an hour after I uploaded the new site (and after >>> I'd gone to bed for a nine hour snooze), the site started serving a >>> 500 error code because too many files were open. >>> >>> OK, figured that out. So I created a singleton and with a method that >>> opens the database in a try/catch block, and closes it in the finally >>> block. Problem solved. >>> >>> Sort of. >>> >>> Apologies for the Scala, but here is the singleton code: >>> >>> object DB { >>> private val context = new Context() >>> new org.basex.core.cmd.Set("dbpath", "/var/www/db").execute(context) >>> >>> def executeQuery(query: String): String = { >>> try { >>> new Open("data").execute(context) >>> new XQuery(query).execute(context) >>> } catch { >>> case _ => >>> new CreateDB("data", "<data/>").execute(context) >>> new XQuery(query).execute(context) >>> } finally { >>> new Close().execute(context) >>> } >>> } >>> } >>> >>> Try blocks in Scala return a value, so this method returns the output >>> of the passed-in XQuery. Works like a charm. >>> >>> When a request is received, my servlet (using Circumflex, a Scala web >>> framework) creates a new RequestRouter class. A "get" method is called >>> and passed the URL. Here is that class and method, which shows how I >>> am calling the above DB object: >>> >>> class Main extends RequestRouter { >>> get(url) = { >>> val query = Rx.getQuery(url) >>> val output = DB.executeQuery(qry) >>> >>> "<!DOCTYPE html>\n" + output >>> } >>> } >>> >>> This passes the URL (url) to a method on another singleton >>> (Rx.getQuery) which returns an XQuery based on the URL. That query is >>> passed to DB.executeQuery which stores the output in the immutable >>> variable "output." I then prepend a doctype and serve it as text/html. >>> >>> This runs great... for about an hour. Then the server stops responding >>> altogether. I've searched the logs, and I found this error: >>> >>> java.io.IOException: Stream Closed >>> java.io.RandomAccessFile.seek(Native Method) >>> org.basex.io.TableDiskAccess.readBlock(TableDiskAccess.java:349) >>> org.basex.io.TableDiskAccess.cursor(TableDiskAccess.java:326) >>> org.basex.io.TableDiskAccess.read1(TableDiskAccess.java:92) >>> org.basex.data.Data.kind(Data.java:305) >>> org.basex.query.item.DBNode$4.next(DBNode.java:273) >>> org.basex.query.path.IterStep$1.next(IterStep.java:45) >>> org.basex.query.path.AxisPath.iter(AxisPath.java:437) >>> org.basex.query.path.AxisPath.iter(AxisPath.java:406) >>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>> org.basex.query.expr.For$1.init(For.java:127) >>> org.basex.query.expr.For$1.next(For.java:92) >>> org.basex.query.expr.FLWR$1.next(FLWR.java:63) >>> org.basex.query.expr.Constr.<init>(Constr.java:53) >>> org.basex.query.expr.CElem.item(CElem.java:112) >>> org.basex.query.expr.CElem.item(CElem.java:1) >>> org.basex.query.expr.CFrag.item(CFrag.java:1) >>> org.basex.query.expr.ParseExpr.iter(ParseExpr.java:49) >>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>> org.basex.query.expr.FLWR$1.next(FLWR.java:67) >>> org.basex.query.iter.Iter.finish(Iter.java:65) >>> org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) >>> org.basex.query.expr.Func.iter(Func.java:80) >>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>> org.basex.query.expr.FuncCall.iter(FuncCall.java:76) >>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>> org.basex.query.expr.TypeCase.iter(TypeCase.java:95) >>> org.basex.query.expr.TypeSwitch.iter(TypeSwitch.java:74) >>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>> org.basex.query.expr.Constr.<init>(Constr.java:52) >>> org.basex.query.up.Replace.item(Replace.java:50) >>> org.basex.query.expr.ParseExpr.iter(ParseExpr.java:49) >>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>> org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) >>> org.basex.query.expr.GFLWOR.iter(GFLWOR.java:298) >>> org.basex.query.expr.GFLWOR.iter(GFLWOR.java:291) >>> org.basex.query.expr.GFLWOR.iter(GFLWOR.java:266) >>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>> org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) >>> org.basex.query.up.Transform.iter(Transform.java:86) >>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>> org.basex.query.expr.FLWR$1.next(FLWR.java:67) >>> org.basex.query.iter.Iter.finish(Iter.java:65) >>> org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) >>> org.basex.query.expr.Func.iter(Func.java:80) >>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>> org.basex.query.expr.FuncCall.iter(FuncCall.java:76) >>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>> >>> I'm guessing that this is some sort of race condition? >>> >>> What is the trick to using Base-X in embedded mode? Do I need to be >>> doing some sort of threading here? (I have managed to avoid learning >>> anything about threads for 15 years now. Hate to ruin that record.) >>> >>> Suggestions? Advice? At this point I'm seriously thinking of just >>> running Base-X as a server and connecting through a connection pool, >>> but I have to figure that out tonight, so if there's an easier answer, >>> I'd sure love to know it. >>> >>> Thanks again for all your help! >>> >>> Chas. >>> _______________________________________________ >>> BaseX-Talk mailing list >>> BaseX-Talk at mailman.uni-konstanz.de >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> >>

On 01/06/2011 8:08 AM, Andreas Weiler wrote: > Hi Chas, > > what BaseX version are you running? > > As a first hint, you don't have to open the database, when you are using > the doc('database') function in your queries. So you don't have to close > it, too. > > Kind regards, > Andreas > > Am 06.01.11 05:20, schrieb Charles F. Munat: >> So I went live with my embedded Base-X website and that has been a >> very painful learning experience. >> >> I had been opening the DB on each request and running various queries, >> but assumed that the process would close it when it exited. That was >> bad, bad, bad. About an hour after I uploaded the new site (and after >> I'd gone to bed for a nine hour snooze), the site started serving a >> 500 error code because too many files were open. >> >> OK, figured that out. So I created a singleton and with a method that >> opens the database in a try/catch block, and closes it in the finally >> block. Problem solved. >> >> Sort of. >> >> Apologies for the Scala, but here is the singleton code: >> >> object DB { >> private val context = new Context() >> new org.basex.core.cmd.Set("dbpath", "/var/www/db").execute(context) >> >> def executeQuery(query: String): String = { >> try { >> new Open("data").execute(context) >> new XQuery(query).execute(context) >> } catch { >> case _ => >> new CreateDB("data", "<data/>").execute(context) >> new XQuery(query).execute(context) >> } finally { >> new Close().execute(context) >> } >> } >> } >> >> Try blocks in Scala return a value, so this method returns the output >> of the passed-in XQuery. Works like a charm. >> >> When a request is received, my servlet (using Circumflex, a Scala web >> framework) creates a new RequestRouter class. A "get" method is called >> and passed the URL. Here is that class and method, which shows how I >> am calling the above DB object: >> >> class Main extends RequestRouter { >> get(url) = { >> val query = Rx.getQuery(url) >> val output = DB.executeQuery(qry) >> >> "<!DOCTYPE html>\n" + output >> } >> } >> >> This passes the URL (url) to a method on another singleton >> (Rx.getQuery) which returns an XQuery based on the URL. That query is >> passed to DB.executeQuery which stores the output in the immutable >> variable "output." I then prepend a doctype and serve it as text/html. >> >> This runs great... for about an hour. Then the server stops responding >> altogether. I've searched the logs, and I found this error: >> >> java.io.IOException: Stream Closed >> java.io.RandomAccessFile.seek(Native Method) >> org.basex.io.TableDiskAccess.readBlock(TableDiskAccess.java:349) >> org.basex.io.TableDiskAccess.cursor(TableDiskAccess.java:326) >> org.basex.io.TableDiskAccess.read1(TableDiskAccess.java:92) >> org.basex.data.Data.kind(Data.java:305) >> org.basex.query.item.DBNode$4.next(DBNode.java:273) >> org.basex.query.path.IterStep$1.next(IterStep.java:45) >> org.basex.query.path.AxisPath.iter(AxisPath.java:437) >> org.basex.query.path.AxisPath.iter(AxisPath.java:406) >> org.basex.query.QueryContext.iter(QueryContext.java:306) >> org.basex.query.expr.For$1.init(For.java:127) >> org.basex.query.expr.For$1.next(For.java:92) >> org.basex.query.expr.FLWR$1.next(FLWR.java:63) >> org.basex.query.expr.Constr.<init>(Constr.java:53) >> org.basex.query.expr.CElem.item(CElem.java:112) >> org.basex.query.expr.CElem.item(CElem.java:1) >> org.basex.query.expr.CFrag.item(CFrag.java:1) >> org.basex.query.expr.ParseExpr.iter(ParseExpr.java:49) >> org.basex.query.QueryContext.iter(QueryContext.java:306) >> org.basex.query.expr.FLWR$1.next(FLWR.java:67) >> org.basex.query.iter.Iter.finish(Iter.java:65) >> org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) >> org.basex.query.expr.Func.iter(Func.java:80) >> org.basex.query.QueryContext.iter(QueryContext.java:306) >> org.basex.query.expr.FuncCall.iter(FuncCall.java:76) >> org.basex.query.QueryContext.iter(QueryContext.java:306) >> org.basex.query.expr.TypeCase.iter(TypeCase.java:95) >> org.basex.query.expr.TypeSwitch.iter(TypeSwitch.java:74) >> org.basex.query.QueryContext.iter(QueryContext.java:306) >> org.basex.query.expr.Constr.<init>(Constr.java:52) >> org.basex.query.up.Replace.item(Replace.java:50) >> org.basex.query.expr.ParseExpr.iter(ParseExpr.java:49) >> org.basex.query.QueryContext.iter(QueryContext.java:306) >> org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) >> org.basex.query.expr.GFLWOR.iter(GFLWOR.java:298) >> org.basex.query.expr.GFLWOR.iter(GFLWOR.java:291) >> org.basex.query.expr.GFLWOR.iter(GFLWOR.java:266) >> org.basex.query.QueryContext.iter(QueryContext.java:306) >> org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) >> org.basex.query.up.Transform.iter(Transform.java:86) >> org.basex.query.QueryContext.iter(QueryContext.java:306) >> org.basex.query.expr.FLWR$1.next(FLWR.java:67) >> org.basex.query.iter.Iter.finish(Iter.java:65) >> org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) >> org.basex.query.expr.Func.iter(Func.java:80) >> org.basex.query.QueryContext.iter(QueryContext.java:306) >> org.basex.query.expr.FuncCall.iter(FuncCall.java:76) >> org.basex.query.QueryContext.iter(QueryContext.java:306) >> >> I'm guessing that this is some sort of race condition? >> >> What is the trick to using Base-X in embedded mode? Do I need to be >> doing some sort of threading here? (I have managed to avoid learning >> anything about threads for 15 years now. Hate to ruin that record.) >> >> Suggestions? Advice? At this point I'm seriously thinking of just >> running Base-X as a server and connecting through a connection pool, >> but I have to figure that out tonight, so if there's an easier answer, >> I'd sure love to know it. >> >> Thanks again for all your help! >> >> Chas. >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >

Am 06.01.11 18:40, schrieb Charles F. Munat: > Where did you get that? I was looking at > > http://www.inf.uni-konstanz.de/dbis/basex/code/ServerExample > > which does not work. The code below looks right. I'll try it. Moving > the db access code out of the singleton definitely fixed the threading > issue, but I think for performance I'd probably be better off in > client/server mode. (Using the embedded database was mostly a learning > exercise for me -- never done it before. I am not wedded to it, and > not even sure I like it.) > > This helps a lot! > > I notice that these are all set up to access the server from an > application (thus the "main" method). But everything I'm doing is in a > servlet. There don't seem to be any examples of best practices for how > to set up a connection pool or access the db from within a servlet, > which has a very distinctive pattern of request/response with objects > being created and then destroyed. I always get confused about what > should be in a singleton and what should be created/destroyed on each > request. > > Do you have any advice on that or links to code examples? > > BTW, if I ever get this working and we determine that it's a good > practice, I'd be happy to provide Scala examples that you could put on > the website for anyone else using Base-X with Scala. Having to convert > everything over from Java is a real pain, especially since I'm not > that great a Java programmer to begin with, having done much more work > in C#, Ruby, and other languages. I'd also be willing to provide a > basic framework using Circumflex (a Scala web framework), Maven, and > Base-X. I'd even be willing to set up a Maven archetype for Circumflex > for it, so people could just generate a Circumflex project with Base-X > ready to go. I can talk to the folks at Circumflex about it. > > Thanks again for all your kind assistance. I am very pleased with > Base-X and am now about to integrate it into another big project and > use it as the basis for all my future dynamic web sites -- a dream > I've had for more than a decade. XML databases make the most sense on > the Web! I'm outputting XML! Why in the world would I want to keep > converting it back and forth to and from objects? It makes so much > more sense to work in an FP paradigm and simply manipulate the data > that is ALWAYS in XML format. That was obvious back in 1999. Finally, > there is a product that allows me to very easily accomplish this (and > works in Maven, which is the killer for me with eXist). > > (Another big issue is that much of the data I work with is > tree-structured. Why in the hell am I having to flatten this stuff out > to stick it in an RDBMS and then rebuild the tree? Finally, PostgreSQL > has recursive queries, which make that a bit easier. But for years I > had to do all this work in the business layer. Why, why, why? XML *is* > a tree structure. Problem not just solved, but eliminated.) > > Chas. > > On 01/06/2011 11:55 AM, Andreas Weiler wrote: >> If you want to achieve concurrently access on your databases, you need >> to use >> the client-server architecture. >> >> Which sample did you mean? >> This one works for sure... >> >> package org.basex.examples.server; >> >> import org.basex.BaseXServer; >> import org.basex.core.BaseXException; >> import org.basex.server.ClientSession; >> >> /** >> * This class demonstrates database access via the client/server >> architecture. >> * >> * @author Workgroup DBIS, University of Konstanz 2005-10, ISC License >> * @author BaseX Team >> */ >> public final class ServerCommands { >> /** Session reference. */ >> static ClientSession session; >> >> /** >> * Runs the example code. >> * @param args (ignored) command-line arguments >> * @throws Exception exception >> */ >> public static void main(final String[] args) throws Exception { >> >> System.out.println("=== ServerExample ==="); >> >> // >> ------------------------------------------------------------------------ >> // Start server on default port 1984 >> BaseXServer server = new BaseXServer(); >> >> // >> ------------------------------------------------------------------------ >> // Create a client session with host name, port, user name and password >> System.out.println("\n* Create a client session."); >> >> session = new ClientSession("localhost", 1984, "admin", "admin"); >> >> // >> ------------------------------------------------------------------------ >> // Create a database >> System.out.println("\n* Create a database."); >> >> session.execute("CREATE DB input etc/xml/input.xml"); >> >> // >> ------------------------------------------------------------------------ >> // Run a query >> System.out.println("\n* Run a query:"); >> >> System.out.println(session.execute("XQUERY //li")); >> >> // >> ------------------------------------------------------------------------ >> // Faster version: specify an output stream and run a query >> System.out.println("\n* Run a query (faster):"); >> >> session.setOutputStream(System.out); >> session.execute("XQUERY //li"); >> >> // Reset output stream >> session.setOutputStream(null); >> >> // >> ------------------------------------------------------------------------ >> // Run a query >> System.out.println("\n* Run a buggy query: "); >> >> try { >> session.execute("XQUERY ///"); >> } catch(final BaseXException ex) { >> System.out.println(ex.getMessage()); >> } >> >> // >> ------------------------------------------------------------------------ >> // Drop the database >> System.out.println("\n* Close and drop the database."); >> >> session.execute("DROP DB input"); >> >> // >> ------------------------------------------------------------------------ >> // Close the client session >> System.out.println("\n* Close the client session."); >> >> session.close(); >> >> // >> ------------------------------------------------------------------------ >> // Stop the server >> System.out.println("\n* Stop the server."); >> >> server.stop(); >> } >> } >> >> >> Andreas >> >> Am 06.01.11 15:46, schrieb Charles F. Munat: >>> The latest version. >>> >>> That's good to know. I haven't changed the query yet because I've been >>> struggling to fix other bugs, but I should be fixing that today. >>> >>> Do you think that performance would be better running client/server >>> instead? If so, do you know any example code that is up-to-date with >>> the latest version. The sample on the website doesn't work as it's >>> currently written (at least it didn't for me). >>> >>> Thanks, >>> >>> Chas. >>> >>> On 01/06/2011 8:08 AM, Andreas Weiler wrote: >>>> Hi Chas, >>>> >>>> what BaseX version are you running? >>>> >>>> As a first hint, you don't have to open the database, when you are >>>> using >>>> the doc('database') function in your queries. So you don't have to >>>> close >>>> it, too. >>>> >>>> Kind regards, >>>> Andreas >>>> >>>> Am 06.01.11 05:20, schrieb Charles F. Munat: >>>>> So I went live with my embedded Base-X website and that has been a >>>>> very painful learning experience. >>>>> >>>>> I had been opening the DB on each request and running various >>>>> queries, >>>>> but assumed that the process would close it when it exited. That was >>>>> bad, bad, bad. About an hour after I uploaded the new site (and after >>>>> I'd gone to bed for a nine hour snooze), the site started serving a >>>>> 500 error code because too many files were open. >>>>> >>>>> OK, figured that out. So I created a singleton and with a method that >>>>> opens the database in a try/catch block, and closes it in the finally >>>>> block. Problem solved. >>>>> >>>>> Sort of. >>>>> >>>>> Apologies for the Scala, but here is the singleton code: >>>>> >>>>> object DB { >>>>> private val context = new Context() >>>>> new org.basex.core.cmd.Set("dbpath", "/var/www/db").execute(context) >>>>> >>>>> def executeQuery(query: String): String = { >>>>> try { >>>>> new Open("data").execute(context) >>>>> new XQuery(query).execute(context) >>>>> } catch { >>>>> case _ => >>>>> new CreateDB("data", "<data/>").execute(context) >>>>> new XQuery(query).execute(context) >>>>> } finally { >>>>> new Close().execute(context) >>>>> } >>>>> } >>>>> } >>>>> >>>>> Try blocks in Scala return a value, so this method returns the output >>>>> of the passed-in XQuery. Works like a charm. >>>>> >>>>> When a request is received, my servlet (using Circumflex, a Scala web >>>>> framework) creates a new RequestRouter class. A "get" method is >>>>> called >>>>> and passed the URL. Here is that class and method, which shows how I >>>>> am calling the above DB object: >>>>> >>>>> class Main extends RequestRouter { >>>>> get(url) = { >>>>> val query = Rx.getQuery(url) >>>>> val output = DB.executeQuery(qry) >>>>> >>>>> "<!DOCTYPE html>\n" + output >>>>> } >>>>> } >>>>> >>>>> This passes the URL (url) to a method on another singleton >>>>> (Rx.getQuery) which returns an XQuery based on the URL. That query is >>>>> passed to DB.executeQuery which stores the output in the immutable >>>>> variable "output." I then prepend a doctype and serve it as >>>>> text/html. >>>>> >>>>> This runs great... for about an hour. Then the server stops >>>>> responding >>>>> altogether. I've searched the logs, and I found this error: >>>>> >>>>> java.io.IOException: Stream Closed >>>>> java.io.RandomAccessFile.seek(Native Method) >>>>> org.basex.io.TableDiskAccess.readBlock(TableDiskAccess.java:349) >>>>> org.basex.io.TableDiskAccess.cursor(TableDiskAccess.java:326) >>>>> org.basex.io.TableDiskAccess.read1(TableDiskAccess.java:92) >>>>> org.basex.data.Data.kind(Data.java:305) >>>>> org.basex.query.item.DBNode$4.next(DBNode.java:273) >>>>> org.basex.query.path.IterStep$1.next(IterStep.java:45) >>>>> org.basex.query.path.AxisPath.iter(AxisPath.java:437) >>>>> org.basex.query.path.AxisPath.iter(AxisPath.java:406) >>>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>>> org.basex.query.expr.For$1.init(For.java:127) >>>>> org.basex.query.expr.For$1.next(For.java:92) >>>>> org.basex.query.expr.FLWR$1.next(FLWR.java:63) >>>>> org.basex.query.expr.Constr.<init>(Constr.java:53) >>>>> org.basex.query.expr.CElem.item(CElem.java:112) >>>>> org.basex.query.expr.CElem.item(CElem.java:1) >>>>> org.basex.query.expr.CFrag.item(CFrag.java:1) >>>>> org.basex.query.expr.ParseExpr.iter(ParseExpr.java:49) >>>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>>> org.basex.query.expr.FLWR$1.next(FLWR.java:67) >>>>> org.basex.query.iter.Iter.finish(Iter.java:65) >>>>> org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) >>>>> org.basex.query.expr.Func.iter(Func.java:80) >>>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>>> org.basex.query.expr.FuncCall.iter(FuncCall.java:76) >>>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>>> org.basex.query.expr.TypeCase.iter(TypeCase.java:95) >>>>> org.basex.query.expr.TypeSwitch.iter(TypeSwitch.java:74) >>>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>>> org.basex.query.expr.Constr.<init>(Constr.java:52) >>>>> org.basex.query.up.Replace.item(Replace.java:50) >>>>> org.basex.query.expr.ParseExpr.iter(ParseExpr.java:49) >>>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>>> org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) >>>>> org.basex.query.expr.GFLWOR.iter(GFLWOR.java:298) >>>>> org.basex.query.expr.GFLWOR.iter(GFLWOR.java:291) >>>>> org.basex.query.expr.GFLWOR.iter(GFLWOR.java:266) >>>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>>> org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) >>>>> org.basex.query.up.Transform.iter(Transform.java:86) >>>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>>> org.basex.query.expr.FLWR$1.next(FLWR.java:67) >>>>> org.basex.query.iter.Iter.finish(Iter.java:65) >>>>> org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) >>>>> org.basex.query.expr.Func.iter(Func.java:80) >>>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>>> org.basex.query.expr.FuncCall.iter(FuncCall.java:76) >>>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>>> >>>>> I'm guessing that this is some sort of race condition? >>>>> >>>>> What is the trick to using Base-X in embedded mode? Do I need to be >>>>> doing some sort of threading here? (I have managed to avoid learning >>>>> anything about threads for 15 years now. Hate to ruin that record.) >>>>> >>>>> Suggestions? Advice? At this point I'm seriously thinking of just >>>>> running Base-X as a server and connecting through a connection pool, >>>>> but I have to figure that out tonight, so if there's an easier >>>>> answer, >>>>> I'd sure love to know it. >>>>> >>>>> Thanks again for all your help! >>>>> >>>>> Chas. >>>>> _______________________________________________ >>>>> BaseX-Talk mailing list >>>>> BaseX-Talk at mailman.uni-konstanz.de >>>>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>> >>>> >>

On 01/06/2011 11:55 AM, Andreas Weiler wrote: > If you want to achieve concurrently access on your databases, you need > to use > the client-server architecture. > > Which sample did you mean? > This one works for sure... > > package org.basex.examples.server; > > import org.basex.BaseXServer; > import org.basex.core.BaseXException; > import org.basex.server.ClientSession; > > /** > * This class demonstrates database access via the client/server > architecture. > * > * @author Workgroup DBIS, University of Konstanz 2005-10, ISC License > * @author BaseX Team > */ > public final class ServerCommands { > /** Session reference. */ > static ClientSession session; > > /** > * Runs the example code. > * @param args (ignored) command-line arguments > * @throws Exception exception > */ > public static void main(final String[] args) throws Exception { > > System.out.println("=== ServerExample ==="); > > // ------------------------------------------------------------------------ > // Start server on default port 1984 > BaseXServer server = new BaseXServer(); > > // ------------------------------------------------------------------------ > // Create a client session with host name, port, user name and password > System.out.println("\n* Create a client session."); > > session = new ClientSession("localhost", 1984, "admin", "admin"); > > // ------------------------------------------------------------------------ > // Create a database > System.out.println("\n* Create a database."); > > session.execute("CREATE DB input etc/xml/input.xml"); > > // ------------------------------------------------------------------------ > // Run a query > System.out.println("\n* Run a query:"); > > System.out.println(session.execute("XQUERY //li")); > > // ------------------------------------------------------------------------ > // Faster version: specify an output stream and run a query > System.out.println("\n* Run a query (faster):"); > > session.setOutputStream(System.out); > session.execute("XQUERY //li"); > > // Reset output stream > session.setOutputStream(null); > > // ------------------------------------------------------------------------ > // Run a query > System.out.println("\n* Run a buggy query: "); > > try { > session.execute("XQUERY ///"); > } catch(final BaseXException ex) { > System.out.println(ex.getMessage()); > } > > // ------------------------------------------------------------------------ > // Drop the database > System.out.println("\n* Close and drop the database."); > > session.execute("DROP DB input"); > > // ------------------------------------------------------------------------ > // Close the client session > System.out.println("\n* Close the client session."); > > session.close(); > > // ------------------------------------------------------------------------ > // Stop the server > System.out.println("\n* Stop the server."); > > server.stop(); > } > } > > > Andreas > > Am 06.01.11 15:46, schrieb Charles F. Munat: >> The latest version. >> >> That's good to know. I haven't changed the query yet because I've been >> struggling to fix other bugs, but I should be fixing that today. >> >> Do you think that performance would be better running client/server >> instead? If so, do you know any example code that is up-to-date with >> the latest version. The sample on the website doesn't work as it's >> currently written (at least it didn't for me). >> >> Thanks, >> >> Chas. >> >> On 01/06/2011 8:08 AM, Andreas Weiler wrote: >>> Hi Chas, >>> >>> what BaseX version are you running? >>> >>> As a first hint, you don't have to open the database, when you are using >>> the doc('database') function in your queries. So you don't have to close >>> it, too. >>> >>> Kind regards, >>> Andreas >>> >>> Am 06.01.11 05:20, schrieb Charles F. Munat: >>>> So I went live with my embedded Base-X website and that has been a >>>> very painful learning experience. >>>> >>>> I had been opening the DB on each request and running various queries, >>>> but assumed that the process would close it when it exited. That was >>>> bad, bad, bad. About an hour after I uploaded the new site (and after >>>> I'd gone to bed for a nine hour snooze), the site started serving a >>>> 500 error code because too many files were open. >>>> >>>> OK, figured that out. So I created a singleton and with a method that >>>> opens the database in a try/catch block, and closes it in the finally >>>> block. Problem solved. >>>> >>>> Sort of. >>>> >>>> Apologies for the Scala, but here is the singleton code: >>>> >>>> object DB { >>>> private val context = new Context() >>>> new org.basex.core.cmd.Set("dbpath", "/var/www/db").execute(context) >>>> >>>> def executeQuery(query: String): String = { >>>> try { >>>> new Open("data").execute(context) >>>> new XQuery(query).execute(context) >>>> } catch { >>>> case _ => >>>> new CreateDB("data", "<data/>").execute(context) >>>> new XQuery(query).execute(context) >>>> } finally { >>>> new Close().execute(context) >>>> } >>>> } >>>> } >>>> >>>> Try blocks in Scala return a value, so this method returns the output >>>> of the passed-in XQuery. Works like a charm. >>>> >>>> When a request is received, my servlet (using Circumflex, a Scala web >>>> framework) creates a new RequestRouter class. A "get" method is called >>>> and passed the URL. Here is that class and method, which shows how I >>>> am calling the above DB object: >>>> >>>> class Main extends RequestRouter { >>>> get(url) = { >>>> val query = Rx.getQuery(url) >>>> val output = DB.executeQuery(qry) >>>> >>>> "<!DOCTYPE html>\n" + output >>>> } >>>> } >>>> >>>> This passes the URL (url) to a method on another singleton >>>> (Rx.getQuery) which returns an XQuery based on the URL. That query is >>>> passed to DB.executeQuery which stores the output in the immutable >>>> variable "output." I then prepend a doctype and serve it as text/html. >>>> >>>> This runs great... for about an hour. Then the server stops responding >>>> altogether. I've searched the logs, and I found this error: >>>> >>>> java.io.IOException: Stream Closed >>>> java.io.RandomAccessFile.seek(Native Method) >>>> org.basex.io.TableDiskAccess.readBlock(TableDiskAccess.java:349) >>>> org.basex.io.TableDiskAccess.cursor(TableDiskAccess.java:326) >>>> org.basex.io.TableDiskAccess.read1(TableDiskAccess.java:92) >>>> org.basex.data.Data.kind(Data.java:305) >>>> org.basex.query.item.DBNode$4.next(DBNode.java:273) >>>> org.basex.query.path.IterStep$1.next(IterStep.java:45) >>>> org.basex.query.path.AxisPath.iter(AxisPath.java:437) >>>> org.basex.query.path.AxisPath.iter(AxisPath.java:406) >>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>> org.basex.query.expr.For$1.init(For.java:127) >>>> org.basex.query.expr.For$1.next(For.java:92) >>>> org.basex.query.expr.FLWR$1.next(FLWR.java:63) >>>> org.basex.query.expr.Constr.<init>(Constr.java:53) >>>> org.basex.query.expr.CElem.item(CElem.java:112) >>>> org.basex.query.expr.CElem.item(CElem.java:1) >>>> org.basex.query.expr.CFrag.item(CFrag.java:1) >>>> org.basex.query.expr.ParseExpr.iter(ParseExpr.java:49) >>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>> org.basex.query.expr.FLWR$1.next(FLWR.java:67) >>>> org.basex.query.iter.Iter.finish(Iter.java:65) >>>> org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) >>>> org.basex.query.expr.Func.iter(Func.java:80) >>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>> org.basex.query.expr.FuncCall.iter(FuncCall.java:76) >>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>> org.basex.query.expr.TypeCase.iter(TypeCase.java:95) >>>> org.basex.query.expr.TypeSwitch.iter(TypeSwitch.java:74) >>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>> org.basex.query.expr.Constr.<init>(Constr.java:52) >>>> org.basex.query.up.Replace.item(Replace.java:50) >>>> org.basex.query.expr.ParseExpr.iter(ParseExpr.java:49) >>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>> org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) >>>> org.basex.query.expr.GFLWOR.iter(GFLWOR.java:298) >>>> org.basex.query.expr.GFLWOR.iter(GFLWOR.java:291) >>>> org.basex.query.expr.GFLWOR.iter(GFLWOR.java:266) >>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>> org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) >>>> org.basex.query.up.Transform.iter(Transform.java:86) >>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>> org.basex.query.expr.FLWR$1.next(FLWR.java:67) >>>> org.basex.query.iter.Iter.finish(Iter.java:65) >>>> org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) >>>> org.basex.query.expr.Func.iter(Func.java:80) >>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>> org.basex.query.expr.FuncCall.iter(FuncCall.java:76) >>>> org.basex.query.QueryContext.iter(QueryContext.java:306) >>>> >>>> I'm guessing that this is some sort of race condition? >>>> >>>> What is the trick to using Base-X in embedded mode? Do I need to be >>>> doing some sort of threading here? (I have managed to avoid learning >>>> anything about threads for 15 years now. Hate to ruin that record.) >>>> >>>> Suggestions? Advice? At this point I'm seriously thinking of just >>>> running Base-X as a server and connecting through a connection pool, >>>> but I have to figure that out tonight, so if there's an easier answer, >>>> I'd sure love to know it. >>>> >>>> Thanks again for all your help! >>>> >>>> Chas. >>>> _______________________________________________ >>>> BaseX-Talk mailing list >>>> BaseX-Talk at mailman.uni-konstanz.de >>>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>> >>> >