BaseX Failing After a Number of Requests
Wednesday, 4 December 2013 · 3 messages · last reply 6 Dec 2013
JM
Jeremy Moseley4 Dec 2013 21:06 #
Hi Guys,
We have implemented BaseX to provide a REST API to access specific information out of the database. Databases range in size from hundreds to 6000+ XML files, and the APIs use the attribute index to quickly locate nodes and return information about them.
We have been noticing that BaseX has been randomly failing after a number of requests in the range of several hundred. I can replicate the failure in a few minutes by looping curl requests from the command line.
The failure is made apparent by a valid request returning a Jetty 404 Error:
HTTP ERROR 404
Quoted text · 8 lines
> > Problem accessing { redacted URL }. Reason: > > No function found that matches the request. > > ------------------------------ > *Powered by Jetty://* >
This message is the same message that is thrown when requesting a URL that does not have a matching path, but this request is valid, and has been used before. It appears that the issue is the BaseX server failing somehow, but I can't track the reason. No information is logged at the time of the crash.
If I connect to the BaseX server (on port 1984) using the basexclient, it connects, but reports that there are no databases available. Once I restart the service, I can reconnect, and it lists all databases as it should.
Any ideas what might be causing these issues?
Thanks,
Jeremy
CG
Christian Grün6 Dec 2013 17:22 #
Hi Jeremy,
I guess your question is related to Joe Templeman’s posting a few days ago?
I would really be interested to hear if other users have encountered this phenomena, as we haven’t encountered this or similar behavior in our own RESTXQ applications. Have you checked out the BaseX debugging output (-d) and, could you please provide us with the last lines of the .logs file contents? Have you tried to start BaseX in embedded mode, setting
HTTPLOCAL to true [1]?
Christian
[1] http://docs.basex.org/wiki/Web_ApplicationQuoted text · Jeremy Moseley · 44 lines
On Wed, Dec 4, 2013 at 8:05 PM, Jeremy Moseley <jeremy at inkling.com> wrote: > Hi Guys, > > We have implemented BaseX to provide a REST API to access specific > information out of the database. Databases range in size from hundreds to > 6000+ XML files, and the APIs use the attribute index to quickly locate > nodes and return information about them. > > We have been noticing that BaseX has been randomly failing after a number of > requests in the range of several hundred. I can replicate the failure in a > few minutes by looping curl requests from the command line. > > The failure is made apparent by a valid request returning a Jetty 404 Error: > >> HTTP ERROR 404 >> >> Problem accessing { redacted URL }. Reason: >> >> No function found that matches the request. >> >> ________________________________ >> Powered by Jetty:// > > > This message is the same message that is thrown when requesting a URL that > does not have a matching path, but this request is valid, and has been used > before. It appears that the issue is the BaseX server failing somehow, but I > can't track the reason. No information is logged at the time of the crash. > > If I connect to the BaseX server (on port 1984) using the basexclient, it > connects, but reports that there are no databases available. Once I restart > the service, I can reconnect, and it lists all databases as it should. > > Any ideas what might be causing these issues? > > Thanks, > > Jeremy > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >
JM
Jeremy Moseley6 Dec 2013 20:55 #
Hi Christian,
After finding the problem it makes sense that others have not encountered it. The problem was the process running out of file descriptors and/or UDP ports. We were calling some native Java classes that grabbed configurations from disk and sent metrics information to an external service, and it appears these classes were not properly disposing of the open file descriptors and the UDP connections. I don't have enough information about the functionality of the Java bindings in BaseX to know exactly where this was falling down, but we solved it by limiting the files opened and the number of UDP connections and the problem went away.
Our hypothesis is that the File open was failing for any file, which explains why BaseX would report that there were no databases loaded.
Thanks for your help,
Jeremy
On Fri, Dec 6, 2013 at 8:22 AM, Christian Grün <christian.gruen at gmail.com>wrote:
Quoted text · Christian Grün · 69 lines
> Hi Jeremy, > > I guess your question is related to Joe Templeman’s posting a few days ago? > > I would really be interested to hear if other users have encountered > this phenomena, as we haven’t encountered this or similar behavior in > our own RESTXQ applications. Have you checked out the BaseX debugging > output (-d) and, could you please provide us with the last lines of > the .logs file contents? Have you tried to start BaseX in embedded > mode, setting HTTPLOCAL to true [1]? > > Christian > > [1] http://docs.basex.org/wiki/Web_Application > ___________________________ > > On Wed, Dec 4, 2013 at 8:05 PM, Jeremy Moseley <jeremy at inkling.com> wrote: > > Hi Guys, > > > > We have implemented BaseX to provide a REST API to access specific > > information out of the database. Databases range in size from hundreds to > > 6000+ XML files, and the APIs use the attribute index to quickly locate > > nodes and return information about them. > > > > We have been noticing that BaseX has been randomly failing after a > number of > > requests in the range of several hundred. I can replicate the failure in > a > > few minutes by looping curl requests from the command line. > > > > The failure is made apparent by a valid request returning a Jetty 404 > Error: > > > >> HTTP ERROR 404 > >> > >> Problem accessing { redacted URL }. Reason: > >> > >> No function found that matches the request. > >> > >> ________________________________ > >> Powered by Jetty:// > > > > > > This message is the same message that is thrown when requesting a URL > that > > does not have a matching path, but this request is valid, and has been > used > > before. It appears that the issue is the BaseX server failing somehow, > but I > > can't track the reason. No information is logged at the time of the > crash. > > > > If I connect to the BaseX server (on port 1984) using the basexclient, it > > connects, but reports that there are no databases available. Once I > restart > > the service, I can reconnect, and it lists all databases as it should. > > > > Any ideas what might be causing these issues? > > > > Thanks, > > > > Jeremy > > > > _______________________________________________ > > BaseX-Talk mailing list > > BaseX-Talk at mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > >
Written for BaseX 7.7. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery