Server Variables, cached vars, etc
Tuesday, 5 September 2017 · 16 messages · last reply 21 Sep 2017
EP
Erik Peterson5 Sep 2017 03:37 #
How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?
XS
Xavier-Laurent SALVADOR5 Sep 2017 08:42 #
Hi,
do you mean something like a repo variable ?
If you define a module name in your repo:
1- module namespace myNs = 'http://www.xxxxx.fr/xxxx-repo’ <http://www.xxxxx.fr/xxxx-repo%E2%80%99>;
You can define in your repo a new variable :
1- declare variable $myNs:testid := db:open(‘xxxxx')/utilisateurs/entry/sessions/session/id=session:id();Then import your module in your restxq:
1- import module namespace isi = 'http://www.isilex.fr/isi-repo’ <http://www.isilex.fr/isi-repo%E2%80%99>;
And access your variables:
1- $myNs:testid
More here: http://docs.basex.org/wiki/XQuery_3.0#External_Variables <http://docs.basex.org/wiki/XQuery_3.0#External_Variables>
Sorry If I’m wrong and didn’t understand,
XLS
Quoted text · Erik Peterson · 3 lines
> Le 5 sept. 2017 à 01:37, Erik Peterson <ep at ardec.com> a écrit : > > How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?
ML
Quoted text · Erik Peterson · 5 lines
On 05/09/2017 01:37, Erik Peterson wrote: > How can I create a variable that is evaluated only once but accessed > across many RestXQ requests and sessions. I'm trying to cache data > that comes from an integration with an expensive operation. Does BaseX > support something similar to server variables like Mark Logic?
Hi Erik,
AFAIK you have the following possibilities to keep a variable live accross multiple RestXQ calls:
1) Use session http://docs.basex.org/wiki/Session_Module
2) Use a database which is builtin in BaseX and is very lightweight. Especially if your data is serializable to XML you could benefit also from indexes to speed up access to your cached objects.
3) Use the file system.
Hope this helps [cit] ;-)
Marco.
FE
Fabrice ETANCHAUD5 Sep 2017 09:29 #
Hi all,
Another solution is to share a main memory database, that behaves like a memory cache. In Client/Server mode, any main memory created by one client is available to all the other ones.
Best regards,
Fabrice
-----Message d'origine-----
Quoted text · Marco Lettere · 17 lines
De : basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Marco Lettere Envoyé : mardi 5 septembre 2017 09:14 À : basex-talk at mailman.uni-konstanz.de Objet : Re: [basex-talk] Server Variables, cached vars, etc On 05/09/2017 01:37, Erik Peterson wrote: > How can I create a variable that is evaluated only once but accessed > across many RestXQ requests and sessions. I'm trying to cache data > that comes from an integration with an expensive operation. Does BaseX > support something similar to server variables like Mark Logic? Hi Erik, AFAIK you have the following possibilities to keep a variable live accross multiple RestXQ calls: 1) Use session http://docs.basex.org/wiki/Session_Module 2) Use a database which is builtin in BaseX and is very lightweight. Especially if your data is serializable to XML you could benefit also from indexes to speed up access to your cached objects. 3) Use the file system. Hope this helps [cit] ;-) Marco.
KS
A problem with that is that it evaluates the variable each time. If you need it to only evaluate once (expensive operation), something else needs to be done. I would like to know an answer to this so that I can schedule jobs on start up without involving processes external to basex.
Quoted text · Xavier-Laurent SALVADOR · 20 lines
From: <basex-talk-bounces at mailman.uni-konstanz.de> on behalf of Xavier-Laurent SALVADOR <xavierlaurent.salvador at gmail.com> Date: Monday, September 4, 2017 at 11:42 PM To: Erik Peterson <ep at ardec.com> Cc: BaseX <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Server Variables, cached vars, etc Hi, do you mean something like a repo variable ? If you define a module name in your repo: 1- module namespace myNs = 'http://www.xxxxx.fr/xxxx-repo’<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.xxxxx.fr_xxxx-2Drepo-25E2-2580-2599&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=6BbM4k_uDRdhjURAbaUe8SgS19qfgxyIM03BQbY2xzk&s=vnUoDRkXSxl-NAs4wmOr8q2saq5Y9K0_k6l71w-jrrM&e=>; You can define in your repo a new variable : 1- declare variable $myNs:testid := db:open(‘xxxxx')/utilisateurs/entry/sessions/session/id=session:id(); Then import your module in your restxq: 1- import module namespace isi = 'http://www.isilex.fr/isi-repo’<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.isilex.fr_isi-2Drepo-25E2-2580-2599&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=6BbM4k_uDRdhjURAbaUe8SgS19qfgxyIM03BQbY2xzk&s=7iT9husSt3pxwCXxx8Kf7Sf1KKDDurqvyhDHERP1p2g&e=>; And access your variables: 1- $myNs:testid More here: http://docs.basex.org/wiki/XQuery_3.0#External_Variables<https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.basex.org_wiki_XQuery-5F3.0-23External-5FVariables&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=6BbM4k_uDRdhjURAbaUe8SgS19qfgxyIM03BQbY2xzk&s=tfVbcjLW6lGgTfWHzfgWtKLPja7kdFONHW74KF_CdiA&e=> Sorry If I’m wrong and didn’t understand, XLS Le 5 sept. 2017 à 01:37, Erik Peterson <ep at ardec.com<mailto:ep at ardec.com>> a écrit : How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?
FE
To be confirmed : there is no 'start script' server option.
I do manually create and populate the mainmem db in the dba query interface.
Best regards,
Fabrice
-----Message d'origine-----
Quoted text · Fabrice ETANCHAUD · 25 lines
De : Fabrice ETANCHAUD Envoyé : mardi 5 septembre 2017 09:29 À : 'Marco Lettere'; basex-talk at mailman.uni-konstanz.de Objet : RE: [basex-talk] Server Variables, cached vars, etc Hi all, Another solution is to share a main memory database, that behaves like a memory cache. In Client/Server mode, any main memory created by one client is available to all the other ones. Best regards, Fabrice -----Message d'origine----- De : basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Marco Lettere Envoyé : mardi 5 septembre 2017 09:14 À : basex-talk at mailman.uni-konstanz.de Objet : Re: [basex-talk] Server Variables, cached vars, etc On 05/09/2017 01:37, Erik Peterson wrote: > How can I create a variable that is evaluated only once but accessed > across many RestXQ requests and sessions. I'm trying to cache data > that comes from an integration with an expensive operation. Does BaseX > support something similar to server variables like Mark Logic? Hi Erik, AFAIK you have the following possibilities to keep a variable live accross multiple RestXQ calls: 1) Use session http://docs.basex.org/wiki/Session_Module 2) Use a database which is builtin in BaseX and is very lightweight. Especially if your data is serializable to XML you could benefit also from indexes to speed up access to your cached objects. 3) Use the file system. Hope this helps [cit] ;-) Marco.
EP
Erik Peterson5 Sep 2017 16:02 #
Thank you all for your replys. It looks like a main memory database is the best "built in" option. However, I have created Jar file to drop him/lib with a Java Singleton object...holding a map. That should be accessible across requests and sessions. The question is how to populate this just once upon start up? Perhaps I could do a job that would do that? Also I could memoize the variables in a global script. That way the expensive operation is only run the first time it is needed.
Any other suggestions welcome. Recommend that a standard built-in feature be added to handle these scenarios.
Quoted text · Fabrice ETANCHAUD · 57 lines
On Tue, Sep 5, 2017 at 1:33 AM Fabrice ETANCHAUD < fetanchaud at pch.cerfrance.fr> wrote: > To be confirmed : there is no 'start script' server option. > I do manually create and populate the mainmem db in the dba query > interface. > > Best regards, > Fabrice > > -----Message d'origine----- > De : Fabrice ETANCHAUD > Envoyé : mardi 5 septembre 2017 09:29 > À : 'Marco Lettere'; basex-talk at mailman.uni-konstanz.de > Objet : RE: [basex-talk] Server Variables, cached vars, etc > > Hi all, > > Another solution is to share a main memory database, that behaves like a > memory cache. > In Client/Server mode, any main memory created by one client is available > to all the other ones. > > Best regards, > Fabrice > > > -----Message d'origine----- > De : basex-talk-bounces at mailman.uni-konstanz.de [mailto: > basex-talk-bounces at mailman.uni-konstanz.de] De la part de Marco Lettere > Envoyé : mardi 5 septembre 2017 09:14 À : > basex-talk at mailman.uni-konstanz.de > Objet : Re: [basex-talk] Server Variables, cached vars, etc > > On 05/09/2017 01:37, Erik Peterson wrote: > > How can I create a variable that is evaluated only once but accessed > > across many RestXQ requests and sessions. I'm trying to cache data > > that comes from an integration with an expensive operation. Does BaseX > > support something similar to server variables like Mark Logic? > > Hi Erik, > > AFAIK you have the following possibilities to keep a variable live accross > multiple RestXQ calls: > > 1) Use session http://docs.basex.org/wiki/Session_Module > > 2) Use a database which is builtin in BaseX and is very lightweight. > Especially if your data is serializable to XML you could benefit also from > indexes to speed up access to your cached objects. > > 3) Use the file system. > > Hope this helps [cit] ;-) > > Marco. > > --
*Erik Peterson*
President, Ardec LLC
281-804-9023 | ep at ardec.com
KS
Kendall Shaw8 Sep 2017 19:12 #
I thought it might be useful to mention advice I was given about startup hooks:
Quoted text · Erik Peterson · 3 lines
> From: "Kirsten, Dirk" Dirk.Kirsten at senacor.com<mailto:Dirk.Kirsten at senacor.com> ,,, > there is currently no way to do this using BaseX itself. But I also don’t think that should be the job of BaseX. Instead you can write a servlet and deploy it using Tomcat which runs some Java application, e.g. which could trigger some BaseXX command. See http://crunchify.com/how-to-run-java-program-automatically-on-tomcat-startup/ for an example how to do this.
I switched from using a cron job, to doing this in order to schedule jobs. I have very simple servlet that is configured with <load-on-startup>2</load-on-startup> (basex has load-on-startup 2). It runs a shell script which schedules the jobs, soon after basex is loaded.
Kendall
Quoted text · Erik Peterson · 41 lines
From: <basex-talk-bounces at mailman.uni-konstanz.de> on behalf of Erik Peterson <ep at ardec.com> Date: Tuesday, September 5, 2017 at 7:02 AM To: Fabrice ETANCHAUD <fetanchaud at pch.cerfrance.fr>, "basex-talk at mailman.uni-konstanz.de" <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Server Variables, cached vars, etc Thank you all for your replys. It looks like a main memory database is the best "built in" option. However, I have created Jar file to drop him/lib with a Java Singleton object...holding a map. That should be accessible across requests and sessions. The question is how to populate this just once upon start up? Perhaps I could do a job that would do that? Also I could memoize the variables in a global script. That way the expensive operation is only run the first time it is needed. Any other suggestions welcome. Recommend that a standard built-in feature be added to handle these scenarios. On Tue, Sep 5, 2017 at 1:33 AM Fabrice ETANCHAUD <fetanchaud at pch.cerfrance.fr<mailto:fetanchaud at pch.cerfrance.fr>> wrote: To be confirmed : there is no 'start script' server option. I do manually create and populate the mainmem db in the dba query interface. Best regards, Fabrice -----Message d'origine----- De : Fabrice ETANCHAUD Envoyé : mardi 5 septembre 2017 09:29 À : 'Marco Lettere'; basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Objet : RE: [basex-talk] Server Variables, cached vars, etc Hi all, Another solution is to share a main memory database, that behaves like a memory cache. In Client/Server mode, any main memory created by one client is available to all the other ones. Best regards, Fabrice -----Message d'origine----- De : basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de> [mailto:basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de>] De la part de Marco Lettere Envoyé : mardi 5 septembre 2017 09:14 À : basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Objet : Re: [basex-talk] Server Variables, cached vars, etc On 05/09/2017 01:37, Erik Peterson wrote: > How can I create a variable that is evaluated only once but accessed > across many RestXQ requests and sessions. I'm trying to cache data > that comes from an integration with an expensive operation. Does BaseX > support something similar to server variables like Mark Logic? Hi Erik, AFAIK you have the following possibilities to keep a variable live accross multiple RestXQ calls: 1) Use session http://docs.basex.org/wiki/Session_Module<https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.basex.org_wiki_Session-5FModule&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=DfAtFmXgoUqMIP116Hn78cCdB0klwv2oiZZ8v8di0Ew&s=VUiY8Im77MBCa2uPCzDsV7HRZEC-Sqx2WNzt_CD9Iok&e=> 2) Use a database which is builtin in BaseX and is very lightweight. Especially if your data is serializable to XML you could benefit also from indexes to speed up access to your cached objects. 3) Use the file system. Hope this helps [cit] ;-) Marco. -- Erik Peterson President, Ardec LLC 281-804-9023 | ep at ardec.com<mailto:ep at ardec.com>
EP
Erik Peterson9 Sep 2017 15:16 #
Thanks Kendal for your reply. What would be the advantage of creating a servlet over a singleton class to do the same thing?
Quoted text · Kendall Shaw · 121 lines
On Fri, Sep 8, 2017 at 11:12 AM, Kendall Shaw <kendall.shaw at workday.com> wrote: > I thought it might be useful to mention advice I was given about startup > hooks: > > > > > From: "Kirsten, Dirk" Dirk.Kirsten at senacor.com > > ,,, > > > there is currently no way to do this using BaseX itself. But I also > don’t think that should be the job of BaseX. Instead you can write a > servlet and deploy it using Tomcat which runs some Java application, e.g. > which could trigger some BaseXX command. See http://crunchify.com/how-to- > run-java-program-automatically-on-tomcat-startup/ for an example how to > do this. > > > > I switched from using a cron job, to doing this in order to schedule > jobs. I have very simple servlet that is configured with > <load-on-startup>2</load-on-startup> (basex has load-on-startup 2). It > runs a shell script which schedules the jobs, soon after basex is loaded. > > > > Kendall > > > > *From: *<basex-talk-bounces at mailman.uni-konstanz.de> on behalf of Erik > Peterson <ep at ardec.com> > *Date: *Tuesday, September 5, 2017 at 7:02 AM > *To: *Fabrice ETANCHAUD <fetanchaud at pch.cerfrance.fr>, " > basex-talk at mailman.uni-konstanz.de" <basex-talk at mailman.uni-konstanz.de> > *Subject: *Re: [basex-talk] Server Variables, cached vars, etc > > > > Thank you all for your replys. It looks like a main memory database is > the best "built in" option. However, I have created Jar file to drop > him/lib with a Java Singleton object...holding a map. That should be > accessible across requests and sessions. The question is how to populate > this just once upon start up? Perhaps I could do a job that would do > that? Also I could memoize the variables in a global script. That way the > expensive operation is only run the first time it is needed. > > > > Any other suggestions welcome. Recommend that a standard built-in feature > be added to handle these scenarios. > > > > On Tue, Sep 5, 2017 at 1:33 AM Fabrice ETANCHAUD < > fetanchaud at pch.cerfrance.fr> wrote: > > To be confirmed : there is no 'start script' server option. > I do manually create and populate the mainmem db in the dba query > interface. > > Best regards, > Fabrice > > -----Message d'origine----- > De : Fabrice ETANCHAUD > Envoyé : mardi 5 septembre 2017 09:29 > À : 'Marco Lettere'; basex-talk at mailman.uni-konstanz.de > Objet : RE: [basex-talk] Server Variables, cached vars, etc > > Hi all, > > Another solution is to share a main memory database, that behaves like a > memory cache. > In Client/Server mode, any main memory created by one client is available > to all the other ones. > > Best regards, > Fabrice > > > -----Message d'origine----- > De : basex-talk-bounces at mailman.uni-konstanz.de [mailto: > basex-talk-bounces at mailman.uni-konstanz.de] De la part de Marco Lettere > Envoyé : mardi 5 septembre 2017 09:14 À : basex-talk at mailman.uni- > konstanz.de > Objet : Re: [basex-talk] Server Variables, cached vars, etc > > On 05/09/2017 01:37, Erik Peterson wrote: > > How can I create a variable that is evaluated only once but accessed > > across many RestXQ requests and sessions. I'm trying to cache data > > that comes from an integration with an expensive operation. Does BaseX > > support something similar to server variables like Mark Logic? > > Hi Erik, > > AFAIK you have the following possibilities to keep a variable live accross > multiple RestXQ calls: > > 1) Use session http://docs.basex.org/wiki/Session_Module > <https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.basex.org_wiki_Session-5FModule&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=DfAtFmXgoUqMIP116Hn78cCdB0klwv2oiZZ8v8di0Ew&s=VUiY8Im77MBCa2uPCzDsV7HRZEC-Sqx2WNzt_CD9Iok&e=> > > 2) Use a database which is builtin in BaseX and is very lightweight. > Especially if your data is serializable to XML you could benefit also from > indexes to speed up access to your cached objects. > > 3) Use the file system. > > Hope this helps [cit] ;-) > > Marco. > > -- > > *Erik Peterson* > > President, Ardec LLC > > 281-804-9023 <(281)%20804-9023> | ep at ardec.com >
--
*Erik Peterson*
President, Ardec LLC
281-804-9023 | ep at ardec.com
KS
Kendall Shaw10 Sep 2017 01:56 #
The servlet could populate your singleton just once upon startup, or run xquery etc. The load-on-startup configuration means that the servlet is initialized after basex has been loaded. So, if you restart jetty or whatever web server/web container you are using basex restarts and then your servlet’s init method is invoked.
Kendall
Quoted text · Erik Peterson · 58 lines
From: Erik Peterson <ep at ardec.com> Date: Saturday, September 9, 2017 at 4:16 AM To: Kendall Shaw <kendall.shaw at workday.com> Cc: "basex-talk at mailman.uni-konstanz.de" <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Server Variables, cached vars, etc Thanks Kendal for your reply. What would be the advantage of creating a servlet over a singleton class to do the same thing? On Fri, Sep 8, 2017 at 11:12 AM, Kendall Shaw <kendall.shaw at workday.com<mailto:kendall.shaw at workday.com>> wrote: I thought it might be useful to mention advice I was given about startup hooks: > From: "Kirsten, Dirk" Dirk.Kirsten at senacor.com<mailto:Dirk.Kirsten at senacor.com> ,,, > there is currently no way to do this using BaseX itself. But I also don’t think that should be the job of BaseX. Instead you can write a servlet and deploy it using Tomcat which runs some Java application, e.g. which could trigger some BaseXX command. See http://crunchify.com/how-to-run-java-program-automatically-on-tomcat-startup/<https://urldefense.proofpoint.com/v2/url?u=http-3A__crunchify.com_how-2Dto-2Drun-2Djava-2Dprogram-2Dautomatically-2Don-2Dtomcat-2Dstartup_&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=fPChED25tvjfNbUFkPSuyRBFfpYGFrqPI1DLNeEovLI&s=0q_rTZpxZNOefv3G77E1Vna_JmrxWcl0OcoKc1EmHAc&e=> for an example how to do this. I switched from using a cron job, to doing this in order to schedule jobs. I have very simple servlet that is configured with <load-on-startup>2</load-on-startup> (basex has load-on-startup 2). It runs a shell script which schedules the jobs, soon after basex is loaded. Kendall From: <basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de>> on behalf of Erik Peterson <ep at ardec.com<mailto:ep at ardec.com>> Date: Tuesday, September 5, 2017 at 7:02 AM To: Fabrice ETANCHAUD <fetanchaud at pch.cerfrance.fr<mailto:fetanchaud at pch.cerfrance.fr>>, "basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de>" <basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de>> Subject: Re: [basex-talk] Server Variables, cached vars, etc Thank you all for your replys. It looks like a main memory database is the best "built in" option. However, I have created Jar file to drop him/lib with a Java Singleton object...holding a map. That should be accessible across requests and sessions. The question is how to populate this just once upon start up? Perhaps I could do a job that would do that? Also I could memoize the variables in a global script. That way the expensive operation is only run the first time it is needed. Any other suggestions welcome. Recommend that a standard built-in feature be added to handle these scenarios. On Tue, Sep 5, 2017 at 1:33 AM Fabrice ETANCHAUD <fetanchaud at pch.cerfrance.fr<mailto:fetanchaud at pch.cerfrance.fr>> wrote: To be confirmed : there is no 'start script' server option. I do manually create and populate the mainmem db in the dba query interface. Best regards, Fabrice -----Message d'origine----- De : Fabrice ETANCHAUD Envoyé : mardi 5 septembre 2017 09:29 À : 'Marco Lettere'; basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Objet : RE: [basex-talk] Server Variables, cached vars, etc Hi all, Another solution is to share a main memory database, that behaves like a memory cache. In Client/Server mode, any main memory created by one client is available to all the other ones. Best regards, Fabrice -----Message d'origine----- De : basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de> [mailto:basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de>] De la part de Marco Lettere Envoyé : mardi 5 septembre 2017 09:14 À : basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Objet : Re: [basex-talk] Server Variables, cached vars, etc On 05/09/2017 01:37, Erik Peterson wrote: > How can I create a variable that is evaluated only once but accessed > across many RestXQ requests and sessions. I'm trying to cache data > that comes from an integration with an expensive operation. Does BaseX > support something similar to server variables like Mark Logic? Hi Erik, AFAIK you have the following possibilities to keep a variable live accross multiple RestXQ calls: 1) Use session http://docs.basex.org/wiki/Session_Module<https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.basex.org_wiki_Session-5FModule&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=DfAtFmXgoUqMIP116Hn78cCdB0klwv2oiZZ8v8di0Ew&s=VUiY8Im77MBCa2uPCzDsV7HRZEC-Sqx2WNzt_CD9Iok&e=> 2) Use a database which is builtin in BaseX and is very lightweight. Especially if your data is serializable to XML you could benefit also from indexes to speed up access to your cached objects. 3) Use the file system. Hope this helps [cit] ;-) Marco. -- Erik Peterson President, Ardec LLC 281-804-9023<tel:(281)%20804-9023> | ep at ardec.com<mailto:ep at ardec.com> -- Erik Peterson President, Ardec LLC 281-804-9023 | ep at ardec.com<mailto:ep at ardec.com>
CG
Christian Grün18 Sep 2017 16:19 #
Hi Erik,
I think that Xavier-Laurent, Marco, Fabrice and Kendall have already given excellent feedback.
In our own projects, we store all global data in databases, or in local configuration files. One advantage is that this data requires no initialization and will automatically be available after a restart.
I don’t know anything about »server variables« in MarkLogic so far, so @Erik: feel free to pass me on a link to the documentation, and I can check if a similar solution could make sense for BaseX.
Talking about the start script server option: The basexserver command comes with a -c flag, which allows you run initial commands [1]. We could add such a flag for basexhttp, or even allow an initial input for both startup commands (similar to basex/basexclient). Would this be helpful for some of you reading this? Quite obviously, this requires BaseX to be run via these scripts (it wouldn’t have any effect if BaseX is deployed as servlet).
Cheers,
Christian
[1] http://docs.basex.org/wiki/Command-Line_Options#Server
Quoted text · Kendall Shaw · 160 lines
On Sun, Sep 10, 2017 at 1:56 AM, Kendall Shaw <kendall.shaw at workday.com> wrote: > The servlet could populate your singleton just once upon startup, or run > xquery etc. The load-on-startup configuration means that the servlet is > initialized after basex has been loaded. So, if you restart jetty or > whatever web server/web container you are using basex restarts and then your > servlet’s init method is invoked. > > > > Kendall > > > > From: Erik Peterson <ep at ardec.com> > Date: Saturday, September 9, 2017 at 4:16 AM > To: Kendall Shaw <kendall.shaw at workday.com> > Cc: "basex-talk at mailman.uni-konstanz.de" > <basex-talk at mailman.uni-konstanz.de> > > > Subject: Re: [basex-talk] Server Variables, cached vars, etc > > > > Thanks Kendal for your reply. What would be the advantage of creating a > servlet over a singleton class to do the same thing? > > > > On Fri, Sep 8, 2017 at 11:12 AM, Kendall Shaw <kendall.shaw at workday.com> > wrote: > > I thought it might be useful to mention advice I was given about startup > hooks: > > > >> From: "Kirsten, Dirk" Dirk.Kirsten at senacor.com > > ,,, > >> there is currently no way to do this using BaseX itself. But I also don’t >> think that should be the job of BaseX. Instead you can write a servlet and >> deploy it using Tomcat which runs some Java application, e.g. which could >> trigger some BaseXX command. See >> http://crunchify.com/how-to-run-java-program-automatically-on-tomcat-startup/ >> for an example how to do this. > > > > I switched from using a cron job, to doing this in order to schedule jobs. > I have very simple servlet that is configured with > <load-on-startup>2</load-on-startup> (basex has load-on-startup 2). It runs > a shell script which schedules the jobs, soon after basex is loaded. > > > > Kendall > > > > From: <basex-talk-bounces at mailman.uni-konstanz.de> on behalf of Erik > Peterson <ep at ardec.com> > Date: Tuesday, September 5, 2017 at 7:02 AM > To: Fabrice ETANCHAUD <fetanchaud at pch.cerfrance.fr>, > "basex-talk at mailman.uni-konstanz.de" <basex-talk at mailman.uni-konstanz.de> > Subject: Re: [basex-talk] Server Variables, cached vars, etc > > > > Thank you all for your replys. It looks like a main memory database is the > best "built in" option. However, I have created Jar file to drop him/lib > with a Java Singleton object...holding a map. That should be accessible > across requests and sessions. The question is how to populate this just > once upon start up? Perhaps I could do a job that would do that? Also I > could memoize the variables in a global script. That way the expensive > operation is only run the first time it is needed. > > > > Any other suggestions welcome. Recommend that a standard built-in feature > be added to handle these scenarios. > > > > On Tue, Sep 5, 2017 at 1:33 AM Fabrice ETANCHAUD > <fetanchaud at pch.cerfrance.fr> wrote: > > To be confirmed : there is no 'start script' server option. > I do manually create and populate the mainmem db in the dba query interface. > > Best regards, > Fabrice > > -----Message d'origine----- > De : Fabrice ETANCHAUD > Envoyé : mardi 5 septembre 2017 09:29 > À : 'Marco Lettere'; basex-talk at mailman.uni-konstanz.de > Objet : RE: [basex-talk] Server Variables, cached vars, etc > > Hi all, > > Another solution is to share a main memory database, that behaves like a > memory cache. > In Client/Server mode, any main memory created by one client is available to > all the other ones. > > Best regards, > Fabrice > > > -----Message d'origine----- > De : basex-talk-bounces at mailman.uni-konstanz.de > [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Marco > Lettere Envoyé : mardi 5 septembre 2017 09:14 À : > basex-talk at mailman.uni-konstanz.de > Objet : Re: [basex-talk] Server Variables, cached vars, etc > > On 05/09/2017 01:37, Erik Peterson wrote: >> How can I create a variable that is evaluated only once but accessed >> across many RestXQ requests and sessions. I'm trying to cache data >> that comes from an integration with an expensive operation. Does BaseX >> support something similar to server variables like Mark Logic? > > Hi Erik, > > AFAIK you have the following possibilities to keep a variable live accross > multiple RestXQ calls: > > 1) Use session http://docs.basex.org/wiki/Session_Module > > 2) Use a database which is builtin in BaseX and is very lightweight. > Especially if your data is serializable to XML you could benefit also from > indexes to speed up access to your cached objects. > > 3) Use the file system. > > Hope this helps [cit] ;-) > > Marco. > > -- > > Erik Peterson > > President, Ardec LLC > > 281-804-9023 | ep at ardec.com > > > > > > -- > > Erik Peterson > > President, Ardec LLC > > 281-804-9023 | ep at ardec.com
ML
Marco Lettere18 Sep 2017 16:39 #
Hi Christian,
welcome back to the list!
Having the possibility to run a script or call any sort of cuntvionality at startup is quite a useful thing.
Doing it in a declarative way would be great maybe exploiting the .basex file or the web.xml?
Btw, hooking up on Server variables, a colleague of mine has just come up with another (somewhat exotic) way of passing content from one RestXQ call to another using
proc:system to set and get environment variables variables. ;-)
Regards,
Marco.Quoted text · Christian Grün · 186 lines
On 18/09/2017 16:19, Christian Grün wrote: > Hi Erik, > > I think that Xavier-Laurent, Marco, Fabrice and Kendall have already > given excellent feedback. > > In our own projects, we store all global data in databases, or in > local configuration files. One advantage is that this data requires no > initialization and will automatically be available after a restart. > > I don’t know anything about »server variables« in MarkLogic so far, so > @Erik: feel free to pass me on a link to the documentation, and I can > check if a similar solution could make sense for BaseX. > > Talking about the start script server option: The basexserver command > comes with a -c flag, which allows you run initial commands [1]. We > could add such a flag for basexhttp, or even allow an initial input > for both startup commands (similar to basex/basexclient). Would this > be helpful for some of you reading this? Quite obviously, this > requires BaseX to be run via these scripts (it wouldn’t have any > effect if BaseX is deployed as servlet). > > Cheers, > Christian > > [1] http://docs.basex.org/wiki/Command-Line_Options#Server > > > > On Sun, Sep 10, 2017 at 1:56 AM, Kendall Shaw <kendall.shaw at workday.com> wrote: >> The servlet could populate your singleton just once upon startup, or run >> xquery etc. The load-on-startup configuration means that the servlet is >> initialized after basex has been loaded. So, if you restart jetty or >> whatever web server/web container you are using basex restarts and then your >> servlet’s init method is invoked. >> >> >> >> Kendall >> >> >> >> From: Erik Peterson <ep at ardec.com> >> Date: Saturday, September 9, 2017 at 4:16 AM >> To: Kendall Shaw <kendall.shaw at workday.com> >> Cc: "basex-talk at mailman.uni-konstanz.de" >> <basex-talk at mailman.uni-konstanz.de> >> >> >> Subject: Re: [basex-talk] Server Variables, cached vars, etc >> >> >> >> Thanks Kendal for your reply. What would be the advantage of creating a >> servlet over a singleton class to do the same thing? >> >> >> >> On Fri, Sep 8, 2017 at 11:12 AM, Kendall Shaw <kendall.shaw at workday.com> >> wrote: >> >> I thought it might be useful to mention advice I was given about startup >> hooks: >> >> >> >>> From: "Kirsten, Dirk" Dirk.Kirsten at senacor.com >> ,,, >> >>> there is currently no way to do this using BaseX itself. But I also don’t >>> think that should be the job of BaseX. Instead you can write a servlet and >>> deploy it using Tomcat which runs some Java application, e.g. which could >>> trigger some BaseXX command. See >>> http://crunchify.com/how-to-run-java-program-automatically-on-tomcat-startup/ >>> for an example how to do this. >> >> >> I switched from using a cron job, to doing this in order to schedule jobs. >> I have very simple servlet that is configured with >> <load-on-startup>2</load-on-startup> (basex has load-on-startup 2). It runs >> a shell script which schedules the jobs, soon after basex is loaded. >> >> >> >> Kendall >> >> >> >> From: <basex-talk-bounces at mailman.uni-konstanz.de> on behalf of Erik >> Peterson <ep at ardec.com> >> Date: Tuesday, September 5, 2017 at 7:02 AM >> To: Fabrice ETANCHAUD <fetanchaud at pch.cerfrance.fr>, >> "basex-talk at mailman.uni-konstanz.de" <basex-talk at mailman.uni-konstanz.de> >> Subject: Re: [basex-talk] Server Variables, cached vars, etc >> >> >> >> Thank you all for your replys. It looks like a main memory database is the >> best "built in" option. However, I have created Jar file to drop him/lib >> with a Java Singleton object...holding a map. That should be accessible >> across requests and sessions. The question is how to populate this just >> once upon start up? Perhaps I could do a job that would do that? Also I >> could memoize the variables in a global script. That way the expensive >> operation is only run the first time it is needed. >> >> >> >> Any other suggestions welcome. Recommend that a standard built-in feature >> be added to handle these scenarios. >> >> >> >> On Tue, Sep 5, 2017 at 1:33 AM Fabrice ETANCHAUD >> <fetanchaud at pch.cerfrance.fr> wrote: >> >> To be confirmed : there is no 'start script' server option. >> I do manually create and populate the mainmem db in the dba query interface. >> >> Best regards, >> Fabrice >> >> -----Message d'origine----- >> De : Fabrice ETANCHAUD >> Envoyé : mardi 5 septembre 2017 09:29 >> À : 'Marco Lettere'; basex-talk at mailman.uni-konstanz.de >> Objet : RE: [basex-talk] Server Variables, cached vars, etc >> >> Hi all, >> >> Another solution is to share a main memory database, that behaves like a >> memory cache. >> In Client/Server mode, any main memory created by one client is available to >> all the other ones. >> >> Best regards, >> Fabrice >> >> >> -----Message d'origine----- >> De : basex-talk-bounces at mailman.uni-konstanz.de >> [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Marco >> Lettere Envoyé : mardi 5 septembre 2017 09:14 À : >> basex-talk at mailman.uni-konstanz.de >> Objet : Re: [basex-talk] Server Variables, cached vars, etc >> >> On 05/09/2017 01:37, Erik Peterson wrote: >>> How can I create a variable that is evaluated only once but accessed >>> across many RestXQ requests and sessions. I'm trying to cache data >>> that comes from an integration with an expensive operation. Does BaseX >>> support something similar to server variables like Mark Logic? >> Hi Erik, >> >> AFAIK you have the following possibilities to keep a variable live accross >> multiple RestXQ calls: >> >> 1) Use session http://docs.basex.org/wiki/Session_Module >> >> 2) Use a database which is builtin in BaseX and is very lightweight. >> Especially if your data is serializable to XML you could benefit also from >> indexes to speed up access to your cached objects. >> >> 3) Use the file system. >> >> Hope this helps [cit] ;-) >> >> Marco. >> >> -- >> >> Erik Peterson >> >> President, Ardec LLC >> >> 281-804-9023 | ep at ardec.com >> >> >> >> >> >> -- >> >> Erik Peterson >> >> President, Ardec LLC >> >> 281-804-9023 | ep at ardec.com
FE
Hello Christian !
Yes, a -c option for the basexhttp would help, as mentioned earlier, for example creating a shared mainmem collection.
Best regards,
Fabrice
-----Message d'origine-----
Quoted text · Christian Grün · 178 lines
De : basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de Christian Grün Envoyé : lundi 18 septembre 2017 16:20 À : Kendall Shaw; coach3pete Cc : basex-talk at mailman.uni-konstanz.de Objet : Re: [basex-talk] Server Variables, cached vars, etc Hi Erik, I think that Xavier-Laurent, Marco, Fabrice and Kendall have already given excellent feedback. In our own projects, we store all global data in databases, or in local configuration files. One advantage is that this data requires no initialization and will automatically be available after a restart. I don’t know anything about »server variables« in MarkLogic so far, so @Erik: feel free to pass me on a link to the documentation, and I can check if a similar solution could make sense for BaseX. Talking about the start script server option: The basexserver command comes with a -c flag, which allows you run initial commands [1]. We could add such a flag for basexhttp, or even allow an initial input for both startup commands (similar to basex/basexclient). Would this be helpful for some of you reading this? Quite obviously, this requires BaseX to be run via these scripts (it wouldn’t have any effect if BaseX is deployed as servlet). Cheers, Christian [1] http://docs.basex.org/wiki/Command-Line_Options#Server On Sun, Sep 10, 2017 at 1:56 AM, Kendall Shaw <kendall.shaw at workday.com> wrote: > The servlet could populate your singleton just once upon startup, or > run xquery etc. The load-on-startup configuration means that the > servlet is initialized after basex has been loaded. So, if you restart > jetty or whatever web server/web container you are using basex > restarts and then your servlet’s init method is invoked. > > > > Kendall > > > > From: Erik Peterson <ep at ardec.com> > Date: Saturday, September 9, 2017 at 4:16 AM > To: Kendall Shaw <kendall.shaw at workday.com> > Cc: "basex-talk at mailman.uni-konstanz.de" > <basex-talk at mailman.uni-konstanz.de> > > > Subject: Re: [basex-talk] Server Variables, cached vars, etc > > > > Thanks Kendal for your reply. What would be the advantage of creating > a servlet over a singleton class to do the same thing? > > > > On Fri, Sep 8, 2017 at 11:12 AM, Kendall Shaw > <kendall.shaw at workday.com> > wrote: > > I thought it might be useful to mention advice I was given about > startup > hooks: > > > >> From: "Kirsten, Dirk" Dirk.Kirsten at senacor.com > > ,,, > >> there is currently no way to do this using BaseX itself. But I also >> don’t think that should be the job of BaseX. Instead you can write a >> servlet and deploy it using Tomcat which runs some Java application, >> e.g. which could trigger some BaseXX command. See >> http://crunchify.com/how-to-run-java-program-automatically-on-tomcat- >> startup/ >> for an example how to do this. > > > > I switched from using a cron job, to doing this in order to schedule jobs. > I have very simple servlet that is configured with > <load-on-startup>2</load-on-startup> (basex has load-on-startup 2). It > runs a shell script which schedules the jobs, soon after basex is loaded. > > > > Kendall > > > > From: <basex-talk-bounces at mailman.uni-konstanz.de> on behalf of Erik > Peterson <ep at ardec.com> > Date: Tuesday, September 5, 2017 at 7:02 AM > To: Fabrice ETANCHAUD <fetanchaud at pch.cerfrance.fr>, > "basex-talk at mailman.uni-konstanz.de" > <basex-talk at mailman.uni-konstanz.de> > Subject: Re: [basex-talk] Server Variables, cached vars, etc > > > > Thank you all for your replys. It looks like a main memory database > is the best "built in" option. However, I have created Jar file to > drop him/lib with a Java Singleton object...holding a map. That should be accessible > across requests and sessions. The question is how to populate this just > once upon start up? Perhaps I could do a job that would do that? > Also I could memoize the variables in a global script. That way the > expensive operation is only run the first time it is needed. > > > > Any other suggestions welcome. Recommend that a standard built-in > feature be added to handle these scenarios. > > > > On Tue, Sep 5, 2017 at 1:33 AM Fabrice ETANCHAUD > <fetanchaud at pch.cerfrance.fr> wrote: > > To be confirmed : there is no 'start script' server option. > I do manually create and populate the mainmem db in the dba query interface. > > Best regards, > Fabrice > > -----Message d'origine----- > De : Fabrice ETANCHAUD > Envoyé : mardi 5 septembre 2017 09:29 > À : 'Marco Lettere'; basex-talk at mailman.uni-konstanz.de > Objet : RE: [basex-talk] Server Variables, cached vars, etc > > Hi all, > > Another solution is to share a main memory database, that behaves like > a memory cache. > In Client/Server mode, any main memory created by one client is > available to all the other ones. > > Best regards, > Fabrice > > > -----Message d'origine----- > De : basex-talk-bounces at mailman.uni-konstanz.de > [mailto:basex-talk-bounces at mailman.uni-konstanz.de] De la part de > Marco Lettere Envoyé : mardi 5 septembre 2017 09:14 À : > basex-talk at mailman.uni-konstanz.de > Objet : Re: [basex-talk] Server Variables, cached vars, etc > > On 05/09/2017 01:37, Erik Peterson wrote: >> How can I create a variable that is evaluated only once but accessed >> across many RestXQ requests and sessions. I'm trying to cache data >> that comes from an integration with an expensive operation. Does >> BaseX support something similar to server variables like Mark Logic? > > Hi Erik, > > AFAIK you have the following possibilities to keep a variable live > accross multiple RestXQ calls: > > 1) Use session http://docs.basex.org/wiki/Session_Module > > 2) Use a database which is builtin in BaseX and is very lightweight. > Especially if your data is serializable to XML you could benefit also > from indexes to speed up access to your cached objects. > > 3) Use the file system. > > Hope this helps [cit] ;-) > > Marco. > > -- > > Erik Peterson > > President, Ardec LLC > > 281-804-9023 | ep at ardec.com > > > > > > -- > > Erik Peterson > > President, Ardec LLC > > 281-804-9023 | ep at ardec.com
CG
Christian Grün20 Sep 2017 19:37 #
Fabrice, Marco, thanks for your asseessment. After I have realized that it’ll take some more time to make this happen, I have added yet another issue for this feature request [1].
[1] https://github.com/BaseXdb/basex/issues/1499
Quoted text · Fabrice ETANCHAUD · 8 lines
On Mon, Sep 18, 2017 at 5:03 PM, Fabrice ETANCHAUD <fetanchaud at pch.cerfrance.fr> wrote: > Hello Christian ! > > Yes, a -c option for the basexhttp would help, as mentioned earlier, for example creating a shared mainmem collection. > > Best regards, > Fabrice
CG
Christian Grün21 Sep 2017 12:52 #
…done [1]. A new 8.7 snapshot is available [1].
[1] http://docs.basex.org/wiki/Command-Line_Options#HTTP_Server [2] http://files.basex.org/releases/latest/
Quoted text · Christian Grün · 16 lines
On Wed, Sep 20, 2017 at 7:37 PM, Christian Grün <christian.gruen at gmail.com> wrote: > Fabrice, Marco, thanks for your asseessment. After I have realized > that it’ll take some more time to make this happen, I have added yet > another issue for this feature request [1]. > > [1] https://github.com/BaseXdb/basex/issues/1499 > > On Mon, Sep 18, 2017 at 5:03 PM, Fabrice ETANCHAUD > <fetanchaud at pch.cerfrance.fr> wrote: >> Hello Christian ! >> >> Yes, a -c option for the basexhttp would help, as mentioned earlier, for example creating a shared mainmem collection. >> >> Best regards, >> Fabrice
ML
Marco Lettere21 Sep 2017 13:37 #
Great!
M.
Quoted text · Christian Grün · 23 lines
On 21/09/2017 12:52, Christian Grün wrote: > …done [1]. A new 8.7 snapshot is available [1]. > > [1] http://docs.basex.org/wiki/Command-Line_Options#HTTP_Server > [2] http://files.basex.org/releases/latest/ > > > On Wed, Sep 20, 2017 at 7:37 PM, Christian Grün > <christian.gruen at gmail.com> wrote: >> Fabrice, Marco, thanks for your asseessment. After I have realized >> that it’ll take some more time to make this happen, I have added yet >> another issue for this feature request [1]. >> >> [1] https://github.com/BaseXdb/basex/issues/1499 >> >> On Mon, Sep 18, 2017 at 5:03 PM, Fabrice ETANCHAUD >> <fetanchaud at pch.cerfrance.fr> wrote: >>> Hello Christian ! >>> >>> Yes, a -c option for the basexhttp would help, as mentioned earlier, for example creating a shared mainmem collection. >>> >>> Best regards, >>> Fabrice
Written for BaseX 8.6. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery