Full memory database
Thursday, 16 July 2015 · 5 messages
C
cmarchand16 Jul 2015 09:47 #
Hello,
is there a way to put all the databases in memory ? We have a 30Gb database, serving as repository for XInclude-like resolution, and I want to avoid disk access (I work with Marc Messeant...)
Using special JVM, I can allocate many memory, and so want to work only in memory. Is this possible ?
Best regards,
Christophe
CG
Christian Grün16 Jul 2015 10:06 #
Hi Christophe,
Quoted text · cmarchand · 6 lines
> is there a way to put all the databases in memory ? We have a 30Gb database, > serving as repository for XInclude-like resolution, and I want to avoid disk > access (I work with Marc Messeant...) > > Using special JVM, I can allocate many memory, and so want to work only in > memory. Is this possible ?
In the early days, BaseX has been a pure main-memory database. Today, some main-memory features are still available. For example, you can set
MAINMEM to true and create a new database completely in main memory.
You can't open existing databases in memory, though. We are currently looking for sponsors to realize this feature [1]. Feel free to give us a note if you are interested? And what are your requirements: do you also need updates, index structures, ...?
Best,
Christian
[1] https://github.com/BaseXdb/basex/issues/948C
cmarchand16 Jul 2015 10:27 #
I think we could use the in-memory database to store read-only documents (our repository), a use disk-persisted database for the resulting documents.
In term of requirements, I'd prefer Marc (added to dest list) to clearly explain what is expected, but I think attribute and node-name index should be enough.
As this, we could avoid the update functionnalty on in-memory database.
Best regards,
Christopeh
Quoted text · Christian Grün · 24 lines
Le 2015-07-16 10:06, Christian Grün a écrit : > Hi Christophe, > >> is there a way to put all the databases in memory ? We have a 30Gb database, >> serving as repository for XInclude-like resolution, and I want to avoid disk >> access (I work with Marc Messeant...) >> >> Using special JVM, I can allocate many memory, and so want to work only in >> memory. Is this possible ? > > In the early days, BaseX has been a pure main-memory database. Today, > some main-memory features are still available. For example, you can > set MAINMEM to true and create a new database completely in main > memory. > > You can't open existing databases in memory, though. We are currently > looking for sponsors to realize this feature [1 [1]]. Feel free to give us > a note if you are interested? And what are your requirements: do you > also need updates, index structures, ...? > > Best, > Christian > > [1] https://github.com/BaseXdb/basex/issues/948 [1]
Links:
------
[1] https://github.com/BaseXdb/basex/issues/948
CG
Hi Luiz,
Quoted text · Luiz Matos · 1 line
> In which config file can I to set MAINMEM value?
Please check out our documentation for more details (e.g. [1]).
Quoted text · Luiz Matos · 1 line
> Besides, I need to know if the system do the pagination automatically on the cases where main memory capacity is lower than the database size.
I am not sure what do you mean by "pagination", but probably the answer is "no": If main memory is not large enough, you will have to resort to disk-based processing. However, identical keys will only be referenced once in main memory, and the representation is much smaller than traditional DOM instances, so it's probably best if you simply give it a try.
Hope this helps,
Christian
[1] http://docs.basex.org/wiki/Databases
LM
Hi Christian, about this comment:
Quoted text · Christian Grün · 4 lines
> In the early days, BaseX has been a pure main-memory database. Today, > some main-memory features are still available. For example, you can > set MAINMEM to true and create a new database completely in main > memory.
In which config file can I to set
MAINMEM value? I searched in .basex and basex.bat files, but I did not see this parameter there. Besides, I need to know if the system do the pagination automatically on the cases where main memory capacity is lower than the database size.
Thank you in advance!--
Luiz Matos
PhD student at Federal Fluminense University
Written for BaseX 8.2. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery