basex, nfs, and local storage
Friday, 20 January 2017 · 8 messages · last reply 24 Jan 2017
BJ
Baysdon, Joseph20 Jan 2017 17:36 #
My baseX database is accessed via an NFS mount, and I’d like to improve its performance by moving its re-creatable files (e.g. indices) to a local disk. For the sake of NFS backup & restore, I’d like to keep the db files which cannot be re-created on the NFS servers.
Would someone please identify which db files I can move to a local disk, and which files cannot be re-created and therefore, must remain on NFS?
Thanks,
Joe
CG
Christian Grün20 Jan 2017 17:45 #
Hi Joe,
I am referring to the files that are listed in our Wiki [1]:
* inf and tbli will be recreated with each update
* tbl, txt and atv will be updated
* the remaining files will be recreated if the index is rebuilt
* when using
UPDINDEX, only atvl, atvr, txtl and txtr will be updated
If you call OPTIMIZE ALL, all files will be recreated.
Does this help?
Christian
[1] http://docs.basex.org/wiki/Storage_LayoutQuoted text · Joseph · 20 lines
On Fri, Jan 20, 2017 at 5:36 PM, Baysdon, Joseph <joseph.baysdon at sap.com> wrote: > My baseX database is accessed via an NFS mount, and I’d like to improve its > performance by moving its re-creatable files (e.g. indices) to a local disk. > For the sake of NFS backup & restore, I’d like to keep the db files which > cannot be re-created on the NFS servers. > > > > Would someone please identify which db files I can move to a local disk, and > which files cannot be re-created and therefore, must remain on NFS? > > > > > > Thanks, > > > > Joe
BJ
Baysdon, Joseph20 Jan 2017 19:24 #
Thanks, Christian. That helps, but I’d like a bit of clarification.
Do I understand correctly that the tbl, txt, and atv files contain the components of the XML data in the db? In other words, those three files are all that is needed to create the other files in the basex/data/fooDb directory; therefore, if I need to restore my db from an NFS backup, I should ensure that the tbl, txt, and atv files are located on an NFS server. (And I do understand that the “raw” directory and its subdirectories will also need to be on NFS.)
Joe
Quoted text · Christian Grün · 1 line
On 1/20/17, 11:45 AM, "Christian Grün" <christian.gruen at gmail.com> wrote:
Hi Joe,
I am referring to the files that are listed in our Wiki [1]:
* inf and tbli will be recreated with each update
* tbl, txt and atv will be updated
* the remaining files will be recreated if the index is rebuilt
* when using
UPDINDEX, only atvl, atvr, txtl and txtr will be updated
If you call OPTIMIZE ALL, all files will be recreated.
Does this help?
Christian
[1] http://docs.basex.org/wiki/Storage_LayoutQuoted text · Joseph · 20 lines
On Fri, Jan 20, 2017 at 5:36 PM, Baysdon, Joseph <joseph.baysdon at sap.com> wrote: > My baseX database is accessed via an NFS mount, and I’d like to improve its > performance by moving its re-creatable files (e.g. indices) to a local disk. > For the sake of NFS backup & restore, I’d like to keep the db files which > cannot be re-created on the NFS servers. > > > > Would someone please identify which db files I can move to a local disk, and > which files cannot be re-created and therefore, must remain on NFS? > > > > > > Thanks, > > > > Joe
CG
Christian Grün20 Jan 2017 19:31 #
Hi Joe,
I am not exactly sure what you mean. Did you have a look at the Wiki link as well? The BaseX database directory ("data" or "BaseXData") contains all databases, which can simply be zipped and unzipped. A single database is composed of .basex files, and the raw files are stored in a "raw" sub-directory.
Cheers,
Christian
Quoted text · Joseph · 49 lines
On Fri, Jan 20, 2017 at 7:24 PM, Baysdon, Joseph <joseph.baysdon at sap.com> wrote: > Thanks, Christian. That helps, but I’d like a bit of clarification. > > Do I understand correctly that the tbl, txt, and atv files contain the components of the XML data in the db? In other words, those three files are all that is needed to create the other files in the basex/data/fooDb directory; therefore, if I need to restore my db from an NFS backup, I should ensure that the tbl, txt, and atv files are located on an NFS server. (And I do understand that the “raw” directory and its subdirectories will also need to be on NFS.) > > Joe > > On 1/20/17, 11:45 AM, "Christian Grün" <christian.gruen at gmail.com> wrote: > > Hi Joe, > > I am referring to the files that are listed in our Wiki [1]: > > * inf and tbli will be recreated with each update > * tbl, txt and atv will be updated > * the remaining files will be recreated if the index is rebuilt > * when using UPDINDEX, only atvl, atvr, txtl and txtr will be updated > > If you call OPTIMIZE ALL, all files will be recreated. > > Does this help? > Christian > > [1] http://docs.basex.org/wiki/Storage_Layout > > > > On Fri, Jan 20, 2017 at 5:36 PM, Baysdon, Joseph <joseph.baysdon at sap.com> wrote: > > My baseX database is accessed via an NFS mount, and I’d like to improve its > > performance by moving its re-creatable files (e.g. indices) to a local disk. > > For the sake of NFS backup & restore, I’d like to keep the db files which > > cannot be re-created on the NFS servers. > > > > > > > > Would someone please identify which db files I can move to a local disk, and > > which files cannot be re-created and therefore, must remain on NFS? > > > > > > > > > > > > Thanks, > > > > > > > > Joe > >
BJ
Baysdon, Joseph20 Jan 2017 20:48 #
Let me rephrase. For a given database, fooDb, which contains a single XML resource at fooDb/a/b/c/bar.xml, which files are required to restore the fooDb from an NFS backup? And for the sake of completeness, which files are not necessary for that restoration?
Thanks,
Joe
Quoted text · Christian Grün · 1 line
On 1/20/17, 1:31 PM, "Christian Grün" <christian.gruen at gmail.com> wrote:
Hi Joe,
I am not exactly sure what you mean. Did you have a look at the Wiki link as well? The BaseX database directory ("data" or "BaseXData") contains all databases, which can simply be zipped and unzipped. A single database is composed of .basex files, and the raw files are stored in a "raw" sub-directory.
Cheers,
Christian
Quoted text · Joseph · 49 lines
On Fri, Jan 20, 2017 at 7:24 PM, Baysdon, Joseph <joseph.baysdon at sap.com> wrote: > Thanks, Christian. That helps, but I’d like a bit of clarification. > > Do I understand correctly that the tbl, txt, and atv files contain the components of the XML data in the db? In other words, those three files are all that is needed to create the other files in the basex/data/fooDb directory; therefore, if I need to restore my db from an NFS backup, I should ensure that the tbl, txt, and atv files are located on an NFS server. (And I do understand that the “raw” directory and its subdirectories will also need to be on NFS.) > > Joe > > On 1/20/17, 11:45 AM, "Christian Grün" <christian.gruen at gmail.com> wrote: > > Hi Joe, > > I am referring to the files that are listed in our Wiki [1]: > > * inf and tbli will be recreated with each update > * tbl, txt and atv will be updated > * the remaining files will be recreated if the index is rebuilt > * when using UPDINDEX, only atvl, atvr, txtl and txtr will be updated > > If you call OPTIMIZE ALL, all files will be recreated. > > Does this help? > Christian > > [1] http://docs.basex.org/wiki/Storage_Layout > > > > On Fri, Jan 20, 2017 at 5:36 PM, Baysdon, Joseph <joseph.baysdon at sap.com> wrote: > > My baseX database is accessed via an NFS mount, and I’d like to improve its > > performance by moving its re-creatable files (e.g. indices) to a local disk. > > For the sake of NFS backup & restore, I’d like to keep the db files which > > cannot be re-created on the NFS servers. > > > > > > > > Would someone please identify which db files I can move to a local disk, and > > which files cannot be re-created and therefore, must remain on NFS? > > > > > > > > > > > > Thanks, > > > > > > > > Joe > >
CG
Christian Grün20 Jan 2017 21:16 #
Quoted text · Baysdon, Joseph · 1 line
> Let me rephrase. For a given database, fooDb, which contains a single XML resource at fooDb/a/b/c/bar.xml, which files are required to restore the fooDb from an NFS backup?
All files in your data/fooDb directory.
Quoted text · Baysdon, Joseph · 1 line
> And for the sake of completeness, which files are not necessary for that restoration?
All other files on your machine ;)
BJ
Baysdon, Joseph24 Jan 2017 19:29 #
Chistian, I thought that *some* of the index files could be created/re-created from scratch (i.e. they don’t contain any data necessary for creating an xml instance for a resource in the db). If that’s not the case, ok. If it is the case, can you identify which index files can be created/re-created from scratch?
Quoted text · Christian Grün · 2 lines
On 1/20/17, 12:16 PM, "Christian Grün" <christian.gruen at gmail.com> wrote: > Let me rephrase. For a given database, fooDb, which contains a single XML resource at fooDb/a/b/c/bar.xml, which files are required to restore the fooDb from an NFS backup?
All files in your data/fooDb directory.
Quoted text · Christian Grün · 1 line
> And for the sake of completeness, which files are not necessary for that restoration?
All other files on your machine ;)
CG
Christian Grün24 Jan 2017 19:36 #
Quoted text · Baysdon, Joseph · 1 line
> Chistian, I thought that *some* of the index files could be created/re-created from scratch (i.e. they don’t contain any data necessary for creating an xml instance for a resource in the db).
That’s true, those are the files listed in our Wiki. However, if indexes are enabled, those files must exist in your database directory, otherwise, you cannot open it.
If you open your database and remove the attribute, text and fulltext index, you will be left with the minimum set of files.
Quoted text · Christian Grün · 11 lines
> On 1/20/17, 12:16 PM, "Christian Grün" <christian.gruen at gmail.com> wrote: > > > Let me rephrase. For a given database, fooDb, which contains a single XML resource at fooDb/a/b/c/bar.xml, which files are required to restore the fooDb from an NFS backup? > > All files in your data/fooDb directory. > > > And for the sake of completeness, which files are not necessary for that restoration? > > All other files on your machine ;) > >
Written for BaseX 8.5, 8.6. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery