Adding documents to database while space on disk drive runs out
Thursday, 22 November 2012 · 3 messages
TK
Thomas Kaltofen22 Nov 2012 19:06 #
Hi,
I just had the problem that the disk space of the drive where a BaseX database instance was stored ran out while adding a new document to the database via the server interface. The result was a half-finished document in the database resulting in ArrayIndexOutOfBounds-exceptions when querying the database and strange behavior of the GUI (views are not shown and even disabled). Would it be possible to check for sufficient disk space before writing to the database files? Moreover, it would be nice if the failed transaction of adding the document while the space ran out could be rolled back somehow to get the database back into a consistent state.
Thank you for your support!
Best regards,
Thomas
CG
Christian Grün22 Nov 2012 19:08 #
Dear Thomas,
I’m sorry that your data got corrupted. I agree that such a test would be a desirable addition, I’m afraid it’s often difficult to guess how much space will be occupied by a specific update operation. We could generally disable update operations when the remaining space falls below a given limit, but once again such a limit would be strongly dependent on the used system and use cases.
Rollbacks are on our long-term todo list (along with, or even without MVCC), similar to incremental backups, but if memory gets low, they may as well lead to unexpected behavior.
All the best,
Christian
Quoted text · Thomas Kaltofen · 10 lines
> I just had the problem that the disk space of the drive where a BaseX database instance was stored ran out while adding a new document to the database via the server interface. The result was a half-finished document in the database resulting in ArrayIndexOutOfBounds-exceptions when querying the database and strange behavior of the GUI (views are not shown and even disabled). Would it be possible to check for sufficient disk space before writing to the database files? Moreover, it would be nice if the failed transaction of adding the document while the space ran out could be rolled back somehow to get the database back into a consistent state. > > Thank you for your support! > > Best regards, > Thomas > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
TK
Thomas Kaltofen22 Nov 2012 19:22 #
Dear Christian,
thank you for your response. Actually, I could recover most of the data by exporting everything from the corrupted database and re-importing into a new database instance (the partial document was of course lost). I think the best solution would be to add a parameter to the global configuration which defines a memory limit that is checked for before performing any write operations. The default value of this parameter could be 0 (= no limit) and that way the current behavior is not changed, but the user could specify his/her own limit based on his/her system and use cases.
Best regards,
Thomas
Quoted text · Christian Grün · 43 lines
> -----Original Message----- > From: Christian Grün [mailto:christian.gruen at gmail.com] > Sent: Thursday, November 22, 2012 7:08 PM > To: Thomas Kaltofen; Thomas Kaltofen > Cc: basex-talk at mailman.uni-konstanz.de > Subject: Re: [basex-talk] Adding documents to database while space on disk > drive runs out > > Dear Thomas, > > I'm sorry that your data got corrupted. I agree that such a test would > be a desirable addition, I'm afraid it's often difficult to guess how > much space will be occupied by a specific update operation. We could > generally disable update operations when the remaining space falls > below a given limit, but once again such a limit would be strongly > dependent on the used system and use cases. > > Rollbacks are on our long-term todo list (along with, or even without > MVCC), similar to incremental backups, but if memory gets low, they > may as well lead to unexpected behavior. > > All the best, > Christian > _________________________________________ > > > I just had the problem that the disk space of the drive where a BaseX > database instance was stored ran out while adding a new document to the > database via the server interface. The result was a half-finished document in > the database resulting in ArrayIndexOutOfBounds-exceptions when > querying the database and strange behavior of the GUI (views are not shown > and even disabled). Would it be possible to check for sufficient disk space > before writing to the database files? Moreover, it would be nice if the failed > transaction of adding the document while the space ran out could be rolled > back somehow to get the database back into a consistent state. > > > > Thank you for your support! > > > > Best regards, > > Thomas > > _______________________________________________ > > BaseX-Talk mailing list > > BaseX-Talk at mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Written for BaseX 7.3. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery