Well done!

Wednesday, 3 February 2010 · 12 messages · last reply 18 Mar 2010
E
ep3 Feb 2010 08:34 #
Congrats for your efforts and well done product. I've worked with Berkeley dbxml, eXist, and Mark Logic. So I really appreciate the clean api, speed!, and compactness. Keep up the great work! Do you have roadmap somewhere? I'd like to get an idea where you are headed with development and if you could use any assistance. Best, Erik Peterson
CG
Christian Grün3 Feb 2010 22:40 #
Thanks Erik for appreciating our work, yes.. we do want to keep up our work for quite a while. Our current focus is enhanced web support for BaseX; we will offer a REST layer soon, and some language bindings for PHP, Python, etc. to simplify its application in web-based scenarios. – Currently, the project is funded by the University of Konstanz. Compared to other projects, BaseX is still young and fresh, and the steadily increasing number of users and the positive feedback indicates that we still need to get better known to attract more users in an even shorter time period. Another thank you for offering us your help. We are glad about everyone who spreads the word on our project, participates in the mailing list, gives feedback on bugs, proposes fixes, and extends the project with additional features that can be included in the main project. Please let us know if you have some more ideas, Christian ___________________________ Christian Gruen Universitaet Konstanz Department of Computer & Information Science D-78457 Konstanz, Germany Tel: +49 (0)7531/88-4449, Fax: +49 (0)7531/88-3577 http://www.inf.uni-konstanz.de/~gruen
Quoted text · ep · 4 lines
On Wed, Feb 3, 2010 at 8:34 AM, <ep at ardec.com> wrote: > Congrats for your efforts and well done product. I've worked with Berkeley dbxml, eXist, and Mark Logic. So I really appreciate the clean api, speed!, and compactness. Keep up the great work! Do you have roadmap somewhere? I'd like to get an idea where you are headed with development and if you could use any assistance. > Best, > Erik Peterson
CG
Christian Grün4 Feb 2010 17:46 #
Hi Erik, all of your suggestions are more than welcome – this is why I posted your answer back to the mailing list. I invite all of you to give personal feedback!
Quoted text · Christian Grün · 2 lines
> A few questions for you about basex and plans... > 1) Can a database store files other than xml? xqy/xql, images, js, css, etc?
Our storage is currently limited to well-formed XML documents.
Quoted text · Christian Grün · 1 line
> 2) Any plans to add collections or may basex does and I've overlooked it?
Yes, BaseX supports collections – as long as they are flat. Databases can store multiple single documents, which can be accessed e.g. by the XQuery collection() function. See also Section 4.4 in http://basex.org/faq.
Quoted text · Christian Grün · 7 lines
> 3) ReST interface: There are two approaches, both have their benefits and I would like to see both available: > a) The rest interface allows manipulating db resources: dbs, collections, files, users, permissions, etc. (The eXist db does this.) > -The main advantage with this is an API easy to integrate into many environments > b) The rest interface allows requests to be rewritten to a delegate xquery file to handle processing the http methods. (Some other db's provide this.) > -This allows building complete applications within a database. > -Would like to see this as the base approach with (a) as an internal implementation of (b) > What are your thoughts?
Interesting ideas.. Some of our current interface features are be inspired by eXist. While GET and POST can be used to send XQuery/Update requests, PUT and DELETE will allow to modify database instances. Here is a simple example how a GET request could look like: http://localhost:8080/rest/db/factbook?query=//country Additionally, we will allow to send BaseX database commands via GET and POST.. http://localhost:8080/rest/db/factbook?command=optimize This way, user management can be handled via REST as well. I'm not sure how users are modified in eXist via REST? Which other XML/REST implementations have you worked with?
Quoted text · Christian Grün · 3 lines
> 4) Any plans for webdav? This makes moving files in and out of the db easy for non-technical users. I've briefly looked at doing this with milton, apache jackrabbit, and from scratch. > 5) What do you think about using OSGI to enable plugging in db services? Bundles would be something like dbServer (the core service), tcpServer, restAdminServer, restServer, webdavServer, webAdmin, cliAdmin, monitorServer clusterServer, etc. This keeps the dbServer core code clean and small, yet allows other services to be developed and evolve independently. It also allows spinning up multiple instances of a server. For example, I could spin up multiple restServers or tcpServers on different ports to segment applications, or multiple webdav servers to partition content management per user/group rights. > 6) What do you think about moving to github? Looking for ways to grow basex community more quickly.
Good suggestions.. Let's see what everyone thinks about it!
Quoted text · Christian Grün · 1 line
> Hope you don't mind me jumping in. I see a great product whose continued and expanded development is important to this industry.
You are welcome, Christian ___________________________ Christian Gruen Universitaet Konstanz Department of Computer & Information Science D-78457 Konstanz, Germany Tel: +49 (0)7531/88-4449, Fax: +49 (0)7531/88-3577 http://www.inf.uni-konstanz.de/~gruen
C
coach3pete5 Feb 2010 07:16 #
See below... Erik
Quoted text · Christian Grün · 11 lines
On Feb 4, 2010, at 9:46 AM, Christian Grün wrote: > Hi Erik, > > all of your suggestions are more than welcome – this is why I posted > your answer back to the mailing list. I invite all of you to give > personal feedback! > >> A few questions for you about basex and plans... >> 1) Can a database store files other than xml? xqy/xql, images, js, css, etc? > > Our storage is currently limited to well-formed XML documents.
Ok, we'll need to think about how to handle packaging apps.
Quoted text · Christian Grün · 8 lines
> > >> 2) Any plans to add collections or may basex does and I've overlooked it? > > Yes, BaseX supports collections – as long as they are flat. Databases > can store multiple single documents, which can be accessed e.g. by the > XQuery collection() function. See also Section 4.4 in > http://basex.org/faq.
Flat meaning one collection of multiple docs per database right? Just thinking about how that impacts partitioning data...more databases...consistent naming scheme, join queries, etc. I'd like to hear how valuable other think collection hierarchies would be? What's the impact on db design?
Quoted text · Christian Grün · 24 lines
> > >> 3) ReST interface: There are two approaches, both have their benefits and I would like to see both available: >> a) The rest interface allows manipulating db resources: dbs, collections, files, users, permissions, etc. (The eXist db does this.) >> -The main advantage with this is an API easy to integrate into many environments >> b) The rest interface allows requests to be rewritten to a delegate xquery file to handle processing the http methods. (Some other db's provide this.) >> -This allows building complete applications within a database. >> -Would like to see this as the base approach with (a) as an internal implementation of (b) >> What are your thoughts? > > Interesting ideas.. Some of our current interface features are be > inspired by eXist. While GET and POST can be used to send > XQuery/Update requests, PUT and DELETE will allow to modify database > instances. Here is a simple example how a GET request could look like: > > http://localhost:8080/rest/db/factbook?query=//country > > Additionally, we will allow to send BaseX database commands via GET and POST.. > > http://localhost:8080/rest/db/factbook?command=optimize > > This way, user management can be handled via REST as well. I'm not > sure how users are modified in eXist via REST? Which other XML/REST > implementations have you worked with?
I've worked with eXist and Mark Logic. I recommend creating a side discussion with a document to collect feedback and review decisions. For example, how ReST purist should we be? GET methods don't modify resources for example. Do we enforce that in the BaseX rest API? How does an application respond with different content (xml, json, other)? The application could be expected to do that or we could provide transformation options in the ReST service. I really like keeping the db clean and simple. But some services pushed down "near" the database can really enable projects. I think this is where osgi could benefit. Providing services in the same jvm as the database, yet independently developed and deployed.
Quoted text · Christian Grün · 24 lines
> > >> 4) Any plans for webdav? This makes moving files in and out of the db easy for non-technical users. I've briefly looked at doing this with milton, apache jackrabbit, and from scratch. > >> 5) What do you think about using OSGI to enable plugging in db services? Bundles would be something like dbServer (the core service), tcpServer, restAdminServer, restServer, webdavServer, webAdmin, cliAdmin, monitorServer clusterServer, etc. This keeps the dbServer core code clean and small, yet allows other services to be developed and evolve independently. It also allows spinning up multiple instances of a server. For example, I could spin up multiple restServers or tcpServers on different ports to segment applications, or multiple webdav servers to partition content management per user/group rights. > >> 6) What do you think about moving to github? Looking for ways to grow basex community more quickly. > > Good suggestions.. Let's see what everyone thinks about it! > > >> Hope you don't mind me jumping in. I see a great product whose continued and expanded development is important to this industry. > > You are welcome, > Christian > > ___________________________ > > Christian Gruen > Universitaet Konstanz > Department of Computer & Information Science > D-78457 Konstanz, Germany > Tel: +49 (0)7531/88-4449, Fax: +49 (0)7531/88-3577 > http://www.inf.uni-konstanz.de/~gruen
EP
Erik Petersonin reply to Christian Grün, 3 Feb 2010, 22:409 Feb 2010 08:35 #
Is xinclude supported? Erik
MS
Michael Seiferle9 Feb 2010 14:57 #
Hi Erik, IIRC correctly XInclude is not supported at the moment. While this is not be exactly what you are looking for, it might still be worth a try: The DeepFS importer is already capable of importing XML and various other content formats in your database. Via: DeepFS > Create DeepFS Mapping > Metadata (Include metadata, include textual contents.) Regards Michael
Quoted text · Erik Peterson · 7 lines
Am 09.02.2010 um 07:36 schrieb Erik Peterson: > Is xinclude supported? > Erik > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
CG
Christian Grünin reply to coach3pete, 5 Feb 2010, 07:1610 Feb 2010 00:53 #
Erik, sorry for a late feedback..
Quoted text · coach3pete · 3 lines
>>> 2) Any plans to add collections or may basex does and I've overlooked it? >> [...] > Flat meaning one collection of multiple docs per database right?
Exactly.. Traditionally, we put our major focus on single, large documents. That's why we decided to keep collections as simple as possible (at least for now). Hierarchical collections could probably implemented rather straight-forward in the backend; conceptual issues would take some more time, however.
Quoted text · coach3pete · 2 lines
>>> 3) ReST interface: There are two approaches, both have their benefits and I would like to see both available: >> [...]
The first beta version of our REST implementation will be released around the end of this month! This might be a good base to discuss further enhancements and changes. We're currently experimenting with two versions, both using Jetty and one of them using JAX-RS.
Quoted text · coach3pete · 1 line
> For example, how ReST purist should we be? GET methods don't modify resources for example. Do we enforce that in the BaseX rest API? How does an application respond with different content (xml, json, other)?
As, in future, XQuery Update might be merged with main XQuery specification, we recently decided to allow XQuery Update via GET and POST. DELETE and PUT will be used to delete/insert collections or documents within collections. All the best, Christian
DG
Dave Glick10 Feb 2010 02:14 #
Hi Christian, First off, thanks for building and continuing to maintain what I think is one of the most underrated/undiscovered pieces of open source code out there. Now to my question... We've been using BaseX 5.7 with very heavy use of the backend for a while now (though we've tried hard not to actually change the BaseX code in our use). On 5.7 we use the "insert" methods of the org.basex.data.Data class to enable mutability (I.e., to add new attributes or elements at specific locations in the document). We prototyped our code from the "insert" proc file. I've noticed that both the insert methods we had been using and the whole "insert" proc command are gone in 6.0. Does 6.0 no longer support mutability? I do notice there are some other insert/update methods instead, but no good calls to them to try and understand how they work. What would be a good procedure for inserting new data given that we have both a Nod and Data reference? Thanks, Dave
CG
Christian Grün10 Feb 2010 11:36 #
Dear Dave,
Quoted text · Dave Glick · 1 line
> First off, thanks for building and continuing to maintain what I think is one of the most underrated/undiscovered pieces of open source code out there. Now to my question...
Thanks for the feedback; don't hesitate to pass this on to everyone…
Quoted text · Dave Glick · 1 line
> We've been using BaseX 5.7 with very heavy use of the backend for a while now (though we've tried hard not to actually change the BaseX code in our use). On 5.7 we use the "insert" methods of the org.basex.data.Data class to enable mutability (I.e., to add new attributes or elements at specific locations in the document). We prototyped our code from the "insert" proc file. I've noticed that both the insert methods we had been using and the whole "insert" proc command are gone in 6.0. Does 6.0 no longer support mutability? I do notice there are some other insert/update methods instead, but no good calls to them to try and understand how they work. What would be a good procedure for inserting new data given that we have both a Nod and Data reference?
As XQuery Update is supported since BaseX 6.0, all update methods in the Data class have been completely revamped and aligned with the logics of the official language specification. If you want to have more insight on how single update operations are working, I recommend you to check out the org.basex.query.up.primitives package. For example, the method InsertAttribute will show you how to add new attributes into a DBNode instance. Hope this helps, Christian ___________________________ Christian Gruen Universitaet Konstanz Department of Computer & Information Science D-78457 Konstanz, Germany Tel: +49 (0)7531/88-4449, Fax: +49 (0)7531/88-3577 http://www.inf.uni-konstanz.de/~gruen
C
coach3petein reply to Michael Seiferle, 9 Feb 2010, 14:5711 Feb 2010 07:12 #
Hi Michael, XInclude is used to spit xml content into reusable fragments that can be referenced by many master documents. This saves having to duplicate content in many places. Would like to see this a high priority for BaseX to implement. I have tried the DeepFS and find it interesting but not useful for our purpose at the moment. Thanks, Erik
Quoted text · Michael Seiferle · 23 lines
On Feb 9, 2010, at 6:57 AM, Michael Seiferle wrote: > Hi Erik, > > IIRC correctly XInclude is not supported at the moment. > > While this is not be exactly what you are looking for, it might still be worth a try: > The DeepFS importer is already capable of importing XML and various other content formats in your database. > > Via: DeepFS > Create DeepFS Mapping > Metadata (Include metadata, include textual contents.) > > Regards > Michael > > > Am 09.02.2010 um 07:36 schrieb Erik Peterson: > >> Is xinclude supported? >> Erik >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >
DG
Dave Glickin reply to Christian Grün, 10 Feb 2010, 11:3618 Mar 2010 01:02 #
I have two (hopefully) quick questions about when indexes get rebuilt and when they need to be manually rebuilt. I've noticed that in the ACreate and CreateDB classes, indexes like the full text index are explicitly rebuilt when a new Data instance is created. If one were to directly create a new Data instance using a Builder, would those indexes need to be built after creation and parsing the way they are in the two mentioned classes? Secondly, I can't find code that updates the indexes on changes to the database. I would assume that performing updates (either through XQuery Update or directly) such as adding a new element should update the corresponding indexes. Where does this happen? Is it incremental (that is, only those parts of the index that need to be changed are modified) or full (all the indexes are totally rebuilt on every change)? Thanks, Dave
CG
Christian Grün18 Mar 2010 16:39 #
Hi Dave,
Quoted text · Dave Glick · 1 line
> I've noticed that in the ACreate and CreateDB classes, indexes like the full text index are explicitly rebuilt when a new Data instance is created. If one were to directly create a new Data instance using a Builder, would those indexes need to be built after creation and parsing the way they are in the two mentioned classes?
All indexes are optional in BaseX, and mainly applied for speeding up query execution – so you shouldn't encounter any troubles if they are missing. By default, the path summary, text and attribute indexes are created (as Prop.TEXTINDEX, ATTRINDEX and PATHINDEX are set to true). You can create all of them in a second step (e.g. by running an instance of the CreateIndex class).
Quoted text · Dave Glick · 1 line
> Secondly, I can't find code that updates the indexes on changes to the database. I would assume that performing updates (either through XQuery Update or directly) such as adding a new element should update the corresponding indexes. Where does this happen? Is it incremental (that is, only those parts of the index that need to be changed are modified) or full (all the indexes are totally rebuilt on every change)?
Your observation is correct – there are no methods for incremental index updates in BaseX. The main reason for dismissing the index structures after updates is that we wanted to keep index structures as small and fast as possible. This means that indexes have to be rebuilt after updates by e.g. triggering the OPTIMIZE command (see also http://basex.org/xquery). Hope this helps - more questions are welcome, Christian ___________________________ Christian Gruen Universitaet Konstanz Department of Computer & Information Science D-78457 Konstanz, Germany Tel: +49 (0)7531/88-4449, Fax: +49 (0)7531/88-3577 http://www.inf.uni-konstanz.de/~gruen

⚡Generated with XQuery