Main Page » Mailing List » October 2012

Create DB from query

Thursday, 25 October 2012 · 9 messages · last reply 26 Oct 2012
BR
bryan rasmussen25 Oct 2012 14:19 #
Hi, I have a db that has all of my xml documents, I would like to make a bunch of new dbs based on the document types which map to the document element name so for example if I do the query /LOV I get back 115491 Items, I would then like to put that into a db named LOV. How do I put the outputs of query on one db into another db, and does anyone have some code that would go through every /* and create a db by the name of the document element if that db does not exist and then place the document into that DB - or would that be insanely process intensive (there are 290422 items, 13 document types)? Thanks, Bryan Rasmussen
CG
Christian Grün25 Oct 2012 14:46 #
Dear Bryan, we are currently including new db:create() and db:drop() functions to BaseX. Until then, there are other options to do what you want. One example is creating a command script from XQuery [1] and run it afterwards, e.g. as follows:
file:write('script.bxs',
  <commands>{
    for $c in distinct-values(db:open('your-db')/*/name())
    return <create-db name="{ $c }"/>
  }</commands>)
In the next step, you could add the documents to the newly created databases:
  for $doc in db:open('Analysedocumentaire')
  let $name := $doc/*/name()
  return db:add($name, $doc)
Hope this helps, Christian [1] http://docs.basex.org/wiki/Commands#Basics
Quoted text · bryan rasmussen · 21 lines
On Thu, Oct 25, 2012 at 2:19 PM, bryan rasmussen <rasmussen.bryan at gmail.com> wrote: > Hi, > > I have a db that has all of my xml documents, I would like to make a > bunch of new dbs based on the document types which map to the document > element name > so for example if I do the query /LOV I get back 115491 Items, I would > then like to put that into a db named LOV. How do I put the outputs of > query on one db into another db, and does anyone have some code that > would go through every /* and create a db by the name of the document > element if that db does not exist and then place the document into > that DB - or would that be insanely process intensive (there are > 290422 items, 13 document types)? > > Thanks, > Bryan Rasmussen > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
BR
bryan rasmussen25 Oct 2012 21:32 #
Quoted text · bryan rasmussen · 17 lines
On Thu, Oct 25, 2012 at 4:32 PM, bryan rasmussen <rasmussen.bryan at gmail.com> wrote: > thanks that was helpful, but the following > >> for $doc in db:open('DK') >> let $name := $doc/*/name() >> return db:add($name, $doc) > where DK is the name of my db. > > returns > Resource path '' is invalid. > > > I tried for $doc in db:open('DK') let $name := > local-name(doc($doc)) return db:add($name, $doc) > > but that gives a FODC0005 error.
Ok I figured out the FODC0005 error, should have realized it, but the other one has me stumped especially considering that for $doc in db:open('DK') return $doc/*/name() outputs all the names of the document nodes no problem. Is there some sort of type conversion that needs to be done? Thanks, Bryan Rasmussen
CG
Christian Grün25 Oct 2012 22:39 #
Quoted text · bryan rasmussen · 4 lines
>>> for $doc in db:open('DK') >>> let $name := $doc/*/name() >>> return db:add($name, $doc) >> where DK is the name of my db.
Maybe one of the returned name strings is empty. Could you send me the result of the following query?
  for $name in db:open('DK')/*/name()
  return element name { $name }
BR
bryan rasmussen25 Oct 2012 23:25 #
Ok, its attached, can't see any issue. Hope you don't mind that I gzipped but there's nearly 300 thousand documents. anyway doesn't seem like that would be possible? I didn't tell basex to ignore errors when importing documents, surely a document where name() on document node returned an empty string would be malformed. Thanks, Bryan Rasmussen
Quoted text · Christian Grün · 12 lines
On Thu, Oct 25, 2012 at 10:39 PM, Christian Grün <christian.gruen at gmail.com> wrote: >>>> for $doc in db:open('DK') >>>> let $name := $doc/*/name() >>>> return db:add($name, $doc) >>> where DK is the name of my db. > > Maybe one of the returned name strings is empty. Could you send me the > result of the following query? > > for $name in db:open('DK')/*/name() > return element name { $name }
CG
Christian Grün25 Oct 2012 23:29 #
Quoted text · bryan rasmussen · 2 lines
> Ok, its attached, can't see any issue. Hope you don't mind that I > gzipped but there's nearly 300 thousand documents.
Yes, the data looks fine. As you already indicated, only invalid data would cause such errors, so I'm still not sure what causes the problem in your data, sorry.. Maybe someone else has an idea, or maybe you can do some more debugging? Christian
BR
bryan rasmussenin reply to Christian Grün, 25 Oct 2012, 14:4626 Oct 2012 11:59 #
Quoted text · Christian Grün · 5 lines
> In the next step, you could add the documents to the newly created databases: > > for $doc in db:open('Analysedocumentaire') > let $name := $doc/*/name() > return db:add($name, $doc)
Since there doesn't seem to be any way around this, is there a way to find more info on the error: Stopped at line 1, column 65: [FODC0007] Resource path '' is invalid. Other possibilities I've tried: Putting db:add in try-catch, doing a counter and an if with db:add inside of the if so that I could see if I could make it work for some files, unfortunately I guess neither of those scenarios is allowed - anyway to do something like that? Is there a way to do a query and export results to a filepath, and then import that filepath into my specific db. Also when I look at the dbs I created they look like the following: Database Properties Name: AFGDOK Size: 4540 Bytes Nodes: 1 Documents: 1 Binaries: 0 Timestamp: 25.10.2012 15:40:19 Resource Properties Timestamp: 25.10.2012 15:40:19 Encoding: UTF-8 Whitespace Chopping: ON Indexes Up-to-date: true Text Index: ON Attribute Index: ON Full-Text Index: OFF I never actually put a document in there, but it says it has a document, is that normal? I figured probably like there is some sort of hidden metadata document in every db? Thanks, Bryan Rasmussen
CG
Christian Grün26 Oct 2012 13:22 #
Hi Bryan, I noticed I overlooked an important detail in your query: there was no indication of how the target document should be called. The following query should work:
  for $doc in db:open('AFGDOK')
  let $name := $doc/*/name()
  let $target := replace(base-uri($doc), '.*?/', '')
  return db:add($name, $doc, $target)
In the existing example, we have a document name, which is encoded in the base uri; but there are many examples in which no document name exists; e.g.:
  db:add('db', document { <a/> }, 'name.xml")
However, I'll think about implicitly assigning the original document name if it exists. If this works out, the former query will work in future, too.
Quoted text · bryan rasmussen · 3 lines
> I never actually put a document in there, but it says it has a > document, is that normal? I figured probably like there is some sort > of hidden metadata document in every db?
Correct! This hidden document has been removed in the latest 7.5 snapshots [1]. Hope this helps, Christian [1] http://files.basex.org/releases/latest/
CG
Christian Grün26 Oct 2012 14:03 #
Hi Bryan, I'm glad to tell I've updated the db:add() function; it now automatically detects and adopts original database document paths. Feel free to check out the latest stable snapshot [1]. Christian [1] http://files.basex.org/releases/latest/
Quoted text · Christian Grün · 34 lines
On Fri, Oct 26, 2012 at 1:22 PM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Bryan, > > I noticed I overlooked an important detail in your query: there was no > indication of how the target document should be called. The following > query should work: > > for $doc in db:open('AFGDOK') > let $name := $doc/*/name() > let $target := replace(base-uri($doc), '.*?/', '') > return db:add($name, $doc, $target) > > In the existing example, we have a document name, which is encoded in > the base uri; but there are many examples in which no document name > exists; e.g.: > > db:add('db', document { <a/> }, 'name.xml") > > However, I'll think about implicitly assigning the original document > name if it exists. If this works out, the former query will work in > future, too. > > >> I never actually put a document in there, but it says it has a >> document, is that normal? I figured probably like there is some sort >> of hidden metadata document in every db? > > Correct! This hidden document has been removed in the latest 7.5 snapshots [1]. > > Hope this helps, > Christian > > [1] http://files.basex.org/releases/latest/

Written for BaseX 7.3. BaseX may have changed since: please check the current documentation.

« BaseX as Windows Servicesbasex on redhat »
⚡Generated with XQuery