Main Page » Mailing List » February 2011

Force default file type to XML during "add to" command?

Tuesday, 1 February 2011 · 6 messages · last reply 2 Feb 2011
CG
Christian Grünin reply to Chad Lancour, 1 Feb 2011, 22:291 Feb 2011 21:39 #
Hi Chad,
Quoted text · Chad Lancour · 2 lines
> Just want to start out with KUDOS on the project, contributors, and > technology. Amazing.
thanks for the kudos, always welcome!
Quoted text · Chad Lancour · 4 lines
> Question: All of the 300k+ xml files do NOT have a file extension. Is there > a way to have BaseX default to xml if there is no file type? Is there > perhaps some other way to handle this other than recursively renaming all > the files?
You're probably looking for the "createfilter" option, which allows you to define a glob syntax for fitering the input files; please try this:
  basex -Vc "create db metadata; alter user admin xxxxxxx"
  basex -Vc "open metadata; set createfilter *; add to \set1 ...
Some more information may be found here: http://docs.basex.org/wiki/Options http://docs.basex.org/wiki/CreateCollection Hope this helps, Christian ___________________________ Christian Grün Uni KN, Box 188 78457 Konstanz, Germany http://www.inf.uni-konstanz.de/~gruen
CL
Chad Lancour1 Feb 2011 22:29 #
Hello- Just want to start out with KUDOS on the project, contributors, and technology. Amazing. We are importing 300K+ xml & binary files from a file system with 30k in nested hierarchical folder structures into basex. The auto-magical capabilities of basex to suck in this data (and the speed at which is does it) is kick-a%$. Question: All of the 300k+ xml files do NOT have a file extension. Is there a way to have BaseX default to xml if there is no file type? Is there perhaps some other way to handle this other than recursively renaming all the files? Fyi -Here is the simple batch file I am using to import the data: java -Xmx1024m -cp %devbase%\basex65\BaseX.jar org.basex.BaseX -Vc "drop db metadata" java -Xmx1024m -cp %devbase%\basex65\BaseX.jar org.basex.BaseX -Vc "create db metadata; alter user admin xxxxxxx" java -Xmx1024m -cp %devbase%\basex65\BaseX.jar org.basex.BaseX -Vc "open metadata; SET INTPARSE ON; add to \set1 %devbase%\code\basex\import\webcat" java -Xmx1024m -cp %devbase%\basex65\BaseX.jar org.basex.BaseX -Vc "open metadata; SET INTPARSE ON; add to \set2 %devbase%\code\basex\import\rpd"
java -Xmx1024m -cp %devbase%\basex65\BaseX.jar org.basex.BaseX -Vc "open
metadata; create index text; create index attribute; create index fulltext;
create index path"
Regards, Chad chad at helmetcameracentral.com
CL
Chad Lancourin reply to Christian Grün, 1 Feb 2011, 21:392 Feb 2011 03:38 #
Thanks Christian. Just tried that. The xml files do not have a file extension such as "file1" "file2", so * would work, but there are also *.ATR files as well - which are binary format. When using *, BaseX chokes on the binary (Error: "root.atr" (Line 1): No text allowed before root tag.). Also, tried variations of FORCECREATE, PARSER. So, tried using a standard wildcard like "*." which returns all files without an extension. This resulted in BaseX not loading any documents. I do have some work-arounds for now... Thx/chad
Quoted text · Christian Grün · 30 lines
-----Original Message----- From: Christian Grün [mailto:christian.gruen at gmail.com] Sent: Tuesday, February 01, 2011 12:40 PM To: Chad Lancour Cc: basex-talk at mailman.uni-konstanz.de Subject: Re: [basex-talk] Force default file type to XML during "add to" command? Hi Chad, > Just want to start out with KUDOS on the project, contributors, and > technology. Amazing. thanks for the kudos, always welcome! > Question: All of the 300k+ xml files do NOT have a file extension. Is there > a way to have BaseX default to xml if there is no file type? Is there > perhaps some other way to handle this other than recursively renaming all > the files? You're probably looking for the "createfilter" option, which allows you to define a glob syntax for fitering the input files; please try this: basex -Vc "create db metadata; alter user admin xxxxxxx" basex -Vc "open metadata; set createfilter *; add to \set1 ... Some more information may be found here: http://docs.basex.org/wiki/Options http://docs.basex.org/wiki/CreateCollection Hope this helps, Christian Christian Grün Uni KN, Box 188 78457 Konstanz, Germany http://www.inf.uni-konstanz.de/~gruen
CG
Christian Grün2 Feb 2011 10:57 #
Thanks Chad, I wasn't aware of the "*." syntax. Unfortunately, Java (version <= 6) has no class for parsing the glob syntax, but I've now extended our parser syntax to also support patterns with trailing dots. Feel free to check out the sources, or give me a message to send you a jar build. Best, Christian
Quoted text · Chad Lancour · 58 lines
On Wed, Feb 2, 2011 at 2:38 AM, Chad Lancour <chad at helmetcameracentral.com> wrote: > Thanks Christian. Just tried that. The xml files do not have a file > extension such as "file1" "file2", so * would work, but there are also *.ATR > files as well - which are binary format. When using *, BaseX chokes on the > binary (Error: "root.atr" (Line 1): No text allowed before root tag.). Also, > tried variations of FORCECREATE, PARSER. > > So, tried using a standard wildcard like "*." which returns all files > without an extension. This resulted in BaseX not loading any documents. > > I do have some work-arounds for now... > > Thx/chad > > -----Original Message----- > From: Christian Grün [mailto:christian.gruen at gmail.com] > Sent: Tuesday, February 01, 2011 12:40 PM > To: Chad Lancour > Cc: basex-talk at mailman.uni-konstanz.de > Subject: Re: [basex-talk] Force default file type to XML during "add to" > command? > > Hi Chad, > >> Just want to start out with KUDOS on the project, contributors, and >> technology. Amazing. > > thanks for the kudos, always welcome! > >> Question: All of the 300k+ xml files do NOT have a file extension. Is > there >> a way to have BaseX default to xml if there is no file type? Is there >> perhaps some other way to handle this other than recursively renaming all >> the files? > > You're probably looking for the "createfilter" option, which allows > you to define a glob syntax for fitering the input files; please try > this: > > basex -Vc "create db metadata; alter user admin xxxxxxx" > basex -Vc "open metadata; set createfilter *; add to \set1 ... > > Some more information may be found here: > > http://docs.basex.org/wiki/Options > http://docs.basex.org/wiki/CreateCollection > > Hope this helps, > Christian > ___________________________ > > Christian Grün > Uni KN, Box 188 > 78457 Konstanz, Germany > http://www.inf.uni-konstanz.de/~gruen > >
CG
Christian Grün2 Feb 2011 11:13 #
…btw, we have just extended our download page with the option to download a latest build; see: http://basex.org/products/download/all-downloads/ Christian
Quoted text · Christian Grün · 71 lines
On Wed, Feb 2, 2011 at 10:57 AM, Christian Grün <christian.gruen at gmail.com> wrote: > Thanks Chad, I wasn't aware of the "*." syntax. Unfortunately, Java > (version <= 6) has no class for parsing the glob syntax, but I've now > extended our parser > syntax to also support patterns with trailing dots. Feel free to check > out the sources, or give me a message to send you a jar build. > > Best, > Christian > > > On Wed, Feb 2, 2011 at 2:38 AM, Chad Lancour > <chad at helmetcameracentral.com> wrote: >> Thanks Christian. Just tried that. The xml files do not have a file >> extension such as "file1" "file2", so * would work, but there are also *.ATR >> files as well - which are binary format. When using *, BaseX chokes on the >> binary (Error: "root.atr" (Line 1): No text allowed before root tag.). Also, >> tried variations of FORCECREATE, PARSER. >> >> So, tried using a standard wildcard like "*." which returns all files >> without an extension. This resulted in BaseX not loading any documents. >> >> I do have some work-arounds for now... >> >> Thx/chad >> >> -----Original Message----- >> From: Christian Grün [mailto:christian.gruen at gmail.com] >> Sent: Tuesday, February 01, 2011 12:40 PM >> To: Chad Lancour >> Cc: basex-talk at mailman.uni-konstanz.de >> Subject: Re: [basex-talk] Force default file type to XML during "add to" >> command? >> >> Hi Chad, >> >>> Just want to start out with KUDOS on the project, contributors, and >>> technology. Amazing. >> >> thanks for the kudos, always welcome! >> >>> Question: All of the 300k+ xml files do NOT have a file extension. Is >> there >>> a way to have BaseX default to xml if there is no file type? Is there >>> perhaps some other way to handle this other than recursively renaming all >>> the files? >> >> You're probably looking for the "createfilter" option, which allows >> you to define a glob syntax for fitering the input files; please try >> this: >> >> basex -Vc "create db metadata; alter user admin xxxxxxx" >> basex -Vc "open metadata; set createfilter *; add to \set1 ... >> >> Some more information may be found here: >> >> http://docs.basex.org/wiki/Options >> http://docs.basex.org/wiki/CreateCollection >> >> Hope this helps, >> Christian >> ___________________________ >> >> Christian Grün >> Uni KN, Box 188 >> 78457 Konstanz, Germany >> http://www.inf.uni-konstanz.de/~gruen >> >> >
JC
Jan Vlčinský (CAD)2 Feb 2011 16:12 #
Christian What filter/parser shall I use to import other people minds? Yesterday I was about to propose something like that (download page with latest builds) and today you deliver it. Awesome. Jan 2011/2/2 Christian Grün <christian.gruen at gmail.com>
Quoted text · Christian Grün · 88 lines
> …btw, we have just extended our download page with the option to > download a latest build; see: > > http://basex.org/products/download/all-downloads/ > > Christian > ___________________________ > > On Wed, Feb 2, 2011 at 10:57 AM, Christian Grün > <christian.gruen at gmail.com> wrote: > > Thanks Chad, I wasn't aware of the "*." syntax. Unfortunately, Java > > (version <= 6) has no class for parsing the glob syntax, but I've now > > extended our parser > > syntax to also support patterns with trailing dots. Feel free to check > > out the sources, or give me a message to send you a jar build. > > > > Best, > > Christian > > > > > > On Wed, Feb 2, 2011 at 2:38 AM, Chad Lancour > > <chad at helmetcameracentral.com> wrote: > >> Thanks Christian. Just tried that. The xml files do not have a file > >> extension such as "file1" "file2", so * would work, but there are also > *.ATR > >> files as well - which are binary format. When using *, BaseX chokes on > the > >> binary (Error: "root.atr" (Line 1): No text allowed before root tag.). > Also, > >> tried variations of FORCECREATE, PARSER. > >> > >> So, tried using a standard wildcard like "*." which returns all files > >> without an extension. This resulted in BaseX not loading any documents. > >> > >> I do have some work-arounds for now... > >> > >> Thx/chad > >> > >> -----Original Message----- > >> From: Christian Grün [mailto:christian.gruen at gmail.com] > >> Sent: Tuesday, February 01, 2011 12:40 PM > >> To: Chad Lancour > >> Cc: basex-talk at mailman.uni-konstanz.de > >> Subject: Re: [basex-talk] Force default file type to XML during "add to" > >> command? > >> > >> Hi Chad, > >> > >>> Just want to start out with KUDOS on the project, contributors, and > >>> technology. Amazing. > >> > >> thanks for the kudos, always welcome! > >> > >>> Question: All of the 300k+ xml files do NOT have a file extension. Is > >> there > >>> a way to have BaseX default to xml if there is no file type? Is there > >>> perhaps some other way to handle this other than recursively renaming > all > >>> the files? > >> > >> You're probably looking for the "createfilter" option, which allows > >> you to define a glob syntax for fitering the input files; please try > >> this: > >> > >> basex -Vc "create db metadata; alter user admin xxxxxxx" > >> basex -Vc "open metadata; set createfilter *; add to \set1 ... > >> > >> Some more information may be found here: > >> > >> http://docs.basex.org/wiki/Options > >> http://docs.basex.org/wiki/CreateCollection > >> > >> Hope this helps, > >> Christian > >> ___________________________ > >> > >> Christian Grün > >> Uni KN, Box 188 > >> 78457 Konstanz, Germany > >> http://www.inf.uni-konstanz.de/~gruen > >> > >> > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >
-- *Ing. Jan Vlčinský* CAD programy Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic tel: +420-597 602 024; mob: +420-608 979 040 skype: janvlcinsky; GoogleTalk: jan.vlcinsky at gmail.com http://cz.linkedin.com/in/vlcinsky

Written for BaseX 6. BaseX may have changed since: please check the current documentation.

⚡Generated with XQuery