Database creation from a directory tree

> Additional question: That was in the GUI. I'd like to do the same > thing using the command line with the .JAR file, but somewhere my > syntax is not right. Additionally, all the XML files have NO > EXTENSION so I don't know how to specify that either in the command > line. So, any help in that area would also be appreciated.

> In the GUI in 6.1, I can create a database from a whole directory > hierarchy of 5,681 individual XML files. When I point to the same > source file directory and using the 6.3.4 GUI (which I installed with > a pre-release version of the installer), it only processes 14 files. > No error message (that I saw) was produced. The behavior that I'm > wanting is what 6.1 does. Glad to help in solving the problem.

Am 28.12.10 12:55, schrieb David leigh: > > I've just installed 6.3.4 and I've noticed a difference from version 6.1. > > In the GUI in 6.1, I can create a database from a whole directory > hierarchy of 5,681 individual XML files. When I point to the same > source file directory and using the 6.3.4 GUI (which I installed with > a pre-release version of the installer), it only processes 14 files. > No error message (that I saw) was produced. The behavior that I'm > wanting is what 6.1 does. Glad to help in solving the problem. > > Additional question: That was in the GUI. I'd like to do the same > thing using the command line with the .JAR file, but somewhere my > syntax is not right. Additionally, all the XML files have NO > EXTENSION so I don't know how to specify that either in the command > line. So, any help in that area would also be appreciated. > > Thank you, > > David > > DavidEmailSig <mailto:david at leighweb.com> > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Am 28.12.10 18:00, schrieb David leigh: > > Here are my command line results from 6.3.4: > > C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX>java -cp > BaseX.jar org.basex.BaseXClient -Uadmin -Padmin > > BaseX 6.3.5 [Client] > > Try "help" to get more information. > > > drop db songs > > Database 'songs' was dropped. > > > create db song > > Database 'song' created in 1683.61 ms. > > > info index > > Tags > > - Structure: Hash > > - Entries: 0 > > Attributes > > - Structure: Hash > > - Entries: 0 > > Text Index > > - Structure: Binary tree > > - Size: 4 Bytes > > - Entries: 0 > > Attribute Index > > - Structure: Binary tree > > - Size: 4 Bytes > > - Entries: 0 > > Full-Text Index > > - Not available > > Path Summary > > doc() 1x > > > info table > > Stopped at line 1, column 10: > > Syntax: INFO ([DATABASE|INDEX|STORAGE]) > > Show information on current database. > > Shows information on the currently opened database: > > - no argument: show global information > > - DATABASE: shows database information > > - INDEX: shows index information > > - STORAGE [start end] | [query]: show internal database table > > > info storage > > PRE DIS SIZ ATS NS KIND CONTENT > > ------------------------------------- > > 0 1 1 1 0 DOC song > > > info > > General Information > > Database Path: C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX\data > > Used Main Memory: 895 KB > > Database Creation > > Whitespace Chopping: ON > > Entity Parsing: OFF > > Indexes > > Path Summary: ON > > Text Index: ON > > Attribute Index: ON > > Full-Text Index: OFF > > > > > So, for 6.3.4, it creates from the command line, but it's empty. The > root of that path has NO files in it. The 5861 files are in all the > subdirectories. > > Then, when I try the same syntax with 6.1, I get the following results: > > > create db songs C:\Users\David\Documents\OpenSong\songpacks\Songs > > Stopped at line 1, column 17: > > Syntax: CREATE [DB|COLL|FS|INDEX|USER] [...] > > Create database, index or user. > > Creates a new database, index or user: > > - DB [path] [name?]: > > creates the database [name] for the file or directory [path] > > - COLL [name]: > > creates an empty database [name] > > - INDEX [TEXT|ATTRIBUTE|FULLTEXT|PATH]: > > creates the specified index > > - FS [path] [name] ([mountpoint] [backingstore]): > > creates filesystem database [name] for [path] > > (using [mountpoint] and [backingstore] if FUSE is activated) > > - USER [name] [password?]: > > creates the specified user > > Summary: > > 6.1 GUI -- works as I want it > > 6.1 command line -- can't get it to create > > 6.3.4 GUI -- creates but with only 14 entries > > 6.3.4 command line -- creates, but it's empty > > I don't mind staying with 6.1, but I DO want to do everything with the > command line. > > Is the ADD syntax the same, in that I can simply specify the path and > it will recurse the whole directory tree? > > Thanks!!! > > David > > DavidEmailSig <mailto:david at leighweb.com> > > *From:*Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] > *Sent:* Tuesday, December 28, 2010 01:29 PM > *To:* david at leighweb.com > *Cc:* basex-talk at mailman.uni-konstanz.de > *Subject:* Re: [basex-talk] Database creation from a directory tree > > Additional question: That was in the GUI. I'd like to do the same > thing using the command line with the .JAR file, but somewhere my > syntax is not right. Additionally, all the XML files have NO > EXTENSION so I don't know how to specify that either in the > command line. So, any help in that area would also be appreciated. > > > Could you provide your syntax? > > The syntax should be: > create db dbname path/to/folder > > The missing extension shouldnt be a problem, i just tested it. > > > In the GUI in 6.1, I can create a database from a whole directory > hierarchy of 5,681 individual XML files. When I point to the same > source file directory and using the 6.3.4 GUI (which I installed with > a pre-release version of the installer), it only processes 14 files. > No error message (that I saw) was produced. The behavior that I'm > wanting is what 6.1 does. Glad to help in solving the problem. > > We will check for modifications between the releases. As quick > workaround you could try to create an empty database and then add the > documents with the Database/Add documents menu. > > Kind regards, > Andreas > > Am 28.12.10 12:55, schrieb David leigh: > > I've just installed 6.3.4 and I've noticed a difference from version 6.1. > > In the GUI in 6.1, I can create a database from a whole directory > hierarchy of 5,681 individual XML files. When I point to the same > source file directory and using the 6.3.4 GUI (which I installed with > a pre-release version of the installer), it only processes 14 files. > No error message (that I saw) was produced. The behavior that I'm > wanting is what 6.1 does. Glad to help in solving the problem. > > Additional question: That was in the GUI. I'd like to do the same > thing using the command line with the .JAR file, but somewhere my > syntax is not right. Additionally, all the XML files have NO > EXTENSION so I don't know how to specify that either in the command > line. So, any help in that area would also be appreciated. > > Thank you, > > David > > DavidEmailSig <mailto:david at leighweb.com> > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de <mailto:BaseX-Talk at mailman.uni-konstanz.de> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

> drop db songs

> create db C:\Users\David\Documents\OpenSong\songpacks\Songs songs

> info index

>

> add C:\Users\David\Documents\OpenSong\songpacks\Songs

> info index

> help add

> add C:\Users\David\Documents\OpenSong\songpacks\Songs\*.*

>

From: Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] Sent: Wednesday, December 29, 2010 11:26 AM To: david at leighweb.com; BaseX-Talk at mailman.uni-konstanz.de Subject: Re: [basex-talk] Database creation from a directory tree Hi David, the syntax of the create command changed between the releases of 6.1 and 6.3.4. In release 6.1 you have to use: create db C:\Users\David\Documents\OpenSong\songpacks\Songs songs Kind regards, Andreas Am 28.12.10 18:00, schrieb David leigh: Here are my command line results from 6.3.4: C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX>java -cp BaseX.jar org.basex.BaseXClient -Uadmin -Padmin BaseX 6.3.5 [Client] Try "help" to get more information. > drop db songs Database 'songs' was dropped. > create db song Database 'song' created in 1683.61 ms. > info index Tags - Structure: Hash - Entries: 0 Attributes - Structure: Hash - Entries: 0 Text Index - Structure: Binary tree - Size: 4 Bytes - Entries: 0 Attribute Index - Structure: Binary tree - Size: 4 Bytes - Entries: 0 Full-Text Index - Not available Path Summary doc() 1x > info table Stopped at line 1, column 10: Syntax: INFO ([DATABASE|INDEX|STORAGE]) Show information on current database. Shows information on the currently opened database: - no argument: show global information - DATABASE: shows database information - INDEX: shows index information - STORAGE [start end] | [query]: show internal database table > info storage PRE DIS SIZ ATS NS KIND CONTENT ------------------------------------- 0 1 1 1 0 DOC song > info General Information Database Path: C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX\data Used Main Memory: 895 KB Database Creation Whitespace Chopping: ON Entity Parsing: OFF Indexes Path Summary: ON Text Index: ON Attribute Index: ON Full-Text Index: OFF > So, for 6.3.4, it creates from the command line, but it's empty. The root of that path has NO files in it. The 5861 files are in all the subdirectories. Then, when I try the same syntax with 6.1, I get the following results: > create db songs C:\Users\David\Documents\OpenSong\songpacks\Songs Stopped at line 1, column 17: Syntax: CREATE [DB|COLL|FS|INDEX|USER] [...] Create database, index or user. Creates a new database, index or user: - DB [path] [name?]: creates the database [name] for the file or directory [path] - COLL [name]: creates an empty database [name] - INDEX [TEXT|ATTRIBUTE|FULLTEXT|PATH]: creates the specified index - FS [path] [name] ([mountpoint] [backingstore]): creates filesystem database [name] for [path] (using [mountpoint] and [backingstore] if FUSE is activated) - USER [name] [password?]: creates the specified user Summary: 6.1 GUI - works as I want it 6.1 command line - can't get it to create 6.3.4 GUI - creates but with only 14 entries 6.3.4 command line - creates, but it's empty I don't mind staying with 6.1, but I DO want to do everything with the command line. Is the ADD syntax the same, in that I can simply specify the path and it will recurse the whole directory tree? Thanks!!! David <mailto:david at leighweb.com> DavidEmailSig From: Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] Sent: Tuesday, December 28, 2010 01:29 PM To: david at leighweb.com Cc: basex-talk at mailman.uni-konstanz.de Subject: Re: [basex-talk] Database creation from a directory tree Additional question: That was in the GUI. I'd like to do the same thing using the command line with the .JAR file, but somewhere my syntax is not right. Additionally, all the XML files have NO EXTENSION so I don't know how to specify that either in the command line. So, any help in that area would also be appreciated. Could you provide your syntax? The syntax should be: create db dbname path/to/folder The missing extension shouldnt be a problem, i just tested it. In the GUI in 6.1, I can create a database from a whole directory hierarchy of 5,681 individual XML files. When I point to the same source file directory and using the 6.3.4 GUI (which I installed with a pre-release version of the installer), it only processes 14 files. No error message (that I saw) was produced. The behavior that I'm wanting is what 6.1 does. Glad to help in solving the problem. We will check for modifications between the releases. As quick workaround you could try to create an empty database and then add the documents with the Database/Add documents menu. Kind regards, Andreas Am 28.12.10 12:55, schrieb David leigh: I've just installed 6.3.4 and I've noticed a difference from version 6.1. In the GUI in 6.1, I can create a database from a whole directory hierarchy of 5,681 individual XML files. When I point to the same source file directory and using the 6.3.4 GUI (which I installed with a pre-release version of the installer), it only processes 14 files. No error message (that I saw) was produced. The behavior that I'm wanting is what 6.1 does. Glad to help in solving the problem. Additional question: That was in the GUI. I'd like to do the same thing using the command line with the .JAR file, but somewhere my syntax is not right. Additionally, all the XML files have NO EXTENSION so I don't know how to specify that either in the command line. So, any help in that area would also be appreciated. Thank you, David <mailto:david at leighweb.com> DavidEmailSig BaseX-Talk mailing list BaseX-Talk at mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Von: Andreas Weiler <andreas.weiler at uni-konstanz.de> An: david at leighweb.com Hi David, thanks a lot for your tests. I couldnt fix the gui bug yet...but i can help you with command line... There is a createfilter option...it can be set with: set createfilter * so all files in the folder are recognized for creating a database. I hope this works for now, Happy new year, Andreas Am 31.12.10 15:13, schrieb David leigh: > > Andreas, > > I just did an interesting test (using the version that the windows > installer installs: 6.3.4). > > I copied my directory tree of files and in the new copy, I gave all my > data files an extension of "xml" (previously they had NO extension). > NOW, both with the command line and the GUI, the full database is > created without any problem. To double check, I went back to the old > directory with no extensions and created a new database using a filter > of "*.*" and only 14 documents were created in the database (instead > of 5000+). > > Conclusions: > > 1.In the command line you must have input documents with an extension > of "xml" since there is no visible way to provide a filter. > > 2.The GUI interfaces differently than the command line with the basex > core since it can receive a filter as well as a path in the database > creation process > > 3.The GUI filter functionality in 6.3.4 is different than in 6.1...for > me it is "broken" vs. how it works in 6.1 > > So, I think for my project to work completely, I need new > functionality: a file name filter that works correctly on the CREATE > DB statement in the command line interface. Does this sound correct > to you? > > Unfortunately, I don't have control over the extension names of my > input files as they are created by another piece of software. Until > I can do a create database in the command line, I'll have to do manual > gui-based (6.1 version) refreshes of the data or some sort of long > copy/rename process in the background. > > Thanks! > > David > > DavidEmailSig <mailto:david at leighweb.com> > > *From:*Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] > *Sent:* Wednesday, December 29, 2010 11:26 AM > *To:* david at leighweb.com; BaseX-Talk at mailman.uni-konstanz.de > *Subject:* Re: [basex-talk] Database creation from a directory tree > > Hi David, > > the syntax of the create command changed between the releases of 6.1 > and 6.3.4. > > In release 6.1 you have to use: > > create db C:\Users\David\Documents\OpenSong\songpacks\Songs songs > > Kind regards, > Andreas > > Am 28.12.10 18:00, schrieb David leigh: > > Here are my command line results from 6.3.4: > > C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX>java -cp > BaseX.jar org.basex.BaseXClient -Uadmin -Padmin > > BaseX 6.3.5 [Client] > > Try "help" to get more information. > > > drop db songs > > Database 'songs' was dropped. > > > create db song > > Database 'song' created in 1683.61 ms. > > > info index > > Tags > > - Structure: Hash > > - Entries: 0 > > Attributes > > - Structure: Hash > > - Entries: 0 > > Text Index > > - Structure: Binary tree > > - Size: 4 Bytes > > - Entries: 0 > > Attribute Index > > - Structure: Binary tree > > - Size: 4 Bytes > > - Entries: 0 > > Full-Text Index > > - Not available > > Path Summary > > doc() 1x > > > info table > > Stopped at line 1, column 10: > > Syntax: INFO ([DATABASE|INDEX|STORAGE]) > > Show information on current database. > > Shows information on the currently opened database: > > - no argument: show global information > > - DATABASE: shows database information > > - INDEX: shows index information > > - STORAGE [start end] | [query]: show internal database table > > > info storage > > PRE DIS SIZ ATS NS KIND CONTENT > > ------------------------------------- > > 0 1 1 1 0 DOC song > > > info > > General Information > > Database Path: C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX\data > > Used Main Memory: 895 KB > > Database Creation > > Whitespace Chopping: ON > > Entity Parsing: OFF > > Indexes > > Path Summary: ON > > Text Index: ON > > Attribute Index: ON > > Full-Text Index: OFF > > > > > So, for 6.3.4, it creates from the command line, but it's empty. The > root of that path has NO files in it. The 5861 files are in all the > subdirectories. > > Then, when I try the same syntax with 6.1, I get the following results: > > > create db songs C:\Users\David\Documents\OpenSong\songpacks\Songs > > Stopped at line 1, column 17: > > Syntax: CREATE [DB|COLL|FS|INDEX|USER] [...] > > Create database, index or user. > > Creates a new database, index or user: > > - DB [path] [name?]: > > creates the database [name] for the file or directory [path] > > - COLL [name]: > > creates an empty database [name] > > - INDEX [TEXT|ATTRIBUTE|FULLTEXT|PATH]: > > creates the specified index > > - FS [path] [name] ([mountpoint] [backingstore]): > > creates filesystem database [name] for [path] > > (using [mountpoint] and [backingstore] if FUSE is activated) > > - USER [name] [password?]: > > creates the specified user > > Summary: > > 6.1 GUI -- works as I want it > > 6.1 command line -- can't get it to create > > 6.3.4 GUI -- creates but with only 14 entries > > 6.3.4 command line -- creates, but it's empty > > I don't mind staying with 6.1, but I DO want to do everything with the > command line. > > Is the ADD syntax the same, in that I can simply specify the path and > it will recurse the whole directory tree? > > Thanks!!! > > David > > DavidEmailSig <mailto:david at leighweb.com> > > *From:*Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] > *Sent:* Tuesday, December 28, 2010 01:29 PM > *To:* david at leighweb.com <mailto:david at leighweb.com> > *Cc:* basex-talk at mailman.uni-konstanz.de > <mailto:basex-talk at mailman.uni-konstanz.de> > *Subject:* Re: [basex-talk] Database creation from a directory tree > > Additional question: That was in the GUI. I'd like to do the same > thing using the command line with the .JAR file, but somewhere my > syntax is not right. Additionally, all the XML files have NO > EXTENSION so I don't know how to specify that either in the > command line. So, any help in that area would also be appreciated. > > > Could you provide your syntax? > > The syntax should be: > create db dbname path/to/folder > > The missing extension shouldnt be a problem, i just tested it. > > > > In the GUI in 6.1, I can create a database from a whole directory > hierarchy of 5,681 individual XML files. When I point to the same > source file directory and using the 6.3.4 GUI (which I installed with > a pre-release version of the installer), it only processes 14 files. > No error message (that I saw) was produced. The behavior that I'm > wanting is what 6.1 does. Glad to help in solving the problem. > > We will check for modifications between the releases. As quick > workaround you could try to create an empty database and then add the > documents with the Database/Add documents menu. > > Kind regards, > Andreas > > Am 28.12.10 12:55, schrieb David leigh: > > I've just installed 6.3.4 and I've noticed a difference from version 6.1. > > In the GUI in 6.1, I can create a database from a whole directory > hierarchy of 5,681 individual XML files. When I point to the same > source file directory and using the 6.3.4 GUI (which I installed with > a pre-release version of the installer), it only processes 14 files. > No error message (that I saw) was produced. The behavior that I'm > wanting is what 6.1 does. Glad to help in solving the problem. > > Additional question: That was in the GUI. I'd like to do the same > thing using the command line with the .JAR file, but somewhere my > syntax is not right. Additionally, all the XML files have NO > EXTENSION so I don't know how to specify that either in the command > line. So, any help in that area would also be appreciated. > > Thank you, > > David > > DavidEmailSig <mailto:david at leighweb.com> > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de <mailto:BaseX-Talk at mailman.uni-konstanz.de> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

Von: David leigh <david at leighweb.com> Antwort an: <david at leighweb.com> An: 'Andreas Weiler' <andreas.weiler at uni-konstanz.de> Wohoo! That works great! Thanks. You've provided the missing link for me! Have a great new year! David DavidEmailSig <mailto:david at leighweb.com> *From:*Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] *Sent:* Friday, December 31, 2010 04:41 PM *To:* david at leighweb.com *Subject:* Re: [basex-talk] Database creation from a directory tree Hi David, thanks a lot for your tests. I couldnt fix the gui bug yet...but i can help you with command line... There is a createfilter option...it can be set with: set createfilter * so all files in the folder are recognized for creating a database. I hope this works for now, Happy new year, Andreas Am 31.12.10 15:13, schrieb David leigh: Andreas, I just did an interesting test (using the version that the windows installer installs: 6.3.4). I copied my directory tree of files and in the new copy, I gave all my data files an extension of "xml" (previously they had NO extension). NOW, both with the command line and the GUI, the full database is created without any problem. To double check, I went back to the old directory with no extensions and created a new database using a filter of "*.*" and only 14 documents were created in the database (instead of 5000+). Conclusions: In the command line you must have input documents with an extension of "xml" since there is no visible way to provide a filter. The GUI interfaces differently than the command line with the basex core since it can receive a filter as well as a path in the database creation process The GUI filter functionality in 6.3.4 is different than in 6.1...for me it is "broken" vs. how it works in 6.1 So, I think for my project to work completely, I need new functionality: a file name filter that works correctly on the CREATE DB statement in the command line interface. Does this sound correct to you? Unfortunately, I don't have control over the extension names of my input files as they are created by another piece of software. Until I can do a create database in the command line, I'll have to do manual gui-based (6.1 version) refreshes of the data or some sort of long copy/rename process in the background. Thanks! David DavidEmailSig <mailto:david at leighweb.com> *From:*Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] *Sent:* Wednesday, December 29, 2010 11:26 AM *To:* david at leighweb.com <mailto:david at leighweb.com>; BaseX-Talk at mailman.uni-konstanz.de <mailto:BaseX-Talk at mailman.uni-konstanz.de> *Subject:* Re: [basex-talk] Database creation from a directory tree Hi David, the syntax of the create command changed between the releases of 6.1 and 6.3.4. In release 6.1 you have to use: create db C:\Users\David\Documents\OpenSong\songpacks\Songs songs Kind regards, Andreas Am 28.12.10 18:00, schrieb David leigh: Here are my command line results from 6.3.4: C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX>java -cp BaseX.jar org.basex.BaseXClient -Uadmin -Padmin BaseX 6.3.5 [Client] Try "help" to get more information. > drop db songs Database 'songs' was dropped. > create db song Database 'song' created in 1683.61 ms. > info index Tags - Structure: Hash - Entries: 0 Attributes - Structure: Hash - Entries: 0 Text Index - Structure: Binary tree - Size: 4 Bytes - Entries: 0 Attribute Index - Structure: Binary tree - Size: 4 Bytes - Entries: 0 Full-Text Index - Not available Path Summary doc() 1x > info table Stopped at line 1, column 10: Syntax: INFO ([DATABASE|INDEX|STORAGE]) Show information on current database. Shows information on the currently opened database: - no argument: show global information - DATABASE: shows database information - INDEX: shows index information - STORAGE [start end] | [query]: show internal database table > info storage PRE DIS SIZ ATS NS KIND CONTENT ------------------------------------- 0 1 1 1 0 DOC song > info General Information Database Path: C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX\data Used Main Memory: 895 KB Database Creation Whitespace Chopping: ON Entity Parsing: OFF Indexes Path Summary: ON Text Index: ON Attribute Index: ON Full-Text Index: OFF > So, for 6.3.4, it creates from the command line, but it's empty. The root of that path has NO files in it. The 5861 files are in all the subdirectories. Then, when I try the same syntax with 6.1, I get the following results: > create db songs C:\Users\David\Documents\OpenSong\songpacks\Songs Stopped at line 1, column 17: Syntax: CREATE [DB|COLL|FS|INDEX|USER] [...] Create database, index or user. Creates a new database, index or user: - DB [path] [name?]: creates the database [name] for the file or directory [path] - COLL [name]: creates an empty database [name] - INDEX [TEXT|ATTRIBUTE|FULLTEXT|PATH]: creates the specified index - FS [path] [name] ([mountpoint] [backingstore]): creates filesystem database [name] for [path] (using [mountpoint] and [backingstore] if FUSE is activated) - USER [name] [password?]: creates the specified user Summary: 6.1 GUI -- works as I want it 6.1 command line -- can't get it to create 6.3.4 GUI -- creates but with only 14 entries 6.3.4 command line -- creates, but it's empty I don't mind staying with 6.1, but I DO want to do everything with the command line. Is the ADD syntax the same, in that I can simply specify the path and it will recurse the whole directory tree? Thanks!!! David DavidEmailSig <mailto:david at leighweb.com> *From:*Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] *Sent:* Tuesday, December 28, 2010 01:29 PM *To:* david at leighweb.com <mailto:david at leighweb.com> *Cc:* basex-talk at mailman.uni-konstanz.de <mailto:basex-talk at mailman.uni-konstanz.de> *Subject:* Re: [basex-talk] Database creation from a directory tree Additional question: That was in the GUI. I'd like to do the same thing using the command line with the .JAR file, but somewhere my syntax is not right. Additionally, all the XML files have NO EXTENSION so I don't know how to specify that either in the command line. So, any help in that area would also be appreciated. Could you provide your syntax? The syntax should be: create db dbname path/to/folder The missing extension shouldnt be a problem, i just tested it. In the GUI in 6.1, I can create a database from a whole directory hierarchy of 5,681 individual XML files. When I point to the same source file directory and using the 6.3.4 GUI (which I installed with a pre-release version of the installer), it only processes 14 files. No error message (that I saw) was produced. The behavior that I'm wanting is what 6.1 does. Glad to help in solving the problem. We will check for modifications between the releases. As quick workaround you could try to create an empty database and then add the documents with the Database/Add documents menu. Kind regards, Andreas Am 28.12.10 12:55, schrieb David leigh: I've just installed 6.3.4 and I've noticed a difference from version 6.1. In the GUI in 6.1, I can create a database from a whole directory hierarchy of 5,681 individual XML files. When I point to the same source file directory and using the 6.3.4 GUI (which I installed with a pre-release version of the installer), it only processes 14 files. No error message (that I saw) was produced. The behavior that I'm wanting is what 6.1 does. Glad to help in solving the problem. Additional question: That was in the GUI. I'd like to do the same thing using the command line with the .JAR file, but somewhere my syntax is not right. Additionally, all the XML files have NO EXTENSION so I don't know how to specify that either in the command line. So, any help in that area would also be appreciated. Thank you, David DavidEmailSig <mailto:david at leighweb.com> BaseX-Talk mailing list BaseX-Talk at mailman.uni-konstanz.de <mailto:BaseX-Talk at mailman.uni-konstanz.de> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Am 03.01.11 11:02, schrieb Andreas Weiler: > > > -------- Original-Nachricht -------- > Betreff: RE: [basex-talk] Database creation from a directory tree > Datum: Fri, 31 Dec 2010 17:59:34 +0100 > Von: David leigh <david at leighweb.com> > Antwort an: <david at leighweb.com> > An: 'Andreas Weiler' <andreas.weiler at uni-konstanz.de> > > > > Wohoo! That works great! > > Thanks. You've provided the missing link for me! > > Have a great new year! > > David > > DavidEmailSig <mailto:david at leighweb.com> > > *From:*Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] > *Sent:* Friday, December 31, 2010 04:41 PM > *To:* david at leighweb.com > *Subject:* Re: [basex-talk] Database creation from a directory tree > > Hi David, > > thanks a lot for your tests. > I couldnt fix the gui bug yet...but i can help you with command line... > > There is a createfilter option...it can be set with: > set createfilter * > > so all files in the folder are recognized for creating a database. > > I hope this works for now, > Happy new year, > Andreas > > Am 31.12.10 15:13, schrieb David leigh: > > Andreas, > > I just did an interesting test (using the version that the windows > installer installs: 6.3.4). > > I copied my directory tree of files and in the new copy, I gave all my > data files an extension of "xml" (previously they had NO extension). > NOW, both with the command line and the GUI, the full database is > created without any problem. To double check, I went back to the old > directory with no extensions and created a new database using a filter > of "*.*" and only 14 documents were created in the database (instead > of 5000+). > > Conclusions: > > In the command line you must have input documents with an extension of > "xml" since there is no visible way to provide a filter. > > The GUI interfaces differently than the command line with the basex > core since it can receive a filter as well as a path in the database > creation process > > The GUI filter functionality in 6.3.4 is different than in 6.1...for > me it is "broken" vs. how it works in 6.1 > > So, I think for my project to work completely, I need new > functionality: a file name filter that works correctly on the CREATE > DB statement in the command line interface. Does this sound correct > to you? > > Unfortunately, I don't have control over the extension names of my > input files as they are created by another piece of software. Until > I can do a create database in the command line, I'll have to do manual > gui-based (6.1 version) refreshes of the data or some sort of long > copy/rename process in the background. > > Thanks! > > David > > DavidEmailSig <mailto:david at leighweb.com> > > *From:*Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] > *Sent:* Wednesday, December 29, 2010 11:26 AM > *To:* david at leighweb.com <mailto:david at leighweb.com>; > BaseX-Talk at mailman.uni-konstanz.de > <mailto:BaseX-Talk at mailman.uni-konstanz.de> > *Subject:* Re: [basex-talk] Database creation from a directory tree > > Hi David, > > the syntax of the create command changed between the releases of 6.1 > and 6.3.4. > > In release 6.1 you have to use: > > create db C:\Users\David\Documents\OpenSong\songpacks\Songs songs > > Kind regards, > Andreas > > Am 28.12.10 18:00, schrieb David leigh: > > Here are my command line results from 6.3.4: > > C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX>java -cp > BaseX.jar org.basex.BaseXClient -Uadmin -Padmin > > BaseX 6.3.5 [Client] > > Try "help" to get more information. > > > drop db songs > > Database 'songs' was dropped. > > > create db song > > Database 'song' created in 1683.61 ms. > > > info index > > Tags > > - Structure: Hash > > - Entries: 0 > > Attributes > > - Structure: Hash > > - Entries: 0 > > Text Index > > - Structure: Binary tree > > - Size: 4 Bytes > > - Entries: 0 > > Attribute Index > > - Structure: Binary tree > > - Size: 4 Bytes > > - Entries: 0 > > Full-Text Index > > - Not available > > Path Summary > > doc() 1x > > > info table > > Stopped at line 1, column 10: > > Syntax: INFO ([DATABASE|INDEX|STORAGE]) > > Show information on current database. > > Shows information on the currently opened database: > > - no argument: show global information > > - DATABASE: shows database information > > - INDEX: shows index information > > - STORAGE [start end] | [query]: show internal database table > > > info storage > > PRE DIS SIZ ATS NS KIND CONTENT > > ------------------------------------- > > 0 1 1 1 0 DOC song > > > info > > General Information > > Database Path: C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX\data > > Used Main Memory: 895 KB > > Database Creation > > Whitespace Chopping: ON > > Entity Parsing: OFF > > Indexes > > Path Summary: ON > > Text Index: ON > > Attribute Index: ON > > Full-Text Index: OFF > > > > > So, for 6.3.4, it creates from the command line, but it's empty. The > root of that path has NO files in it. The 5861 files are in all the > subdirectories. > > Then, when I try the same syntax with 6.1, I get the following results: > > > create db songs C:\Users\David\Documents\OpenSong\songpacks\Songs > > Stopped at line 1, column 17: > > Syntax: CREATE [DB|COLL|FS|INDEX|USER] [...] > > Create database, index or user. > > Creates a new database, index or user: > > - DB [path] [name?]: > > creates the database [name] for the file or directory [path] > > - COLL [name]: > > creates an empty database [name] > > - INDEX [TEXT|ATTRIBUTE|FULLTEXT|PATH]: > > creates the specified index > > - FS [path] [name] ([mountpoint] [backingstore]): > > creates filesystem database [name] for [path] > > (using [mountpoint] and [backingstore] if FUSE is activated) > > - USER [name] [password?]: > > creates the specified user > > Summary: > > 6.1 GUI -- works as I want it > > 6.1 command line -- can't get it to create > > 6.3.4 GUI -- creates but with only 14 entries > > 6.3.4 command line -- creates, but it's empty > > I don't mind staying with 6.1, but I DO want to do everything with the > command line. > > Is the ADD syntax the same, in that I can simply specify the path and > it will recurse the whole directory tree? > > Thanks!!! > > David > > DavidEmailSig <mailto:david at leighweb.com> > > *From:*Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] > *Sent:* Tuesday, December 28, 2010 01:29 PM > *To:* david at leighweb.com <mailto:david at leighweb.com> > *Cc:* basex-talk at mailman.uni-konstanz.de > <mailto:basex-talk at mailman.uni-konstanz.de> > *Subject:* Re: [basex-talk] Database creation from a directory tree > > Additional question: That was in the GUI. I'd like to do the same > thing using the command line with the .JAR file, but somewhere my > syntax is not right. Additionally, all the XML files have NO > EXTENSION so I don't know how to specify that either in the > command line. So, any help in that area would also be appreciated. > > > Could you provide your syntax? > > The syntax should be: > create db dbname path/to/folder > > The missing extension shouldnt be a problem, i just tested it. > > > > > In the GUI in 6.1, I can create a database from a whole directory > hierarchy of 5,681 individual XML files. When I point to the same > source file directory and using the 6.3.4 GUI (which I installed with > a pre-release version of the installer), it only processes 14 files. > No error message (that I saw) was produced. The behavior that I'm > wanting is what 6.1 does. Glad to help in solving the problem. > > We will check for modifications between the releases. As quick > workaround you could try to create an empty database and then add the > documents with the Database/Add documents menu. > > Kind regards, > Andreas > > Am 28.12.10 12:55, schrieb David leigh: > > I've just installed 6.3.4 and I've noticed a difference from version 6.1. > > In the GUI in 6.1, I can create a database from a whole directory > hierarchy of 5,681 individual XML files. When I point to the same > source file directory and using the 6.3.4 GUI (which I installed with > a pre-release version of the installer), it only processes 14 files. > No error message (that I saw) was produced. The behavior that I'm > wanting is what 6.1 does. Glad to help in solving the problem. > > Additional question: That was in the GUI. I'd like to do the same > thing using the command line with the .JAR file, but somewhere my > syntax is not right. Additionally, all the XML files have NO > EXTENSION so I don't know how to specify that either in the command > line. So, any help in that area would also be appreciated. > > Thank you, > > David > > DavidEmailSig <mailto:david at leighweb.com> > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de <mailto:BaseX-Talk at mailman.uni-konstanz.de> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Am 06.01.11 20:06, schrieb David leigh: > > Andreas, > > I spoke too soon about my success. > > If I do: > > set createfilter * > > from an */interactive/* commandline, it works. > > If I do the same thing as a */passed command/* it does NOT work, BUT a > passed command is accepted if I do: > > set createfilter '*' > > Unfortunately the filter is then incorrect! > > Here's my results from the DOS console: > > */C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX>java -cp > BaseX.jar org.basex.BaseXClient -Uadmin -Padmin -c set createfilter */* > > */Error: Stopped at line 1, column 24:/* > > */Syntax: SET [option] ([value])/* > > */ Set global options (current values: try 'info')./* > > *//* > > */Available options [option] with value [value] = on/off:/* > > */- QUERYINFO: Display of query info/* > > */- DEBUG: Display of debug info/* > > */- SERIALIZE: Serialization of query results/* > > */- CHOP: Chopping of XML whitespaces/* > > */- ENTITY: Parsing of XML entities/* > > */- TEXTINDEX: Text indexing/* > > */- ATTRINDEX: Attribute value indexing/* > > */- FTINDEX: Full-text indexing/* > > */- PATHINDEX: Path indexing/* > > *//* > > */C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX>java -cp > BaseX.jar org.basex.BaseXClient -Uadmin -Padmin/* > > */BaseX 6.3.5 [Client]/* > > */Try "help" to get more information./* > > *//* > > */> set createfilter */* > > */CREATEFILTER: */* > > */>/* > > So, I'm stuck again as it appears that the parser for the "-c" option > on the command line is slightly different from the command parser on > the interactive command line. Is there another way to do this? > > Thanks! > > David > > DavidEmailSig <mailto:david at leighweb.com> > > *From:*Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] > *Sent:* Friday, December 31, 2010 04:41 PM > *To:* david at leighweb.com > *Subject:* Re: [basex-talk] Database creation from a directory tree > > Hi David, > > thanks a lot for your tests. > I couldnt fix the gui bug yet...but i can help you with command line... > > There is a createfilter option...it can be set with: > set createfilter * > > so all files in the folder are recognized for creating a database. > > I hope this works for now, > Happy new year, > Andreas > > Am 31.12.10 15:13, schrieb David leigh: > > Andreas, > > I just did an interesting test (using the version that the windows > installer installs: 6.3.4). > > I copied my directory tree of files and in the new copy, I gave all my > data files an extension of "xml" (previously they had NO extension). > NOW, both with the command line and the GUI, the full database is > created without any problem. To double check, I went back to the old > directory with no extensions and created a new database using a filter > of "*.*" and only 14 documents were created in the database (instead > of 5000+). > > Conclusions: > > In the command line you must have input documents with an extension of > "xml" since there is no visible way to provide a filter. > > The GUI interfaces differently than the command line with the basex > core since it can receive a filter as well as a path in the database > creation process > > The GUI filter functionality in 6.3.4 is different than in 6.1...for > me it is "broken" vs. how it works in 6.1 > > So, I think for my project to work completely, I need new > functionality: a file name filter that works correctly on the CREATE > DB statement in the command line interface. Does this sound correct > to you? > > Unfortunately, I don't have control over the extension names of my > input files as they are created by another piece of software. Until > I can do a create database in the command line, I'll have to do manual > gui-based (6.1 version) refreshes of the data or some sort of long > copy/rename process in the background. > > Thanks! > > David > > DavidEmailSig <mailto:david at leighweb.com> > > *From:*Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] > *Sent:* Wednesday, December 29, 2010 11:26 AM > *To:* david at leighweb.com <mailto:david at leighweb.com>; > BaseX-Talk at mailman.uni-konstanz.de > <mailto:BaseX-Talk at mailman.uni-konstanz.de> > *Subject:* Re: [basex-talk] Database creation from a directory tree > > Hi David, > > the syntax of the create command changed between the releases of 6.1 > and 6.3.4. > > In release 6.1 you have to use: > > create db C:\Users\David\Documents\OpenSong\songpacks\Songs songs > > Kind regards, > Andreas > > Am 28.12.10 18:00, schrieb David leigh: > > Here are my command line results from 6.3.4: > > C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX>java -cp > BaseX.jar org.basex.BaseXClient -Uadmin -Padmin > > BaseX 6.3.5 [Client] > > Try "help" to get more information. > > > drop db songs > > Database 'songs' was dropped. > > > create db song > > Database 'song' created in 1683.61 ms. > > > info index > > Tags > > - Structure: Hash > > - Entries: 0 > > Attributes > > - Structure: Hash > > - Entries: 0 > > Text Index > > - Structure: Binary tree > > - Size: 4 Bytes > > - Entries: 0 > > Attribute Index > > - Structure: Binary tree > > - Size: 4 Bytes > > - Entries: 0 > > Full-Text Index > > - Not available > > Path Summary > > doc() 1x > > > info table > > Stopped at line 1, column 10: > > Syntax: INFO ([DATABASE|INDEX|STORAGE]) > > Show information on current database. > > Shows information on the currently opened database: > > - no argument: show global information > > - DATABASE: shows database information > > - INDEX: shows index information > > - STORAGE [start end] | [query]: show internal database table > > > info storage > > PRE DIS SIZ ATS NS KIND CONTENT > > ------------------------------------- > > 0 1 1 1 0 DOC song > > > info > > General Information > > Database Path: C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX\data > > Used Main Memory: 895 KB > > Database Creation > > Whitespace Chopping: ON > > Entity Parsing: OFF > > Indexes > > Path Summary: ON > > Text Index: ON > > Attribute Index: ON > > Full-Text Index: OFF > > > > > So, for 6.3.4, it creates from the command line, but it's empty. The > root of that path has NO files in it. The 5861 files are in all the > subdirectories. > > Then, when I try the same syntax with 6.1, I get the following results: > > > create db songs C:\Users\David\Documents\OpenSong\songpacks\Songs > > Stopped at line 1, column 17: > > Syntax: CREATE [DB|COLL|FS|INDEX|USER] [...] > > Create database, index or user. > > Creates a new database, index or user: > > - DB [path] [name?]: > > creates the database [name] for the file or directory [path] > > - COLL [name]: > > creates an empty database [name] > > - INDEX [TEXT|ATTRIBUTE|FULLTEXT|PATH]: > > creates the specified index > > - FS [path] [name] ([mountpoint] [backingstore]): > > creates filesystem database [name] for [path] > > (using [mountpoint] and [backingstore] if FUSE is activated) > > - USER [name] [password?]: > > creates the specified user > > Summary: > > 6.1 GUI -- works as I want it > > 6.1 command line -- can't get it to create > > 6.3.4 GUI -- creates but with only 14 entries > > 6.3.4 command line -- creates, but it's empty > > I don't mind staying with 6.1, but I DO want to do everything with the > command line. > > Is the ADD syntax the same, in that I can simply specify the path and > it will recurse the whole directory tree? > > Thanks!!! > > David > > DavidEmailSig <mailto:david at leighweb.com> > > *From:*Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] > *Sent:* Tuesday, December 28, 2010 01:29 PM > *To:* david at leighweb.com <mailto:david at leighweb.com> > *Cc:* basex-talk at mailman.uni-konstanz.de > <mailto:basex-talk at mailman.uni-konstanz.de> > *Subject:* Re: [basex-talk] Database creation from a directory tree > > Additional question: That was in the GUI. I'd like to do the same > thing using the command line with the .JAR file, but somewhere my > syntax is not right. Additionally, all the XML files have NO > EXTENSION so I don't know how to specify that either in the > command line. So, any help in that area would also be appreciated. > > > Could you provide your syntax? > > The syntax should be: > create db dbname path/to/folder > > The missing extension shouldnt be a problem, i just tested it. > > > > > In the GUI in 6.1, I can create a database from a whole directory > hierarchy of 5,681 individual XML files. When I point to the same > source file directory and using the 6.3.4 GUI (which I installed with > a pre-release version of the installer), it only processes 14 files. > No error message (that I saw) was produced. The behavior that I'm > wanting is what 6.1 does. Glad to help in solving the problem. > > We will check for modifications between the releases. As quick > workaround you could try to create an empty database and then add the > documents with the Database/Add documents menu. > > Kind regards, > Andreas > > Am 28.12.10 12:55, schrieb David leigh: > > I've just installed 6.3.4 and I've noticed a difference from version 6.1. > > In the GUI in 6.1, I can create a database from a whole directory > hierarchy of 5,681 individual XML files. When I point to the same > source file directory and using the 6.3.4 GUI (which I installed with > a pre-release version of the installer), it only processes 14 files. > No error message (that I saw) was produced. The behavior that I'm > wanting is what 6.1 does. Glad to help in solving the problem. > > Additional question: That was in the GUI. I'd like to do the same > thing using the command line with the .JAR file, but somewhere my > syntax is not right. Additionally, all the XML files have NO > EXTENSION so I don't know how to specify that either in the command > line. So, any help in that area would also be appreciated. > > Thank you, > > David > > DavidEmailSig <mailto:david at leighweb.com> > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de <mailto:BaseX-Talk at mailman.uni-konstanz.de> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

From: Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] Sent: Thursday, January 06, 2011 08:37 PM To: david at leighweb.com; BaseX-Talk at mailman.uni-konstanz.de Subject: Re: [basex-talk] Database creation from a directory tree Hi David, just surround the command with quotes: java -cp BaseX.jar org.basex.BaseXClient -c "set createfilter *" Kind regards, Andreas Am 06.01.11 20:06, schrieb David leigh: Andreas, I spoke too soon about my success. If I do: set createfilter * from an interactive commandline, it works. If I do the same thing as a passed command it does NOT work, BUT a passed command is accepted if I do: set createfilter '*' Unfortunately the filter is then incorrect! Here's my results from the DOS console: C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX>java -cp BaseX.jar org.basex.BaseXClient -Uadmin -Padmin -c set createfilter * Error: Stopped at line 1, column 24: Syntax: SET [option] ([value]) Set global options (current values: try 'info'). Available options [option] with value [value] = on/off: - QUERYINFO: Display of query info - DEBUG: Display of debug info - SERIALIZE: Serialization of query results - CHOP: Chopping of XML whitespaces - ENTITY: Parsing of XML entities - TEXTINDEX: Text indexing - ATTRINDEX: Attribute value indexing - FTINDEX: Full-text indexing - PATHINDEX: Path indexing C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX>java -cp BaseX.jar org.basex.BaseXClient -Uadmin -Padmin BaseX 6.3.5 [Client] Try "help" to get more information. > set createfilter * CREATEFILTER: * > So, I'm stuck again as it appears that the parser for the "-c" option on the command line is slightly different from the command parser on the interactive command line. Is there another way to do this? Thanks! David <mailto:david at leighweb.com> DavidEmailSig From: Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] Sent: Friday, December 31, 2010 04:41 PM To: david at leighweb.com Subject: Re: [basex-talk] Database creation from a directory tree Hi David, thanks a lot for your tests. I couldnt fix the gui bug yet...but i can help you with command line... There is a createfilter option...it can be set with: set createfilter * so all files in the folder are recognized for creating a database. I hope this works for now, Happy new year, Andreas Am 31.12.10 15:13, schrieb David leigh: Andreas, I just did an interesting test (using the version that the windows installer installs: 6.3.4). I copied my directory tree of files and in the new copy, I gave all my data files an extension of "xml" (previously they had NO extension). NOW, both with the command line and the GUI, the full database is created without any problem. To double check, I went back to the old directory with no extensions and created a new database using a filter of "*.*" and only 14 documents were created in the database (instead of 5000+). Conclusions: In the command line you must have input documents with an extension of "xml" since there is no visible way to provide a filter. The GUI interfaces differently than the command line with the basex core since it can receive a filter as well as a path in the database creation process The GUI filter functionality in 6.3.4 is different than in 6.1.for me it is "broken" vs. how it works in 6.1 So, I think for my project to work completely, I need new functionality: a file name filter that works correctly on the CREATE DB statement in the command line interface. Does this sound correct to you? Unfortunately, I don't have control over the extension names of my input files as they are created by another piece of software. Until I can do a create database in the command line, I'll have to do manual gui-based (6.1 version) refreshes of the data or some sort of long copy/rename process in the background. Thanks! David <mailto:david at leighweb.com> DavidEmailSig From: Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] Sent: Wednesday, December 29, 2010 11:26 AM To: david at leighweb.com; BaseX-Talk at mailman.uni-konstanz.de Subject: Re: [basex-talk] Database creation from a directory tree Hi David, the syntax of the create command changed between the releases of 6.1 and 6.3.4. In release 6.1 you have to use: create db C:\Users\David\Documents\OpenSong\songpacks\Songs songs Kind regards, Andreas Am 28.12.10 18:00, schrieb David leigh: Here are my command line results from 6.3.4: C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX>java -cp BaseX.jar org.basex.BaseXClient -Uadmin -Padmin BaseX 6.3.5 [Client] Try "help" to get more information. > drop db songs Database 'songs' was dropped. > create db song Database 'song' created in 1683.61 ms. > info index Tags - Structure: Hash - Entries: 0 Attributes - Structure: Hash - Entries: 0 Text Index - Structure: Binary tree - Size: 4 Bytes - Entries: 0 Attribute Index - Structure: Binary tree - Size: 4 Bytes - Entries: 0 Full-Text Index - Not available Path Summary doc() 1x > info table Stopped at line 1, column 10: Syntax: INFO ([DATABASE|INDEX|STORAGE]) Show information on current database. Shows information on the currently opened database: - no argument: show global information - DATABASE: shows database information - INDEX: shows index information - STORAGE [start end] | [query]: show internal database table > info storage PRE DIS SIZ ATS NS KIND CONTENT ------------------------------------- 0 1 1 1 0 DOC song > info General Information Database Path: C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX\data Used Main Memory: 895 KB Database Creation Whitespace Chopping: ON Entity Parsing: OFF Indexes Path Summary: ON Text Index: ON Attribute Index: ON Full-Text Index: OFF > So, for 6.3.4, it creates from the command line, but it's empty. The root of that path has NO files in it. The 5861 files are in all the subdirectories. Then, when I try the same syntax with 6.1, I get the following results: > create db songs C:\Users\David\Documents\OpenSong\songpacks\Songs Stopped at line 1, column 17: Syntax: CREATE [DB|COLL|FS|INDEX|USER] [...] Create database, index or user. Creates a new database, index or user: - DB [path] [name?]: creates the database [name] for the file or directory [path] - COLL [name]: creates an empty database [name] - INDEX [TEXT|ATTRIBUTE|FULLTEXT|PATH]: creates the specified index - FS [path] [name] ([mountpoint] [backingstore]): creates filesystem database [name] for [path] (using [mountpoint] and [backingstore] if FUSE is activated) - USER [name] [password?]: creates the specified user Summary: 6.1 GUI - works as I want it 6.1 command line - can't get it to create 6.3.4 GUI - creates but with only 14 entries 6.3.4 command line - creates, but it's empty I don't mind staying with 6.1, but I DO want to do everything with the command line. Is the ADD syntax the same, in that I can simply specify the path and it will recurse the whole directory tree? Thanks!!! David <mailto:david at leighweb.com> DavidEmailSig From: Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] Sent: Tuesday, December 28, 2010 01:29 PM To: david at leighweb.com Cc: basex-talk at mailman.uni-konstanz.de Subject: Re: [basex-talk] Database creation from a directory tree Additional question: That was in the GUI. I'd like to do the same thing using the command line with the .JAR file, but somewhere my syntax is not right. Additionally, all the XML files have NO EXTENSION so I don't know how to specify that either in the command line. So, any help in that area would also be appreciated. Could you provide your syntax? The syntax should be: create db dbname path/to/folder The missing extension shouldnt be a problem, i just tested it. In the GUI in 6.1, I can create a database from a whole directory hierarchy of 5,681 individual XML files. When I point to the same source file directory and using the 6.3.4 GUI (which I installed with a pre-release version of the installer), it only processes 14 files. No error message (that I saw) was produced. The behavior that I'm wanting is what 6.1 does. Glad to help in solving the problem. We will check for modifications between the releases. As quick workaround you could try to create an empty database and then add the documents with the Database/Add documents menu. Kind regards, Andreas Am 28.12.10 12:55, schrieb David leigh: I've just installed 6.3.4 and I've noticed a difference from version 6.1. In the GUI in 6.1, I can create a database from a whole directory hierarchy of 5,681 individual XML files. When I point to the same source file directory and using the 6.3.4 GUI (which I installed with a pre-release version of the installer), it only processes 14 files. No error message (that I saw) was produced. The behavior that I'm wanting is what 6.1 does. Glad to help in solving the problem. Additional question: That was in the GUI. I'd like to do the same thing using the command line with the .JAR file, but somewhere my syntax is not right. Additionally, all the XML files have NO EXTENSION so I don't know how to specify that either in the command line. So, any help in that area would also be appreciated. Thank you, David <mailto:david at leighweb.com> DavidEmailSig BaseX-Talk mailing list BaseX-Talk at mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Am 06.01.11 20:48, schrieb David leigh: > > Great! In the meantime I discovered that if I stack commands together > it works as well -- like this: > > -c drop database Songs;SET CREATEFILTER *;SET ENTITY true;SET FTINDEX > true;SET INTPARSE true;SET PARSER xml;SET WILDCARDS true;CREATE DB > Songs "C:/Users/David/Documents/OpenSong/Songs" > > I guess the parser needed that semi-colon after the asterisk to > correctly identify the filter. > > Thanks! > > David > > DavidEmailSig <mailto:david at leighweb.com> > > *From:*Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] > *Sent:* Thursday, January 06, 2011 08:37 PM > *To:* david at leighweb.com; BaseX-Talk at mailman.uni-konstanz.de > *Subject:* Re: [basex-talk] Database creation from a directory tree > > Hi David, > > just surround the command with quotes: > java -cp BaseX.jar org.basex.BaseXClient -c "set createfilter *" > > Kind regards, > Andreas > > Am 06.01.11 20:06, schrieb David leigh: > > Andreas, > > I spoke too soon about my success. > > If I do: > > set createfilter * > > from an */interactive/* commandline, it works. > > If I do the same thing as a */passed command/* it does NOT work, BUT a > passed command is accepted if I do: > > set createfilter '*' > > Unfortunately the filter is then incorrect! > > Here's my results from the DOS console: > > */C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX>java -cp > BaseX.jar org.basex.BaseXClient -Uadmin -Padmin -c set createfilter */* > > */Error: Stopped at line 1, column 24:/* > > */Syntax: SET [option] ([value])/* > > */ Set global options (current values: try 'info')./* > > *//* > > */Available options [option] with value [value] = on/off:/* > > */- QUERYINFO: Display of query info/* > > */- DEBUG: Display of debug info/* > > */- SERIALIZE: Serialization of query results/* > > */- CHOP: Chopping of XML whitespaces/* > > */- ENTITY: Parsing of XML entities/* > > */- TEXTINDEX: Text indexing/* > > */- ATTRINDEX: Attribute value indexing/* > > */- FTINDEX: Full-text indexing/* > > */- PATHINDEX: Path indexing/* > > *//* > > */C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX>java -cp > BaseX.jar org.basex.BaseXClient -Uadmin -Padmin/* > > */BaseX 6.3.5 [Client]/* > > */Try "help" to get more information./* > > *//* > > */> set createfilter */* > > */CREATEFILTER: */* > > */> /* > > So, I'm stuck again as it appears that the parser for the "-c" option > on the command line is slightly different from the command parser on > the interactive command line. Is there another way to do this? > > Thanks! > > David > > DavidEmailSig <mailto:david at leighweb.com> > > *From:*Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] > *Sent:* Friday, December 31, 2010 04:41 PM > *To:* david at leighweb.com <mailto:david at leighweb.com> > *Subject:* Re: [basex-talk] Database creation from a directory tree > > Hi David, > > thanks a lot for your tests. > I couldnt fix the gui bug yet...but i can help you with command line... > > There is a createfilter option...it can be set with: > set createfilter * > > so all files in the folder are recognized for creating a database. > > I hope this works for now, > Happy new year, > Andreas > > Am 31.12.10 15:13, schrieb David leigh: > > Andreas, > > I just did an interesting test (using the version that the windows > installer installs: 6.3.4). > > I copied my directory tree of files and in the new copy, I gave all my > data files an extension of "xml" (previously they had NO extension). > NOW, both with the command line and the GUI, the full database is > created without any problem. To double check, I went back to the old > directory with no extensions and created a new database using a filter > of "*.*" and only 14 documents were created in the database (instead > of 5000+). > > Conclusions: > > In the command line you must have input documents with an extension of > "xml" since there is no visible way to provide a filter. > > The GUI interfaces differently than the command line with the basex > core since it can receive a filter as well as a path in the database > creation process > > The GUI filter functionality in 6.3.4 is different than in 6.1...for > me it is "broken" vs. how it works in 6.1 > > So, I think for my project to work completely, I need new > functionality: a file name filter that works correctly on the CREATE > DB statement in the command line interface. Does this sound correct > to you? > > Unfortunately, I don't have control over the extension names of my > input files as they are created by another piece of software. Until > I can do a create database in the command line, I'll have to do manual > gui-based (6.1 version) refreshes of the data or some sort of long > copy/rename process in the background. > > Thanks! > > David > > DavidEmailSig <mailto:david at leighweb.com> > > *From:*Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] > *Sent:* Wednesday, December 29, 2010 11:26 AM > *To:* david at leighweb.com <mailto:david at leighweb.com>; > BaseX-Talk at mailman.uni-konstanz.de > <mailto:BaseX-Talk at mailman.uni-konstanz.de> > *Subject:* Re: [basex-talk] Database creation from a directory tree > > Hi David, > > the syntax of the create command changed between the releases of 6.1 > and 6.3.4. > > In release 6.1 you have to use: > > create db C:\Users\David\Documents\OpenSong\songpacks\Songs songs > > Kind regards, > Andreas > > Am 28.12.10 18:00, schrieb David leigh: > > Here are my command line results from 6.3.4: > > C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX>java -cp > BaseX.jar org.basex.BaseXClient -Uadmin -Padmin > > BaseX 6.3.5 [Client] > > Try "help" to get more information. > > > drop db songs > > Database 'songs' was dropped. > > > create db song > > Database 'song' created in 1683.61 ms. > > > info index > > Tags > > - Structure: Hash > > - Entries: 0 > > Attributes > > - Structure: Hash > > - Entries: 0 > > Text Index > > - Structure: Binary tree > > - Size: 4 Bytes > > - Entries: 0 > > Attribute Index > > - Structure: Binary tree > > - Size: 4 Bytes > > - Entries: 0 > > Full-Text Index > > - Not available > > Path Summary > > doc() 1x > > > info table > > Stopped at line 1, column 10: > > Syntax: INFO ([DATABASE|INDEX|STORAGE]) > > Show information on current database. > > Shows information on the currently opened database: > > - no argument: show global information > > - DATABASE: shows database information > > - INDEX: shows index information > > - STORAGE [start end] | [query]: show internal database table > > > info storage > > PRE DIS SIZ ATS NS KIND CONTENT > > ------------------------------------- > > 0 1 1 1 0 DOC song > > > info > > General Information > > Database Path: C:\Users\David\Documents\OpenSong\OpenSongSearch\BaseX\data > > Used Main Memory: 895 KB > > Database Creation > > Whitespace Chopping: ON > > Entity Parsing: OFF > > Indexes > > Path Summary: ON > > Text Index: ON > > Attribute Index: ON > > Full-Text Index: OFF > > > > > So, for 6.3.4, it creates from the command line, but it's empty. The > root of that path has NO files in it. The 5861 files are in all the > subdirectories. > > Then, when I try the same syntax with 6.1, I get the following results: > > > create db songs C:\Users\David\Documents\OpenSong\songpacks\Songs > > Stopped at line 1, column 17: > > Syntax: CREATE [DB|COLL|FS|INDEX|USER] [...] > > Create database, index or user. > > Creates a new database, index or user: > > - DB [path] [name?]: > > creates the database [name] for the file or directory [path] > > - COLL [name]: > > creates an empty database [name] > > - INDEX [TEXT|ATTRIBUTE|FULLTEXT|PATH]: > > creates the specified index > > - FS [path] [name] ([mountpoint] [backingstore]): > > creates filesystem database [name] for [path] > > (using [mountpoint] and [backingstore] if FUSE is activated) > > - USER [name] [password?]: > > creates the specified user > > Summary: > > 6.1 GUI -- works as I want it > > 6.1 command line -- can't get it to create > > 6.3.4 GUI -- creates but with only 14 entries > > 6.3.4 command line -- creates, but it's empty > > I don't mind staying with 6.1, but I DO want to do everything with the > command line. > > Is the ADD syntax the same, in that I can simply specify the path and > it will recurse the whole directory tree? > > Thanks!!! > > David > > DavidEmailSig <mailto:david at leighweb.com> > > *From:*Andreas Weiler [mailto:andreas.weiler at uni-konstanz.de] > *Sent:* Tuesday, December 28, 2010 01:29 PM > *To:* david at leighweb.com <mailto:david at leighweb.com> > *Cc:* basex-talk at mailman.uni-konstanz.de > <mailto:basex-talk at mailman.uni-konstanz.de> > *Subject:* Re: [basex-talk] Database creation from a directory tree > > Additional question: That was in the GUI. I'd like to do the same > thing using the command line with the .JAR file, but somewhere my > syntax is not right. Additionally, all the XML files have NO > EXTENSION so I don't know how to specify that either in the > command line. So, any help in that area would also be appreciated. > > > Could you provide your syntax? > > The syntax should be: > create db dbname path/to/folder > > The missing extension shouldnt be a problem, i just tested it. > > > > > > In the GUI in 6.1, I can create a database from a whole directory > hierarchy of 5,681 individual XML files. When I point to the same > source file directory and using the 6.3.4 GUI (which I installed with > a pre-release version of the installer), it only processes 14 files. > No error message (that I saw) was produced. The behavior that I'm > wanting is what 6.1 does. Glad to help in solving the problem. > > We will check for modifications between the releases. As quick > workaround you could try to create an empty database and then add the > documents with the Database/Add documents menu. > > Kind regards, > Andreas > > Am 28.12.10 12:55, schrieb David leigh: > > I've just installed 6.3.4 and I've noticed a difference from version 6.1. > > In the GUI in 6.1, I can create a database from a whole directory > hierarchy of 5,681 individual XML files. When I point to the same > source file directory and using the 6.3.4 GUI (which I installed with > a pre-release version of the installer), it only processes 14 files. > No error message (that I saw) was produced. The behavior that I'm > wanting is what 6.1 does. Glad to help in solving the problem. > > Additional question: That was in the GUI. I'd like to do the same > thing using the command line with the .JAR file, but somewhere my > syntax is not right. Additionally, all the XML files have NO > EXTENSION so I don't know how to specify that either in the command > line. So, any help in that area would also be appreciated. > > Thank you, > > David > > DavidEmailSig <mailto:david at leighweb.com> > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de <mailto:BaseX-Talk at mailman.uni-konstanz.de> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >