Difference between revisions of "Parsers"
Line 16: | Line 16: | ||
===Command Line=== | ===Command Line=== | ||
− | To | + | To turn the internal XML parser and DTD parsing on/off, modfify the <code>INTPARSE</code> and <code>DTD</code> options: |
SET [[Options#INTPARSE|INTPARSE]] ON | SET [[Options#INTPARSE|INTPARSE]] ON |
Revision as of 00:59, 23 September 2011
This article is part of the Advanced User's Guide. It presents various parsers for importing various data source into BaseX databases.
Contents
XML Parsers
BaseX provides two parsers to import XML data:
- by default, the standard Java SAXParser is used to parse the input XML documents. Some documents may not be fully processed by the default parser. This may e.g. be the case if they are too large or contain too many or incorrect entities.
- the internal, built-in XML parser of BaseX is more fault-tolerant than the default parser, and supports all standard HTML entities out-of-the-box. In turn, it does not support all peculiarities introduced by DTDs.
GUI
Go to Menu Database → New, then choose the Parsing tab and (de)activate Use internal XML parser. The parsing of DTDs can be turned on/off by selecting the checkbox below.
Command Line
To turn the internal XML parser and DTD parsing on/off, modfify the INTPARSE
and DTD
options:
SET INTPARSE ON SET DTD ON
XQuery
The db:add() or db:replace()
function can be used as well to add new XML documents to the database.
The following example query adds all files to the database DB
that are found in the directory 2Bimported
:
for $file in file:list("2Bimported") return db:add('DB', $file)
CSV Parser
BaseX can be used to import CSV documents. Different alternatives how to proceed are shown in the following:
GUI
Go to Menu Database → New, choose the Parsing tab and activate "csv" in the input format combobox. You can set the following options for parsing CSV documents:
- Header: Activate this option if the incoming CSV files have a header line.
- Column Separator: Choose the column separator of the CSV file (possible:
comma
,semicolon
,tab
). - XML format: Choose the XML format (possible:
verbose
,simple
).
Command Line
Turn on the CSV Parser before parsing documents, and set some optional, parser-specific options:
SET PARSER CSV SET PARSEROPT encoding=utf-8, lines=true, format=verbose, header=false, separator=comma
XQuery
The CSV parser can also be specified in the prolog of an XQuery expression.
The db:add() or db:replace()
function can be used to add the specified source files into the database.
The following example query adds all CSV files to the database DB
that are found in the directory 2Bimported
, and interprets the
first lines as column headers:
declare option db:parser "csv"; declare option db:parseropt "header=yes"; for $file in file:list("2Bimported", false(), "*.csv") return db:add('DB', $file)
Text Parser
Plain text can be imported as well:
GUI
Go to Menu Database → New, choose the Parsing tab and activate "text" in the input format combobox. You can set the following option for parsing text documents:
- Lines: Activate this option to create a
<line>...</line>
element for each line of the input text file.
Command Line
Turn on the CSV Parser before parsing documents and set some optional, parser-specific options:
SET PARSER text SET PARSEROPT lines=yes
XQuery
Again, the text parser can also be specified in the prolog of an XQuery expression, and
the db:add() or db:replace()
function can be used to add the specified source files into the database.
The following example query adds all text files to the database DB
that are found in the directory 2Bimported
and its sub-directories:
declare option db:parser "text"; for $file in file:list("2Bimported", true(), "*.txt") return db:add('DB', $file)