Bureaucrats, editor, reviewer, Administrators
13,550
edits
Changes
Jump to navigation
Jump to search
With If [http://vrici.lojban.org/~cowan/XML/tagsoup/ TagSoup] is found in the [[Startup#Distributions|classpath]], HTML can be imported in BaseX without any problems. TagSoup ensures that only well-formed HTML arrives at the XML parser (correct opening and closing tags, etc.). Hence, if If TagSoup is not available on a system, there the default XML parser will be a lot of cases where importing HTML failsused. (Only) if the input is well-formed XML, no matter whether you use the GUI or the standalone modeimport will succeed.
no edit summary
BaseX provides two alternatives for parsing XML:
* By default, Java’s [httphttps://downloaddocs.oracle.com/en/java/javase/611/docs/api/java.xml/javax/xml/parsers/SAXParser.html SAXParser] is used to parse XML documents.
* The internal, built-in XML parser is more fault-tolerant than Java’s XML parser. It supports standard HTML entities out of the box, and it is faster than the default parser, in particular if small documents are to be parsed. However, the internal parser does not support the full range of DTD features and cannot resolve [[Catalog_Resolver|catalogs]].
To turn the internal XML parser and DTD parsing on/off, modify the <code>INTPARSE</code> and <code>DTD</code> options:
SET [[Options#INTPARSE{{Option|INTPARSE]] }} true SET [[Options#DTD{{Option|DTD]] }} true
==XQuery==
The [[Database Module#db:add|db:add]] and [[Database Module#db:replace|db:replace]] functions can also be used to add new XML documents to the database. The following example query uses the internal XML parser and adds all files to the database <code>DB</code> that are found in the directory <code>2Bimported</code>:
<pre classsyntaxhighlight lang="brush:xquery">
for $file in file:list("2Bimported")
return db:add('DB', $file, '', map { 'intparse': true() })
</presyntaxhighlight>
=HTML Parser=
==Installation==
An easy way to add TagSoup to your project is to follow this steps:
1. visit Visit [httphttps://mvnrepository.com/artifact/org.ccil.cowan.tagsoup/tagsoup/ MVN TagSoup Repository]
2. click Click on the version you want
3. on On the first tab, you can see an XML snippet like this:
<pre classsyntaxhighlight lang="brush:xml">
<dependency>
<groupId>org.ccil.cowan.tagsoup</groupId>
<version>1.2.1</version>
</dependency>
</presyntaxhighlight>
4. copy that in your own maven project’s <code>pom.xml</code> file into the <code><dependencies></code> element.
TagSoup offers a variety of options to customize the HTML conversion. For the complete list
please visit the [http://homevrici.ccillojban.org/~cowan/XML/tagsoup/#program TagSoup] website. BaseX supports
most of these options with a few exceptions:
Turn on the HTML Parser before parsing documents, and set a file filter:
SET [[Options#PARSER{{Option|PARSER]] }} html SET [[Options#HTMLPARSER{{Option|HTMLPARSER]] }} method=xml,nons=true,nocdata=true,nodefaults=true,nobogons=true,nocolons=true,ignorable=true SET [[Options#CREATEFILTER{{Option|CREATEFILTER]] }} *.html
===XQuery===
as function arguments:
<pre classsyntaxhighlight lang="brush:xquery">
fetch:xml("index.html", map {
'parser': 'html',
'htmlparser': map { 'html': false(), 'nodefaults': true() }
})
</presyntaxhighlight>
=JSON Parser=
Turn on the JSON Parser before parsing documents, and set some optional, parser-specific options and a file filter:
SET [[Options#PARSER{{Option|PARSER]] }} json SET [[Options#JSONPARSER{{Option|JSONPARSER]] }} encoding=utf-8, jsonml=true SET [[Options#CREATEFILTER{{Option|CREATEFILTER]] }} *.json
==XQuery==
Turn on the CSV Parser before parsing documents, and set some optional, parser-specific options and a file filter. Unicode code points can be specified as separators; {{Code|32}} is the code point for spaces:
SET [[Options#PARSER{{Option|PARSER]] }} csv SET [[Options#CSVPARSER{{Option|CSVPARSER]] }} encoding=utf-8, lines=true, header=false, separator=space SET [[Options#CREATEFILTER{{Option|CREATEFILTER]] }} *.csv
==XQuery==
the first lines as column headers:
<pre classsyntaxhighlight lang="brush:xquery">
for $file in file:list("2Bimported", false(), "*.csv")
return db:add("DB", $file, "", map {
'csvparser': map { 'header': true() }
})
</presyntaxhighlight>
=Text Parser=
Turn on the CSV Parser before parsing documents and set some optional, parser-specific options and a file filter:
SET [[Options#PARSER{{Option|PARSER]] }} text SET [[Options#TEXTPARSER{{Option|TEXTPARSER]] }} lines=yes SET [[Options#CREATEFILTER{{Option|CREATEFILTER]] }} *
==XQuery==
Similar to the other formats, the text parser can also be specified via XQuery:
<pre classsyntaxhighlight lang="brush:xquery">
for $file in file:list("2Bimported", true(), "*.txt")
return db:add("DB", $file, "", map { 'parser': 'text' })
</presyntaxhighlight>
=Changelog=
