HTML serialization error
Wednesday, 27 November 2013 · 3 messages · last reply 28 Nov 2013
LJ
Lars Johnsen27 Nov 2013 16:01 #
When converting files from xml to html, there appeared a serialization error saying something to the effect that x84 was an illegal html character. The files were written using
file:write with parameter $params defined as: let $params := <output:serialization-parameters><output:method
value="html"/></
output:serialization-parameters>When sending the html directly to the browser (not writing to file and using the above declaration), the browser (chrome) appeared to be ok, and displayed the full html.
Processing the text through normalize-unicode() didn't help. The error persisted. Is there a way to fix the text before submitting it to
file:write()?
All the best
Lars G Johnsen
National Library of NorwayCG
Christian Grün28 Nov 2013 00:11 #
Hi Lars,
Quoted text · Lars Johnsen · 3 lines
> When converting files from xml to html, there appeared a serialization error > saying something to the effect that x84 was an illegal html character. The > files were written using file:write with parameter $params defined as:
Do you have some idea how the x84 byte was stored into the database?
Quoted text · Lars Johnsen · 2 lines
> Is there a way to fix the text before submitting it to > file:write()?
There are probably several ways to do this, but one standard XQuery solution I just got in mind looks as follows:
let $invalid := 132
let $valid := string-to-codepoints("?")
for $text in db:open('db')//text()
let $cps := string-to-codepoints($string) !
(if (. eq $invalid) then $valid else .)
let $new := codepoints-to-string($cps)
return $textAll strings are converted to their codepoints, and the invalid codes are replaced with an alternative (here: ?). The text are then returned as result.
The following query will replace all texts in the database..
let $invalid := 132
let $valid := string-to-codepoints("?")
for $text in db:open('db')//text()
let $cps := string-to-codepoints($string) !
(if (. eq $invalid) then $valid else .)
let $new := codepoints-to-string($cps)
return replace value of node $text with $new..and the last one replaces the texts in the main memory
representation of the document:
copy $db := db:open('db')
modify (
let $invalid := 132
let $valid := string-to-codepoints("?")
for $text in $db//text()
let $cps := string-to-codepoints($string) !
(if (. eq $invalid) then $valid else .)
let $new := codepoints-to-string($cps)
return replace value of node $text with $new
)
return $dbHope this helps,
Christian
LJ
Lars Johnsen28 Nov 2013 08:41 #
Hi Christian and thanks for the quick solution!
Quoted text · Christian Grün · 2 lines
> Do you have some idea how the x84 byte was stored into the database? >
Each file is a digitized book, where the bytes comes stem the OCR-process. The words themselves are stored as values of attributes, for example <STRING CONTENT="word">.
All the best
Lars
Written for BaseX 7.7. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery