unicode characters missing in the output
Tuesday, 9 June 2026 · 3 messages
GP
Guillaume Porte9 Jun 2026 11:58 #
Hi,
I'm trying to figure out a character display issue when using basex.
I have a document originally created by using a unicode font called fedorovsk.otf (https://sci.ponomar.net/fonts.html)
When I open the file inside a text editor, I can see all characters and ligatures (since the font is installed on the system), and it also works in the browser when using the font with @font-face.
When I load the document with basex (either by the specifying the path or after indexing it), ir returns the word with certain characters missing.
You can see the difference here : https://gprt.fr/unicode/test.html
Apparently, the codes for the first entity are  or 
When adding those codes to the XML document, they disappear in the output.
Do you have any idea on what's going on and how to fix it?
Regards
Guillaume
GS
Graydon Saunders9 Jun 2026 14:09 #
If I try char(0xE16E) in the GUI, I get as part of the stuff in the Info window
Compiling:
- evaluate
fn:char(value): char(57710) -> ""
E16E is in the Private Use Area and https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-char says of char() that "The result must consist of permitted characters."
"Permitted characters" are "[Definition: A permitted character is one within the repertoire accepted by the implementation.]"
At least as of 12.4, it looks like BaseX doesn't include private use characters in the character repertoire. (https://docs.basex.org/main/Search?input=repertoire gives me nothing at all, so I'm guessing.)
Whether this is on purpose or not I have no idea, but it looks like that's what's happening.Quoted text · Guillaume Porte · 28 lines
On Tue, Jun 9, 2026, at 05:58, Guillaume Porte via BaseX-Talk wrote: > Hi, > > I'm trying to figure out a character display issue when using basex. > > I have a document originally created by using a unicode font called > fedorovsk.otf (https://sci.ponomar.net/fonts.html) > > When I open the file inside a text editor, I can see all characters and > ligatures (since the font is installed on the system), and it also works > in the browser when using the font with @font-face. > > When I load the document with basex (either by the specifying the path > or after indexing it), ir returns the word with certain characters missing. > > You can see the difference here : https://gprt.fr/unicode/test.html > > Apparently, the codes for the first entity are  or  > > When adding those codes to the XML document, they disappear in the output. > > Do you have any idea on what's going on and how to fix it? > > Regards > > Guillaume > >
GR
Hi Guillaume,
when you say the characters are missing, I presume you refer to the result display in the BaseX GUI.
U+E16E is in the private use area U+E000 through U+F8FF. The BaseX GUI uses code points in this range for internal purposes, so they are blocked when being displayed, if they occur in actual content. This is a limitation of the GUI implementation.
However this affects the GUI only. The BaseX engine handles code point U+E16E just like any other valid Unicode code point, it is not dropped from the document. So when you send your results somewhere else than to GUI result panes, e.g. to files or REST responses, the affected code points should be present as expected.
Hope this helps.
Best regards,
Gunther
Gesendet: Dienstag, 9. Juni 2026 um 11:58
Quoted text · Guillaume Porte · 18 lines
Von: "Guillaume Porte via BaseX-Talk" <basex-talk at mailman.uni-konstanz.de> An: basex-talk at mailman.uni-konstanz.de Betreff: [basex-talk] unicode characters missing in the output Hi, I'm trying to figure out a character display issue when using basex. I have a document originally created by using a unicode font called fedorovsk.otf (https://sci.ponomar.net/fonts.html) When I open the file inside a text editor, I can see all characters and ligatures (since the font is installed on the system), and it also works in the browser when using the font with @font-face. When I load the document with basex (either by the specifying the path or after indexing it), ir returns the word with certain characters missing. You can see the difference here : https://gprt.fr/unicode/test.html Apparently, the codes for the first entity are  or  When adding those codes to the XML document, they disappear in the output. Do you have any idea on what's going on and how to fix it? Regards Guillaume
Written for BaseX 12.4. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery