Changes

Jump to navigation Jump to search
12 bytes removed ,  16:30, 27 March 2015
no edit summary
Thank you to [http://blog.infinite.jp Toshio HIRAI] for integrating the lexer in BaseX!
==Introduction==
The lexical analysis of Japanese documents is performed by
* NAIST Dictionary: http://files.basex.org/etc/naistdic.zip
==Lexical Analysis==
The example sentence "私は本を書きました。(I wrote a book.)"
morpheme are used in indexing and stemming.
==Parsing==
During indexing and parsing, the input strings are split into single ''tokens''.
for each token.
==Token Processing==
"Fullwidth" and "Halfwidth" (which is defined by
[http://www.w3.org/TR/xpath-full-text-10/#ftdiacriticsoption Diacritics] Option.
==Stemming==
Stemming in Japanese means to analyze the results of morphological analysis
</pre>
==Wildcards==
The Wildcard option in XQuery Full-Text is available for Japanese as well.
Bureaucrats, editor, reviewer, Administrators
13,550

edits

Navigation menu