Builds full-text information.
Monday, 26 September 2011 · 2 messages
TH
Toshio HIRAI26 Sep 2011 09:41 #
Hi.
Although I was trying implements of Japanese Lexer to 6.8 beta, I have noticed that the result of a
ft:mark() function shifts.
55 line of
https://github.com/BaseXdb/basex/blob/master/src/main/java/org/basex/query/util/DataFTBuilder.java
final FTLexer lex = new FTLexer(ftopt).sc().init(nd.atom());
FTLexer will generate the tokenizer of the default language instead of an original language, if it does not perform in this way.
How is it?
In code which I am correcting, ft:mark() operated normally by carrying out like this.
thanks.--
Toshio HIRAI
CG
Christian Grün26 Sep 2011 10:26 #
Toshio HIRAI,
thanks for developing the Japanese Lexer for BaseX! I agree that the code snippet in question seems to lack a reference to the original fulltext options (it could be passed on to the DataFTBuilder constructor as well). I can have a second look at this.
Have you already been successful with 6.7.1, or is it the first time you're adapting the GUI visualization to your tokenizer?
I've added a GitHub issue on that..
https://github.com/BaseXdb/basex/issues/208
..and I'll keep you updated,
Christian
Quoted text · Toshio HIRAI · 23 lines
On Mon, Sep 26, 2011 at 9:41 AM, Toshio HIRAI <toshio.hirai at gmail.com> wrote: > Hi. > Although I was trying implements of Japanese Lexer to 6.8 beta, I have > noticed that the result of a ft:mark() function shifts. > 55 line of > https://github.com/BaseXdb/basex/blob/master/src/main/java/org/basex/query/util/DataFTBuilder.java > > final FTLexer lex = new FTLexer(ftopt).sc().init(nd.atom()); > FTLexer will generate the tokenizer of the default language instead of an > original language, if it does not perform in this way. > How is it? > In code which I am correcting, ft:mark() operated normally by carrying out > like this. > thanks. > -- > Toshio HIRAI > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >
Written for BaseX 6.7. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery