ft:normalize: potential bug?

On Thu, Jun 17, 2021 at 6:21 PM Alexander Witzigmann <alexander.witzigmann at tanner.de> wrote: > > hi, > > if you call ft:normalize using a string containing "*" > ft:normalize("a*",map { 'stemming': 'true', 'language':'de' }) > > this results in > > Improper use? Potential bug? Your feedback is welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 9.5.2 > Java: Oracle Corporation, 13.0.2 > OS: Windows 10, amd64 > Stack Trace: > java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 8 > at org.basex.util.TokenBuilder.get(TokenBuilder.java:191) > at org.basex.util.ft.GermanStemmer.resub(GermanStemmer.java:150) > at org.basex.util.ft.GermanStemmer.stem(GermanStemmer.java:42) > at org.basex.util.ft.Stemmer.nextToken(Stemmer.java:99) > at org.basex.util.ft.FTLexer.nextToken(FTLexer.java:156) > at org.basex.query.func.ft.FtTokenize.tokens(FtTokenize.java:49) > at org.basex.query.func.ft.FtNormalize.item(FtNormalize.java:23) > .... > > anything I missed? > > Alex >

Am 17.06.2021 um 18:48 schrieb Christian Grün: > Hi Alex, > > Thanks for the easily reproducible test case. The German stemmer has > been fixed, a new snapshot is available [1]. > > Greetings to the other end of the lake, > Christian > > [1] https://files.basex.org/releases/latest/ > > > On Thu, Jun 17, 2021 at 6:21 PM Alexander Witzigmann > <alexander.witzigmann at tanner.de> wrote: >> hi, >> >> if you call ft:normalize using a string containing "*" >> ft:normalize("a*",map { 'stemming': 'true', 'language':'de' }) >> >> this results in >> >> Improper use? Potential bug? Your feedback is welcome: >> Contact: basex-talk at mailman.uni-konstanz.de >> Version: BaseX 9.5.2 >> Java: Oracle Corporation, 13.0.2 >> OS: Windows 10, amd64 >> Stack Trace: >> java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 8 >> at org.basex.util.TokenBuilder.get(TokenBuilder.java:191) >> at org.basex.util.ft.GermanStemmer.resub(GermanStemmer.java:150) >> at org.basex.util.ft.GermanStemmer.stem(GermanStemmer.java:42) >> at org.basex.util.ft.Stemmer.nextToken(Stemmer.java:99) >> at org.basex.util.ft.FTLexer.nextToken(FTLexer.java:156) >> at org.basex.query.func.ft.FtTokenize.tokens(FtTokenize.java:49) >> at org.basex.query.func.ft.FtNormalize.item(FtNormalize.java:23) >> .... >> >> anything I missed? >> >> Alex >>