Bug creating full-text index
Wednesday, 6 May 2020 · 3 messages
CG
Hi Sebastian,
I am afraid the input size seems to be too large for a full-text index. I will check if we can improve the error feedback (usually, you should get an info message that indicates that your data exceeds the maximum size for an index instance).
If you don’t plan to query all texts of your document, you can reduce memory consumption by restricting your index to specific elements (see [1] for more information).
Did you successfully manage to create your database without full-text index?
Saludos (y buena salud) a Argentina,
Christian
[1] https://docs.basex.org/wiki/Indexes#Selective_Indexing
Quoted text · Sebastian Guerrero · 38 lines
On Wed, May 6, 2020 at 1:18 PM Sebastian Guerrero <chapeti at gmail.com> wrote: > > Hello from Argentina!!!, thanks for the excellent software, I'm so happy learning a bit more about BaseX! > > Sadly I get an error trying to create a full-text index for a 51GB database: > > Improper use? Potential bug? Your feedback is welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 9.3.2 > Java: Oracle Corporation, 1.8.0_231 > OS: Windows 10, amd64 > Stack Trace: > java.lang.NegativeArraySizeException > at org.basex.index.ft.FTList.next(FTList.java:93) > at org.basex.index.ft.FTBuilder.merge(FTBuilder.java:239) > at org.basex.index.ft.FTBuilder.write(FTBuilder.java:147) > at org.basex.index.ft.FTBuilder.build(FTBuilder.java:86) > at org.basex.index.ft.FTBuilder.build(FTBuilder.java:23) > at org.basex.data.DiskData.createIndex(DiskData.java:198) > at org.basex.core.cmd.CreateIndex.create(CreateIndex.java:100) > at org.basex.core.cmd.Optimize.optimize(Optimize.java:181) > at org.basex.core.cmd.Optimize.optimize(Optimize.java:163) > at org.basex.core.cmd.Optimize.optimize(Optimize.java:92) > at org.basex.core.cmd.Optimize.finish(Optimize.java:82) > at org.basex.core.cmd.ACreate.update(ACreate.java:96) > at org.basex.core.cmd.CreateIndex.run(CreateIndex.java:64) > at org.basex.core.Command.run(Command.java:257) > at org.basex.core.Command.execute(Command.java:93) > at org.basex.core.Command.execute(Command.java:116) > at org.basex.gui.dialog.DialogProgress.lambda$execute$0(DialogProgress.java:182) > at java.lang.Thread.run(Unknown Source) > > I'm using BaseX GUI. > > Is it a known problem? any advice? > > Regards, > Sebastian.
SG
Sebastian Guerrero6 May 2020 15:18 #
Hello from Argentina!!!, thanks for the excellent software, I'm so happy learning a bit more about BaseX!
Sadly I get an error trying to create a full-text index for a 51GB database:
*Improper use? Potential bug? Your feedback is welcome:Contact: basex-talk at mailman.uni-konstanz.de
<basex-talk at mailman.uni-konstanz.de>Version: BaseX 9.3.2Java: Oracle Corporation, 1.8.0_231OS: Windows 10, amd64Stack Trace:
java.lang.NegativeArraySizeException at
org.basex.index.ft.FTList.next(FTList.java:93) at
org.basex.index.ft.FTBuilder.merge(FTBuilder.java:239) at
org.basex.index.ft.FTBuilder.write(FTBuilder.java:147) at
org.basex.index.ft.FTBuilder.build(FTBuilder.java:86) at
org.basex.index.ft.FTBuilder.build(FTBuilder.java:23) at
org.basex.data.DiskData.createIndex(DiskData.java:198) at
org.basex.core.cmd.CreateIndex.create(CreateIndex.java:100) at org.basex.core.cmd.Optimize.optimize(Optimize.java:181) at
org.basex.core.cmd.Optimize.optimize(Optimize.java:163) at
org.basex.core.cmd.Optimize.optimize(Optimize.java:92) at
org.basex.core.cmd.Optimize.finish(Optimize.java:82) at
org.basex.core.cmd.ACreate.update(ACreate.java:96) at
org.basex.core.cmd.CreateIndex.run(CreateIndex.java:64) at
org.basex.core.Command.run(Command.java:257) at
org.basex.core.Command.execute(Command.java:93) at
org.basex.core.Command.execute(Command.java:116) at
org.basex.gui.dialog.DialogProgress.lambda$execute$0(DialogProgress.java:182) at java.lang.Thread.run(Unknown Source)*
I'm using BaseX GUI.
Is it a known problem? any advice?
Regards,
Sebastian.
SG
Hi Christian,
Thank you very much for your answer.
You were right: I've added a field named "GoodsServicesDescription" into full-index text and that was the cause of the error.
I removed it and the full-index now is created like a charm. And yes: searching by that field is not a requirement for now.
BTW, we are evaluating BaseX to work with the UKIPO trademarks database ( several millions of records ), and so far it looks amazing.
I'll let you know how it goes.
Grüße aus dem Land der Gauchos nach Deutschland!
Sebastian.
Quoted text · Christian Grün · 66 lines
On Wed, May 6, 2020 at 8:48 AM Christian Grün <christian.gruen at gmail.com> wrote: > Hi Sebastian, > > I am afraid the input size seems to be too large for a full-text > index. I will check if we can improve the error feedback (usually, you > should get an info message that indicates that your data exceeds the > maximum size for an index instance). > > If you don’t plan to query all texts of your document, you can reduce > memory consumption by restricting your index to specific elements (see > [1] for more information). > > Did you successfully manage to create your database without full-text > index? > > Saludos (y buena salud) a Argentina, > Christian > > [1] https://docs.basex.org/wiki/Indexes#Selective_Indexing > > > > On Wed, May 6, 2020 at 1:18 PM Sebastian Guerrero <chapeti at gmail.com> > wrote: > > > > Hello from Argentina!!!, thanks for the excellent software, I'm so happy > learning a bit more about BaseX! > > > > Sadly I get an error trying to create a full-text index for a 51GB > database: > > > > Improper use? Potential bug? Your feedback is welcome: > > Contact: basex-talk at mailman.uni-konstanz.de > > Version: BaseX 9.3.2 > > Java: Oracle Corporation, 1.8.0_231 > > OS: Windows 10, amd64 > > Stack Trace: > > java.lang.NegativeArraySizeException > > at org.basex.index.ft.FTList.next(FTList.java:93) > > at org.basex.index.ft.FTBuilder.merge(FTBuilder.java:239) > > at org.basex.index.ft.FTBuilder.write(FTBuilder.java:147) > > at org.basex.index.ft.FTBuilder.build(FTBuilder.java:86) > > at org.basex.index.ft.FTBuilder.build(FTBuilder.java:23) > > at org.basex.data.DiskData.createIndex(DiskData.java:198) > > at org.basex.core.cmd.CreateIndex.create(CreateIndex.java:100) > > at org.basex.core.cmd.Optimize.optimize(Optimize.java:181) > > at org.basex.core.cmd.Optimize.optimize(Optimize.java:163) > > at org.basex.core.cmd.Optimize.optimize(Optimize.java:92) > > at org.basex.core.cmd.Optimize.finish(Optimize.java:82) > > at org.basex.core.cmd.ACreate.update(ACreate.java:96) > > at org.basex.core.cmd.CreateIndex.run(CreateIndex.java:64) > > at org.basex.core.Command.run(Command.java:257) > > at org.basex.core.Command.execute(Command.java:93) > > at org.basex.core.Command.execute(Command.java:116) > > at > org.basex.gui.dialog.DialogProgress.lambda$execute$0(DialogProgress.java:182) > > at java.lang.Thread.run(Unknown Source) > > > > I'm using BaseX GUI. > > > > Is it a known problem? any advice? > > > > Regards, > > Sebastian. >
Written for BaseX 9.3. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery