BaseX performance issue

> We noticed that when # of documents increased in BaseX database, it will > double time to retrieve documents in database. > Our test cases execution time double when # of documents increased from 200 > to 400. And If number of document reach to 5000, > The test case execution time increase much more. > > Is that any performance testing BaseX team can share with us to compare, or > there is any documentation or suggestion for BaseX database turning that we > should follow? > > > Thanks > > Yvonne > > >

On 6/21/16, 1:44 PM, "Christian Grün" <christian.gruen at gmail.com> wrote: >Hi Yvonne, > >could you give us some hints how you retrieve documents? Do you use >one of the existing APIs, db:open, or any other functions? > >TIA >Christian > > >> We noticed that when # of documents increased in BaseX database, it will >> double time to retrieve documents in database. >> Our test cases execution time double when # of documents increased from 200 >> to 400. And If number of document reach to 5000, >> The test case execution time increase much more. >> >> Is that any performance testing BaseX team can share with us to compare, or >> there is any documentation or suggestion for BaseX database turning that we >> should follow? >> >> >> Thanks >> >> Yvonne >> >> >>

> Currently all document we installed is in binary format. > So we use bin:decode-String(db:retrieve(‘schema’, path)

> > > > Thanks > > Yvonne > > > > > On 6/21/16, 1:44 PM, "Christian Grün" <christian.gruen at gmail.com> wrote: > >>Hi Yvonne, >> >>could you give us some hints how you retrieve documents? Do you use >>one of the existing APIs, db:open, or any other functions? >> >>TIA >>Christian >> >> >>> We noticed that when # of documents increased in BaseX database, it will >>> double time to retrieve documents in database. >>> Our test cases execution time double when # of documents increased from 200 >>> to 400. And If number of document reach to 5000, >>> The test case execution time increase much more. >>> >>> Is that any performance testing BaseX team can share with us to compare, or >>> there is any documentation or suggestion for BaseX database turning that we >>> should follow? >>> >>> >>> Thanks >>> >>> Yvonne >>> >>> >>>

On 6/21/16, 11:41 PM, "Christian Grün" <christian.gruen at gmail.com> wrote: >> Currently all document we installed is in binary format. >> So we use bin:decode-String(db:retrieve(‘schema’, path) > >I see. In that case, the degradation is due to your filesystem (find >some details in [1]). You could either try to run BaseX on a partition >with a file system that is capable of handling a larger amount of >files, or you can distribute your files in different sub-paths. > > 0/1/file01.txt > 1/2/file12.txt > >Out of interest: BaseX is particularly good at handling XML >structures. Why do you store everything are binary data? > >[1] http://docs.basex.org/wiki/Binary_Data > > >> >> >> >> Thanks >> >> Yvonne >> >> >> >> >> On 6/21/16, 1:44 PM, "Christian Grün" <christian.gruen at gmail.com> wrote: >> >>>Hi Yvonne, >>> >>>could you give us some hints how you retrieve documents? Do you use >>>one of the existing APIs, db:open, or any other functions? >>> >>>TIA >>>Christian >>> >>> >>>> We noticed that when # of documents increased in BaseX database, it will >>>> double time to retrieve documents in database. >>>> Our test cases execution time double when # of documents increased from 200 >>>> to 400. And If number of document reach to 5000, >>>> The test case execution time increase much more. >>>> >>>> Is that any performance testing BaseX team can share with us to compare, or >>>> there is any documentation or suggestion for BaseX database turning that we >>>> should follow? >>>> >>>> >>>> Thanks >>>> >>>> Yvonne >>>> >>>> >>>>

> Actually, they are json format files, during testing, we only use very small size of .json format files, > But our test cases including insert, update, replace, get and delete using bases db functions.

On Wed, Jun 22, 2016 at 7:17 PM, Wang, Yvonne <yvonne.wang at sap.com> wrote: > Actually, they are json format files, during testing, we only use very small size of .json format files, > But our test cases including insert, update, replace, get and delete using bases db functions. > > > Thanks > > Yvonne > > > > On 6/21/16, 11:41 PM, "Christian Grün" <christian.gruen at gmail.com> wrote: > >>> Currently all document we installed is in binary format. >>> So we use bin:decode-String(db:retrieve(‘schema’, path) >> >>I see. In that case, the degradation is due to your filesystem (find >>some details in [1]). You could either try to run BaseX on a partition >>with a file system that is capable of handling a larger amount of >>files, or you can distribute your files in different sub-paths. >> >> 0/1/file01.txt >> 1/2/file12.txt >> >>Out of interest: BaseX is particularly good at handling XML >>structures. Why do you store everything are binary data? >> >>[1] http://docs.basex.org/wiki/Binary_Data >> >> >>> >>> >>> >>> Thanks >>> >>> Yvonne >>> >>> >>> >>> >>> On 6/21/16, 1:44 PM, "Christian Grün" <christian.gruen at gmail.com> wrote: >>> >>>>Hi Yvonne, >>>> >>>>could you give us some hints how you retrieve documents? Do you use >>>>one of the existing APIs, db:open, or any other functions? >>>> >>>>TIA >>>>Christian >>>> >>>> >>>>> We noticed that when # of documents increased in BaseX database, it will >>>>> double time to retrieve documents in database. >>>>> Our test cases execution time double when # of documents increased from 200 >>>>> to 400. And If number of document reach to 5000, >>>>> The test case execution time increase much more. >>>>> >>>>> Is that any performance testing BaseX team can share with us to compare, or >>>>> there is any documentation or suggestion for BaseX database turning that we >>>>> should follow? >>>>> >>>>> >>>>> Thanks >>>>> >>>>> Yvonne >>>>> >>>>> >>>>>