Replace command long execution time
Hello, The size of my database is about 23GB and about 650,000 records. The execution of the REPLACE command can take about 10 minutes.
What could be the reason?
Hi Andrey,
How long does it take to run a query against the documents you are replacing?
db:open('database', '/path/to/your/doc.xml')Best, Christian
Quoted text · Andrey Fedosenko · 7 lines
On Sat, Apr 14, 2018 at 10:34 AM, Andrey Fedosenko <andrey.fedosenko at startmatter.com> wrote: > Hello, > The size of my database is about 23GB and about 650,000 records. > The execution of the REPLACE command can take about 10 minutes. > > What could be the reason?
Hi Christian,
Query executed in 283.27 ms.
2018-04-14 11:37 GMT+03:00 Christian Grün <christian.gruen at gmail.com>:
Quoted text · Christian Grün · 17 lines
> Hi Andrey, > > How long does it take to run a query against the documents you are replacing? > > db:open('database', '/path/to/your/doc.xml') > > Best, > Christian > > > On Sat, Apr 14, 2018 at 10:34 AM, Andrey Fedosenko > <andrey.fedosenko at startmatter.com> wrote: >> Hello, >> The size of my database is about 23GB and about 650,000 records. >> The execution of the REPLACE command can take about 10 minutes. >> >> What could be the reason?
Hi Andrey,
Thanks for the quick feedback. So the document index seems to be fast enough, and it seems to be the updating process. You could try to distribute your documents in several smaller databases.
However, I am still surprised by a such a long delay. Is it due to an older hard disk, or do you even utilize SSDs?
Hope this helps, Christian
Quoted text · Andrey Fedosenko · 24 lines
On Sat, Apr 14, 2018 at 10:48 AM, Andrey Fedosenko <andrey.fedosenko at startmatter.com> wrote: > Hi Christian, > > Query executed in 283.27 ms. > > 2018-04-14 11:37 GMT+03:00 Christian Grün <christian.gruen at gmail.com>: >> Hi Andrey, >> >> How long does it take to run a query against the documents you are replacing? >> >> db:open('database', '/path/to/your/doc.xml') >> >> Best, >> Christian >> >> >> On Sat, Apr 14, 2018 at 10:34 AM, Andrey Fedosenko >> <andrey.fedosenko at startmatter.com> wrote: >>> Hello, >>> The size of my database is about 23GB and about 650,000 records. >>> The execution of the REPLACE command can take about 10 minutes. >>> >>> What could be the reason?
The my database is inside Amazons "Elastic Container Service". There was never a problem with the speed of the hard drive.
2018-04-14 11:52 GMT+03:00 Christian Grün <christian.gruen at gmail.com>:
Quoted text · Christian Grün · 38 lines
> Hi Andrey, > > Thanks for the quick feedback. So the document index seems to be fast > enough, and it seems to be the updating process. You could try to > distribute your documents in several smaller databases. > > However, I am still surprised by a such a long delay. Is it due to an > older hard disk, or do you even utilize SSDs? > > Hope this helps, > Christian > > > > On Sat, Apr 14, 2018 at 10:48 AM, Andrey Fedosenko > <andrey.fedosenko at startmatter.com> wrote: >> Hi Christian, >> >> Query executed in 283.27 ms. >> >> 2018-04-14 11:37 GMT+03:00 Christian Grün <christian.gruen at gmail.com>: >>> Hi Andrey, >>> >>> How long does it take to run a query against the documents you are replacing? >>> >>> db:open('database', '/path/to/your/doc.xml') >>> >>> Best, >>> Christian >>> >>> >>> On Sat, Apr 14, 2018 at 10:34 AM, Andrey Fedosenko >>> <andrey.fedosenko at startmatter.com> wrote: >>>> Hello, >>>> The size of my database is about 23GB and about 650,000 records. >>>> The execution of the REPLACE command can take about 10 minutes. >>>> >>>> What could be the reason?
Quoted text · Andrey Fedosenko · 2 lines
> The my database is inside Amazons "Elastic Container Service". There > was never a problem with the speed of the hard drive.
Good to know. It would be interesting, though, to see the difference to a local database instance. If you have a chance to do some more comparisons, feel free to give me some more feedback.
Quoted text · Andrey Fedosenko · 39 lines
> 2018-04-14 11:52 GMT+03:00 Christian Grün <christian.gruen at gmail.com>: >> Hi Andrey, >> >> Thanks for the quick feedback. So the document index seems to be fast >> enough, and it seems to be the updating process. You could try to >> distribute your documents in several smaller databases. >> >> However, I am still surprised by a such a long delay. Is it due to an >> older hard disk, or do you even utilize SSDs? >> >> Hope this helps, >> Christian >> >> >> >> On Sat, Apr 14, 2018 at 10:48 AM, Andrey Fedosenko >> <andrey.fedosenko at startmatter.com> wrote: >>> Hi Christian, >>> >>> Query executed in 283.27 ms. >>> >>> 2018-04-14 11:37 GMT+03:00 Christian Grün <christian.gruen at gmail.com>: >>>> Hi Andrey, >>>> >>>> How long does it take to run a query against the documents you are replacing? >>>> >>>> db:open('database', '/path/to/your/doc.xml') >>>> >>>> Best, >>>> Christian >>>> >>>> >>>> On Sat, Apr 14, 2018 at 10:34 AM, Andrey Fedosenko >>>> <andrey.fedosenko at startmatter.com> wrote: >>>>> Hello, >>>>> The size of my database is about 23GB and about 650,000 records. >>>>> The execution of the REPLACE command can take about 10 minutes. >>>>> >>>>> What could be the reason?
Written for BaseX 9.0. BaseX may have changed since: please check the current documentation.