BaseX performance 7.x vs 8.x
Thursday, 28 July 2016 · 11 messages · last reply 4 Aug 2016
BC
Bondeson, Carl28 Jul 2016 19:36 #
I am seeing huge differences in the performance of a multi-threaded JBoss application recently upgraded from 7.9 to 8.4. The problem is that 8.4 is much slower than 7.9. Any ideas? Do I need to add indices or something along those lines?
Carl R Bondeson
IT Analyst 3
Information Technology
Connecticut Department of Public Health
410 Capitol Ave
Hartford, CT 06134
Phone: 860-509-7434
carl.bondeson at ct.gov<mailto:carl.bondeson at ct.gov>
[logosmall]
BC
Bondeson, Carl28 Jul 2016 20:36 #
As a follow up for performance a few queries are listed with execution times
declare option output:item-separator "
"; for $h in //ValidationDictionary/Validation let $g := $h/Parent[@Element='PatientDefinition'] let $i := $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='Gender'] for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/> Version: 7.9 execution time: 3.82 ms
Version: 8.5.1 execution time: 138.8 ms
declare option output:item-separator "
"; for $h in //ValidationDictionary/Validation let $g := $h/Parent[@Element='PatientDefinition'] let $i := $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='FirstName'] for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/> Version: 7.9 execution time: 3.3 ms
Version: 8.5.1 execution time: 133.8 ms
Query:declare option output:item-separator "
";for $g in //ValidationDictionary//Validation let $h := $g//Parent[@Element='PatientDefinition']let $i := $h//Child[@Element='PartyDefinition'] where $g//ProductCode='ZIKA' return <BaseXValidation Parent='{$h/@Element}' Child='{$i/@Element}'/> Version: 7.9 execution time: 2.2 ms
Version: 8.5.1 execution time: 156.1 ms
These queries are called from an XSLT transformer via all callout and are performed in very type loops. I was wondering why the transforms were never completing. 40-70x decrease in performance would slow any process down.
Carl R Bondeson
IT Analyst 3
Information Technology
Connecticut Department of Public Health
410 Capitol Ave
Hartford, CT 06134
Phone: 860-509-7434
carl.bondeson at ct.gov<mailto:carl.bondeson at ct.gov>
[logosmall]
Quoted text · basex-talk-bounces at mailman.uni-konstanz.de · 14 lines
From: basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of Bondeson, Carl Sent: Thursday, July 28, 2016 1:36 PM To: basex-talk at mailman.uni-konstanz.de Subject: [basex-talk] BaseX performance 7.x vs 8.x I am seeing huge differences in the performance of a multi-threaded JBoss application recently upgraded from 7.9 to 8.4. The problem is that 8.4 is much slower than 7.9. Any ideas? Do I need to add indices or something along those lines? Carl R Bondeson IT Analyst 3 Information Technology Connecticut Department of Public Health 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson at ct.gov<mailto:carl.bondeson at ct.gov> [logosmall]
LM
Hello Carl and everyone,
very interest this question, mainly because now I'm investigating the performance of the position() XPath function in BaseX. On older versions (7.*) the access time for elements in different positions is practically constant (see [1]). However on latest versions (8.*) we detected that this time increases according we increase the position values.
P.S.: all indexes were created.
Any ideas?
[1] http://goo.gl/fnzVaX
Luiz Matos
PhD Student
Quoted text · Bondeson, Carl · 16 lines
> I am seeing huge differences in the performance of a multi-threaded JBoss > application recently upgraded from 7.9 to 8.4. The problem is that 8.4 is > much slower than 7.9. Any ideas? Do I need to add indices or something > along those lines? > > Carl R Bondeson > IT Analyst 3 > Information Technology > Connecticut Department of Public Health > 410 Capitol Ave > Hartford, CT 06134 > Phone: 860-509-7434 > carl.bondeson at ct.gov<mailto:carl.bondeson at ct.gov> > [logosmall] > >
--
Luiz Matos
PhD student at Federal Fluminense University
CG
Hi Carl, hi Luiz,
With every version of BaseX, we are adding new query optimizations (usually, this can be experienced if you run newer code with older version of BaseX). Existing optimization should of course be preserved. To be able to understand why your queries run slower than before, I would be more than glad if you could both provide me with little examples that we can try by ourselves. The Query Info output for both versions of BaseX would be helpful as well; it indicates which optimizations are performed or not.
Thanks in advance
Christian
Quoted text · Carl · 117 lines
On Thu, Jul 28, 2016 at 8:36 PM, Bondeson, Carl <Carl.Bondeson at ct.gov> wrote: > As a follow up for performance a few queries are listed with execution > times > > > > declare option output:item-separator "
"; for $h in > //ValidationDictionary/Validation let $g := > $h/Parent[@Element='PatientDefinition'] let $i := > $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='Gender'] > for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation > Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' > Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' > Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' > LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' > Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/> > > Version: 7.9 execution time: 3.82 ms > > Version: 8.5.1 execution time: 138.8 ms > > > > declare option output:item-separator "
"; for $h in > //ValidationDictionary/Validation let $g := > $h/Parent[@Element='PatientDefinition'] let $i := > $g/Child[@Element='PartyDefinition'] let $j := > $i/Item[@Attribute='FirstName'] for $k in $j/@Type where > $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' > Child='{$i/@Element}' Action='{$k/../@Action}' > Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' > Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' > LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' > Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/> > > Version: 7.9 execution time: 3.3 ms > > Version: 8.5.1 execution time: 133.8 ms > > > > Query:declare option output:item-separator "
";for $g in > //ValidationDictionary//Validation let $h := > $g//Parent[@Element='PatientDefinition']let $i := > $h//Child[@Element='PartyDefinition'] where $g//ProductCode='ZIKA' return > <BaseXValidation Parent='{$h/@Element}' Child='{$i/@Element}'/> > > Version: 7.9 execution time: 2.2 ms > > Version: 8.5.1 execution time: 156.1 ms > > > > > > These queries are called from an XSLT transformer via all callout and are > performed in very type loops. I was wondering why the transforms were never > completing. 40-70x decrease in performance would slow any process down. > > > > Carl R Bondeson > > IT Analyst 3 > > Information Technology > > Connecticut Department of Public Health > > 410 Capitol Ave > > Hartford, CT 06134 > > Phone: 860-509-7434 > > carl.bondeson at ct.gov > > [image: logosmall] > > > > *From:* basex-talk-bounces at mailman.uni-konstanz.de [mailto: > basex-talk-bounces at mailman.uni-konstanz.de] *On Behalf Of *Bondeson, Carl > *Sent:* Thursday, July 28, 2016 1:36 PM > *To:* basex-talk at mailman.uni-konstanz.de > *Subject:* [basex-talk] BaseX performance 7.x vs 8.x > > > > I am seeing huge differences in the performance of a multi-threaded JBoss > application recently upgraded from 7.9 to 8.4. The problem is that 8.4 is > much slower than 7.9. Any ideas? Do I need to add indices or something > along those lines? > > > > Carl R Bondeson > > IT Analyst 3 > > Information Technology > > Connecticut Department of Public Health > > 410 Capitol Ave > > Hartford, CT 06134 > > Phone: 860-509-7434 > > carl.bondeson at ct.gov > > [image: logosmall] > > >
BC
Bondeson, Carl29 Jul 2016 13:34 #
I have put together a set of queries, the output, and corresponding xml. The files are attached. These queries were performed using BaseX 8.5.2 on JBoss 6.4.9, and Java 8.102.
I appreciate your support.
Carl R Bondeson
IT Analyst 3
Information Technology
Connecticut Department of Public Health
410 Capitol Ave
Hartford, CT 06134
Phone: 860-509-7434
carl.bondeson at ct.gov<mailto:carl.bondeson at ct.gov>
[logosmall]
Quoted text · Christian Grün · 44 lines
From: Christian Grün [mailto:christian.gruen at gmail.com] Sent: Friday, July 29, 2016 5:07 AM To: Bondeson, Carl <Carl.Bondeson at ct.gov>; Luiz Matos <luizmatos at ic.uff.br> Cc: basex-talk at mailman.uni-konstanz.de Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x Hi Carl, hi Luiz, With every version of BaseX, we are adding new query optimizations (usually, this can be experienced if you run newer code with older version of BaseX). Existing optimization should of course be preserved. To be able to understand why your queries run slower than before, I would be more than glad if you could both provide me with little examples that we can try by ourselves. The Query Info output for both versions of BaseX would be helpful as well; it indicates which optimizations are performed or not. Thanks in advance Christian On Thu, Jul 28, 2016 at 8:36 PM, Bondeson, Carl <Carl.Bondeson at ct.gov<mailto:Carl.Bondeson at ct.gov>> wrote: As a follow up for performance a few queries are listed with execution times declare option output:item-separator "
"; for $h in //ValidationDictionary/Validation let $g := $h/Parent[@Element='PatientDefinition'] let $i := $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='Gender'] for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/> Version: 7.9 execution time: 3.82 ms Version: 8.5.1 execution time: 138.8 ms declare option output:item-separator "
"; for $h in //ValidationDictionary/Validation let $g := $h/Parent[@Element='PatientDefinition'] let $i := $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='FirstName'] for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/> Version: 7.9 execution time: 3.3 ms Version: 8.5.1 execution time: 133.8 ms Query:declare option output:item-separator "
";for $g in //ValidationDictionary//Validation let $h := $g//Parent[@Element='PatientDefinition']let $i := $h//Child[@Element='PartyDefinition'] where $g//ProductCode='ZIKA' return <BaseXValidation Parent='{$h/@Element}' Child='{$i/@Element}'/> Version: 7.9 execution time: 2.2 ms Version: 8.5.1 execution time: 156.1 ms These queries are called from an XSLT transformer via all callout and are performed in very type loops. I was wondering why the transforms were never completing. 40-70x decrease in performance would slow any process down. Carl R Bondeson IT Analyst 3 Information Technology Connecticut Department of Public Health 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson at ct.gov<mailto:carl.bondeson at ct.gov> [logosmall] From: basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de> [mailto:basex-talk-bounces at mailman.uni-konstanz.de<mailto:basex-talk-bounces at mailman.uni-konstanz.de>] On Behalf Of Bondeson, Carl Sent: Thursday, July 28, 2016 1:36 PM To: basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Subject: [basex-talk] BaseX performance 7.x vs 8.x I am seeing huge differences in the performance of a multi-threaded JBoss application recently upgraded from 7.9 to 8.4. The problem is that 8.4 is much slower than 7.9. Any ideas? Do I need to add indices or something along those lines? Carl R Bondeson IT Analyst 3 Information Technology Connecticut Department of Public Health 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson at ct.gov<mailto:carl.bondeson at ct.gov> [logosmall]
CG
Christian Grün29 Jul 2016 13:45 #
Quoted text · Bondeson, Carl · 1 line
> I have put together a set of queries, the output, and corresponding xml.
I have run the first query; it yields no results. Is this expected? If you want, you can send me some more data in private.
Could you please pass me on the query info output for both 8.5.2 and 7.9?
Quoted text · Bondeson, Carl · 132 lines
> I appreciate your support. > > > > Carl R Bondeson > > IT Analyst 3 > > Information Technology > > Connecticut Department of Public Health > > 410 Capitol Ave > > Hartford, CT 06134 > > Phone: 860-509-7434 > > carl.bondeson at ct.gov > > > > From: Christian Grün [mailto:christian.gruen at gmail.com] > Sent: Friday, July 29, 2016 5:07 AM > To: Bondeson, Carl <Carl.Bondeson at ct.gov>; Luiz Matos <luizmatos at ic.uff.br> > Cc: basex-talk at mailman.uni-konstanz.de > Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x > > > > Hi Carl, hi Luiz, > > > > With every version of BaseX, we are adding new query optimizations (usually, this can be experienced if you run newer code with older version of BaseX). Existing optimization should of course be preserved. To be able to understand why your queries run slower than before, I would be more than glad if you could both provide me with little examples that we can try by ourselves. The Query Info output for both versions of BaseX would be helpful as well; it indicates which optimizations are performed or not. > > > > Thanks in advance > > Christian > > > > > > > > On Thu, Jul 28, 2016 at 8:36 PM, Bondeson, Carl <Carl.Bondeson at ct.gov> wrote: > > As a follow up for performance a few queries are listed with execution times > > > > declare option output:item-separator "
"; for $h in //ValidationDictionary/Validation let $g := $h/Parent[@Element='PatientDefinition'] let $i := $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='Gender'] for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/> > > Version: 7.9 execution time: 3.82 ms > > Version: 8.5.1 execution time: 138.8 ms > > > > declare option output:item-separator "
"; for $h in //ValidationDictionary/Validation let $g := $h/Parent[@Element='PatientDefinition'] let $i := $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='FirstName'] for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/> > > Version: 7.9 execution time: 3.3 ms > > Version: 8.5.1 execution time: 133.8 ms > > > > Query:declare option output:item-separator "
";for $g in //ValidationDictionary//Validation let $h := $g//Parent[@Element='PatientDefinition']let $i := $h//Child[@Element='PartyDefinition'] where $g//ProductCode='ZIKA' return <BaseXValidation Parent='{$h/@Element}' Child='{$i/@Element}'/> > > Version: 7.9 execution time: 2.2 ms > > Version: 8.5.1 execution time: 156.1 ms > > > > > > These queries are called from an XSLT transformer via all callout and are performed in very type loops. I was wondering why the transforms were never completing. 40-70x decrease in performance would slow any process down. > > > > Carl R Bondeson > > IT Analyst 3 > > Information Technology > > Connecticut Department of Public Health > > 410 Capitol Ave > > Hartford, CT 06134 > > Phone: 860-509-7434 > > carl.bondeson at ct.gov > > > > From: basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of Bondeson, Carl > Sent: Thursday, July 28, 2016 1:36 PM > To: basex-talk at mailman.uni-konstanz.de > Subject: [basex-talk] BaseX performance 7.x vs 8.x > > > > I am seeing huge differences in the performance of a multi-threaded JBoss application recently upgraded from 7.9 to 8.4. The problem is that 8.4 is much slower than 7.9. Any ideas? Do I need to add indices or something along those lines? > > > > Carl R Bondeson > > IT Analyst 3 > > Information Technology > > Connecticut Department of Public Health > > 410 Capitol Ave > > Hartford, CT 06134 > > Phone: 860-509-7434 > > carl.bondeson at ct.gov > > > >
BC
Bondeson, Carl3 Aug 2016 19:00 #
I have not been able to get back to the same performance as was seen in 7.9. I attempted a change in architecture to use separate databases rather than having them combined in one. I created classes to support them, and updates to the XSLT tier to address the namespace additions. So rather than having 1 database with 14 MB of data, I now have 4 databases served by their own classes. In the end the performance was once again much worse than 7.9 with a single database. I have to look at the cost of calling up to 3 constructors in the XSLT transform, while looking at the gains, if any, or working with smaller individual databases.
I will save this branch for another day. I have too many deliverables to afford any more resources at this time.
Carl R Bondeson
IT Analyst 3
Information Technology
Connecticut Department of Public Health
410 Capitol Ave
Hartford, CT 06134
Phone: 860-509-7434
carl.bondeson at ct.gov
Quoted text · Christian Grün · 142 lines
-----Original Message----- From: Christian Grün [mailto:christian.gruen at gmail.com] Sent: Friday, July 29, 2016 7:45 AM To: Bondeson, Carl <Carl.Bondeson at ct.gov> Cc: Luiz Matos <luizmatos at ic.uff.br>; basex-talk at mailman.uni-konstanz.de Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x > I have put together a set of queries, the output, and corresponding xml. I have run the first query; it yields no results. Is this expected? If you want, you can send me some more data in private. Could you please pass me on the query info output for both 8.5.2 and 7.9? > I appreciate your support. > > > > Carl R Bondeson > > IT Analyst 3 > > Information Technology > > Connecticut Department of Public Health > > 410 Capitol Ave > > Hartford, CT 06134 > > Phone: 860-509-7434 > > carl.bondeson at ct.gov > > > > From: Christian Grün [mailto:christian.gruen at gmail.com] > Sent: Friday, July 29, 2016 5:07 AM > To: Bondeson, Carl <Carl.Bondeson at ct.gov>; Luiz Matos <luizmatos at ic.uff.br> > Cc: basex-talk at mailman.uni-konstanz.de > Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x > > > > Hi Carl, hi Luiz, > > > > With every version of BaseX, we are adding new query optimizations (usually, this can be experienced if you run newer code with older version of BaseX). Existing optimization should of course be preserved. To be able to understand why your queries run slower than before, I would be more than glad if you could both provide me with little examples that we can try by ourselves. The Query Info output for both versions of BaseX would be helpful as well; it indicates which optimizations are performed or not. > > > > Thanks in advance > > Christian > > > > > > > > On Thu, Jul 28, 2016 at 8:36 PM, Bondeson, Carl <Carl.Bondeson at ct.gov> wrote: > > As a follow up for performance a few queries are listed with execution times > > > > declare option output:item-separator "
"; for $h in //ValidationDictionary/Validation let $g := $h/Parent[@Element='PatientDefinition'] let $i := $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='Gender'] for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/> > > Version: 7.9 execution time: 3.82 ms > > Version: 8.5.1 execution time: 138.8 ms > > > > declare option output:item-separator "
"; for $h in //ValidationDictionary/Validation let $g := $h/Parent[@Element='PatientDefinition'] let $i := $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='FirstName'] for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/> > > Version: 7.9 execution time: 3.3 ms > > Version: 8.5.1 execution time: 133.8 ms > > > > Query:declare option output:item-separator "
";for $g in //ValidationDictionary//Validation let $h := $g//Parent[@Element='PatientDefinition']let $i := $h//Child[@Element='PartyDefinition'] where $g//ProductCode='ZIKA' return <BaseXValidation Parent='{$h/@Element}' Child='{$i/@Element}'/> > > Version: 7.9 execution time: 2.2 ms > > Version: 8.5.1 execution time: 156.1 ms > > > > > > These queries are called from an XSLT transformer via all callout and are performed in very type loops. I was wondering why the transforms were never completing. 40-70x decrease in performance would slow any process down. > > > > Carl R Bondeson > > IT Analyst 3 > > Information Technology > > Connecticut Department of Public Health > > 410 Capitol Ave > > Hartford, CT 06134 > > Phone: 860-509-7434 > > carl.bondeson at ct.gov > > > > From: basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of Bondeson, Carl > Sent: Thursday, July 28, 2016 1:36 PM > To: basex-talk at mailman.uni-konstanz.de > Subject: [basex-talk] BaseX performance 7.x vs 8.x > > > > I am seeing huge differences in the performance of a multi-threaded JBoss application recently upgraded from 7.9 to 8.4. The problem is that 8.4 is much slower than 7.9. Any ideas? Do I need to add indices or something along those lines? > > > > Carl R Bondeson > > IT Analyst 3 > > Information Technology > > Connecticut Department of Public Health > > 410 Capitol Ave > > Hartford, CT 06134 > > Phone: 860-509-7434 > > carl.bondeson at ct.gov > > > >
CG
Christian Grün3 Aug 2016 19:01 #
Sorry to hear that. If you can provide us your data, we could have another look.
Quoted text · Carl · 165 lines
On Wed, Aug 3, 2016 at 7:00 PM, Bondeson, Carl <Carl.Bondeson at ct.gov> wrote: > I have not been able to get back to the same performance as was seen in 7.9. I attempted a change in architecture to use separate databases rather than having them combined in one. I created classes to support them, and updates to the XSLT tier to address the namespace additions. So rather than having 1 database with 14 MB of data, I now have 4 databases served by their own classes. In the end the performance was once again much worse than 7.9 with a single database. I have to look at the cost of calling up to 3 constructors in the XSLT transform, while looking at the gains, if any, or working with smaller individual databases. > > I will save this branch for another day. I have too many deliverables to afford any more resources at this time. > > > Carl R Bondeson > IT Analyst 3 > Information Technology > Connecticut Department of Public Health > 410 Capitol Ave > Hartford, CT 06134 > Phone: 860-509-7434 > carl.bondeson at ct.gov > > > -----Original Message----- > From: Christian Grün [mailto:christian.gruen at gmail.com] > Sent: Friday, July 29, 2016 7:45 AM > To: Bondeson, Carl <Carl.Bondeson at ct.gov> > Cc: Luiz Matos <luizmatos at ic.uff.br>; basex-talk at mailman.uni-konstanz.de > Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x > >> I have put together a set of queries, the output, and corresponding xml. > > I have run the first query; it yields no results. Is this expected? > If you want, you can send me some more data in private. > > Could you please pass me on the query info output for both 8.5.2 and 7.9? > > > > >> I appreciate your support. >> >> >> >> Carl R Bondeson >> >> IT Analyst 3 >> >> Information Technology >> >> Connecticut Department of Public Health >> >> 410 Capitol Ave >> >> Hartford, CT 06134 >> >> Phone: 860-509-7434 >> >> carl.bondeson at ct.gov >> >> >> >> From: Christian Grün [mailto:christian.gruen at gmail.com] >> Sent: Friday, July 29, 2016 5:07 AM >> To: Bondeson, Carl <Carl.Bondeson at ct.gov>; Luiz Matos <luizmatos at ic.uff.br> >> Cc: basex-talk at mailman.uni-konstanz.de >> Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x >> >> >> >> Hi Carl, hi Luiz, >> >> >> >> With every version of BaseX, we are adding new query optimizations (usually, this can be experienced if you run newer code with older version of BaseX). Existing optimization should of course be preserved. To be able to understand why your queries run slower than before, I would be more than glad if you could both provide me with little examples that we can try by ourselves. The Query Info output for both versions of BaseX would be helpful as well; it indicates which optimizations are performed or not. >> >> >> >> Thanks in advance >> >> Christian >> >> >> >> >> >> >> >> On Thu, Jul 28, 2016 at 8:36 PM, Bondeson, Carl <Carl.Bondeson at ct.gov> wrote: >> >> As a follow up for performance a few queries are listed with execution times >> >> >> >> declare option output:item-separator "
"; for $h in //ValidationDictionary/Validation let $g := $h/Parent[@Element='PatientDefinition'] let $i := $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='Gender'] for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/> >> >> Version: 7.9 execution time: 3.82 ms >> >> Version: 8.5.1 execution time: 138.8 ms >> >> >> >> declare option output:item-separator "
"; for $h in //ValidationDictionary/Validation let $g := $h/Parent[@Element='PatientDefinition'] let $i := $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='FirstName'] for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/> >> >> Version: 7.9 execution time: 3.3 ms >> >> Version: 8.5.1 execution time: 133.8 ms >> >> >> >> Query:declare option output:item-separator "
";for $g in //ValidationDictionary//Validation let $h := $g//Parent[@Element='PatientDefinition']let $i := $h//Child[@Element='PartyDefinition'] where $g//ProductCode='ZIKA' return <BaseXValidation Parent='{$h/@Element}' Child='{$i/@Element}'/> >> >> Version: 7.9 execution time: 2.2 ms >> >> Version: 8.5.1 execution time: 156.1 ms >> >> >> >> >> >> These queries are called from an XSLT transformer via all callout and are performed in very type loops. I was wondering why the transforms were never completing. 40-70x decrease in performance would slow any process down. >> >> >> >> Carl R Bondeson >> >> IT Analyst 3 >> >> Information Technology >> >> Connecticut Department of Public Health >> >> 410 Capitol Ave >> >> Hartford, CT 06134 >> >> Phone: 860-509-7434 >> >> carl.bondeson at ct.gov >> >> >> >> From: basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of Bondeson, Carl >> Sent: Thursday, July 28, 2016 1:36 PM >> To: basex-talk at mailman.uni-konstanz.de >> Subject: [basex-talk] BaseX performance 7.x vs 8.x >> >> >> >> I am seeing huge differences in the performance of a multi-threaded JBoss application recently upgraded from 7.9 to 8.4. The problem is that 8.4 is much slower than 7.9. Any ideas? Do I need to add indices or something along those lines? >> >> >> >> Carl R Bondeson >> >> IT Analyst 3 >> >> Information Technology >> >> Connecticut Department of Public Health >> >> 410 Capitol Ave >> >> Hartford, CT 06134 >> >> Phone: 860-509-7434 >> >> carl.bondeson at ct.gov >> >> >> >>
BC
Bondeson, Carl3 Aug 2016 19:06 #
I am looking at creating a deliverable jar, in the not too distant future, which illustrates the issues. I just don't have the time right now. It is on my to-do list. Thanks again for your support!
Carl R Bondeson
IT Analyst 3
Information Technology
Connecticut Department of Public Health
410 Capitol Ave
Hartford, CT 06134
Phone: 860-509-7434
carl.bondeson at ct.gov
Quoted text · Christian Grün · 172 lines
-----Original Message----- From: Christian Grün [mailto:christian.gruen at gmail.com] Sent: Wednesday, August 03, 2016 1:02 PM To: Bondeson, Carl <Carl.Bondeson at ct.gov> Cc: Luiz Matos <luizmatos at ic.uff.br>; basex-talk at mailman.uni-konstanz.de Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x Sorry to hear that. If you can provide us your data, we could have another look. On Wed, Aug 3, 2016 at 7:00 PM, Bondeson, Carl <Carl.Bondeson at ct.gov> wrote: > I have not been able to get back to the same performance as was seen in 7.9. I attempted a change in architecture to use separate databases rather than having them combined in one. I created classes to support them, and updates to the XSLT tier to address the namespace additions. So rather than having 1 database with 14 MB of data, I now have 4 databases served by their own classes. In the end the performance was once again much worse than 7.9 with a single database. I have to look at the cost of calling up to 3 constructors in the XSLT transform, while looking at the gains, if any, or working with smaller individual databases. > > I will save this branch for another day. I have too many deliverables to afford any more resources at this time. > > > Carl R Bondeson > IT Analyst 3 > Information Technology > Connecticut Department of Public Health > 410 Capitol Ave > Hartford, CT 06134 > Phone: 860-509-7434 > carl.bondeson at ct.gov > > > -----Original Message----- > From: Christian Grün [mailto:christian.gruen at gmail.com] > Sent: Friday, July 29, 2016 7:45 AM > To: Bondeson, Carl <Carl.Bondeson at ct.gov> > Cc: Luiz Matos <luizmatos at ic.uff.br>; basex-talk at mailman.uni-konstanz.de > Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x > >> I have put together a set of queries, the output, and corresponding xml. > > I have run the first query; it yields no results. Is this expected? > If you want, you can send me some more data in private. > > Could you please pass me on the query info output for both 8.5.2 and 7.9? > > > > >> I appreciate your support. >> >> >> >> Carl R Bondeson >> >> IT Analyst 3 >> >> Information Technology >> >> Connecticut Department of Public Health >> >> 410 Capitol Ave >> >> Hartford, CT 06134 >> >> Phone: 860-509-7434 >> >> carl.bondeson at ct.gov >> >> >> >> From: Christian Grün [mailto:christian.gruen at gmail.com] >> Sent: Friday, July 29, 2016 5:07 AM >> To: Bondeson, Carl <Carl.Bondeson at ct.gov>; Luiz Matos <luizmatos at ic.uff.br> >> Cc: basex-talk at mailman.uni-konstanz.de >> Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x >> >> >> >> Hi Carl, hi Luiz, >> >> >> >> With every version of BaseX, we are adding new query optimizations (usually, this can be experienced if you run newer code with older version of BaseX). Existing optimization should of course be preserved. To be able to understand why your queries run slower than before, I would be more than glad if you could both provide me with little examples that we can try by ourselves. The Query Info output for both versions of BaseX would be helpful as well; it indicates which optimizations are performed or not. >> >> >> >> Thanks in advance >> >> Christian >> >> >> >> >> >> >> >> On Thu, Jul 28, 2016 at 8:36 PM, Bondeson, Carl <Carl.Bondeson at ct.gov> wrote: >> >> As a follow up for performance a few queries are listed with execution times >> >> >> >> declare option output:item-separator "
"; for $h in //ValidationDictionary/Validation let $g := $h/Parent[@Element='PatientDefinition'] let $i := $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='Gender'] for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/> >> >> Version: 7.9 execution time: 3.82 ms >> >> Version: 8.5.1 execution time: 138.8 ms >> >> >> >> declare option output:item-separator "
"; for $h in //ValidationDictionary/Validation let $g := $h/Parent[@Element='PatientDefinition'] let $i := $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='FirstName'] for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/> >> >> Version: 7.9 execution time: 3.3 ms >> >> Version: 8.5.1 execution time: 133.8 ms >> >> >> >> Query:declare option output:item-separator "
";for $g in //ValidationDictionary//Validation let $h := $g//Parent[@Element='PatientDefinition']let $i := $h//Child[@Element='PartyDefinition'] where $g//ProductCode='ZIKA' return <BaseXValidation Parent='{$h/@Element}' Child='{$i/@Element}'/> >> >> Version: 7.9 execution time: 2.2 ms >> >> Version: 8.5.1 execution time: 156.1 ms >> >> >> >> >> >> These queries are called from an XSLT transformer via all callout and are performed in very type loops. I was wondering why the transforms were never completing. 40-70x decrease in performance would slow any process down. >> >> >> >> Carl R Bondeson >> >> IT Analyst 3 >> >> Information Technology >> >> Connecticut Department of Public Health >> >> 410 Capitol Ave >> >> Hartford, CT 06134 >> >> Phone: 860-509-7434 >> >> carl.bondeson at ct.gov >> >> >> >> From: basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of Bondeson, Carl >> Sent: Thursday, July 28, 2016 1:36 PM >> To: basex-talk at mailman.uni-konstanz.de >> Subject: [basex-talk] BaseX performance 7.x vs 8.x >> >> >> >> I am seeing huge differences in the performance of a multi-threaded JBoss application recently upgraded from 7.9 to 8.4. The problem is that 8.4 is much slower than 7.9. Any ideas? Do I need to add indices or something along those lines? >> >> >> >> Carl R Bondeson >> >> IT Analyst 3 >> >> Information Technology >> >> Connecticut Department of Public Health >> >> 410 Capitol Ave >> >> Hartford, CT 06134 >> >> Phone: 860-509-7434 >> >> carl.bondeson at ct.gov >> >> >> >>
MG
Maximilian Gärber4 Aug 2016 10:24 #
Hi,
your query only reads data, right? With a db that small you could make sure it is all in memory by doing let $db := db:open('foo') update { }
Then, at least you don't have a penalty for disk/index access. As well, If I want to make sure that some data is availabe really fast (and to avoid n*n cost), I use the
map:merge [1] function to build a quick look-up structure.
[1] http://docs.basex.org/wiki/Map_Module#map:merge
Regards,
Max
2016-08-03 19:06 GMT+02:00 Bondeson, Carl <Carl.Bondeson at ct.gov>:Quoted text · Bondeson, Carl · 188 lines
> I am looking at creating a deliverable jar, in the not too distant future, which illustrates the issues. I just don't have the time right now. It is on my to-do list. Thanks again for your support! > > Carl R Bondeson > IT Analyst 3 > Information Technology > Connecticut Department of Public Health > 410 Capitol Ave > Hartford, CT 06134 > Phone: 860-509-7434 > carl.bondeson at ct.gov > > > > -----Original Message----- > From: Christian Grün [mailto:christian.gruen at gmail.com] > Sent: Wednesday, August 03, 2016 1:02 PM > To: Bondeson, Carl <Carl.Bondeson at ct.gov> > Cc: Luiz Matos <luizmatos at ic.uff.br>; basex-talk at mailman.uni-konstanz.de > Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x > > Sorry to hear that. If you can provide us your data, we could have another look. > > > On Wed, Aug 3, 2016 at 7:00 PM, Bondeson, Carl <Carl.Bondeson at ct.gov> wrote: >> I have not been able to get back to the same performance as was seen in 7.9. I attempted a change in architecture to use separate databases rather than having them combined in one. I created classes to support them, and updates to the XSLT tier to address the namespace additions. So rather than having 1 database with 14 MB of data, I now have 4 databases served by their own classes. In the end the performance was once again much worse than 7.9 with a single database. I have to look at the cost of calling up to 3 constructors in the XSLT transform, while looking at the gains, if any, or working with smaller individual databases. >> >> I will save this branch for another day. I have too many deliverables to afford any more resources at this time. >> >> >> Carl R Bondeson >> IT Analyst 3 >> Information Technology >> Connecticut Department of Public Health >> 410 Capitol Ave >> Hartford, CT 06134 >> Phone: 860-509-7434 >> carl.bondeson at ct.gov >> >> >> -----Original Message----- >> From: Christian Grün [mailto:christian.gruen at gmail.com] >> Sent: Friday, July 29, 2016 7:45 AM >> To: Bondeson, Carl <Carl.Bondeson at ct.gov> >> Cc: Luiz Matos <luizmatos at ic.uff.br>; basex-talk at mailman.uni-konstanz.de >> Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x >> >>> I have put together a set of queries, the output, and corresponding xml. >> >> I have run the first query; it yields no results. Is this expected? >> If you want, you can send me some more data in private. >> >> Could you please pass me on the query info output for both 8.5.2 and 7.9? >> >> >> >> >>> I appreciate your support. >>> >>> >>> >>> Carl R Bondeson >>> >>> IT Analyst 3 >>> >>> Information Technology >>> >>> Connecticut Department of Public Health >>> >>> 410 Capitol Ave >>> >>> Hartford, CT 06134 >>> >>> Phone: 860-509-7434 >>> >>> carl.bondeson at ct.gov >>> >>> >>> >>> From: Christian Grün [mailto:christian.gruen at gmail.com] >>> Sent: Friday, July 29, 2016 5:07 AM >>> To: Bondeson, Carl <Carl.Bondeson at ct.gov>; Luiz Matos <luizmatos at ic.uff.br> >>> Cc: basex-talk at mailman.uni-konstanz.de >>> Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x >>> >>> >>> >>> Hi Carl, hi Luiz, >>> >>> >>> >>> With every version of BaseX, we are adding new query optimizations (usually, this can be experienced if you run newer code with older version of BaseX). Existing optimization should of course be preserved. To be able to understand why your queries run slower than before, I would be more than glad if you could both provide me with little examples that we can try by ourselves. The Query Info output for both versions of BaseX would be helpful as well; it indicates which optimizations are performed or not. >>> >>> >>> >>> Thanks in advance >>> >>> Christian >>> >>> >>> >>> >>> >>> >>> >>> On Thu, Jul 28, 2016 at 8:36 PM, Bondeson, Carl <Carl.Bondeson at ct.gov> wrote: >>> >>> As a follow up for performance a few queries are listed with execution times >>> >>> >>> >>> declare option output:item-separator "
"; for $h in //ValidationDictionary/Validation let $g := $h/Parent[@Element='PatientDefinition'] let $i := $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='Gender'] for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/> >>> >>> Version: 7.9 execution time: 3.82 ms >>> >>> Version: 8.5.1 execution time: 138.8 ms >>> >>> >>> >>> declare option output:item-separator "
"; for $h in //ValidationDictionary/Validation let $g := $h/Parent[@Element='PatientDefinition'] let $i := $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='FirstName'] for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/> >>> >>> Version: 7.9 execution time: 3.3 ms >>> >>> Version: 8.5.1 execution time: 133.8 ms >>> >>> >>> >>> Query:declare option output:item-separator "
";for $g in //ValidationDictionary//Validation let $h := $g//Parent[@Element='PatientDefinition']let $i := $h//Child[@Element='PartyDefinition'] where $g//ProductCode='ZIKA' return <BaseXValidation Parent='{$h/@Element}' Child='{$i/@Element}'/> >>> >>> Version: 7.9 execution time: 2.2 ms >>> >>> Version: 8.5.1 execution time: 156.1 ms >>> >>> >>> >>> >>> >>> These queries are called from an XSLT transformer via all callout and are performed in very type loops. I was wondering why the transforms were never completing. 40-70x decrease in performance would slow any process down. >>> >>> >>> >>> Carl R Bondeson >>> >>> IT Analyst 3 >>> >>> Information Technology >>> >>> Connecticut Department of Public Health >>> >>> 410 Capitol Ave >>> >>> Hartford, CT 06134 >>> >>> Phone: 860-509-7434 >>> >>> carl.bondeson at ct.gov >>> >>> >>> >>> From: basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of Bondeson, Carl >>> Sent: Thursday, July 28, 2016 1:36 PM >>> To: basex-talk at mailman.uni-konstanz.de >>> Subject: [basex-talk] BaseX performance 7.x vs 8.x >>> >>> >>> >>> I am seeing huge differences in the performance of a multi-threaded JBoss application recently upgraded from 7.9 to 8.4. The problem is that 8.4 is much slower than 7.9. Any ideas? Do I need to add indices or something along those lines? >>> >>> >>> >>> Carl R Bondeson >>> >>> IT Analyst 3 >>> >>> Information Technology >>> >>> Connecticut Department of Public Health >>> >>> 410 Capitol Ave >>> >>> Hartford, CT 06134 >>> >>> Phone: 860-509-7434 >>> >>> carl.bondeson at ct.gov >>> >>> >>> >>>
BC
Bondeson, Carl4 Aug 2016 13:59 #
I am using the Java client session to manage BaseX. I create a database by
// create database and add mapper document
InputStream is = new FileInputStream(docmapperPath + "Mapper.xml");
instance.mapperDBSession.create("Mapper", is);
is.close();What is the analogous function to update {} in the Java API?
Thanks for your interest and feedback!
Carl R Bondeson
IT Analyst 3
Information Technology
Connecticut Department of Public Health
410 Capitol Ave
Hartford, CT 06134
Phone: 860-509-7434
carl.bondeson at ct.gov
Quoted text · Maximilian Gärber · 234 lines
-----Original Message----- From: gaerber at axxepta.net [mailto:gaerber at axxepta.net] On Behalf Of Maximilian Gärber Sent: Thursday, August 04, 2016 4:24 AM To: Bondeson, Carl <Carl.Bondeson at ct.gov> Cc: Christian Grün <christian.gruen at gmail.com>; basex-talk at mailman.uni-konstanz.de Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x Hi, your query only reads data, right? With a db that small you could make sure it is all in memory by doing let $db := db:open('foo') update { } Then, at least you don't have a penalty for disk/index access. As well, If I want to make sure that some data is availabe really fast (and to avoid n*n cost), I use the map:merge [1] function to build a quick look-up structure. [1] http://docs.basex.org/wiki/Map_Module#map:merge Regards, Max 2016-08-03 19:06 GMT+02:00 Bondeson, Carl <Carl.Bondeson at ct.gov>: > I am looking at creating a deliverable jar, in the not too distant future, which illustrates the issues. I just don't have the time right now. It is on my to-do list. Thanks again for your support! > > Carl R Bondeson > IT Analyst 3 > Information Technology > Connecticut Department of Public Health > 410 Capitol Ave > Hartford, CT 06134 > Phone: 860-509-7434 > carl.bondeson at ct.gov > > > > -----Original Message----- > From: Christian Grün [mailto:christian.gruen at gmail.com] > Sent: Wednesday, August 03, 2016 1:02 PM > To: Bondeson, Carl <Carl.Bondeson at ct.gov> > Cc: Luiz Matos <luizmatos at ic.uff.br>; > basex-talk at mailman.uni-konstanz.de > Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x > > Sorry to hear that. If you can provide us your data, we could have another look. > > > On Wed, Aug 3, 2016 at 7:00 PM, Bondeson, Carl <Carl.Bondeson at ct.gov> wrote: >> I have not been able to get back to the same performance as was seen in 7.9. I attempted a change in architecture to use separate databases rather than having them combined in one. I created classes to support them, and updates to the XSLT tier to address the namespace additions. So rather than having 1 database with 14 MB of data, I now have 4 databases served by their own classes. In the end the performance was once again much worse than 7.9 with a single database. I have to look at the cost of calling up to 3 constructors in the XSLT transform, while looking at the gains, if any, or working with smaller individual databases. >> >> I will save this branch for another day. I have too many deliverables to afford any more resources at this time. >> >> >> Carl R Bondeson >> IT Analyst 3 >> Information Technology >> Connecticut Department of Public Health >> 410 Capitol Ave >> Hartford, CT 06134 >> Phone: 860-509-7434 >> carl.bondeson at ct.gov >> >> >> -----Original Message----- >> From: Christian Grün [mailto:christian.gruen at gmail.com] >> Sent: Friday, July 29, 2016 7:45 AM >> To: Bondeson, Carl <Carl.Bondeson at ct.gov> >> Cc: Luiz Matos <luizmatos at ic.uff.br>; >> basex-talk at mailman.uni-konstanz.de >> Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x >> >>> I have put together a set of queries, the output, and corresponding xml. >> >> I have run the first query; it yields no results. Is this expected? >> If you want, you can send me some more data in private. >> >> Could you please pass me on the query info output for both 8.5.2 and 7.9? >> >> >> >> >>> I appreciate your support. >>> >>> >>> >>> Carl R Bondeson >>> >>> IT Analyst 3 >>> >>> Information Technology >>> >>> Connecticut Department of Public Health >>> >>> 410 Capitol Ave >>> >>> Hartford, CT 06134 >>> >>> Phone: 860-509-7434 >>> >>> carl.bondeson at ct.gov >>> >>> >>> >>> From: Christian Grün [mailto:christian.gruen at gmail.com] >>> Sent: Friday, July 29, 2016 5:07 AM >>> To: Bondeson, Carl <Carl.Bondeson at ct.gov>; Luiz Matos >>> <luizmatos at ic.uff.br> >>> Cc: basex-talk at mailman.uni-konstanz.de >>> Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x >>> >>> >>> >>> Hi Carl, hi Luiz, >>> >>> >>> >>> With every version of BaseX, we are adding new query optimizations (usually, this can be experienced if you run newer code with older version of BaseX). Existing optimization should of course be preserved. To be able to understand why your queries run slower than before, I would be more than glad if you could both provide me with little examples that we can try by ourselves. The Query Info output for both versions of BaseX would be helpful as well; it indicates which optimizations are performed or not. >>> >>> >>> >>> Thanks in advance >>> >>> Christian >>> >>> >>> >>> >>> >>> >>> >>> On Thu, Jul 28, 2016 at 8:36 PM, Bondeson, Carl <Carl.Bondeson at ct.gov> wrote: >>> >>> As a follow up for performance a few queries are listed with >>> execution times >>> >>> >>> >>> declare option output:item-separator "
"; for $h in >>> //ValidationDictionary/Validation let $g := >>> $h/Parent[@Element='PatientDefinition'] let $i := >>> $g/Child[@Element='PartyDefinition'] let $j := >>> $i/Item[@Attribute='Gender'] for $k in $j/@Type where >>> $h/ProductCode='ZIKA' return <BaseXValidation >>> Parent='{$g/@Element}' Child='{$i/@Element}' >>> Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' >>> DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' >>> Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' >>> Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' >>> Extra3='{$k/../@Extra3}'/> >>> >>> Version: 7.9 execution time: 3.82 ms >>> >>> Version: 8.5.1 execution time: 138.8 ms >>> >>> >>> >>> declare option output:item-separator "
"; for $h in >>> //ValidationDictionary/Validation let $g := >>> $h/Parent[@Element='PatientDefinition'] let $i := >>> $g/Child[@Element='PartyDefinition'] let $j := >>> $i/Item[@Attribute='FirstName'] for $k in $j/@Type where >>> $h/ProductCode='ZIKA' return <BaseXValidation >>> Parent='{$g/@Element}' Child='{$i/@Element}' >>> Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' >>> DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' >>> Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' >>> Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' >>> Extra3='{$k/../@Extra3}'/> >>> >>> Version: 7.9 execution time: 3.3 ms >>> >>> Version: 8.5.1 execution time: 133.8 ms >>> >>> >>> >>> Query:declare option output:item-separator "
";for $g in >>> //ValidationDictionary//Validation let $h := >>> $g//Parent[@Element='PatientDefinition']let $i := >>> $h//Child[@Element='PartyDefinition'] where $g//ProductCode='ZIKA' >>> return <BaseXValidation Parent='{$h/@Element}' >>> Child='{$i/@Element}'/> >>> >>> Version: 7.9 execution time: 2.2 ms >>> >>> Version: 8.5.1 execution time: 156.1 ms >>> >>> >>> >>> >>> >>> These queries are called from an XSLT transformer via all callout and are performed in very type loops. I was wondering why the transforms were never completing. 40-70x decrease in performance would slow any process down. >>> >>> >>> >>> Carl R Bondeson >>> >>> IT Analyst 3 >>> >>> Information Technology >>> >>> Connecticut Department of Public Health >>> >>> 410 Capitol Ave >>> >>> Hartford, CT 06134 >>> >>> Phone: 860-509-7434 >>> >>> carl.bondeson at ct.gov >>> >>> >>> >>> From: basex-talk-bounces at mailman.uni-konstanz.de >>> [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of >>> Bondeson, Carl >>> Sent: Thursday, July 28, 2016 1:36 PM >>> To: basex-talk at mailman.uni-konstanz.de >>> Subject: [basex-talk] BaseX performance 7.x vs 8.x >>> >>> >>> >>> I am seeing huge differences in the performance of a multi-threaded JBoss application recently upgraded from 7.9 to 8.4. The problem is that 8.4 is much slower than 7.9. Any ideas? Do I need to add indices or something along those lines? >>> >>> >>> >>> Carl R Bondeson >>> >>> IT Analyst 3 >>> >>> Information Technology >>> >>> Connecticut Department of Public Health >>> >>> 410 Capitol Ave >>> >>> Hartford, CT 06134 >>> >>> Phone: 860-509-7434 >>> >>> carl.bondeson at ct.gov >>> >>> >>> >>>
Written for BaseX 8.5. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery