Xquery update performance on Base 8.1

On Wed, Apr 15, 2015 at 4:11 AM, Erol Akarsu <eakarsu at gmail.com> wrote: > > Hello, > > I am having noticeable performance degradation for updating existing > database in this function with Basex 81. I can say this function never > finish with 81. But when I replace nodes call with file:append (writing > result to a file), I can see it is slowness. > > But Basex_79 is much faster than Basex81 with regard to this update and > finish this function quickly without any issue. > > I like to upgrade to 81 but because of this issue I could not. > > Can you help me? > > declare %updating function local:addCRMSegmentsIntoProducts () > { > > let $allProds := fn:doc("AccumulatedProducts") > let $segments := > ("Aburcubur","Ãay_Kahve","İçecek","Karma_Az","Meyve_Sebze","Saç_Bakım","Süt_Su-Madeaze_Tüketim","Temizlik") > let $productsMap := > map:new( > for $rec in $allProds//doc > let $prodIDField := $rec/field[@name eq "ProductID"] > let $pid := $prodIDField/text() > return > map:entry($pid,$prodIDField)) > > for tumbling window $prodGroup in fn:doc("CRMSegments")//record > start $first next $second when fn:true() > end $last next $beyond when $last/PRODUCT_ID ne $beyond/PRODUCT_ID > > let $pid := $prodGroup[1]/PRODUCT_ID > return > let $prodEntry := map:get($productsMap,$pid)/.. > return > if (fn:empty($prodEntry)) then () > else > let $segData := > for $segGroup in $prodGroup > let $sid := $segGroup/SON_SEGMENT/text() > let $sid := > fn:concat("10",fn:index-of($segments,$sid)) > group by $sid > return > let $segAmount := sum($segGroup//AMOUNT/text()) > let $segOrderCount := sum($segGroup//ORDER_COUNT) > let $segFields := (<field > name="SegAmount_{$sid}">{$segAmount}</field>, > <field > name="SegOrderCount_{$sid}">{$segOrderCount}</field>) > return $segFields > > return > insert nodes ($segData) as last into $prodEntry > > }; >

> Today, I am trying to export a Bade DB about 2.5G but getting error with JDK > 8.1 > > With Basex 79 with JDK 7 was fine.

> >> export AccumulatedProducts > Improper use? Potential bug? Your feedback is welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 7.9 > Java: Oracle Corporation, 1.8.0_25 > OS: Linux, amd64 > Stack Trace: > java.lang.NegativeArraySizeException > at org.basex.io.random.DataAccess.readBytes(DataAccess.java:216) > at org.basex.io.random.DataAccess.readToken(DataAccess.java:194) > at org.basex.io.random.DataAccess.readToken(DataAccess.java:185) > at org.basex.data.DiskData.txt(DiskData.java:267) > at org.basex.data.DiskData.text(DiskData.java:235) > at org.basex.io.serial.Serializer.serialize(Serializer.java:459) > at org.basex.io.serial.Serializer.serialize(Serializer.java:335) > at org.basex.io.serial.Serializer.serialize(Serializer.java:135) > at org.basex.io.serial.Serializer.serialize(Serializer.java:115) > at org.basex.io.serial.Serializer.serialize(Serializer.java:105) > at org.basex.core.cmd.Export.export(Export.java:122) > at org.basex.core.cmd.Export.export(Export.java:66) > at org.basex.core.cmd.Export.run(Export.java:44) > at org.basex.core.Command.run(Command.java:329) > at org.basex.core.Command.execute(Command.java:94) > at org.basex.server.ClientListener.run(ClientListener.java:145) > > > > On Wed, Apr 15, 2015 at 3:06 AM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> Hi Erol, >> >> Could you possibly provide us with an sscce [1]? >> >> Thanks in advance, >> Christian >> >> [1] http://sscce.org/ >> >> >> >> On Wed, Apr 15, 2015 at 4:11 AM, Erol Akarsu <eakarsu at gmail.com> wrote: >> > >> > Hello, >> > >> > I am having noticeable performance degradation for updating existing >> > database in this function with Basex 81. I can say this function never >> > finish with 81. But when I replace nodes call with file:append (writing >> > result to a file), I can see it is slowness. >> > >> > But Basex_79 is much faster than Basex81 with regard to this update and >> > finish this function quickly without any issue. >> > >> > I like to upgrade to 81 but because of this issue I could not. >> > >> > Can you help me? >> > >> > declare %updating function local:addCRMSegmentsIntoProducts () >> > { >> > >> > let $allProds := fn:doc("AccumulatedProducts") >> > let $segments := >> > >> > ("Aburcubur","Ãay_Kahve","İçecek","Karma_Az","Meyve_Sebze","Saç_Bakım","Süt_Su-Madeaze_Tüketim","Temizlik") >> > let $productsMap := >> > map:new( >> > for $rec in $allProds//doc >> > let $prodIDField := $rec/field[@name eq "ProductID"] >> > let $pid := $prodIDField/text() >> > return >> > map:entry($pid,$prodIDField)) >> > >> > for tumbling window $prodGroup in fn:doc("CRMSegments")//record >> > start $first next $second when fn:true() >> > end $last next $beyond when $last/PRODUCT_ID ne >> > $beyond/PRODUCT_ID >> > >> > let $pid := $prodGroup[1]/PRODUCT_ID >> > return >> > let $prodEntry := map:get($productsMap,$pid)/.. >> > return >> > if (fn:empty($prodEntry)) then () >> > else >> > let $segData := >> > for $segGroup in $prodGroup >> > let $sid := $segGroup/SON_SEGMENT/text() >> > let $sid := >> > fn:concat("10",fn:index-of($segments,$sid)) >> > group by $sid >> > return >> > let $segAmount := sum($segGroup//AMOUNT/text()) >> > let $segOrderCount := >> > sum($segGroup//ORDER_COUNT) >> > let $segFields := (<field >> > name="SegAmount_{$sid}">{$segAmount}</field>, >> > <field >> > name="SegOrderCount_{$sid}">{$segOrderCount}</field>) >> > return $segFields >> > >> > return >> > insert nodes ($segData) as last into $prodEntry >> > >> > }; >> > > >

> export AccumulatedProducts

On Wed, Apr 15, 2015 at 3:06 AM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Erol, > > Could you possibly provide us with an sscce [1]? > > Thanks in advance, > Christian > > [1] http://sscce.org/ > > > > On Wed, Apr 15, 2015 at 4:11 AM, Erol Akarsu <eakarsu at gmail.com> wrote: > > > > Hello, > > > > I am having noticeable performance degradation for updating existing > > database in this function with Basex 81. I can say this function never > > finish with 81. But when I replace nodes call with file:append (writing > > result to a file), I can see it is slowness. > > > > But Basex_79 is much faster than Basex81 with regard to this update and > > finish this function quickly without any issue. > > > > I like to upgrade to 81 but because of this issue I could not. > > > > Can you help me? > > > > declare %updating function local:addCRMSegmentsIntoProducts () > > { > > > > let $allProds := fn:doc("AccumulatedProducts") > > let $segments := > > > ("Aburcubur","Ãay_Kahve","İçecek","Karma_Az","Meyve_Sebze","Saç_Bakım","Süt_Su-Madeaze_Tüketim","Temizlik") > > let $productsMap := > > map:new( > > for $rec in $allProds//doc > > let $prodIDField := $rec/field[@name eq "ProductID"] > > let $pid := $prodIDField/text() > > return > > map:entry($pid,$prodIDField)) > > > > for tumbling window $prodGroup in fn:doc("CRMSegments")//record > > start $first next $second when fn:true() > > end $last next $beyond when $last/PRODUCT_ID ne > $beyond/PRODUCT_ID > > > > let $pid := $prodGroup[1]/PRODUCT_ID > > return > > let $prodEntry := map:get($productsMap,$pid)/.. > > return > > if (fn:empty($prodEntry)) then () > > else > > let $segData := > > for $segGroup in $prodGroup > > let $sid := $segGroup/SON_SEGMENT/text() > > let $sid := > > fn:concat("10",fn:index-of($segments,$sid)) > > group by $sid > > return > > let $segAmount := sum($segGroup//AMOUNT/text()) > > let $segOrderCount := sum($segGroup//ORDER_COUNT) > > let $segFields := (<field > > name="SegAmount_{$sid}">{$segAmount}</field>, > > <field > > name="SegOrderCount_{$sid}">{$segOrderCount}</field>) > > return $segFields > > > > return > > insert nodes ($segData) as last into $prodEntry > > > > }; > > >

On Wed, Apr 15, 2015 at 2:40 PM, Christian Grün <christian.gruen at gmail.com> wrote: > > Today, I am trying to export a Bade DB about 2.5G but getting error with > JDK > > 8.1 > > > > With Basex 79 with JDK 7 was fine. > > So you mean you can export the same database with JDK 7, but not with > JDK 8? This indicates you should probably stick with Java 7 and wait > for newer versions of the JDK. > > What happens with JRE 8? Does JDK 8.1 exist at all? Afaik, the latest > version is 8u45. > > C. > > > > > >> export AccumulatedProducts > > Improper use? Potential bug? Your feedback is welcome: > > Contact: basex-talk at mailman.uni-konstanz.de > > Version: BaseX 7.9 > > Java: Oracle Corporation, 1.8.0_25 > > OS: Linux, amd64 > > Stack Trace: > > java.lang.NegativeArraySizeException > > at org.basex.io.random.DataAccess.readBytes(DataAccess.java:216) > > at org.basex.io.random.DataAccess.readToken(DataAccess.java:194) > > at org.basex.io.random.DataAccess.readToken(DataAccess.java:185) > > at org.basex.data.DiskData.txt(DiskData.java:267) > > at org.basex.data.DiskData.text(DiskData.java:235) > > at org.basex.io.serial.Serializer.serialize(Serializer.java:459) > > at org.basex.io.serial.Serializer.serialize(Serializer.java:335) > > at org.basex.io.serial.Serializer.serialize(Serializer.java:135) > > at org.basex.io.serial.Serializer.serialize(Serializer.java:115) > > at org.basex.io.serial.Serializer.serialize(Serializer.java:105) > > at org.basex.core.cmd.Export.export(Export.java:122) > > at org.basex.core.cmd.Export.export(Export.java:66) > > at org.basex.core.cmd.Export.run(Export.java:44) > > at org.basex.core.Command.run(Command.java:329) > > at org.basex.core.Command.execute(Command.java:94) > > at org.basex.server.ClientListener.run(ClientListener.java:145) > > > > > > > > On Wed, Apr 15, 2015 at 3:06 AM, Christian Grün < > christian.gruen at gmail.com> > > wrote: > >> > >> Hi Erol, > >> > >> Could you possibly provide us with an sscce [1]? > >> > >> Thanks in advance, > >> Christian > >> > >> [1] http://sscce.org/ > >> > >> > >> > >> On Wed, Apr 15, 2015 at 4:11 AM, Erol Akarsu <eakarsu at gmail.com> wrote: > >> > > >> > Hello, > >> > > >> > I am having noticeable performance degradation for updating existing > >> > database in this function with Basex 81. I can say this function > never > >> > finish with 81. But when I replace nodes call with file:append > (writing > >> > result to a file), I can see it is slowness. > >> > > >> > But Basex_79 is much faster than Basex81 with regard to this update > and > >> > finish this function quickly without any issue. > >> > > >> > I like to upgrade to 81 but because of this issue I could not. > >> > > >> > Can you help me? > >> > > >> > declare %updating function local:addCRMSegmentsIntoProducts () > >> > { > >> > > >> > let $allProds := fn:doc("AccumulatedProducts") > >> > let $segments := > >> > > >> > > ("Aburcubur","Ãay_Kahve","İçecek","Karma_Az","Meyve_Sebze","Saç_Bakım","Süt_Su-Madeaze_Tüketim","Temizlik") > >> > let $productsMap := > >> > map:new( > >> > for $rec in $allProds//doc > >> > let $prodIDField := $rec/field[@name eq "ProductID"] > >> > let $pid := $prodIDField/text() > >> > return > >> > map:entry($pid,$prodIDField)) > >> > > >> > for tumbling window $prodGroup in fn:doc("CRMSegments")//record > >> > start $first next $second when fn:true() > >> > end $last next $beyond when $last/PRODUCT_ID ne > >> > $beyond/PRODUCT_ID > >> > > >> > let $pid := $prodGroup[1]/PRODUCT_ID > >> > return > >> > let $prodEntry := map:get($productsMap,$pid)/.. > >> > return > >> > if (fn:empty($prodEntry)) then () > >> > else > >> > let $segData := > >> > for $segGroup in $prodGroup > >> > let $sid := $segGroup/SON_SEGMENT/text() > >> > let $sid := > >> > fn:concat("10",fn:index-of($segments,$sid)) > >> > group by $sid > >> > return > >> > let $segAmount := > sum($segGroup//AMOUNT/text()) > >> > let $segOrderCount := > >> > sum($segGroup//ORDER_COUNT) > >> > let $segFields := (<field > >> > name="SegAmount_{$sid}">{$segAmount}</field>, > >> > <field > >> > name="SegOrderCount_{$sid}">{$segOrderCount}</field>) > >> > return $segFields > >> > > >> > return > >> > insert nodes ($segData) as last into $prodEntry > >> > > >> > }; > >> > > > > > >