java.lang.NegativeArraySizeException

Thursday, 28 December 2017 · 2 messages · last reply 15 Jan 2018
SN
Stefanie Nageldiek28 Dec 2017 16:05 #
Hi! I'm trying to move a XSLT Transformation into XQuery. First step is Creating a db and trying to run the XSLT with XQuery. I have a small input file (50 MB) and a large input file (2GB) for testing. So I set up the db with the small input file, ran a parametrizised query calling the XSLT Transformation and everything went well. To be sure everything will work on the bigger db too, I tried to do the same, which fails - I had to allow for more memory which I did. (To be sure the stylesheet works with the big input file for the db I checked it (it takes 20 Minutes to produce a ~1 GB File). This is the error I get: Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk at mailman.uni-konstanz.de Version: BaseX 8.6.7 Java: Oracle Corporation, 1.8.0_151 OS: Windows 10, amd64 Stack Trace:
java.lang.NegativeArraySizeException
    at java.util.Arrays.copyOf(Unknown Source)
    at org.basex.io.out.ArrayOutput.write(ArrayOutput.java:25)
    at org.basex.io.out.PrintOutput.print(PrintOutput.java:76)
    at org.basex.io.out.NewlineOutput.print(NewlineOutput.java:33)
    at
org.basex.io.serial.MarkupSerializer.printChar(MarkupSerializer.java:254)
    at
org.basex.io.serial.MarkupSerializer.attribute(MarkupSerializer.java:131)
    at org.basex.io.serial.Serializer.node(Serializer.java:416)
    at org.basex.io.serial.Serializer.node(Serializer.java:158)
    at
org.basex.io.serial.StandardSerializer.node(StandardSerializer.java:105)
    at
org.basex.io.serial.AdaptiveSerializer.node(AdaptiveSerializer.java:75)
    at org.basex.io.serial.Serializer.serialize(Serializer.java:109)
    at
org.basex.io.serial.AdaptiveSerializer.serialize(AdaptiveSerializer.java:66)
    at org.basex.query.value.Value.serialize(Value.java:222)
    at org.basex.query.value.Value.serialize(Value.java:205)
    at org.basex.query.func.xslt.XsltTransform.read(XsltTransform.java:75)
    at
org.basex.query.func.xslt.XsltTransform.transform(XsltTransform.java:47)
    at org.basex.query.func.xslt.XsltTransform.item(XsltTransform.java:33)
    at org.basex.query.expr.ParseExpr.value(ParseExpr.java:65)
    at org.basex.query.QueryContext.value(QueryContext.java:406)
    at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:185)
    at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:93)
    at org.basex.query.scope.MainModule$1.next(MainModule.java:125)
    at org.basex.query.QueryContext.cache(QueryContext.java:618)
    at org.basex.query.QueryProcessor.cache(QueryProcessor.java:112)
    at org.basex.core.cmd.AQuery.query(AQuery.java:86)
    at org.basex.core.cmd.XQuery.run(XQuery.java:22)
    at org.basex.core.Command.run(Command.java:257)
    at org.basex.core.Command.execute(Command.java:93)
    at org.basex.gui.GUI.exec(GUI.java:474)
    at org.basex.gui.GUI.access$4(GUI.java:428)
    at org.basex.gui.GUI$6.run(GUI.java:416)
Compiling:
- pre-evaluate root() to document-node()
- pre-evaluate map { "param1":"default", "param2":"products" } to map
- pre-evaluate map { "method":"xml", "indent":"no",
"omit-xml-declaration":"no" } to map
- inline $path_2
- pre-evaluate concat("c:/work/xslt_to_xquery/",
"xslt_post_processing_bmwp-2526.xsl") to xs:string
- inline $style_3
Optimized Query:
let $document_4 := xslt:transform(db:open-pre("big-xml",0)/*,
"c:/work/xslt_to_xquery/i-know-which.xsl", map { "param1": "default",
"param2": "products", ... }) return
file:write(concat("c:/work/xslt_to_xquery/", db:name(.), "_out_",
"default", "products", ".xml"), $document_4, map { "omit-xml-declaration":
"no", "method": "xml", ... })
Query:
declare variable $param1 external := "default"; declare variable $param2
external := "products"; let $path := 'c:/work/xslt_to_xquery/' let $style
:= concat($path, 'i-know-which.xsl') let $document := xslt:transform(/*,
$style, map { "param1": $param1, "param2": $param2 }) return
file:write(concat($path, db:name(.), '_out_', $param1, $param2, '.xml'),
$document, map { "method": "xml", "indent": "no",
"omit-xml-declaration":"no" })
Query plan:
<QueryPlan compiled="true">
  <GFLWOR>
    <Let>
      <Var name="$document" id="4"/>
      <XsltTransform name="transform(input,stylesheet[,params[,options]])">
        <IterPath>
          <DBNode name="big-xml" pre="0"/>
          <IterStep axis="child" test="*"/>
        </IterPath>
        <Str value="c:/work/xslt_to_xquery/i-know-which.xsl"
type="xs:string"/>
        <Map size="2">
          <Str value="param1" type="xs:string"/>
          <Str value="default" type="xs:string"/>
          <Str value="param2" type="xs:string"/>
          <Str value="products" type="xs:string"/>
        </Map>
      </XsltTransform>
    </Let>
    <FileWrite name="write(path,data[,params])">
      <FnConcat name="concat(atom1,atom2[,...])">
        <Str value="c:/work/xslt_to_xquery/" type="xs:string"/>
        <DbName name="name(node)">
          <ContextValue/>
        </DbName>
        <Str value="_out_" type="xs:string"/>
        <Str value="default" type="xs:string"/>
        <Str value="products" type="xs:string"/>
        <Str value=".xml" type="xs:string"/>
      </FnConcat>
      <VarRef>
        <Var name="$document" id="4"/>
      </VarRef>
      <Map size="3">
        <Str value="omit-xml-declaration" type="xs:string"/>
        <Str value="no" type="xs:string"/>
        <Str value="method" type="xs:string"/>
        <Str value="xml" type="xs:string"/>
        <Str value="indent" type="xs:string"/>
        <Str value="no" type="xs:string"/>
      </Map>
    </FileWrite>
  </GFLWOR>
</QueryPlan>
Please let me know which information you need. Many thanks Steffie
CG
Christian Grün15 Jan 2018 12:59 #
Dear Steffie, Sorry for letting you wait; I’m back again after a longer winter break. As BaseX is no native XSLT Transformer, the memory limits for transforming data are much tighter than for XQuery expressions. XSLT results will be cached in memory before they will be passed on to the XSLT processor. However, I tried to tweak the code a bit, and I have improved the error message. I invite you to check out the latest stable snapshot and see if it allows you to process your input [1]. If not, you could try to: • Reduce the input to maybe 50% of the size in order to see if your XSLT processor (Java, Saxon) can handle this size of data. A general note: If there is some chance for you to do some more pre-processing in XQuery, you will definitely get better performance and require less memory. Hope this helps, Christian [1] http://files.basex.org/releases/latest/
Quoted text · Stefanie Nageldiek · 134 lines
On Thu, Dec 28, 2017 at 4:05 PM, <st.haupt at gmail.com> wrote: > Hi! > > I'm trying to move a XSLT Transformation into XQuery. First step is Creating > a db and trying to run the XSLT with XQuery. I have a small input file (50 > MB) and a large input file (2GB) for testing. > So I set up the db with the small input file, ran a parametrizised query > calling the XSLT Transformation and everything went well. To be sure > everything will work on the bigger db too, I tried to do the same, which > fails - I had to allow for more memory which I did. (To be sure the > stylesheet works with the big input file for the db I checked it (it takes > 20 Minutes to produce a ~1 GB File). > > This is the error I get: > > Error: > Improper use? Potential bug? Your feedback is welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 8.6.7 > Java: Oracle Corporation, 1.8.0_151 > OS: Windows 10, amd64 > Stack Trace: > java.lang.NegativeArraySizeException > at java.util.Arrays.copyOf(Unknown Source) > at org.basex.io.out.ArrayOutput.write(ArrayOutput.java:25) > at org.basex.io.out.PrintOutput.print(PrintOutput.java:76) > at org.basex.io.out.NewlineOutput.print(NewlineOutput.java:33) > at > org.basex.io.serial.MarkupSerializer.printChar(MarkupSerializer.java:254) > at > org.basex.io.serial.MarkupSerializer.attribute(MarkupSerializer.java:131) > at org.basex.io.serial.Serializer.node(Serializer.java:416) > at org.basex.io.serial.Serializer.node(Serializer.java:158) > at > org.basex.io.serial.StandardSerializer.node(StandardSerializer.java:105) > at > org.basex.io.serial.AdaptiveSerializer.node(AdaptiveSerializer.java:75) > at org.basex.io.serial.Serializer.serialize(Serializer.java:109) > at > org.basex.io.serial.AdaptiveSerializer.serialize(AdaptiveSerializer.java:66) > at org.basex.query.value.Value.serialize(Value.java:222) > at org.basex.query.value.Value.serialize(Value.java:205) > at org.basex.query.func.xslt.XsltTransform.read(XsltTransform.java:75) > at > org.basex.query.func.xslt.XsltTransform.transform(XsltTransform.java:47) > at org.basex.query.func.xslt.XsltTransform.item(XsltTransform.java:33) > at org.basex.query.expr.ParseExpr.value(ParseExpr.java:65) > at org.basex.query.QueryContext.value(QueryContext.java:406) > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:185) > at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:93) > at org.basex.query.scope.MainModule$1.next(MainModule.java:125) > at org.basex.query.QueryContext.cache(QueryContext.java:618) > at org.basex.query.QueryProcessor.cache(QueryProcessor.java:112) > at org.basex.core.cmd.AQuery.query(AQuery.java:86) > at org.basex.core.cmd.XQuery.run(XQuery.java:22) > at org.basex.core.Command.run(Command.java:257) > at org.basex.core.Command.execute(Command.java:93) > at org.basex.gui.GUI.exec(GUI.java:474) > at org.basex.gui.GUI.access$4(GUI.java:428) > at org.basex.gui.GUI$6.run(GUI.java:416) > Compiling: > - pre-evaluate root() to document-node() > - pre-evaluate map { "param1":"default", "param2":"products" } to map > - pre-evaluate map { "method":"xml", "indent":"no", > "omit-xml-declaration":"no" } to map > - inline $path_2 > - pre-evaluate concat("c:/work/xslt_to_xquery/", > "xslt_post_processing_bmwp-2526.xsl") to xs:string > - inline $style_3 > Optimized Query: > let $document_4 := xslt:transform(db:open-pre("big-xml",0)/*, > "c:/work/xslt_to_xquery/i-know-which.xsl", map { "param1": "default", > "param2": "products", ... }) return > file:write(concat("c:/work/xslt_to_xquery/", db:name(.), "_out_", "default", > "products", ".xml"), $document_4, map { "omit-xml-declaration": "no", > "method": "xml", ... }) > Query: > declare variable $param1 external := "default"; declare variable $param2 > external := "products"; let $path := 'c:/work/xslt_to_xquery/' let $style := > concat($path, 'i-know-which.xsl') let $document := xslt:transform(/*, > $style, map { "param1": $param1, "param2": $param2 }) return > file:write(concat($path, db:name(.), '_out_', $param1, $param2, '.xml'), > $document, map { "method": "xml", "indent": "no", > "omit-xml-declaration":"no" }) > Query plan: > <QueryPlan compiled="true"> > <GFLWOR> > <Let> > <Var name="$document" id="4"/> > <XsltTransform name="transform(input,stylesheet[,params[,options]])"> > <IterPath> > <DBNode name="big-xml" pre="0"/> > <IterStep axis="child" test="*"/> > </IterPath> > <Str value="c:/work/xslt_to_xquery/i-know-which.xsl" > type="xs:string"/> > <Map size="2"> > <Str value="param1" type="xs:string"/> > <Str value="default" type="xs:string"/> > <Str value="param2" type="xs:string"/> > <Str value="products" type="xs:string"/> > </Map> > </XsltTransform> > </Let> > <FileWrite name="write(path,data[,params])"> > <FnConcat name="concat(atom1,atom2[,...])"> > <Str value="c:/work/xslt_to_xquery/" type="xs:string"/> > <DbName name="name(node)"> > <ContextValue/> > </DbName> > <Str value="_out_" type="xs:string"/> > <Str value="default" type="xs:string"/> > <Str value="products" type="xs:string"/> > <Str value=".xml" type="xs:string"/> > </FnConcat> > <VarRef> > <Var name="$document" id="4"/> > </VarRef> > <Map size="3"> > <Str value="omit-xml-declaration" type="xs:string"/> > <Str value="no" type="xs:string"/> > <Str value="method" type="xs:string"/> > <Str value="xml" type="xs:string"/> > <Str value="indent" type="xs:string"/> > <Str value="no" type="xs:string"/> > </Map> > </FileWrite> > </GFLWOR> > </QueryPlan> > > Please let me know which information you need. > Many thanks > > Steffie

⚡Generated with XQuery