archive:create - Out Of Main Memory
Saturday, 25 May 2019 · 3 messages · last reply 29 May 2019
HP
Han (US), Paul P25 May 2019 00:09 #
Hello,
We encountered Out Of Main Memory when trying to create archive of a 4GB XML file. I'm using version BaseX922.war. We allocated 20GB with -Xmx flag for tomcat. Please let me know if there is a way to overcome this issue.
let $files := file:list('/tmp', false(), '4gb.xml')
let $zip := archive:create($files ! element archive:entry { . }, file:read-binary('/tmp/4gb.xml'))
return file:write-binary('4gb.xml.zip', $zip)Thanks,
Paul
CG
Christian Grün27 May 2019 19:25 #
Hi Paul,
In the current version of BaseX, new archives cannot be streamed to files.
I think we could propagate the file output stream to
archive:create and other functions. This could potentially be financed as sponsored feature. Does Boeing support open source projects?
Best,
Christian
Han (US), Paul P <Paul.P.Han at boeing.com> schrieb am Sa., 25. Mai 2019, 00:10:Quoted text · Han (US), Paul P · 26 lines
> Hello, > > > > We encountered Out Of Main Memory when trying to create archive of a 4GB > XML file. I’m using version BaseX922.war. We allocated 20GB with –Xmx > flag for tomcat. Please let me know if there is a way to overcome this > issue. > > > > let $files := file:list('/tmp', false(), '4gb.xml') > > let $zip := archive:create($files ! element archive:entry { . > }, file:read-binary('/tmp/4gb.xml')) > > return file:write-binary('4gb.xml.zip', $zip) > > > > > > Thanks, > > Paul >
HP
Han (US), Paul P29 May 2019 23:46 #
Hello Christian,
I have raised your inquiry to management. They would like to get a sample agreement to sponsor a solution.
Fortunately we are able to use zip:zip-file function to overcome the current limitation of the
archive:create function.
Regards,
PaulQuoted text · Christian Grün · 18 lines
From: Christian Grün [mailto:christian.gruen at gmail.com] Sent: Monday, May 27, 2019 10:25 AM To: Han (US), Paul P <Paul.P.Han at boeing.com> Cc: BaseX <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] archive:create - Out Of Main Memory Hi Paul, In the current version of BaseX, new archives cannot be streamed to files. I think we could propagate the file output stream to archive:create and other functions. This could potentially be financed as sponsored feature. Does Boeing support open source projects? Best, Christian Han (US), Paul P <Paul.P.Han at boeing.com<mailto:Paul.P.Han at boeing.com>> schrieb am Sa., 25. Mai 2019, 00:10: Hello, We encountered Out Of Main Memory when trying to create archive of a 4GB XML file. I’m using version BaseX922.war. We allocated 20GB with –Xmx flag for tomcat. Please let me know if there is a way to overcome this issue. let $files := file:list('/tmp', false(), '4gb.xml') let $zip := archive:create($files ! element archive:entry { . }, file:read-binary('/tmp/4gb.xml')) return file:write-binary('4gb.xml.zip', $zip) Thanks, Paul
Written for BaseX 9.2. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery