problems using Archive module with certain zip files

Hello, I'm using BaseX in several file conversion projects that involve unpacking a zip file using the Archive module, modifying the files that are found inside with the File module and XSLT module, and creating new zip file(s) using the Archive module again. BaseX is very useful for these kinds of transformations and makes it quick to develop a transformation complete with unit tests. I'm running in to 2 kinds of problems with certain zip files and getting the following error messages from the Archive module. 1. When attempting to extract certain zip files

archive:extract-to() or archive:entries() produce this error message "Operation failed: only DEFLATED entries can have EXT descriptor."

The zip files that produce this error can be extracted using another program such as 7zip, which might be better at handling variations in the structure of some zip files. 2. When attempting to create a zip file larger than about 2 Gb. After extracting a zip file larger than about 2 Gb using archive:extract-to (which works), when trying to create a new zip file

file:write-binary($newZip, archive:create-from($tempDir))

produces a stack trace that begins with: java.lang.ArrayIndexOutOfBoundsException: Maximum array size reached. at org.basex.util.Array.checkCapacity(Array.java:322) at org.basex.util.Array.newCapacity(Array.java:313) at org.basex.util.Array.newCapacity(Array.java:301) at org.basex.io.out.ArrayOutput.write(ArrayOutput.java:32) at java.base/java.io.OutputStream.write(OutputStream.java:157) at java.base/java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:253) at java.base/java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:211) at java.base/java.util.zip.ZipOutputStream.write(ZipOutputStream.java:332) at java.base/java.io.FilterOutputStream.write(FilterOutputStream.java:108) at org.basex.query.func.archive.ZIPOut.write(ZIPOut.java:44) at org.basex.query.func.archive.ArchiveCreate.add(ArchiveCreate.java:138) I'm currently using BaseX version 9.4.1 and Java 64-bit openjdk version "11.0.7" 2020-04-14. This might be a bug in BaseX handling of zip files. Is there any solution or workaround? Thanks, Vincent ______________________________________________ Vincent M. Lizzi Head of Information Standards | Taylor & Francis Group 530 Walnut St., Suite 850, Philadelphia, PA 19106 E-Mail: vincent.lizzi at taylorandfrancis.com Phone: 215-606-4221 Web: www.tandfonline.com Taylor & Francis is a trading name of Informa UK Limited, registered in England under no. 1072954 "Everything should be made as simple as possible, but not simpler." Information Classification: General

Hi Vincent,

This error seems to be caused by to a bug in the JDK [1]. As the bug entry has been opened in 2015, I’m not sure if we can count on a quick fix.

This is a known restriction indeed. The best option might be to directly stream archives to the specified file on disk. If enough people are interested in such an extension, we could try to realize this as sponsored feature. Best, Christian [1] https://bugs.openjdk.java.net/browse/JDK-8143613