FULL PRODUCT VERSION :
java version "1.8.0_151"
Java(TM) SE Runtime Environment (build 1.8.0_151-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.151-b12, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows [Version 6.1.7601]

EXTRA RELEVANT SYSTEM CONFIGURATION :
Should probably work anywhere, as it seems to be a bug in the java code.

A DESCRIPTION OF THE PROBLEM :
When a zip file is openend to overwrite an existing entry, a ByteArrayOutputStream is created with the initial size of the existing entry. However, the size argument is an int, so if the size of the existing entry is somewhere between 2^31 and 2 ^32 (i.e. between 2 and 4 GB), the int will be negative, resulting in an IllegalArgumentException.
I didn't check what happens if the size is even larger than 2^32.


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Execute the code below.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Getting an OutputStream in which I can write the new content of the zip file entry.
ACTUAL -
IllegalArgumentException.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
Exception in thread "main" java.lang.IllegalArgumentException: Negative initial size: -2147479552
at java.io.ByteArrayOutputStream.<init>(ByteArrayOutputStream.java:74)
at com.sun.nio.zipfs.ZipFileSystem.getOutputStream(ZipFileSystem.java:1378)
at com.sun.nio.zipfs.ZipFileSystem.newOutputStream(ZipFileSystem.java:524)
at com.sun.nio.zipfs.ZipPath.newOutputStream(ZipPath.java:792)
at com.sun.nio.zipfs.ZipFileSystemProvider.newOutputStream(ZipFileSystemProvider.java:285)
at java.nio.file.Files.newOutputStream(Files.java:216)
at ZipFileSystemBug.main(ZipFileSystemBug.java:33)

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.io.File;
import java.io.OutputStream;
import java.net.URI;
import java.nio.file.FileSystem;
import java.nio.file.FileSystems;
import java.nio.file.Files;
import java.nio.file.StandardOpenOption;
import java.util.HashMap;
import java.util.Map;

public class ZipFileSystemBug {

    public static void main(String[] args) throws Exception {

        // Preparation: Create a Zip file with an entry larger than 2^31 bytes.
        File file = File.createTempFile("large", ".zip");
        file.delete();
        final URI uri = URI.create("jar:" + file.toURI());
        final Map<String, Object> env = new HashMap<>();
        env.put("create", "true");
        // Uncomment this for a temporary fix using the file system instead of memory (undocumented feature?)
        // env.put("useTempFile", Boolean.TRUE);
        System.out.println(uri);
        try (FileSystem fs = FileSystems.newFileSystem(uri, env)) {
            byte[] buf = new byte[4096];
            try (OutputStream out = Files.newOutputStream(fs.getPath("large"), StandardOpenOption.CREATE)) {
                for (int i = 0; i < 524289; i++) {
                    out.write(buf);
                }
            }
        }

        // Try to rewrite that file:
        try (FileSystem fs = FileSystems.newFileSystem(uri, env)) {
            Files.newOutputStream(fs.getPath("large"), StandardOpenOption.WRITE);
        } finally {
            // Cleanup.
            file.delete();
        }
    }
}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
You can bypass the problem by setting "useTempFile" to Boolean.TRUE (see comment in code).