Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8298530

ZipInputStream.readEnd fails for certain ZIP64 archives with small files

XMLWordPrintable

      ADDITIONAL SYSTEM INFORMATION :
      Tested on Linux and the following JDKs:
      JDK 1.8.0_341
      OpenJDK 11.0.17
      OpenJDK 17.0.2
      OpenJDK 19.0.1

      A DESCRIPTION OF THE PROBLEM :
      ZipInputStream fails to close an entry under these conditions:
       - File size and compressed size are under 4 GiB
       - The local file header is ZIP64-encoded
       - Actual file sizes are contained within a trailing data descriptor

      ZipInputStream interprets the data descriptor as 32-bit despite the local file header being ZIP64 and therefore reads the file size incorrectly. The issue is on line 349 in readEnd():
              if (inf.getBytesWritten() > ZIP64_MAGICVAL ||
                      inf.getBytesRead() > ZIP64_MAGICVAL) {

      Despite the local file header indicating 64-bit file sizes, the Inflator reports a < 4 GiB file, and ZIS reads 32-bit size fields from the data descriptor. This leaves the stream in a misaligned state and always results in uncompressed size being interpreted as 0 bytes.

      The bug does not affect ZipFile because it reads sizes from the central record instead of the data descriptors.

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      Use ZipInputStream to read a ZIP file encoded with the following properties:
        - Data descriptors present (sizes in header marked as zero)
        - ZIP64 (8-bit) size fields
        - Contain files of size less than 4 GiB
      A sample file with these conditions is included in test case.

      Read the entry using getNextEntry()
      Read all file bytes using read()

      The next call to read() or closeEntry() when at the end of the data stream throws an exception.

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      Test case should execute with no exception
      ACTUAL -
      Entry name: Content.txt
      Entry compressed size: -1
      Entry uncompressed size: -1
      Stream failed at byte 199
      Post readEnd() compressed size: 141
      Post readEnd() uncompressed size: 0
      java.util.zip.ZipException: invalid entry size (expected 0 but got 199 bytes)
      at java.util.zip.ZipInputStream.readEnd(ZipInputStream.java:384)
      at java.util.zip.ZipInputStream.read(ZipInputStream.java:196)
      at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:122)
      at main(DemonstrationTest2.java:39)


      ---------- BEGIN SOURCE ----------
      import java.io.BufferedInputStream;
      import java.io.ByteArrayInputStream;
      import java.io.InputStream;
      import java.util.Base64;
      import java.util.zip.ZipEntry;
      import java.util.zip.ZipInputStream;

      public class DemonstrationTest2 {
        
        public static void main(String[] args) throws Exception {
          // Base64-encoded ZIP file that triggers the bug
          String zipExample = "UEsDBC0ACAgIABNO+lQAAAAA//////////8LABQAQ29udGVudC50eHQBABAAAAAA"
                            + "AAAAAAAAAAAAAAAAABWNsZHDMAwEc1VxFagKJ449XwBknv8xQ5ASAXhcvvHZ3Qa7"
                            + "j/SnnhoYfNJdQ0KPdCwaONR2/IxGMHDKUolcuFIFNjs9VBziJ5fOxf/paRxN8Ccr"
                            + "DRk77r1PTBvq0N4LNh4a9d7slXU2sGhJ/Uqp3m32kkG6FmvQ9JiQuJLlg8lHjQja"
                            + "OZfglb+yb9v2BVBLBwjbriq7jQAAAAAAAADHAAAAAAAAAFBLAwQtAAgICAATTvpU"
                            + "AAAAAP//////////CwAUAEFiY2RlZmcudHh0AQAQAAAAAAAAAAAAAAAAAAAAAAAV"
                            + "jbGRwzAMBHNVcRWoCieOPV8AZJ7/MUOQEgF4XL7x2d0Gu4/0p54aGHzSXUNCj3Qs"
                            + "GjjUdvyMRjBwylKJXLhSBTY7PVQc4ieXzsX/6WkcTfAnKw0ZO+69T0wb6tDeCzYe"
                            + "GvXe7JV1NrBoSf1Kqd5t9pJBuhZr0PSYkLiS5YPJR40I2jmX4JW/sm/b9gVQSwcI"
                            + "264qu40AAAAAAAAAxwAAAAAAAABQSwECPwMUAAIACAC3tXxV264qu40AAADHAAAA"
                            + "CwAAAAAAAAAAAAAApIEAAAAAQ29udGVudC50eHRQSwECPwMUAAIACAC3tXxV264q"
                            + "u40AAADHAAAACwAAAAAAAAAAAAAApIHiAAAAQWJjZGVmZy50eHRQSwUGAAAAAAIA"
                            + "AgByAAAAxAEAAAAA";
          
          try (InputStream fileStream = new ByteArrayInputStream(Base64.getDecoder().decode(zipExample));
               InputStream bufferedStream = new BufferedInputStream(fileStream);
               ZipInputStream zipStream = new ZipInputStream(bufferedStream)) {
            
            ZipEntry entry = zipStream.getNextEntry();
            System.out.println("Entry name: " + entry.getName());
            System.out.println("Entry compressed size: " + entry.getCompressedSize());
            System.out.println("Entry uncompressed size: " + entry.getSize());
            
            // Consume all bytes
            StringBuilder content = new StringBuilder();
            try {
              for (int b; (b = zipStream.read()) >= 0;) content.append((char) b);
            } catch (Exception e) {
              System.err.println("Stream failed at byte " + content.length());
              System.out.println("Post readEnd() compressed size: " + entry.getCompressedSize());
              System.out.println("Post readEnd() uncompressed size: " + entry.getSize());
              e.printStackTrace();
            }
          }
          
        }
      }

      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      One workaround is to modify readEnd() in ZipInputStream to check if the local file header was ZIP64, and if so, read the data descriptor as ZIP64 as well.

      A workaround using reflection to manually advance the stream:
      https://github.com/cjgriscom/ZipInputStreamPatch64/blob/main/src/main/java/io/chandler/zip/patch64/ZipInputStreamPatch64.java

      FREQUENCY : always


            lancea Lance Andersen
            webbuggrp Webbug Group
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: