Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-7022417

JDK6u23,6u24 GZIPInputStream readHeader broke handling of GZIP 'extra fields'

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • P2
    • None
    • 6u24
    • core-libs

    Description

      FULL PRODUCT VERSION :
      java version "1.6.0_24"
      Java(TM) SE Runtime Environment (build 1.6.0_24-b07)
      Java HotSpot(TM) Client VM (build 19.1-b02, mixed mode, sharing)


      A DESCRIPTION OF THE PROBLEM :
      Changes for 6u23 (probably for bug 4691425) broke GZIPInputStream.readHeader so it no longer properly handles the optional 'extra fields' GZIP data may include.

      Specifically, code that used to read:

      // Skip optional extra field
      if ((flg & FEXTRA) == FEXTRA) {
      skipBytes(in, readUShort(in));
      }

      ...now reads...

              // Skip optional extra field
              if ((flg & FEXTRA) == FEXTRA) {
                  skipBytes(in, readUShort(in));
                  int m = readUShort(in);
                  skipBytes(in, m);
                  n += m + 2;
              }

      It looks to me like a sloppy edit that meant to completely replace the first skipBytes(), but didn't. As a result, too many bytes are skipped (and the 'n' this method is trying to report can't be right). Depending on the subsequent data misinterpreted as a length, and subsequent overskip, several things could go wrong, including an error "java.util.zip.ZipException: invalid stored block lengths", an unexpected EOFException, and possibly other corrupted output/checksum warnings.

      I believe removing the leftover, untallied skipBytes() line will fix.

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      Try to use JDK6u23/JDK6u24 GZIPInputStream on GZIP data including any 'extra fields'. (See attached code for small example.)

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      Expected result is receiving the uncompressed data, as worked (at least for the first member encountered) pre-JDK6u23.
      ACTUAL -
      An exception is thrown. Some legal inputs might also return corrupted data, if the skip happened to land at an offset where the compressed data looked legal for a while.

      ERROR MESSAGES/STACK TRACES THAT OCCUR :
      The test below will generate:

      Exception in thread "main" java.util.zip.ZipException: invalid stored block lengths
      at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:147)
      at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:92)
      at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:105)
      at GZIPTest.main(GZIPTest.java:49)

      As first encountered, I received an unexpected EOFException (because my data at the time caused the bug to skip past end of compressed data).

      It's probably possible to receive corrupt uncompressed data for a while, then receive a GZIP checksum error or other indication of corruption after a while.

      REPRODUCIBILITY :
      This bug can be reproduced always.

      ---------- BEGIN SOURCE ----------
      import java.io.ByteArrayInputStream;
      import java.io.IOException;
      import java.util.zip.GZIPInputStream;

      /**
       * Demonstrate JDK6u23-JDK6u24 GZIPInputStream FEXTRA bug.
       *
       * Pre-JDK6u23, will print zero length of both GZIP inputs. JDK6u23-24,
       * will work on input without extra-field but throw exception for input with
       * extra-field.
       */
      public class GZIPTest {
          // Identical to what GZIPOutputStream creates with zero-length input
          static byte[] empty_gz = {
                    0x1f, (byte)0x8b, // ID1, ID2: GZIP MAGIC
                    0x08, // CM: DEFLATE
                    0x00, // FLG: NO FLAGS
                    0x00, 0x00, 0x00, 0x00, // MTIME
                    0x00, // XFL
                    0x00, // OS
                    0x03, 0x00, // compressed block
                    0x00, 0x00, 0x00, 0x00, // CRC32
                    0x00, 0x00, 0x00, 0x00, // ISIZE
          };
          
          // Above with added, legal (RFC1952) "extra field"
          static byte[] empty_extra_gz = {
                  0x1f, (byte)0x8b, // ID1, ID2: GZIP MAGIC
                  0x08, // CM: DEFLATE
                  0x04, // FLG: FEXTRA (0x04) set
                  0x00, 0x00, 0x00, 0x00, // MTIME
                  0x00, // XFL
                  0x00, // OS
                  
                  0x04, 0x00, // XLEN: 4 bytes of "extra field"
                  0x00, 0x00, 0x00, 0x00, // SI1, SI2, LEN: irrelevant
                  
                  0x03, 0x00, // compressed block
                  0x00, 0x00, 0x00, 0x00, // CRC32
                  0x00, 0x00, 0x00, 0x00, // ISIZE
        };
       
          public static void main(String [] args) throws IOException {
              GZIPInputStream in = new GZIPInputStream(new ByteArrayInputStream(empty_gz));
              long emptyCount = 0;
              while(in.read()>-1) {
                  emptyCount++;
              }
              System.out.println("empty_gz:"+emptyCount);
              GZIPInputStream in2 = new GZIPInputStream(new ByteArrayInputStream(empty_extra_gz));
              long emptyExtraCount = 0;
              while(in2.read()>-1) {
                  emptyExtraCount++;
              }
              System.out.println("empty_extra_gz:"+emptyExtraCount);
          }
      }
      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      Move back to pre-JDK6u23 releases. Copy the GZIPInputStream code elsewhere and remove t

      Attachments

        Issue Links

          Activity

            People

              sherman Xueming Shen
              webbuggrp Webbug Group
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                Imported:
                Indexed: