Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8133170

Deflater.deflate with small output buffers fails

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: P4 P4
    • 9
    • 8u45, 9
    • core-libs
    • b127
    • x86_64
    • linux

      FULL PRODUCT VERSION :
      java version "1.8.0_45"
      Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
      Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)


      ADDITIONAL OS VERSION INFORMATION :
      Linux lapetus 3.19.8-100.fc20.x86_64 #1 SMP Tue May 12 17:08:50 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

      A DESCRIPTION OF THE PROBLEM :
      Originally asked as question on stackoverflow
      http://stackoverflow.com/questions/31861983/deflater-deflate-and-small-output-buffers

      Copied here (I am author of original stackoverflow question)

      I'm seeing a strange situation with small output buffers with Java 8u45 and the java.util.Deflater.deflate(byte[] b, int off, int len, int flush) method when used with small output buffers.

      (I'm working on some low level networking code related to WebSocket's upcoming permessage-deflate extension, so small buffers are a reality for me)

      If a buffer of 5 or 6 bytes is used as an output buffer for .deflate() with SYNC_FLUSH then there is no way for the deflate to finish.



      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      Run test class

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      In the provided example class, I'm attempting to generate compressed bytes for the input "Hello" using an output buffer of 5 bytes in length.

      3 buffers through use of example code attempting to compress "Hello" (5 bytes) using an output buffer of 5 bytes.

      buffer 1 [ F2 48 CD C9 C9 ]
      buffer 2 [ 07 00 00 00 FF ]
      buffer 3 [ FF ]

      Which translates as

      [ F2 48 CD C9 C9 07 00 ] <-- the compressed data
      [ 00 00 FF FF ] <-- the deflate tail bytes

      Other possible outcome :

      A RuntimeException on use of small output buffers sizes when using SYNC_FLUSH with .deflate()


      ACTUAL -
      Infinite loop

      Resulting output of running the provided demo code ...

      input is 5 bytes - [48 65 6C 6C 6F]
      compressed 5 bytes - [F2 48 CD C9 C9]
      compressed 5 bytes - [07 00 00 00 FF]
      compressed 5 bytes - [FF 00 00 00 FF]
      compressed 5 bytes - [FF 00 00 00 FF]
      compressed 5 bytes - [FF 00 00 00 FF]
      compressed 5 bytes - [FF 00 00 00 FF]
      compressed 5 bytes - [FF 00 00 00 FF]
      compressed 5 bytes - [FF 00 00 00 FF]
      compressed 5 bytes - [FF 00 00 00 FF]
      compressed 5 bytes - [FF 00 00 00 FF]
      Exited compress (maxloops left -1)



      REPRODUCIBILITY :
      This bug can be reproduced always.

      ---------- BEGIN SOURCE ----------
      package deflate;

      import java.nio.charset.StandardCharsets;
      import java.util.zip.Deflater;

      public class DeflaterSmallBufferBug
      {
          public static void main(String[] args)
          {
              boolean nowrap = true;
              Deflater deflater = new Deflater(Deflater.DEFAULT_COMPRESSION,nowrap);

              byte[] input = "Hello".getBytes(StandardCharsets.UTF_8);

              System.out.printf("input is %,d bytes - %s%n",input.length,getHex(input,0,input.length));

              deflater.setInput(input);

              byte[] output = new byte[input.length];

              // break out of infinite loop seen with bug
              int maxloops = 10;

              // Compress the data
              while (maxloops-- > 0)
              {
                  int compressed = deflater.deflate(output,0,output.length,Deflater.SYNC_FLUSH);
                  System.out.printf("compressed %,d bytes - %s%n",compressed,getHex(output,0,compressed));

                  if (compressed < output.length)
                  {
                      System.out.printf("Compress success");
                      return;
                  }
              }

              System.out.printf("Exited compress (maxloops left %d)%n",maxloops);
          }

          private static String getHex(byte[] buf, int offset, int len)
          {
              StringBuilder hex = new StringBuilder();
              hex.append('[');
              for (int i = offset; i < (offset + len); i++)
              {
                  if (i > offset)
                  {
                      hex.append(' ');
                  }
                  hex.append(String.format("%02X",buf[i]));
              }
              hex.append(']');
              return hex.toString();
          }
      }
      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      After asking the question on stackoverflow, Mark Adler (co-author of zlib/gzip) answered and pointed out that there is a (greater than) 6 byte output buffer requirement for use of SYNC_FLUSH successfully with zlib.



            sherman Xueming Shen
            webbuggrp Webbug Group
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: