-
Bug
-
Resolution: Fixed
-
P4
-
8u45, 9
-
b127
-
x86_64
-
linux
FULL PRODUCT VERSION :
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
ADDITIONAL OS VERSION INFORMATION :
Linux lapetus 3.19.8-100.fc20.x86_64 #1 SMP Tue May 12 17:08:50 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
A DESCRIPTION OF THE PROBLEM :
Originally asked as question on stackoverflow
http://stackoverflow.com/questions/31861983/deflater-deflate-and-small-output-buffers
Copied here (I am author of original stackoverflow question)
I'm seeing a strange situation with small output buffers with Java 8u45 and the java.util.Deflater.deflate(byte[] b, int off, int len, int flush) method when used with small output buffers.
(I'm working on some low level networking code related to WebSocket's upcoming permessage-deflate extension, so small buffers are a reality for me)
If a buffer of 5 or 6 bytes is used as an output buffer for .deflate() with SYNC_FLUSH then there is no way for the deflate to finish.
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Run test class
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
In the provided example class, I'm attempting to generate compressed bytes for the input "Hello" using an output buffer of 5 bytes in length.
3 buffers through use of example code attempting to compress "Hello" (5 bytes) using an output buffer of 5 bytes.
buffer 1 [ F2 48 CD C9 C9 ]
buffer 2 [ 07 00 00 00 FF ]
buffer 3 [ FF ]
Which translates as
[ F2 48 CD C9 C9 07 00 ] <-- the compressed data
[ 00 00 FF FF ] <-- the deflate tail bytes
Other possible outcome :
A RuntimeException on use of small output buffers sizes when using SYNC_FLUSH with .deflate()
ACTUAL -
Infinite loop
Resulting output of running the provided demo code ...
input is 5 bytes - [48 65 6C 6C 6F]
compressed 5 bytes - [F2 48 CD C9 C9]
compressed 5 bytes - [07 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
Exited compress (maxloops left -1)
REPRODUCIBILITY :
This bug can be reproduced always.
---------- BEGIN SOURCE ----------
package deflate;
import java.nio.charset.StandardCharsets;
import java.util.zip.Deflater;
public class DeflaterSmallBufferBug
{
public static void main(String[] args)
{
boolean nowrap = true;
Deflater deflater = new Deflater(Deflater.DEFAULT_COMPRESSION,nowrap);
byte[] input = "Hello".getBytes(StandardCharsets.UTF_8);
System.out.printf("input is %,d bytes - %s%n",input.length,getHex(input,0,input.length));
deflater.setInput(input);
byte[] output = new byte[input.length];
// break out of infinite loop seen with bug
int maxloops = 10;
// Compress the data
while (maxloops-- > 0)
{
int compressed = deflater.deflate(output,0,output.length,Deflater.SYNC_FLUSH);
System.out.printf("compressed %,d bytes - %s%n",compressed,getHex(output,0,compressed));
if (compressed < output.length)
{
System.out.printf("Compress success");
return;
}
}
System.out.printf("Exited compress (maxloops left %d)%n",maxloops);
}
private static String getHex(byte[] buf, int offset, int len)
{
StringBuilder hex = new StringBuilder();
hex.append('[');
for (int i = offset; i < (offset + len); i++)
{
if (i > offset)
{
hex.append(' ');
}
hex.append(String.format("%02X",buf[i]));
}
hex.append(']');
return hex.toString();
}
}
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
After asking the question on stackoverflow, Mark Adler (co-author of zlib/gzip) answered and pointed out that there is a (greater than) 6 byte output buffer requirement for use of SYNC_FLUSH successfully with zlib.
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
ADDITIONAL OS VERSION INFORMATION :
Linux lapetus 3.19.8-100.fc20.x86_64 #1 SMP Tue May 12 17:08:50 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
A DESCRIPTION OF THE PROBLEM :
Originally asked as question on stackoverflow
http://stackoverflow.com/questions/31861983/deflater-deflate-and-small-output-buffers
Copied here (I am author of original stackoverflow question)
I'm seeing a strange situation with small output buffers with Java 8u45 and the java.util.Deflater.deflate(byte[] b, int off, int len, int flush) method when used with small output buffers.
(I'm working on some low level networking code related to WebSocket's upcoming permessage-deflate extension, so small buffers are a reality for me)
If a buffer of 5 or 6 bytes is used as an output buffer for .deflate() with SYNC_FLUSH then there is no way for the deflate to finish.
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Run test class
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
In the provided example class, I'm attempting to generate compressed bytes for the input "Hello" using an output buffer of 5 bytes in length.
3 buffers through use of example code attempting to compress "Hello" (5 bytes) using an output buffer of 5 bytes.
buffer 1 [ F2 48 CD C9 C9 ]
buffer 2 [ 07 00 00 00 FF ]
buffer 3 [ FF ]
Which translates as
[ F2 48 CD C9 C9 07 00 ] <-- the compressed data
[ 00 00 FF FF ] <-- the deflate tail bytes
Other possible outcome :
A RuntimeException on use of small output buffers sizes when using SYNC_FLUSH with .deflate()
ACTUAL -
Infinite loop
Resulting output of running the provided demo code ...
input is 5 bytes - [48 65 6C 6C 6F]
compressed 5 bytes - [F2 48 CD C9 C9]
compressed 5 bytes - [07 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
Exited compress (maxloops left -1)
REPRODUCIBILITY :
This bug can be reproduced always.
---------- BEGIN SOURCE ----------
package deflate;
import java.nio.charset.StandardCharsets;
import java.util.zip.Deflater;
public class DeflaterSmallBufferBug
{
public static void main(String[] args)
{
boolean nowrap = true;
Deflater deflater = new Deflater(Deflater.DEFAULT_COMPRESSION,nowrap);
byte[] input = "Hello".getBytes(StandardCharsets.UTF_8);
System.out.printf("input is %,d bytes - %s%n",input.length,getHex(input,0,input.length));
deflater.setInput(input);
byte[] output = new byte[input.length];
// break out of infinite loop seen with bug
int maxloops = 10;
// Compress the data
while (maxloops-- > 0)
{
int compressed = deflater.deflate(output,0,output.length,Deflater.SYNC_FLUSH);
System.out.printf("compressed %,d bytes - %s%n",compressed,getHex(output,0,compressed));
if (compressed < output.length)
{
System.out.printf("Compress success");
return;
}
}
System.out.printf("Exited compress (maxloops left %d)%n",maxloops);
}
private static String getHex(byte[] buf, int offset, int len)
{
StringBuilder hex = new StringBuilder();
hex.append('[');
for (int i = offset; i < (offset + len); i++)
{
if (i > offset)
{
hex.append(' ');
}
hex.append(String.format("%02X",buf[i]));
}
hex.append(']');
return hex.toString();
}
}
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
After asking the question on stackoverflow, Mark Adler (co-author of zlib/gzip) answered and pointed out that there is a (greater than) 6 byte output buffer requirement for use of SYNC_FLUSH successfully with zlib.