Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8161903

Thread safety problem with java.util.zip.Deflator's input array

XMLWordPrintable

      FULL PRODUCT VERSION :
      java version "1.8.0_91"
      Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
      Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)

      ADDITIONAL OS VERSION INFORMATION :
      Darwin C02NH1CUG3QT.local 15.5.0 Darwin Kernel Version 15.5.0: Tue Apr 19 18:36:36 PDT 2016; root:xnu-3248.50.21~8/RELEASE_X86_64 x86_64


      A DESCRIPTION OF THE PROBLEM :
      Starting with 8u40, if you turn on -Xcheck:jni it will always make a copy of the array when GetPrimitiveArrayCritical/ReleasePrimitiveArrayCritical is using. The native code for Deflater uses that on the input array. If you are using the array as a ring buffer and compressing one chunk while writing to another, you can "lose" writes when it copies the original array back in.

      I think the bug is in the Deflater logic, the change to -Xcheck:jni is per spec as they mentioned when they added it: https://bugs.openjdk.java.net/browse/JDK-6311046 .



      REGRESSION. Last worked in version 7u80

      ADDITIONAL REGRESSION INFORMATION:
      I went by the comment in https://bugs.openjdk.java.net/browse/JDK-6311046, I didn't check against 1.7.

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      Compile the test case code submitted below.

      Run like this:
      ---
      $ java -cp . -Xcheck:jni -Xms64m -Xmx512m DeflateGate
      ---

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      The program should run until manually killed, reporting progress.
      ACTUAL -
      You get runs like this:

      ---
      $ java -cp . -Xcheck:jni -Xms64m -Xmx512m DeflateGate
      WARNING in native method: JNI call made without checking exceptions when required to from CallStaticObjectMethod
      WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod
      DEFLATE:: bytesDeflated: 1048902, fill val: 5
      FAIL: filler: -120, backer[x]: -76, x: 0
      DEFLATE:: bytesDeflated: 1048902, fill val: -76
      $ java -cp . -Xcheck:jni -Xms64m -Xmx512m DeflateGate
      WARNING in native method: JNI call made without checking exceptions when required to from CallStaticObjectMethod
      WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod
      FAIL: filler: -1, backer[x]: 50, x: 0
      DEFLATE:: bytesDeflated: 1048902, fill val: 50
      $ java -cp . -Xcheck:jni -Xms64m -Xmx512m DeflateGate
      WARNING in native method: JNI call made without checking exceptions when required to from CallStaticObjectMethod
      WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod
      DEFLATE:: bytesDeflated: 1048902, fill val: -102
      DEFLATE:: bytesDeflated: 1048902, fill val: -72
      DEFLATE:: bytesDeflated: 1048902, fill val: 117
      FAIL: filler: -48, backer[x]: 9, x: 0
      DEFLATE:: bytesDeflated: 1048902, fill val: 9
      ---

      REPRODUCIBILITY :
      This bug can be reproduced often.

      ---------- BEGIN SOURCE ----------
      import java.util.Arrays;
      import java.util.Random;
      import java.util.zip.Deflater;

      // I get this to fail every time using 1.8.0_51 and 91 from the command like using:
      // $ java -cp . -Xcheck:jni -Xms64m -Xmx512m DeflateGate


      public class DeflateGate {

      public static void main(String[] args) {
      int bufSize = 2 * 1024 * 1024;;
      byte[] backer = new byte[bufSize];

      new Random().nextBytes(backer);

      Thread t = new Thread() {
      public void run() {
      byte[] dumpster;
      Deflater deflater = new Deflater();
      while (true) {
      dumpster = new byte[2 * bufSize];
      deflater.reset();
      deflater.setInput(backer, bufSize/2, bufSize/2);
      deflater.finish();
      int bytesDeflated = deflater.deflate(dumpster);
      System.out.println("DEFLATE:: bytesDeflated: " + bytesDeflated + ", fill val: " + backer[0]);
      }
      }
      };
      t.start();

      byte filler = 1;
      long count = 1;
      while (true) {
      //Arrays.fill(backer, filler);
      backer[0] = filler;
      //System.gc();

      // I started out doing the whole array and then wanted to speed it up.
      // If you remove this one iteration loop, it won't fail. I don't know why.
      for (int x=0; x < 1; x++) {
      if (backer[x] != filler) {
      System.out.println("FAIL: filler: " + filler + ", backer[x]: " + backer[x] + ", x: " + x);
      System.exit(-1);
      }
      }
      if (count % 10000000 == 0) {
      System.out.println("MAIN:: Done with count: " + count);
      }
      filler++;
      count++;
      }

      }

      }
      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      If you synchronize on the array around the array write and the Deflater.delate() call it won't fail.

      If you remove the -Xcheck:jni flag, it won't fail.

            sherman Xueming Shen
            webbuggrp Webbug Group
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: