Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8292158

AES-CTR cipher state corruption with AVX-512

XMLWordPrintable

    • b17
    • x86_64
    • Verified

        A DESCRIPTION OF THE PROBLEM :
        The internal state of the "AES/CTR/NoPadding" cipher can become corrupted when encrypting/decrypting byte arrays with a size less than 16. This only occurs when the AVX-512 AES CTR intrinsic is enabled.

        REGRESSION : Last worked in version 8u341

        STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
        Run provided test code with -XX:UseAVX=3


        ---------- BEGIN SOURCE ----------
        import javax.crypto.Cipher;
        import javax.crypto.spec.IvParameterSpec;
        import javax.crypto.spec.SecretKeySpec;
        import java.security.Key;
        import java.security.SecureRandom;
        import java.util.Arrays;

        public class Main {
            private static final int LOOPS = 1000;

            public static void main(String[] args) throws Exception {
                SecureRandom random = new SecureRandom();

                byte[] keyBytes = new byte[32];
                random.nextBytes(keyBytes);
                Key key = new SecretKeySpec(keyBytes, "AES");

                byte[] ivBytes = new byte[16];
                random.nextBytes(ivBytes);
                IvParameterSpec iv = new IvParameterSpec(ivBytes);

                Cipher encrypt = Cipher.getInstance("AES/CTR/NoPadding");
                Cipher decrypt = Cipher.getInstance("AES/CTR/NoPadding");

                while (true) {
                    byte[][] unencryptedStuff = new byte[LOOPS][];
                    byte[][] encryptedStuff = new byte[LOOPS][];

                    encrypt.init(Cipher.ENCRYPT_MODE, key, iv);
                    for (int i = 0; i < LOOPS; i++) {
                        int size = (i % 15) + 1;

                        byte[] unencrypted = new byte[size];
                        random.nextBytes(unencrypted);
                        unencryptedStuff[i] = unencrypted;

                        byte[] encrypted = encrypt.update(unencrypted);
                        encryptedStuff[i] = encrypted;
                    }

                    decrypt.init(Cipher.DECRYPT_MODE, key, iv);
                    for (int i = 0; i < LOOPS; i++) {
                        byte[] decrypted = decrypt.update(encryptedStuff[i]);
                        byte[] original = unencryptedStuff[i];

                        if (!Arrays.equals(original, decrypted)) {
                            throw new Exception("Array mismatch. " + Arrays.toString(original) + " " + Arrays.toString(decrypted));
                        }
                    }
                }
            }
        }
        ---------- END SOURCE ----------

        CUSTOMER SUBMITTED WORKAROUND :
        Set -XX:UseAVX=2 or -XX:+UnlockDiagnosticVMOptions -XX:-UseAESCTRIntrinsics

        FREQUENCY : often


              svkamath Smita Kamath (Inactive)
              webbuggrp Webbug Group
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

                Created:
                Updated:
                Resolved: