SunJCE support of password-based encryption scheme 2 params (PBES2) not working

FULL PRODUCT VERSION :
Java version "1.8.0_40"
Java(TM) SE Runtime Environment (build 1.8.0_40-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)


ADDITIONAL OS VERSION INFORMATION :
Independent of Operating System

EXTRA RELEVANT SYSTEM CONFIGURATION :
Tested with both OpenJDK and Oracle JDK.

A DESCRIPTION OF THE PROBLEM :
The class com.sun.crypto.provider.PBES2Parameters, is supposed to provide implementation for PBES2-params as defined in PKCS#5. Hopelessly, the current implementation has several issues. This is a new implementation introduced by JDK-6383200.

1) it provides decoding of PBES2Algorithms in place of PBES2-params. The PBES2Parameters should not decode the id_PBES2 algorithm identifier itself, but only its parameters, so a sequence of 2 items (PBES2-KDFs and PBES2-Enc). This is not the algorithm but its parameters that should be implemented here.

2) the decoding of PBKDF2-params is not implemented properly. The prf AlgorithmIdentifier of the derivation function could be defaulted to algid-hmacWithSHA1. So it means that after reading the iterationCount, it may happen that no more data is available. Moreover (pBKDF2_params.tag == DerValue.tag_Sequence) is always true and unrelated here. So actually, the optional key length is unsupported

3) the usage of the default prf is not supported, due to the same issue above.

4) The current implementation only support certain algorithm for the cipher (AES only). While in JDK7 the decoding wasn't done, any algo was supported, so it is already a regression. Moreover, as a general behavior, when it is not supported by this provider does not means that it cannot be hold in this parameter structure for being used by other providers later.

5) Encoding of PBES2Parameters produce the full algorithm identifier with params, in place of just the params alone.

PS: Sorry if all these should have been sent as separate issues, but without direct access to JIRA, it would have been a pain to do, without any way to link information together properly, causing tons of repetition and potential misunderstanding.

REGRESSION. Last worked in version 7u76

ADDITIONAL REGRESSION INFORMATION:
The first test case works properly in older versions, since parameters were not decoded.

java version "1.7.0_76"
Java(TM) SE Runtime Environment (build 1.7.0_76-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.76-b04, mixed mode)


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Execute the test cases provided.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
No exception, nor any output.
Note that these tests case use internal proprietary API for more direct testing, but that javax.crypto.EncryptedPrivateKeyInfo use this same code path for algorithm and parameter decoding.
ACTUAL -
1) Currently fails here because the expectation of PBES2Parameters is also an algorithm id.
java.io.IOException: ObjectIdentifier() -- data isn't an object ID (tag = 48)
at sun.security.util.ObjectIdentifier.<init>(ObjectIdentifier.java:253)
at sun.security.util.DerInputStream.getOID(DerInputStream.java:281)
at com.sun.crypto.provider.PBES2Parameters.engineInit(PBES2Parameters.java:267)
at java.security.AlgorithmParameters.init(AlgorithmParameters.java:293)
at sun.security.x509.AlgorithmId.decodeParams(AlgorithmId.java:132)
at sun.security.x509.AlgorithmId.<init>(AlgorithmId.java:114)
at sun.security.x509.AlgorithmId.parse(AlgorithmId.java:372)
at PBES2AlgorithmId.main(PBES2AlgorithmId.java:218)
1) Currently fails here because the expectation of PBES2Parameters is also an algorithm id.
java.io.IOException: DerInputStream.getLength(): lengthTag=127, too big.
at sun.security.util.DerInputStream.getLength(DerInputStream.java:561)
at sun.security.util.DerValue.<init>(DerValue.java:252)
at sun.security.util.DerInputStream.getDerValue(DerInputStream.java:417)
at com.sun.crypto.provider.PBES2Parameters.parseKDF(PBES2Parameters.java:319)
at com.sun.crypto.provider.PBES2Parameters.engineInit(PBES2Parameters.java:281)
at java.security.AlgorithmParameters.init(AlgorithmParameters.java:293)
at PBES2AlgorithmId.main(PBES2AlgorithmId.java:227)
2) Currently fails here because PBES2Parameters does not support optional Keylength properly.
java.io.IOException: DerInputStream.getLength(): lengthTag=127, too big.
at sun.security.util.DerInputStream.getLength(DerInputStream.java:561)
at sun.security.util.DerValue.<init>(DerValue.java:252)
at sun.security.util.DerInputStream.getDerValue(DerInputStream.java:417)
at com.sun.crypto.provider.PBES2Parameters.parseKDF(PBES2Parameters.java:324)
at com.sun.crypto.provider.PBES2Parameters.engineInit(PBES2Parameters.java:281)
at java.security.AlgorithmParameters.init(AlgorithmParameters.java:293)
at PBES2AlgorithmId.main(PBES2AlgorithmId.java:235)
3) Currently fails here because PBES2Parameters does not support default prf.
java.io.IOException: DerInputStream.getLength(): lengthTag=127, too big.
at sun.security.util.DerInputStream.getLength(DerInputStream.java:561)
at sun.security.util.DerValue.<init>(DerValue.java:252)
at sun.security.util.DerInputStream.getDerValue(DerInputStream.java:417)
at com.sun.crypto.provider.PBES2Parameters.parseKDF(PBES2Parameters.java:324)
at com.sun.crypto.provider.PBES2Parameters.engineInit(PBES2Parameters.java:281)
at java.security.AlgorithmParameters.init(AlgorithmParameters.java:293)
at PBES2AlgorithmId.main(PBES2AlgorithmId.java:243)
4) Currently fails here because PBES2Parameters does not support non AES ciphers.
java.io.IOException: PBE parameter parsing error: expecting the object identifier for AES cipher
at com.sun.crypto.provider.PBES2Parameters.parseES(PBES2Parameters.java:371)
at com.sun.crypto.provider.PBES2Parameters.engineInit(PBES2Parameters.java:287)
at java.security.AlgorithmParameters.init(AlgorithmParameters.java:293)
at PBES2AlgorithmId.main(PBES2AlgorithmId.java:251)
5) Currently fails here because PBES2Parameters does not encode params properly.
Expected:[48, 77, 48, 44, 6, 9, 42, -122, 72, -122, -9, 13, 1, 5, 12, 48, 31, 4, 8, 24, 23, -27, -87, -26, 69, -49, -53, 2, 2, 8, 0, 2, 1, 16, 48, 12, 6, 8, 42, -122, 72, -122, -9, 13, 2, 7, 5, 0, 48, 29, 6, 9, 96, -122, 72, 1, 101, 3, 4, 1, 2, 4, 16, 11, 109, 58, -68, -102, -12, 59, 64, 11, 109, 58, -68, -102, -12, 59, 64]
Actual:[48, 90, 6, 9, 42, -122, 72, -122, -9, 13, 1, 5, 13, 48, 77, 48, 44, 6, 9, 42, -122, 72, -122, -9, 13, 1, 5, 12, 48, 31, 4, 8, 24, 23, -27, -87, -26, 69, -49, -53, 2, 2, 8, 0, 2, 1, 16, 48, 12, 6, 8, 42, -122, 72, -122, -9, 13, 2, 7, 5, 0, 48, 29, 6, 9, 96, -122, 72, 1, 101, 3, 4, 1, 2, 4, 16, 11, 109, 58, -68, -102, -12, 59, 64, 11, 109, 58, -68, -102, -12, 59, 64]
Actual is the full algorithm id, not the params alone.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.security.AlgorithmParameters;
import java.util.Arrays;

import sun.security.util.DerValue;
import sun.security.x509.AlgorithmId;

/*
 * @test
 * @summary Bad encoding/decoding of password-based encryption scheme 2 parameters (PBES2)
 */
public class PBES2AlgorithmId {
    public static void main(String[] args) throws Exception
    {
        final byte[] PBES2_PARAM = new byte[]{
            // SEQUENCE(2 elem)
            0x30, 0x3D,
            // OBJECT IDENTIFIER 1.2.840.113549.1.5.13
            0x06, 0x09, 0x2A, (byte) 0x86, 0x48, (byte) 0x86, (byte) 0xF7, 0x0D, 0x01, 0x05, 0x0D,
            // SEQUENCE(2 elem)
            0x30, 0x30,
            // SEQUENCE(2 elem)
            0x30, 0x1B,
            // OBJECT IDENTIFIER 1.2.840.113549.1.5.12
            0x06, 0x09, 0x2A, (byte) 0x86, 0x48, (byte) 0x86, (byte) 0xF7, 0x0D, 0x01, 0x05, 0x0C,
            // SEQUENCE(2 elem)
            0x30, 0x0E,
            // OCTET STRING(8 bytes) 1817E5A9E645CFCB
            0x04, 0x08, 0x18, 0x17, (byte) 0xE5, (byte) 0xA9, (byte) 0xE6, 0x45, (byte) 0xCF, (byte) 0xCB,
            // INTEGER 2048
            0x02, 0x02, 0x08, 0x00,
            // SEQUENCE(2 elem)
            0x30, 0x11,
            // OBJECT IDENTIFIER 1.3.14.3.2.7
            0x06, 0x05, 0x2B, 0x0E, 0x03, 0x02, 0x07,
            // OCTET STRING(8 byte) 0B6D3ABC9AF43B40
            0x04, 0x08, 0x0B, 0x6D, 0x3A, (byte) 0xBC, (byte) 0x9A, (byte) 0xF4, 0x3B, 0x40
        };

        final byte[] PBES2_ALG_AND_PARAM_WITHOUT_KEYLENGTH = new byte[]{
            // SEQUENCE(2 elem)
            0x30, 0x4B,
            // OBJECT IDENTIFIER 1.2.840.113549.1.5.13
            0x06, 0x09, 0x2A, (byte) 0x86, 0x48, (byte) 0x86, (byte) 0xF7, 0x0D, 0x01, 0x05, 0x0D,
            // SEQUENCE(2 elem)
            0x30, 0x3E,
            // SEQUENCE(2 elem)
            0x30, 0x29,
            // OBJECT IDENTIFIER 1.2.840.113549.1.5.12
            0x06, 0x09, 0x2A, (byte) 0x86, 0x48, (byte) 0x86, (byte) 0xF7, 0x0D, 0x01, 0x05, 0x0C,
            // SEQUENCE(2 elem)
            0x30, 0x1C,
            // OCTET STRING(8 bytes) 1817E5A9E645CFCB
            0x04, 0x08, 0x18, 0x17, (byte) 0xE5, (byte) 0xA9, (byte) 0xE6, 0x45, (byte) 0xCF, (byte) 0xCB,
            // INTEGER 2048
            0x02, 0x02, 0x08, 0x00,
            // SEQUENCE(2 elem)
            0x30, 0x0C,
            // OBJECT IDENTIFIER 1.2.840.113549.2.7
            0x06, 0x08, 0x2A, (byte) 0x86, 0x48, (byte) 0x86, (byte) 0xF7, 0x0D, 0x02, 0x07,
            // NULL
            0x05, 0x00,
            // SEQUENCE(2 elem)
            0x30, 0x11,
            // OBJECT IDENTIFIER 1.3.14.3.2.7
            0x06, 0x05, 0x2B, 0x0E, 0x03, 0x02, 0x07,
            // OCTET STRING(8 byte) 0B6D3ABC9AF43B40
            0x04, 0x08, 0x0B, 0x6D, 0x3A, (byte) 0xBC, (byte) 0x9A, (byte) 0xF4, 0x3B, 0x40
        };

        final byte[] PBES2_ALG_AND_PARAM_WITHOUT_PRF = new byte[]{
            // SEQUENCE(2 elem)
            0x30, 0x40,
            // OBJECT IDENTIFIER 1.2.840.113549.1.5.13
            0x06, 0x09, 0x2A, (byte) 0x86, 0x48, (byte) 0x86, (byte) 0xF7, 0x0D, 0x01, 0x05, 0x0D,
            // SEQUENCE(2 elem)
            0x30, 0x33,
            // SEQUENCE(2 elem)
            0x30, 0x1E,
            // OBJECT IDENTIFIER 1.2.840.113549.1.5.12
            0x06, 0x09, 0x2A, (byte) 0x86, 0x48, (byte) 0x86, (byte) 0xF7, 0x0D, 0x01, 0x05, 0x0C,
            // SEQUENCE(2 elem)
            0x30, 0x11,
            // OCTET STRING(8 bytes) 1817E5A9E645CFCB
            0x04, 0x08, 0x18, 0x17, (byte) 0xE5, (byte) 0xA9, (byte) 0xE6, 0x45, (byte) 0xCF, (byte) 0xCB,
            // INTEGER 2048
            0x02, 0x02, 0x08, 0x00,
            // INTEGER 16
            0x02, 0x01, 0x10,
            // SEQUENCE(2 elem)
            0x30, 0x11,
            // OBJECT IDENTIFIER 1.3.14.3.2.7
            0x06, 0x05, 0x2B, 0x0E, 0x03, 0x02, 0x07,
            // OCTET STRING(8 byte) 0B6D3ABC9AF43B40
            0x04, 0x08, 0x0B, 0x6D, 0x3A, (byte) 0xBC, (byte) 0x9A, (byte) 0xF4, 0x3B, 0x40
        };

        final byte[] PBES2_ALG_AND_PARAM_WITH_KEYLENGHT_AND_PRF_BUT_NOT_AN_AES_CIPHER = new byte[]{
            // SEQUENCE(2 elem)
            0x30, 0x4E,
            // OBJECT IDENTIFIER 1.2.840.113549.1.5.13
            0x06, 0x09, 0x2A, (byte) 0x86, 0x48, (byte) 0x86, (byte) 0xF7, 0x0D, 0x01, 0x05, 0x0D,
            // SEQUENCE(2 elem)
            0x30, 0x41,
            // SEQUENCE(2 elem)
            0x30, 0x2C,
            // OBJECT IDENTIFIER 1.2.840.113549.1.5.12
            0x06, 0x09, 0x2A, (byte) 0x86, 0x48, (byte) 0x86, (byte) 0xF7, 0x0D, 0x01, 0x05, 0x0C,
            // SEQUENCE(2 elem)
            0x30, 0x1F,
            // OCTET STRING(8 bytes) 1817E5A9E645CFCB
            0x04, 0x08, 0x18, 0x17, (byte) 0xE5, (byte) 0xA9, (byte) 0xE6, 0x45, (byte) 0xCF, (byte) 0xCB,
            // INTEGER 2048
            0x02, 0x02, 0x08, 0x00,
            // INTEGER 16
            0x02, 0x01, 0x10,
            // SEQUENCE(2 elem)
            0x30, 0x0C,
            // OBJECT IDENTIFIER 1.2.840.113549.2.7
            0x06, 0x08, 0x2A, (byte) 0x86, 0x48, (byte) 0x86, (byte) 0xF7, 0x0D, 0x02, 0x07,
            // NULL
            0x05, 0x00,
            // SEQUENCE(2 elem)
            0x30, 0x11,
            // OBJECT IDENTIFIER 1.3.14.3.2.7
            0x06, 0x05, 0x2B, 0x0E, 0x03, 0x02, 0x07,
            // OCTET STRING(8 byte) 0B6D3ABC9AF43B40
            0x04, 0x08, 0x0B, 0x6D, 0x3A, (byte) 0xBC, (byte) 0x9A, (byte) 0xF4, 0x3B, 0x40
        };

        final byte[] PBES2_ALG_AND_PARAM_WITH_KEYLENGHT_AND_PRF_AND_AES_CIPHER = new byte[]{
            // SEQUENCE(2 elem)
            0x30, 0x5A,
            // OBJECT IDENTIFIER 1.2.840.113549.1.5.13
            0x06, 0x09, 0x2A, (byte) 0x86, 0x48, (byte) 0x86, (byte) 0xF7, 0x0D, 0x01, 0x05, 0x0D,
            // SEQUENCE(2 elem)
            0x30, 0x4D,
            // SEQUENCE(2 elem)
            0x30, 0x2C,
            // OBJECT IDENTIFIER 1.2.840.113549.1.5.12
            0x06, 0x09, 0x2A, (byte) 0x86, 0x48, (byte) 0x86, (byte) 0xF7, 0x0D, 0x01, 0x05, 0x0C,
            // SEQUENCE(2 elem)
            0x30, 0x1F,
            // OCTET STRING(8 bytes) 1817E5A9E645CFCB
            0x04, 0x08, 0x18, 0x17, (byte) 0xE5, (byte) 0xA9, (byte) 0xE6, 0x45, (byte) 0xCF, (byte) 0xCB,
            // INTEGER 2048
            0x02, 0x02, 0x08, 0x00,
            // INTEGER 16
            0x02, 0x01, 0x10,
            // SEQUENCE(1 elem)
            0x30, 0x0C,
            // OBJECT IDENTIFIER 1.2.840.113549.2.7
            0x06, 0x08, 0x2A, (byte) 0x86, 0x48, (byte) 0x86, (byte) 0xF7, 0x0D, 0x02, 0x07,
            // NULL
            0x05, 0x00,
            // SEQUENCE(2 elem)
            0x30, 0x1D,
            // OBJECT IDENTIFIER 2.16.840.1.101.3.4.1.2
            0x06, 0x09, 0x60,(byte) 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x01, 0x02,
            // OCTET STRING(8 byte) 0B6D3ABC9AF43B40
            0x04, 0x10, 0x0B, 0x6D, 0x3A, (byte) 0xBC, (byte) 0x9A, (byte) 0xF4, 0x3B, 0x40,
                        0x0B, 0x6D, 0x3A, (byte) 0xBC, (byte) 0x9A, (byte) 0xF4, 0x3B, 0x40
        };

        final byte[] PBES2_PARAM_WITH_KEYLENGH_AND_PRF_AND_AES_CIPHER = new byte[]{
            // SEQUENCE(2 elem)
            0x30, 0x4D,
            // SEQUENCE(2 elem)
            0x30, 0x2C,
            // OBJECT IDENTIFIER 1.2.840.113549.1.5.12
            0x06, 0x09, 0x2A, (byte) 0x86, 0x48, (byte) 0x86, (byte) 0xF7, 0x0D, 0x01, 0x05, 0x0C,
            // SEQUENCE(2 elem)
            0x30, 0x1F,
            // OCTET STRING(8 bytes) 1817E5A9E645CFCB
            0x04, 0x08, 0x18, 0x17, (byte) 0xE5, (byte) 0xA9, (byte) 0xE6, 0x45, (byte) 0xCF, (byte) 0xCB,
            // INTEGER 2048
            0x02, 0x02, 0x08, 0x00,
            // INTEGER 16
            0x02, 0x01, 0x10,
            // SEQUENCE(1 elem)
            0x30, 0x0C,
            // OBJECT IDENTIFIER 1.2.840.113549.2.7
            0x06, 0x08, 0x2A, (byte) 0x86, 0x48, (byte) 0x86, (byte) 0xF7, 0x0D, 0x02, 0x07,
            // NULL
            0x05, 0x00,
            // SEQUENCE(2 elem)
            0x30, 0x1D,
            // OBJECT IDENTIFIER 2.16.840.1.101.3.4.1.2
            0x06, 0x09, 0x60,(byte) 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x01, 0x02,
            // OCTET STRING(8 byte) 0B6D3ABC9AF43B40
            0x04, 0x10, 0x0B, 0x6D, 0x3A, (byte) 0xBC, (byte) 0x9A, (byte) 0xF4, 0x3B, 0x40,
            0x0B, 0x6D, 0x3A, (byte) 0xBC, (byte) 0x9A, (byte) 0xF4, 0x3B, 0x40
        };

        try {
            AlgorithmId algId = AlgorithmId.parse(new DerValue(PBES2_PARAM));
        } catch (Exception e) {
            System.out.println("1) Currently fails here because the expectation of PBES2Parameters is also an algorithm id.");
            e.printStackTrace();
        }

        // WARNING !!!!
        // From here, submitting directly to PBES2Parameters the Algorithm Id, to make it happy and pursue.
        // Proper test case needs the two first line of the bytes arrays above containing the PBES2 algorithm to be
        // removed once the first issue above has been solved.
        try {
            AlgorithmParameters algParams = AlgorithmParameters.getInstance("1.2.840.113549.1.5.13","SunJCE");
            algParams.init(PBES2_PARAM);
        } catch (Exception e) {
            System.out.println("1) Currently fails here because the expectation of PBES2Parameters is also an algorithm id.");
            e.printStackTrace();
        }

        try {
            AlgorithmParameters algParams = AlgorithmParameters.getInstance("1.2.840.113549.1.5.13","SunJCE");
            algParams.init(PBES2_ALG_AND_PARAM_WITHOUT_KEYLENGTH);
        } catch (Exception e) {
            System.out.println("2) Currently fails here because PBES2Parameters does not support optional Keylength properly.");
            e.printStackTrace();
        }

        try {
            AlgorithmParameters algParams = AlgorithmParameters.getInstance("1.2.840.113549.1.5.13","SunJCE");
            algParams.init(PBES2_ALG_AND_PARAM_WITHOUT_PRF);
        } catch (Exception e) {
            System.out.println("3) Currently fails here because PBES2Parameters does not support default prf.");
            e.printStackTrace();
        }

        try {
            AlgorithmParameters algParams = AlgorithmParameters.getInstance("1.2.840.113549.1.5.13","SunJCE");
            algParams.init(PBES2_ALG_AND_PARAM_WITH_KEYLENGHT_AND_PRF_BUT_NOT_AN_AES_CIPHER);
        } catch (Exception e) {
            System.out.println("4) Currently fails here because PBES2Parameters does not support non AES ciphers.");
            e.printStackTrace();
        }

        AlgorithmParameters algParams = AlgorithmParameters.getInstance("1.2.840.113549.1.5.13","SunJCE");
        algParams.init(PBES2_ALG_AND_PARAM_WITH_KEYLENGHT_AND_PRF_AND_AES_CIPHER);
        if (!Arrays.equals(algParams.getEncoded(),PBES2_PARAM_WITH_KEYLENGH_AND_PRF_AND_AES_CIPHER)) {
            System.out.println("5) Currently fails here because PBES2Parameters does not encode params properly.");
            System.out.print("Expected:");
            System.out.println(Arrays.toString(PBES2_PARAM_WITH_KEYLENGH_AND_PRF_AND_AES_CIPHER));
            System.out.print("Actual:");
            System.out.println(Arrays.toString(algParams.getEncoded()));
            if (Arrays.equals(algParams.getEncoded(),PBES2_ALG_AND_PARAM_WITH_KEYLENGHT_AND_PRF_AND_AES_CIPHER)) {
                System.out.println("Actual is the full algorithm id, not the params alone.");
            }
        }
    }
}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Use a better provider than the SunJCE.