Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-7031830

bad_record_mac failure on TLSv1.2 enabled connection with SSLEngine

XMLWordPrintable

    • b12
    • x86
    • linux, windows_xp
    • Verified

        FULL PRODUCT VERSION :
        java version "1.7.0-ea"
        Java(TM) SE Runtime Environment (build 1.7.0-ea-b134)
        Java HotSpot(TM) Client VM (build 21.0-b04, mixed mode, sharing)

        A DESCRIPTION OF THE PROBLEM :
        About sun.security.ssl.CipherBox class' "decrypt(ByteBuffer)" function : if a ByteBuffer(created by wraping a byte array) is passed to this function, it does array copy on this internal byte array without considering offset of the wrapper ByteBuffer.

        As a result, copying on internal byte array is improper so data and mac values is corrupted. It fails in mac verification with bad_record_mac error.

        ps: This problem also applies to encrypt(ByteBuffer) function of the same class.

         int decrypt(ByteBuffer bytebuffer) throws BadPaddingException
            {
                int i = bytebuffer.remaining();
                if(cipher == null)
                {
                    bytebuffer.position(bytebuffer.limit());
                    return i;
                }
                try
                {
                    int pos = bytebuffer.position();
                    ByteBuffer bytebuffer1 = bytebuffer.duplicate();
                    int k = cipher.update(bytebuffer1, bytebuffer);
                    if(k != i)
                        throw new RuntimeException(/*...*/);

        //some debug logging operation
                    if(blockSize != 0)
                    {
                        bytebuffer.position(pos);
                        k = removePadding(bytebuffer, blockSize, protocolVersion);
                        if(protocolVersion.v >= ProtocolVersion.TLS11.v)
                        {
                            if(k < blockSize)
                                throw new BadPaddingException("invalid explicit IV");
                            Object obj = null;
                            int lim = bytebuffer.limit();
                            if(bytebuffer.hasArray())
                            {
                                byte inarray[] = bytebuffer.array();
        /*HERE IS THE FAULTY PART*/
                                System.arraycopy(inarray, pos + blockSize, inarray, pos, lim - pos - blockSize);
                                bytebuffer.limit(lim - blockSize);
                            } else
                            {
                                byte inarray[] = new byte[lim - pos - blockSize];
                                bytebuffer.position(pos + blockSize);
                                bytebuffer.get(inarray);
                                bytebuffer.position(pos);
                                bytebuffer.put(inarray);
                                bytebuffer.limit(lim - blockSize);
                            }
                            lim = bytebuffer.limit();
                            bytebuffer.position(lim);
                        }
                    }
                    return k;
                }
                catch(ShortBufferException shortbufferexception)
                { /*error handling*/ }
            }

        STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
        Any TLSv1.2 protocol enabled client-server application with NIO(SSLEngine)!


        REPRODUCIBILITY :
        This bug can be reproduced always.

        CUSTOMER SUBMITTED WORKAROUND :
        System.arraycopy(inarray, pos + blockSize, inarray, pos, lim - pos - blockSize);

        part can be replaced with:

        System.arraycopy(inarray, pos + offset + blockSize, inarray, pos + offset, lim - pos - blockSize);

              wetmore Bradford Wetmore
              wetmore Bradford Wetmore
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved:
                Imported:
                Indexed: