Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8178374

Problematic ByteBuffer handling in CipherSpi.bufferCrypt method

XMLWordPrintable

    • b18
    • generic
    • generic
    • Verified

        Existing buffering code inside the javax.crypto.CipherSpi.bufferCrypt method has several shortcomings:
        1) does not work well with AES/GCM/NoPadding decryption where data is buffered internally until tag is verified, i.e. when doFinal() method is called
        2) high number of buffer re-allocations (see below report from external source)
        While running some performance tests and analyzing with Yourkit, Our customer noticed an excessive number of ShortBufferExceptions being thrown. After digging into the back trace provided in Yourkit, it appears that there is a logic flaw in the way that the output buffer is allocated inside the bufferCrypt method.

        Near the top of the method, the size needed in the output buffer is calculated with

        int outLenNeeded = engineGetOutputSize(inLen);

        In the third conditional block identified by the comment "output is not backed by an accessible byte[]", the output buffer is initially allocated with

        byte[] outArray = new byte\[getTempArraySize(outLenNeeded)\];

        The method "getTempArraySize" is a static method that returns the minimum between outLenNeeded and 4096. As a result, outArray is never bigger than 4096. As a result, if the needed length is greater, then the call to engineDoFinal will fail with ShortBufferException.

        Immediately after the exception is thrown, a catch block catches the exception then reallocates the output buffer to the correct size. This new output buffer is resubmitted to the engineDoFinal call. Since it is now big enough, it succeeds.

        Then end result is that a buffer is allocated that is known to fail, submitted, the exception caught, then reallocated and resubmitted. Instead, the behavior should be to allocate the correct buffer initially so that the call succeeds on it's first attempt.

        Attached is a simulation of the problem. The bufferCrypt method is unmodified from CipherSpi. The supporting methods return values that were observed when stepping thru the code.

              valeriep Valerie Peng
              gucheng Guixin Cheng
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated:
                Resolved: