Comment on security-dev, brought up that GCM is using a lot of memory.
--------



From Tim Whittington <jdk-security-dev@whittington.net.nz>
On 16/01/2015, at 9:18 pm, Anthony Scarpino <anthony.scarpino@oracle.com> wrote:
>> On Jan 15, 2015, at 12:26 PM, Florian Weimer <fweimer@redhat.com> wrote:
>>
>> Yes, they are going to help quite a bit as well. The other thing we need to fix for TLS is that
>> AES-GCM is a garbage collector stress test. Last time I looked, for each transferred byte,
>> there were four bytes allocated on the heap.
>
> What exactly are you referring to here? Is there a problem in TLS where it is using too much
> memory for AES-GCM specifically or AES-GCM itself is using too much memory? What test did
> you do to see this heap allocation or is this code inspection?
>

The GCM implementation (jdk/src/share/classes/com/sun/crypto/provider/GaloisCounterMode.java) does a lot of unnecessary data copying/allocation:

- AAD is buffered in a ByteArrayOutputStream and accessed with toByteArray() = unnecessary copy of all AAD to buffer and to new array
- on decrypt, all ciphertext is buffered in a ByteArrayOutputStream, accessed with toByteArray() = unnecessary copying of all ciphertext data to new array (twice if you don’t buy that it should buffer ciphertext)
- all of the block pad/length block operations allocate new arrays (they could all use a preallocated per-cipher buffer)
- there are a bunch of other byte[] allocations that could probably be eliminated

ByteArrayOutputStream buffer can be trivially accessed to avoid copy, and the AAD buffer can probably be avoided entirely.

I haven’t checked to see if JSSE is using the in[], out[] variants of update/doFinal to avoid copying there.

cheers
tim