GCM using more memory than necessary with in-place operations

XMLWordPrintable

    • Type: Bug
    • Resolution: Fixed
    • Priority: P2
    • 20
    • Affects Version/s: 18, 19
    • Component/s: security-libs

      With the addition of the AVX512 intrinsic that combined gctr and ghash operations, caused a check for in-place operations to require a new buffer for decryption to avoid complications with tag mismatches. This below mail thread and test showed that TLS, who uses in-place byte[] or heap bytebuffers can drive up memory usage when the system. Direct ByteBuffers are unaffected as the intrinsic does not call them directly.

      https://mail.openjdk.org/pipermail/security-dev/2022-October/032723.html
      https://mail.openjdk.org/pipermail/security-dev/2022-November/032757.html

            Assignee:
            Anthony Scarpino
            Reporter:
            Anthony Scarpino
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: