Summary

Modify the TLS 1.3 shared secret derivation to use the Generic algorithm. This is a retroactive CSR.

Problem

TLS 1.3 changed the way the shared Diffie-Hellman secret is derived, The negotiated key is now left-padded with zeros up to the size of the prime. TLS 1.2 stripped the leading zeros. See RFC 8446, section 7.4.1. JSSE uses the TLS 1.2 key derivation method for all TLS handshakes. This causes roughly 1 in 256 handshakes between JSSE and a compliant implementation using TLS 1.3 and DHE to fail.

Handshakes between two JSSE implementations complete normally, as both implementations derive the same incorrect secret. Handshakes using elliptic curves (ECDHE) are not affected.

Solution

Use the Generic key derivation algorithm instead of TlsPremasterSecret for all TLS 1.3 Diffie-Hellman exchanges, including both DHE and ECDHE. That algorithm returns the full (padded) shared secret, and is compatible with key derivation functions used by TLS 1.3.

Alternative solutions:

• Pad the secret key after derivation. This does not change the JSSE interface, but only works when the shared key bytes are accessible. Some providers (NSS in FIPS mode) restrict access to the key bytes.
• Keep the existing code. FFDHE is rarely used, and the failure is infrequent.
• Use the Generic algorithm, but fall back to the TlsPremasterSecret in case of a failure. This might require performing the key derivation twice, as the KeyAgreement#generateSecret method is specified to reset the object to the initial state.
• Use the Generic algorithm for FFDHE only, and keep using TlsPremasterSecret for ECDHE. This would require additional coding, as the information about the selected exchange type is not readily available.

Risk mitigation:

Not implemented in the original ticket. We could implement a system property jdk.tls.legacyKeyDerivation to fall back to the old derivation algorithm.

Specification

The algorithm type used by JSSE is not defined in the specification.