A DESCRIPTION OF THE PROBLEM :
When performing Diffie-Hellman key agreement for SSL/TLS, the TLS 1.3 specification (RFC 8446) says that "The negotiated key (Z) is converted to a byte string by encoding in big-endian form and left-padded with zeros up to the size of the prime. [...] Note that this construction differs from previous versions of TLS which removed leading zeros."

However, since JDK-8014618, com.sun.crypto.provider.DHKeyAgreement.java strips leading zero bytes. This causes approximately 1 out 256 TLS1.3 handshakes using ffdhe named groups to fail (when the leading byte happens, by chance, to be zero).


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Start a simple JSSE socket server with -Djavax.net.debug=all.

2. Connect to the server with e.g. OpenSSL command line tool, ensuring that ffdhe group gets selected (e.g. " openssl s_client -groups ffdhe2048 -connect 192.168.81.1:9999 " ) repeatedly. Other SSL clients can be used -- this is not an OpenSSL bug (see below).

3. Repeat the connection. After a couple of hundred successful connections, the connection will fail with "decryption failed or bad record mac".

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Expected result is that every connection succeed.
ACTUAL -
Roughly one out of 256 connections fail.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
Java server:

import javax.net.ssl.SSLServerSocket;
import javax.net.ssl.SSLServerSocketFactory;
import javax.net.ssl.SSLSocket;

public class TestServer {
    public static void main(String args[]) throws Exception {
SSLServerSocketFactory ssf = (SSLServerSocketFactory) SSLServerSocketFactory.getDefault();
SSLServerSocket ss = (SSLServerSocket) ssf.createServerSocket(9999);
System.out.println( " Listening on port 9999 " );
while (true) {
    SSLSocket s = (SSLSocket) ss.accept();
    System.out.println( " Connected with " +s.getSession().getCipherSuite());
    s.close();
}
    }
}

Run as as follows:

keytool -storepass " password " -keypass " password " -genkey -keyalg RSA -keystore test_keystore.jks -dname CN=test
javac TestServer.java
java -Djavax.net.debug=all -Djavax.net.ssl.keyStore=./test_keystore.jks -Djavax.net.ssl.keyStorePassword=password TestServer

OpenSSL client:

set -e
while true; do
  openssl s_client -groups ffdhe2048 -connect 127.0.0.1:9999 -quiet -no_ign_eof < /dev/null
done



---------- END SOURCE ----------