Intermittent SSL handshake error with DH-based ciphers

FULL PRODUCT VERSION :
java version " 1.7.0_21 "
Java(TM) SE Runtime Environment (build 1.7.0_21-b12)
Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
I can reproduce this on my mac:

Darwin bebop.local 12.3.0 Darwin Kernel Version 12.3.0: Sun Jan 6 22:37:10 PST 2013; root:xnu-2050.22.13~1/RELEASE_X86_64 x86_64

My users have seen this on Ubuntu 12.04.2 LTS with JDK 7u15-2.3.7-0ubuntu1~12.04.1

My users have also seen this on RHEL 6.4 with java-1.7.0-oracle-1.7.0.17-1jpp.1.el6_4.x86_64

A DESCRIPTION OF THE PROBLEM :
The best description is in the following forum thread:

https://forums.oracle.com/forums/thread.jspa?messageID=10999587

When running an SSL server on the JVM using any JDK beyond 1.7u5, clients will eventually (around 5% of the time in my testing) receive a handshake error due to a problem with handling padding in the Diffie Hellman ciphers. I've included precise steps to reproduce the issue along with this ticket.

The problem goes away completely if one chooses non-diffie-hellman cipher suites. This, however, doesn't solve the problem for people that require DHE for business reasons.

ActiveMQ Apollo has run into the same issue:

https://issues.apache.org/jira/browse/APLO-287

As has PuppetDB:

http://projects.puppetlabs.com/issues/19884

REGRESSION. Last worked in version 6u45

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Here is a setup that replicates the problem. Create a keypair in a java keystore, and run this server:

package edu.berkeley.ist.scratch;

import javax.net.ssl.SSLServerSocket;
import javax.net.ssl.SSLServerSocketFactory;
import javax.net.ssl.SSLSocket;
import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;

public class EchoSSLServer {
public static void main(String[] arstring) {
try {
SSLServerSocketFactory sslserversocketfactory =
(SSLServerSocketFactory) SSLServerSocketFactory.getDefault();
SSLServerSocket sslserversocket =
(SSLServerSocket) sslserversocketfactory.createServerSocket(9999);

while (true) {
SSLSocket sslsocket = (SSLSocket) sslserversocket.accept();
InputStream inputstream = sslsocket.getInputStream();
InputStreamReader inputstreamreader = new InputStreamReader(inputstream);
BufferedReader bufferedreader = new BufferedReader(inputstreamreader);
String string = null;
while ((string = bufferedreader.readLine()) != null) {
System.out.println(string);
System.out.flush();
}
}
} catch (Exception exception) {
exception.printStackTrace();
}
}
}

Then run this bash program; it will repeatedly connect to the above server:

count=0
while [ 1 -eq 1 ]
do
echo " Hello " | openssl s_client -connect localhost:9999 >/dev/null
if [ $? -ne 0 ]; then
exit 1
fi
((count+=1))
echo $count
done

After a few hundered iterations, you will see something like this on the client side:

failed connect or ssl handshake: ,IO::Socket::INET6 configuration failederror:00000000:lib(0):func(0):reason(0) at ssl-client.pl line 6.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
I'd expect zero handshake errors.
ACTUAL -
After a few hundered iterations, you will see something like this on the client side:

failed connect or ssl handshake: ,IO::Socket::INET6 configuration failederror:00000000:lib(0):func(0):reason(0) at ssl-client.pl line 6.

So a certain percentage of the time, clients will not be able to connect to the JVM SSL socket.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
Given the above code to reproduce the problem, you see the following error on the JVM side:

And this on the server side:

javax.net.ssl.SSLHandshakeException: Invalid Padding length: 81
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1886)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:974)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:882)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:154)
at java.io.BufferedReader.readLine(BufferedReader.java:317)
at java.io.BufferedReader.readLine(BufferedReader.java:382)
at edu.berkeley.ist.scratch.EchoSSLServer.main(EchoSSLServer.java:67)
Caused by: javax.crypto.BadPaddingException: Invalid Padding length: 81
at sun.security.ssl.CipherBox.removePadding(CipherBox.java:684)
at sun.security.ssl.CipherBox.decrypt(CipherBox.java:423)
at sun.security.ssl.InputRecord.decrypt(InputRecord.java:154)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:969)
... 11 more
After enabling debug logging for SSL handshake, I see this:

main, READ: SSL v2, contentType = Handshake, translated length = 89
*** ClientHello, TLSv1
RandomCookie: GMT: 0 bytes = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 93, 75, 180, 53, 239, 108, 51, 69, 35, 182, 61, 245, 71, 68, 28, 211 }
Session ID: {}
Cipher Suites: [TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_SEED_CBC_SHA, TLS_DHE_DSS_WITH_SEED_CBC_SHA, TLS_RSA_WITH_SEED_CBC_SHA, SSL_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_MD5, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_RSA_EXPORT_WITH_RC2_CBC_40_MD5, SSL_RSA_EXPORT_WITH_RC4_40_MD5, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods: { 0 }
***
%% Initialized: [Session-232, SSL_NULL_WITH_NULL_NULL]
%% Negotiating: [Session-232, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA]
*** ServerHello, TLSv1
RandomCookie: GMT: 1361908238 bytes = { 210, 137, 20, 192, 173, 40, 205, 3, 242, 185, 80, 19, 198, 183, 44, 63, 3, 149, 97, 175, 153, 239, 243, 97, 92, 200, 110, 212 }
Session ID: {81, 45, 18, 14, 137, 154, 98, 252, 20, 124, 81, 4, 214, 8, 231, 121, 175, 133, 142, 252, 20, 12, 99, 201, 24, 9, 83, 15, 239, 34, 39, 61}
Cipher Suite: SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA
Compression Method: 0
Extension renegotiation_info, renegotiated_connection: <empty>
***
Cipher suite: SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA
*** Certificate chain
chain [0] = [
[
Version: V3
  Subject: CN=localhost, OU=IST, O= " University of California, Berkeley " , L=Berkeley, ST=CA, C=US
Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11

Key: Sun RSA public key, 1024 bits
modulus: 100318692587772653487949066254487989918157604545853798061718708748206860158041498924763974217777890645501700365160396601829010852095657193155138078345180085242824268946142257050055468216742995753689139662471328635164222926500994444770172537176817688500963591823961418447303232823489913199105654849046538472289
public exponent: 65537
Validity: [From: Mon Feb 25 11:43:44 PST 2013,
  To: Thu Feb 20 11:43:44 PST 2014]
Issuer: CN=localhost, OU=IST, O= " University of California, Berkeley " , L=Berkeley, ST=CA, C=US
SerialNumber: [ 2cba5851]

Certificate Extensions: 1
[1]: ObjectId: 2.5.29.14 Criticality=false
  SubjectKeyIdentifier [
KeyIdentifier [
0000: EC 0B B0 6A 3F 7E D1 9A FD 97 E7 DD C5 7C 00 1E ...j?...........
0010: BD 35 5D 31 .5]1
]
]

]
Algorithm: [SHA256withRSA]
Signature:
0000: 4D C2 29 84 58 2B D6 D2 47 D9 A9 45 68 54 80 9C M.).X+..G..EhT..
0010: 9F CD B1 4D FD 9E B6 AC 74 DF F8 B2 B6 16 BB 9C ...M....t.......
0020: 55 F9 60 DF 3E 86 9E AE 84 65 DC 59 86 92 84 EC U.`.>....e.Y....
0030: F7 7F D2 EE 39 53 26 87 3F B8 69 9E 58 20 C5 75 ....9S&.?.i.X .u
0040: A8 38 23 78 85 2D 61 F8 19 15 8F 5F C6 BF 08 6C .8#x.-a...._...l
0050: DA AD F5 C3 B2 84 67 B3 D4 A2 3C 88 D8 B4 15 15 ......g...<.....
0060: 10 10 B5 D1 8F 93 2F F4 62 D8 A2 24 74 C2 27 1F ....../.b..$t.'.
0070: 33 E4 1C 6A 0C 61 1F DB DC E5 4C F3 B0 83 46 CB 3..j.a....L...F.

]
***
*** Diffie-Hellman ServerKeyExchange
DH Modulus: { 233, 230, 66, 89, 157, 53, 95, 55, 201, 127, 253, 53, 103, 18, 11, 142, 37, 201, 205, 67, 233, 39, 179, 169, 103, 15, 190, 197, 216, 144, 20, 25, 34, 210, 195, 179, 173, 36, 128, 9, 55, 153, 134, 157, 30, 132, 106, 171, 73, 250, 176, 173, 38, 210, 206, 106, 34, 33, 157, 71, 11, 206, 125, 119, 125, 74, 33, 251, 233, 194, 112, 181, 127, 96, 112, 2, 243, 206, 248, 57, 54, 148, 207, 69, 238, 54, 136, 193, 26, 140, 86, 171, 18, 122, 61, 175 }
DH Base: { 48, 71, 10, 213, 160, 5, 251, 20, 206, 45, 157, 205, 135, 227, 139, 199, 209, 177, 197, 250, 203, 174, 203, 233, 95, 25, 10, 167, 163, 29, 35, 196, 219, 188, 190, 6, 23, 69, 68, 64, 26, 91, 44, 2, 9, 101, 216, 194, 189, 33, 113, 211, 102, 132, 69, 119, 31, 116, 186, 8, 77, 32, 41, 216, 60, 28, 21, 133, 71, 243, 169, 241, 162, 113, 91, 226, 61, 81, 174, 77, 62, 90, 31, 106, 112, 100, 243, 22, 147, 58, 52, 109, 63, 82, 146, 82 }
Server DH Public Key: { 172, 67, 73, 181, 25, 206, 209, 77, 79, 31, 226, 46, 83, 69, 140, 140, 22, 195, 111, 42, 172, 99, 197, 235, 173, 157, 97, 136, 79, 75, 116, 206, 222, 83, 158, 57, 134, 161, 188, 133, 1, 207, 15, 220, 98, 36, 187, 164, 222, 202, 244, 16, 77, 55, 212, 165, 120, 146, 239, 61, 5, 118, 28, 83, 254, 68, 1, 221, 73, 103, 218, 108, 174, 252, 122, 71, 71, 31, 195, 106, 87, 23, 53, 162, 130, 82, 44, 61, 9, 149, 25, 50, 93, 91, 9, 76 }
Signed with a DSA or RSA public key
*** ServerHelloDone
main, WRITE: TLSv1 Handshake, length = 1185
main, READ: TLSv1 Handshake, length = 102
*** ClientKeyExchange, DH
DH Public key: { 40, 45, 195, 251, 225, 147, 146, 211, 158, 138, 201, 109, 148, 41, 22, 10, 146, 233, 14, 87, 55, 145, 189, 247, 21, 113, 123, 26, 198, 8, 225, 169, 164, 202, 107, 59, 148, 133, 111, 72, 128, 82, 20, 47, 223, 39, 108, 44, 36, 17, 31, 170, 126, 254, 201, 192, 233, 192, 67, 155, 77, 194, 190, 155, 70, 169, 219, 48, 55, 236, 150, 214, 255, 183, 255, 87, 253, 228, 7, 231, 211, 72, 14, 159, 156, 34, 22, 133, 64, 21, 163, 97, 80, 3, 67, 165 }
SESSION KEYGEN:
PreMaster Secret:
0000: 00 90 1E 38 9D 9E 79 DD 7B 1F B5 0C 7A 58 82 38 ...8..y.....zX.8
0010: 8E 19 8A CD A0 A3 EF A1 DC 3B B1 3E FC 35 C8 97 .........;.>.5..
0020: 1E AF 62 4A 7C 95 52 6B A0 8E A6 94 25 D1 20 06 ..bJ..Rk....%. .
0030: 77 6B 7A 19 7A C7 D8 12 DF 61 97 5E 8E 10 40 E2 wkz.z....a.^..@.
0040: 3B FA A5 64 9B EA D8 50 9C 35 84 36 31 12 04 B2 ;..d...P.5.61...
0050: 4D CE 33 BB 9D E1 8A 2C 99 38 13 99 78 75 8F BF M.3....,.8..xu..
CONNECTION KEYGEN:
Client Nonce:
0000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0010: 5D 4B B4 35 EF 6C 33 45 23 B6 3D F5 47 44 1C D3 ]K.5.l3E#.=.GD..
Server Nonce:
0000: 51 2D 12 0E D2 89 14 C0 AD 28 CD 03 F2 B9 50 13 Q-.......(....P.
0010: C6 B7 2C 3F 03 95 61 AF 99 EF F3 61 5C C8 6E D4 ..,?..a....a\.n.
Master Secret:
0000: 6D 72 A2 A0 98 23 31 53 0F 02 E9 8A 95 8A 6E B2 mr...#1S......n.
0010: 4E 61 4F 04 37 16 4C FB D5 3C DC 89 38 5A 30 37 NaO.7.L..<..8Z07
0020: AB E3 E0 4D 36 01 68 1B C1 D6 33 A7 C0 CB FD 1C ...M6.h...3.....
Client MAC write Secret:
0000: 38 45 A5 8C BA 1D 04 9E 19 21 3F C1 2F 1C A0 82 8E.......!?./...
0010: 93 8E 89 88 ....
Server MAC write Secret:
0000: C8 26 BC FA 05 EC DA 66 56 98 31 87 79 87 F3 37 .&.....fV.1.y..7
0010: CC 7B 09 A3 ....
Client write key:
0000: EA E5 A4 37 30 55 20 61 14 51 09 DF 50 EC 8A 94 ...70U a.Q..P...
0010: E4 4A BD 84 0B D4 7E 9D .J......
Server write key:
0000: A3 5C C0 D0 03 49 8E BA 1C 62 88 BF 98 61 10 CC .\...I...b...a..
0010: 57 E6 72 BB 3B 88 9F 3E W.r.;..>
Client write IV:
0000: 64 A7 95 AA F7 61 82 7B d....a..
Server write IV:
0000: A8 62 ED B7 C9 8C 1D 76 .b.....v
main, READ: TLSv1 Change Cipher Spec, length = 1
main, READ: TLSv1 Handshake, length = 40
%% Invalidated: [Session-232, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA]
main, SEND TLSv1 ALERT: fatal, description = handshake_failure
main, WRITE: TLSv1 Alert, length = 2
main, called closeSocket()
main, handling exception: javax.net.ssl.SSLHandshakeException: Invalid Padding length: 81
javax.net.ssl.SSLHandshakeException: Invalid Padding length: 81
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1886)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:974)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:882)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:154)
at java.io.BufferedReader.readLine(BufferedReader.java:317)
at java.io.BufferedReader.readLine(BufferedReader.java:382)
at edu.berkeley.ist.scratch.EchoSSLServer.main(EchoSSLServer.java:67)
Caused by: javax.crypto.BadPaddingException: Invalid Padding length: 81
at sun.security.ssl.CipherBox.removePadding(CipherBox.java:684)
at sun.security.ssl.CipherBox.decrypt(CipherBox.java:423)
at sun.security.ssl.InputRecord.decrypt(InputRecord.java:154)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:969)
... 11 more


REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
package ssl.bug;

import javax.net.ssl.SSLServerSocket;
import javax.net.ssl.SSLServerSocketFactory;
import javax.net.ssl.SSLSocket;
import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;

public class EchoSSLServer {
public static void main(String[] arstring) {
try {
SSLServerSocketFactory sslserversocketfactory =
(SSLServerSocketFactory) SSLServerSocketFactory.getDefault();
SSLServerSocket sslserversocket =
(SSLServerSocket) sslserversocketfactory.createServerSocket(9999);

while (true) {
SSLSocket sslsocket = (SSLSocket) sslserversocket.accept();
InputStream inputstream = sslsocket.getInputStream();
InputStreamReader inputstreamreader = new InputStreamReader(inputstream);
BufferedReader bufferedreader = new BufferedReader(inputstreamreader);
String string = null;
while ((string = bufferedreader.readLine()) != null) {
System.out.println(string);
System.out.flush();
}
}
} catch (Exception exception) {
exception.printStackTrace();
}
}
}

Then run this bash program; it will repeatedly connect to the above server and will show occasional handshake errors:

count=0
while [ 1 -eq 1 ]
do
echo " Hello " | openssl s_client -connect localhost:9999 >/dev/null
if [ $? -ne 0 ]; then
exit 1
fi
((count+=1))
echo $count
done
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Disable Diffie-Hellman based cipher suites. This doesn't work in environments where DHE is required for security/certificate reasons.