TLS 1.3 half-close and synchronization issues

Summary

Provide a workaround if an application does not work well with the TLS 1.3 half-close policy.

Problem

Unlike TLS 1.2 and prior versions, TLS 1.3 uses a half-close policy. In TLS 1.3, the inbound and outbound close_notify alerts are independent. While in TLS1.2 and prior versions, if one party closes its write side by sending the close notify alert (closeOutbound) , then it MUST receive a close_notify alert response from the peer and if the peer has closed its write side of connection by sending a close notify alert, then upon receipt of the alert, the party MUST send a close notify alert response.

In practice, an application may only close outbound even if it intends to close the inbound as well, or close the connection completely. These use cases work for TLS 1.2 and prior versions, but not for TLS 1.3 because of the TLS 1.3 specification changes. With TLS 1.3, the application may hang waiting for further operations. This issue could be solved by the application explicitly closing both inbound and outbound in each side of the connection, but that requires source code updates which may not be practical for many applications.

Solution

In order to mitigate the impact, a new System Property, "jdk.tls.acknowledgeCloseNotify", is introduced. If the System Property is set to "true", when receiving the close_notify alert, a corresponding close_notify alert will be sent, and the connection can be duplex closed accordingly. This is a JDK specific property that will be supported by the SunJSSE implementation.

Specification

Add a new System Property:

A new System Property, "jdk.tls.acknowledgeCloseNotify", is added. The default value of the System Property is "false". If the System Property is set to "true", a corresponding close_notify alert will be sent when receiving a close_notify alert, and the connection will be duplex closed.

TLS 1.2 and prior versions use a duplex-close policy, while TLS 1.3 uses a half-close policy. The inbound and the outbound close_notify alerts for TLS 1.3 are independent. When upgrading to TLS 1.3, unexpected behavior may occur if your application shuts down the (D)TLS connection using only one of the SSLEngine.closeInbound() or SSLEngine.closeOutbound() APIs, but not both in each side of the connection. If your application exhibits unexpected hangs or timeouts when the underlying (D)TLS transportation is not duplex closed, you may need to set this property to true.

Add a SSLSocket apiNote;

@apiNote When the connection is no longer needed, the client and server applications should each close both sides of their respective connection. For {@code SSLSocket} objects, for example, an application can call {@link Socket#shutdownOutput} or {@link OutputStream#close} to close the output stream and call {@link Socket#shutdownInput} or {@link InputStream#close} to close the input stream. Note that in some cases, closing the input stream may depend on the peer's output stream being closed first. If the connection is not closed in an orderly manner (for example {@link Socket#shutdownInput} is called before the peer's write closure notification has been received), exceptions may be raised to indicate that an error has occurred. Once an {@code SSLSocket} is closed, it is not reusable: a new {@code SSLSocket} must be created.

Update the closure note in SSLEngine:

Replace:

Closure - When the connection is no longer needed, the application should close the {@code SSLEngine} and should send/receive any remaining messages to the peer before closed, it is not reusable: a new {@code SSLEngine} must be created.

with:

Closure - When the connection is no longer needed, the client and the server applications should each close both sides of their respective connections. For {@code SSLEngine} objects, an application should call {@link SSLEngine#closeOutbound} and send any remaining messages to the peer. Likewise, an application should receive any remaining messages from the peer before calling {@link SSLEngine#closeInbound}. The underlying transport mechanism can then be closed after both sides of the {@code SSLEngine} have been closed. If the connection is not closed in an orderly manner (for example {@link SSLEngine#closeInbound} is called before the peer's write closure notification has been received), exceptions will be raised to indicate that an error has occurred. Once an engine is closed, it is not reusable: a new {@code SSLEngine} must be created.

Compatibility Risk

Low.