Summary

Fix the javax.net.debug TLS debug system property so that it operates as documented in the help output menu. Tighten the implementation to ensure that sub-options are only recognized when a valid parent option is specified. Update the help menu to remove obsolete debug options and add any newly supported ones.

Problem

Since the introduction of TLSv1.3, use of the javax.net.debug TLS debug system property has been buggy. While a significant portion of these issues was addressed in the recent JDK-8350582 fix, overly lax parsing still exists.

Specifically, the current implementation allows users to declare the verbose logging option on its own, contrary to the documentation, which states that verbose should only be used in conjunction with the handshake option. Similarly, the sub-options packet and plaintext can be specified without the required record parent option.

In general, the lax parsing allows any of the options [defaultctx, handshake, keymanager, record, respmgr, session, sslctx, trustmanager] and sub-options [verbose, packet, plaintext] to be specified independently, without enforcing that the appropriate master option is provided. Additionally, the ssl master option should be required for all these options, but this is not enforced.

The help menu (javax.net.debug=help) is also inaccurate: it still contains obsolete options, while omitting valid ones.

Solution

Update the javax.net.debug implementation to enforce the correct use of options and sub-options as stated in the help menu. Incorrect combinations will cause those options to be disregarded, and only valid option/sub-option combinations will have an effect.

A release note will highlight the enforcement of valid debug option combinations, specifying that invalid combinations will be ignored. The security documentation will also be updated accordingly.

Finally, update the help menu to accurately reflect supported options. The missing respmgr option is added; the obsolete keygen, pluggability, and (from verbose section) data options are removed. The revised help output can be seen in the Specification section.

Specification

The usage instructions displayed in the javax.net.debug help menu are now strictly enforced. Only valid option and sub-option values as displayed in the help option output will have an effect in TLS debug log generation.

E.g. for plaintext expand option, -Djavax.net.debug=plaintext would have displayed such data in the past, now that option needs to be: -Djavax.net.debug=ssl:record:plaintext

help             print this help message and exit
expand           expanded (less compact) output format

all              turn on all debugging
ssl              turn on ssl debugging

The following filters can be used with ssl:
    defaultctx     print default SSL initialization
    handshake      print each handshake message
      verbose        verbose handshake message printing (widens handshake)
    keymanager     print key manager tracing
    record         enable per-record tracing
      packet         print raw SSL/TLS packets (widens record)
      plaintext      hex dump of record plaintext (widens record)
    respmgr        print OCSP response tracing
    session        print session activity
    sessioncache   print session cache tracing
    sslctx         print SSLContext tracing
    trustmanager   print trust manager tracing

Adding valid filter options to "ssl" will log messages to include
just those filtered categories.

If "ssl" is specified by itself, all non-widening filters are enabled.