This observed once in jdk 9 dev nightly. Re-run it 10 times on the same machine with the same build but not reproduced.

Snippet of test error:
***
*** Testing configuration (port=62217): /scratch/export/home/aurora/sandbox/gresults/testoutput/jdk_management/JTwork/classes/sun/management/jmxremote/bootstrap/management_ssltest11_ok.properties
***
Connection failed for no credentials: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
Test Failed: Failed to connect to agent {url=service:jmx:rmi://scaaa608.us.oracle.com/jndi/rmi://localhost:62217/jmxrmi}: java.lang.RuntimeException: Test /scratch/export/home/aurora/sandbox/gresults/testoutput/jdk_management/JTwork/classes/sun/management/jmxremote/bootstrap/management_ssltest11_ok.properties failed with 1 error(s)