The following tests rely on the JDI ProcessAttachingConnector:

com/sun/jdi/ProcessAttachTest.java
com/sun/jdi/ReattachStressTest.java
con/sun/jdi/DataDumpTest.java

The ProcessAttachingConnector will determine the socket the debuggee is listening on by using the Attach API to fetch the sun.jdwp.listenerAddress property. This property is not setup until after the debug agent has started. In order to make sure the tests do not try to attach too early, the following code is used:

             p = pb.start();

            // Wait for the process to start
            InputStream is = p.getInputStream();
            is.read();

After this code exits the test tries to attach using the ProcessAttachingConnector as describe above.

The code above is waiting for the first letter of the debug agent's "Listening for transport..." output. Usually this is the first output from the process, so when the first character is read the debug agent is ready. However, if certain logging is enabled, or someone introduces a debugging printf during the jvm startup, it won't be the first output. This can lead to trying to attach too soon, before the sun.jdwp.listenerAddress property is ready. The result is:

 java.io.IOException: Not a debuggee, or not listening for debugger to attach
at jdk.jdi/com.sun.tools.jdi.ProcessAttachingConnector.attach(ProcessAttachingConnector.java:114)
at ProcessAttachTest.tryDebug(ProcessAttachTest.java:104)
at ProcessAttachTest.runTest(ProcessAttachTest.java:84)
at ProcessAttachTest.main(ProcessAttachTest.java:67)

JDK-8346827 was an example of this type of failure. Some error logging was added to the -XX:+UseNUMA, and this error logging was done before the debug agent was ready. It was eventually addressed by only doing the logging with logging is explicitly enabled, but the underlying issue is still there.

This issue can be fixed either by waiting explicitly for the "Listening for transport..." output, or by doing retries with a short delay in between retries.