Summary

JDI spec needs to clarify when OpaqueFrameException and NativeMethodException are thrown

Problem

There is a long standing issue with JDI throwing NativeMethodException for reasons other than a native method. This happens with the ThreadReference popFrames() and forceEarlyReturn() APIs. At the JVMTI level, these APIs can produce JVMTI_ERROR_OPAQUE_FRAME, which is converted to a JDWP OPAQUE_FRAME error. JDI assumed that this can only happen due to a native method, so it converted OPAQUE_FRAME error to a NativeMethodException. However, it could happen due to other reasons specific to the JVMTI implementation. Although very rare, one example is an inability to de-opt the method.

With the addition of virtual threads, JDI introduced OpaqueFrameException for ThreadReference popFrames() and forceEarlyReturn(). This is because with virtual threads, there were other reasons why these APIs might fail. Specifically, the thread had to be suspended at a breakpoint to guarantee the API would succeed. Otherwise JVMTI could produce JVMTI_ERROR_OPAQUE_FRAME, and JDI would then throw OpaqueFrameException if it detected that a native method was not the cause of the OPAQUE_FRAME_EXCEPTION. The wording added to the spec was:

 * @throws OpaqueFrameException if this thread is a suspended virtual thread and the
 * target VM was unable to pop the frames.

 * @throws OpaqueFrameException if this thread is a suspended virtual thread and the
 * target VM is unable to force the method to return.

What was not realized at this time was the other possible reasons for JVMTI_ERROR_OPAQUE_FRAME (such as failure to de-opt), so not only was the above description not fully correct for virtual threads, but we also missed the fact that these other JVMTI_ERROR_OPAQUE_FRAME failures could apply to platform threads too. As a result, OpaqueFrameException was reserved for just virtual threads, and platforms threads continued to always throw NativeMethodException for JDWP OPAQUE_FRAME error. Basically for virtual threads some extra logic was added when JDWP OPAQUE_FRAME error was received. That logic determines if a native method was the cause of the JDWP OPAQUE_FRAME error. If it was then NativeMethodException is thrown. If not, then OpaqueFrameException is thrown.

Solution

Update the spec to clarify the meaning of OpaqueFrameException for ThreadReference popFrames() and forceEarlyReturn(), and for these two APIs change the logic that handles JDWP OPAQUE_FRAME error so always applies the native method check instead of just doing it for virtual threads.

Specification

For context, the first diff below is for ThreadReference.popFrames() and the 2nd diff is for ThreadReference.forceEarlyReturn(). You can also find the webrev at the following link, but it also includes the implementation changes: https://webrevs.openjdk.org/?repo=jdk&pr=26335&range=02

diff --git a/src/jdk.jdi/share/classes/com/sun/jdi/ThreadReference.java b/src/jdk.jdi/share/classes/com/sun/jdi/ThreadReference.java
index bccdf4cc8bf57..67b8e391bf30d 100644
--- a/src/jdk.jdi/share/classes/com/sun/jdi/ThreadReference.java
+++ b/src/jdk.jdi/share/classes/com/sun/jdi/ThreadReference.java
@@ -402,8 +402,8 @@ List<MonitorInfo> ownedMonitorsAndFrames()
      * @throws java.lang.IllegalArgumentException if <CODE>frame</CODE>
      * is not on this thread's call stack.
      *
-     * @throws OpaqueFrameException if this thread is a suspended virtual thread and the
-     * target VM was unable to pop the frames.
+     * @throws OpaqueFrameException if the target VM is unable to pop this frame
+     * (e.g. a virtual thread is suspended, but not at an event).
      *
      * @throws NativeMethodException if one of the frames that would be
      * popped is that of a native method or if the frame previous to
@@ -484,8 +484,8 @@ List<MonitorInfo> ownedMonitorsAndFrames()
      * @throws IncompatibleThreadStateException if this
      * thread is not suspended.
      *
-     * @throws OpaqueFrameException if this thread is a suspended virtual thread and the
-     * target VM is unable to force the method to return.
+     * @throws OpaqueFrameException if the target VM is unable to force the method to return
+     * (e.g. a virtual thread is suspended, but not at an event).
      *
      * @throws NativeMethodException if the frame to be returned from
      * is that of a native method.

Also note that earlier in the description of these two APIs you can find the following (or similar) describing the virtual threads support:

 * This method may be used to pop frames of a virtual thread when
 * it is suspended at an event. An implementation may support popping
 * the frames of a suspended virtual thread in other cases.