Thread suspension (Java-level or JVM TI) is ultimately implemented by JavaThread::java_suspend() which basically uses a safepoint VM op to notify the target thread(s) of the suspension request:

 VM_ThreadSuspend vm_suspend;
 VMThread::execute(&vm_suspend);

When a thread blocks for the safepoint it will execute SafepointSynchronize::block and as it is released from the safepoint it will execute:

 // Check for pending. async. exceptions or suspends - except if the
  // thread was blocked inside the VM. has_special_runtime_exit_condition()
  // is called last since it grabs a lock and we only want to do that when
  // we must.
  ...
    if (state != _thread_blocked_trans &&
      state != _thread_in_vm_trans &&
      thread->has_special_runtime_exit_condition()) {
    thread->handle_special_runtime_exit_condition(
      !thread->is_at_poll_safepoint() && (state != _thread_in_native_trans));
  }

and in turn handle_special_runtime_exit_condition will execute java_suspend_self() if a suspend request is pending.

If the target of the suspend request is the current thread the path is somewhat different. VMThread::execute will cause the current thread to block in the VM and so it does not execute Safepoint::block and so will not see the suspend request. Further the ThreadBlockInVM destructor which transitions the thread back to _thread_in_vm does not check for pending suspend requests or async exceptions, and so the call to java_suspend() will actually return even though the current thread has been suspended.

For java.lang.Thread.suspend this is not a problem because the return to Java via ThreadInVMFromJava will call handle_special_runtime_exit_condition() which will do the self-suspend.

But for JVM TI SuspendThread, from application native code, we use an entry point that does a ThreadInVMFromNative, which checks for suspension going into the VM but not going out again. Consequently the suspended thread will continue to execute until such time as it does a transition that does check for the pending suspend request.

Similarly, if we call JVM TI SuspendThread from a JVM TI callback, then we start with _thread_in_vm, and use a JvmtiJavaThreadEventTransition which uses a ThreadToNativeFromVM transition, which again checks for suspension going in, but not coming out.

We must have the check on the exit path for self-suspend to occur.

There is a simple fix however - in java_suspend():

+ if (Thread::current() == this) {
+ // Safely self-suspend.
+ // If we don't do this explicitly it will implicitly happen
+ // before we transition back to Java, and on some other thread-state
+ // transition paths, but not as we exit a JVM TI SuspendThread call.
+ // As SuspendThread(current) must not return (until resumed) we must
+ // self-suspend here.
+ ThreadBlockInVM tbivm(this);
+ java_suspend_self();
+ } else {
+ VM_ThreadSuspend vm_suspend;
+ VMThread::execute(&vm_suspend);
+ }