NotifyFramePop throws unexpected error: JVMTI_ERROR_THREAD_NOT_SUSPENDED

We got
  nsk/jvmti/unit/ForceEarlyReturn/earlyretbase

test failure in SPB nightly.
I was also able to reproduce it in SCA.

The test fails under:
 JDK: JDK6b88 (it also fails at least starting from JDK6b77, didn't check with other)
 platform: linux-amd64
 flags: -server -Xcomp
 hosts: vm-v20z-2.sfbay (2 x AMD Opteron 2.2 GHz, SLES 9.0)
           opteron003.sfbay (2 x AMD Opteron 1.4 GHz, RedHat Linux 4.0AS)

I wasn't able to reproduce it with '-Xmixed' and '-Xint' as well as with -client.
And I was not able to reproduce it on single CPU machine.
So, this makes me think that it could be 'compiler2' bug and not JVMTI, but I am not sure.

The bug looks like it is not related to EarlyReturn functionality, it seems related to
JVMTI functions SuspendThread and NotifyFramePop.

Here is the log:
[2006-06-16T23:16:20.19] earlyretThread (Thread[Tested Thread,5,main]): notifying main thread
[2006-06-16T23:16:23.66] earlyretThread (Thread[Tested Thread,5,main]): inside activeMethod()
[2006-06-16T23:16:23.66] Going to suspend the thread...
[2006-06-16T23:16:23.66] VM option '+UseUnsupportedDeprecatedJVMPI'
[2006-06-16T23:16:23.66] >>>>>>>> Invoke SuspendThread()
[2006-06-16T23:16:23.66] <<<<<<<< SuspendThread() is successfully done
[2006-06-16T23:16:23.66] Forcing early return...
[2006-06-16T23:16:23.66] (NotifyFramePop) unexpected error: JVMTI_ERROR_THREAD_NOT_SUSPENDED (13)
[2006-06-16T23:16:23.66] TEST: failed to force early return
[2006-06-16T23:16:23.66] >>>>>>>> Invoke ResumeThread()
[2006-06-16T23:16:23.66] <<<<<<<< ResumeThread() is successfully done
[2006-06-16T23:16:23.66] TEST FAILED: a tested frame wasn't returned
[2006-06-16T23:16:23.66] Expected result: 1995 - Java Birth Year
[2006-06-16T23:16:23.66] Returned result: 0
[2006-06-16T23:16:23.66] Wrong result returned
[2006-06-16T23:16:23.66] earlyretThread (Thread[Tested Thread,5,main]): exiting...
[2006-06-16T23:16:23.66] # Test level exit status: 97


As you can see NotifyFramePop throws JVMTI_ERROR_THREAD_NOT_SUSPENDED error but
at the same time (according to the log file) SuspendThread() was finished successfully.

The bug is not always reproduced. Steps to reproduce:
1. cp /home/ep155969/bugs/earlyretbase/ep155969.Linux.amd64/earlyretbase/earlyretbase.tlog YOUR_DIR
2. cp /home/ep155969/bugs/earlyretbase/ep155969.Linux.amd64/earlyretbase/reproduce_bug.sh YOUR_DIR
3. cd YOUR_DIR
4. bash reproduce_bug.sh earlyretbase.tlog
Also, a couple of new tests are affected by this bug:
nsk/jvmti/ForceEarlyReturn/ForceEarlyReturn001
nsk/jvmti/ForceEarlyReturn/ForceEarlyReturn002