Test test/hotspot/jtreg/serviceability/jvmti/vthread/ThreadStateTest/ shows the issue:
The test uses native library to implement JVMTI agent and to implement native methods of the test class.
Native method testGetThreadState uses agent's jvmtiEnv to call GetThreadState for the current thread.
It's expected that the thread state value is "JVMTI_THREAD_STATE_ALIVE | JVMTI_THREAD_STATE_RUNNABLE", but the call is made from native method, so it should include JVMTI_THREAD_STATE_IN_NATIVE bit