-
Bug
-
Resolution: Fixed
-
P3
-
8, 11, 12
-
b26
The crash was originally noticed when attaching async-profiler to a running JVM. While investigating the problem I discovered a race condition in HotSpot related to posting JVMTI dynamic_code_generated event for vtable_stub.
#
# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (c:/Users/Andrei/java/jdk/src/hotspot/share/prims/jvmtiExport.cpp:2280), pid=15040, tid=12008
# guarantee(collector != NULL) failed: attempt to register stub without event collector
#
# JRE version: OpenJDK Runtime Environment (11.0+28) (fastdebug build 11+28)
# Java VM: OpenJDK 64-Bit Server VM (fastdebug 12-internal+0-adhoc.Andrei.jdk, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [jvm.dll+0xb70c71] os::platform_print_native_stack+0xf1 (os_windows_x86.cpp:373)
V [jvm.dll+0xd8f72a] VMError::report+0xe6a (vmerror.cpp:698)
V [jvm.dll+0xd91044] VMError::report_and_die+0x854 (vmerror.cpp:1445)
V [jvm.dll+0xd91704] VMError::report_and_die+0x64 (vmerror.cpp:1269)
V [jvm.dll+0x594d0e] report_vm_error+0x7e (debug.cpp:233)
V [jvm.dll+0x9756be] JvmtiExport::post_dynamic_code_generated_while_holding_locks+0xbe (jvmtiexport.cpp:2280)
V [jvm.dll+0xda692f] VtableStubs::find_stub+0x2ef (vtablestubs.cpp:238)
V [jvm.dll+0x538cd4] CompiledIC::set_to_megamorphic+0x564 (compiledic.cpp:246)
V [jvm.dll+0xc3bd72] SharedRuntime::handle_ic_miss_helper+0xa62 (sharedruntime.cpp:1640)
V [jvm.dll+0xc3c9f0] SharedRuntime::handle_wrong_method_ic_miss+0x240 (sharedruntime.cpp:1403)
C 0x000001d0ab6a052c
I attached the reliable reproducer of this issue.
Run java -agentpath:jvmtiRace.dll
The problem is that JVMTI_EVENT_DYNAMIC_CODE_GENERATED can be enabled at any arbitrary moment of time, in particular, after JvmtiDynamicCodeEventCollector is constructed but before vtable_stub is generated.
The other manifestation of this race condition is the following assertion failure in fastdebug build.
#
# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (c:/Users/Andrei/java/jdk/src/hotspot/share/runtime/mutex.cpp:1312), pid=15484, tid=15808
# fatal error: acquiring lock JvmtiThreadState_lock/22 out of order with lock CompiledIC_lock/22 -- possible deadlock
#
# JRE version: OpenJDK Runtime Environment (11.0+28) (fastdebug build 11+28)
# Java VM: OpenJDK 64-Bit Server VM (fastdebug 12-internal+0-adhoc.Andrei.jdk, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [jvm.dll+0xb70c71] os::platform_print_native_stack+0xf1 (os_windows_x86.cpp:373)
V [jvm.dll+0xd8f72a] VMError::report+0xe6a (vmerror.cpp:698)
V [jvm.dll+0xd91044] VMError::report_and_die+0x854 (vmerror.cpp:1445)
V [jvm.dll+0xd91704] VMError::report_and_die+0x64 (vmerror.cpp:1269)
V [jvm.dll+0x594a90] report_fatal+0x80 (debug.cpp:253)
V [jvm.dll+0xb1fcd1] Monitor::set_owner_implementation+0x2a1 (mutex.cpp:1310)
V [jvm.dll+0x966497] JvmtiThreadState::state_for+0x77 (jvmtithreadstate.inline.hpp:91)
V [jvm.dll+0x97566a] JvmtiExport::post_dynamic_code_generated_while_holding_locks+0x6a (jvmtiexport.cpp:2275)
V [jvm.dll+0xda692f] VtableStubs::find_stub+0x2ef (vtablestubs.cpp:238)
V [jvm.dll+0x538cd4] CompiledIC::set_to_megamorphic+0x564 (compiledic.cpp:246)
V [jvm.dll+0xc3bd72] SharedRuntime::handle_ic_miss_helper+0xa62 (sharedruntime.cpp:1640)
V [jvm.dll+0xc3c9f0] SharedRuntime::handle_wrong_method_ic_miss+0x240 (sharedruntime.cpp:1403)
C 0x000001dad032052c
Again, if JVMTI_EVENT_DYNAMIC_CODE_GENERATED is concurrently enabled after acquiring CompiledIC_lock in SharedRuntime::handle_ic_miss_helper(), the following post_dynamic_code_generated_while_holding_locks() will attempt to create new JvmtiThreadState and acquire JvmtiThreadState_lock which has the same priority.
#
# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (c:/Users/Andrei/java/jdk/src/hotspot/share/prims/jvmtiExport.cpp:2280), pid=15040, tid=12008
# guarantee(collector != NULL) failed: attempt to register stub without event collector
#
# JRE version: OpenJDK Runtime Environment (11.0+28) (fastdebug build 11+28)
# Java VM: OpenJDK 64-Bit Server VM (fastdebug 12-internal+0-adhoc.Andrei.jdk, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [jvm.dll+0xb70c71] os::platform_print_native_stack+0xf1 (os_windows_x86.cpp:373)
V [jvm.dll+0xd8f72a] VMError::report+0xe6a (vmerror.cpp:698)
V [jvm.dll+0xd91044] VMError::report_and_die+0x854 (vmerror.cpp:1445)
V [jvm.dll+0xd91704] VMError::report_and_die+0x64 (vmerror.cpp:1269)
V [jvm.dll+0x594d0e] report_vm_error+0x7e (debug.cpp:233)
V [jvm.dll+0x9756be] JvmtiExport::post_dynamic_code_generated_while_holding_locks+0xbe (jvmtiexport.cpp:2280)
V [jvm.dll+0xda692f] VtableStubs::find_stub+0x2ef (vtablestubs.cpp:238)
V [jvm.dll+0x538cd4] CompiledIC::set_to_megamorphic+0x564 (compiledic.cpp:246)
V [jvm.dll+0xc3bd72] SharedRuntime::handle_ic_miss_helper+0xa62 (sharedruntime.cpp:1640)
V [jvm.dll+0xc3c9f0] SharedRuntime::handle_wrong_method_ic_miss+0x240 (sharedruntime.cpp:1403)
C 0x000001d0ab6a052c
I attached the reliable reproducer of this issue.
Run java -agentpath:jvmtiRace.dll
The problem is that JVMTI_EVENT_DYNAMIC_CODE_GENERATED can be enabled at any arbitrary moment of time, in particular, after JvmtiDynamicCodeEventCollector is constructed but before vtable_stub is generated.
The other manifestation of this race condition is the following assertion failure in fastdebug build.
#
# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (c:/Users/Andrei/java/jdk/src/hotspot/share/runtime/mutex.cpp:1312), pid=15484, tid=15808
# fatal error: acquiring lock JvmtiThreadState_lock/22 out of order with lock CompiledIC_lock/22 -- possible deadlock
#
# JRE version: OpenJDK Runtime Environment (11.0+28) (fastdebug build 11+28)
# Java VM: OpenJDK 64-Bit Server VM (fastdebug 12-internal+0-adhoc.Andrei.jdk, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [jvm.dll+0xb70c71] os::platform_print_native_stack+0xf1 (os_windows_x86.cpp:373)
V [jvm.dll+0xd8f72a] VMError::report+0xe6a (vmerror.cpp:698)
V [jvm.dll+0xd91044] VMError::report_and_die+0x854 (vmerror.cpp:1445)
V [jvm.dll+0xd91704] VMError::report_and_die+0x64 (vmerror.cpp:1269)
V [jvm.dll+0x594a90] report_fatal+0x80 (debug.cpp:253)
V [jvm.dll+0xb1fcd1] Monitor::set_owner_implementation+0x2a1 (mutex.cpp:1310)
V [jvm.dll+0x966497] JvmtiThreadState::state_for+0x77 (jvmtithreadstate.inline.hpp:91)
V [jvm.dll+0x97566a] JvmtiExport::post_dynamic_code_generated_while_holding_locks+0x6a (jvmtiexport.cpp:2275)
V [jvm.dll+0xda692f] VtableStubs::find_stub+0x2ef (vtablestubs.cpp:238)
V [jvm.dll+0x538cd4] CompiledIC::set_to_megamorphic+0x564 (compiledic.cpp:246)
V [jvm.dll+0xc3bd72] SharedRuntime::handle_ic_miss_helper+0xa62 (sharedruntime.cpp:1640)
V [jvm.dll+0xc3c9f0] SharedRuntime::handle_wrong_method_ic_miss+0x240 (sharedruntime.cpp:1403)
C 0x000001dad032052c
Again, if JVMTI_EVENT_DYNAMIC_CODE_GENERATED is concurrently enabled after acquiring CompiledIC_lock in SharedRuntime::handle_ic_miss_helper(), the following post_dynamic_code_generated_while_holding_locks() will attempt to create new JvmtiThreadState and acquire JvmtiThreadState_lock which has the same priority.
- relates to
-
JDK-8214816 os::read() should not transition to _thread_blocked with safepoint check on Solaris
-
- Resolved
-