Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8212155

Race condition when posting dynamic_code_generated event leads to JVM crash

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: P3 P3
    • 17
    • 8, 11, 12
    • hotspot
    • b26

      The crash was originally noticed when attaching async-profiler to a running JVM. While investigating the problem I discovered a race condition in HotSpot related to posting JVMTI dynamic_code_generated event for vtable_stub.

      #
      # A fatal error has been detected by the Java Runtime Environment:
      #
      # Internal Error (c:/Users/Andrei/java/jdk/src/hotspot/share/prims/jvmtiExport.cpp:2280), pid=15040, tid=12008
      # guarantee(collector != NULL) failed: attempt to register stub without event collector
      #
      # JRE version: OpenJDK Runtime Environment (11.0+28) (fastdebug build 11+28)
      # Java VM: OpenJDK 64-Bit Server VM (fastdebug 12-internal+0-adhoc.Andrei.jdk, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)

      Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
      V [jvm.dll+0xb70c71] os::platform_print_native_stack+0xf1 (os_windows_x86.cpp:373)
      V [jvm.dll+0xd8f72a] VMError::report+0xe6a (vmerror.cpp:698)
      V [jvm.dll+0xd91044] VMError::report_and_die+0x854 (vmerror.cpp:1445)
      V [jvm.dll+0xd91704] VMError::report_and_die+0x64 (vmerror.cpp:1269)
      V [jvm.dll+0x594d0e] report_vm_error+0x7e (debug.cpp:233)
      V [jvm.dll+0x9756be] JvmtiExport::post_dynamic_code_generated_while_holding_locks+0xbe (jvmtiexport.cpp:2280)
      V [jvm.dll+0xda692f] VtableStubs::find_stub+0x2ef (vtablestubs.cpp:238)
      V [jvm.dll+0x538cd4] CompiledIC::set_to_megamorphic+0x564 (compiledic.cpp:246)
      V [jvm.dll+0xc3bd72] SharedRuntime::handle_ic_miss_helper+0xa62 (sharedruntime.cpp:1640)
      V [jvm.dll+0xc3c9f0] SharedRuntime::handle_wrong_method_ic_miss+0x240 (sharedruntime.cpp:1403)
      C 0x000001d0ab6a052c

      I attached the reliable reproducer of this issue.
      Run java -agentpath:jvmtiRace.dll

      The problem is that JVMTI_EVENT_DYNAMIC_CODE_GENERATED can be enabled at any arbitrary moment of time, in particular, after JvmtiDynamicCodeEventCollector is constructed but before vtable_stub is generated.

      The other manifestation of this race condition is the following assertion failure in fastdebug build.

      #
      # A fatal error has been detected by the Java Runtime Environment:
      #
      # Internal Error (c:/Users/Andrei/java/jdk/src/hotspot/share/runtime/mutex.cpp:1312), pid=15484, tid=15808
      # fatal error: acquiring lock JvmtiThreadState_lock/22 out of order with lock CompiledIC_lock/22 -- possible deadlock
      #
      # JRE version: OpenJDK Runtime Environment (11.0+28) (fastdebug build 11+28)
      # Java VM: OpenJDK 64-Bit Server VM (fastdebug 12-internal+0-adhoc.Andrei.jdk, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)

      Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
      V [jvm.dll+0xb70c71] os::platform_print_native_stack+0xf1 (os_windows_x86.cpp:373)
      V [jvm.dll+0xd8f72a] VMError::report+0xe6a (vmerror.cpp:698)
      V [jvm.dll+0xd91044] VMError::report_and_die+0x854 (vmerror.cpp:1445)
      V [jvm.dll+0xd91704] VMError::report_and_die+0x64 (vmerror.cpp:1269)
      V [jvm.dll+0x594a90] report_fatal+0x80 (debug.cpp:253)
      V [jvm.dll+0xb1fcd1] Monitor::set_owner_implementation+0x2a1 (mutex.cpp:1310)
      V [jvm.dll+0x966497] JvmtiThreadState::state_for+0x77 (jvmtithreadstate.inline.hpp:91)
      V [jvm.dll+0x97566a] JvmtiExport::post_dynamic_code_generated_while_holding_locks+0x6a (jvmtiexport.cpp:2275)
      V [jvm.dll+0xda692f] VtableStubs::find_stub+0x2ef (vtablestubs.cpp:238)
      V [jvm.dll+0x538cd4] CompiledIC::set_to_megamorphic+0x564 (compiledic.cpp:246)
      V [jvm.dll+0xc3bd72] SharedRuntime::handle_ic_miss_helper+0xa62 (sharedruntime.cpp:1640)
      V [jvm.dll+0xc3c9f0] SharedRuntime::handle_wrong_method_ic_miss+0x240 (sharedruntime.cpp:1403)
      C 0x000001dad032052c

      Again, if JVMTI_EVENT_DYNAMIC_CODE_GENERATED is concurrently enabled after acquiring CompiledIC_lock in SharedRuntime::handle_ic_miss_helper(), the following post_dynamic_code_generated_while_holding_locks() will attempt to create new JvmtiThreadState and acquire JvmtiThreadState_lock which has the same priority.

            lmesnik Leonid Mesnik
            apangin Andrei Pangin
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: