Deadlock caused by jdwp agent

Our customer reported a deadlock caused by jdwp agent with jdk11, the stack trace is attached.


Thread 4: while holding classTrackLock monitor tries to allocate memory, that triggers state transition and gets blocked by on going GC safepoint.

#12 ThreadInVMfromNative (thread=0x7ffef4036000, this=0x7ffed41e1560) at src/hotspot/share/runtime/interfaceSupport.inline.hpp:267
#13 jvmti_Allocate (env=0x7ffff000bd70, size=24, mem_ptr=0x7ffed41e15c0, this=<optimized out>) at build/hotspot/variant-server/gensrc/jvmtifiles/jvmtiEnter.cpp:336
#14 0x00007ffff49c6786 in jvmtiAllocate (numBytes=numBytes@entry=24) at src/jdk.jdwp.agent/share/native/libjdwp/util.c:1808
#15 0x00007ffff49a92ad in bagCreateBag (itemSize=itemSize@entry=8, initialAllocation=initialAllocation@entry=10) at src/jdk.jdwp.agent/share/native/libjdwp/bag.c:40 << == Holding classTrackLock monitor
#16 0x00007ffff49a97b0 in classTrack_processUnloads (env=env@entry=0x7ffef4036348) at src/jdk.jdwp.agent/share/native/libjdwp/classTrack.c:91
#17 0x00007ffff49b1253 in event_callback (env=env@entry=0x7ffef4036348, evinfo=evinfo@entry=0x7ffed41e1700) at src/jdk.jdwp.agent/share/native/libjdwp/eventHandler.c:583
#18 0x00007ffff49b215e in cbClassPrepare (jvmti_env=<optimized out>, env=0x7ffef4036348, thread=0x7ffee00010d0, klass=0x7ffee00010d8) at src/jdk.jdwp.agent/share/native/libjdwp/eventHandler.c:892
#19 0x00007ffff6681094 in JvmtiExport::post_class_prepare (thread=0x7ffef4036000, klass=0x840362840) at src/hotspot/share/prims/jvmtiExport.cpp:1337
#20 0x00007ffff64cf5c8 in InstanceKlass::link_class_impl (this=0x840362840, throw_verifyerror=<optimized out>, __the_thread__=0x7ffef4036000) at src/hotspot/share/oops/instanceKlass.cpp:838

Thread 88: VMThread is running G1 Remark code, processing weak roots and trying to post object free event and blocked on classTrackLock monitor

#2 0x00007ffff668f57c in SimpleEnter (Self=<optimized out>, this=<optimized out>) at src/hotspot/share/prims/jvmtiRawMonitor.cpp:150
#3 JvmtiRawMonitor::raw_enter (this=this@entry=0x7ffff07c0000, __the_thread__=__the_thread__@entry=0x7ffff0755800) at src/hotspot/share/prims/jvmtiRawMonitor.cpp:304
#4 0x00007ffff6671cb8 in JvmtiEnv::RawMonitorEnter (this=<optimized out>, rmonitor=0x7ffff07c0000) at src/hotspot/share/prims/jvmtiEnv.cpp:3324
#5 0x00007ffff49c4a7b in debugMonitorEnter (monitor=0x7ffff07c0000) at src/jdk.jdwp.agent/share/native/libjdwp/util.c:1043 << == acquire classTrackLock monitor
#6 0x00007ffff49a9758 in cbTrackingObjectFree (jvmti_env=<optimized out>, tag=140729502422448) at src/jdk.jdwp.agent/share/native/libjdwp/classTrack.c:67
#7 0x00007ffff667fe1d in JvmtiExport::post_object_free (env=0x7ffff07c00f0, tag=140729502422448) at src/hotspot/share/prims/jvmtiExport.cpp:1470
#8 0x00007ffff66a098c in JvmtiTagMap::do_weak_oops (this=0x7ffff07c1940, is_alive=is_alive@entry=0x7fffb4377380, f=f@entry=0x7ffff7182150 <do_nothing_cl>) at src/hotspot/share/prims/jvmtiTagMap.cpp:3377
#9 0x00007ffff66a0ac5 in JvmtiTagMap::weak_oops_do (is_alive=is_alive@entry=0x7fffb4377380, f=f@entry=0x7ffff7182150 <do_nothing_cl>) at src/hotspot/share/prims/jvmtiTagMap.cpp:3325
#10 0x00007ffff667eeca in JvmtiExport::weak_oops_do (is_alive=is_alive@entry=0x7fffb4377380, f=f@entry=0x7ffff7182150 <do_nothing_cl>) at src/hotspot/share/prims/jvmtiExport.cpp:2590
#11 0x00007ffff6b44d11 in WeakProcessor::weak_oops_do (is_alive=is_alive@entry=0x7fffb4377380, keep_alive=0x7ffff7182150 <do_nothing_cl>) at src/hotspot/share/gc/shared/weakProcessor.cpp:38
#12 0x00007ffff640e9a3 in G1ConcurrentMark::weak_refs_work (this=this@entry=0x7ffff00708f0, clear_all_soft_refs=clear_all_soft_refs@entry=false) at src/hotspot/share/gc/g1/g1ConcurrentMark.cpp:1674
#13 0x00007ffff640fc59 in G1ConcurrentMark::remark (this=0x7ffff00708f0) at src/hotspot/share/gc/g1/g1ConcurrentMark.cpp:1163


Start with JDK-8212879 in jdk16, object free events are no longer posted during weak root processing, but they are still posted at safepoints, so potential has the same issue(?)