ConcurrentLocksDump::dump_at_safepoint() creates a GrowableArray, which gets allocated in a resource area. This array is than passed down a call chain, where it can't control that another ResourceMark isn't created. In the leaf of this call chain, a closure (FindInstanceClosure) is executed, which appends to the array, which means it might need to be resized. This doesn't work if a new ResourceMark has been created, since the array resize will happen in a nested ResourceArea context. As a result, the append operation fails in GenericGrowableArray::check_nesting().

This has so far gone unnoticed because CollectedHeap::object_iterate() in existing collectors typically don't create new ResourceMarks. This is not true for ZGC (and potentially other concurrent collectors), which needs to walk thread stacks, which in turn requires a ResourceMark.
The proposed fix is to make this array C Heap allocated.

diff --git a/src/hotspot/share/services/threadService.cpp b/src/hotspot/share/services/threadService.cpp
--- a/src/hotspot/share/services/threadService.cpp
+++ b/src/hotspot/share/services/threadService.cpp
@@ -675,15 +675,15 @@
   // dump all locked concurrent locks
   assert(SafepointSynchronize::is_at_safepoint(), "all threads are stopped");
 
- ResourceMark rm;
-
- GrowableArray<oop>* aos_objects = new GrowableArray<oop>(INITIAL_ARRAY_SIZE);
+ GrowableArray<oop>* aos_objects = new (ResourceObj::C_HEAP, mtInternal) GrowableArray<oop>(INITIAL_ARRAY_SIZE, true /* C_heap */);
 
   // Find all instances of AbstractOwnableSynchronizer
   HeapInspection::find_instances_at_safepoint(SystemDictionary::abstract_ownable_synchronizer_klass(),
                                                 aos_objects);
   // Build a map of thread to its owned AQS locks
   build_map(aos_objects);
+
+ delete aos_objects;
 }