Investigating a timeout of the applications/runthese/RunThese30M.java closed stress test the following deadlock was observed.

This thread has been suspended, but the signal has hit inside the pthread_create library code:

thread #34
    frame #0: 0x00000001890c0b98 libsystem_kernel.dylib`__sigsuspend + 8
    frame #1: 0x000000010a5c0420 libjvm.dylib`SR_handler(int, __siginfo*, void*) + 332
    frame #2: 0x00000001890ff744 libsystem_platform.dylib`_sigtramp + 56
    frame #3: 0x00000001890f72dc libsystem_pthread.dylib`_pthread_create + 904
    frame #4: 0x000000010a45ccfc libjvm.dylib`os::create_thread(Thread*, os::ThreadType, unsigned long) + 312
    frame #5: 0x0000000109ea95f4 libjvm.dylib`JavaThread::JavaThread(void (*)(JavaThread*, JavaThread*), unsigned long, MemTag) + 68
    frame #6: 0x000000010a00d6c4 libjvm.dylib`JVM_StartThread + 1136

Here was have the resuming thread:

thread #41
    frame #0: 0x00000001890c0c74 libsystem_kernel.dylib`__ulock_wait2 + 8
    frame #1: 0x00000001890fd3c4 libsystem_platform.dylib`_os_unfair_lock_lock_slow + 176
    frame #2: 0x00000001890f5800 libsystem_pthread.dylib`pthread_kill + 160
    frame #3: 0x000000010a5bf8c8 libjvm.dylib`PosixSignals::do_resume(OSThread*) + 128
    frame #4: 0x000000010a5bfa4c libjvm.dylib`SuspendedThreadTask::internal_do_task() + 80
    frame #5: 0x0000000109f40290 libjvm.dylib`JfrSamplerThread::sample_java_thread(JavaThread*) + 76
    frame #6: 0x0000000109f3fff4 libjvm.dylib`JfrSamplerThread::task_stacktrace(JfrSampleRequestType, JavaThread**) + 332
    frame #7: 0x0000000109f3fe7c libjvm.dylib`JfrSamplerThread::run() + 300

but it is blocked in pthread_kill trying to send the resumption signal, because the suspended thread holds an internal pthreads lock.

You can see other threads piling up on the same lock:

thread #14
    frame #0: 0x00000001890c0c74 libsystem_kernel.dylib`__ulock_wait2 + 8
    frame #1: 0x00000001890fd3c4 libsystem_platform.dylib`_os_unfair_lock_lock_slow + 176
    frame #2: 0x00000001890f72dc libsystem_pthread.dylib`_pthread_create + 904
    frame #3: 0x000000010a45ccfc libjvm.dylib`os::create_thread(Thread*, os::ThreadType, unsigned long) + 312
    frame #4: 0x0000000109ea959c libjvm.dylib`JavaThread::JavaThread(void (*)(JavaThread*, JavaThread*), unsigned long, MemTag) + 68
    frame #5: 0x0000000109ac9b90 libjvm.dylib`CompilerThread::CompilerThread(CompileQueue*, CompilerCounters*) + 40
    frame #6: 0x0000000109aac2c0 libjvm.dylib`CompileBroker::make_thread(CompileBroker::ThreadType, _jobject*, CompileQueue*, AbstractCompiler*, JavaThread*) + 768
    frame #7: 0x0000000109aac9f0 libjvm.dylib`CompileBroker::possibly_add_compiler_threads(JavaThread*) + 82