I am observing the following assert:

# Internal Error (/home/rkennke/src/openjdk/loom/src/hotspot/share/runtime/stackWatermark.cpp:178), pid=54418, tid=54534
# assert(is_frame_safe(f)) failed: Frame must be safe

Stack: [0x00007f9c28e85000,0x00007f9c28f86000], sp=0x00007f9c28f7ef30, free space=999k
Thread 0x00007f9c485e8260 [54534]
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x1d9fb2f] StackWatermark::assert_is_frame_safe(frame const&)+0x6f
V [libjvm.so+0x1da12ec] StackWatermarkSet::on_iteration(JavaThread*, frame const&)+0xac
V [libjvm.so+0x1022f4f] frame::sender(RegisterMap*) const+0x10f
V [libjvm.so+0x1014d78] frame::real_sender(RegisterMap*) const+0x18
V [libjvm.so+0x1f9ec4e] vframe::sender() const+0x11e
V [libjvm.so+0x1f9f12d] vframe::java_sender() const+0xd
V [libjvm.so+0x1da24dc] LiveFrameStream::next()+0x21c
V [libjvm.so+0x1da5328] StackWalk::fill_in_frames(long, BaseFrameStream&, int, int, objArrayHandle, int&, Thread*)+0x568
V [libjvm.so+0x1da6068] StackWalk::fetchNextBatch(Handle, long, long, int, int, objArrayHandle, Thread*)+0x1e8
V [libjvm.so+0x149c0eb] JVM_MoreStackWalk+0x17b

This can be reproduced with Shenandoah:
CONF=linux-x86_64-server-fastdebug make run-test TEST=java/lang/StackWalker TEST_VM_OPTS="-XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=aggressive"

or ZGC:
CONF=linux-x86_64-server-fastdebug make run-test TEST=java/lang/StackWalker TEST_VM_OPTS="-XX:+UnlockExperimentalVMOptions -XX:+UseZGC -XX:ZCollectionInterval=0.01"

Working hypothesis (by @stefank) is that in StackWalk::fetchNextBatch() we prepare the entire stack to be processed by calling StackWatermarkSet::finish_processing(jt, NULL, StackWatermarkKind::gc), but then subsequently, during frames scan, perform allocations to fill in the frame information (fill_in_frames => LiveFrameStream::fill_frame => fill_live_stackframe) at where we could safepoint for GC, which could reset the stack watermark.

This is only relevant for GCs that use the StackWatermark, e.g. ZGC and Shenandoah at the moment.