G1 does not keep weak nmethod oops alive

Summary after investigation (details can be found in the comment section):
This is an issue with the implementation of the @Stable annotation in scenarios where the field is initialized multiple times due to a race condition between threads. In this case, indy string concat code lazily initializes the stable NEW_STRING field holding a MethodHandle but multiple threads can succeed in writing to the field, see (https://github.com/openjdk/jdk/blob/5cdb4b196047d4f2d69df0fc73102c102bf042f7/src/java.base/share/classes/java/lang/invoke/StringConcatFactory.java#L855). Now according to the comments in Stable.java, this could simply be treated as a user error:

 * A field may be annotated as stable if all of its component variables
 * changes value at most once.
[...]
 * It is (currently) undefined what happens if a field annotated as stable
 * is given a third value (by explicitly updating a stable field, a component of
 * a stable array, or a final stable field via reflection or other means).
 * Since the HotSpot VM promotes a non-null component value to constant, it may
 * be that the Java memory model would appear to be broken, if such a constant
 * (the second value of the field) is used as the value of the field even after
 * the field value has changed (to a third value).

However, in this particular scenario, C2's FoldStableValues optimization will constant fold the load at compile time and embed the current oop value in compiled code. If the field is then overwritten, the compiled code is the only (weak) reference to the MethodHandle object. G1 does not keep such references alive, leading to a dead oop and corresponding asserts/crashes.


Running runtime/cds/appcds/dynamicArchive/LotsUnloadTest with -XX:+VerifyBeforeGC -XX:+VerifyAfterGC -XX:+VerifyDuringGC shows some references to dead oops (java.lang.invoke.BoundMethodHandle$Species_L) in some stack frames.

Use the attached patch to improve reproducability. Does not reproduce with c1 (TieredStopAtLevel=1) or interpreter (-Xint)

The change to use single bitmaps in G1 (JDK-8210708) made this to occur a bit more often, but it has been reproduced in master too.

This issue started with JDK 17b13, JDK-8219555 that "increased the aggressiveness of C2 compilation".

# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (workspace/open/src/hotspot/share/gc/g1/g1HeapVerifier.cpp:500), pid=3070816, tid=3070823
# fatal error: there should not have been any failures
#
# JRE version: Java(TM) SE Runtime Environment (20.0+9) (fastdebug build 20-ea+9-487)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 20-ea+9-487, compiled mode, tiered, compressed class ptrs, g1 gc, linux-amd64)
# Problematic frame:
# V [libjvm.so+0xe1413f] G1HeapVerifier::verify(VerifyOption)+0x4cf
#
# Core dump will be written. Default location: Core dumps may be processed with "/opt/core.sh %p" (or dumping to /opt/mach5/mesos/work_dir/slaves/0c72054a-24ab-4dbb-944f-97f9341a1b96-S45803/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/5b60ce54-d431-4916-ac28-d78c20d6446b/runs/099177ec-b020-48a9-af28-e9f618d83cec/testoutput/test-support/jtreg_open_test_hotspot_jtreg_runtime_cds_appcds_dynamicArchive_LotsUnloadTest3_java/scratch/0/core.3070816)
#
# If you would like to submit a bug report, please visit:
# https://bugreport.java.com/bugreport/crash.jsp
#

Stack: [0x00007f8b7da2a000,0x00007f8b7db2a000], sp=0x00007f8b7db27db0, free space=1015k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0xe1413f] G1HeapVerifier::verify(VerifyOption)+0x4cf
V [libjvm.so+0x1adb2cb] Universe::verify(VerifyOption, char const*)+0x74b
V [libjvm.so+0xda2863] G1ConcurrentMark::verify_during_pause(G1HeapVerifier::G1VerifyType, G1ConcurrentMark::VerifyLocation)+0xf3
V [libjvm.so+0xda860d] G1ConcurrentMark::remark()+0x45d
V [libjvm.so+0xe90dee] VM_G1PauseConcurrent::doit()+0x18e
V [libjvm.so+0x1b848a2] VM_Operation::evaluate()+0x182
V [libjvm.so+0x1ba969a] VMThread::evaluate_operation(VM_Operation*)+0x18a
V [libjvm.so+0x1baaf3f] VMThread::inner_execute(VM_Operation*)+0x40f
V [libjvm.so+0x1bab115] VMThread::loop()+0xc5
V [libjvm.so+0x1bab230] VMThread::run()+0xb0
V [libjvm.so+0x1a8ebf0] Thread::call_run()+0x100
V [libjvm.so+0x174c663] thread_native_entry(Thread*)+0x103

[7.924s][error][gc,verify ] GC(25) Root location 0x00007f8b7cf19b20 points to dead obj 0x00007f8b8ce00c90 in region 2:(O)[0x00007f8b8ce00000,0x00007f8b8ceebb00,0x00007f8b8cf00000]
[7.924s][error][gc,verify ] GC(25) java.lang.invoke.BoundMethodHandle$Species_L
[7.924s][error][gc,verify ] GC(25) {0x00007f8b8ce00c90} - klass: 'java/lang/invoke/BoundMethodHandle$Species_L'
[7.924s][error][gc,verify ] GC(25) - ---- fields (total size 7 words):
[7.924s][error][gc,verify ] GC(25) - private 'customizationCount' 'B' @12 0
[7.924s][error][gc,verify ] GC(25) - private volatile 'updateInProgress' 'Z' @13 false
[7.924s][error][gc,verify ] GC(25) - private final 'type' 'Ljava/lang/invoke/MethodType;' @16 a 'java/lang/invoke/MethodType'{0x00007f8b8ce0ac28} = ([BJ)Ljava/lang/String; (8ce0ac28 7f8b)
[7.924s][error][gc,verify ] GC(25) - final 'form' 'Ljava/lang/invoke/LambdaForm;' @24 a 'java/lang/invoke/LambdaForm'{0x00007f8b8ce009c8} => a 'java/lang/invoke/MemberName'{0x00007f8b8ce220b0} = {method} {0x00007f8b745cf2e0} 'reinvoke_L' '(Ljava/lang/Object;Ljava/lang/Object;J)Ljava/lang/Object;' in 'java/lang/invoke/DelegatingMethodHandle$Holder' (8ce009c8 7f8b)
[7.924s][error][gc,verify ] GC(25) - private 'asTypeCache' 'Ljava/lang/invoke/MethodHandle;' @32 NULL (0 0)
[7.924s][error][gc,verify ] GC(25) - private 'asTypeSoftCache' 'Ljava/lang/ref/SoftReference;' @40 NULL (0 0)
[7.925s][error][gc,verify ] GC(25) - final 'argL0' 'Ljava/lang/Object;' @48 a 'java/lang/invoke/DirectMethodHandle'{0x00007f8b8ce13940} (8ce13940 7f8b)
[7.938s][error][gc,verify ] GC(25) Heap after failed verification (kind 0):
[7.938s][error][gc,verify ] GC(25) garbage-first heap total 65536K, used 46035K [0x00007f8b8cc00000, 0x00007f8b90c00000)
[7.938s][error][gc,verify ] GC(25) region size 1024K, 2 young (2048K), 1 survivors (1024K)
[7.938s][error][gc,verify ] GC(25) Metaspace used 7479K, committed 7680K, reserved 1114112K
[7.938s][error][gc,verify ] GC(25) class space used 518K, committed 640K, reserved 1048576K