Since JDK-8348426, when creating the AOT cache with -XX:AOTMode=create, at the very end of normal Java execution, right before we enter the safepoint VM_PopulateDumpSharedSpace, we reset the states of a few Java objects.

https://github.com/openjdk/jdk/blob/a21fa463c4f8d067c18c09a072f3cdfa772aea5e/src/hotspot/share/cds/metaspaceShared.cpp#L983-L1012

Some of the reset operations may violate some assumptions by JIT compilers. We have seen failures in both C2 and graal (see linked issues).

Note: we have seen the crashes only when -XX:+AOTClassLinking is specified. The following call

    HeapShared::reset_archived_object_states(CHECK)

happens even when AOTClassLinking is disabled, so it's unlikely to be the cause of the crash. The crash is probably related to the other two calls, which happen only with -XX:+AOTClassLinking

    MethodType::createArchivedObjects()
    ReflectionFactory::resetArchivedStates()

Crash example: runtime/cds/appcds/applications/JavacBench.java#aot

Command-line:

Command Line: -XX:MaxRAMPercentage=6.25 -Xcomp -ea -esa -XX:CompileThreshold=100 -XX:+UnlockExperimentalVMOptions -XX:-TieredCompilation -XX:+DeoptimizeALot -Xlog:cds -Xlog:cds+heap=error -XX:AOTMode=create -XX:AOTConfiguration=JavacBench.aotconfig -XX:AOTCache=JavacBench.aot -Xlog:cds=debug,cds+class=debug,cds+heap=warning,cds+resolve=debug:file=JavacBench.aot.log::filesize=0

# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (/workspace/open/src/hotspot/share/opto/library_call.cpp:6975), pid=70128, tid=24583
# assert(field != nullptr) failed: undefined field Ljava/lang/Object; jdk/internal/util/WeakReferenceKey referent
#

Current CompileTask:
C2:1829 718 jdk.internal.util.ReferencedKeyMap$$Lambda/0x800000165::apply (10 bytes)

Stack: [0x000000016d7a8000,0x000000016d9ab000], sp=0x000000016d9a7eb0, free space=2047k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.dylib+0x11b8da4] VMError::report(outputStream*, bool)+0x1b00 (library_call.cpp:6975)
V [libjvm.dylib+0x11bc53c] VMError::report_and_die(int, char const*, char const*, char*, Thread*, unsigned char*, void const*, void const*, char const*, int, unsigned long)+0x56c
V [libjvm.dylib+0x5879e8] print_error_for_unit_test(char const*, char const*, char*)+0x0
V [libjvm.dylib+0xc63fc4] LibraryCallKit::load_field_from_object(Node*, char const*, char const*, unsigned long long, bool, ciInstanceKlass*)+0x478
V [libjvm.dylib+0xc4c450] LibraryCallKit::inline_reference_get()+0x8c
V [libjvm.dylib+0xc31cc8] LibraryIntrinsic::generate(JVMState*)+0x184
V [libjvm.dylib+0x62b45c] Parse::do_call()+0x8a0
V [libjvm.dylib+0xec5ca8] Parse::do_one_bytecode()+0x204
V [libjvm.dylib+0xeb6ea4] Parse::do_one_block()+0x4ac
V [libjvm.dylib+0xeb4ed4] Parse::do_all_blocks()+0x480
V [libjvm.dylib+0xeb16a0] Parse::Parse(JVMState*, ciMethod*, float)+0xa34
V [libjvm.dylib+0x3a04d0] ParseGenerator::generate(JVMState*)+0x118
V [libjvm.dylib+0x4daa54] Compile::Compile(ciEnv*, ciMethod*, int, Options, DirectiveSet*)+0xe5c
V [libjvm.dylib+0x39e978] C2Compiler::compile_method(ciEnv*, ciMethod*, int, bool, DirectiveSet*)+0x21c
V [libjvm.dylib+0x4faa80] CompileBroker::invoke_compiler_on_method(CompileTask*)+0x920
V [libjvm.dylib+0x4f9e00] CompileBroker::compiler_thread_loop()+0x398
V [libjvm.dylib+0x8da744] JavaThread::thread_main_inner()+0x1c8
V [libjvm.dylib+0x1103ed4] Thread::call_run()+0xf4
V [libjvm.dylib+0xe84108] thread_native_entry(Thread*)+0x138
C [libsystem_pthread.dylib+0x726c] _pthread_start+0x94
Lock stack of current Java thread (top to bottom):