Reported by Oli Gillespie from Amazon:

I observed a JVM crash while running the jtreg test jdk/jfr/event/oldobject/TestLargeRootSet.java as part of the collection test/jdk:jdk_jfr.

I have tried unsuccessfully to reproduce this issue, running the same tests on repeat overnight in the same environment.

Here are the relevant snippets from hs_err log, plus some observations from debugging the core file. I can provide more information from the core file on request.

#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f8cac4be9a0, pid=14860, tid=14870
#
# JRE version: OpenJDK Runtime Environment Corretto-18.0.0.34.1 (18.0+34) (build 18+34-Nightly)
# Java VM: OpenJDK 64-Bit Server VM Corretto-18.0.0.34.1 (18+34-Nightly, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
# Problematic frame:
# V [libjvm.so+0x6c69a0] EdgeStore::get_id(Edge const*) const+0x0
#
# Core dump will be written. Default location: /home/jenkins/node/workspace/Corretto18/generic_linux/x64/jtreg_jfr/JTwork/scratch/4/core.14860
#
# JFR recording file will be written. Location: /home/jenkins/node/workspace/Corretto18/generic_linux/x64/jtreg_jfr/JTwork/scratch/4/hs_err_pid14860.jfr
#
# If you would like to submit a bug report, please visit:
# https://github.com/corretto/corretto-18/issues/
#

...

--------------- T H R E A D ---------------

Current thread (0x00007f8ca420f760): VMThread "VM Thread" [stack: 0x00007f8c7daaf000,0x00007f8c7dbaf000] [id=14870]

Stack: [0x00007f8c7daaf000,0x00007f8c7dbaf000], sp=0x00007f8c7dbad678, free space=1017k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x6c69a0] EdgeStore::get_id(Edge const*) const+0x0
V [libjvm.so+0x6daaec] EventEmitter::write_events(ObjectSampler*, EdgeStore*, bool)+0x15c
V [libjvm.so+0xc91c49] PathToGcRootsOperation::doit()+0x489
V [libjvm.so+0xf532c3] VM_Operation::evaluate()+0x173
V [libjvm.so+0xf54d4d] VMThread::evaluate_operation(VM_Operation*)+0xbd
V [libjvm.so+0xf553ff] VMThread::inner_execute(VM_Operation*)+0x38f
V [libjvm.so+0xf555e7] VMThread::run()+0xb7
V [libjvm.so+0xecddcb] Thread::call_run()+0xbb
V [libjvm.so+0xc556a1] thread_native_entry(Thread*)+0xe1


siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x0000000000000008

Register to memory mapping:

RAX=0x0 is NULL
RBX=0x00007f8c2c0b4f00 points into unknown readable memory: 0x00007f8c2c0b1290 | 90 12 0b 2c 8c 7f 00 00
RCX=0x00007f8c2c016d19 points into unknown readable memory: 00 80 80 08 00 22 54
RDX=0x00007f8c6c01a390 points into unknown readable memory: 0x0000000000000000 | 00 00 00 00 00 00 00 00
RSP=0x00007f8c7dbad678 points into unknown readable memory: 0x00007f8cac4d2730 | 30 27 4d ac 8c 7f 00 00
RBP=0x00007f8c7dbad710 points into unknown readable memory: 0x00007f8c7dbad770 | 70 d7 ba 7d 8c 7f 00 00
RSI=0x0 is NULL
RDI=0x00007f8c7c6611c0 is pointing into the stack for thread: 0x00007f8ca428e550
R8 =0x0 is NULL
R9 =0x00007f8c2c016a70 points into unknown readable memory: 0x0000000000000000 | 00 00 00 00 00 00 00 00
R10=0x00007f8c2c016d1a points into unknown readable memory: 80 80 08 00 22 54
R11=0x0000000000000028 is an unknown value
R12=0x00007f8c7dbad7e0 points into unknown readable memory: 0x00007f8cad3c98a0 | a0 98 3c ad 8c 7f 00 00
R13=0x0 is NULL
R14=0x00007f8c7c6611c0 is pointing into the stack for thread: 0x00007f8ca428e550
R15=0x00007f8c2c0b4f00 points into unknown readable memory: 0x00007f8c2c0b1290 | 90 12 0b 2c 8c 7f 00 00


Registers:
RAX=0x0000000000000000, RBX=0x00007f8c2c0b4f00, RCX=0x00007f8c2c016d19, RDX=0x00007f8c6c01a390
RSP=0x00007f8c7dbad678, RBP=0x00007f8c7dbad710, RSI=0x0000000000000000, RDI=0x00007f8c7c6611c0
R8 =0x0000000000000000, R9 =0x00007f8c2c016a70, R10=0x00007f8c2c016d1a, R11=0x0000000000000028
R12=0x00007f8c7dbad7e0, R13=0x0000000000000000, R14=0x00007f8c7c6611c0, R15=0x00007f8c2c0b4f00
RIP=0x00007f8cac4be9a0, EFLAGS=0x0000000000010206, CSGSFS=0x002b000000000033, ERR=0x0000000000000004
  TRAPNO=0x000000000000000e

Top of Stack: (sp=0x00007f8c7dbad678)
0x00007f8c7dbad678: 00007f8cac4d2730 00000000b049796e
0x00007f8c7dbad688: 00000000b1893fca 00007f8c00000101
0x00007f8c7dbad698: 00000000b02894ca 00000000b02894ca
0x00007f8c7dbad6a8: 0000000000000028 000000000020e4a4

Instructions: (pc=0x00007f8cac4be9a0)
0x00007f8cac4be8a0: c0 74 2d 48 3b 70 28 75 0d eb 15 0f 1f 44 00 00
0x00007f8cac4be8b0: 48 39 70 28 74 0a 48 8b 00 48 85 c0 75 f2 5d c3
0x00007f8cac4be8c0: 48 83 c0 08 5d c3 66 2e 0f 1f 84 00 00 00 00 00
0x00007f8cac4be8d0: 31 c0 5d c3 66 90 66 2e 0f 1f 84 00 00 00 00 00
0x00007f8cac4be8e0: 55 48 89 f2 48 89 e5 41 54 53 49 89 fc 48 8d 7d
0x00007f8cac4be8f0: d0 48 89 f3 48 83 ec 20 31 f6 48 83 e3 fc e8 9d
0x00007f8cac4be900: fa ff ff 66 0f ef c0 bf 38 00 00 00 4d 8b 24 24
0x00007f8cac4be910: 0f 29 45 e0 e8 77 94 1a 00 48 85 c0 48 89 c1 74
0x00007f8cac4be920: 2c 48 c7 00 00 00 00 00 66 0f 6f 45 d0 0f 11 40
0x00007f8cac4be930: 08 48 8b 45 e0 48 89 59 28 48 89 41 18 48 8b 45
0x00007f8cac4be940: e8 48 c7 41 30 00 00 00 00 48 89 41 20 48 89 d8
0x00007f8cac4be950: 31 d2 49 f7 74 24 08 48 8b 05 72 8d f0 00 48 83
0x00007f8cac4be960: c0 01 48 89 05 67 8d f0 00 48 89 41 30 49 8b 04
0x00007f8cac4be970: 24 48 c1 e2 03 48 01 d0 48 8b 00 48 89 01 49 03
0x00007f8cac4be980: 14 24 48 89 0a 49 83 44 24 18 01 48 83 c4 20 5b
0x00007f8cac4be990: 48 8d 41 08 41 5c 5d c3 0f 1f 84 00 00 00 00 00
0x00007f8cac4be9a0: 48 8b 4e 08 55 31 d2 48 8b 3f 48 89 e5 48 83 e1
0x00007f8cac4be9b0: fc 48 89 c8 48 f7 77 08 48 8b 07 48 8d 04 d0 48
0x00007f8cac4be9c0: 8b 00 48 85 c0 74 17 48 3b 48 28 75 09 eb 0f 90
0x00007f8cac4be9d0: 48 39 48 28 74 08 48 8b 00 48 85 c0 75 f2 48 8b
0x00007f8cac4be9e0: 40 30 5d c3 66 90 66 2e 0f 1f 84 00 00 00 00 00
0x00007f8cac4be9f0: 48 8b 46 10 48 85 c0 74 07 f3 c3 0f 1f 44 00 00
0x00007f8cac4bea00: 55 48 89 e5 53 48 89 fb 48 89 f7 48 83 ec 08 e8
0x00007f8cac4bea10: cc 14 00 00 48 8b 48 08 48 8b 33 31 d2 48 83 e1
0x00007f8cac4bea20: fc 48 89 c8 48 f7 76 08 48 8b 06 48 8d 04 d0 48
0x00007f8cac4bea30: 8b 00 48 85 c0 75 11 eb 15 0f 1f 80 00 00 00 00
0x00007f8cac4bea40: 48 8b 00 48 85 c0 74 06 48 3b 48 28 75 f2 48 8b
0x00007f8cac4bea50: 40 30 48 83 c4 08 5b 5d c3 90 66 0f 1f 44 00 00
0x00007f8cac4bea60: 31 c0 48 83 f9 63 77 08 f3 c3 66 0f 1f 44 00 00
0x00007f8cac4bea70: 55 48 89 e5 41 57 41 56 41 55 41 54 49 89 fd 53
0x00007f8cac4bea80: 48 8d 59 9d 49 89 f6 49 89 d4 48 83 ec 28 48 8b
0x00007f8cac4bea90: 3a 48 89 de e8 77 14 00 00 49 89 c7 49 8b 06 31


Stack slot to memory mapping:
stack at sp + 0 slots: 0x00007f8cac4d2730: <offset 0x00000000006da730> in /home/jenkins/node/workspace/Corretto18/generic_linux/x64/jtreg_jfr/jdk/lib/server/libjvm.so at 0x00007f8cabdf8000
stack at sp + 1 slots: 0x00000000b049796e is an unknown value
stack at sp + 2 slots: 0x00000000b1893fca is an unknown value
stack at sp + 3 slots: 0x00007f8c00000101 points into unknown readable memory: 00 00 00 8c 7f 00 00
stack at sp + 4 slots: 0x00000000b02894ca is an unknown value
stack at sp + 5 slots: 0x00000000b02894ca is an unknown value
stack at sp + 6 slots: 0x0000000000000028 is an unknown value
stack at sp + 7 slots: 0x000000000020e4a4 is an unknown value

VM_Operation (0x00007f8c7c6611d0): JFROldObject, mode: safepoint, requested by thread 0x00007f8ca428e550
...

It seems that the edge passed in to https://github.com/openjdk/jdk18/blob/6dae52f8e3993d529033147de8e34ad1e7d48c53/src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp#L82 is NULL.

Here are my observations from the core dump:

R15 contains the ObjectSample we're looking up
(gdb) p (struct ObjectSample) 0x00007f8c2c0b4f00
{
<JfrCHeapObj> = {<CHeapObj<(MEMFLAGS)16>> = {<No data fields>}, <No data fields>},
_next = 0x7f8c2c0b1290,
_previous = 0x0,
_stacktrace = {_ptr = 0x0},
_thread = {_ptr = 0x7f8c2c0c82b0},
_type_set = {_ptr = 0x0},
_object = {_obj = 0x7f8c2c0c8dc8},
_allocation_time = {<CompositeCounterRepresentation<CompositeElapsedCounterSource>>= {_rep = {val1 = 2956793916,val2 = 2956793916}}, <No data fields>}, <No data fields>},
_stack_trace_id = 31,
_thread_id = 22,
_index = 0,
_span = 24,
_allocated = 24,
_heap_used_at_last_gc = 186036224,
_stack_trace_hash = 1161530239
}

EdgeStore is at RDI
(gdb) p (EdgeStore) 0x00007f8c7c6611c0
$18 = {<CHeapObj<(MEMFLAGS)16>> = {<No data fields>}, static _edge_id_counter = 257, _edges = 0x7f8c2c0caf20}

Read _edges:
(gdb) p ('EdgeStore::EdgeHashTable') 0x7f8c2c0caf20
$24 = {<JfrBasicHashtable<StoredEdge>> = {
<CHeapObj<(MEMFLAGS)16>> = {<No data fields>},
_buckets = 0x7f8c2c103ba0,
_table_size = 1009,
_entry_size = 56,
_number_of_entries = 257
},_callback = 0x7f8c7c6611c0}

p (((ObjectSample) *0x00007f8c2c0b4f00)._object._obj._mark._value >> markWord::lock_bits)
$42 = 3

Therefore link_sample_with_edge in EventEmitter::write_events presumably returned early under the assumption: "Associated with an edge (chain) already during heap traversal.",
which appears to be incorrect since the EdgeStore then returned NULL for get(sample).