Details
Description
We have seen crashes in AsyncGetCallTrace on Zulu17.38+21-CA (17.0.5+8) (build 17.0.5+8-LTS) in containerized deployments (docker) with the following flags:
-XX:+UseCompressedOops -XX:+UnlockExperimentalVMOptions -XX:+UseZGC -XX:ConcGCThreads=4 -XX:+CrashOnOutOfMemoryError -XX:MaxMetaspaceSize=256M -XX:ReservedCodeCacheSize=256m -XX:+UnlockDiagnosticVMOptions -XX:+ShowHiddenFrames -XX:+UseContainerSupport -XX:+UseContainerCpuShares -XX:NativeMemoryTracking=summary -XX:+PrintNMTStatistics -XX:InitialRAMPercentage=62 -XX:MaxRAMPercentage=62
--------------- T H R E A D ---------------
Current thread (0x00007f246c12ea70): JavaThread "redacted" [_thread_in_Java, id=466, stack(0x00007f22dbaff000,0x00007f22dbc00000)]
Stack: [0x00007f22dbaff000,0x00007f22dbc00000], sp=0x00007f22dbbfaef8, free space=1007k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x6c4438] frame::entry_frame_is_first() const+0x8
V [libjvm.so+0x6c368d] forte_fill_call_trace_given_top(JavaThread*, ASGCT_CallTrace*, int, frame) [clone .isra.22]+0x7bd
V [libjvm.so+0x6c3e8e] AsyncGetCallTrace+0x21e
C [libjavaProfiler2332108121502657858.so+0x1f99a] Profiler::getJavaTraceAsync(void*, ASGCT_CallFrame*, int, StackContext*, bool*) [clone .isra.486]+0xda
C [libjavaProfiler2332108121502657858.so+0x203f2] Profiler::recordSample(void*, unsigned long long, int, int, Event*)+0x2c2
C [libjavaProfiler2332108121502657858.so+0x21217] PerfEvents::signalHandler(int, siginfo_t*, void*)+0x107
C [libpthread.so.0+0x14420]
V [libjvm.so+0xf3d36b] ZBarrierSetRuntime::load_barrier_on_oop_field_preloaded(oopDesc*, oopDesc**)+0x3b
siginfo: si_signo: 11 (SIGSEGV), si_code: 128 (SI_KERNEL), si_addr: 0x0000000000000000
Register to memory mapping:
RAX=0x4d8b187d8b4c2021 is an unknown value
RBX=0x00007f22dbbfb0c0 is pointing into the stack for thread: 0x00007f246c12ea70
RCX=0x00007f27203fcb80 is pointing into interpreter code (not bytecode specific)
RDX=0x00007f273807bcb0 points into unknown readable memory: 0x00007f273807bc90 | 90 bc 07 38 27 7f 00 00
RSP=0x00007f22dbbfaef8 is pointing into the stack for thread: 0x00007f246c12ea70
RBP=0x00007f22dbbfaf90 is pointing into the stack for thread: 0x00007f246c12ea70
RSI=0x00007f27203f4cc2 is at begin+131 in a stub
StubRoutines::call_stub [0x00007f27203f4c3f, 0x00007f27203f4d26] (231 bytes)
RDI=0x00007f22dbbfb0c0 is pointing into the stack for thread: 0x00007f246c12ea70
R8 =0x00007f27203f4b80 points into unknown readable memory: 0x00000000000000ec | ec 00 00 00 00 00 00 00
R9 =0x00007f273efd5000 points into unknown readable memory: 0x0100050403020100 | 00 01 02 03 04 05 00 01
R10=0x00000000000004a5 is an unknown value
R11=0x0000000000000001 is an unknown value
R12=0x0000000000000002 is an unknown value
R13=0x0000000000000800 is an unknown value
R14=0x00007f2740608080: <offset 0x000000000135c080> in /usr/lib/jvm/zulu-17-amd64/lib/server/libjvm.so at 0x00007f273f2ac000
R15=0x00007f22dbbfb090 is pointing into the stack for thread: 0x00007f246c12ea70
Registers:
RAX=0x4d8b187d8b4c2021, RBX=0x00007f22dbbfb0c0, RCX=0x00007f27203fcb80, RDX=0x00007f273807bcb0
RSP=0x00007f22dbbfaef8, RBP=0x00007f22dbbfaf90, RSI=0x00007f27203f4cc2, RDI=0x00007f22dbbfb0c0
R8 =0x00007f27203f4b80, R9 =0x00007f273efd5000, R10=0x00000000000004a5, R11=0x0000000000000001
R12=0x0000000000000002, R13=0x0000000000000800, R14=0x00007f2740608080, R15=0x00007f22dbbfb090
RIP=0x00007f273f970438, EFLAGS=0x0000000000010246, CSGSFS=0x002b000000000033, ERR=0x0000000000000000
TRAPNO=0x000000000000000d
Top of Stack: (sp=0x00007f22dbbfaef8)
0x00007f22dbbfaef8: 00007f273f96e9bc 01007f247f1f4418
0x00007f22dbbfaf08: 00007f22dbbfaf10 00007f22dbbfd590
0x00007f22dbbfaf18: 00007f27203f4cc2 00007f27203f4b90
0x00007f22dbbfaf28: 00007f2400000000 00007f27203f4cc2
Instructions: (pc=0x00007f273f970438)
0x00007f273f970338: 16 48 8b 07 ff 50 60 84 c0 74 0c 48 89 df e8 35
0x00007f273f970348: 29 00 00 84 c0 75 91 4c 89 f2 48 89 de 4c 89 ef
0x00007f273f970358: e8 33 33 00 00 f3 0f 6f 8d 10 ed ff ff f3 0f 6f
0x00007f273f970368: 95 20 ed ff ff f3 0f 6f 9d 30 ed ff ff 0f 29 8d
0x00007f273f970378: 40 ed ff ff 0f 29 95 50 ed ff ff 0f 29 9d 60 ed
0x00007f273f970388: ff ff e9 11 ff ff ff 90 48 8b 85 60 ed ff ff 48
0x00007f273f970398: 8b 40 d0 48 8b 50 20 48 85 d2 75 89 48 8b 50 20
0x00007f273f9703a8: b8 01 00 00 00 48 85 d2 0f 85 3b ff ff ff e9 52
0x00007f273f9703b8: ff ff ff 0f 1f 44 00 00 49 8b 04 24 48 39 85 48
0x00007f273f9703c8: ed ff ff 75 82 48 8b 85 60 ed ff ff 48 8b 40 d0
0x00007f273f9703d8: 48 8b 50 20 48 85 d2 0f 85 6a ff ff ff 48 8b 50
0x00007f273f9703e8: 20 b8 01 00 00 00 48 85 d2 0f 85 fa fe ff ff e9
0x00007f273f9703f8: 11 ff ff ff 0f 1f 40 00 48 89 df e8 78 28 00 00
0x00007f273f970408: 48 81 c4 d0 12 00 00 5b 41 5c 41 5d 41 5e 5d c3
0x00007f273f970418: 48 8b 85 60 ed ff ff 48 8b 40 d0 e9 7c ff ff ff
0x00007f273f970428: 0f 1f 84 00 00 00 00 00 48 8b 47 20 48 8b 40 d0
0x00007f273f970438: 48 8b 40 20 48 85 c0 0f 94 c0 c3 90 66 66 2e 0f
0x00007f273f970448: 1f 84 00 00 00 00 00 90 48 8b 4f 20 48 8d 05 6d
0x00007f273f970458: 6d c9 00 48 8d 3d 6e 6d c9 00 48 8b 00 48 8d 51
0x00007f273f970468: d0 48 03 07 48 8d 3d 4d 6d c9 00 48 03 07 48 03
0x00007f273f970478: 86 d0 03 00 00 48 39 c2 72 09 48 39 96 50 02 00
0x00007f273f970488: 00 77 05 31 c0 c3 66 90 48 8b 41 d0 c3 90 66 2e
0x00007f273f970498: 0f 1f 84 00 00 00 00 00 48 8b 4f 20 48 8b 51 d0
0x00007f273f9704a8: 48 3b 96 50 02 00 00 0f 92 c0 48 39 ca 0f 97 c1
0x00007f273f9704b8: 20 c8 75 04 c3 0f 1f 00 48 8b 42 20 48 39 07 0f
0x00007f273f9704c8: 92 c0 c3 90 0f 1f 40 00 83 7f 18 01 74 52 48 8b
0x00007f273f9704d8: 47 10 48 85 c0 74 49 55 48 89 e5 53 48 89 fb 48
0x00007f273f9704e8: 89 c7 48 83 ec 08 48 8b 10 ff 52 58 84 c0 74 38
0x00007f273f9704f8: 48 8b 7b 10 48 8b 47 70 48 85 c0 74 2b f6 40 29
0x00007f273f970508: 01 75 25 8b 47 68 85 c0 74 1e 48 8b 73 08 e8 45
0x00007f273f970518: 0d f1 ff 48 83 c4 08 83 f0 01 5b 5d c3 0f 1f 00
0x00007f273f970528: 31 c0 c3 0f 1f 44 00 00 48 83 c4 08 31 c0 5b 5d
Stack slot to memory mapping:
stack at sp + 0 slots: 0x00007f273f96e9bc: <offset 0x00000000006c29bc> in /usr/lib/jvm/zulu-17-amd64/lib/server/libjvm.so at 0x00007f273f2ac000
stack at sp + 1 slots: 0x01007f247f1f4418 is an unknown value
stack at sp + 2 slots: 0x00007f22dbbfaf10 is pointing into the stack for thread: 0x00007f246c12ea70
stack at sp + 3 slots: 0x00007f22dbbfd590 is pointing into the stack for thread: 0x00007f246c12ea70
stack at sp + 4 slots: 0x00007f27203f4cc2 is at begin+131 in a stub
StubRoutines::call_stub [0x00007f27203f4c3f, 0x00007f27203f4d26] (231 bytes)
stack at sp + 5 slots: 0x00007f27203f4b90 is pointing to an (unnamed) stub routine
stack at sp + 6 slots: 0x00007f2400000000 points into unknown readable memory: 0x00007f2400000020 | 20 00 00 00 24 7f 00 00
stack at sp + 7 slots: 0x00007f27203f4cc2 is at begin+131 in a stub
StubRoutines::call_stub [0x00007f27203f4c3f, 0x00007f27203f4d26] (231 bytes)
-XX:+UseCompressedOops -XX:+UnlockExperimentalVMOptions -XX:+UseZGC -XX:ConcGCThreads=4 -XX:+CrashOnOutOfMemoryError -XX:MaxMetaspaceSize=256M -XX:ReservedCodeCacheSize=256m -XX:+UnlockDiagnosticVMOptions -XX:+ShowHiddenFrames -XX:+UseContainerSupport -XX:+UseContainerCpuShares -XX:NativeMemoryTracking=summary -XX:+PrintNMTStatistics -XX:InitialRAMPercentage=62 -XX:MaxRAMPercentage=62
--------------- T H R E A D ---------------
Current thread (0x00007f246c12ea70): JavaThread "redacted" [_thread_in_Java, id=466, stack(0x00007f22dbaff000,0x00007f22dbc00000)]
Stack: [0x00007f22dbaff000,0x00007f22dbc00000], sp=0x00007f22dbbfaef8, free space=1007k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x6c4438] frame::entry_frame_is_first() const+0x8
V [libjvm.so+0x6c368d] forte_fill_call_trace_given_top(JavaThread*, ASGCT_CallTrace*, int, frame) [clone .isra.22]+0x7bd
V [libjvm.so+0x6c3e8e] AsyncGetCallTrace+0x21e
C [libjavaProfiler2332108121502657858.so+0x1f99a] Profiler::getJavaTraceAsync(void*, ASGCT_CallFrame*, int, StackContext*, bool*) [clone .isra.486]+0xda
C [libjavaProfiler2332108121502657858.so+0x203f2] Profiler::recordSample(void*, unsigned long long, int, int, Event*)+0x2c2
C [libjavaProfiler2332108121502657858.so+0x21217] PerfEvents::signalHandler(int, siginfo_t*, void*)+0x107
C [libpthread.so.0+0x14420]
V [libjvm.so+0xf3d36b] ZBarrierSetRuntime::load_barrier_on_oop_field_preloaded(oopDesc*, oopDesc**)+0x3b
siginfo: si_signo: 11 (SIGSEGV), si_code: 128 (SI_KERNEL), si_addr: 0x0000000000000000
Register to memory mapping:
RAX=0x4d8b187d8b4c2021 is an unknown value
RBX=0x00007f22dbbfb0c0 is pointing into the stack for thread: 0x00007f246c12ea70
RCX=0x00007f27203fcb80 is pointing into interpreter code (not bytecode specific)
RDX=0x00007f273807bcb0 points into unknown readable memory: 0x00007f273807bc90 | 90 bc 07 38 27 7f 00 00
RSP=0x00007f22dbbfaef8 is pointing into the stack for thread: 0x00007f246c12ea70
RBP=0x00007f22dbbfaf90 is pointing into the stack for thread: 0x00007f246c12ea70
RSI=0x00007f27203f4cc2 is at begin+131 in a stub
StubRoutines::call_stub [0x00007f27203f4c3f, 0x00007f27203f4d26] (231 bytes)
RDI=0x00007f22dbbfb0c0 is pointing into the stack for thread: 0x00007f246c12ea70
R8 =0x00007f27203f4b80 points into unknown readable memory: 0x00000000000000ec | ec 00 00 00 00 00 00 00
R9 =0x00007f273efd5000 points into unknown readable memory: 0x0100050403020100 | 00 01 02 03 04 05 00 01
R10=0x00000000000004a5 is an unknown value
R11=0x0000000000000001 is an unknown value
R12=0x0000000000000002 is an unknown value
R13=0x0000000000000800 is an unknown value
R14=0x00007f2740608080: <offset 0x000000000135c080> in /usr/lib/jvm/zulu-17-amd64/lib/server/libjvm.so at 0x00007f273f2ac000
R15=0x00007f22dbbfb090 is pointing into the stack for thread: 0x00007f246c12ea70
Registers:
RAX=0x4d8b187d8b4c2021, RBX=0x00007f22dbbfb0c0, RCX=0x00007f27203fcb80, RDX=0x00007f273807bcb0
RSP=0x00007f22dbbfaef8, RBP=0x00007f22dbbfaf90, RSI=0x00007f27203f4cc2, RDI=0x00007f22dbbfb0c0
R8 =0x00007f27203f4b80, R9 =0x00007f273efd5000, R10=0x00000000000004a5, R11=0x0000000000000001
R12=0x0000000000000002, R13=0x0000000000000800, R14=0x00007f2740608080, R15=0x00007f22dbbfb090
RIP=0x00007f273f970438, EFLAGS=0x0000000000010246, CSGSFS=0x002b000000000033, ERR=0x0000000000000000
TRAPNO=0x000000000000000d
Top of Stack: (sp=0x00007f22dbbfaef8)
0x00007f22dbbfaef8: 00007f273f96e9bc 01007f247f1f4418
0x00007f22dbbfaf08: 00007f22dbbfaf10 00007f22dbbfd590
0x00007f22dbbfaf18: 00007f27203f4cc2 00007f27203f4b90
0x00007f22dbbfaf28: 00007f2400000000 00007f27203f4cc2
Instructions: (pc=0x00007f273f970438)
0x00007f273f970338: 16 48 8b 07 ff 50 60 84 c0 74 0c 48 89 df e8 35
0x00007f273f970348: 29 00 00 84 c0 75 91 4c 89 f2 48 89 de 4c 89 ef
0x00007f273f970358: e8 33 33 00 00 f3 0f 6f 8d 10 ed ff ff f3 0f 6f
0x00007f273f970368: 95 20 ed ff ff f3 0f 6f 9d 30 ed ff ff 0f 29 8d
0x00007f273f970378: 40 ed ff ff 0f 29 95 50 ed ff ff 0f 29 9d 60 ed
0x00007f273f970388: ff ff e9 11 ff ff ff 90 48 8b 85 60 ed ff ff 48
0x00007f273f970398: 8b 40 d0 48 8b 50 20 48 85 d2 75 89 48 8b 50 20
0x00007f273f9703a8: b8 01 00 00 00 48 85 d2 0f 85 3b ff ff ff e9 52
0x00007f273f9703b8: ff ff ff 0f 1f 44 00 00 49 8b 04 24 48 39 85 48
0x00007f273f9703c8: ed ff ff 75 82 48 8b 85 60 ed ff ff 48 8b 40 d0
0x00007f273f9703d8: 48 8b 50 20 48 85 d2 0f 85 6a ff ff ff 48 8b 50
0x00007f273f9703e8: 20 b8 01 00 00 00 48 85 d2 0f 85 fa fe ff ff e9
0x00007f273f9703f8: 11 ff ff ff 0f 1f 40 00 48 89 df e8 78 28 00 00
0x00007f273f970408: 48 81 c4 d0 12 00 00 5b 41 5c 41 5d 41 5e 5d c3
0x00007f273f970418: 48 8b 85 60 ed ff ff 48 8b 40 d0 e9 7c ff ff ff
0x00007f273f970428: 0f 1f 84 00 00 00 00 00 48 8b 47 20 48 8b 40 d0
0x00007f273f970438: 48 8b 40 20 48 85 c0 0f 94 c0 c3 90 66 66 2e 0f
0x00007f273f970448: 1f 84 00 00 00 00 00 90 48 8b 4f 20 48 8d 05 6d
0x00007f273f970458: 6d c9 00 48 8d 3d 6e 6d c9 00 48 8b 00 48 8d 51
0x00007f273f970468: d0 48 03 07 48 8d 3d 4d 6d c9 00 48 03 07 48 03
0x00007f273f970478: 86 d0 03 00 00 48 39 c2 72 09 48 39 96 50 02 00
0x00007f273f970488: 00 77 05 31 c0 c3 66 90 48 8b 41 d0 c3 90 66 2e
0x00007f273f970498: 0f 1f 84 00 00 00 00 00 48 8b 4f 20 48 8b 51 d0
0x00007f273f9704a8: 48 3b 96 50 02 00 00 0f 92 c0 48 39 ca 0f 97 c1
0x00007f273f9704b8: 20 c8 75 04 c3 0f 1f 00 48 8b 42 20 48 39 07 0f
0x00007f273f9704c8: 92 c0 c3 90 0f 1f 40 00 83 7f 18 01 74 52 48 8b
0x00007f273f9704d8: 47 10 48 85 c0 74 49 55 48 89 e5 53 48 89 fb 48
0x00007f273f9704e8: 89 c7 48 83 ec 08 48 8b 10 ff 52 58 84 c0 74 38
0x00007f273f9704f8: 48 8b 7b 10 48 8b 47 70 48 85 c0 74 2b f6 40 29
0x00007f273f970508: 01 75 25 8b 47 68 85 c0 74 1e 48 8b 73 08 e8 45
0x00007f273f970518: 0d f1 ff 48 83 c4 08 83 f0 01 5b 5d c3 0f 1f 00
0x00007f273f970528: 31 c0 c3 0f 1f 44 00 00 48 83 c4 08 31 c0 5b 5d
Stack slot to memory mapping:
stack at sp + 0 slots: 0x00007f273f96e9bc: <offset 0x00000000006c29bc> in /usr/lib/jvm/zulu-17-amd64/lib/server/libjvm.so at 0x00007f273f2ac000
stack at sp + 1 slots: 0x01007f247f1f4418 is an unknown value
stack at sp + 2 slots: 0x00007f22dbbfaf10 is pointing into the stack for thread: 0x00007f246c12ea70
stack at sp + 3 slots: 0x00007f22dbbfd590 is pointing into the stack for thread: 0x00007f246c12ea70
stack at sp + 4 slots: 0x00007f27203f4cc2 is at begin+131 in a stub
StubRoutines::call_stub [0x00007f27203f4c3f, 0x00007f27203f4d26] (231 bytes)
stack at sp + 5 slots: 0x00007f27203f4b90 is pointing to an (unnamed) stub routine
stack at sp + 6 slots: 0x00007f2400000000 points into unknown readable memory: 0x00007f2400000020 | 20 00 00 00 24 7f 00 00
stack at sp + 7 slots: 0x00007f27203f4cc2 is at begin+131 in a stub
StubRoutines::call_stub [0x00007f27203f4c3f, 0x00007f27203f4d26] (231 bytes)