Kitchensink fails with
#
# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (open/src/hotspot/share/gc/z/zRootsIterator.cpp:156), pid=2052, tid=2057
# assert(ZNMethod::is_armed(nm) == _should_disarm_nmethods) failed: Invalid state
#
# JRE version: Java(TM) SE Runtime Environment (14.0+27) (fastdebug build 14-ea+27-1297)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 14-ea+27-1297, mixed mode, tiered, z gc, linux-amd64)
# Problematic frame:
# V [libjvm.so+0x1830d23] ZRootsIteratorCodeBlobClosure::do_code_blob(CodeBlob*)+0x73
#
# Core dump will be written. Default location: Core dumps may be processed with "/opt/core.sh %p" (or dumping to /opt/mach5/mesos/work_dir/slaves/fcf4c0c4-d73e-4321-860c-6613427db92b-S1793/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/8436a87f-0cb7-458a-ada1-a7ec3f525fe6/runs/2d8ffc73-0050-4316-a8b7-ad7fa3cc3930/testoutput/test-support/jtreg_closed_test_hotspot_jtreg_applications_kitchensink_Kitchensink_java/scratch/0/core.2052)
#
# If you would like to submit a bug report, please visit:
# https://bugreport.java.com/bugreport/crash.jsp
#

--------------- S U M M A R Y ------------

Command Line: -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI -XX:MaxRAMPercentage=6 -XX:+CreateCoredumpOnCrash -XX:+UnlockExperimentalVMOptions -XX:+UseZGC -XX:MaxRAMPercentage=50 -XX:MaxMetaspaceSize=256m -XX:+HeapDumpOnOutOfMemoryError -XX:+CrashOnOutOfMemoryError -Djava.net.preferIPv6Addresses=false -XX:+DisplayVMOutputToStderr -Xlog:gc*,gc+heap=debug:gc.log:uptime,timemillis,level,tags -XX:+DisableExplicitGC -XX:+StartAttachListener --add-exports=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-exports=java.xml/com.sun.org.apache.xerces.internal.parsers=ALL-UNNAMED --add-exports=java.xml/com.sun.org.apache.xerces.internal.util=ALL-UNNAMED -Djava.io.tmpdir=/opt/mach5/mesos/work_dir/slaves/fcf4c0c4-d73e-4321-860c-6613427db92b-S1793/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/8436a87f-0cb7-458a-ada1-a7ec3f525fe6/runs/2d8ffc73-0050-4316-a8b7-ad7fa3cc3930/testoutput/test-support/jtreg_closed_test_hotspot_jtreg_applications_kitchensink_Kitchensink_java/scratch/0/java.io.tmpdir -Duser.home=/opt/mach5/mesos/work_dir/slaves/fcf4c0c4-d73e-4321-860c-6613427db92b-S1793/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/8436a87f-0cb7-458a-ada1-a7ec3f525fe6/runs/2d8ffc73-0050-4316-a8b7-ad7fa3cc3930/testoutput/test-support/jtreg_closed_test_hotspot_jtreg_applications_kitchensink_Kitchensink_java/scratch/0/user.home -agentpath:/opt/mach5/mesos/work_dir/jib-master/install/jdk-14+27-1297/linux-x64-debug.test/hotspot/jtreg/native/libJvmtiStressModule.so -XX:NativeMemoryTracking=detail applications.kitchensink.process.stress.Main /opt/mach5/mesos/work_dir/slaves/fcf4c0c4-d73e-4321-860c-6613427db92b-S1793/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/8436a87f-0cb7-458a-ada1-a7ec3f525fe6/runs/2d8ffc73-0050-4316-a8b7-ad7fa3cc3930/testoutput/test-support/jtreg_closed_test_hotspot_jtreg_applications_kitchensink_Kitchensink_java/scratch/0/kitchensink.final.properties

Host: Intel(R) Xeon(R) Platinum 8167M CPU @ 2.00GHz, 8 cores, 58G, Oracle Linux Server release 7.6
Time: Tue Dec 10 16:35:24 2019 UTC elapsed time: 633 seconds (0d 0h 10m 33s)

--------------- T H R E A D ---------------

Current thread (0x00007f550c08b780): GCTaskThread "ZWorker#0" [stack: 0x00007f5511884000,0x00007f5511984000] [id=2057]

Stack: [0x00007f5511884000,0x00007f5511984000], sp=0x00007f5511982c00, free space=1019k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x1830d23] ZRootsIteratorCodeBlobClosure::do_code_blob(CodeBlob*)+0x73
V [libjvm.so+0x15200a8] ServiceThread::oops_do(OopClosure*, CodeBlobClosure*)+0x38
V [libjvm.so+0x1830daf] ZRootsIteratorThreadClosure::do_thread(Thread*)+0x5f
V [libjvm.so+0x16990c9] Threads::possibly_parallel_threads_do(bool, ThreadClosure*)+0x169
V [libjvm.so+0x182eb08] ZRootsIterator::oops_do(ZRootsIteratorClosure*)+0x1f8
V [libjvm.so+0x18066f0] ZMarkRootsTask::work()+0x20
V [libjvm.so+0x183a09d] ZTask::GangTask::work(unsigned int)+0x1d
V [libjvm.so+0x17c4cd4] GangWorker::run_task(WorkData)+0x84
V [libjvm.so+0x17c4e18] GangWorker::loop()+0x48
V [libjvm.so+0x169bbb6] Thread::call_run()+0xf6
V [libjvm.so+0x13c9cae] thread_native_entry(Thread*)+0x10e

Register to memory mapping:

RAX=0x00007f5515434000 points into unknown readable memory: 58 00 00 00 00 00 00 00
RBX=0x00007f54fc93c010 is at entry_point+-496 in (nmethod*)0x00007f54fc93c010
RCX=0x00007f551436cc10: <offset 0x00000000019cbc10> in /opt/mach5/mesos/work_dir/jib-master/install/jdk-14+27-1297/linux-x64-debug.jdk/jdk-14/fastdebug/lib/server/libjvm.so at 0x00007f55129a1000
RDX=0x00007f551434b058: <offset 0x00000000019aa058> in /opt/mach5/mesos/work_dir/jib-master/install/jdk-14+27-1297/linux-x64-debug.jdk/jdk-14/fastdebug/lib/server/libjvm.so at 0x00007f55129a1000
RSP=0x00007f5511982c00 points into unknown readable memory: 50 2c 98 11 55 7f 00 00
RBP=0x00007f5511982c10 points into unknown readable memory: 40 2c 98 11 55 7f 00 00
RSI=0x000000000000009c is an unknown value
RDI=0x00007f551434b028: <offset 0x00000000019aa028> in /opt/mach5/mesos/work_dir/jib-master/install/jdk-14+27-1297/linux-x64-debug.jdk/jdk-14/fastdebug/lib/server/libjvm.so at 0x00007f55129a1000
R8 =0x000043ab02000004 is an unknown value
R9 =0x0000000000000001 is an unknown value
R10=0x0 is NULL
R11=0x00007f52df000000 points into unknown readable memory: 0f 00 00 00 00 00 00 00
R12=0x00007f5511982c50 points into unknown readable memory: f0 61 70 14 55 7f 00 00
R13=0x0000000000000240 is an unknown value
R14=0x00007f5298192468 points into unknown readable memory: 00 00 00 00 00 00 00 00
R15=0x00007f5511982ce0 points into unknown readable memory: 18 62 70 14 55 7f 00 00


Registers:
RAX=0x00007f5515434000, RBX=0x00007f54fc93c010, RCX=0x00007f551436cc10, RDX=0x00007f551434b058
RSP=0x00007f5511982c00, RBP=0x00007f5511982c10, RSI=0x000000000000009c, RDI=0x00007f551434b028
R8 =0x000043ab02000004, R9 =0x0000000000000001, R10=0x0000000000000000, R11=0x00007f52df000000
R12=0x00007f5511982c50, R13=0x0000000000000240, R14=0x00007f5298192468, R15=0x00007f5511982ce0
RIP=0x00007f55141d1d23, EFLAGS=0x0000000000010297, CSGSFS=0x002b000000000033, ERR=0x0000000000000006
  TRAPNO=0x000000000000000e

Top of Stack: (sp=0x00007f5511982c00)
0x00007f5511982c00: 00007f5511982c50 00007f54f41ad6f8
0x00007f5511982c10: 00007f5511982c40 00007f5513ec10a8
0x00007f5511982c20: 00007f5511982ce0 00007f550c5777d0
0x00007f5511982c30: 00007f5511982ce0 0000000000000240

Instructions: (pc=0x00007f55141d1d23)
0x00007f55141d1c23: 07 ff 50 10 48 8b 03 48 85 c0 74 09 5b 41 5c 5d
0x00007f55141d1c33: c3 0f 1f 40 00 5b 49 83 44 24 10 01 41 5c 5d c3
0x00007f55141d1c43: 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 48 8d 05
0x00007f55141d1c53: a1 a6 53 00 55 be 8c 01 00 00 48 8d 3d c4 93 17
0x00007f55141d1c63: 00 48 8b 00 48 89 e5 c6 00 58 e8 7e 02 1d ff 5d
0x00007f55141d1c73: e9 b8 09 b9 ff 0f 1f 84 00 00 00 00 00 48 8d 05
0x00007f55141d1c83: 71 a6 53 00 55 be 8c 01 00 00 48 8d 3d 94 93 17
0x00007f55141d1c93: 00 48 8b 00 48 89 e5 c6 00 58 e8 4e 02 1d ff 5d
0x00007f55141d1ca3: e9 88 09 b9 ff 0f 1f 84 00 00 00 00 00 55 48 89
0x00007f55141d1cb3: e5 41 54 49 89 fc 48 89 f7 53 48 8b 06 48 89 f3
0x00007f55141d1cc3: ff 50 10 48 85 db 74 10 84 c0 74 0c 48 89 df e8
0x00007f55141d1cd3: 59 6a b1 ff 84 c0 75 05 5b 41 5c 5d c3 49 8b 74
0x00007f55141d1ce3: 24 08 48 89 df e8 73 04 fe ff 48 89 df e8 0b 04
0x00007f55141d1cf3: fe ff 41 0f b6 54 24 10 38 d0 74 39 48 8d 05 f2
0x00007f55141d1d03: a5 53 00 48 8d 15 4b 93 17 00 48 8d 0d fc ae 19
0x00007f55141d1d13: 00 be 9c 00 00 00 48 8d 3d 08 93 17 00 48 8b 00
0x00007f55141d1d23: c6 00 58 31 c0 e8 13 fd 1c ff e8 fe 08 b9 ff 41
0x00007f55141d1d33: 0f b6 54 24 10 84 d2 74 9f 48 89 df 5b 41 5c 5d
0x00007f55141d1d43: e9 e8 03 fe ff 0f 1f 84 00 00 00 00 00 55 48 8d
0x00007f55141d1d53: 05 98 44 53 00 48 8d 0d 61 e8 f9 ff 48 89 e5 41
0x00007f55141d1d63: 54 49 89 fc 53 48 89 f3 48 83 ec 20 48 8b 77 08
0x00007f55141d1d73: 48 89 45 d0 48 89 75 d8 48 8b 06 48 8b 50 28 31
0x00007f55141d1d83: c0 48 39 ca 75 47 48 8d 0d 5c bd 53 00 88 45 e0
0x00007f55141d1d93: 48 8d 55 d0 48 89 df 48 8b 03 80 39 00 b9 00 00
0x00007f55141d1da3: 00 00 48 0f 44 d1 ff 90 c8 00 00 00 49 8b 7c 24
0x00007f55141d1db3: 08 48 8d 15 f5 e7 f9 ff 48 8b 07 48 8b 40 20 48
0x00007f55141d1dc3: 39 d0 75 19 48 83 c4 20 5b 41 5c 5d c3 48 89 f7
0x00007f55141d1dd3: ff d2 49 8b 74 24 08 eb ad 0f 1f 40 00 48 89 de
0x00007f55141d1de3: ff d0 48 83 c4 20 5b 41 5c 5d c3 66 90 0f 1f 80
0x00007f55141d1df3: 00 00 00 00 00 00 00 00 00 00 00 00 00 48 8d 05
0x00007f55141d1e03: d5 0e 5e 00 8b 00 c3 66 0f 1f 44 00 00 48 89 f8
0x00007f55141d1e13: c3 66 66 2e 0f 1f 84 00 00 00 00 00 90 e9 4b 33


Stack slot to memory mapping:
stack at sp + 0 slots: 0x00007f5511982c50 points into unknown readable memory: f0 61 70 14 55 7f 00 00
stack at sp + 1 slots: 0x00007f54f41ad6f8 points into unknown readable memory: 58 5a 70 14 55 7f 00 00
stack at sp + 2 slots: 0x00007f5511982c40 points into unknown readable memory: 80 2c 98 11 55 7f 00 00
stack at sp + 3 slots: 0x00007f5513ec10a8: <offset 0x00000000015200a8> in /opt/mach5/mesos/work_dir/jib-master/install/jdk-14+27-1297/linux-x64-debug.jdk/jdk-14/fastdebug/lib/server/libjvm.so at 0x00007f55129a1000
stack at sp + 4 slots: 0x00007f5511982ce0 points into unknown readable memory: 18 62 70 14 55 7f 00 00
stack at sp + 5 slots: 0x00007f550c5777d0 is a thread
stack at sp + 6 slots: 0x00007f5511982ce0 points into unknown readable memory: 18 62 70 14 55 7f 00 00
stack at sp + 7 slots: 0x0000000000000240 is an unknown value