runtime/cds/appcds/cacheObject/HeapFragmentationTest.java crashed with OutOfMemory

The following test failed in the JDK17 CI:

runtime/cds/appcds/cacheObject/HeapFragmentationTest.java

Here's a snippet from the log file:

0.820s][info][gc,heap] GC(4) Eden regions: 0->0(1)
[0.820s][info][gc,heap] GC(4) Survivor regions: 0->0(5)
[0.820s][info][gc,heap] GC(4) Old regions: 1->1
[0.820s][info][gc,heap] GC(4) Archive regions: 0->0
[0.820s][info][gc,heap] GC(4) Humongous regions: 0->0
[0.820s][info][gc ] GC(4) Pause Full (G1 Humongous Allocation) 0M->0M(8M) 90.869ms
[0.833s][info][gc ] GC(1) Concurrent Undo Cycle 415.336ms
Aborting due to java.lang.OutOfMemoryError: Java heap space
# To suppress the following error report, specify this argument
# after -XX: or in .hotspotrc: SuppressErrorAt=/debug.cpp:364
#
# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (/opt/mach5/mesos/work_dir/slaves/3c846bae-ce30-4a97-93ee-9fef4497ccb6-S79213/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/3d136cf9-b484-41b0-a4b4-5ded75b927ce/runs/990bcb78-f7c1-4ada-8b7a-5a7801a37528/workspace/open/src/hotspot/share/utilities/debug.cpp:364), pid=32046, tid=32049
# fatal error: OutOfMemory encountered: Java heap space
#
# JRE version: Java(TM) SE Runtime Environment (17.0+24) (fastdebug build 17-ea+24-LTS-2133)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 17-ea+24-LTS-2133, mixed mode, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
# Core dump will be written. Default location: Core dumps may be processed with "/opt/core.sh %p" (or dumping to /opt/mach5/mesos/work_dir/slaves/3c846bae-ce30-4a97-93ee-9fef4497ccb6-S79213/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/2c4217ec-0c42-4111-a920-a26f6eeef273/runs/52d08d41-3773-43fb-bd87-ef71779f483b/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_runtime/scratch/2/core.32046)
#
# An error report file with more information is saved as:
# /opt/mach5/mesos/work_dir/slaves/3c846bae-ce30-4a97-93ee-9fef4497ccb6-S79213/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/2c4217ec-0c42-4111-a920-a26f6eeef273/runs/52d08d41-3773-43fb-bd87-ef71779f483b/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_runtime/scratch/2/hs_err_pid32046.log
];
 stderr: [java version "17-ea" 2021-09-14 LTS
Java(TM) SE Runtime Environment (fastdebug build 17-ea+24-LTS-2133)
Java HotSpot(TM) 64-Bit Server VM (fastdebug build 17-ea+24-LTS-2133, mixed mode)
]
 exitValue = 134

java.lang.RuntimeException: 'array.length = 1048576000' missing from stdout/stderr

at jdk.test.lib.process.OutputAnalyzer.shouldContain(OutputAnalyzer.java:206)
at jdk.test.lib.cds.CDSTestUtils.checkMatches(CDSTestUtils.java:483)
at jdk.test.lib.cds.CDSTestUtils$Result.assertNormalExit(CDSTestUtils.java:194)
at HeapFragmentationTest.main(HeapFragmentationTest.java:80)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298)
at java.base/java.lang.Thread.run(Thread.java:831)

JavaTest Message: Test threw exception: java.lang.RuntimeException
JavaTest Message: shutting down test

result: Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: 'array.length = 1048576000' missing from stdout/stderr


Here's the crashing thread's stack:

--------------- T H R E A D ---------------

Current thread (0x00007f9ab4039900): JavaThread "main" [_thread_in_vm, id=32049, stack(0x00007f9abb7d7000,0x00007f9abb8d8000)]

Stack: [0x00007f9abb7d7000,0x00007f9abb8d8000], sp=0x00007f9abb8d62e0, free space=1020k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x198a0f0] VMError::report_and_die(int, char const*, char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned long)+0x1a0
V [libjvm.so+0xa81ceb] report_fatal(VMErrorType, char const*, int, char const*, ...)+0x12b
V [libjvm.so+0xa81f9d] report_java_out_of_memory(char const*)+0xed
V [libjvm.so+0x141114a] MemAllocator::Allocation::check_out_of_memory()+0xda
V [libjvm.so+0x14139f1] MemAllocator::allocate() const+0x191
V [libjvm.so+0x18e05a6] TypeArrayKlass::allocate_common(int, bool, JavaThread*)+0x2e6
V [libjvm.so+0x154ce3f] oopFactory::new_typeArray(BasicType, int, JavaThread*)+0xff
V [libjvm.so+0xe565bf] InterpreterRuntime::newarray(JavaThread*, BasicType, int)+0x10f
j HeapFragmentationApp.main([Ljava/lang/String;)V+20
v ~StubRoutines::call_stub
V [libjvm.so+0xe704b4] JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*)+0x504
V [libjvm.so+0xfa9275] jni_invoke_static(JNIEnv_*, JavaValue*, _jobject*, JNICallType, _jmethodID*, JNI_ArgumentPusher*, JavaThread*) [clone .constprop.1]+0x375
V [libjvm.so+0xfac8b5] jni_CallStaticVoidMethod+0x1c5
C [libjli.so+0x47d7] JavaMain+0xd37
C [libjli.so+0x7ce9] ThreadJavaMain+0x9

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j HeapFragmentationApp.main([Ljava/lang/String;)V+20
v ~StubRoutines::call_stub