I've observed that we hit the MemLimit with vmTestbase/vm/mlvm/meth/stress/compiler/i2c_c2i/Test.java in a seemingly unrelated patch over and over again but never with mainline. I've now observed it again with a completely different patch. I'm therefore filing this bug to further investigate since I don't think that my patches have anything to do with it.

Additional VM flags required:

-Xcomp -ea -esa -XX:CompileThreshold=100 -XX:+UnlockExperimentalVMOptions -server -XX:-TieredCompilation -XX:+DeoptimizeALot

#
# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (/opt/mach5/mesos/work_dir/slaves/a4a7850a-7c35-410a-b879-d77fbb2f6087-S15438/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/cac83830-bb2d-420d-b575-debda39cf5a6/runs/092497a8-b7ab-448c-acd2-5d0fd1e146cc/workspace/open/src/hotspot/share/compiler/compilationMemoryStatistic.cpp:559), pid=4003, tid=4104
# fatal error: c2 java/lang/invoke/LambdaForm$MH+0x00007f3af3efb000::invoke((Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;): Hit MemLimit (limit: 1073741824 now: 1073752152)


Stack: [0x00007f3b338fa000,0x00007f3b339fa000], sp=0x00007f3b339f4d30, free space=1003k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x18a3fa2] VMError::report_and_die(int, char const*, char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned long)+0x482 (compilationMemoryStatistic.cpp:559)
V [libjvm.so+0xa96855] report_fatal(VMErrorType, char const*, int, char const*, ...)+0x105 (debug.cpp:214)
V [libjvm.so+0x9c5eed] CompilationMemoryStatistic::on_arena_change(long, Arena const*)+0x3fd (compilationMemoryStatistic.cpp:559)
V [libjvm.so+0x5f2efc] Arena::grow(unsigned long, AllocFailStrategy::AllocFailEnum)+0x10c (arena.cpp:299)
V [libjvm.so+0xe09bdf] IndexSet::initialize(unsigned int)+0x20f (arena.hpp:119)
V [libjvm.so+0xdf1016] PhaseIFG::init(unsigned int)+0xf6 (ifg.cpp:56)
V [libjvm.so+0xd697aa] PhaseCFG::global_code_motion()+0x78a (gcm.cpp:1577)
V [libjvm.so+0xd6c171] PhaseCFG::do_global_code_motion()+0x51 (gcm.cpp:1644)
V [libjvm.so+0x9ec994] Compile::Code_Gen()+0x2a4 (compile.cpp:2966)
V [libjvm.so+0x9ef6ac] Compile::Compile(ciEnv*, ciMethod*, int, Options, DirectiveSet*)+0x1bdc (compile.cpp:886)
V [libjvm.so+0x844875] C2Compiler::compile_method(ciEnv*, ciMethod*, int, bool, DirectiveSet*)+0x1d5 (c2compiler.cpp:142)
V [libjvm.so+0x9fb3e8] CompileBroker::invoke_compiler_on_method(CompileTask*)+0x928 (compileBroker.cpp:2303)
V [libjvm.so+0x9fc078] CompileBroker::compiler_thread_loop()+0x478 (compileBroker.cpp:1961)
V [libjvm.so+0xea461c] JavaThread::thread_main_inner()+0xcc (javaThread.cpp:759)
V [libjvm.so+0x17c0dd6] Thread::call_run()+0xb6 (thread.cpp:225)
V [libjvm.so+0x14a7ed7] thread_native_entry(Thread*)+0x127 (os_linux.cpp:846)