G1: Excessive c_heap (malloc) consumption

G1 use a lot of c_heap (malloc) memory per GC thread (8Mb) which leads sometimes to OOM error. I collected some date using PrintMalloc:

src/share/vm/memory/universe.cpp
     }
   }
 
+ PrintMalloc=true;
   jint status = Universe::initialize_heap();
+ PrintMalloc=false;
   if (status != JNI_OK) {
     return status;
   }
@@ -1154,7 +1156,11 @@
   }
 
   // ("weak") refs processing infrastructure initialization
+ PrintMalloc=true;
+ tty->cr();
+ tty->print_cr("post_initialize");
   Universe::heap()->post_initialize();
+ PrintMalloc=false;
 
   GC_locker::unlock(); // allow gc after bootstrapping


sparc

G1:

java -d64 -XX:CICompilerCount=1 -XX:ParallelGCThreads=10 -Xms20g -Xmx20g -XX:+UseG1GC -version
os::malloc 2032 bytes --> 0x00000001001686d8
...
os::malloc 25 bytes --> 0x000000010ec0d388
-------
total 246,045,104 bytes

java -d64 -XX:CICompilerCount=1 -XX:ParallelGCThreads=20 -Xms20g -Xmx20g -XX:+UseG1GC -version
os::malloc 2032 bytes --> 0x00000001001686d8
...
os::malloc 25 bytes --> 0x00000001132b8048
-------
total 320,141,680 bytes


ParallelOldGC mostly used malloc in post_initialize() phase:

java -d64 -XX:-ZapUnusedHeapArea -XX:CICompilerCount=1 -XX:ParallelGCThreads=20 -Xms20g -Xmx20g -XX:+UseParallelOldGC -version > PS_20_malloc.log2
os::free 800 bytes --> 0x000000010015aaa8
os::free 200 bytes --> 0x0000000100164138
os::free 80 bytes --> 0x0000000100171f88
java version "1.6.0_25"
Java(TM) SE Runtime Environment (build 1.6.0_25-b06)
Java HotSpot(TM) 64-Bit Server VM (build 24.0-b18-internal-jvmg, mixed mode)

os::malloc 152 bytes --> 0x0000000100167a48
...
os::malloc 25 bytes --> 0x0000000100199e68

post_initialize
os::malloc 112 bytes --> 0x00000001002e6f98
...
os::malloc 1048576 bytes --> 0x0000000104290ef8
os::malloc 472 bytes --> 0x0000000104390f38
os::malloc 1048576 bytes --> 0x0000000104391148
------------------------------------------------
total 68,854,192 bytes
See public comments.