Failing to initialize VM with small initial heap when NUMA and large pages are enabled

While investigating another NUMA isseu I ran into this problem while running some jtreg-tests. It turns out that the test doesn't have anything to do with it, since it is reproducible simply by running:
java -XX:+UseNUMA -XX:+UseLargePages -Xms8m -version

When running this I get:
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000607600000, 3145728, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 3145728 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /localhome/sjohanss/jpg/bugs/jbs8016155/c-repro/hs_err_pid3647.log

I've investigated the issue and the problem seems to be that when using large pages and eden is small we can end up in a situation where we use mmap with MAP_FIXED but without MAP_HUGETLB on an address that is already mapped with MAP_HUGETLB. If the size of the MAP_FIXED mapping isn't a multiple of 2MB (huge page size) this will fail with errno ENOMEM (strange errno since we are not really out of memory). This will in turn lead to failing to initialize the VM even though there is memory left on the system.