The os::available_memory call is supposed to report, at a snapshot, how much more memory is available for taking from the OS. This is used by testing frameworks to see how many JVMs can run at the same time. Unfortunately, the implementation is wrong, which is why there have been numerous reports that you need "2x" or "3x" swap on Solaris. As a matter of fact, it is only a problem that the available_memory implementation is wrong, leading to overcommitting when creating JVMs.

Unfortunately, the implementation uses sysconf(_SC_AVPHYS_PAGES) to query how much memory is available, assuming mmapped memory would make this number smaller.

The problem is that if you mmap NORESERVE, which is the case for the reserved rather than committed portion of the Java heap, then it doesn't count as used memory by Solaris, since it merely reserves a range of virtual address space, with no link to actual memory backing it up.

So in summary, after starting a JVM that reserves 1 TB max heap size, only the portion of the heap that is committed and used, is reported as used as observed by os::available_memory, resulting in overcommits and lots of swapping problems on Solaris.

I think at the core of the problem is what we mean by a max size for the Java heap. It either means "I want to make sure I can allocate this much memory for the heap", or it is just a conservative maximum meaning "I know I won't allocate more than this", allowing the JVM to build appropriate auxiliary GC datastructures, but not expecting to be able to use more memory than available from the OS. It should be either one of those two at a time, and now it is a blend of them.

I propose a JVM flag, e.g. -XX:+UseDeterministicHeapReservation to mean you want a guarantee you will get this much memory, and hence allocate enough swap to be able to deal with it straight away.

Note that when mmapping in memory, it is only mapped to physical memory once used. So then we could mmap without NORESERVE when reserving the heap, and os::available_memory will report sane values. This is what the test runs should use, when deterministic behavior is wanted.

When using -XX:-UseDeterministicHeapReservation, mmap NORESEVE would be used instead, and we would never be able to know if that max heap can ever be reached, but allocate no resources at all for it, and simply deal with overcommits as we commit. Now, os::available_memory only reports how much actual memory is being used, and hence does not give deterministic behavior with any form of guarantees of how many JVMs we can run, which we happily accept.