In 2021 we discussed the possibility of letting the JVM auto-trim the native heap to combat Glibc memory retention.

The Glibc allocator is reluctant to return memory to the OS, much more so than other allocators. Temporary malloc spikes often carry over as permanent RSS increase.

Note that C-heap retention is difficult to observe. Since it is freed memory, it won't show up in NMT, it is just a part of private RSS.

Theoretically, retained memory is not lost since it will be reused by future mallocs. Retaining memory is therefore a bet on the future behavior of the app. The allocator bets on the application needing memory in the near future, and to satisfy that need via malloc.

But an app's malloc load can fluctuate wildly, with temporary spikes and long idle periods. And if the app rolls its own custom allocators atop of mmap, as hotspot does, a lot of that memory cannot be reused even though it counts toward its memory footprint.

To help, Glibc exports an API to trim the C-heap: malloc_trim(3). It would be good if we were to integrate it into the JVM as an optional GC step.


[1] https://mail.openjdk.org/pipermail/hotspot-dev/2021-August/054323.html