Introduce a Hotspot option DumpPrivateMappingsInCore to determine if file-backed private mappings of the process can be dumped into the core file

Summary

Introduce a Hotspot option DumpPrivateMappingsInCore to determine if file-backed private mappings of the process can be dumped into the core file.

Problem

With JDK-8200613, changes are being introduced such that on Linux, the file backed private mappings for a process get dumped into the process core file by default (if UseSharedSpaces is on). These file backed private mappings would also include any NIO file mapping with MapMode.PRIVATE. and file mmap()-ed by any native library with MAP_PRIVATE. This results in a size increase for the core files on Linux.

Solution

The proposed solution is to introduce a VM option (DumpPrivateMappingsInCore) on Linux, so that users have an option of not getting these file backed private mappings dumped into the core files. The value of this option is true by default.

Specification

The proposal is to add the following snippet to: : src/hotspot/os/linux/globals_linux.hpp

• product(bool, DumpPrivateMappingsInCore, true, \
• "If true, sets bit 2 of /proc/PID/coredump_filter, thus " \
• "resulting in file-backed private mappings of the process to "\
• "be dumped into the corefile, if UseSharedSpaces is true.") \
• \