Summary

Introduce an API for dynamically creating memory tags for tracking native memory via Hotspot's NMT (Native Memory Tracking) subsystem. Expose this new API to the entire JDK, which will enable tracking of native memory for both native C libraries and libraries using FFM, with the goal of achieving a higher level of granularity for native memory tracking, which will help with diagnosing native memory issues.

Goals

• Improve native memory diagnosis within the JVM

Adding dynamic memory tags to the Hotspot VM allows for more granular memory tracking. This may improve native memory diagnosis capabilities within the VM.

• Enable usage of NMT for the JDK's C libraries

The JDK today has multiple native libraries which are called into by Java code via JNI. Our plan is to expose an NMT interface through a jvm.h API. This will allow native core libraries developers to opt into NMT, which will in turn enable native memory tracking and diagnostics for the JDK's native libraries.

• Enable usage of NMT for the JDK's FFM libraries

The future of the JDK's native code capabilities lie in project Panama and its new FFM interface. NMT ought to prepare for this future right now, and support a Java interface to NMT. The plan is to implement such an interface with this JEP. Similarly to the C interface, this will enable core library developers to opt into using NMT, which will lead to further improvement of native memory tracking and diagnostics.

Non-Goals

• Improving memory tracking for users of the Unsafe API

While we do not exclude any improvements to the tracking of the Unsafe API in the future, this JEP will not implement any such improvements.

• Implementing the usage of NMT within the JDK's C libraries or FFM code

Using the new APIs will be up to the maintainers and developers of said systems who wish to integrate with NMT.

• Enable usage of NMT for code not in the JDK repository

It is not planned in this JEP to support any code which is outside of the JDK repository. In other words, these new APIs will be for internal usage only.

• Enable usage of NMT's virtual memory registration API to the rest of the JDK

NMT consists of two interfaces:

1. The malloc registration interface
2. The virtual mapping registration interface

The goal with this JEP is to port the malloc registration interface to the rest of the JDK. As off-heap memory in FFM today is allocated using malloc, this gives us the most value for the least cost of implementation. The work presented in this JEP does not prevent porting the virtual mapping interface to the rest of the JDK in the future.

• Enabling Java stack traces in NMT's detailed mode

NMT has the capability of associating allocations with stack traces in its detailed mode. This is a very powerful feature, but it does not see as much use as the summary mode does. Therefore, detailed mode stack tracing will not be made available for FFM code with this JEP. This JEP will not prevent implementing this feature in the future.

Motivation

The NMT subsystem of Hotspot has been a valuable tool for JDK maintainers and Java users alike. It has been used for, among other things, diagnosing memory leaks and finding allocation hot spots, as well as basic memory safety utilizing its canary system. However, today this system is only available for tracking Hotspot code. As Java code is expected to interface more and more with native libraries through the usage of APIs such as that offered by FFM, the limitation of only tracking Hotspot code is becoming a true bottleneck in NMT's usefulness. Extending the usage of this subsystem to the entirety of the JDK is critical for utilizing the full potential of this tool. This will also allow us to gain experience and learn the lessons required to port NMT for usage outside of the JDK as well.

Description

NMT is a Hotspot subsystem which provides tracking of memory allocations. Each memory allocation is tagged with a category at the time of allocation, and statistics is kept by NMT for each category. A runningg JVM can produce a NMT statistics report by, for example, responding to a request made using the JCmd facility, or as part of JFR event production. This facility has turned out to be quite useful at detecting memory leaks in the VM, as you can see which categories are monotonically increasing by sampling at a regular interval. Today, NMT defines its categories statically. That is, they are specified within a C++ source file and any addition or removal necessitates a recompilation of the JVM. This work will do three things. First, we will make memory category creation dynamic. Allowing memory categories to be created at runtime means that no changes have to be made to the Hotspot source code in order to add a memory category. This opens up for the capability of doing the second step, which is to expose a JNI-compatible C API to NMT. This API can then be used by OpenJDK developers in order to gain native memory tracking for their C libraries. Finally, this JNI C API can be exposed to Java, through the usage of native methods in a NativeMemoryTracking class. Such a class can then be incorporated and used by FFM, in order to gain native memory tracking for Java as well. It is expected that these two final steps will be developed with input from the maintainers of the libraries which will use NMT.

Alternatives

With this proposal, NMT is something that a developer opts into by writing code which calls the NMT interface. An alternative solution to this is to implement the NMT interface as a dynamically loadable library. This can then replace the ordinary definitions of malloc and free, by using a mechanism such as LD_PRELOAD on Linux. In this way, tracking can be achieved automatically, without developer opt-in. The major issue with such an implementation is the tagging of the memory: what category should a specific allocation be considered as part of? The developer doesn't opt-in, and so cannot decide for themselves. Instead, NMT must decide, through some heuristic. Such a heuristic could be to name the category according to some function name, discovered through stack walking. This type of heuristic may lead to a larger than necessary amount of memory categories being allocated, and may lead to a poor categorization of the tracked memory. This idea was discarded because of such issues.