Summary

Automatically and dynamically adapt the maximum Java heap size according to the environment when using the G1 garbage collector (G1).

Goals

When using G1:

• Dynamically adapt the maximum Java heap size to changes to the available memory of the environment.
• Typical performance should not change noticeably compared to when the end user manually correctly tunes Java heap size.

Non-Goals

It is not a goal of this JEP to:

• Find a static optimal maximum Java heap size.
• Remove the existing configurability of static heap bounds using existing Java heap sizing options.
• Change existing dynamic adaption of the current Java heap size to the workload.

Motivation

The Java Virtual Machine (JVM) stores most Java objects in the (Java object) heap, where they are garbage-collected when no longer needed. The size of the heap significantly impacts the performance and memory usage of an application, so the JVM allows the size to be constrained manually: java -Xms... sets the minimum and initial heap size and -Xmx... sets the maximum heap size. All garbage collectors (GCs) respect these settings.

Within these boundaries, there is a delicate performance trade-off between memory use and an applications's throughput. In a highly simplified view of tracing garbage collectors — the kind offered by the JVM — the amount of work required to collect garbage is not related to the amount of dead objects in the heap but to the amount of live objects, and so it is independent of the heap size. However, with a larger heap, the GC will need to perform less frequent collection cycles than with a smaller heap, and so the overall CPU usage of garbage collection is lower the bigger the heap and vice-versa. This overhead may come at the expense of application throughput, and so a larger heap will yield lower GC overhead and possibly higher application throughput.

When users choose the G1 garbage collector, the maximum heap size is the only memory-related option they should need to set. Unfortunately, setting a good maximum heap size is notoriously difficult. An application's memory usage or requirements depend on the input and changes during different phases of execution, yet the user is asked to set a maximum heap size across the entire runtime of the application in advance. In addition to the Java heap memory, the JVM itself allocates a certain, non-fixed amount of memory for its own purpose that also depends on the selected maximum heap size which the user needs to consider when determining it. If the maximum Java heap size is set too low, the application can run out of memory; if set too high, the machine can run out of memory. Finding a good maximum heap size involves measuring application memory usage and throughput for different maximum Java heap sizes in an experimental setup that provides a representative workload to an application. Setting up such an environment alone is challenging for most developers.

An application's actual memory consumption is dependent not only on the application's code but also on the maximum heap size setting. G1 interprets a higher maximum heap size as an allowance given by the user for consuming more memory to meet the CPU overhead and pause time goals determined by the -XX:GCTimeRatio and -XX:MaxGCPauseMillis options respectively. The G1 garbage collector is allowed to use close to the maximum heap size to meet these goals.

We believe that the JVM can automatically control the maximum Java heap size and balance CPU and memory consumption better than the user can manually with explicit configuration. The JVM should monitor the overall free memory usage of the machine, and use that current information to automatically determine a maximum heap size. Users should not need to set -Xmx when choosing G1. G1 should be allowed to use all the memory of the machine if needed, just as a C application does, yet timely respond to a reduction in free memory on the machine. Finally, users should be able to express a preference between less garbage collector CPU usage (but more memory use) and more garbage collector CPU usage (but less memory use) similarly to what JEP XXX provides for the ZGC collector.

Note: Because the memory consumption of a Java process depends not only on how the application code uses memory but also on the selected garbage collector and maximum heap size, the same application may use more or less memory with different GCs and maximum size settings. This means that reporting memory usage in benchmarks is meaningless unless the heap configuration is detailed.

With automatic heap sizing, the memory consumption may further depend on how much free memory is available on the machine. The same application may consume more memory if it is not needed for other applications and less memory when other applications running on the same machine are using more memory.

Description

G1 will automatically select a maximum heap size that dynamically adapts to changing circumstances in the environment.

The selected heap size will lie between the minimum (-Xms) and maximum (-Xmx) heap configuration, but when one or both are not configured by the user, the default maximum and minimum heap sizes will be changed to give the automatic heap sizing as much flexibility as possible, as follows:

• Default minimum and initial heap sizes (-Xms) are changed to 4 MB.
• Default maximum heap size (-Xmx) is changed to 100% of the available RAM of the computer or the compressed oops boundary if compressed oops are in use, whichever is lower (see Initial maximum heap size), minus a small reserve.

Automatic heap sizing for G1 provides the following features, which will be expounded on below:

• A dynamic maximum heap size that adapts to changes in the availability of free memory on the machine.
• The dynamic maximum heap size automatically shrinks in response to memory on the machine being used up by other applications. Excess memory will be given back to the operating system in a timely manner.
• The dynamic maximum heap size quickly grows to accommodate sudden increases in application allocation requests.
• The dynamic maximum heap size proactively grows to accommodate future behavior of the application — as predicted by the JVM - to meet garbage collector CPU usage and pause time goals. This expansion is done concurrently with the application; the added memory is proactively and concurrently committed and paged to avoid any slowdowns due to OS operations.
• A new method to indicate the user's intent about the performance/memory footprint tradeoff.

With these changes, the need for configuring the maximum heap size when using G1 should drop significantly.

Automatic heap resizing will be enabled by default when using the G1 garbage collector. Since the G1 garbage collector is currently the default collector, there is typically nothing to do for the end user to benefit from these changes.

Rapid expansion

Decent startup performance is an important goal for a balanced garbage collector like G1. When the JVM boots with an initial heap size of 4 MB on a large computer with many cores, it will quickly find itself in a situation where that amount of Java heap is not enough. The application might require a heap size of, for example, 20 GB, in which case the garbage collector will need to expand the heap and do so very quickly.

With the heap starting out small, garbage collection will likely trigger early on, and the application is likely to need more memory faster than the garbage collector can free it. To accommodate the application's allocation, G1 will expand the heap as per existing heuristics.

In addition to expanding the heap in the event of sudden allocations, the garbage collector will use the recent CPU usage of the garbage collector and application activity to grow the heap as necessary. Growing the heap allows G1 to reduce the frequency of collections, which in turn reduces the garbage collector's CPU usage and so may improve the application's throughput.

Automatic dynamic tuning

After finding an initial maximum heap size, G1 continuously monitors the behavior of the garbage collector, the application, and the environment, adjusts the maximum heap size, and applies incremental tuning of actual heap size using existing mechanisms.

Initial maximum heap size

G1 determines the initial absolute maximum heap size using 100% the available RAM in the environment and observing current settings for compressed oops minus a small reserve of free memory.

G1 limits the default absolute maximum heap size to the compressed oops boundary. This is approximately 32 GB without additional options. This keeps G1 profiting from the performance advantages compressed oops provide in the common case of applications requiring less than that amount of Java heap. The user may manually expand the range of compressed oops via -XX:ObjectAlignmentInBytes or disable compressed oops completely via -XX:-UseCompressedOops to increase this absolute maximum heap size. When disabling compressed oops, G1 may use all of the available memory in the environment minus the reserve.

A reserve of system memory unused by the JVM is generally helpful, even in even in single-application deployments. For example, it allows for file caches to be populated, which typically improves the performance of the system. At the same time, as explained in rapid expansion this unused memory acts as a safety buffer that can be used to avoid unexpected responses by the garbage collection algorithm that manifest in whole heap collections if the application's allocation rate rises suddenly.

G1GCIntensity

Developers may have different preferences on how to trade garbage collector CPU usage for memory footprint for different applications.

If the user is unhappy with the default, use the existing -XX:GCTimeRatio flag to control the expected CPU usage/memory footprint tradeoff. For compatibility with other collectors, there is a new option -XX:G1GCIntensity that influences the same tradeoff with a different scale. It takes an integer value between 0 and infinity, although a reasonable value would be between 0 and 10. The default is 5. Its default value corresponds to a reasonable balance of CPU and memory usage. Raise it for a larger CPU usage and a smaller heap; lower it for less CPU usage and a larger heap.

We intentionally do not define an exact relationship between -XX:G1GCIntensity and -XX:GCTimeRatio. This is so we may evolve and improve the automatic tuning policies over time. Instead, the guiding principle is that higher values for -XX:G1GCIntensity let G1 spend more time on garbage collection, resulting in more frequent collections, higher CPU usage and lower memory usage; lower values make the GC less intensive, triggering less frequent collections but requiring a larger heap.

The -XX:G1GCIntensity flag is manageable, meaning it may be updated at runtime, if desired.

Measuring CPU overhead

G1 is a generational garbage collector. Young objects are placed in a young generation, collected more frequently during young-only phases. Old objects are promoted to the old generation without collecting it. The old generation is collected less frequently in the space reclamation phase (which collects both generations) according to the G1 garbage collection cycle.

To decide whether to grow or shrink the heap G1 tracks garbage collection CPU usage, trying to keep the target garbage collection CPU usage derived from the current value of -XX:GCTimeRatio, and modified by -XX:G1GCIntensity, the current maximum heap size, and other existing variables. In the existing heuristics, G1 examines the overall GC CPU usage at every young collection. If a sequence of garbage collections consumes more CPU than the CPU target, then the heap expands. Conversely, if a sequence of collections consumes less CPU than that CPU target, the heap shrinks.

Garbage collection activity is not the only CPU overhead imposed by the GC. Frequent collection cycles impose other CPU penalties on the application, such as increased execution of GC barriers (instructions performed when accessing Java objects while a garbage collection cycle is in progress). The existing automatic heap resizing heuristics take this into account and expand the heap to minimize such impacts. These impacts can be larger the more CPU the application itself consumes, and this, too, is taken into consideration.

Responding to machine memory pressure

The JVM process can automatically find an appropriate heap size for the given target garbage collector CPU usage. However, if we let the JVM use as much memory as it wants, the machine may not have enough memory available to run other processes. Therefore, in addition to monitoring the behavior of the Java application, G1 will also continuously monitor the overall available free memory on the machine. In response to less free memory on the machine, G1 will adjust its internal target CPU usage and attempt to shrink the heap.

This new maximum heap size will automatically consider the JVM's internal native memory usage: If the JVM requires more native memory, the amount of free memory in the environment adjusts, and so the automatically determined maximum heap size.

Other applications running on the machine may also deplete that reserve, which will have the effect of increasing the GC frequency, making it consume less memory to shrink the heap at the cost of spending more CPU. (Multiple JVMs using G1 and running on the same machine will reach an equilibrium rather than fight over memory with each other.)

On MacOS or Windows with memory compression enabled, the ratio of compressed and uncompressed memory is continuously monitored. The perceived size of the memory reserve is scaled according to that compression ratio. When the OS starts compressing more memory, the GC will work harder to reclaim garbage and give memory back to the OS, relieving its compression pressure.

Testing

This enhancement primarily affects performance metrics. Therefore, it will be thoroughly performance-tested with a wide variety of workloads. The defined success metrics will be tested on said workloads.

Risks and Assumptions

By changing the default maximum heap size from 25% of the available memory to all available memory, there is a risk that the new heuristics use more memory than the current implementation would, and so other processes may run out of memory. However, even with a 25% default maximum heap policy there is already a risk of that happening when several JVMs using that default run on the same machine. Moreover, the dynamically updated maximum heap size is very likely to be able to throw an out-of-memory error before exceeding the computer memory limits.