Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8276094

JEP 423: Region Pinning for G1

    XMLWordPrintable

    Details

    • Author:
      Hamlin Li
    • JEP Type:
      Feature
    • Exposure:
      Open
    • Subcomponent:
      gc
    • Scope:
      Implementation
    • Discussion:
      hotspot dash gc dash dev at openjdk dot java dot net
    • Effort:
      L
    • Duration:
      M
    • JEP Number:
      423

      Description

      Summary

      Reduce latency by implementing region pinning to G1, so that garbage collection need not be disabled during Java Native Interface (JNI) critical regions.

      Goals

      • No stalling of threads due to JNI critical regions.
      • No additional latency to start a garbage collection due to JNI critical regions.
      • No regressions in GC pause times when no JNI critical regions are active.
      • Minimal regressions in GC pause times when JNI critical regions are active.

      Motivation

      For interoperability with unmanaged programming languages such as C and C++, JNI defines functions to get and then release direct pointers to Java objects. These functions must always be used in pairs: First, get a pointer to an object (e.g., via GetPrimitiveArrayCritical); then, after using the object, release the pointer (e.g., via ReleasePrimitiveArrayCritical). Code within such function pairs is considered to run in a critical region, and the Java object available for use during that time is a critical object.

      When a Java thread is in a critical region, the JVM must take care not to move the associated critical object during garbage collection. It can do this by pinning such objects to their locations, essentially locking them in place as the GC moves other objects. Alternatively, it may simply disable GC whenever a thread is in a critical region.

      The default GC, G1, takes the latter approach, disabling GC during every critical region. This has a significant impact on latency: If a Java thread triggers a GC then it must wait until no other threads are in critical regions. The severity of the impact depends upon the frequency and duration of critical regions. In the worst cases users report critical sections blocking GC and their entire application for minutes, unnecessary out-of-memory conditions due to thread starvation, and even premature VM shutdown. Due to these problems, the maintainers of some Java libraries and frameworks have chosen not to use critical regions by default (e.g., JavaCPP) or even at all (e.g., Netty), even though doing so can adversely affect throughput.

      With the change that we propose here, Java threads will never wait for a G1 GC operation to complete.

      Description

      Background

      G1 partitions the heap into fixed-size memory regions (not to be confused with critical regions). G1 is a generational collector, so any non-empty region is a member of either the young generation or the old generation. In any particular collection operation, objects are evacuated (i.e., moved) from only a subset of the regions.

      When G1 is unable to find space to evacuate an object during a minor (i.e., young-generation) collection then it leaves the object in place and marks both it and its containing region as having failed evacuation. After evacuation, G1 fixes up the failed regions by promoting them from the young generation to the old generation,

      G1 is already capable of pinning objects to their memory locations during major (i.e., full) collection operations, simply by not evacuating the regions that contain them. G1 permanently pins humongous regions, which contain large objects, and archive regions, which contain objects loaded from class data sharing (CDS) archives. It also pins, for the duration of a single collection, any region that exceeds a specified liveness threshold.

      G1 cannot pin arbitrary regions during minor collection operations, though it does exclude humongous and archive regions from such collections.

      Pinning regions during minor collection operations

      We aim to achieve the goals of this JEP by extending G1 to pin arbitrary regions during both major and minor collection operations, as follows:

      • Maintain a count of the number of critical objects in each region: Increment it when a critical object in that region is obtained, and decrement it when that object is released. When the count is zero then garbage-collect the region normally; when the count is non-zero, consider the region to be pinned.

      • During a major collection, do not evacuate any pinned region.

      • During a minor collection, treat pinned regions in the young generation as having failed evacuation, thus promoting them to the old generation. Do not evacuate pinned regions in the old generation.

      Once we have done this then we can implement JNI critical regions — without disabling GC — by pinning regions that contain critical objects and continuing to collect garbage in unpinned regions.

      Handling failed evacuations more efficiently

      The current evacuation-failure mechanism assumes that failures are rare and that few objects are involved. G1 typically avoids evacuation failures entirely by sizing generations properly and by performing preventive garbage collections, so these are reasonable assumptions. A consequence of these assumptions, however, is that the current mechanism is not well suited to our purposes:

      • The code that records evacuation failures is not well optimized.

      • The post-evacuation fix-up phase walks through failed regions linearly and can concurrently process only whole regions, rather than individual objects.

      • Regions that are promoted to the old generation because they fail evacuation typically do not contain many live objects, and reclaiming the unused space in these regions requires significant time and effort. If many regions are promoted to the old generation then heap exhaustion becomes a real risk.

      Thomas Schatzl has suggested how to address these issues in a blog post and a set of related JBS issues.

      Alternatives

      The JNI specification suggests two other ways to implement critical regions:

      • At the start of a critical region, copy the critical object to the C heap, where it will not be moved; at the end of the critical region, copy it back. This is very inefficient in both time and space.

        In G1 we could do this only for critical objects in regions that cannot be pinned. Those regions are in the young generation, however, in which most object use and modification typically occurs, so we do not expect that this would help much.

      • Pin objects individually. G1 can only evacuate whole regions, so a single pinned object in a region would prevent the collection of that region. The end result would be little different from what we propose above except that it would have higher overhead, since tracking individual pinned objects is more costly than maintaining per-region counts of critical objects.

      Testing

      Aside from functionality tests, we will do benchmarking and performance measurements to collect the performance data necessary to ensure that our goals are met.

      Risks and Assumptions

      We assume that there will be no changes to the expected usage of JNI critical regions: They will continue to be used sparingly, and they will be short in duration.

      There is a risk of heap exhaustion when an application pins many regions at the same time. We have no solution for this, but the fact that the Shenandoah GC pins memory regions during JNI critical regions and does not have this problem suggests that it will not be a problem for G1.

        Attachments

          Issue Links

          1.
          G1: Fully support pinned regions for full gc Sub-task Resolved Thomas Schatzl  
          2.
          G1: Forwarding pointer removal thread sizing Sub-task Resolved Thomas Schatzl  
          3.
          Improve g1 evacuation failure injector performance Sub-task Resolved Thomas Schatzl  
          4.
          Compile in G1 evacuation failure injection code based on define Sub-task Resolved Thomas Schatzl  
          5.
          G1: Record regions where evacuation failed to provide targeted iteration Sub-task Resolved Hamlin Li  
          6.
          G1: Factor out concurrent segmented array from G1CardSetAllocator Sub-task Resolved Hamlin Li  
          7.
          G1: Optimize evacuation failure for regions with few failed objects Sub-task Resolved Hamlin Li  
          8.
          G1: Allow forced evacuation failure of first N regions in collection set Sub-task Resolved Hamlin Li  
          9.
          G1: Log basic statistics for evacuation failure Sub-task Resolved Hamlin Li  
          10.
          G1: Log further detailed statistics of evacuation failure Sub-task New Unassigned  
          11.
          G1: Log per region statistics of evacuation failure Sub-task New Unassigned  
          12.
          G1: Add an additional logging category such as "gc+evacfail" to switch on the most detailed statistics Sub-task New Unassigned  
          13.
          G1: Extend the gc+heap=debug messages to also show the number of failed regions of that category Sub-task New Unassigned  
          14.
          G1: Improve parallelism in regions that failed evacuation Sub-task In Progress Hamlin Li  
          15.
          G1: Move G1CardSetFreePool and related classes to separate files Sub-task Resolved Hamlin Li  
          16.
          G1: Rename G1CardSetFreePool and related classes Sub-task Resolved Hamlin Li  
          17.
          G1: Make reclaiming memory functionality more general in G1SegmentedArrayFreePool and G1SegmentedArrayFreeMemoryTask Sub-task New Hamlin Li  
          18.
          G1: Factor out G1CardSetFreePool and related classes from G1CardSetXxx Sub-task In Progress Hamlin Li  
          19.
          G1: Support reclaiming memory used in G1EvacFailureObjectsSet Sub-task Resolved Hamlin Li  
          20.
          G1: Consider putting regions where evacuation failed into next collection set Sub-task Open Hamlin Li  
          21.
          G1: support concurrent freeing of segments after GC in evacuation failure handling Sub-task Open Hamlin Li  
          22.
          G1: Improve generation placement heuristics for regions that could not be evacuated Sub-task Open Hamlin Li  
          23.
          G1: Improve thread sizing for evacuation failure Sub-task Open Hamlin Li  
          24.
          G1: Improve evacuation failure for regions with many objects Sub-task Open Hamlin Li  
          25.
          G1: Add objArray splitting when scanning object with evacuation failure Sub-task Open Hamlin Li  
          26.
          G1: Allow random selection of forced evacuation failure of N regions in collection set Sub-task Open Hamlin Li  
          27.
          G1: Distinguish logging between the real evac failure and region pinning Sub-task New Unassigned  
          28.
          G1: Enable region pinning in Young GC Sub-task New Unassigned  
          29.
          G1: Enable region pinning in Full GC Sub-task New Hamlin Li  
          30.
          G1: Remove GCLocker usage and related code in G1 Sub-task New Hamlin Li  

            Activity

              People

              Assignee:
              mli Hamlin Li
              Reporter:
              mli Hamlin Li
              Owner:
              Hamlin Li Hamlin Li
              Reviewed By:
              Thomas Schatzl, Vladimir Kozlov
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Dates

                Created:
                Updated: