Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8290225

Class loading errors may cause excessive native memory use

    XMLWordPrintable

Details

    • Enhancement
    • Resolution: Won't Fix
    • P4
    • tbd
    • 11, 17, 18, 19, 20
    • hotspot

    Description

      Class load errors may increase the native memory footprint of the VM over time. From the outside this looks like a leak (observable via NMT "mtClass" malloc size), even though it is not.

      When a class loading error occurs we may leave a newly created ClassLoaderData structure in the CLDG, essentially empty, with zero classes loaded. It may or may not have allocated metaspace depending on when during class loading the error occurred.

      It will in any case cost at least ~1.3...2.2 KB per class loader since we pay for the CLD itself as well as its daughter structures (e.g. the initially-sized dictionary). The exact size of the waste depends on whether NMT is enabled (malloc headers), the bitness of the VM, and whether it's a release or a debug build.

      It can lead to the native memory footprint monotonously rising over time in case a broken java program endlessly repeats the attempt to load a class. We saw this at a customer with Dynatrace. It is not a real leak since the CLDG will be purged eventually, and this memory released, on the next GC that does class unloading. But that may not happen if the loaders are referenced. Even if they are cleaned, the next purge may take a while because a raise in native memory is not directly observed by the GC, especially if no metaspace is allocated.

      This problem is aggravated by malloc allocator variants that are bad at memory reclamation, e.g. glibc's malloc. Even if class loading happens, RSS may very well stay up instead of recovering.

      ------

      Reproduction:

      See simple java program attached.

      See output of nmt.txt after 128k failed attempts to load a class:

      ```
      139505:

      Native Memory Tracking:

      (Omitting categories weighting less than 1KB)

      Total: reserved=18912396KB, committed=3559812KB
             malloc: 403504KB #1452847
             mmap: reserved=18508892KB, committed=3156308KB

      - Java Heap (reserved=16449536KB, committed=2990080KB)
                                  (mmap: reserved=16449536KB, committed=2990080KB)
       
      - Class (reserved=1225828KB, committed=177380KB)
                                  (classes #597)
                                  ( instance classes #496, array classes #101)
                              >>> (malloc=177252KB #525284) <<<
                                  (mmap: reserved=1048576KB, committed=128KB)
                                  ( Metadata: )
                                  ( reserved=65536KB, committed=384KB)
                                  ( used=335KB)
                                  ( waste=49KB =12,74%)
                                  ( Class space:)
                                  ( reserved=1048576KB, committed=128KB)
                                  ( used=7KB)
                                  ( waste=121KB =94,32%)

      ```

      Attachments

        1. ContinuousLoadErrors.java
          1 kB
        2. nmt.txt
          7 kB
        3. classloaders.txt
          16.13 MB

        Activity

          People

            stuefe Thomas Stuefe
            stuefe Thomas Stuefe
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: