Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8296671

[JFR] jdk.ContainerConfiguration event should include host total memory

    XMLWordPrintable

Details

    • Enhancement
    • Resolution: Fixed
    • P3
    • 20
    • None
    • hotspot
    • None
    • jfr
    • b26
    • generic
    • linux

    Description

      The documentation of the jdk.PhysicalMemory event says:

      /**
       * PhysicalMemory event represents:
       *
       * @totalSize == The amount of physical memory (hw) installed and reported by the OS, in bytes.
       * @usedSize == The amount of physical memory currently in use in the system (reserved/committed), in bytes.
       *
       * Both fields are systemwide, i.e. represents the entire OS/HW environment.
       * These fields do not include virtual memory.
       *
       * If running inside a guest OS on top of a hypervisor in a virtualized environment,
       * the total memory reported is the amount of memory configured for the guest OS by the hypervisor.
       */

      Yet, when run in a container, it reports the container memory rather than the host value. Given that jdk.ContainerConfiguration reports the container values, it would make sense to report the memory of the physical host.

      Reproducer:
      $ git clone https://github.com/Karm/mandrel-integration-tests.git
      $ cd mandrel-integration-tests/apps/debug-symbols-smoke
      $ mvn package
      $ sudo podman run --memory=1G --memory-swap=1G --rm -ti -v /disk/openjdk/upstream-sources/git/jdk-jdk/build/linux-x86_64-server-fastdebug/images/jdk:/opt/jdk:z -v $(pwd)/target:/opt/target:z fedora:37
      [root@4b003616eab8 /]# /opt/jdk/bin/java -XX:StartFlightRecording=filename=flight-java.jfr -Xlog:jfr -jar /opt/target/debug-symbols-smoke.jar
      [1.030s][info][jfr] Flight Recorder initialized
      [1.030s][info][jfr] Repository base directory: /tmp
      [1.259s][info][jfr] Started recording "1" (1) {dumponexit=true, filename=/flight-java.jfr}
      [1.260s][info][jfr,startup] Started recording 1. No limit specified, using maxsize=250MB as default.
      [1.260s][info][jfr,startup]
      [1.260s][info][jfr,startup] Use jcmd 26 JFR.dump name=1 to copy recording data to file.
      Q to quit
      Q
      e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
      [3.585s][info][jfr ] Stopped recording "1" (1). Reason "Dump on exit".
      [3.671s][info][jfr ] Transferred 222333 bytes from the disk repository
      [3.671s][info][jfr ] Wrote recording "1" (1) to /flight-java.jfr
      [3.672s][info][jfr ] Closed recording "1" (1)
      [3.681s][info][jfr ] Removed repository /tmp/2022_11_09_13_53_17_26
      [root@4b003616eab8 /]# /opt/jdk/bin/jfr print --events jdk.ContainerConfiguration flight-java.jfr
      jdk.ContainerConfiguration {
        startTime = 13:53:17.658 (2022-11-09)
        containerType = "cgroupv1"
        cpuSlicePeriod = 100 ms
        cpuQuota = -0.00100 ms
        cpuShares = -1
        effectiveCpuCount = 12
        memorySoftLimit = -1 byte
        memoryLimit = 1.0 GB
        swapMemoryLimit = 1.0 GB
        eventThread = "main" (javaThreadId = 1)
      }

      [root@4b003616eab8 /]# /opt/jdk/bin/jfr print --events jdk.PhysicalMemory flight-java.jfr
      jdk.PhysicalMemory {
        startTime = 13:53:17.648 (2022-11-09)
        totalSize = 1.0 GB
        usedSize = 130.3 MB
      }

      jdk.PhysicalMemory {
        startTime = 13:53:19.991 (2022-11-09)
        totalSize = 1.0 GB
        usedSize = 171.8 MB
      }

      Unfortunately, with this behaviour it's not easy to know the (container) hosts' total memory which is useful for some monitoring use-cases. Note that hotspot container trace logging (`os+container`) shows this info after JDK-8292083:

      [root@4b003616eab8 /]# /opt/jdk/bin/java -Xlog:os+container=trace -version
      [0.007s][trace][os,container] OSContainer::init: Initializing Container Support
      [0.007s][debug][os,container] Detected optional pids controller entry in /proc/cgroups
      [0.007s][debug][os,container] Detected cgroups hybrid or legacy hierarchy, using cgroups v1 controllers
      [0.007s][trace][os,container] Path to /memory.use_hierarchy is /sys/fs/cgroup/memory/memory.use_hierarchy
      [0.008s][trace][os,container] Use Hierarchy is: 1
      [0.010s][trace][os,container] Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
      [0.010s][trace][os,container] CPU Quota is: -1
      [0.010s][trace][os,container] Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
      [0.010s][trace][os,container] CPU Period is: 100000
      [0.010s][trace][os,container] OSContainer::active_processor_count: 12
      [0.010s][trace][os,container] CgroupSubsystem::active_processor_count (cached): 12
      [0.010s][trace][os,container] total physical memory: 67174506496
      [0.010s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes
      [0.010s][trace][os,container] Memory Limit is: 1073741824
      [0.028s][trace][os,container] CgroupSubsystem::active_processor_count (cached): 12
      [0.050s][trace][os,container] total physical memory: 67174506496
      [0.050s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes
      [0.050s][trace][os,container] Memory Limit is: 1073741824
      [0.155s][trace][os,container] Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
      [0.155s][trace][os,container] CPU Quota is: -1
      [0.155s][trace][os,container] Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
      [0.155s][trace][os,container] CPU Period is: 100000
      [0.155s][trace][os,container] OSContainer::active_processor_count: 12
      openjdk version "20-internal" 2023-03-21
      OpenJDK Runtime Environment (fastdebug build 20-internal-adhoc.sgehwolf.jdk-jdk)
      OpenJDK 64-Bit Server VM (fastdebug build 20-internal-adhoc.sgehwolf.jdk-jdk, mixed mode, sharing)

      Edit:
      After some discussion https://mail.openjdk.org/pipermail/hotspot-jfr-dev/2022-November/004360.html it seems this is expected behaviour. But since there is no way to know the host total memory via JFR the proposal is to amend the jdk.ContainerConfiguration event with host total memory field.

      Attachments

        Issue Links

          Activity

            People

              sgehwolf Severin Gehwolf
              sgehwolf Severin Gehwolf
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: