Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8307977

jcmd and jstack broken for target processes running with elevated capabilities

XMLWordPrintable

    • x86_64
    • linux

      ADDITIONAL SYSTEM INFORMATION :
      Ubuntu 20.04 and 22.04
      OpenJDK 17.0.6+10-Ubuntu-0ubuntu120.04.1 and Temurin 17.0.6+10

      A DESCRIPTION OF THE PROBLEM :
      Tools like jcmd and jstack do not work if the target process uses elevated capabilities like CAP_NET_BIND_SERVICE (using systemd's AmbientCapabilities or setcap cap_net_bind_service+ep /path/to/java) and when jcmd/jstack is run as the same user as the target process.

      This worked fine in Java 8, but since that, the dynamic attach mechanism has been changed from using /tmp/.attach_pidXXXX to /proc/XXXX/root/tmp/.attach_pidXXXX (as seen in current mainline https://github.com/openjdk/jdk/blob/c0b4957fcce530290fe3b1e730b593b6458285aa/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java#L212-L218).

      https://github.com/openjdk/jdk/blob/c0b4957fcce530290fe3b1e730b593b6458285aa/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java#L212-L218

      Using /proc/XXXX/root/tmp/.attach_pidXXXX is a sensible default (see https://bugs.openjdk.org/browse/JDK-8179498 for rationale), I think falling back to /tmp/.attach_pidXXXX outside container environments would make sense.

      REGRESSION : Last worked in version 8u361

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      xx@desk:~/reproducer$ sudo apt-get install openjdk-17-jdk-headless
      xx@desk:~/reproducer$ cat Reproducer.java
      import java.io.IOException;
      import java.net.InetSocketAddress;
      import java.net.ServerSocket;

      public class Reproducer {
        public static void main(String[] args) throws InterruptedException, IOException {
          System.out.println("Hello, World!");
          try (var server = new ServerSocket()) {
            server.bind(new InetSocketAddress("localhost", 81));
            System.out.println("Bound to port 81");
            while (true) {
              Thread.sleep(1_000L);
            }
          }
        }
      }

      xx@desk:~/reproducer$ cat reproducer.service
      [Service]
      Type=simple
      ExecStart=/usr/lib/jvm/java-17-openjdk-amd64/bin/java /home/xx/reproducer/Reproducer.java

      User=xx
      Group=xx
      ReadWritePaths=/tmp
      AmbientCapabilities=CAP_NET_BIND_SERVICE
      xx@desk:~/reproducer$ sudo cp -a reproducer.service /etc/systemd/system/
      xx@desk:~/reproducer$ sudo systemctl daemon-reload
      xx@desk:~/reproducer$ sudo systemctl start reproducer.service
      xx@desk:~/reproducer$ ls -lh /proc/$(pgrep -f Reproducer.java)/root
      ls: cannot read symbolic link '/proc/2322887/root': Permission denied
      lrwxrwxrwx 1 xx xx 0 apr 17 14:43 /proc/2322887/root
      xx@desk:~/reproducer$ jcmd $(pgrep -f Reproducer.java) VM.version

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      Change the port in the Java source file from 81 to 8081.
      Remove AmbientCapabilities=CAP_NET_BIND_SERVICE from /etc/systemd/system/reproducer.service.

      xx@desk:~/reproducer$ sudo systemctl daemon-reload
      xx@desk:~/reproducer$ sudo systemctl restart reproducer.service
      xx@desk:~/reproducer$ jcmd $(pgrep -f Reproducer.java) VM.version
      2327242:
      OpenJDK 64-Bit Server VM version 17.0.6+10-Ubuntu-0ubuntu122.04
      JDK 17.0.6
      ACTUAL -
      xx@desk:~/reproducer$ jcmd $(pgrep -f Reproducer.java) VM.version
      2322887:
      com.sun.tools.attach.AttachNotSupportedException: Unable to open socket file /proc/2322887/root/tmp/.java_pid2322887: target process 2322887 doesn't respond within 10500ms or HotSpot VM not loaded
      at jdk.attach/sun.tools.attach.VirtualMachineImpl.<init>(VirtualMachineImpl.java:104)
      at jdk.attach/sun.tools.attach.AttachProviderImpl.attachVirtualMachine(AttachProviderImpl.java:58)
      at jdk.attach/com.sun.tools.attach.VirtualMachine.attach(VirtualMachine.java:207)
      at jdk.jcmd/sun.tools.jcmd.JCmd.executeCommandForPid(JCmd.java:113)
      at jdk.jcmd/sun.tools.jcmd.JCmd.main(JCmd.java:97)

      ---------- BEGIN SOURCE ----------
      See above.
      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      Not acceptable in all environments, but running jcmd/jstack as root works.

      Change the port in the Java source file back to 81.
      Enable AmbientCapabilities=CAP_NET_BIND_SERVICE in /etc/systemd/system/reproducer.service.

      xx@desk:~/reproducer$ sudo systemctl daemon-reload
      xx@desk:~/reproducer$ sudo systemctl restart reproducer.service
      xx@desk:~/reproducer$ sudo jcmd $(pgrep -f Reproducer.java) VM.version
      2328529:
      OpenJDK 64-Bit Server VM version 17.0.6+10-Ubuntu-0ubuntu122.04
      JDK 17.0.6

      FREQUENCY : always


            lcable Larry Cable
            webbuggrp Webbug Group
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: