-
Sub-task
-
Resolution: Delivered
-
P4
-
21
-
linux
Since JDK 13, executing commands in a sub-process uses the so-called `POSIX_SPAWN` launching mechanism (that is, `-Djdk.lang.Process.launchMechanism=POSIX_SPAWN`) by default on Linux. In cases where the parent JVM process terminates abnormally before the handshake between the JVM and the newly created `jspawnhelper` process has completed, `jspawnhelper` can hang indefinitely in JDK 13 to JDK 20. This issue is fixed in JDK 21. The issue was especially harmful if the parent process had open sockets, because in that case, the forked `jspawnhelper` process will inherit them and keep all the corresponding ports open, effectively preventing other processes from binding to them.
This misbehavior has been observed with applications which frequently fork child processes in environments with tight memory constraints. In such cases, the OS can kill the JVM in the middle of the forking process leading to the described issue. Restarting the JVM process after such a crash will be impossible if the new process tries to bind to the same ports as the initial application because they will be blocked by the hanging `jspawnhelper` child process.
The root cause of this issue is `jspawnhelper`'s omission to close its writing end of the pipe, which is used for the handshake with the parent JVM. It was fixed by closing the writing end of the communication pipe before attempting to read data from the parent process. This way, `jspawnhelper` will reliably read an EOF event from the communication pipe and terminate once the parent process dies prematurely.
A second variant of this issue could happen because the handshaking code in the JDK didn't handle interrupts to `write(2)` correctly. This could lead to incomplete messages being sent to the `jspawnhelper` child process. The result is a deadlock between the parent thread and the child process which manifests itself in a `jspawnhelper` process being blocked while reading from a pipe and the following stack trace in the corresponding parent Java process:
```
java.lang.Thread.State: RUNNABLE
at java.lang.ProcessImpl.forkAndExec(java.base@17.0.7/Native Method)
at java.lang.ProcessImpl.<init>(java.base@17.0.7/ProcessImpl.java:314)
at java.lang.ProcessImpl.start(java.base@17.0.7/ProcessImpl.java:244)
at java.lang.ProcessBuilder.start(java.base@17.0.7/ProcessBuilder.java:1110)
at java.lang.ProcessBuilder.start(java.base@17.0.7/ProcessBuilder.java:1073)
```
This misbehavior has been observed with applications which frequently fork child processes in environments with tight memory constraints. In such cases, the OS can kill the JVM in the middle of the forking process leading to the described issue. Restarting the JVM process after such a crash will be impossible if the new process tries to bind to the same ports as the initial application because they will be blocked by the hanging `jspawnhelper` child process.
The root cause of this issue is `jspawnhelper`'s omission to close its writing end of the pipe, which is used for the handshake with the parent JVM. It was fixed by closing the writing end of the communication pipe before attempting to read data from the parent process. This way, `jspawnhelper` will reliably read an EOF event from the communication pipe and terminate once the parent process dies prematurely.
A second variant of this issue could happen because the handshaking code in the JDK didn't handle interrupts to `write(2)` correctly. This could lead to incomplete messages being sent to the `jspawnhelper` child process. The result is a deadlock between the parent thread and the child process which manifests itself in a `jspawnhelper` process being blocked while reading from a pipe and the following stack trace in the corresponding parent Java process:
```
java.lang.Thread.State: RUNNABLE
at java.lang.ProcessImpl.forkAndExec(java.base@17.0.7/Native Method)
at java.lang.ProcessImpl.<init>(java.base@17.0.7/ProcessImpl.java:314)
at java.lang.ProcessImpl.start(java.base@17.0.7/ProcessImpl.java:244)
at java.lang.ProcessBuilder.start(java.base@17.0.7/ProcessBuilder.java:1110)
at java.lang.ProcessBuilder.start(java.base@17.0.7/ProcessBuilder.java:1073)
```