-
Bug
-
Resolution: Won't Fix
-
P4
-
17
In cgroups environments the available CPU resources are described by the minimal guaranteed amount and maximal allowed amount (see eg. here - https://engineering.squarespace.com/blog/2017/understanding-linux-container-scheduling#:~:text=CPU%20shares%20provide%20tasks%20in,total%20number%20of%20shares%20available.).
The current active_processor_count computation makes the assumption that the minimal guaranteed amount of CPU resources translates to the number of available CPUs reported by the container. Unfortunately, this is not completely true and a container is free to use whatever CPUs are available leading to the actual CPU usage being higher than the reported number of available CPUs.
Just for the record, the algorithm is a bit more involved - it computes both values, the one based on the minimal guaranteed amount (if specified) as well as the one based on the maximal allowed amount (again, if specified) and then takes the lesser of the two. In reality, when both parts are set the minimal guaranteed amount will always be less or equal to the maximal allowed amount so, as a simplification, we can consider the minimal guaranteed amount to be the base for the available CPU count calculation if it is set.
For systems with 'elastic' setup where the minimal guaranteed amount and maximal allowed amount are not equal this definition of available CPUs can lead to underreporting of the available CPU resources and misconfiguration of anything relying on the reported number of cores - eg. number of GC threads, compiler thread or the fork-join pool size.
The available processor count calculation should disregard the minimal guaranteed amount in the calculation when PreferContainerQuotaForCPUCount JVM flag is set to true (currently default). This would allow fallback to the original calculation based on the minimal guaranteed amount by specifying -XX:-PreferContainerQuotaForCPUCount.
With this change in place the number of available processors will only be limited by the cpuset and the maximal allowed amount specified by cgroups.
The current active_processor_count computation makes the assumption that the minimal guaranteed amount of CPU resources translates to the number of available CPUs reported by the container. Unfortunately, this is not completely true and a container is free to use whatever CPUs are available leading to the actual CPU usage being higher than the reported number of available CPUs.
Just for the record, the algorithm is a bit more involved - it computes both values, the one based on the minimal guaranteed amount (if specified) as well as the one based on the maximal allowed amount (again, if specified) and then takes the lesser of the two. In reality, when both parts are set the minimal guaranteed amount will always be less or equal to the maximal allowed amount so, as a simplification, we can consider the minimal guaranteed amount to be the base for the available CPU count calculation if it is set.
For systems with 'elastic' setup where the minimal guaranteed amount and maximal allowed amount are not equal this definition of available CPUs can lead to underreporting of the available CPU resources and misconfiguration of anything relying on the reported number of cores - eg. number of GC threads, compiler thread or the fork-join pool size.
The available processor count calculation should disregard the minimal guaranteed amount in the calculation when PreferContainerQuotaForCPUCount JVM flag is set to true (currently default). This would allow fallback to the original calculation based on the minimal guaranteed amount by specifying -XX:-PreferContainerQuotaForCPUCount.
With this change in place the number of available processors will only be limited by the cpuset and the maximal allowed amount specified by cgroups.
- links to
-
Review
openjdk/jdk/3177