-
Bug
-
Resolution: Unresolved
-
P4
-
21
-
generic
-
generic
A DESCRIPTION OF THE PROBLEM :
We had an Apache Solr instance start crashing with this same issue after upgrading to JDK 21. It appears that explicit monitor deflation leads to the in_use_list_ceiling being erroneously bumped, eventually leading to wrap around a divide by zero:
[2024-10-30T16:38:33.163+0000][info][monitorinflation] Async deflation needed: explicit request
[2024-10-30T16:38:33.163+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:38:33.163+0000][info][monitorinflation] before handshaking: unlinked_count=1, in_use_list stats: ceiling=17587870461276979200, count=18, max=1188
[2024-10-30T16:38:33.163+0000][info][monitorinflation] after handshaking: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:38:33.163+0000][info][monitorinflation] deflated_count=1, {unlinked,deleted}_count=1 monitors in 0.0000050 secs
[2024-10-30T16:38:33.163+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:38:33.406+0000][info][monitorinflation] Async deflation needed: explicit request
[2024-10-30T16:38:33.407+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:38:33.407+0000][info][monitorinflation] before handshaking: unlinked_count=1, in_use_list stats: ceiling=17587870461276979200, count=18, max=1188
[2024-10-30T16:38:33.407+0000][info][monitorinflation] after handshaking: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:38:33.407+0000][info][monitorinflation] deflated_count=1, {unlinked,deleted}_count=1 monitors in 0.0000051 secs
[2024-10-30T16:38:33.407+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:39:33.423+0000][info][monitorinflation] Async deflation needed: guaranteed interval (60000 ms) is greater than time since last deflation (60015 ms)
[2024-10-30T16:39:33.423+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:39:33.423+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:39:34.034+0000][info][monitorinflation] Async deflation needed: explicit request
[2024-10-30T16:39:34.034+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276980224, count=138, max=1188
[2024-10-30T16:39:34.034+0000][info][monitorinflation] before handshaking: unlinked_count=119, in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:39:34.034+0000][info][monitorinflation] after handshaking: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:39:34.034+0000][info][monitorinflation] deflated_count=119, {unlinked,deleted}_count=119 monitors in 0.0000427 secs
[2024-10-30T16:39:34.034+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:39:34.317+0000][info][monitorinflation] Async deflation needed: explicit request
[2024-10-30T16:39:34.317+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:39:34.317+0000][info][monitorinflation] before handshaking: unlinked_count=1, in_use_list stats: ceiling=17587870461276980224, count=18, max=1188
[2024-10-30T16:39:34.317+0000][info][monitorinflation] after handshaking: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:39:34.317+0000][info][monitorinflation] deflated_count=1, {unlinked,deleted}_count=1 monitors in 0.0000047 secs
[2024-10-30T16:39:34.317+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:40:33.242+0000][info][monitorinflation] Async deflation needed: explicit request
[2024-10-30T16:40:33.242+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:40:33.242+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:40:33.489+0000][info][monitorinflation] Async deflation needed: explicit request
[2024-10-30T16:40:33.490+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:40:33.490+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:41:33.169+0000][info][monitorinflation] Async deflation needed: explicit request
[2024-10-30T16:41:33.169+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:41:33.169+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:41:33.415+0000][info][monitorinflation] Async deflation needed: explicit request
[2024-10-30T16:41:33.415+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:41:33.415+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:41:33.915+0000][info][monitorinflation] Too many deflations without progress; bumping in_use_list_ceiling from 17587870461276979200 to 0
Thread dumps cause this due to the explicit deflation request here:
https://github.com/openjdk/jdk/blob/688e92e7f5febddd2935cb7f500dd3f10fbd9401/src/hotspot/share/runtime/vmOperations.cpp#L402-L410
We had an Apache Solr instance start crashing with this same issue after upgrading to JDK 21. It appears that explicit monitor deflation leads to the in_use_list_ceiling being erroneously bumped, eventually leading to wrap around a divide by zero:
[2024-10-30T16:38:33.163+0000][info][monitorinflation] Async deflation needed: explicit request
[2024-10-30T16:38:33.163+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:38:33.163+0000][info][monitorinflation] before handshaking: unlinked_count=1, in_use_list stats: ceiling=17587870461276979200, count=18, max=1188
[2024-10-30T16:38:33.163+0000][info][monitorinflation] after handshaking: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:38:33.163+0000][info][monitorinflation] deflated_count=1, {unlinked,deleted}_count=1 monitors in 0.0000050 secs
[2024-10-30T16:38:33.163+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:38:33.406+0000][info][monitorinflation] Async deflation needed: explicit request
[2024-10-30T16:38:33.407+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:38:33.407+0000][info][monitorinflation] before handshaking: unlinked_count=1, in_use_list stats: ceiling=17587870461276979200, count=18, max=1188
[2024-10-30T16:38:33.407+0000][info][monitorinflation] after handshaking: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:38:33.407+0000][info][monitorinflation] deflated_count=1, {unlinked,deleted}_count=1 monitors in 0.0000051 secs
[2024-10-30T16:38:33.407+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:39:33.423+0000][info][monitorinflation] Async deflation needed: guaranteed interval (60000 ms) is greater than time since last deflation (60015 ms)
[2024-10-30T16:39:33.423+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:39:33.423+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:39:34.034+0000][info][monitorinflation] Async deflation needed: explicit request
[2024-10-30T16:39:34.034+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276980224, count=138, max=1188
[2024-10-30T16:39:34.034+0000][info][monitorinflation] before handshaking: unlinked_count=119, in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:39:34.034+0000][info][monitorinflation] after handshaking: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:39:34.034+0000][info][monitorinflation] deflated_count=119, {unlinked,deleted}_count=119 monitors in 0.0000427 secs
[2024-10-30T16:39:34.034+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:39:34.317+0000][info][monitorinflation] Async deflation needed: explicit request
[2024-10-30T16:39:34.317+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:39:34.317+0000][info][monitorinflation] before handshaking: unlinked_count=1, in_use_list stats: ceiling=17587870461276980224, count=18, max=1188
[2024-10-30T16:39:34.317+0000][info][monitorinflation] after handshaking: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:39:34.317+0000][info][monitorinflation] deflated_count=1, {unlinked,deleted}_count=1 monitors in 0.0000047 secs
[2024-10-30T16:39:34.317+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:40:33.242+0000][info][monitorinflation] Async deflation needed: explicit request
[2024-10-30T16:40:33.242+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:40:33.242+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:40:33.489+0000][info][monitorinflation] Async deflation needed: explicit request
[2024-10-30T16:40:33.490+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:40:33.490+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276980224, count=19, max=1188
[2024-10-30T16:41:33.169+0000][info][monitorinflation] Async deflation needed: explicit request
[2024-10-30T16:41:33.169+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:41:33.169+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:41:33.415+0000][info][monitorinflation] Async deflation needed: explicit request
[2024-10-30T16:41:33.415+0000][info][monitorinflation] begin deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:41:33.415+0000][info][monitorinflation] end deflating: in_use_list stats: ceiling=17587870461276979200, count=19, max=1188
[2024-10-30T16:41:33.915+0000][info][monitorinflation] Too many deflations without progress; bumping in_use_list_ceiling from 17587870461276979200 to 0
Thread dumps cause this due to the explicit deflation request here:
https://github.com/openjdk/jdk/blob/688e92e7f5febddd2935cb7f500dd3f10fbd9401/src/hotspot/share/runtime/vmOperations.cpp#L402-L410
- relates to
-
JDK-8332506 SIGFPE In ObjectSynchronizer::is_async_deflation_needed() causing JVM Crash
- Open