-
Bug
-
Resolution: Unresolved
-
P3
-
21.0.5
-
x86_64
-
linux
ADDITIONAL SYSTEM INFORMATION :
Host: Intel(R) Xeon(R) Gold 6348 CPU @ 2.60GHz, 8 cores, 15G, Ubuntu 20.04.6 LTS
Time: Sun Dec 29 15:44:45 2024 UTC elapsed time: 307008.344300 seconds (3d 13h 16m 48s)
JRE version: Java(TM) SE Runtime Environment (21.0.5+9) (build 21.0.5+9-LTS-239)
Java VM: Java HotSpot(TM) 64-Bit Server VM (21.0.5+9-LTS-239, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
A DESCRIPTION OF THE PROBLEM :
The java process suddenly crashed and exited. According to the error file, it was caused by the actuator obtaining thread status information.
REGRESSION : Last worked in version 21.0.5
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Environment: linux, Ubuntu newer version, openJDK21.02-21.05
First write a springboot java application, which needs to expose the /actuator/prometheus interface (default)
Because the err log display is related to obtaining thread information, it is recommended to have a variable thread pool in the process (the process in question has a nacos client pulling thread)
Then run the java application normally and regularly pull and call the /actuator/prometheus interface.
The crash frequency will not be particularly high. The frequency of our pull is not particularly high, and it needs to continue running for a period of time. We have encountered several situations in the production environment where problems occurred in 2 days, 3 days, or more than ten days.
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The JVM process will suddenly crash and exit
ACTUAL -
The JVM process will suddenly crash and exit
---------- BEGIN SOURCE ----------
Stack: [0x00007f157d9fa000,0x00007f157dafb000], sp=0x00007f157daf83e0, free space=1016k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0xeb3b99] ThreadSnapshot::initialize(ThreadsList*, JavaThread*)+0x179
V [libjvm.so+0xeb3ed4] ThreadDumpResult::add_thread_snapshot(JavaThread*)+0x64
V [libjvm.so+0xc11c35] jmm_GetThreadInfo+0x4a5
J 35905 sun.management.ThreadImpl.getThreadInfo1([JI[Ljava/lang/management/ThreadInfo;)V java.management@21.0.5 (0 bytes) @ 0x00007f16c8efaf95 [0x00007f16c8efaea0+0x00000000000000f5]
J 52840 c2 io.micrometer.core.instrument.binder.jvm.JvmThreadMetrics$$Lambda+0x00007f167f47d800.applyAsDouble(Ljava/lang/Object;)D (12 bytes) @ 0x00007f16c94491bc [0x00007f16c9448980+0x000000000000083c]
J 42273 c2 io.micrometer.prometheus.PrometheusMeterRegistry$$Lambda+0x00007f167f44f788.samples(Ljava/lang/String;Ljava/util/List;)Ljava/util/stream/Stream; (14 bytes) @ 0x00007f16ca88af64 [0x00007f16ca88ae00+0x0000000000000164]
J 42590 c2 io.micrometer.prometheus.MicrometerCollector.collect()Ljava/util/List; (104 bytes) @ 0x00007f16ca8f0094 [0x00007f16ca8ef780+0x0000000000000914]
J 38520 c2 io.prometheus.client.exporter.common.TextFormat.write004(Ljava/io/Writer;Ljava/util/Enumeration;)V (568 bytes) @ 0x00007f16ca343c3c [0x00007f16ca343820+0x000000000000041c]
J 58984 c2 org.springframework.boot.actuate.metrics.export.prometheus.PrometheusScrapeEndpoint.scrape(Lorg/springframework/boot/actuate/metrics/export/prometheus/TextOutputFormat;Ljava/util/Set;)Lorg/springframework/boot/actuate/endpoint/web/WebEndpointResponse; (85 bytes) @ 0x00007f16cbd5f9c0 [0x00007f16cbd5ef60+0x0000000000000a60]
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
have not
FREQUENCY : occasionally
Host: Intel(R) Xeon(R) Gold 6348 CPU @ 2.60GHz, 8 cores, 15G, Ubuntu 20.04.6 LTS
Time: Sun Dec 29 15:44:45 2024 UTC elapsed time: 307008.344300 seconds (3d 13h 16m 48s)
JRE version: Java(TM) SE Runtime Environment (21.0.5+9) (build 21.0.5+9-LTS-239)
Java VM: Java HotSpot(TM) 64-Bit Server VM (21.0.5+9-LTS-239, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
A DESCRIPTION OF THE PROBLEM :
The java process suddenly crashed and exited. According to the error file, it was caused by the actuator obtaining thread status information.
REGRESSION : Last worked in version 21.0.5
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Environment: linux, Ubuntu newer version, openJDK21.02-21.05
First write a springboot java application, which needs to expose the /actuator/prometheus interface (default)
Because the err log display is related to obtaining thread information, it is recommended to have a variable thread pool in the process (the process in question has a nacos client pulling thread)
Then run the java application normally and regularly pull and call the /actuator/prometheus interface.
The crash frequency will not be particularly high. The frequency of our pull is not particularly high, and it needs to continue running for a period of time. We have encountered several situations in the production environment where problems occurred in 2 days, 3 days, or more than ten days.
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The JVM process will suddenly crash and exit
ACTUAL -
The JVM process will suddenly crash and exit
---------- BEGIN SOURCE ----------
Stack: [0x00007f157d9fa000,0x00007f157dafb000], sp=0x00007f157daf83e0, free space=1016k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0xeb3b99] ThreadSnapshot::initialize(ThreadsList*, JavaThread*)+0x179
V [libjvm.so+0xeb3ed4] ThreadDumpResult::add_thread_snapshot(JavaThread*)+0x64
V [libjvm.so+0xc11c35] jmm_GetThreadInfo+0x4a5
J 35905 sun.management.ThreadImpl.getThreadInfo1([JI[Ljava/lang/management/ThreadInfo;)V java.management@21.0.5 (0 bytes) @ 0x00007f16c8efaf95 [0x00007f16c8efaea0+0x00000000000000f5]
J 52840 c2 io.micrometer.core.instrument.binder.jvm.JvmThreadMetrics$$Lambda+0x00007f167f47d800.applyAsDouble(Ljava/lang/Object;)D (12 bytes) @ 0x00007f16c94491bc [0x00007f16c9448980+0x000000000000083c]
J 42273 c2 io.micrometer.prometheus.PrometheusMeterRegistry$$Lambda+0x00007f167f44f788.samples(Ljava/lang/String;Ljava/util/List;)Ljava/util/stream/Stream; (14 bytes) @ 0x00007f16ca88af64 [0x00007f16ca88ae00+0x0000000000000164]
J 42590 c2 io.micrometer.prometheus.MicrometerCollector.collect()Ljava/util/List; (104 bytes) @ 0x00007f16ca8f0094 [0x00007f16ca8ef780+0x0000000000000914]
J 38520 c2 io.prometheus.client.exporter.common.TextFormat.write004(Ljava/io/Writer;Ljava/util/Enumeration;)V (568 bytes) @ 0x00007f16ca343c3c [0x00007f16ca343820+0x000000000000041c]
J 58984 c2 org.springframework.boot.actuate.metrics.export.prometheus.PrometheusScrapeEndpoint.scrape(Lorg/springframework/boot/actuate/metrics/export/prometheus/TextOutputFormat;Ljava/util/Set;)Lorg/springframework/boot/actuate/endpoint/web/WebEndpointResponse; (85 bytes) @ 0x00007f16cbd5f9c0 [0x00007f16cbd5ef60+0x0000000000000a60]
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
have not
FREQUENCY : occasionally
- relates to
-
JDK-8323792 ThreadSnapshot::initialize can cause assert in Thread::check_for_dangling_thread_pointer (possibility of dangling Thread pointer)
-
- Open
-