Description
The following test failed in my stress testing for jdk-20+33:
jdk/jfr/api/consumer/streaming/TestCrossProcessStreaming.java
Here's a snippet from the log file:
[Event-Producer] [15.752s][info][jfr,startup] Started recording 1. No limit specified, using maxsize=250MB as default.
[Event-Producer] [15.752s][info][jfr,startup]
[Event-Producer] [15.752s][info][jfr,startup] Use jcmd 15245 JFR.dump name=1 filename=FILEPATH to copy recording data to file.
[Event-Producer] MAIN_STARTED
JFR repository: /private/var/folders/5g/66j5b5kn7fl10mkt696_lzhh00grqb/T/2023_01_29_05_41_30_15245
[Event-Producer] Events generated: 25000
[Event-Producer] Events generated: 8000
[Event-Producer] # To suppress the following error report, specify this argument
[Event-Producer] # after -XX: or in .hotspotrc: SuppressErrorAt=/jfrThreadLocal.cpp:331
[Event-Producer] #
[Event-Producer] # A fatal error has been detected by the Java Runtime Environment:
[Event-Producer] #
[Event-Producer] # Internal Error (/System/Volumes/Data/work/shared/bug_hunt/thread_SMR_stress/jdk20_exp.git/open/src/hotspot/share/jfr/support/jfrThreadLocal.cpp:331), pid=15245, tid=40451
[Event-Producer] # assert(other_thread_id != 0) failed: invariant
[Event-Producer] #
[Event-Producer] # JRE version: Java(TM) SE Runtime Environment (20.0) (slowdebug build 20-internal-2023-01-26-1529122.dcubed...)
[Event-Producer] # Java VM: Java HotSpot(TM) 64-Bit Server VM (slowdebug 20-internal-2023-01-26-1529122.dcubed..., mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-aarch64)
[Event-Producer] # Core dump will be written. Default location: ../core.15245
[Event-Producer] #
[Event-Producer] # JFR recording file will be written. Location: /System/Volumes/Data/work/shared/bug_hunt/thread_SMR_stress/jdk20_exp.git/build/macosx-aarch64-normal-server-slowdebug/test-support/jtreg_open_test_jdk_jdk_svc/scratch/2/hs_err_pid15245.jfr
[Event-Producer] #
[Event-Producer] # An error report file with more information is saved as:
[Event-Producer] # /System/Volumes/Data/work/shared/bug_hunt/thread_SMR_stress/jdk20_exp.git/build/macosx-aarch64-normal-server-slowdebug/test-support/jtreg_open_test_jdk_jdk_svc/scratch/2/hs_err_pid15245.log
[Event-Producer] [thread 26883 also had an error]
[Event-Producer] #
[Event-Producer] # If you would like to submit a bug report, please visit:
[Event-Producer] # https://bugreport.java.com/bugreport/crash.jsp
[Event-Producer] #
----------System.err:(12/707)----------
Here's the crashing thread's stack:
--------------- T H R E A D ---------------
Current thread (0x000000014a09f210): JavaThread "JFR Periodic Tasks" daemon [_thread_in_vm, id=40451, stack(0x000000016dc6c000,0x000000016de6f000)] _threads_hazard_ptr=0x0000000159e7fad0, _nested_threads_hazard_ptr_cnt=0
Stack: [0x000000016dc6c000,0x000000016de6f000], sp=0x000000016de6df20, free space=2055k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.dylib+0x107c00c] VMError::report_and_die(int, char const*, char const*, char*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned long)+0x990 (jfrThreadLocal.cpp:331)
V [libjvm.dylib+0x107c65c] VMError::report_and_die(Thread*, void*, char const*, int, char const*, char const*, char*)+0x68
V [libjvm.dylib+0x58b1b8] report_vm_error(char const*, int, char const*, char const*, ...)+0x88
V [libjvm.dylib+0x97ab14] JfrThreadLocal::impersonate(Thread const*, unsigned long long)+0x68
V [libjvm.dylib+0x975280] JfrThreadCPULoadEvent::send_events()+0x14c
V [libjvm.dylib+0x941d0c] JfrPeriodicEventSet::requestThreadCPULoad()+0xc
V [libjvm.dylib+0x92fc20] JfrPeriodicEventSet::requestEvent(JfrEventId, long, PeriodicType)+0xd4
V [libjvm.dylib+0x92faec] jfr_emit_event+0xa4
j jdk.jfr.internal.JVM.emitEvent(JJJ)Z+0 jdk.jfr@20-internal
j jdk.jfr.internal.RequestEngine$RequestHook.emitJVMEvent(Ljdk/jfr/internal/PlatformEventType;JLjdk/jfr/internal/RequestEngine$PeriodicType;)V+20 jdk.jfr@20-internal
j jdk.jfr.internal.RequestEngine$RequestHook.execute(JLjdk/jfr/internal/RequestEngine$PeriodicType;)V+36 jdk.jfr@20-internal
j jdk.jfr.internal.RequestEngine.run_requests(Ljava/util/Collection;J)J+141 jdk.jfr@20-internal
j jdk.jfr.internal.RequestEngine.doPeriodic()J+6 jdk.jfr@20-internal
j jdk.jfr.internal.PlatformRecorder.periodicTask()V+47 jdk.jfr@20-internal
j jdk.jfr.internal.PlatformRecorder.lambda$startDiskMonitor$1()V+1 jdk.jfr@20-internal
j jdk.jfr.internal.PlatformRecorder$$Lambda$73+0x00000008010424d8.run()V+4 jdk.jfr@20-internal
j java.lang.Thread.runWith(Ljava/lang/Object;Ljava/lang/Runnable;)V+5 java.base@20-internal
j java.lang.Thread.run()V+19 java.base@20-internal
v ~StubRoutines::call_stub 0x000000011b1b817c
V [libjvm.dylib+0x8d63f4] JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*)+0x424
V [libjvm.dylib+0xd96f3c] os::os_exception_wrapper(void (*)(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*), JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*)+0x38
V [libjvm.dylib+0x8d52c8] JavaCalls::call(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*)+0x38
V [libjvm.dylib+0x8d5108] JavaCalls::call_virtual(JavaValue*, Klass*, Symbol*, Symbol*, JavaCallArguments*, JavaThread*)+0x148
V [libjvm.dylib+0x8d5344] JavaCalls::call_virtual(JavaValue*, Handle, Klass*, Symbol*, Symbol*, JavaThread*)+0x70
V [libjvm.dylib+0x9f4af0] thread_entry(JavaThread*, JavaThread*)+0xa8
V [libjvm.dylib+0x8f0fbc] JavaThread::thread_main_inner()+0x108
V [libjvm.dylib+0x8f0ea8] JavaThread::run()+0x124
V [libjvm.dylib+0xfd6398] Thread::call_run()+0x154
V [libjvm.dylib+0xd94198] thread_native_entry(Thread*)+0x138
C [libsystem_pthread.dylib+0x7878] _pthread_start+0x140
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j jdk.jfr.internal.JVM.emitEvent(JJJ)Z+0 jdk.jfr@20-internal
j jdk.jfr.internal.RequestEngine$RequestHook.emitJVMEvent(Ljdk/jfr/internal/PlatformEventType;JLjdk/jfr/internal/RequestEngine$PeriodicType;)V+20 jdk.jfr@20-internal
j jdk.jfr.internal.RequestEngine$RequestHook.execute(JLjdk/jfr/internal/RequestEngine$PeriodicType;)V+36 jdk.jfr@20-internal
j jdk.jfr.internal.RequestEngine.run_requests(Ljava/util/Collection;J)J+141 jdk.jfr@20-internal
j jdk.jfr.internal.RequestEngine.doPeriodic()J+6 jdk.jfr@20-internal
j jdk.jfr.internal.PlatformRecorder.periodicTask()V+47 jdk.jfr@20-internal
j jdk.jfr.internal.PlatformRecorder.lambda$startDiskMonitor$1()V+1 jdk.jfr@20-internal
j jdk.jfr.internal.PlatformRecorder$$Lambda$73+0x00000008010424d8.run()V+4 jdk.jfr@20-internal
j java.lang.Thread.runWith(Ljava/lang/Object;Ljava/lang/Runnable;)V+5 java.base@20-internal
j java.lang.Thread.run()V+19 java.base@20-internal
v ~StubRoutines::call_stub 0x000000011b1b817c
jdk/jfr/api/consumer/streaming/TestCrossProcessStreaming.java
Here's a snippet from the log file:
[Event-Producer] [15.752s][info][jfr,startup] Started recording 1. No limit specified, using maxsize=250MB as default.
[Event-Producer] [15.752s][info][jfr,startup]
[Event-Producer] [15.752s][info][jfr,startup] Use jcmd 15245 JFR.dump name=1 filename=FILEPATH to copy recording data to file.
[Event-Producer] MAIN_STARTED
JFR repository: /private/var/folders/5g/66j5b5kn7fl10mkt696_lzhh00grqb/T/2023_01_29_05_41_30_15245
[Event-Producer] Events generated: 25000
[Event-Producer] Events generated: 8000
[Event-Producer] # To suppress the following error report, specify this argument
[Event-Producer] # after -XX: or in .hotspotrc: SuppressErrorAt=/jfrThreadLocal.cpp:331
[Event-Producer] #
[Event-Producer] # A fatal error has been detected by the Java Runtime Environment:
[Event-Producer] #
[Event-Producer] # Internal Error (/System/Volumes/Data/work/shared/bug_hunt/thread_SMR_stress/jdk20_exp.git/open/src/hotspot/share/jfr/support/jfrThreadLocal.cpp:331), pid=15245, tid=40451
[Event-Producer] # assert(other_thread_id != 0) failed: invariant
[Event-Producer] #
[Event-Producer] # JRE version: Java(TM) SE Runtime Environment (20.0) (slowdebug build 20-internal-2023-01-26-1529122.dcubed...)
[Event-Producer] # Java VM: Java HotSpot(TM) 64-Bit Server VM (slowdebug 20-internal-2023-01-26-1529122.dcubed..., mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-aarch64)
[Event-Producer] # Core dump will be written. Default location: ../core.15245
[Event-Producer] #
[Event-Producer] # JFR recording file will be written. Location: /System/Volumes/Data/work/shared/bug_hunt/thread_SMR_stress/jdk20_exp.git/build/macosx-aarch64-normal-server-slowdebug/test-support/jtreg_open_test_jdk_jdk_svc/scratch/2/hs_err_pid15245.jfr
[Event-Producer] #
[Event-Producer] # An error report file with more information is saved as:
[Event-Producer] # /System/Volumes/Data/work/shared/bug_hunt/thread_SMR_stress/jdk20_exp.git/build/macosx-aarch64-normal-server-slowdebug/test-support/jtreg_open_test_jdk_jdk_svc/scratch/2/hs_err_pid15245.log
[Event-Producer] [thread 26883 also had an error]
[Event-Producer] #
[Event-Producer] # If you would like to submit a bug report, please visit:
[Event-Producer] # https://bugreport.java.com/bugreport/crash.jsp
[Event-Producer] #
----------System.err:(12/707)----------
Here's the crashing thread's stack:
--------------- T H R E A D ---------------
Current thread (0x000000014a09f210): JavaThread "JFR Periodic Tasks" daemon [_thread_in_vm, id=40451, stack(0x000000016dc6c000,0x000000016de6f000)] _threads_hazard_ptr=0x0000000159e7fad0, _nested_threads_hazard_ptr_cnt=0
Stack: [0x000000016dc6c000,0x000000016de6f000], sp=0x000000016de6df20, free space=2055k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.dylib+0x107c00c] VMError::report_and_die(int, char const*, char const*, char*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned long)+0x990 (jfrThreadLocal.cpp:331)
V [libjvm.dylib+0x107c65c] VMError::report_and_die(Thread*, void*, char const*, int, char const*, char const*, char*)+0x68
V [libjvm.dylib+0x58b1b8] report_vm_error(char const*, int, char const*, char const*, ...)+0x88
V [libjvm.dylib+0x97ab14] JfrThreadLocal::impersonate(Thread const*, unsigned long long)+0x68
V [libjvm.dylib+0x975280] JfrThreadCPULoadEvent::send_events()+0x14c
V [libjvm.dylib+0x941d0c] JfrPeriodicEventSet::requestThreadCPULoad()+0xc
V [libjvm.dylib+0x92fc20] JfrPeriodicEventSet::requestEvent(JfrEventId, long, PeriodicType)+0xd4
V [libjvm.dylib+0x92faec] jfr_emit_event+0xa4
j jdk.jfr.internal.JVM.emitEvent(JJJ)Z+0 jdk.jfr@20-internal
j jdk.jfr.internal.RequestEngine$RequestHook.emitJVMEvent(Ljdk/jfr/internal/PlatformEventType;JLjdk/jfr/internal/RequestEngine$PeriodicType;)V+20 jdk.jfr@20-internal
j jdk.jfr.internal.RequestEngine$RequestHook.execute(JLjdk/jfr/internal/RequestEngine$PeriodicType;)V+36 jdk.jfr@20-internal
j jdk.jfr.internal.RequestEngine.run_requests(Ljava/util/Collection;J)J+141 jdk.jfr@20-internal
j jdk.jfr.internal.RequestEngine.doPeriodic()J+6 jdk.jfr@20-internal
j jdk.jfr.internal.PlatformRecorder.periodicTask()V+47 jdk.jfr@20-internal
j jdk.jfr.internal.PlatformRecorder.lambda$startDiskMonitor$1()V+1 jdk.jfr@20-internal
j jdk.jfr.internal.PlatformRecorder$$Lambda$73+0x00000008010424d8.run()V+4 jdk.jfr@20-internal
j java.lang.Thread.runWith(Ljava/lang/Object;Ljava/lang/Runnable;)V+5 java.base@20-internal
j java.lang.Thread.run()V+19 java.base@20-internal
v ~StubRoutines::call_stub 0x000000011b1b817c
V [libjvm.dylib+0x8d63f4] JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*)+0x424
V [libjvm.dylib+0xd96f3c] os::os_exception_wrapper(void (*)(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*), JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*)+0x38
V [libjvm.dylib+0x8d52c8] JavaCalls::call(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*)+0x38
V [libjvm.dylib+0x8d5108] JavaCalls::call_virtual(JavaValue*, Klass*, Symbol*, Symbol*, JavaCallArguments*, JavaThread*)+0x148
V [libjvm.dylib+0x8d5344] JavaCalls::call_virtual(JavaValue*, Handle, Klass*, Symbol*, Symbol*, JavaThread*)+0x70
V [libjvm.dylib+0x9f4af0] thread_entry(JavaThread*, JavaThread*)+0xa8
V [libjvm.dylib+0x8f0fbc] JavaThread::thread_main_inner()+0x108
V [libjvm.dylib+0x8f0ea8] JavaThread::run()+0x124
V [libjvm.dylib+0xfd6398] Thread::call_run()+0x154
V [libjvm.dylib+0xd94198] thread_native_entry(Thread*)+0x138
C [libsystem_pthread.dylib+0x7878] _pthread_start+0x140
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j jdk.jfr.internal.JVM.emitEvent(JJJ)Z+0 jdk.jfr@20-internal
j jdk.jfr.internal.RequestEngine$RequestHook.emitJVMEvent(Ljdk/jfr/internal/PlatformEventType;JLjdk/jfr/internal/RequestEngine$PeriodicType;)V+20 jdk.jfr@20-internal
j jdk.jfr.internal.RequestEngine$RequestHook.execute(JLjdk/jfr/internal/RequestEngine$PeriodicType;)V+36 jdk.jfr@20-internal
j jdk.jfr.internal.RequestEngine.run_requests(Ljava/util/Collection;J)J+141 jdk.jfr@20-internal
j jdk.jfr.internal.RequestEngine.doPeriodic()J+6 jdk.jfr@20-internal
j jdk.jfr.internal.PlatformRecorder.periodicTask()V+47 jdk.jfr@20-internal
j jdk.jfr.internal.PlatformRecorder.lambda$startDiskMonitor$1()V+1 jdk.jfr@20-internal
j jdk.jfr.internal.PlatformRecorder$$Lambda$73+0x00000008010424d8.run()V+4 jdk.jfr@20-internal
j java.lang.Thread.runWith(Ljava/lang/Object;Ljava/lang/Runnable;)V+5 java.base@20-internal
j java.lang.Thread.run()V+19 java.base@20-internal
v ~StubRoutines::call_stub 0x000000011b1b817c