-
Bug
-
Resolution: Fixed
-
P4
-
17, 21, 25, 26
-
b07
I am working on JDK-8340297, which will add the ability to spot use-after-free problems with Metaspace objects. Note, it is not pushed yet.
With that patch activated, I get asserts in some Shenandoah tests:
- gc/shenandoah/mxbeans/TestPauseNotifications#aggressive
- serviceability/dcmd/gc/HeapDumpCompressedTest#Shenandoah
- gc/shenandoah/mxbeans/TestChurnNotifications#aggressive
- compiler/gcbarriers/UnsafeIntrinsicsTest#Shenandoah
```
Stack: [0x00007f8888a8a000,0x00007f8888b8a000], sp=0x00007f8888b85f30, free space=1007k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x14d9f88] NativeStackPrinter::print_stack(outputStream*, char*, int, unsigned char*&, bool, int)+0x74 (shenandoahHeap.cpp:1166)
V [libjvm.so+0x19f350c] VMError::report(outputStream*, bool)+0x1ba4 (vmError.cpp:960)
V [libjvm.so+0x19f6ea5] VMError::report_and_die(int, char const*, char const*, __va_list_tag*, Thread*, unsigned char*, void const*, void const*, char const*, int, unsigned long)+0x8b5 (vmError.cpp:1846)
V [libjvm.so+0xbb5d3c] report_vm_status_error(char const*, int, char const*, int, char const*)+0x0 (debug.cpp:195)
V [libjvm.so+0xbb5aa5] print_error_for_unit_test(char const*, char const*, __va_list_tag*)+0x0 (debug.cpp:148)
V [libjvm.so+0x16b226a] ShenandoahAsserts::assert_in_heap_bounds(void*, oopDesc*, char const*, int)+0x0 (shenandoahAsserts.cpp:173)
V [libjvm.so+0x16b2830] ShenandoahAsserts::assert_correct(void*, oopDesc*, char const*, int)+0x4c6 (shenandoahAsserts.cpp:289)
V [libjvm.so+0x16b2c30] ShenandoahAsserts::assert_marked(void*, oopDesc*, char const*, int)+0x32 (shenandoahAsserts.cpp:365)
V [libjvm.so+0x1736693] ShenandoahConcurrentEvacuateRegionObjectClosure::do_object(oopDesc*)+0x31 (shenandoahHeap.cpp:1166)
V [libjvm.so+0x17390a1] void ShenandoahHeap::marked_object_iterate<ShenandoahConcurrentEvacuateRegionObjectClosure>(ShenandoahHeapRegion*, ShenandoahConcurrentEvacuateRegionObjectClosure*, HeapWordImpl**)+0x4dd (shenandoahHeap.inline.hpp:565)
V [libjvm.so+0x1737df9] void ShenandoahHeap::marked_object_iterate<ShenandoahConcurrentEvacuateRegionObjectClosure>(ShenandoahHeapRegion*, ShenandoahConcurrentEvacuateRegionObjectClosure*)+0x3b (shenandoahHeap.inline.hpp:508)
V [libjvm.so+0x17368da] ShenandoahEvacuationTask::do_work()+0xd6 (shenandoahHeap.cpp:1207)
V [libjvm.so+0x1736799] ShenandoahEvacuationTask::work(unsigned int)+0x59 (shenandoahHeap.cpp:1193)
V [libjvm.so+0x1a8fc6e] WorkerTaskDispatcher::worker_run_task()+0x84 (workerThread.cpp:69)
V [libjvm.so+0x1a9036a] WorkerThread::run()+0x34 (workerThread.cpp:200)
V [libjvm.so+0x193cc26] Thread::call_run()+0x1a8 (thread.cpp:243)
V [libjvm.so+0x1550efc] thread_native_entry(Thread*)+0x1c3 (os_linux.cpp:868)
```
with
```
#
# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (/shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp:1166), pid=36555, tid=36562
# Error: Shenandoah assert_correct failed; Instance class mirror should point to Metaspace
Referenced from:
no interior location recorded (probably a plain heap scan, or detached oop)
Object:
0x00000000ff68f498 - klass 0x00000000641ca5a8 java.lang.Class
not allocated after mark start
not after update watermark
marked strong
not marked weak
in collection set
mark: marked(0x00000000ffe69d23)
region: | 474|CS |Y|BTE ff680000, ff6a3a98, ff6c0000|TAMS ff69bc50|UWM ff6a3a98|U 142K|T 126K|G 0B|S 16400B|L 49784B|CP 0
0x00000000ff68f498: 00000000ffe69d23 00000000001ca5a8 #...............
0x00000000ff68f4a8: 00007f1fa819be60 0000000000000000 `...............
Forwardee:
0x00000000ffe69d20 - klass 0x00000000641ca5a8 java.lang.Class
allocated after mark start
not after update watermark
marked strong
marked weak
not in collection set
mark: mark(is_unlocked no_hash age=0)
region: | 505|R |Y|BTE ffe40000, ffe80000, ffe80000|TAMS ffe40000|UWM ffe80000|U 256K|T 0B|G 212K|S 44352B|L 0B|CP 0
0x00000000ffe69d20: 0000000000000001 00000000001ca5a8 ................
0x00000000ffe69d30: 0000000000000000 0000000000000000 ................
```
In all cases, the assertion trips over an invalid mirror object. I added the hex dump of both oops. It shows that the forwarded oop carries a pointer to the deallocated Klass (0x7f1fa819be60) , the forwardee oop's Klass field is nulled out correctly.
==================
What happens:
Object is the mirror for class "jdk.internal.event.Event". This type is always redefined by JFR right after loading. Class redefinition causes the creation of a new Klass and a new mirror object. The old Klass is discarded (reclaimed), and now, with JDK-8340297, zapped. The old class mirror's Klass reference is nulled out, and the old mirror is abandoned to be collected eventually.
However, if this happens during concurrent evacuation, the class mirror may be in a forwarded state. Class redefinition only fixes the forwardee version of the class mirror. The old, forwarded version still contains the old Klass reference to the now reclaimed Klass. If we check that forwarded object, e.g. as part of a heap dump, we will assert in `ShenandoaAsserts::assert_correct` when encountering and checking the mirror for the "jdk.internal.event.Event" class.
With that patch activated, I get asserts in some Shenandoah tests:
- gc/shenandoah/mxbeans/TestPauseNotifications#aggressive
- serviceability/dcmd/gc/HeapDumpCompressedTest#Shenandoah
- gc/shenandoah/mxbeans/TestChurnNotifications#aggressive
- compiler/gcbarriers/UnsafeIntrinsicsTest#Shenandoah
```
Stack: [0x00007f8888a8a000,0x00007f8888b8a000], sp=0x00007f8888b85f30, free space=1007k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x14d9f88] NativeStackPrinter::print_stack(outputStream*, char*, int, unsigned char*&, bool, int)+0x74 (shenandoahHeap.cpp:1166)
V [libjvm.so+0x19f350c] VMError::report(outputStream*, bool)+0x1ba4 (vmError.cpp:960)
V [libjvm.so+0x19f6ea5] VMError::report_and_die(int, char const*, char const*, __va_list_tag*, Thread*, unsigned char*, void const*, void const*, char const*, int, unsigned long)+0x8b5 (vmError.cpp:1846)
V [libjvm.so+0xbb5d3c] report_vm_status_error(char const*, int, char const*, int, char const*)+0x0 (debug.cpp:195)
V [libjvm.so+0xbb5aa5] print_error_for_unit_test(char const*, char const*, __va_list_tag*)+0x0 (debug.cpp:148)
V [libjvm.so+0x16b226a] ShenandoahAsserts::assert_in_heap_bounds(void*, oopDesc*, char const*, int)+0x0 (shenandoahAsserts.cpp:173)
V [libjvm.so+0x16b2830] ShenandoahAsserts::assert_correct(void*, oopDesc*, char const*, int)+0x4c6 (shenandoahAsserts.cpp:289)
V [libjvm.so+0x16b2c30] ShenandoahAsserts::assert_marked(void*, oopDesc*, char const*, int)+0x32 (shenandoahAsserts.cpp:365)
V [libjvm.so+0x1736693] ShenandoahConcurrentEvacuateRegionObjectClosure::do_object(oopDesc*)+0x31 (shenandoahHeap.cpp:1166)
V [libjvm.so+0x17390a1] void ShenandoahHeap::marked_object_iterate<ShenandoahConcurrentEvacuateRegionObjectClosure>(ShenandoahHeapRegion*, ShenandoahConcurrentEvacuateRegionObjectClosure*, HeapWordImpl**)+0x4dd (shenandoahHeap.inline.hpp:565)
V [libjvm.so+0x1737df9] void ShenandoahHeap::marked_object_iterate<ShenandoahConcurrentEvacuateRegionObjectClosure>(ShenandoahHeapRegion*, ShenandoahConcurrentEvacuateRegionObjectClosure*)+0x3b (shenandoahHeap.inline.hpp:508)
V [libjvm.so+0x17368da] ShenandoahEvacuationTask::do_work()+0xd6 (shenandoahHeap.cpp:1207)
V [libjvm.so+0x1736799] ShenandoahEvacuationTask::work(unsigned int)+0x59 (shenandoahHeap.cpp:1193)
V [libjvm.so+0x1a8fc6e] WorkerTaskDispatcher::worker_run_task()+0x84 (workerThread.cpp:69)
V [libjvm.so+0x1a9036a] WorkerThread::run()+0x34 (workerThread.cpp:200)
V [libjvm.so+0x193cc26] Thread::call_run()+0x1a8 (thread.cpp:243)
V [libjvm.so+0x1550efc] thread_native_entry(Thread*)+0x1c3 (os_linux.cpp:868)
```
with
```
#
# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (/shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp:1166), pid=36555, tid=36562
# Error: Shenandoah assert_correct failed; Instance class mirror should point to Metaspace
Referenced from:
no interior location recorded (probably a plain heap scan, or detached oop)
Object:
0x00000000ff68f498 - klass 0x00000000641ca5a8 java.lang.Class
not allocated after mark start
not after update watermark
marked strong
not marked weak
in collection set
mark: marked(0x00000000ffe69d23)
region: | 474|CS |Y|BTE ff680000, ff6a3a98, ff6c0000|TAMS ff69bc50|UWM ff6a3a98|U 142K|T 126K|G 0B|S 16400B|L 49784B|CP 0
0x00000000ff68f498: 00000000ffe69d23 00000000001ca5a8 #...............
0x00000000ff68f4a8: 00007f1fa819be60 0000000000000000 `...............
Forwardee:
0x00000000ffe69d20 - klass 0x00000000641ca5a8 java.lang.Class
allocated after mark start
not after update watermark
marked strong
marked weak
not in collection set
mark: mark(is_unlocked no_hash age=0)
region: | 505|R |Y|BTE ffe40000, ffe80000, ffe80000|TAMS ffe40000|UWM ffe80000|U 256K|T 0B|G 212K|S 44352B|L 0B|CP 0
0x00000000ffe69d20: 0000000000000001 00000000001ca5a8 ................
0x00000000ffe69d30: 0000000000000000 0000000000000000 ................
```
In all cases, the assertion trips over an invalid mirror object. I added the hex dump of both oops. It shows that the forwarded oop carries a pointer to the deallocated Klass (0x7f1fa819be60) , the forwardee oop's Klass field is nulled out correctly.
==================
What happens:
Object is the mirror for class "jdk.internal.event.Event". This type is always redefined by JFR right after loading. Class redefinition causes the creation of a new Klass and a new mirror object. The old Klass is discarded (reclaimed), and now, with JDK-8340297, zapped. The old class mirror's Klass reference is nulled out, and the old mirror is abandoned to be collected eventually.
However, if this happens during concurrent evacuation, the class mirror may be in a forwarded state. Class redefinition only fixes the forwardee version of the class mirror. The old, forwarded version still contains the old Klass reference to the now reclaimed Klass. If we check that forwarded object, e.g. as part of a heap dump, we will assert in `ShenandoaAsserts::assert_correct` when encountering and checking the mirror for the "jdk.internal.event.Event" class.
- caused by
-
JDK-8337213 Shenandoah: Add verification for class mirrors
-
- Resolved
-
- duplicates
-
JDK-8340381 Shenandoah: Class mirrors verification should check forwarded objects
-
- Closed
-
- relates to
-
JDK-8340364 Shenandoah: Dead class mirrors crash GC
-
- Open
-
-
JDK-8340297 Use-after-free recognition for metaspace and class space
-
- In Progress
-
- links to
-
Commit(master) openjdk/jdk/445e5ecd
-
Review(master) openjdk/jdk/26187
(1 links to)