JSR 292: +VerifyMethodHandles in product JVM can overflow buffer

On Oct 25, 2011, at 12:23 PM, Tom Rodriguez wrote:

Thanks for the pointer. I checked in crashes, where they are normally captured and didn't see anything. I didn't expect the bundle to end up with that data.

I tried adding some code to verify that the boundaries on the code blobs are being respected but didn't see any problems. Looking more closely at the core file and a regular run I think I see what's happening though. The location of mappings in the JVM that crashed are slightly worse from a code generation perspective so we have to emit 8 instructions to emit some constants. This causes code expansion which causes us to overflow the buffer. Most of the time the values are smaller so that's why it's hard to reproduce the crash.

An interim fix that could be pushed into the JDK would be to remove the VerifyMethodHandles option from that test and maybe others in that java/lang/invoke suite. I'm not actually sure why that's a product flag in the first place. That's causes a bunch extra assembly to be generated that we wouldn't normally allow in a product JVM. Turning it off should bring the total space required down to a safe level.


I think what's happening is that a stub is running off the end its code blob and overwriting space which is on the free list. If search_freelist looks at it after that then it crashes. The free list pointer appears to be 0x7fffffff78067900.

(dbx) x 0x7fffffff78067900/4lx
0x7fffffff78067900: 0xd875000a9a10000e 0x1f1fffff9e13e3ff
0x7fffffff78067910: 0x9f2bf00c9e13e780 0x9f2bf00a9e13e014
(dbx) p -flx $l7
$l7 = 0x9f2bf00c9e13e780
(dbx) dis 0x7fffffff78067900
0x7fffffff78067900: stx %o4, [%l4 + %o2]
0x7fffffff78067904: mov %sp, %o5
0x7fffffff78067908: sethi %hi(0x7ffffc00), %o7
0x7fffffff7806790c: bset 0x00000000000003ff, %o7 ! 0x7fffffff
0x7fffffff78067910: sllx %o7, 12, %o7

disassembly further back looks like this:

dbx) dis 0x7fffffff78067890
0x7fffffff78067890: ldx [%o0 + 24], %o2
0x7fffffff78067894: stx %o2, [%g4 + 24]
0x7fffffff78067898: ldx [%o0 + 16], %o2
0x7fffffff7806789c: stx %o2, [%g4 + 16]
0x7fffffff780678a0: ldx [%o0 + 8], %o2
0x7fffffff780678a4: stx %o2, [%g4 + 8]
0x7fffffff780678a8: ldx [%o0], %o2
0x7fffffff780678ac: stx %o2, [%g4]
0x7fffffff780678b0: and %l5, 255, %o2
0x7fffffff780678b4: add %l4, -8, %o3
(dbx) dis
0x7fffffff780678b8: cmp %o2, 0
0x7fffffff780678bc: be,pn %icc,0x7fffffff780678f4 ! 0x7fffffff780678f4
0x7fffffff780678c0: nop
0x7fffffff780678c4: sllx %o2, 3, %o4
0x7fffffff780678c8: add %l4, %o4, %o0
0x7fffffff780678cc: add %l4, 0, %o4
0x7fffffff780678d0: cmp %o4, %o0
0x7fffffff780678d4: bcc,pn %xcc,0x7fffffff780678f4 ! 0x7fffffff780678f4
0x7fffffff780678d8: nop
0x7fffffff780678dc: ldx [%o4], %g5
(dbx) dis
0x7fffffff780678e0: stx %g5, [%o4 - 8]
0x7fffffff780678e4: inc 8, %o4
0x7fffffff780678e8: cmp %o4, %o0
0x7fffffff780678ec: bcs,pt %xcc,0x7fffffff780678dc ! 0x7fffffff780678dc
0x7fffffff780678f0: nop
0x7fffffff780678f4: mov %o3, %l4
0x7fffffff780678f8: clr %o4
0x7fffffff780678fc: sllx %o2, 3, %o2
0x7fffffff78067900: stx %o4, [%l4 + %o2]
0x7fffffff78067904: mov %sp, %o5

I thought it was an arraycopy stub at first but I think it's one of the stubs for method handles. I'll take a look and see what's happening.

tom