Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8336042

Caller/callee param size mismatch in deoptimization causes crash

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: P3 P3
    • 25
    • 8, 11, 17, 18, 21, 23, 24, 25
    • hotspot
    • b13

      For interpreter frames, the caller contains the locals. Some platforms also align the callee frame, so they compute locals in layout_activation relative to the caller last_sp and not the callee sender_sp. When calling certain MethodHandle.linkTo* methods, the interpreter pushes 1 extra trailing arg for the MemberName, which is removed before calling the callee taret method. However, the interpreter takes the snapshot of last_sp before popping the trailing arg, and the return entry point pops the number of args represented by the resolved method in the constant pool. This is a synthetic method whose signature includes the MemberName. The deoptimization code in vframeArray::unpack_to_stack has logic to check has_member_arg() on the resolved target method and make the proper adjustments. However, the adjustment is missing for the oldest/bottom deoptimized frame, causing the computation of the callee locals based on the caller last_sp to be off by one, allowing the last local to overwrite the bottom of the callee frame (on aarch64 this is the return pc).

      Instead of the correct logic for the is_bottom frame, there is code that sets caller_was_method_handle based on if the invoke is invokedynamic or invokehandle. This code dates back to ricochet frames when there could be an arbitrary mismatch between the arg count of the caller's invoke and the callee. However, this code fails to take into account all cases of calls to has_member_arg() MH linkTo* intrinsics.

      In addition, the last_frame_adjust() logic is also broken. The adjustment can be done when not needed, and the same caller frame can be adjusted multiple times (every time the callee is deoptimized), resulting in unbounded frame/stack size growth.

            dlong Dean Long
            dlong Dean Long
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: