[lworld] Handle mismatches of the preload attribute in the calling convention

C2 passes inline type arguments in scalarized form if the calling convention of the resolved method supports this. That means that the calling conventions of the overriding methods need to support scalarization as well. The preload attribute added with JDK-8281116 does not guarantee any consistency between the overridden and overriding method. We need to fix the calling convention accordingly.

Below is a summary I wrote a while ago.

-------------------------------------------------------------------------------------------

Not following the "global cache" version of option (3) for now, I was thinking about other ways to
make it work.

In general, we would always stick to the calling convention of the parent method. Our "multiple
interface example" would be handled like this:

interface I1 {
  void m(L*MyValue); // Scalarized
}

interface I2 {
  void m(LMyValue); // Non-scalarized
}

class C implements I1, I2 {
  void m(LMyValue) { }
}

Once C::m is linked, we detect the mismatch and force C::m to always use the non-scalarized calling
convention. Now if the method does not have any other scalarized arguments, that would mean that the
scalarized entry point is unused and just points to the non-scalarized entry point. But in this case
we could always create it and add code that triggers deoptimization and re-execution in the caller.

That's similar to option (2) in that we resolve the mismatch with deoptimization but it is easier to
implement and deoptimization would only be triggered in edge cases (otherwise we would just use the
non-scalarized calling convention right from the beginning). Also, I found several flaws with option
(2) and resolving them would add lots of additional complexity.

Potentially, we could also implement it the other way around: Always use the scalarized calling
convention for C::m but there are some technical challenges to make that work.


-------------------------------------------------------------------------------------------


I had a Zoom session with John this morning. I'm trying to summarize some of what we discussed.

----------------
Argument passing
----------------

- The (optimized) calling convention is fixed at method link time (when the holder klass is loaded).
- The interpreter and C1 always pass arguments in non-scalarized form.
- C2 passes inline type arguments in scalarized form if the calling convention of the resolved
method supports this. That means that the calling conventions of the overriding methods need to
support scalarization as well. Otherwise, we have a mismatch that can only be detected at runtime.
With Q, such a mismatch is not possible but with L* it is.
- Translations between non-scalarized and scalarized arguments are performed in the method entry
points for compiled-to-compiled calls and in the adapter for compiled-to-interpreted or
interpreted-to-compiled calls. It's not feasible to emit multiple entry points and adapters to
handle all the possible mismatches.

Options for handling mismatches:
(1) Reject classes that introduce mismatches during class loading.
(2) Detect mismatches at runtime during resolution of a C2 compiled call when the calling convention
used by the resolved method differs from the calling convention expected by the selected method. To
resolve the mismatch, deoptimize the C2 compiled caller and re-execute the call in the interpreter.
Potentially re-compile the caller without using the scalarized calling convention.
(3) Always use the calling convention of the overridden method and ignore the stars attached to the
arguments of the overriding method.

As I understand, (1) is not an option because we need to support migration.

For (2), we would need to implement deoptimization from call resolution and re-execution of a call
in the interpreter. Without prototyping, it's hard to tell how complicated that would be. This also
has the side effect that once a mismatch is detected for a (virtual) call, we would fall back to
using the non-scalarized calling convention for that call (i.e. for all arguments / callees).

Option (3) would make mismatches impossible. But there's one scenario that we can't handle, namely a
class implementing two interfaces both defining the same method with mismatching star settings:

interface I1 {
  void m(L*MyValue); // Scalarized
}

interface I2 {
  void m(LMyValue); // Non-scalarized
}

class C implements I1, I2 {
  void m(LMyValue) { }
}

No matter what calling convention we choose for C::m, a mismatch is possible through I1 (scalarized)
or I2 (non-scalarized) interface calls.

One variant of option (3) that John and I discussed, would be a dictionary that keeps track of the
calling convention used for each class.

- If I1 is loaded first, 'MyValue' would be loaded early due to the * and the dictionary would keep
track of 'MyValue' being scalarized. When I2::m and C::m are then loaded, we would consult the
dictionary and find that the scalarized calling convention should be used.
- If I2 is loaded first, 'MyValue' would not be loaded early and the dictionary would keep track of
'MyValue' being non-scalarized. When I1::m and C::m are then loaded, we would consult the dictionary
and find that the non-scalarized calling convention should be used.

I'm not sure how feasible that is though. Open questions are:
- Where would we keep track of this?
- Is the footprint overhead acceptable?
- What about concurrent loading of classes?

Also, this option has the side effect that the star setting is often ignored and the class loading
order has an impact on the calling convention of methods of completely unrelated classes.


-----------------
Return processing
-----------------

To avoid costly runtime checks in C2 compiled code to handle scalarized and non-scalarized returns,
the interpreter, C1 and C2 are *always* returning Q's in a scalarized form. Because the interpreter
and C1 do not support scalarization, an additional "field" is returned that either contains a
pointer to a heap buffer or a specially encoded Klass*. The interpreter and C1 can then check that
field and either use the pointer to the buffer or re-buffer.

Now current code partially relies on the fact that a Q return is always scalarized and an L return
is never scalarized. With the L* proposal, that would no longer be guaranteed because there can be a
mismatch between the resolved method having an L (non-scalarized) return and the selected method
having a L* (scalarized) return.

Our options are similar to the argument passing ones. For (2) we would detect the mismatch by adding
additional runtime checks to the interpreter and C1 (as I described in my earlier emails). I think
the overhead would be negligible because we already need these checks and would just apply them in
some additional cases. For (3) we would always choose the return convention the dictionary tells us
to use and mismatches would be impossible.