The JVMTI RetransformClasses gives class class format error or crashes on the invokedynamic bytecodes. It is currently combined in one CR but may need to split into two different CRs.

Below is the email from Robert Field:

-------- Original Message --------
The failing test is a java.lang.instrument test
(IsModifiableClassAgent). To get at the underlying issue, I wrote a
JVMTI test. This test shows that there are at least two bugs: the class
reconstruction done by retransformClass for a class with invokedynamic
generates bad class bytes, and retransformClasses handed the original
bytes, at least for java.utiil.Arrays (which uses invokedynamic) crashes
Hotspot.

I talked to John about these, he was aware of a bug report involving
redefineClasses and popFrame -- I found that one. I could not find a
bug for what I'm seeing.

The JVMTI test is attached (cfe.c) with the crash log (and a hello world
class I used for testing).

As attached this will show the original bytes crash. To see the bad
reconstruction, dump the incoming class bytes or pass it through instead
of the original bytes.
. . .

Just to underscore, this test fails only under the lambda repo where invokedynamic instructions are now generated by the compiler into class file. I would assume it would also fail if the offending class file was loaded and retransformed in a mainline build.

> ----------- from Serguei ---------
> Ok, I see you wrote this:
> > ... , at least for java.utiil.Arrays (which uses invokedynamic) crashes Hotspot.
>

Right, in the lambda repo, this and some other library classes which have lambda expressions are compiled into code with invokedynamic instructions and the corresponding new sections in the class file.

> How do you know this issue is related to invokedynamic bytecodes?

It happily churns through lots of classes before dying on this one. And this test did not fail until flipping the switch to use invokedynamic.

-Robert
------------------------------------

Also, please, see the hs_err dump and the test (provided by Robert) in the comment sections.