-
Type:
Bug
-
Resolution: Unresolved
-
Priority:
P4
-
Affects Version/s: 17, 21, 25
-
Component/s: hotspot
We identified a bug in the way records with type annotation are retransformed corrupt the in-memory class representation leading to NoSuchFieldError when using the retransformed record.
Here the steps to reproduce:
1. create a java file named NoSuchFieldErrorOnRecord.java with the following content:
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
public class NoSuchFieldErrorOnRecord {
public static void main(String[] args) throws Exception {
// Load the record class
try {
MyRecord.parse(null);
} catch (Exception e) {
// swallow exception
}
// Wait for retransformation
Thread.sleep(500);
// calling again the record
System.out.println(MyRecord.parse("foo"));
}
}
@MyTypeAnnotation
record MyRecord(@MyTypeUseAnnotation String filter) {
public static MyRecord parse(String param) {
if (param == null) {
throw new IllegalArgumentException("Filter cannot be null");
}
return new MyRecord(param);
}
}
@Target({ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@interface MyTypeAnnotation {
}
@Target({ ElementType.TYPE_USE })
@Retention(RetentionPolicy.RUNTIME)
@interface MyTypeUseAnnotation {
}
2. Here is the tricky part: for retransforming the record we are using ASM lib to read and rewrite WITHOUT any changes the cladssfile. But the ASM processing for this
will implicitly rewrite the constant pool indices which will also trigger the rewrite of the constant pool during VM_RedefineClass operation.
here the instrumenting java agent:
public class Agent {
public static void premain(String arg, Instrumentation inst) {
new Thread(() -> retransformLoop(inst, arg)).start();
}
private static void retransformLoop(Instrumentation instrumentation, String className) {
System.out.println("Starting retransform loop for: " + className);
Class<?> classToRetransform = null;
while (classToRetransform == null) {
for (Class<?> clazz : instrumentation.getAllLoadedClasses()) {
if (clazz.getName().equals(className)) {
System.out.println("found class: " + className);
classToRetransform = clazz;
break;
}
}
}
try {
IdentityTransformer identityTransformer = new IdentityTransformer(className);
instrumentation.addTransformer(identityTransformer, true);
instrumentation.retransformClasses(classToRetransform);
instrumentation.removeTransformer(identityTransformer);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
class IdentityTransformer implements ClassFileTransformer {
private final String className;
public IdentityTransformer(String className) {
this.className = className;
}
@Override
public byte[] transform(ClassLoader loader, String classPath, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {
if (classPath != null && classPath.equals(className.replace('.', '/'))) {
System.out.println("Transforming " + className);
try {
ClassReader reader = new ClassReader(classfileBuffer);
ClassNode classNode = new ClassNode();
reader.accept(classNode, ClassReader.SKIP_FRAMES);
ClassWriter writer = new ClassWriter(ClassWriter.COMPUTE_FRAMES);
classNode.accept(writer);
return writer.toByteArray();
} catch (Throwable t) {
t.printStackTrace();
return null;
}
}
return null;
}
}
3. after generating the jar for this agent with the following Manifest:
"Premain-Class": "Agent",
"Can-Retransform-Classes": true
4. we are running the main class as such:
java "-Xlog:redefine*=debug" -javaagent:inst-agent-1.0-SNAPSHOT-all.jar=MyRecord NoSuchFieldErrorOnRecord
and here the output:
Starting retransform loop for: MyRecord
found class: MyRecord
[0,046s][debug][redefine,class,load] loading name=MyRecord kind=101 (avail_mem=658848K)
Transforming MyRecord
[0,066s][info ][redefine,class,constantpool] old_cp_len=65, scratch_cp_len=65
[0,066s][debug][redefine,class,constantpool] after pass 0: merge_cp_len=65
[0,066s][debug][redefine,class,constantpool] after pass 1a: merge_cp_len=65, scratch_i=65, index_map_len=63
[0,066s][info ][redefine,class,constantpool] merge_cp_len=65, index_map_len=63
[0,066s][debug][redefine,class,annotation ] num_annotations=1
[0,066s][debug][redefine,class,annotation ] type_index=4864 num_element_value_pairs=14
[0,066s][debug][redefine,class,annotation ] element_name_index=0
[0,066s][debug][redefine,class,annotation ] length() is too small for a tag
[0,066s][debug][redefine,class,annotation ] bad element_value at 0
[0,066s][debug][redefine,class,annotation ] bad annotation_struct at 0
[0,066s][debug][redefine,class,annotation ] bad record_component_type_annotations at 0
[0,066s][debug][redefine,class,load ] loaded name=MyRecord (avail_mem=654848K)
[0,066s][info ][redefine,class,load ] redefined name=MyRecord, count=1 (avail_mem=654848K)
[0,067s][debug][redefine,class,nmethod ] Marked dependent nmethods for deopt
[0,067s][info ][redefine,class,update ] adjust: name=MyRecord
[0,067s][info ][redefine,class,timer ] vm_op: all=20 prologue=20 doit=0
[0,067s][info ][redefine,class,timer ] redefine_single_class: phase1=0 phase2=0
Exception in thread "main" java.lang.NoSuchFieldError: Class MyRecord does not have member field 'java.lang.String filter'
at MyRecord.<init>(NoSuchFieldErrorOnRecord.java:22)
at MyRecord.parse(NoSuchFieldErrorOnRecord.java:27)
at NoSuchFieldErrorOnRecord.main(NoSuchFieldErrorOnRecord.java:17)
Notice the debug logs:
- length() is too small for a tag
- bad element_value at 0
- bad annotation_struct at 0
- bad record_component_type_annotations at 0
indicating that something wrong is happening when redefining the class. Though no exception is thrown.
Looking at the source code we can notice in VM_RedefineClasses::load_new_class_versions:
res = merge_cp_and_rewrite(the_class, scratch_class, THREAD);
if (HAS_PENDING_EXCEPTION) {
Symbol* ex_name = PENDING_EXCEPTION->klass()->name();
log_info(redefine, class, load, exceptions)("merge_cp_and_rewrite exception: '%s'", ex_name->as_C_string());
CLEAR_PENDING_EXCEPTION;
if (ex_name == vmSymbols::java_lang_OutOfMemoryError()) {
return JVMTI_ERROR_OUT_OF_MEMORY;
} else {
return JVMTI_ERROR_INTERNAL;
}
}
the result of the call to merge_cp_and_rewrite is not handled at all. Therefore the error is not propagated.
if we add:
if (res != JVMTI_ERROR_NONE) {
return res;
}
then we get:
[0.265s][debug][redefine,class,annotation ] length() is too small for a tag
[0.265s][debug][redefine,class,annotation ] bad element_value at 0
[0.265s][debug][redefine,class,annotation ] bad annotation_struct at 0
[0.265s][debug][redefine,class,annotation ] bad record_component_type_annotations at 0
Exception in thread "Thread-0" java.lang.InternalError
at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:221)
at com.bempel.Agent.retransformLoop(Agent.java:64)
at com.bempel.Agent.lambda$premain$0(Agent.java:27)
at java.base/java.lang.Thread.run(Thread.java:1516)
which prevent bad redefinitions and notify the caller that something bad happen and can bail out the new version of the class and revert to the original class.
Now the main issue is located in VM_RedefineClasses::rewrite_cp_refs_in_record_attribute:
AnnotationArray* type_annotations = component->type_annotations();
if (type_annotations != nullptr && type_annotations->length() != 0) {
log_debug(redefine, class, annotation)("rewriting type annot for record. type_annotations->length()=%d", type_annotations->length());
int byte_i = 0; // byte index into annotations
if (!rewrite_cp_refs_in_annotations_typeArray(type_annotations, byte_i)) {
log_debug(redefine, class, annotation)("bad record_component_type_annotations at %d", i);
// propagate failure back to caller
return false;
}
}
for the type annotation array we are calling rewrite_cp_refs_in_annotations_typeArray instead of rewrite_cp_refs_in_type_annotations_typeArray, and therefore trying to decode the incorrect struct from the buffer. An error is indeed detected but not propagated as mentioned above, leaving the class in bad state leading to the NoSuchFieldError in our case.
Here the steps to reproduce:
1. create a java file named NoSuchFieldErrorOnRecord.java with the following content:
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
public class NoSuchFieldErrorOnRecord {
public static void main(String[] args) throws Exception {
// Load the record class
try {
MyRecord.parse(null);
} catch (Exception e) {
// swallow exception
}
// Wait for retransformation
Thread.sleep(500);
// calling again the record
System.out.println(MyRecord.parse("foo"));
}
}
@MyTypeAnnotation
record MyRecord(@MyTypeUseAnnotation String filter) {
public static MyRecord parse(String param) {
if (param == null) {
throw new IllegalArgumentException("Filter cannot be null");
}
return new MyRecord(param);
}
}
@Target({ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@interface MyTypeAnnotation {
}
@Target({ ElementType.TYPE_USE })
@Retention(RetentionPolicy.RUNTIME)
@interface MyTypeUseAnnotation {
}
2. Here is the tricky part: for retransforming the record we are using ASM lib to read and rewrite WITHOUT any changes the cladssfile. But the ASM processing for this
will implicitly rewrite the constant pool indices which will also trigger the rewrite of the constant pool during VM_RedefineClass operation.
here the instrumenting java agent:
public class Agent {
public static void premain(String arg, Instrumentation inst) {
new Thread(() -> retransformLoop(inst, arg)).start();
}
private static void retransformLoop(Instrumentation instrumentation, String className) {
System.out.println("Starting retransform loop for: " + className);
Class<?> classToRetransform = null;
while (classToRetransform == null) {
for (Class<?> clazz : instrumentation.getAllLoadedClasses()) {
if (clazz.getName().equals(className)) {
System.out.println("found class: " + className);
classToRetransform = clazz;
break;
}
}
}
try {
IdentityTransformer identityTransformer = new IdentityTransformer(className);
instrumentation.addTransformer(identityTransformer, true);
instrumentation.retransformClasses(classToRetransform);
instrumentation.removeTransformer(identityTransformer);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
class IdentityTransformer implements ClassFileTransformer {
private final String className;
public IdentityTransformer(String className) {
this.className = className;
}
@Override
public byte[] transform(ClassLoader loader, String classPath, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {
if (classPath != null && classPath.equals(className.replace('.', '/'))) {
System.out.println("Transforming " + className);
try {
ClassReader reader = new ClassReader(classfileBuffer);
ClassNode classNode = new ClassNode();
reader.accept(classNode, ClassReader.SKIP_FRAMES);
ClassWriter writer = new ClassWriter(ClassWriter.COMPUTE_FRAMES);
classNode.accept(writer);
return writer.toByteArray();
} catch (Throwable t) {
t.printStackTrace();
return null;
}
}
return null;
}
}
3. after generating the jar for this agent with the following Manifest:
"Premain-Class": "Agent",
"Can-Retransform-Classes": true
4. we are running the main class as such:
java "-Xlog:redefine*=debug" -javaagent:inst-agent-1.0-SNAPSHOT-all.jar=MyRecord NoSuchFieldErrorOnRecord
and here the output:
Starting retransform loop for: MyRecord
found class: MyRecord
[0,046s][debug][redefine,class,load] loading name=MyRecord kind=101 (avail_mem=658848K)
Transforming MyRecord
[0,066s][info ][redefine,class,constantpool] old_cp_len=65, scratch_cp_len=65
[0,066s][debug][redefine,class,constantpool] after pass 0: merge_cp_len=65
[0,066s][debug][redefine,class,constantpool] after pass 1a: merge_cp_len=65, scratch_i=65, index_map_len=63
[0,066s][info ][redefine,class,constantpool] merge_cp_len=65, index_map_len=63
[0,066s][debug][redefine,class,annotation ] num_annotations=1
[0,066s][debug][redefine,class,annotation ] type_index=4864 num_element_value_pairs=14
[0,066s][debug][redefine,class,annotation ] element_name_index=0
[0,066s][debug][redefine,class,annotation ] length() is too small for a tag
[0,066s][debug][redefine,class,annotation ] bad element_value at 0
[0,066s][debug][redefine,class,annotation ] bad annotation_struct at 0
[0,066s][debug][redefine,class,annotation ] bad record_component_type_annotations at 0
[0,066s][debug][redefine,class,load ] loaded name=MyRecord (avail_mem=654848K)
[0,066s][info ][redefine,class,load ] redefined name=MyRecord, count=1 (avail_mem=654848K)
[0,067s][debug][redefine,class,nmethod ] Marked dependent nmethods for deopt
[0,067s][info ][redefine,class,update ] adjust: name=MyRecord
[0,067s][info ][redefine,class,timer ] vm_op: all=20 prologue=20 doit=0
[0,067s][info ][redefine,class,timer ] redefine_single_class: phase1=0 phase2=0
Exception in thread "main" java.lang.NoSuchFieldError: Class MyRecord does not have member field 'java.lang.String filter'
at MyRecord.<init>(NoSuchFieldErrorOnRecord.java:22)
at MyRecord.parse(NoSuchFieldErrorOnRecord.java:27)
at NoSuchFieldErrorOnRecord.main(NoSuchFieldErrorOnRecord.java:17)
Notice the debug logs:
- length() is too small for a tag
- bad element_value at 0
- bad annotation_struct at 0
- bad record_component_type_annotations at 0
indicating that something wrong is happening when redefining the class. Though no exception is thrown.
Looking at the source code we can notice in VM_RedefineClasses::load_new_class_versions:
res = merge_cp_and_rewrite(the_class, scratch_class, THREAD);
if (HAS_PENDING_EXCEPTION) {
Symbol* ex_name = PENDING_EXCEPTION->klass()->name();
log_info(redefine, class, load, exceptions)("merge_cp_and_rewrite exception: '%s'", ex_name->as_C_string());
CLEAR_PENDING_EXCEPTION;
if (ex_name == vmSymbols::java_lang_OutOfMemoryError()) {
return JVMTI_ERROR_OUT_OF_MEMORY;
} else {
return JVMTI_ERROR_INTERNAL;
}
}
the result of the call to merge_cp_and_rewrite is not handled at all. Therefore the error is not propagated.
if we add:
if (res != JVMTI_ERROR_NONE) {
return res;
}
then we get:
[0.265s][debug][redefine,class,annotation ] length() is too small for a tag
[0.265s][debug][redefine,class,annotation ] bad element_value at 0
[0.265s][debug][redefine,class,annotation ] bad annotation_struct at 0
[0.265s][debug][redefine,class,annotation ] bad record_component_type_annotations at 0
Exception in thread "Thread-0" java.lang.InternalError
at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:221)
at com.bempel.Agent.retransformLoop(Agent.java:64)
at com.bempel.Agent.lambda$premain$0(Agent.java:27)
at java.base/java.lang.Thread.run(Thread.java:1516)
which prevent bad redefinitions and notify the caller that something bad happen and can bail out the new version of the class and revert to the original class.
Now the main issue is located in VM_RedefineClasses::rewrite_cp_refs_in_record_attribute:
AnnotationArray* type_annotations = component->type_annotations();
if (type_annotations != nullptr && type_annotations->length() != 0) {
log_debug(redefine, class, annotation)("rewriting type annot for record. type_annotations->length()=%d", type_annotations->length());
int byte_i = 0; // byte index into annotations
if (!rewrite_cp_refs_in_annotations_typeArray(type_annotations, byte_i)) {
log_debug(redefine, class, annotation)("bad record_component_type_annotations at %d", i);
// propagate failure back to caller
return false;
}
}
for the type annotation array we are calling rewrite_cp_refs_in_annotations_typeArray instead of rewrite_cp_refs_in_type_annotations_typeArray, and therefore trying to decode the incorrect struct from the buffer. An error is indeed detected but not propagated as mentioned above, leaving the class in bad state leading to the NoSuchFieldError in our case.
- links to
-
Review(master)
openjdk/jdk/29445