The test "applications/ctw/modules/java_xml.java" runs into this assertion on AIX when compiling "com.sun.org.apache.xalan.internal.xsltc.dom.DOMWSFilter::<init>" with C2. The Klass which can't be encoded is 0x0a00010091130af8
com.sun.org.apache.xalan.internal.xsltc.runtime.AbstractTranslet {0x0a00010091130af8}
 - instance size: 18
 - klass size: 118
 - access: public synchronized abstract
 - flags: has_nonstatic_fields has_localvariable_table has_miranda_methods has_final_method
 - state: loaded
 - name: &apos;com/sun/org/apache/xalan/internal/xsltc/runtime/AbstractTranslet&apos;
 - super: &apos;java/lang/Object&apos;
 - sub:
 - arrays: null
 - methods: Array&lt;T&gt;(0x0a0001009112d130)
 - method ordering: Array&lt;T&gt;(0x0a00010090000018)
 - local interfaces: Array&lt;T&gt;(0x0a0001009112d120)
 - trans. interfaces: Array&lt;T&gt;(0x0a0001009112d120)
 - secondary supers: Array&lt;T&gt;(0x0a0001009112d120)
 - hash_slot: 45
 - bitmap: 0x0010000000000000
 - constants: constant pool [727]/operands[35] {0x0a0001009112b8a8} for &apos;com/sun/org/apache/xalan/internal/xsltc/runtime/AbstractTranslet&apos;
 - class loader data: loader data: 0x0000000114020150 of &apos;bootstrap&apos;
 - source file: &apos;AbstractTranslet.java&apos;
 - inner classes: Array&lt;T&gt;(0x0a000100911309b0)
 - nest members: Array&lt;T&gt;(0x0a00010090000020)
 - permitted subclasses: Array&lt;T&gt;(0x0a00010090000020)

C2 tries to encode it although it is abstract. The issue has only been observed on AIX so far.

Stack snippet:
0x00000001157ba0f0 - 0x090000001bb87ed0 libjvm.so::CompressedKlassPointers::encode_not_null(Klass*, unsigned char*, int)+0x128 (C++ uses_alloca saves_lr stores_bc gpr_saved:1 fixedparms:3 parmsonstk:1)
0x00000001157ba170 - 0x090000001c4d1fe4 libjvm.so::loadConNKlass_ExNode::postalloc_expand(GrowableArray<Node*>*, PhaseRegAlloc*)+0x23c (C++ uses_alloca saves_lr stores_bc gpr_saved:10 fixedparms:3 parmsonstk:1)
0x00000001157ba230 - 0x090000001c6e7ec0 libjvm.so::PhaseCFG::postalloc_expand(PhaseRegAlloc*)+0x5bc (C++ uses_alloca saves_cr saves_lr stores_bc gpr_saved:18 fixedparms:2 parmsonstk:1)
0x00000001157ba450 - 0x090000001c87eb50 libjvm.so::Compile::Code_Gen()+0x394 (C++ uses_alloca saves_lr stores_bc gpr_saved:6 fixedparms:1 parmsonstk:1)

"loadConNKlass_ExNode" is trying to load a ConNKlass into a RegN.

The transformation here is no longer valid with Klass pointers which can't be encoded: https://github.com/openjdk/jdk/blob/ab9b72c50a5f324e53b8c6535f401cc185b98c75/src/hotspot/share/opto/compile.cpp#L3317

I think that all platforms are affected on which Matcher::const_klass_prefer_decode() can become true. These are aarch64, ppc64, riscv, s390. However, the problem only arrises when C2 encounters a Klass outside the encodable range which depends on the location of the metaspace.