With JDK 22 we're starting to see various issues arising from jdk.internal.vm.FillerArray. It's not clear whether this is a duplicate of JDK-8319548 or not, so I'm filing this issue to track it.

The issue we see is, "java.lang.ClassCastException: class Ljdk.internal.vm.FillerArray; cannot be cast to class ...", which seems to appear in different execution paths, and with different targets of the cast, e.g. java.nio.ByteBuffer, java.util.Map, or even a non-JDK type org.elasticsearch.index.engine.LiveVersionMap$VersionLookup.

I originally thought that this was only occurring during heap dump, but I now think that it could occur at any time when the GC is under pressure?

Here are some snippets of stacktraces that we see:

java.lang.ClassCastException: class Ljdk.internal.vm.FillerArray; cannot be cast to class java.nio.ByteBuffer (Ljdk.internal.vm.FillerArray; and java.nio.ByteBuffer are in module java.base of loader 'bootstrap')
  at io.netty.buffer@4.1.107.Final/io.netty.buffer.PoolChunk.allocate(PoolChunk.java:354)
  at io.netty.buffer@4.1.107.Final/io.netty.buffer.PoolChunkList.allocate(PoolChunkList.java:108)
  at io.netty.buffer@4.1.107.Final/io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:204)
  ...

In one particular case, I see hundreds of these, all appearing around the same time:

java.lang.ClassCastException: class Ljdk.internal.vm.FillerArray; cannot be cast to class org.elasticsearch.index.engine.LiveVersionMap$VersionLookup (Ljdk.internal.vm.FillerArray; is in module java.base of loader 'bootstrap'; org.elasticsearch.index.engine.LiveVersionMap$VersionLookup is in module org.elasticsearch.server@8.14.0 of loader 'app')
  at org.elasticsearch.stateless@8.14.0/co.elastic.elasticsearch.stateless.engine.StatelessLiveVersionMapArchive.getRamBytesUsed(StatelessLiveVersionMapArchive.java:156)
  at org.elasticsearch.server@8.14.0/org.elasticsearch.index.engine.LiveVersionMap.ramBytesUsedForRefresh(LiveVersionMap.java:483)
  at org.elasticsearch.server@8.14.0/org.elasticsearch.index.engine.InternalEngine.getIndexBufferRAMBytesUsed(InternalEngine.java:2573)
  at org.elasticsearch.server@8.14.0/org.elasticsearch.index.shard.IndexShard.getIndexBufferRAMBytesUsed(IndexShard.java:2355)
  ...

java.lang.IncompatibleClassChangeError: Class Ljdk.internal.vm.FillerArray; does not implement the requested interface java.util.Map
  at org.apache.lucene.core@9.10.0/org.apache.lucene.index.ReadersAndUpdates.getNumDVUpdates(ReadersAndUpdates.java:168)
  at org.apache.lucene.core@9.10.0/org.apache.lucene.index.ReaderPool.anyDocValuesChanges(ReaderPool.java:384)
  at org.apache.lucene.core@9.10.0/org.apache.lucene.index.IndexWriter.nrtIsCurrent(IndexWriter.java:5776)
  at org.apache.lucene.core@9.10.0/org.apache.lucene.index.StandardDirectoryReader.isCurrent(StandardDirectoryReader.java:455)
  at org.apache.lucene.core@9.10.0/org.apache.lucene.index.FilterDirectoryReader.isCurrent(FilterDirectoryReader.java:133)
  at org.elasticsearch.server@8.14.0/org.elasticsearch.index.engine.Engine.refreshNeeded(Engine.java:1093)
  ...