Java code is executed during CDS dump time. Currently (as of 2018/03/06, JDK 10 & 11) none of the code paths would create any VM anonymous classes (due to careful tuning of the core lib code). However, it's possible to create VM anonymous classes by dumping with "-Xshare:dump -javaagent:<jar>". It's also possible that in the future the core lib might be changed to create VM anonymous classes during class loading operations.

With the current JDK 11, the follow crash happens with a new test case that uses "-Xshare:dump -javaagent:<jar>"

===
ClassFileTransformer.premain() is called
Dumping class files to DUMP_CLASS_FILES/...
dump: DUMP_CLASS_FILES/java/lang/invoke/LambdaForm$MH000.class
dump: DUMP_CLASS_FILES/java/lang/invoke/LambdaForm$MH001.class
Invoked inside a Lambda
Loading classes to share ...
Loading classes to share: done.
[2.357s][info][cds] Shared spaces: preloaded 1181 classes
Rewriting and linking classes ...
Rewriting and linking classes: done
Number of classes 1297
    instance classes = 1202
    obj array classes = 87
    type array classes = 8
Updating ConstMethods ... done.
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f2b29edc33b, pid=6539, tid=6599
#
# JRE version: Java(TM) SE Runtime Environment (11.0) (slowdebug build 11-internal+0-adhoc.iklam.open)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (slowdebug 11-internal+0-adhoc.iklam.open, interpreted mode, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# V [libjvm.so+0xb6a33b] BasicHashtable<(MemoryType)1>::table_size() const+0x1f
#
# Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport %p %s %c %P" (or dumping to /jdk/tmp/jtreg/work/scratch/core.6539)
#
# An error report file with more information is saved as:
# /jdk/tmp/jtreg/work/scratch/hs_err_pid6539.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#
Current thread is 6599
Dumping core ...

===============


Therefore, for VM stability and future-proof, we need to ensure that CDS dump time can properly handle the existence of VM anonymous classes.

The VM needs to be fixed and a regression test case needs to be added.