[Nestmates] Assertion failure when the compiler thread appears to trigger classloading

Here's stack trace:

V [libjvm.so+0x166e66f] VMError::report_and_die(Thread*, char const*, int, char const*, char const*, __va_list_tag*)+0x2f
V [libjvm.so+0xaee89d] report_vm_error(char const*, int, char const*, char const*, ...)+0xdd
V [libjvm.so+0x159207b] SystemDictionary::resolve_or_null(Symbol*, Handle, Handle, Thread*)+0x27b
V [libjvm.so+0x159224e] SystemDictionary::resolve_or_fail(Symbol*, Handle, Handle, bool, Thread*)+0x1e
V [libjvm.so+0xad37bb] ConstantPool::klass_at_impl(constantPoolHandle const&, int, bool, Thread*)+0x31b
V [libjvm.so+0xe12d89] InstanceKlass::nest_top(Thread*)+0xe9
V [libjvm.so+0xe12fb2] InstanceKlass::has_nestmate_access_to(InstanceKlass*, Thread*)+0x22
V [libjvm.so+0x14557cf] Reflection::verify_field_access(Klass const*, Klass const*, Klass const*, AccessFlags, bool, bool)+0x28f
V [libjvm.so+0x10dd3ff] LinkResolver::check_method_accessability(Klass*, Klass*, Klass*, methodHandle const&, Thread*)+0x6f
V [libjvm.so+0x10de3dd] LinkResolver::resolve_method(LinkInfo const&, Bytecodes::Code, Thread*)+0x14d
V [libjvm.so+0x10dfbf3] LinkResolver::linktime_resolve_static_method(LinkInfo const&, Thread*)+0x63
V [libjvm.so+0x10dff82] LinkResolver::resolve_static_call(CallInfo&, LinkInfo const&, bool, Thread*)+0x32
V [libjvm.so+0x10e37e4] LinkResolver::resolve_static_call_or_null(LinkInfo const&)+0xb4
V [libjvm.so+0x8da556] ciEnv::lookup_method(ciInstanceKlass*, ciKlass*, Symbol*, Symbol*, Bytecodes::Code, constantTag)+0x1e6
V [libjvm.so+0x8dac45] ciEnv::get_method_by_index_impl(constantPoolHandle const&, int, Bytecodes::Code, ciInstanceKlass*)+0x1b5
V [libjvm.so+0x8db5e6] ciEnv::get_method_by_index(constantPoolHandle const&, int, Bytecodes::Code, ciInstanceKlass*)+0x276
V [libjvm.so+0x94f6b5] ciBytecodeStream::get_method(bool&, ciSignature**)+0x335
V [libjvm.so+0x78b963] GraphBuilder::invoke(Bytecodes::Code)+0x33
V [libjvm.so+0x78e604] GraphBuilder::iterate_bytecodes_for_block(int)+0x1c14
V [libjvm.so+0x78ea7f] GraphBuilder::iterate_all_blocks(bool)+0x9f
V [libjvm.so+0x78fb4f] GraphBuilder::GraphBuilder(Compilation*, IRScope*)+0x38f
V [libjvm.so+0x799a57] IRScope::IRScope(Compilation*, IRScope*, int, ciMethod*, int, bool)+0x277
V [libjvm.so+0x79adc1] IR::IR(Compilation*, ciMethod*, int)+0x121
V [libjvm.so+0x762bfd] Compilation::build_hir() [clone .part.116]+0x2fd
V [libjvm.so+0x7687d8] Compilation::compile_java_method() [clone .part.117]+0xbf8
V [libjvm.so+0x768e97] Compilation::compile_method()+0x247
V [libjvm.so+0x76a7ee] Compilation::Compilation(AbstractCompiler*, ciEnv*, ciMethod*, int, BufferBlob*, DirectiveSet*)+0x44e
V [libjvm.so+0x76bb6e] Compiler::compile_method(ciEnv*, ciMethod*, int, DirectiveSet*)+0x17e
V [libjvm.so+0xa663c5] CompileBroker::invoke_compiler_on_method(CompileTask*)+0x3e5
V [libjvm.so+0xa67590] CompileBroker::compiler_thread_loop()+0x340
V [libjvm.so+0x15d0f8e] JavaThread::thread_main_inner()+0x22e
V [libjvm.so+0x133540a] thread_native_entry(Thread*)+0xfa
C [libpthread.so.0+0x7e9a] start_thread+0xda

and the code

Klass* SystemDictionary::resolve_or_null(Symbol* class_name, Handle class_loader, Handle protection_domain, TRAPS) {
  assert(THREAD->can_call_java(),
         "can not load classes with compiler thread: class=%s, classloader=%s",
         class_name->as_C_string(),
         class_loader.is_null() ? "null" : class_loader->klass()->name()->as_C_string());

The problem is that the access check requires "loading" the nest-top class, and the compiler thread is not allowed to "load classes".

But actually the compiler thread is not allowed to execute Java code, which is different to not being able to load classes per-se. Further when we call resolve_or_null we may not need to load anything because it may already be loaded - hence the assertion is placed too early in the class-loading process.