Background
Sjavac splits the workload into chunks in order to run jobs in parallel. If 3 or more jobs are available to run in parallel, the main JDK build job is run in 3 chunks, which usually works well. If not enough memory is available, there is some heuristics that will help avoid OOME by splitting the workload into more chunks, so that less memory is needed. This seriously affects performance, but is a reasonable fallback to keep the build working.

Problem
The split into more chunks uncovers a problem. It seems that if a class in one chunk refers to an inner class outside of its chunk, that inner class will not be found by javac. When the JDK classes are split into 3 chunks this doesn't happen, but if the heuristics are triggered, the fallback is 20 chunks and the missing class problem always happens.

Repro
To reproduce this, configure with --enable-sjavac --with-memory-size=1000. This will force sjavac into splitting into 20 chunks. The build will take a lot longer than usual but eventually fail with:

/localhome/hg/jdk8-build/jdk/src/share/classes/org/jcp/xml/dsig/internal/dom/DOMSignatureMethod.java:165: error: cannot access Instance
                    : Signature.getInstance(getJCAAlgorithm(), p);
                               ^
  class file for sun.security.jca.GetInstance$Instance not found
/localhome/hg/jdk8-build/jdk/src/share/classes/sun/security/jca/GetInstance.java:238: error: cannot find symbol
        return new Instance(s.getProvider(), instance);
                   ^
  symbol: class Instance
  location: class GetInstance
/localhome/hg/jdk8-build/jdk/src/share/classes/sun/security/jca/GetInstance.java:245: error: cannot find symbol
        return new Instance(s.getProvider(), instance);
                   ^
  symbol: class Instance
  location: class GetInstance
3 errors

The easiest way I have found to debug this is to edit spec.gmk (in the root of the output directory created by configure) and add the debug parameters to SJAVAC_SERVER_JAVA.