From Tom Ball
-----------------------
As you all know, javac and javafxc file parsing is I/O intensive. I tried parsing files as separate tasks using the java.util.concurrent package, and found that parse time is indeed cut in half on my two-CPU system. Attached are the diffs for the two changed files.

JavaCompiler.parseFiles() creates an ExecutorService while has a fixed thread pool equal to the number of available processors (this change is bypassed if there is only one). Brian may find my simplistic use of his package inelegant, but it gets the job done: submit each file as a separate task, shutdown the pool (which blocks until all tasks have completed), then fetch the generated compilation unit from each task and return the list.

The reason each Parser instance needs its own TreeMaker is because TreeMaker has a stateful position variable. TreeMaker also has references to shared compiler services, but those are only used after parsing by subsequent phases. As far as I can tell, that pos variable is the only shared state used during parsing.

I wish a similar parallel-ization could be done for ClassReader and ClassWriter, but I don't think it would improve performance; ClassReader symbol completions are demand-driven to conserve memory and avoid unnecessary completions, while ClassWriter pretty much just dumps to the disk as fast as possible.

Tom
Index: JavaCompiler.java
===================================================================
--- JavaCompiler.java (revision 258)
+++ JavaCompiler.java (working copy)
@@ -62,6 +62,10 @@
// TEMP, until we have a more efficient way to save doc comment info
import com.sun.tools.javac.parser.DocCommentScanner;

+import java.util.concurrent.Callable;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
import javax.lang.model.SourceVersion;

/** This class could be the main entry point for GJC when GJC is used as a
@@ -811,8 +815,32 @@

        //parse all files
        ListBuffer<JCCompilationUnit> trees = lb();
- for (JavaFileObject fileObject : fileObjects)
- trees.append(parse(fileObject));
+ int ncores = Runtime.getRuntime().availableProcessors();
+ if (ncores > 1 && fileObjects.size() > 1) {
+ ExecutorService executor = Executors.newFixedThreadPool(ncores);
+ java.util.List<Future<JCCompilationUnit>> tasks = new java.util.ArrayList<Future<JCCompilationUnit>>();
+ for (JavaFileObject fileObject : fileObjects) {
+ final JavaFileObject jfo = fileObject;
+ tasks.add(executor.submit(new Callable<JCCompilationUnit>() {
+ @Override
+ public JCCompilationUnit call() throws Exception {
+ return parse(jfo);
+ }
+ }));
+ }
+ executor.shutdown(); // wait for all files to be parsed
+ for (Future<JCCompilationUnit> task : tasks)
+ try {
+ trees.append(task.get());
+ } catch (Exception e) {
+ log.printLines(log.noticeWriter, "parsing failed: " + e);
+ ++log.nerrors;
+ }
+ }
+ else {
+ for (JavaFileObject fileObject : fileObjects)
+ trees.append(parse(fileObject));
+ }
        return trees.toList();
    }

Index: Parser.java
===================================================================
--- Parser.java (revision 258)
+++ Parser.java (working copy)
@@ -74,7 +74,8 @@
        /** Create a new parser factory. */
        protected Factory(Context context) {
            context.put(parserFactoryKey, this);
- this.F = TreeMaker.instance(context);
+ TreeMaker treeMaker = TreeMaker.instance(context);
+ this.F = treeMaker.forToplevel(null);
            this.log = Log.instance(context);
            this.names = Name.Table.instance(context);
            this.keywords = Keywords.instance(context);