Clang provides some options to investigate compilation time. See "shenandoahOldGC.json" for an example.

ClangBuildAnalyzer allows aggregating this data and extracting human-readable information from it. I attached an example output to this ticket (clang_build_analyzer.txt).

In JDK-8365053 I investigated the "Expensive headers" section. Other sections may be interesting to explore too to reduce build time.

**** Templates that took longest to instantiate:
 45007 ms: AccessInternal::OopLoadProxy<oopDesc *, 524288>::operator oopDesc * (1232 times, avg 36 ms)
 44433 ms: AccessInternal::load<524292UL, oopDesc *, oopDesc *> (1232 times, avg 36 ms)

**** Template sets that took longest to instantiate:
371271 ms: AccessInternal::BarrierResolver<$>::resolve_barrier (24728 times, avg 15 ms)
370278 ms: AccessInternal::BarrierResolver<$>::resolve_barrier_rt (24728 times, avg 14 ms)

**** Functions that took longest to compile:
  2550 ms: State::MachNodeGenerator(int) (/jdk/build/trace/hotspot/variant-server/gensrc/adfiles/ad_x86_gen.cpp)
  2539 ms: CompilerToVM::initialize_intrinsics(JVMCIEnv*) (/jdk/src/hotspot/share/jvmci/jvmciCompilerToVMInit.cpp)

References:
- https://clang.llvm.org/docs/analyzer/developer-docs/PerformanceInvestigation.html
- https://github.com/aras-p/ClangBuildAnalyzer