While investigating a minor performance regression in a System.gc() microbenchmark I saw that the Concurrent Remap Roots phase took a significant portion of the GC cycle time.

It turns out that even though we use constexpr we don't get the inlining of the the functions as we expected and this results in a noticeable performance loss.

When running System.gc() with on old GC worker thread with a 16TB max heap size we spend around 300 ms just iterating over the page table and its indices via ZIndexDistributorClaimTree. If the inlining is fixed this drops down to ~70 ms, which is similar to what we see on Linux and MacOS.

We already have a patch out to remove the last usage of the ZIndexDistributor mechanism in https://github.com/openjdk/jdk/pull/25345, but ZIndexDistributor has less restrictions and is easier to use so we might want to keep it around (with fixed performance) so that it can be used for prototyping and maybe future features.