CodeCache is dynamic. Code comes in and out. Code can be used most of the time. Code can be used periodically. Code can be used some time and never used after. Frequently called code can be spread across CodeCache and become sparse. From performance analysis of applications affected by sparse code we know sparse code mostly affects prediction of calls. CPU branch predictor might be less effective in case of sparse code. On Graviton systems code sparsity can be measured: https://github.com/aws/aws-graviton-getting-started/blob/main/perfrunbook/debug_hw_perf.md#drill-down-front-end-stalls.

We spend 80-90% in C2 nmethods. Most of frequently called Java methods will be C2 nmethods. At high level the process is:
- Detect frequently called C2 nmethods.
- Identify candidates for relocation to reduce sparsity.
- Relocate them close to each other.

## Detection of frequently called C2 nmethods

C2 nmethods don't collect profiles. We add a sampling-based profiler that runs in a non-Java thread and periodically collects PC samples. We start profiling when the amount of C2 code exceeds a threshold. The threshold should be CPU specific because different CPUs tolerate different degree of CodeCache sparsity. We record nmethods owning those PC samples. Nmethods can be purged during profiling. We track those nmethods to remove them from a set of recorded nmethods. We stop profiling when we collect enough samples to identify hot methods. A sampling period is chosen randomly from the range 5 - 15 ms for better accuracy. We estimate time needed to collect required samples. Not collecting enough samples withing the time is a sign of low application activity. We stop profiling and wait for an application to become more active. We track how many new C2 nmethods have been registered. The number of new C2 nmethods exceeding a threshold signals an increased application activity. This triggers reprofiling.

## Relocation of frequently called C2 nmethods

Currently we can relocate a nmethod to a new place only through deoptimization and recompilation. Recompilations, especially C2, are expensive. Having many of them can cause performance issues. Also recompiled code might differ from the originally compiled code.

Instead of recompilations we need copy like recolations where we copy a nmethod to a new place and relink its code.

We need to group nmethods somewhere. We can group them in the same code heap which they belong to. Or we can use a separate code heap. Support of a separate code heap is simpler to implement. From public benchmark and real-life applications we estimate that nmethods to group will be ~10%-20% of all nmethods. So a separate code heap won't need a lot of memory.

We can consider not to relocate some nmethods if they are already close to each other.

## Removal of nmethod from a group

Currently GC is responsible for flushing nmethods from CodeCache.
We can either of:
- Mark nmethods needed to be removed as cold and rely on GC to flush them.
- Exclude the special code from GC flushing. Copy nmethods back to a general code heap.