Throttling is similar in concept to the existing implementation of ShenandoahPacer. However, it is designed to have much less impact on typical workloads.

The allocation overhead of ShenandoahPacer is so high that it is impractical for many "frugal" service deployments. These services have found that occasional degenerated and even full GCs are preferable to the use of ShenandoahPacer in providing higher throughput and lower high-percentile end-to-end service latency.

The goal of ShenandoahThrottling is to avoid degenerated cycles in the rare case that a spike in workload has fooled the GC into triggering a bit late, while having no measurable overhead when GC is at pace with workload allocation requests.

A secondary goal of ShenandoahThrottling is to provide more fine-grained resolution to triggering heuristics than is practical before throttling. When throttling is required during a particuiar GC cycle, this should feed into the triggering heuristic to cause triggering to be more aggressive. Traditionally, this happens only when degenerated or full gc cycles are required. With throttling, we can provide this feedback without requiring a heavyweight degenerated cycle.

ShenandoahThrottling is not a substitute for proper provisioning of resources. If a workload consistently requires degenerated cycles, it is under provisioned. For such workloads, throttling will still degenerate.

TBD: do we want to add this to traditional Shenandoah in addition to GenShen?