The JVM attempts to place the class space at a location usable for unscaled or zero-based encoding.

It does so by:
1. attempting to piggyback on Java heap placement
2. alternatively, attempt to map at HeapBaseMinAddress
3. (ppc and aarch64 only) attempt to map within the lower 32 GB with a "ladder approach"

This approach has disadvantages:

- We don't try particularly hard. (1) relies on the heap using CompressedOops *and* being reserved in low address ranges *and* the range above the heap being free. (2) depends on HeapBaseMinAddress being free. (3) is restricted to two platforms.

- Only (2) has even a chance to get us into unscaled (zero base zero shift) territory. (1) and (3) only give us zero-based non-zero-shift encoding.

- HeapBaseMinAddress is an odd choice. It is 2GB on all platforms, and that denies us half of the valuable low address range below 4G right away, further reducing the chance of ever going unscaled.

- It is not well randomized: (1) and (2) are not at all, (3) is somewhat randomized since JDK-8296565 but with a value range of only 64.

- Importantly, we only try for zero-based encoding if CDS=off or CDS=dump. For CDS runtime, atm we need to set the encoding base to the start of the archive, and therefore will never run zero-based. The reason is that if the archive contains heap objects, those will come with pre-computed narrow Klass IDs, which will be based on the start address of the archive. At runtime, the archive can be placed anywhere (and since JDK-8296565, that placement is randomized), but then the encoding base has to point to the mapping start.

- It reduces the chance of getting a zero-based java heap - this is because when attempting to place the heap, we leave a gap for the future class space. But a zero-based Java heap is more valuable than a zero-based class space - it makes sense to optimize for the former and take the latter if possible.

- It assumes narrow Klass encoding uses the same geometry (bit size, shift) as narrow Oops, which is not guaranteed with future developments (lilliput).

- It introduces an unnecessary dependency between heap reservation, narrow Oop encoding, and class space reservation. That makes the code base brittle.


Proposal for a better solution:

Add a new mem reservation API that takes:
- an address range [min, max)
- an alignment
- byte size

Let that API reserve within the given address range with the given alignment. Let it be smart and quick about it (e.g., balancing randomization with attempting to reduce fragmentation). This API is a building block that can be used for all kinds of "in address range" reservations while automatically providing enough randomization to be safe. Use that API for reserving class space and CDS regions. In future RFEs, this can also be used for similar allocations, e.g., java heap or Shenandoah collection sets.

This proposal goes alongside JDK-8310823 - one plan is to re-calculate the narrow Klass IDs for materialized heap objects at CDS runtime. That will allow us to use zero-based encoding also at CDS runtime, which would be great since it affects the majority of use cases.