In G1CollectorPolicy.cpp there is this code to determine whether to continue to put regions into the collection set:

466 if ((2.0 * _sigma) * (double) bytes_to_copy > (double) free_bytes) {
467 // end condition 3: out-of-space (conservatively!)
468 }

It happens that _sigma is 0.5 by default. So this code will take in more regions until there is really no space left, assuming that the GC can copy without any fragmentation.

That is far from conservative and a recipe for evacuation failure. Reconsider this formula, at least taking the expected waste in buffers into account (ParallelGCBufferWastePct).

Do not forget to add necessary documentation for the changes.