This is an old bug that was uncovered as a result of a change that went into
CR 6948538. The specific change was that since the product build used
-XX:-BlockOffsetArrayUseUnallocatedBlock, the fastdebug builds were also
changed (from trueInDebug to false) so that the same code paths were
exercised in both, so we were testing the product bits. This revealed
a long-standing bug in the code that tripped an assertion that was a tad
too strong. Note that the bug itself exists in the product builds preceding
6948538; it was just that the bug was not revealed until the recent change in the
setting of the flag for debug builds.

More specifically, the CMS sweep is limited to the high water mark (if any)
of allocation at the point at which the concurrent marking cycle was started.
When we do not use +BlockOffsetArrayUseUnallocatedBlock, this is the "end"
of the committed space for a CMS generation. If the CMS generation is not
fully committed at this point and is expanded during the sweep, then the
newly expanded portion is coalesced with the previously co-terminal chunk
in order to limit fragmentation. That means that a previously recorded
"limit" (being the address of the previous "end") is no longer a block
boundary and may not be encountered by the sweep, which may skip over that
specific address. This fell afoul of the code and assertion in question,
which is reproduced below:-

     7936 size_t SweepClosure::do_blk_careful(HeapWord* addr) {

     7937 FreeChunk* fc = (FreeChunk*)addr;

     7938 size_t res;

     7939

     7940 // check if we are done sweepinrg

     7941 if (addr == _limit) { // we have swept up to the limit, do nothing more

     7942 assert(_limit >= _sp->bottom() && _limit <= _sp->end(),

     7943 "sweep _limit out of bounds");

     7944 // help the closure application finish

     7945 return pointer_delta(_sp->end(), _limit);

     7946 }

     7947 assert(addr <= _limit, "sweep invariant");


The fix is to relax the code to allow a "skipping over" the limit,
rather than "stepping on the limit", see suggested fix section.