A significant part of remembered set clearing is spent on clearing the coarse remembered set.

One way to avoid this work most of the time is to only clear the coarse bitmaps if they have actually been dirtied.