G1RemSetScanState is a helper class containing too many nested classes.

Improve readability by refactoring this code.

The original comment from the review for JDK-8213108 is:

" 82 class G1RemSetScanState : public CHeapObj<mtGC> {

I have a preference for non-trivial nested classes being declared in
the owning class but defined outside it. Similarly for non-trivial
functions. I find the style used here pretty hard to read, having
trouble finding the APIs for classes and the boundaries between
functions and between classes. (I've seen code editors that can help with that, but not everyone uses such.)

There are certainly lots of non-trivial classes in hotspot whose
entire definition is nested in the class definition. (Not that I'm a
fan of such.) But this seems kind of excessive. The class definition is nearly 500 lines long, with 7(?) nested classes. (I was almost surprised not to find any multiply nested classes; unless I missed them.)

(And I *do* like nested classes as a scoping mechanism, just not laid out like this.)"