To be able to complete the GC Interface work for nmethods described in JDK-8220343, I need to get rid of the usage of scavenge_root_nmethods_do from VM_HeapWalkOperation::collect_simple_roots.

VM_HeapWalkOperation::collect_simple_roots visits strong like our class unloading GCs, but in addition it also visits nmethods with pointer into the young gen, but only for Serial, Parallel, and CMS, but not for G1.

Set of roots:
  JNIHandles::oops_do(&blk);
  SystemDictionary::oops_do(&blk);
  CLDToOopClosure cld_closure(&blk, false);
  ClassLoaderDataGraph::always_strong_cld_do(&cld_closure);
  ObjectSynchronizer::oops_do(&blk);
  Universe::oops_do(&blk);
  CodeCache::scavenge_root_nmethods_do(&look_in_blobs);

If we compare this to the set of roots in PSMarkSweep:
  Universe::oops_do(mark_and_push_closure());
  JNIHandles::oops_do(mark_and_push_closure());
  ObjectSynchronizer::oops_do(mark_and_push_closure());
  Management::oops_do(mark_and_push_closure());
  JvmtiExport::oops_do(mark_and_push_closure());
  SystemDictionary::oops_do(mark_and_push_closure());
  ClassLoaderDataGraph::always_strong_cld_do(...);
  // Do not treat nmethods as strong roots for mark/sweep, since we can unload them.
  //CodeCache::scavenge_root_nmethods_do(...);
  AOTLoader::oops_do(mark_and_push_closure());

Comparing these two lists, its also apparent that the HeapWalk code is missing some roots, but that needs to be dealt with in a separate RFE.

For this RFE the interesting part is that the VM_HeapWalkOperation iterates over the scavenge_root_nmethods_do, even though our Serial collector (or other class unloading GCs) does not.

I propose that we remove this call.

This change will Effect the following JVMTI functionality:
 IterateOverReachableObjects
 IterateOverObjectsReachableFromObject
 FollowReferences