There are different ways to describe how many worker threads:

FlexibleWorkGang::active_workers()
AbstractWorkGang::total_workers()
ParallelGCThreads
ConcurrentMark::_active_tasks
CollectedHeap::n_par_threads()
... etc ...

Most code path use active_workers() to determine the number of worker threads to spawn for a task, but the code is sprinkled with asserts, checks and unnecessary set_par_threads calls.

We need to clean out some of this redundant code, so that we have a better understanding of how many workers threads are executing.

Some examples from the code:
---
This code path assumes that n_par_threads() and active_workers() are the same:
   bool is_par = GenCollectedHeap::heap()->n_par_threads() > 0; \
   if (is_par) { \
     assert(GenCollectedHeap::heap()->n_par_threads() == \
            GenCollectedHeap::heap()->workers()->active_workers(), "Mismatch"); \
---
Here set_par_threads() is called although n_par_threads isn't used by the task:
   G1ParNoteEndTask g1_par_note_end_task(g1h, &_cleanup_list, n_workers);
   g1h->set_par_threads((int)n_workers);
   g1h->workers()->run_task(&g1_par_note_end_task);
   g1h->set_par_threads(0);
---
This is another way G1 setups the n_par_threads. Note that the set_par_thread is a G1CollectedHeap function:
   G1StringDedupUnlinkOrOopsDoTask task(is_alive, keep_alive, allow_resize_and_rehash, phase_times);
   G1CollectedHeap* g1h = G1CollectedHeap::heap();
   g1h->set_par_threads();
   g1h->workers()->run_task(&task);
   g1h->set_par_threads(0);
---
...
   assert(n_threads > 0, "Error: expected n_threads > 0");
   assert((n_threads == 1 && ParallelGCThreads == 0) ||
          n_threads <= (int)ParallelGCThreads,
          "# worker threads != # requested!");
   assert(!Thread::current()->is_VM_thread() || (n_threads == 1), "There is only 1 VM thread");
   assert(UseDynamicNumberOfGCThreads ||
          !FLAG_IS_DEFAULT(ParallelGCThreads) ||
          n_threads == (int)ParallelGCThreads,
          "# worker threads != # requested!");
---
     // Caller (process_roots()) claims that all GC threads
     // execute this call. With UseDynamicNumberOfGCThreads now all
     // active GC threads execute this call. The number of active GC
     // threads needs to be passed to par_non_clean_card_iterate_work()
     // to get proper partitioning and termination.
     //
     // This is an example of where n_par_threads() is used instead
     // of workers()->active_workers(). n_par_threads can be set to 0 to
     // turn off parallelism. For example when this code is called as
     // part of verification during root processing then n_par_threads()
     // may have been set to 0. active_workers is not overloaded with
     // the meaning that it is a switch to disable parallelism and so keeps
     // the meaning of the number of active gc workers. If parallelism has
     // not been shut off by setting n_par_threads to 0, then n_par_threads
     // should be equal to active_workers. When a different mechanism for
     // shutting off parallelism is used, then active_workers can be used in
     // place of n_par_threads.
     int n_threads = GenCollectedHeap::heap()->n_par_threads();
     bool is_par = n_threads > 0;
     if (is_par) {
---
We need to "register" the number of threads in the SubTaskDone system.
void GenCollectedHeap::set_par_threads(uint t) {
  assert(t == 0 || !UseSerialGC, "Cannot have parallel threads");
  CollectedHeap::set_par_threads(t);
  set_n_termination(t);
}

void GenCollectedHeap::set_n_termination(uint t) {
  _process_strong_tasks->set_n_threads(t);
}
---

This RFE will serve as an umbrella CR for some of the coming cleanups.