Investigate TaskTerminator improvements

During review of JDK-8245721 several suggestions to improve the task termination came up that we considered out of scope for that (refactoring) change:

- always use notify_all() and not bother with potentially time-consuming gathering of the number of tasks:

"How bad would it be to not bother collecting the number of tasks and just have prepare_for_return unconditionally notify_all?

Alternatively, if that's bad and the cost of tasks_in_queue_set() is
also a problem (which I can well believe), maybe the taskqueue
facility can have some specialized helper function added for use here.
All we really want to know in the current code is how many tasks are
available up to the number of workers, or more than the number of
workers and we don't need a precise answer."

- getting the number of tasks could potentially be sped up (if desirable) by the fact that we have a good estimate on which threads are still working - the ones that have not offered termination. Threads not in there can by definition not have work left to do, and since we use stealing only, other threads are not actively pushing into different thread's queues.
Also the current task getting the task count is guaranteed to not contain any elements.
I.e. it is not required to check all task queues, just the active ones.

- "Another idea would be to record the task count in the terminator,
along with a last update epoch counter. (Both are only touched under the lock, so no lock-free synchronization to worry about.) Use the recorded count if it's epoch is more recent than the locally captured epoch from the most recent local use."

- one could optimize the notify_all calls after determining that we need to exit: the first one noticing that we should terminate, sets a flag that he's notified everyone. The others would then not do the notify_all call any more.

- "One question:
 197 } else {
 198 size_t tasks = tasks_in_queue_set();
 199 if (exit_termination(tasks, terminator)) {
 200 prepare_for_return(the_thread, tasks);
 201 _offered_termination--;
 202 return false;
 203 }
 204 }
Do we actually need this clause? This is the wait-timed-out case. If there is an active spin-master, then it seems like we should just go back to sleep. If there isn't an active spin-master, we could just take that over. Either way, dropping this clause and just looping would work. And this is the only place where we count tasks under the lock."

This would have the drawback that if that thread obtained the spin master, would do an extra unlock/lock in case we are actually done.

https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-June/030185.html
https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-July/030326.html