In G1CMRemarkTask::work() there is this code:

if (worker_id < _cm->active_tasks()) {
  // do remark
}

and we start the G1CMRemarkTask with all available gc worker threads. This means that basically 3/4 of threads are started to do nothing.

Just start the G1CMRemarkTask with the appropriate amount of threads in the first place.