Rebalancing Reference lists costs a lot of time because it is single threaded; the main issue is because the list of discovered references is a single linked list per discovery thread per reference type that need to be walked through one by one.

Build an index containing locations within the discovered lists at fixed intervals to speed up rebalancing the discovered lists.

This needs to be done either during discovery (if precleaning is disabled) or during precleaning, and potentially rebuilt every time elements are removed.