Create a rule that triggers if the time to get to safe point is too high compared to what is actually going on in the safepoint.
Compare SafepointBegin duration with SafepointEnd.startTime - SafepointBegin.endTime

Also the actual SafepointBegin could probably have a limit where the rule starts to warn.
One thing to note about this is that SafepointEnd is not enabled by default in default or profile settings.

From discussion with dsimms