Mutex safepoint checking options and flags should be scoped enums

Making Mutex::SafepointCheckFlag and Mutex::SafepointCheckRequired be scoped enums would improve type checking in some places. For example, JDK-8255595 would have produced a compile-time error.