src/java.desktop/macosx/native/libawt_lwawt/java2d/metal/EncoderManager.m line 370:

368: if (_encoder != nil) {
369: if (_destination != dest || renderOptions->isAA != _encoderStates.aa ||
370: renderOptions->isAAShader != _encoderStates.aaShader) {

We don’t need to end the encoder here - the appropriate state will be recovered later in the code (by updatePipelineState method).