ADDITIONAL SYSTEM INFORMATION :
Mac OS X version 10.13.5 (17F77)

java version "10.0.1" 2018-04-17
Java(TM) SE Runtime Environment 18.3 (build 10.0.1+10)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.1+10, mixed mode)

Reproducible with Java 11:
java version "11-ea" 2018-09-25
Java(TM) SE Runtime Environment 18.9 (build 11-ea+20)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11-ea+20, mixed mode)

Also reproducible with multiple versions of Java 8 (1.8.0_171-b11, 1.8.0_161-b12, 1.8.0_112-b16).


A DESCRIPTION OF THE PROBLEM :
When "zoom" is turned on from the accessibility features, a FileDialog cannot be launched from a modal dialog that is hidden prior to displaying the FileDialog (even if the FD's parent is the main window).

When zoom is disabled, the dialog is displayed without error.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Enabled Zoom via System Preferences > Accessibility, then select Zoom from the left menu and turn on any keyboard shortcut or mouse gesture
2. Click "Press the button" in the main window
3. Zoom in using the keyboard/mouse shortcut set
4. In the modal dialog that appears, click "Open a FileDialog".


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
FileDialog should be displayed.
ACTUAL -
The dialog is suppressed with the following logging:

2018-07-04 09:34:33.781 java[54681:6729357] -[NSApplication runModalForWindow:] may not be invoked inside of transaction begin/commit pair, or inside of transaction commit (usually this means it was invoked inside of a view's -drawRect: method.) The modal dialog has been suppressed to avoid deadlock. (
        0 AppKit 0x00007fff4fcb6947 -[NSApplication runModalForWindow:] + 187
        1 AppKit 0x00007fff5018fa94 -[NSSavePanel runModal] + 531
        2 libawt_lwawt.dylib 0x000000012ef34c92 -[CFileDialog safeSaveOrLoad] + 359
        3 Foundation 0x00007fff546322b5 __NSThreadPerformPerform + 334
        4 CoreFoundation 0x00007fff5250ca61 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
        5 CoreFoundation 0x00007fff525c647c __CFRunLoopDoSource0 + 108
        6 CoreFoundation 0x00007fff524ef4c0 __CFRunLoopDoSources0 + 208
        7 CoreFoundation 0x00007fff524ee93d __CFRunLoopRun + 1293
        8 CoreFoundation 0x00007fff524ee1a3 CFRunLoopRunSpecific + 483
        9 Foundation 0x00007fff545eaf26 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 277
        10 libawt_lwawt.dylib 0x000000012ef56a9d Java_sun_lwawt_macosx_LWCToolkit_doAWTRunLoopImpl + 275
        11 ??? 0x0000000111e67b50 0x0 + 4595284816
)


---------- BEGIN SOURCE ----------
import javax.swing.*;
import java.awt.*;

public class Main extends JFrame {
    MyPanel panel;

    public Main() {
        setTitle("This is a frame");
        setSize(300, 200);
        panel = new MyPanel(this);
        add(panel);

        setLocationRelativeTo(null);
        setDefaultCloseOperation(EXIT_ON_CLOSE);
    }

    public static void main(String[] args) {
        EventQueue.invokeLater(() -> {
            Main frame = new Main();
            frame.pack();
            frame.setVisible(true);
        });
    }

    private static class MyPanel extends JPanel {

        int dialogCounter = 1;
        final JFrame theParent;

        public MyPanel(JFrame parent) {
            super();
            theParent = parent;
            setPreferredSize(new Dimension(300, 200));
            JButton button = new JButton("Press the button");
            button.addActionListener(e -> showDialog(theParent));

            add(button);
        }

        private void showDialog(Frame parent) {
            JDialog dialog = new JDialog(parent, "This is dialog " + dialogCounter, false);
            setupDialog(dialog);
        }

        private void setupDialog(JDialog dialog) {
            JPanel dialogPanel = new JPanel();
            dialogPanel.setPreferredSize(new Dimension(300, 200));
            dialogPanel.add(new JLabel("Current dialog count: " + dialogCounter++));
            JButton button = new JButton("Open a FileDialog");
            FileDialog fileDialog = new FileDialog(theParent, "a file dialog", FileDialog.SAVE);

            button.addActionListener(e -> {
                dialog.setVisible(false);
                fileDialog.setVisible(true);
            });
            dialogPanel.add(button);
            dialog.add(dialogPanel);
            dialog.pack();
            dialog.setModal(true);
            dialog.setVisible(true);
        }
    }
}

---------- END SOURCE ----------

FREQUENCY : always