FULL PRODUCT VERSION :
java version "1.6.0"
Java(TM) SE Runtime Environment (build 1.6.0-b105)
Java HotSpot(TM) Client VM (build 1.6.0-b105, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows XP Professional Version 2002 Service Pack 2

A DESCRIPTION OF THE PROBLEM :
The documentation in Dialog states for setVisible(false): "hides the Dialog and then causes setVisible(true) to return if it is currently blocked". But with the document-modality it only returns immediately if no other modal dialog was opened later. The UI isn't blocked, but the code after setVisible(true) is blocked till all later opened modal dialogs are hidden. Although they use a different document and the modal dialogs are document-modal. It's only reproducable if the setVisible(true) code of the document-modal dialog ist in the EventDispatcherThread. If the code is in another thread it works.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
The Example opens two frames. Each frame has a button to open a document-modal dialog. After returning from the modal dialog, the code prints text on the console. If you open the first dialog before the second dialog and you close the second dialog before the first dialog all works fine. But if you close the dialogs in the reverse order (first dialog before second dialog), the print code of the first dialog will executet soon after the second dialog is closed.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The expected result is that the code after setVisible(true) will executed immediately after the dialog is hidden.
ACTUAL -
The actual result is that the code after setVisible(true) will executed after all later opened dialog are hidden.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.awt.*;
import java.awt.Dialog.ModalityType;
import java.awt.event.*;

public class ModalityTest
{

   Frame f1, f2;
   Dialog d1, d2;
   Button b1, b2;

   public ModalityTest()
   {

      f1 = new Frame("Test Frame1");

      d1 = new Dialog(f1, "Modal Dialog", true);
      d1.setModalityType(ModalityType.DOCUMENT_MODAL);
      d1.setSize(150, 100);
      d1.setLocation(225, 400);
      d1.addWindowListener(new WindowAdapter()
      {
         public void windowClosing(WindowEvent we)
         {
            d1.setVisible(false);
         }
      });

      f2 = new Frame("Test Frame2");

      d2 = new Dialog(f2, "Modal Dialog", true);
      d2.setModalityType(ModalityType.DOCUMENT_MODAL);
      d2.setSize(150, 100);
      d2.setLocation(500, 400);
      d2.addWindowListener(new WindowAdapter()
      {
         public void windowClosing(WindowEvent we)
         {
            d2.setVisible(false);
         }
      });

      b1 = new Button("Show Modal Dialog 1");
      b1.addActionListener(new ActionListener()
      {
         public void actionPerformed(ActionEvent e)
         {
            d1.setVisible(true);
            System.out.println("d1 finished");
         }
      });
      f1.add(b1);
      f1.setSize(175, 100);
      f1.setLocation(0, 400);

      b2 = new Button("Show Modal Dialog 2");
      b2.addMouseListener(new MouseAdapter()
      {
         public void mouseClicked(MouseEvent e)
         {
            d2.setVisible(true);
            System.out.println("d2 finished");
         }
      });
      f2.add(b2);
      f2.setSize(175, 100);
      f2.setLocation(400, 400);

      f2.setVisible(true);
      f1.setVisible(true);

   }// ModalDialogTest()

   public static void main(String args[])
   {
      new ModalityTest();
   }
}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
In some cases it is a workaround to set the dialog visible in another thread. But if your method runs in the EventDispatcherThread and you want to return after the dialog is closed this workaround doesn't work.

new Thread(new Runnable()
            {

               public void run()
               {
                  d1.setVisible(true);
                  SwingUtilities.invokeLater(new Runnable()
                  {
                     public void run()
                     {
                        System.out.println("d1 finished");
                     }
                  });
               }

            }).start();