Bug Description from Christopher Harrison (Oracle) email:

===========================================================
We have run in to a deadlock caused by Javasoft's fix for bug #4048664.
Here is a descriptionof the problem, in case you haven't seen this yet:

AWT-Window Thread AWT-EventQueue Thread
----------------- ---------------------
                                         calls WDialogPeer.hide()
                                           calls AwtDialog::EndModal()
                                             enters critical section
receives a WM_AWT_COMPONENT_SHOW
  calls ShowWindow()
    receives a WM_KILLFOCUS for Dialog
      calls SendFocusEvent()
        blocks to enter critical section
                                               posts WM_AWT_QUEUE_SYNC to the
                                               Window Thread and waits for it
                                               to be processed


I have stopped this deadlock from occurring by removing the critical
section from ~AwtDialog and reducing the scope of the critical section
in AwtDialog::EndModal() as shown below. Exiting the critical section
before waiting for the AWT-Window Thread to process WM_AWT_QUEUE_SYNC
prevents the deadlock.

I am not sure whether this fix is thread safe however. Is it safe to
leave this critical section before executing the rest of this function?
If not the bug fix for #4048664 will have to be modified.


AwtDialog::~AwtDialog() {
    // Oracle Bug #515756: Critical section is entered in EndModal
--> // CriticalSection::Lock l(GetLock());
    if (m_modalWnd != NULL) {
        EndModal();
    }
    if (m_disabledWnds != NULL) {
        delete[] m_disabledWnds;
    }
}

void AwtDialog::EndModal()
{
    if (AwtToolkit::GetInstance().GetModalDialog() != this) { // Fix for bug #4045610 -
Oracle problem with
                                                                 // "deadlocking" (more accurately, failing to
                                                                 // re-enable the modal dialog's parent...)
                                                                 // when closing nested dialogs (out of order).
        return;
    }
    if ((GetStyle() & WS_VISIBLE) == 0) {
        return;
    }

// Oracle Bug #515756: Release critical section after posting messages, so that we
// don't deadlock waiting for the Window Thread.
--> {
--> CriticalSection::Lock l(GetLock());

AwtToolkit::GetInstance().ResetModal(m_previousModalDialog);
Hide();

// Enable all previously disabled windows.
if (m_disabledWnds != NULL) {
for (int i = 0; i < m_nWnds; i++) {
// Post "enable" messages, since this object is locked. This is necessary because one
// of them may yank focus from this object, causing a FocusEvent to be fired -- posting
// that event requires this component's lock.
::PostMessage(m_disabledWnds[i], WM_AWT_COMPONENT_ENABLE, TRUE, 0);
}
delete[] m_disabledWnds;
m_disabledWnds = NULL;
m_nWnds = 0;
}
---> }

    // Bug #4048664 (Hide does not work on modal dialog)
    // Wait for all posted messages to be processed
    HANDLE hevt = CreateEvent(NULL, TRUE, FALSE, NULL);
    ASSERT(hevt);
    ::PostMessage(GetHWnd(), WM_AWT_QUEUE_SYNC, (WPARAM)hevt, 0);
    WaitForSingleObject(hevt, INFINITE);
    CloseHandle(hevt);

    if (m_onMainThread) {
        AwtToolkit::GetInstance().StopMessagePump();
    } else {
        Hsun_awt_windows_WDialogPeer* peer = GetPeer();
        monitorEnter(obj_monitor(peer));
        monitorNotifyAll(obj_monitor(peer));
        monitorExit(obj_monitor(peer));
    }

    if (m_previousModalDialog != NULL) {
        ::SetForegroundWindow(m_previousModalDialog->GetHWnd());
        m_previousModalDialog = NULL;
    } else {
        AwtComponent* parent = GetParent();
        if (parent != NULL) {
            ::SetForegroundWindow(parent->GetHWnd());
        }
    }

    // Bug #4048664 (Hide does not work on modal dialog)
    m_modalWnd = NULL;
    ::SetProp(GetHWnd(), "hidden", (HANDLE)1);
}