I could reproduce this issue on multiple machines. I guess this not machine specific issue.

I could reprodcue the same issue with application version of FIU dialog a well.

How to reprodcue this issue:

1.) Install 6u10 and onwards release of JRE (Base JRE)
2.) Launch IE Browser which loads some applet. (http://www.babynamewizard.com/voyager).
3.) Invoke the installer (6u18). PIT build b03
4.) Navigate through the installer.
5.) Verify the following in File in Use dialog.
    * Verify File in Use dialog showing "Windows Internet Explorer" only.
    * Verify that on clicking cancel button pops up another message window with (OK and Cancel button)
    * Verify on clicking cancel in pop up window control comes back to File In Use dialog.

Expected behavior:
Clicking on cancel button in pop up window, control should come back to File In Use dialog.

Actual Behavior:
Installer is exited