Every time I open Java from System Preferences it shows "Java Control Panel" dialog on the screen and immediately moves it to the bottom left corner, no matter where I leave it before.

That wasn't the case with Java 7.