Illegal -Xmx argument, when specified and passed s runtime argument, would fail to launch the subordinate JVM rather than simply ignore the argument.
Try to load the following html, if applets fails tolaod then bug is reproduced
http://sqeweb.sfbay/deployment2/jitu/deployment_mustang_exec_ws/deployment/src/plugin/runtimeArgs/runtimeArgs/html/secure/TestUntrustedSecureInvalid.html