FULL PRODUCT VERSION :
Java version "1.8.0_25" (x86) onwards. Also reprodeuced with latest release of "1.8.0_91" (x86).


FULL OS VERSION :
Windows 7 x64 and x86

A DESCRIPTION OF THE PROBLEM :
If a create a JVM through the invocation interface using
JVM.DLL -> JNI_CreateJavaVM and the -Xmx parameter
specifies a heap larger than the VM can allocate, the VM
prints a message to stderr and *completely exits the
process*.

I am using the invocation interface to create a VM in-
process of another application. The application is not
JAVA.EXE but a GUI application. The message to stderr is
never seen, and the app disappears w/ no error or acces
violation messages of any kind.

As per JNI (invocation interface) documentation the CreateJavaVM call should return an error code of -4. Which is the behavior wit Java version 1.7.0_705 (x86).

From your bug database I found a similar issue reported against Java 1.4 and was fixed for Java 1.5. So it seems to have been re-introduced in Java 1.8. Here is the link to that Bug report : https://bugs.openjdk.java.net/browse/JDK-4898900

THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: Did not try

REGRESSION. Last worked in version 7u76

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Create a GUI app that creates its own VM via the
invocation interface and passes a very large value as an -
Xmx parameter (like 1800M).
2. Watch the app disappear at runtime w/ no error message.
Running from the command line also results in no error output since the app is not a console app. The only way to capture the error message is to redirect the app's output
to a file (either from the command line or programmatically) and view the output file after the app disappears.

In Java 1.7 createJavaVM() would return with an error code of -4 which would allow us to reset the Xmx value to 256 and get the application started.


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED VERSUS ACTUAL BEHAVIOR :
I would expect the JNI_CreateJavaVM to return a failure code of -4 as documented in JNI error codes and allow my application to continue, handling the error gracefully if possible (for instance, if the VM is only needed for one feature that can be disabled at runtime
if the VM
REPRODUCIBILITY :
This bug can be reproduced always.

CUSTOMER SUBMITTED WORKAROUND :
Not found any.