Invalid critical JNI function lookup

On 10/10/2016 10:34 AM, Ioannis Tsakpinis wrote:
> This patch fixes the lookup of critical JNI functions on Windows x86.
>
> There are two problems with the argument size calculation in the
> lookup_critical_entry function:
>
> 1) Critical natives do not have a JNIEnv parameter. Critical natives are
> always static, but do not have a jclass parameter. The current code assumes
> that both parameters exist and counts them against the total argument size.
>
> 2) For each Java array parameter, the critical native gets an additional
> length parameter for that array. The current code does not count them.
>
> On the 32-bit VM, the argument size is used to apply stdcall decorations to
> the function name. A wrong size is calculated with the current code, so the
> name used for the lookup is invalid (unless the function happens to have
> exactly two array parameters).
>
> diff -r fec31089c2ef src/share/vm/prims/nativeLookup.cpp
> --- a/src/share/vm/prims/nativeLookup.cpp Thu Oct 06 18:05:53 2016 -0700
> +++ b/src/share/vm/prims/nativeLookup.cpp Sun Oct 09 22:44:54 2016 +0300
> @@ -293,10 +293,12 @@
> char* critical_name = critical_jni_name(method);
>
> // Compute argument size
> - int args_size = 1 // JNIEnv
> - + (method->is_static() ? 1 : 0) // class for static methods
> - + method->size_of_parameters(); // actual parameters
> -
> + int args_size = method->size_of_parameters(); // actual parameters
> + for (SignatureStream ss(signature); !ss.at_return_type(); ss.next()) {
> + if (ss.is_array()) {
> + args_size += T_INT_size; // array length parameter
> + }
> + }
>
> // 1) Try JNI short style
> entry = lookup_critical_style(method, critical_name, "",
> args_size, true);
>
> In steps 3 and 4 the function lookup is done without a prefix/suffix, so a
> workaround is available. On msvc JNI functions can be exported without
> decorations, but it's not without pain: it requires pragmas or a .DEF file.
>
> Regards,
> Ioannis
>