In jni_util.c, InitializeEncoding takes the sun_jnu_encoding string, typically containing the system locale's encoding, and uses NewStringUTF and NewGlobalRef to create a Java String object with the encoding's name and assign a handle to it to the global variable jnuEncoding. The variable is then used in functions such as JNU_NewStringPlatform and JNU_GetStringPlatformChars for Java calls.

Instead of storing a String object, it should look up the appropriate Charset object right away and store a handle to that, which would avoid a Charset lookup by name on each invocation, although there are fast fully-native code paths for common encodings. InitializeEncoding already calls Charset.isSupported which does such a lookup.

Besides an optimization, this would be an important change for GraalVM Native Image. We support multiple VM instances ("isolates") in one process at the same time (e.g. through calling JNI_CreateJavaVM several times) which share the data section, but not the Java object heap. Therefore, the handle in jnuEncoding refers to a String object in the first created VM instance which cannot be accessed by other VM instances. Charset objects, however, are preexisting objects in the native image, and if InitializeEncoding looked up and stored a handle to the Charset object, that handle could be used across VM instances. It is not our goal to support different encodings for different VM instances, only to be able to safely use the system locale's encoding in multiple VM instances.

I am happy to come up with and test a patch.