The current implementation of jdk.nashorn.internal.runtime.linker.JavaAdapterFactory is written in such a way that it can lead to Metaspace OOM errors.

Each time a new script is loaded by the Nashorn engine, a cache of <CodeSource, StaticClass> key-value pairs is built up. It is stored in the following Map [1]:

private final Map<CodeSource, StaticClass> instanceAdapters = new ConcurrentHashMap<>();

This may lead to Metaspace OOM errors as this caching behaviour is tightly coupled to class loading.

This occurs as often the Metaspace is significantly smaller than the regular Java heap.

A good solution to this would be to enforce an upper limit on how large that cache can get, as instanceAdapter classes can be regenerated when needed

[1] https://github.com/openjdk/jdk11u-dev/blob/master/src/jdk.scripting.nashorn/share/classes/jdk/nashorn/internal/runtime/linker/JavaAdapterFactory.java