FULL PRODUCT VERSION :
java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Darwin acbc32c7fb93.ant.amazon.com 15.6.0 Darwin Kernel Version 15.6.0: Thu Sep 1 15:01:16 PDT 2016; root:xnu-3248.60.11~2/RELEASE_X86_64 x86_64

A DESCRIPTION OF THE PROBLEM :
Given a FI like this:

@FunctionalInterface
public static interface ITree {
Object node(List<Object> children);
}

Registered as "h" in the engine.

When executing JavaScript code like this:

h([ h([]), h([]) , h([]), h([]) ])

The engine fails with: Cannot cast jdk.nashorn.api.scripting.ScriptObjectMirror to java.util.List exception.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Run the following piece of code:

@FunctionalInterface
public static interface ITree {
Object node(List<Object> children);
}

ScriptEngine engine = new ScriptEngineManager().getEngineByName("nashorn");
engine.put("h", (ITree) c -> null);
engine.eval("h([ h([]), h([]) , h([]), h([]) ])");


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The evaluation should succeed and the resulting array correctly passed to the "node" method.
ACTUAL -
java.lang.ClassCastException: Cannot cast jdk.nashorn.api.scripting.ScriptObjectMirror to java.util.List

ERROR MESSAGES/STACK TRACES THAT OCCUR :
java.lang.ClassCastException: Cannot cast jdk.nashorn.api.scripting.ScriptObjectMirror to java.util.List
at java.lang.invoke.MethodHandleImpl.newClassCastException(MethodHandleImpl.java:361)
at java.lang.invoke.MethodHandleImpl.castReference(MethodHandleImpl.java:356)
at jdk.nashorn.internal.scripts.Script$1$\^eval\_.:scopeCall(<eval>)
at jdk.nashorn.internal.scripts.Script$1$\^eval\_.:program(<eval>:1)
at jdk.nashorn.internal.runtime.ScriptFunctionData.invoke(ScriptFunctionData.java:637)
at jdk.nashorn.internal.runtime.ScriptFunction.invoke(ScriptFunction.java:494)
at jdk.nashorn.internal.runtime.ScriptRuntime.apply(ScriptRuntime.java:393)
at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:446)
at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:403)
at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:399)
at jdk.nashorn.api.scripting.NashornScriptEngine.eval(NashornScriptEngine.java:155)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
at NashornArrayTypeCastInstabilityBugTest.nestedFunctionCallsOneElementFiveLevelsDeep(NashornArrayTypeCastInstabilityBugTest.java:49)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)



REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.util.List;

import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;
import javax.script.ScriptException;

import org.junit.Before;
import org.junit.Test;

public class NashornFunctionalInterfaceListArgumentTypeCastInstabilityBugTest {
ScriptEngine engine = new ScriptEngineManager().getEngineByName("nashorn");

@FunctionalInterface
public static interface ITree {
Object node(List<Object> children);
}

@Before
public void registerApi() {
engine.put("h", (ITree) c -> null);
}

@Test
public void nestedFunctionCallsOneElementTwoLevelsDeep() throws ScriptException {
engine.eval("h([ h([]) ])");
}

@Test
public void nestedFunctionCallsOneElementThreeLevelsDeep() throws ScriptException {
engine.eval("h([ h([ h([]) ]) ])");
}

@Test
public void nestedFunctionCallsOneElementFourLevelsDeep() throws ScriptException {
engine.eval("h([ h([ h([ h([]) ]) ]) ])");
}

// As of Java(TM) SE build 1.8.0_111-b14 this will fail with:
// java.lang.ClassCastException: Cannot cast jdk.nashorn.api.scripting.ScriptObjectMirror to java.util.List
@Test
public void nestedFunctionCallsOneElementFiveLevelsDeep() throws ScriptException {
engine.eval("h([ h([ h([ h([ h([]) ]) ]) ]) ])");
}

@Test
public void nestedFunctionCallsTwoElementsTwoLevelsDeep() throws ScriptException {
engine.eval("h([ h([]), h([]) ])");
}

// As of Java(TM) SE build 1.8.0_111-b14 this will fail with:
// java.lang.ClassCastException: Cannot cast jdk.nashorn.api.scripting.ScriptObjectMirror to java.util.List
@Test
public void nestedFunctionCallsTwoElementsThreeLevelsDeep() throws ScriptException {
engine.eval("h([ h([ h([]) ]), h([ h([]) ]) ])");
}

// As of Java(TM) SE build 1.8.0_111-b14 this will fail with:
// java.lang.ClassCastException: Cannot cast jdk.nashorn.api.scripting.ScriptObjectMirror to java.util.List
@Test
public void nestedFunctionCallsFourElementsTwoLevelsDeep() throws ScriptException {
engine.eval("h([ h([]), h([]) , h([]), h([]) ])");
}

@Test
public void nestedFunctionCallsOneElementsFiveLevelsDeepDirectMethodCall() throws ScriptException {
engine.eval("h.node([ h.node([ h.node([ h.node([ h.node([]) ]) ]) ]) ])");
}

@Test
public void nestedFunctionCallsTwoElementsThreeLevelsDeepDirectMethodCall() throws ScriptException {
engine.eval("h.node([ h.node([ h.node([]) ]), h.node([ h.node([]) ]) ])");
}
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Calling the method on the FunctionalInterface by name seems to work. e.g.

h.node([ h.node([ h.node([]) ]), h.node([ h.node([]) ]) ])