Ensure consistent treatment of private/static methods during verification

During execution of an invokevirtual instruction, JVM implementations have always "skipped" private methods when performing method selection in the receiver's class hierarchy. This is for two reasons [1][2]. However, the JVMS7 definition of overriding used by invokevirtual allows a private method to override a non-private method such that a private method may be the result of selection; this was also the behavior specified in JVMS2. To bring the JVMS in line with implementations, the Lambda spec modifies 5.4.5 Method Overriding so that a private method is not deemed to override a non-private method.

For consistency, the type-checking verifier should deem that a private method does not override anything. Also, the type-checking verifier should deem that a static method does not override anything. This will involve short-circuiting doesNotOverrideFinalMethod:

doesNotOverrideFinalMethod(class('java/lang/Object', L), Method) :-
    isBootstrapLoader(L).

// NEW
doesNotOverrideFinalMethod(Class, Method) :-
    isPrivate(Method, Class).

// NEW
doesNotOverrideFinalMethod(Class, Method) :-
    isStatic(Method, Class).

doesNotOverrideFinalMethod(Class, Method) :-
    isNotPrivate(Method, Class), // NEW
    isNotStatic(Method, Class), // NEW
    doesNotOverrideFinalMethodOfSuperclass(Class, Method).

It is necessary to add a new intermediate clause between doesNotOverrideFinalMethod and finalMethodNotOverridden, because doesNotOverrideFinalMethod now assumes the method exists in the class whereas finalMethodNotOverridden needs to be able to recurse upwards into a class with no methods.

// NEW
doesNotOverrideFinalMethodOfSuperclass(Class, Method) :-
    classSuperClassName(Class, SuperclassName),
    classDefiningLoader(Class, L),
    loadedClass(SuperclassName, L, Superclass),
    classMethods(Superclass, MethodList),
    finalMethodNotOverridden(Method, Superclass, MethodList).

We skip over a superclass if it has a non-final method which "doesn't count" due to being private or static:

finalMethodNotOverridden(Method, Superclass, SuperMethodList) :-
    methodName(Method, Name),
    methodDescriptor(Method, Descriptor),
    member(method(_, Name, Descriptor), SuperMethodList),
    isNotFinal(Method, Superclass),
    isPrivate(Method, Superclass),
    doesNotOverrideFinalMethodOfSuperclass(Superclass, Method).

finalMethodNotOverridden(Method, Superclass, SuperMethodList) :-
    methodName(Method, Name),
    methodDescriptor(Method, Descriptor),
    member(method(_, Name, Descriptor), SuperMethodList),
    isNotFinal(Method, Superclass),
    isStatic(Method, Superclass),
    doesNotOverrideFinalMethodOfSuperclass(Superclass, Method).

Separately, if it is discovered that a (non-private, non-static) method overrides a final method, then the final method is really only overridden if it's non-static and non-private. If the final method is static or private, then it's not really overridden; per JLS7 13.4.17, it should be binary compatible to add 'final' to a static method. In fact, the 'final' modifier is an aberration; a static or private method cannot logically be overridden so the presence of 'final' is irrelevant. In Prolog terms:

finalMethodNotOverridden(Method, Superclass, SuperMethodList) :-
    methodName(Method, Name),
    methodDescriptor(Method, Descriptor),
    member(method(_, Name, Descriptor), SuperMethodList),
    isFinal(Method, Superclass),
    isPrivate(Method, Superclass).

finalMethodNotOverridden(Method, Superclass, SuperMethodList) :-
    methodName(Method, Name),
    methodDescriptor(Method, Descriptor),
    member(method(_, Name, Descriptor), SuperMethodList),
    isFinal(Method, Superclass),
    isStatic(Method, Superclass).

Finally, the "success" and "recursive" cases are similar to JLS7:

finalMethodNotOverridden(Method, Superclass, SuperMethodList) :-
    methodName(Method, Name),
    methodDescriptor(Method, Descriptor),
    member(method(_, Name, Descriptor), SuperMethodList),
    isNotFinal(Method, Superclass),
    isNotStatic(Method, Superclass), // NEW; we covered non-final static methods above
    isNotPrivate(Method, Superclass). // NEW; we covered non-final private methods above

finalMethodNotOverridden(Method, Superclass, SuperMethodList) :-
    methodName(Method, Name),
    methodDescriptor(Method, Descriptor),
    notMember(method(_, Name, Descriptor), SuperMethodList),
    doesNotOverrideFinalMethodOfSuperclass(Superclass, Method). // CHANGED

Need to add new accessors in 4.10.1.1:

isPrivate(Method, Class) - True iff Method in class Class is private.
isNotPrivate(Method, Class) - True iff Method in class Class is not private.
isStatic(Method, Class) - True iff Method in class Class is static.
isNotStatic(Method, Class) - True iff Method in class Class is not static.

[1] In the Java language, an overriding method cannot have less accessibility than an overridden method; it is a compile-time error if a private method [attempts to] override a non-private method. As such, the JVM need not be sympathetic to a class hierarchy where a private method "overrides" a non-private method. Still, it would be rather unforgiving for the JVM to give error (e.g. ICCE) upon detecting the private method, so JVM implementations simply act as if the private method didn't exist and continue climbing the class hierarchy.

[2] To the extent that private methods are legitimate in a class hierarchy, they should be invoked via invokespecial, not invokevirtual. So as in [1], it's not unreasonable for JVM implementations to act as if private methods are invisible during invokevirtual.