*****
Rationale and proposed changes for JDK-7039764
"Support expression names and type arguments
in method and constructor invocations"
*****



1) Drop NonWildTypeArguments to simplify the presentation

The nonterminal NonWildTypeArguments is introduced in 8.8.7.1 for
explicit constructor invocations and is reused in 15.12 for method
invocations. Use of the nonterminal implies it is a mere syntactic
error to pass a wildcard type argument. However, javac gives a
semantic error if a wildcard type argument is passed in either kind of
invocation. It is appropriate to add a semantic error in 8.8.7.1 ("It
is a compile-time error if any of the type arguments passed explicitly
before 'this' or 'super' are wildcard type arguments.") and weave a
similar error into the clauses in 15.12.1.

Then, there is no need for the NonWildTypeArguments
nonterminal. 8.8.7.1 and 15.12 should follow 15.9 in using
TypeArguments in the grammar, then relying on a semantic error in
normative text.

(The redundancy of NonWildTypeArguments is clear when one considers
that the JLS did not bother to introduce UnboundedWildTypeArguments
for array creation expressions and the instanceof operator, both of
which require any wildcard type arguments to be unbounded.)



2) Fix method invocations to allow expression names and type arguments

JLS7 15.12 defined a method invocation expression as follows:
(reordered for clarity)

MethodInvocation:
  MethodName ( ArgumentList_opt )
  TypeName  . NonWildTypeArguments Identifier ( ArgumentList_opt )
  Primary   . NonWildTypeArguments_opt Identifier ( ArgumentList_opt )
  super     . NonWildTypeArguments_opt Identifier ( ArgumentList_opt )
  ClassName . super . NonWildTypeArguments_opt Identifier ( ArgumentList_opt )

This allows:

  m ()                   // MethodName ...
  x.m ()                 // MethodName ...
  Foo.m ()               // MethodName ...
  Foo.<Bar>m ()          // TypeName . NonWildTypeArguments Identifier ...
  (new Foo()).<Bar>m ()  // Primary . NonWildTypeArguments_opt Identifier ...

but wrongly disallows:

  x.<Bar>m ()    // ExpressionName . NonWildTypeArguments Identifier ...

Type arguments are only possible with a TypeName or a Primary before
the explicit ".". A qualified MethodName includes its own "." but does
not allow type arguments. In my view, it is a mistake to allow a
qualified MethodName in MethodInvocation. The solution is to allow
only a simple method name in MethodInvocation, and then allow an
ExpressionName to the left of the ".", as follows:

MethodInvocation:
  SimpleMethodName ( ArgumentList_opt )
  TypeName       . TypeArguments_opt Identifier ( ArgumentList_opt )  // REFORMULATION
  ExpressionName . TypeArguments_opt Identifier ( ArgumentList_opt )  // REFORMULATION
  Primary        . TypeArguments_opt Identifier ( ArgumentList_opt )
  super          . TypeArguments_opt Identifier ( ArgumentList_opt )
  TypeName       . super . TypeArguments_opt Identifier ( ArgumentList_opt ) 

The question then arises of how we know that a.b in:

  a.b.<Bar>m ()

is an ExpressionName or a TypeName. The answer is easy: apply the
normal disambiguation rules to any name which occurs to the left of
the ".", as follows:

- Delete from 6.5.1: "A name is syntactically classified as an
  AmbiguousName to the left of the . in a qualified MethodName"

- Add to 6.5.1: "A name is syntactically classified as an
  AmbiguousName to the left of the rightmost . that occurs before the
  ( in a method invocation expression."

This classifies as ambiguous exactly the same names which were to the
left of the "." in a qualified MethodName in JLS7. For example,
consider the method invocation expression x.f.m() - JLS7 classified
x.f.m as a MethodName and x.f as an AmbiguousName to be reclassified;
JLS8 parses m as an Identifier and classifies x.f as an AmbiguousName
to be reclassified. Example:

public class MethodOrTypeName {
  <T> void m() { System.out.println("class MethodOrTypeName # m"); }
   
  public static void main(String[] args) {
    MethodOrTypeName x = new MethodOrTypeName();
    x.<String>m();
  /* Informally, local variable x obscures class x.
     Formally, x is an AmbiguousName reclassified to an ExpressionName
     by 6.5.2, where visible variables are preferred to visible classes.
     If the variable declaration is commented out, then x is
     reclassified as a TypeName and the program prints 
     "class x # m" instead of "class MethodOrTypeName # m". */
  }
}
class x {
  static <T> void m() { System.out.println("class x # m"); }
}

Further changes to "de-qualify" the method name:

- 15.12.1 and 15.12.4.1 refer to a MethodName being a qualified name
  of the form FieldName.Identifier. This becomes
  ExpressionName.Identifier.

- 15.12.1, 15.12.3, and 15.12.4.1 refer to the form of a method
  invocation being a TypeName.Identifier as distinct from a
  TypeName.NonWildTypeArguments Identifier, so we need to merge the
  handling of these forms.

- 6.2 must account for the five right-hand sides of MethodInvocation
  which have an Identifier rather than a name.

We can now drop the concept of qualified MethodNames altogether. The
only context that used them was a method invocation expression, which
now uses a simple MethodName. Their lack of utility can be seen from
the fact that 6.5.7.2 "Qualified Method Names" was mostly a simplified
form of method selection in 15.12. In more detail:

- 6.5.7.2 said that a qualified MethodName of the form
  "ExpressionName.m" meant there had to be at least one method called
  m in the type of the ExpressionName; this duplicates the search for
  potentially applicable methods in 15.12.2.

- 6.5.7.2 also said that a qualified MethodName of the form
  "TypeName.m" meant there had to be at least one static method m in
  type TypeName; this duplicates the check that the "compile-time
  declaration" is static in 15.12.3.

- Finally, 6.5.7.2 also said that "If a method name is of the form
  Q.Id, then Q has already been classified as a package name, a type
  name, or an expression name ... If Q is a package name, then a
  compile-time error occurs." If 6.5.7.2 is removed, what becomes of
  this semantic error? Let's look into how Q became a package name in
  the first place, and what similar situations exist.

  We claimed previously that reclassification of the ambiguous name in
  a MethodInvocation would always produce the same type name or
  expression name to the left of the "." as it did in SE 7. This is
  true if a type or member exists, because reclassification did and
  will continue to find it and reclassify the ambiguous name
  accordingly. What if the type or member doesn't exist?

  To answer that, we need to realize that reclassification has a
  secondary role: flagging a non-existent type or member by
  reclassifying it as a PackageName for which "a later step determines
  whether or not a package of that name actually exists." When a
  reclassification-as-PackageName happens in the overall context of a
  qualified type name, the non-existence of the package is detected
  when the type name's package name component is given meaning. When a
  reclassification-as-PackageName happens in the overall context of a
  qualified expression name, the non-existence of the package is not
  even checked; rather, the mere fact that a package is trying to
  qualify an expression is banned in 6.5.6.2 "Qualified Expression
  Names" ("If Q is a package name, then a compile-time error occurs.")

  (Technically, the clause "A later step determines whether or not a
  package of that name actually exists" which is often cited during
  reclassification is wrong, because the package-exists check is only
  made in the overall context of a qualified type name, not a
  qualified expression name.)

  The following code uses a qualified method name 'p.m' where the name
  'p' is classified as an ambiguous name then reclassified. (The fact
  that package p exists is not relevant, since only "a later step" is
  presumed to care about existence.) There is no visible type or
  member called 'p', so the name 'p' is reclassified as a package
  name. We expect a compile-time error by 6.5.7.2's requirement that Q
  is not a PackageName.

  package p;
  public class Test {
    void m() {
      p.m();
    }
  }

  Interestingly, the actual javac error does not mention package names:

  p/Test.java:4: error: cannot find symbol
    p.m();
    ^
  symbol:   variable p
  location: class Test
  1 error 

  It turns out that javac expects either a type name or an expression
  name to the left of the ".". (The error message should say "variable
  or type p" rather than "variable p".) This expectation is congruent
  with the new MethodInvocation production that allows TypeName or
  ExpressionName to the left of the ".". But note that the JLS has to
  reclassify an ambiguous name to get there, and there will be a
  serious underspecification if that gives PackageName but all the
  rules of 15.12 expect a TypeName or ExpressionName. We will have to
  acknowledge in 15.12 that the name to the left of the "." must not
  be classified as a PackageName if method invocation is to proceed.

So, to drop qualified MethodNames:

- Define MethodName in 6.5 as only an Identifier and modify the first
  RHS of MethodInvocation to use MethodName rather than
  SimpleMethodName.

- Add to 15.12: "It is a compile-time error if the name to the left of
  the leftmost "." in a MethodInvocation cannot be classified as a
  TypeName or an ExpressionName (6.5.2)."

- Delete 6.5.7.2 !

Since we are on the subject of MethodNames:

It is conceptually wrong to interpret the element name in an
annotation's element-value pair as a simple method name. The point of
a simple method name is that it appears in the scope of the method
declaration that introduced the name. The scope of a method declared
in an annotation type (i.e. an element) is the body of the type
declaration, so an annotation of that type which appears on arbitrary
program elements is not in that scope. Consequently, an element-value
pair in the annotation uses an identifier, not a simple name, to
denote an element. This is confirmed by the ElementValuePair
production in 9.7.1, which has Identifier to the left of the
"=". Therefore:

- Add to 6.2: In the list of "not a name", include the identifier that
  appears in an element-value pair of an annotation to denote an
  element.

- Delete from 6.5.1: "A name is syntactically classified as a
  MethodName: - to the left of the = sign in an annotation's element
  value pair."

- Delete from 6.5.7.1: "A simple method name may appear as the element
  name in an element-value pair. In that case, ... or a compile-time
  error occurs." This rule duplicates a rule in 9.7.1.

*****
Sidebar:
Poor handling of the "TypeName . NonWildTypeArguments Identifier" case

For a method invocation expression of the form:
  ClassName . super . NonWildTypeArguments_opt Identifier (ArgumentList_opt)

JLS7 6.5.1 classified the ClassName before the super as a
TypeName. However, 6.5.1 forgot to classify the TypeName in a method
invocation expression of the form:

  TypeName . NonWildTypeArguments Identifier (ArgumentList_opt)

For example, 6.5.1 failed to classify the Foo in Foo.<Bar>m() as a
TypeName. Happily, the new 6.5.1 clause above will catch the Foo to
the left of the ".". Foo will be classified initially as an
AmbiguousName, then reclassified as a TypeName by 6.5.2 in the normal
way (assuming Foo.<Bar>m() appears in the scope of a type declaration
called Foo, of course).

In addition, JLS7 6.2 forgot to list the Identifier after the
NonWildTypeArguments as "not a name". The change to 6.2 ensures all
Identifiers in the MethodInvocation production are accounted for as
"not a name".
*****



3) Fix explicit constructor invocations and qualified CICEs to allow
   expression names and type arguments

JLS7 8.8.7.1 defined an explicit constructor invocation statement as
follows:

ExplicitConstructorInvocation:
  NonWildTypeArguments_opt this ...
  NonWildTypeArguments_opt super ...
  Primary . NonWildTypeArguments_opt super ...

This allows:

  Primary.super()
  Primary.<Bar>super()

but wrongly disallows:

  x.super()
  x.<Bar>super()

(JLS7 18 allows x.super() but disallows x.<Bar>super() )

Similarly, JLS7 15.9.1 defined a class instance creation expression as
follows:

ClassInstanceCreationExpression:
  new TypeArguments_opt TypeDeclSpecifier ...
  Primary . new TypeArguments_opt Identifier ...

This allows:

  Primary.new Foo()
  Primary.new <Bar> Foo()

but wrongly disallows:

  x.new Foo()
  x.new <Bar> Foo()

(JLS7 18 allows both x.new Foo() and x.new <Bar> Foo() )

The solution is straightforward: allow ExpressionName as well as
Primary expressions in the 8.8.7.1 and 15.9.1 productions:

ExplicitConstructorInvocation:
  TypeArguments_opt this ...
  TypeArguments_opt super ...
  Primary . TypeArguments_opt super ... 
  ExpressionName . TypeArguments_opt super ...  // NEW

ClassInstanceCreationExpression:
  new TypeArguments_opt TypeDeclSpecifier ...
  Primary . new TypeArguments_opt Identifier ...
  ExpressionName . new TypeArguments_opt identifier ...  // NEW

Every mention of a qualified superclass constructor invocation in
8.8.7.1 must be updated so that the qualifying expression may be an
ExpressionName as well as a Primary. This also applies to every
mention of a qualified class instance creation in 15.9, 15.9.1, and
15.9.2.

*****
Sidebar: 
Poor handling of the "Primary . NonWildTypeArguments_opt super" case

Since JLS2, 6.5.1 has classified the name that represents the
qualifying expression of a qualified superclass constructor invocation
statement or qualified CICE as an ExpressionName. This is perfect for
x.super() and x.new Foo(), despite these forms being disallowed by the
grammars in 8.8.7.1 and 15.9.1! The only forms allowed by the grammars
- Primary.super() and Primary.new Foo() - do not have names in the
qualifying expression position, so are ignored by 6.5.1. In other
words, two of the classifications in 6.5.1 were no-ops.
*****