Summary

In the body of a basic for loop, allow lambda expressions to reference the loop variable as long as it is effectively final within the body of the loop, similar to enhanced for loops.

Goals

• Eliminate the difference in treatment between basic for and enhanced for statements with respect to when loop variables may be referenced by lambdas, nested classes, and switch case guards in the body of the loop.

• Do not change the rules for variables that are not basic for loop variables.

• Do not change the rules for basic for loop variables outside the for loop body.

• Do not change the rules for final and "effectively final" variables.

• Do not change the rules for definite assignment.

Motivation

Java allows lambdas, nested classes, and switch case guards to reference variables that are declared in a containing scope. In the example below, the Runnable is allowed to access the variable whom, even though it may not be invoked until some arbitrary later time:

String whom = "world";
Runnable r = () -> System.out.println("Hello " + whom);

While this capability is very useful, Java requires that such variables must either be explicitly declared final or else be "effectively final", which means they are only assigned a value once (and therefore could just as well have been declared final).

The purpose of this rule is to eliminate any ambiguity about the variable's possible value. Without this rule, it would be unclear whether and how the modification of a variable in one scope might affect that same variable's value in a different scope, as these scopes can execute at different times.

The following example, which is not legal Java, demonstrates this ambiguity:

String today = "Tuesday";
Runnable r = () -> {
    System.out.println("Today is " + today);
    today = "Friday";
};
today = "Thursday";
r.run();                                  // what would this print?
System.out.println("Today is " + today);  // what would this print?

By requiring a variable to be final or effectively final if referenced in a lambda, Java ensures that the variable can only ever have one value. This eliminates any possible confusion because the variable must have the same value no matter where, or when, it is used.

Loop Variables in for Loops

However, certain language constructs eliminate any ambiguity about a variable's possible value by their very design.

For example, consider code like this:

List<Runnable> toDoList = ...
for (int i = 1; i <= 3; i++)
    toDoList.add(() -> System.out.println("Do item #" + i));    // error!

This example is currently not valid Java, because the variable i is neither final nor effectively final, yet there is no ambiguity about the intent. In general, when a for loop variable is referenced from a lambda expression in the body of the loop, the intent is to "capture" the current value of the variable for that particular iteration of the loop, as computed by the loop header. This allows the lambda expression, whenever it is invoked, to execute in the context of that particular iteration. As long as the loop variable is final or effectively final within the body of the loop, there is no ambiguity about its current value in each iteration.

On the other hand, if the loop variable were to be modifed within the body of the loop, that would reintroduce the ambiguity that the current rules are designed to avoid.

Therefore, because of the way they are structured, when for loop bodies contain lambdas that reference a loop variable, what's important is not that the variable be final or effectively final, but that it be final or effectively final within the body of the loop.

Enhanced for Loops

In fact, this is already how Java handles lambdas inside enhanced for statement bodies:

List<Runnable> toDoList = ...
for (int i : new int[] { 1, 2, 3 })
    toDoList.add(() -> System.out.println("Do item #" + i));    // allowed

In the example above, even though i is assigned a new value for each element in the array, it is still allowed to be referenced by the lambda in the loop body because it is effectively final in that context.

Of course, if the loop variable is not effectively final in the loop body, then it may no longer be referenced by a lambda:

List<Runnable> toDoList = ...
for (int i : new int[] { 1, 2, 3 }) {
    toDoList.add(() -> System.out.println("Do item #" + i)); // error!
    i++;   // "i" is mutated here
}

In practice, for both basic for loops and enhanced for loops, it is relatively uncommon for loop variables to be modified in the body of the loop. With enhanced for loops, lambdas commonly take advantage of this and reference the loop variable.

The Java language specifies that the rules for enhanced for statements are equivalent to how the normal rules would apply if the loop were rewritten to declare a "copy variable" at the start of the body of the loop:

for (int i : new int[] { 1, 2, 3 }) {
    int i_copy = i;
    toDoList.add(() -> System.out.println("Do item #" + i_copy));
}

As long as the copy variable would remain effectively final, the the original variable can be referenced by a lambda. Requiring the copy variable to be effectively final is equivalent to requiring that the original loop variable be effectively final in the body of the loop.

Basic for Loops

The basic for loop was included in the original Java language, before lambda expressions were added. Partly as a result of this historical quirk, basic for loops were never granted the same ability for lambdas to reference loop variables in the loop body.

As a result, the following code using a basic for statement does not compile, instead reporting a "local variables referenced from a lambda expression must be final or effectively final" error:

List<Runnable> toDoList = ...
for (int i = 1; i <= 3; i++)
    toDoList.add(() -> System.out.println("Do item #" + i));    // error!

In this situation, the standard workaround is to manually create a copy variable:

List<Runnable> toDoList = ...
for (int i = 1; i <= 3; i++) {
    int i_copy = i;
    toDoList.add(() -> System.out.println("Do item #" + i_copy));   // ok!
}

Note how this workaround matches what the Java language specifies for enhanced for loop variables.

Even though basic for loops were added to the language before lambdas, the rationale for allowing lambdas to reference loop variables in enhanced for loops applies just as well to basic for loops. In other words, as long as the loop variable is effectively final in the body of the loop, a lambda should be allowed to reference it.

For loop variables that are effectively final in the body of the loop, this discrepancy between basic for and enhanced for seems unnecessary. Developers should be relieved of having to manually create copy variables which clutter their code.

Description

The rule for when a lambda inside a basic for loop body is allowed to reference a loop variable will be changed to permit referencing loop variables which are effectively final in the body of the loop.

This will allow this earlier example to compile successfully:

List<Runnable> toDoList = ...
for (int i = 1; i <= 3; i++)
    toDoList.add(() -> System.out.println("Do item #" + i));     // ok!

This change aligns the behavior of basic for loop variables to be consistent with enhanced for loop variables, and obviates the need for copy variables.

More precisely, the requirement allowing a variable to be referenced inside the body of a for loop will be augmented to admit, in addition to final and effectively final variables, variables for which the following are both true:

• The variable is declared in the initialization section of the for loop
• The variable is effectively final in the body of the for loop

where a variable is defined as effectively final in the body of the loop if the following are both true in the body of the loop:

• Wherever the variable occurs as the left hand side in an assignment expression, it is definitely unassigned
• The variable never occurs as the operand of a prefix or postfix increment or decrement operator

Note that the above conditions are necessary, but not sufficient, to allow a variable to be captured; captured variables still must be definitely assigned, for example.

These changes are intended to be the minimum required to achieve parity with the enhanced for statement. As such, they do not change how loop variables are treated in the initialization, condition, and step sections of the basic for loop, or to change how definite assignment analysis is performed anywhere. So for example, a lambda that captures a mutating loop variable in the condition or the step would still fail to compile.

Other Loops

A natural question to ask is, "What about while and do loops?" For example, should this code be allowed?

List<Runnable> toDoList = ...
int i = 0;
while (++i <= 3)
    toDoList.add(() -> System.out.println("Do item #" + i));    // ok??

The problem with while and do loops is that they don't explicitly define "loop variables" in a well-defined scope that is limited to the statement itself. As a result, it wouldn't always be well-defined which variables are the "loop variables". Even if they were identified, they could be mutated both before and after the loop, which would create additional ambiguity. So unlike the basic for statement, while and do statements are not clear candidates for this change.

Testing

We will test the compiler changes with existing unit tests, unchanged except for those tests that verify changed behavior, plus new positive and negative test cases as appropriate.

We will compile all JDK classes using the previous and new versions of the compiler and verify that the resulting bytecode is identical.

No platform-specific testing should be required.

Risks and Assumptions

These changes are source and behavioral compatible. They strictly expand the set of legal Java programs; the specified behavior of all existing programs remains unchanged.