-
Enhancement
-
Resolution: Fixed
-
P3
-
9
-
b33
-
generic
-
generic
A DESCRIPTION OF THE REQUEST :
If a "for" loop iterates over a large list/array it keeps a temporary (implicit auto-generated) reference to the memory, and this memory might be locked for garbage collecting in the next steps which causes OutOfMemoryError.
The problem might happen if one uses:
- a large list or arrays which allocates, for example, more than half of available heap space
- a new list/array should be allocated immediately after the "for" loop and the old one expected to be discarded
JUSTIFICATION :
As soon as explicit reference to any (large) memory block is released it should be available for collecting as a garbage, otherwise an application may fall to OutOfMemoryError.
I found this example when i needed to process two large XML files: after parsing the first one i fell to OutOfMemoryError when i tried to parse the second one, and i discarded on declared references to the previously parsed XML file. Note, if i avoid "for" loop and use explicit "Iterator-while-hasNext-next-iterator=null" construction - the application works fine.
In the examples below i simplified the issue demonstration.
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
No OutOfMemoryError when i don't have any explicit references to allocated memory.
As soon as "for" block is finished the platform (not sure javac or JVM) must release those auto-generated implicit reference and make the memory block available for garbage collecting.
ACTUAL -
The program falls to OutOfMemoryError.
The program keeps it's own implicit auto-generated reference to the memory block till next implicit usage. If one tries to allocate a memory before that "next implicit usage" - OutOfMemoryError occurs.
---------- BEGIN SOURCE ----------
/**
* See more examples here: https://bitbucket.org/snippets/radistao/geerj
* <p>
* Run with:
* <pre>javac IteratorInOneScope.java && java IteratorInOneScope</pre>
*
* and try two cases:<ul>
* <li>with the <b>for</b> loop: falls to <i>java.lang.OutOfMemoryError: Java heap space</i></li>
* <li>without <b>for</b> loop: completes successfully</li>
* </ul>
* <p>
* <b>Conclusion</b>: <b>for</b> loop created automatically a (temporary) reference to a memory block,
* but has not released it after usage before the next memory allocation.
*/
public class IteratorInOneScope {
private static final int HALF_OF_MEMORY = (int) (Runtime.getRuntime().maxMemory() * 0.5);
public static void main(String[] args) {
byte[] data = new byte[HALF_OF_MEMORY];
for (byte b : data); // <-- if you comment this line - the application finished successfully
data = null; // this expects to discard reference -> allow to release the memory
byte[] data2 = new byte[HALF_OF_MEMORY]; // the memory can't be allocated second time, if the "for" loop statement above is used
System.out.println("Success");
}
}
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
See more examples and workarounds in this snippet:
https://bitbucket.org/snippets/radistao/geerj
1. If you avoid those "for" loops - it works fine (no OutOfMemoryError)
2. If you use explicit code block with final nullify, like
Iterator itr = list.iterator();
while (itr.hasNext()) {
Object t = itr.next();
}
itr = null; // <-- saves our souls
3. If you make intermediate operations between the "for" loop and the next memory allocation - it may work fine, but guess there is not guarantee that exactly the same reference will be reused in those intermediate operation (see example https://bitbucket.org/snippets/radistao/geerj#file-IteratorInOtherScope.java where "System.gc()" or "double d = Math.random()" is used before second allocation)
4. The first open investigation started here
http://stackoverflow.com/questions/42403347/java-for-statement-implementation-prevents-garbage-collecting
as the explantation why those implicit reference occur.
5. Tested on versions from 1.8.0_111 and 1.8.0_121, macos and windows, + 3 online java compilers (like this one: https://ideone.com/GJ1qoI )
If a "for" loop iterates over a large list/array it keeps a temporary (implicit auto-generated) reference to the memory, and this memory might be locked for garbage collecting in the next steps which causes OutOfMemoryError.
The problem might happen if one uses:
- a large list or arrays which allocates, for example, more than half of available heap space
- a new list/array should be allocated immediately after the "for" loop and the old one expected to be discarded
JUSTIFICATION :
As soon as explicit reference to any (large) memory block is released it should be available for collecting as a garbage, otherwise an application may fall to OutOfMemoryError.
I found this example when i needed to process two large XML files: after parsing the first one i fell to OutOfMemoryError when i tried to parse the second one, and i discarded on declared references to the previously parsed XML file. Note, if i avoid "for" loop and use explicit "Iterator-while-hasNext-next-iterator=null" construction - the application works fine.
In the examples below i simplified the issue demonstration.
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
No OutOfMemoryError when i don't have any explicit references to allocated memory.
As soon as "for" block is finished the platform (not sure javac or JVM) must release those auto-generated implicit reference and make the memory block available for garbage collecting.
ACTUAL -
The program falls to OutOfMemoryError.
The program keeps it's own implicit auto-generated reference to the memory block till next implicit usage. If one tries to allocate a memory before that "next implicit usage" - OutOfMemoryError occurs.
---------- BEGIN SOURCE ----------
/**
* See more examples here: https://bitbucket.org/snippets/radistao/geerj
* <p>
* Run with:
* <pre>javac IteratorInOneScope.java && java IteratorInOneScope</pre>
*
* and try two cases:<ul>
* <li>with the <b>for</b> loop: falls to <i>java.lang.OutOfMemoryError: Java heap space</i></li>
* <li>without <b>for</b> loop: completes successfully</li>
* </ul>
* <p>
* <b>Conclusion</b>: <b>for</b> loop created automatically a (temporary) reference to a memory block,
* but has not released it after usage before the next memory allocation.
*/
public class IteratorInOneScope {
private static final int HALF_OF_MEMORY = (int) (Runtime.getRuntime().maxMemory() * 0.5);
public static void main(String[] args) {
byte[] data = new byte[HALF_OF_MEMORY];
for (byte b : data); // <-- if you comment this line - the application finished successfully
data = null; // this expects to discard reference -> allow to release the memory
byte[] data2 = new byte[HALF_OF_MEMORY]; // the memory can't be allocated second time, if the "for" loop statement above is used
System.out.println("Success");
}
}
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
See more examples and workarounds in this snippet:
https://bitbucket.org/snippets/radistao/geerj
1. If you avoid those "for" loops - it works fine (no OutOfMemoryError)
2. If you use explicit code block with final nullify, like
Iterator itr = list.iterator();
while (itr.hasNext()) {
Object t = itr.next();
}
itr = null; // <-- saves our souls
3. If you make intermediate operations between the "for" loop and the next memory allocation - it may work fine, but guess there is not guarantee that exactly the same reference will be reused in those intermediate operation (see example https://bitbucket.org/snippets/radistao/geerj#file-IteratorInOtherScope.java where "System.gc()" or "double d = Math.random()" is used before second allocation)
4. The first open investigation started here
http://stackoverflow.com/questions/42403347/java-for-statement-implementation-prevents-garbage-collecting
as the explantation why those implicit reference occur.
5. Tested on versions from 1.8.0_111 and 1.8.0_121, macos and windows, + 3 online java compilers (like this one: https://ideone.com/GJ1qoI )
- relates to
-
JDK-8194836 delta apply changesets for JDK-8192885 and JDK-8175883
-
- Resolved
-
-
JDK-8192885 Compiler in JDK 10-ea+33 misses to include entry in LineNumberTable for goto instruction of foreach loop
-
- Closed
-