ADDITIONAL SYSTEM INFORMATION :
java version "10.0.1" 2018-04-17
Java(TM) SE Runtime Environment 18.3 (build 10.0.1+10)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.1+10, mixed mode)
System: Mac OS X 10.13.4
uname -a: Darwin Jessicas-MacBook-Pro.local 17.5.0 Darwin Kernel Version 17.5.0: Fri Apr 13 19:32:32 PDT 2018; root:xnu-4570.51.2~1/RELEASE_X86_64 x86_64
Java 11 EA information:
java version "11-ea" 2018-09-25
Java(TM) SE Runtime Environment 18.9 (build 11-ea+13)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11-ea+13, mixed mode)
A DESCRIPTION OF THE PROBLEM :
When running with -XX:+UseG1GC and using Runtime.freeMemory() to estimate how memory is available, I sometimes run into OutOfMemoryError even though it appears there's enough memory left for what the object I'm trying to allocate -- this is not true when running with -XX:+UseConcMarkSweepGC, where the reported value from Runtime.freeMemory() gets relatively close to the size I'm trying to allocate before OOME is thrown. But ConcMarkSweepGC has been deprecated since Java 9, which concerns me.
When running the attached code with -XX:+UseConcMarkSweepGC, it finishes with the output:
Finished when availableMemory was 129,240 and is now 18,576. Free=18,576. Allocated blocks=421,216
With -XX:+UseG1GC, it finishes with:
Got out of memory trying to allocate memory when availableMemory was 28,715,936 and is now 28,233,984. Free=28,233,984. Allocated blocks=421,792
I've tried using Java 11 as well, with the same result.
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Run the attached code with -XX:+UseG1GC
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Runtime.freeMemory() should report somewhere in the vicinity of 10,000 bytes remaining when I run into OOME allocating a 10,000 byte array.
ACTUAL -
With G1GC, Runtime.freeMemory() reports 28,715,936 bytes -- several orders of magnitude more than I'm trying to allocate!
---------- BEGIN SOURCE ----------
package com.me;
import java.util.ArrayList;
import java.util.List;
public class GCTest {
public static void main(String[] args) {
int initialCapacity = 10_000_000;
List<Object> stuff = new ArrayList<Object>(initialCapacity);
while (true) {
long availableMemory = getAvailableMemory();
try {
for (int i = 0; i < 100; i++) {
stuff.add(new byte[10_000]);
}
} catch (OutOfMemoryError e) {
long availableNow = getAvailableMemory();
long freeMemory = Runtime.getRuntime().freeMemory();
int blocksAllocated = stuff.size();
stuff = null; // So that we GC now and the following methods don't throw OutOfMemoryError
if (availableMemory > 2_000_000) {
System.err.printf("Got out of memory trying to allocate memory when availableMemory was %,d and is now %,d. Free=%,d. Allocated blocks=%,d\n", availableMemory, availableNow, freeMemory, blocksAllocated);
break;
}
System.out.printf("Finished when availableMemory was %,d and is now %,d. Free=%,d. Allocated blocks=%,d\n", availableMemory, availableNow, freeMemory, blocksAllocated);
break;
}
long usedMemory = getUsedMemory();
availableMemory = getAvailableMemory();
if (availableMemory > 1_500_000) // Don't do this when low on memory so that the println doesn't throw OutOfMemoryError
System.out.printf("Used=%,d available=%,d of %,d size=%,d max=%,d total=%,d free=%,d\n", usedMemory, availableMemory, usedMemory + availableMemory, stuff.size(), Runtime.getRuntime().maxMemory(), Runtime.getRuntime().totalMemory(), Runtime.getRuntime().freeMemory());
if (stuff.size() > initialCapacity) {
System.err.printf("initial capacity should have been higher\n");
break;
}
}
}
private static long getAvailableMemory() {
return Runtime.getRuntime().maxMemory() - getUsedMemory();
}
private static long getUsedMemory() {
Runtime runtime = Runtime.getRuntime();
return runtime.totalMemory() - runtime.freeMemory();
}
}
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
Use deprecated ConcMarkSweepGC
FREQUENCY : always
java version "10.0.1" 2018-04-17
Java(TM) SE Runtime Environment 18.3 (build 10.0.1+10)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.1+10, mixed mode)
System: Mac OS X 10.13.4
uname -a: Darwin Jessicas-MacBook-Pro.local 17.5.0 Darwin Kernel Version 17.5.0: Fri Apr 13 19:32:32 PDT 2018; root:xnu-4570.51.2~1/RELEASE_X86_64 x86_64
Java 11 EA information:
java version "11-ea" 2018-09-25
Java(TM) SE Runtime Environment 18.9 (build 11-ea+13)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11-ea+13, mixed mode)
A DESCRIPTION OF THE PROBLEM :
When running with -XX:+UseG1GC and using Runtime.freeMemory() to estimate how memory is available, I sometimes run into OutOfMemoryError even though it appears there's enough memory left for what the object I'm trying to allocate -- this is not true when running with -XX:+UseConcMarkSweepGC, where the reported value from Runtime.freeMemory() gets relatively close to the size I'm trying to allocate before OOME is thrown. But ConcMarkSweepGC has been deprecated since Java 9, which concerns me.
When running the attached code with -XX:+UseConcMarkSweepGC, it finishes with the output:
Finished when availableMemory was 129,240 and is now 18,576. Free=18,576. Allocated blocks=421,216
With -XX:+UseG1GC, it finishes with:
Got out of memory trying to allocate memory when availableMemory was 28,715,936 and is now 28,233,984. Free=28,233,984. Allocated blocks=421,792
I've tried using Java 11 as well, with the same result.
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Run the attached code with -XX:+UseG1GC
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Runtime.freeMemory() should report somewhere in the vicinity of 10,000 bytes remaining when I run into OOME allocating a 10,000 byte array.
ACTUAL -
With G1GC, Runtime.freeMemory() reports 28,715,936 bytes -- several orders of magnitude more than I'm trying to allocate!
---------- BEGIN SOURCE ----------
package com.me;
import java.util.ArrayList;
import java.util.List;
public class GCTest {
public static void main(String[] args) {
int initialCapacity = 10_000_000;
List<Object> stuff = new ArrayList<Object>(initialCapacity);
while (true) {
long availableMemory = getAvailableMemory();
try {
for (int i = 0; i < 100; i++) {
stuff.add(new byte[10_000]);
}
} catch (OutOfMemoryError e) {
long availableNow = getAvailableMemory();
long freeMemory = Runtime.getRuntime().freeMemory();
int blocksAllocated = stuff.size();
stuff = null; // So that we GC now and the following methods don't throw OutOfMemoryError
if (availableMemory > 2_000_000) {
System.err.printf("Got out of memory trying to allocate memory when availableMemory was %,d and is now %,d. Free=%,d. Allocated blocks=%,d\n", availableMemory, availableNow, freeMemory, blocksAllocated);
break;
}
System.out.printf("Finished when availableMemory was %,d and is now %,d. Free=%,d. Allocated blocks=%,d\n", availableMemory, availableNow, freeMemory, blocksAllocated);
break;
}
long usedMemory = getUsedMemory();
availableMemory = getAvailableMemory();
if (availableMemory > 1_500_000) // Don't do this when low on memory so that the println doesn't throw OutOfMemoryError
System.out.printf("Used=%,d available=%,d of %,d size=%,d max=%,d total=%,d free=%,d\n", usedMemory, availableMemory, usedMemory + availableMemory, stuff.size(), Runtime.getRuntime().maxMemory(), Runtime.getRuntime().totalMemory(), Runtime.getRuntime().freeMemory());
if (stuff.size() > initialCapacity) {
System.err.printf("initial capacity should have been higher\n");
break;
}
}
}
private static long getAvailableMemory() {
return Runtime.getRuntime().maxMemory() - getUsedMemory();
}
private static long getUsedMemory() {
Runtime runtime = Runtime.getRuntime();
return runtime.totalMemory() - runtime.freeMemory();
}
}
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
Use deprecated ConcMarkSweepGC
FREQUENCY : always