Name: rmT116609 Date: 04/01/2004


A DESCRIPTION OF THE REQUEST :
Currently, System.gc() always forces a full, stop-the-world, collection regardless of the collector policy being used.

When the concurrent mark-sweep (CMS) collector is in use, it would be better to have System.gc() trigger a concurrent full collection instead of a stop-the-world collection.

As some callers of System.gc() (e.g. NIO and RMI's DGC) rely on System.gc() only returning after the collection is completed, System.gc() would still need to block until the collection completes; the change is that when CMS is in use, only the calling thread would be blocked during the concurrent phases of GC, not all threads.

JUSTIFICATION :
The CMS collector is used mostly for applications where GC pauses must be kept low -- i.e. latency-sensitive applications. A stop-the-world full collection causes an unacceptably large pause for these applications -- part of tuning the CMS collector for an application involves making sure a stop-the-world full collection never occurs in normal operation.

The current System.gc() implementation always forces a full collection. The only way to avoid this currently is to pass -XX:+DisableExplicitGC to turn System.gc() into a no-op. Avoiding calls to System.gc() entirely is not possible as some calls are made from within the standard Java libraries (e.g. NIO and RMI DGC).

However, completely disabling System.gc() in this way does not work well either, as callers of System.gc() generally do so for a reason -- e.g. NIO calls it to reclaim direct buffers when it runs out of direct buffer space, DGC calls it to get a more accurate view of live RMI references.

The attached testcase shows some of the problems with NIO and -XX:+DisableExplicitGC. Without -XX:+DisableExplicitGC, full stop-the-world GCs are forced. With it, NIO buffer allocation fails.

Note that the attached testcase exposes another NIO-related bug (submitted with review ID 233528) which can cause spurious OutOfMemoryErrors even when System.gc() is enabled.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
-XX:+DisableExplicitGC should not be needed to avoid stop-the-world collections. The testcase should run with concurrent collections only.
ACTUAL -
With System.gc() enabled:

oliver@flood:~/nio-bugs$ java -verbose:gc -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:MaxDirectMemorySize=1M NIOAllocate 1000 10000
Allocating 10000 NIO Direct buffers of 1000 bytes each.
0 buffers allocated.
1001 buffers allocated.
[GC 246K->207K(16320K), 0.0162750 secs]
[Full GC 207K->206K(16320K), 0.0348480 secs]
Caught OOME allocating buffer #1049
2002 buffers allocated.
[GC 330K->329K(16320K), 0.0063360 secs]
[Full GC 329K->206K(16320K), 0.0292560 secs]
Caught OOME allocating buffer #2097
3003 buffers allocated.
[GC 329K->329K(16320K), 0.0050900 secs]
[Full GC 329K->206K(16320K), 0.0291510 secs]
Caught OOME allocating buffer #3145
4004 buffers allocated.
[GC 329K->329K(16320K), 0.0053420 secs]
[Full GC 329K->206K(16320K), 0.0297570 secs]
Caught OOME allocating buffer #4193
5005 buffers allocated.
[GC 329K->329K(16320K), 0.0050510 secs]
[Full GC 329K->206K(16320K), 0.0291440 secs]
Caught OOME allocating buffer #5241
6006 buffers allocated.
[GC 329K->329K(16320K), 0.0053410 secs]
[Full GC 329K->206K(16320K), 0.0288150 secs]
Caught OOME allocating buffer #6289
7007 buffers allocated.
[GC 329K->329K(16320K), 0.0059880 secs]
[Full GC 329K->206K(16320K), 0.0287610 secs]
Caught OOME allocating buffer #7337
8008 buffers allocated.
[GC 329K->329K(16320K), 0.0051440 secs]
[Full GC 329K->206K(16320K), 0.0293400 secs]
Caught OOME allocating buffer #8385
9009 buffers allocated.
[GC 329K->329K(16320K), 0.0051870 secs]
[Full GC 329K->206K(16320K), 0.0287630 secs]
Caught OOME allocating buffer #9433
Done.

With System.gc() disabled:

oliver@flood:~/nio-bugs$ java -verbose:gc -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:MaxDirectMemorySize=1M -XX:+DisableExplicitGC NIOAllocate 1000 10000
Allocating 10000 NIO Direct buffers of 1000 bytes each.
0 buffers allocated.
1001 buffers allocated.
Caught OOME allocating buffer #1049
Caught OOME allocating buffer #1049
Caught OOME allocating buffer #1049
Caught OOME allocating buffer #1049
Caught OOME allocating buffer #1049
Caught OOME allocating buffer #1049
Caught OOME allocating buffer #1049
Caught OOME allocating buffer #1049
Caught OOME allocating buffer #1049
Caught OOME allocating buffer #1049
Caught OOME allocating buffer #1049
Caught OOME allocating buffer #1049
Caught OOME allocating buffer #1049
Caught OOME allocating buffer #1049
Caught OOME allocating buffer #1049
Caught OOME allocating buffer #1049
[.. continues indefinitely ..]


---------- BEGIN SOURCE ----------
public class NIOAllocate {
    public static void main(String[] args) throws Exception {
        if (args.length < 2) {
            System.err.println("syntax: java NIOAllocate <buffer size> <buffer count>");
            return;
        }

        int bufferSize = Integer.parseInt(args[0]);
        int bufferCount = Integer.parseInt(args[1]);
        int progressSize = bufferCount / 10 + 1;
        
        System.err.println("Allocating " + bufferCount + " NIO Direct buffers of " + bufferSize + " bytes each.");

        for (int i = 0; i < bufferCount; ++i) {
            if (i % progressSize == 0)
                System.err.println(i + " buffers allocated.");
            
            try {
                java.nio.ByteBuffer testBuffer = java.nio.ByteBuffer.allocateDirect(bufferSize);
            } catch (OutOfMemoryError oome) {
                System.err.println("Caught OOME allocating buffer #" + (i+1));
                Thread.sleep(500);
                --i; // Try again.
            }
        }

        System.err.println("Done.");
    }
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
None. Currently we must avoid using any code that requires calls to System.gc() (e.g. NIO direct buffers) to operate correctly, and run with +XX:+DisableExplicitGC.
(Incident Review ID: 233534)
======================================================================