Name: rmT116609 Date: 04/29/2003


FULL PRODUCT VERSION :
java version "1.4.1_01"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.1_01-b01)
Java HotSpot(TM) Client VM (build 1.4.1_01-b01, mixed mode)

java version "1.4.2-beta"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2-beta-b19)
Java HotSpot(TM) Client VM (build 1.4.2-beta-b19, mixed mode)

FULL OPERATING SYSTEM VERSION :
Microsoft Windows 2000 [Version 5.00.2195]


A DESCRIPTION OF THE PROBLEM :
The HotSpot VM sometimes throws an OutOfMemoryError even
though there is plenty of reclaimable garbage present. It
appears that an allocation failure on an application
request does trigger a full garbage collection, but an
OutOfMemoryError may be thrown regardless of the results of
that GC.

In the process of debugging an out-of-memory exit in a
complex application, I wrote a simple test program that
allocates a series of 4KB blocks (byte arrays). When the
amount of free space on the heap gets small, say <1MB free,
the program begins dropping its references to some of the
blocks & continues to allocate new blocks. Depending on
various heap option settings, an OutOfMemory exception may
be thrown. An example of the output in one of these cases
follows:

C:\heaptest>java -server -verbose:gc -XX:NewRatio=4 -x100m -
Xms100m HeapTest

requested allocation size = 4096 actual size = 4112 bytes
[Full GC 446K->330K(100352K), 0.0104416 secs]
--- starting allocations -------
[GC 16714K->16714K(100352K), 0.0470250 secs]
[GC 33097K->33097K(100352K), 0.0497818 secs]
[GC 49481K->49481K(100352K), 0.0480004 secs]
[GC 65865K->65865K(100352K), 0.0489293 secs]
[Full GC 82249K->82249K(100352K), 0.0753784 secs]
24908: may throw OutOfMemory. free heap is 4632
[Full GC 100348K->98307K(100352K), 0.0306476 secs]
[Full GC 98307K->98307K(100352K), 0.1124120 secs]

24908: *** Out of Memory Exception caught *** (expected
2032K garbage)
 free heap is 2092448 bytes
 total blocks allocated = 24399

The line beginning 24908: is logged indicating that the
free heap space (as returned by Runtime.freeMemory()) is
low enough that the next 4KB allocation attempt may fail.
However, at this point there are roughly 2MB of previously
allocated but now unreferenced blocks in the heap. The next
line shows a full GC, presumably caused by the allocation
failure.
  [Full GC 100348K->98307K(100352K), 0.0306476 secs]
Note that this full GC reclaimed around 2MB, yet the
OutOfMemoryError exception is still thrown. Furthermore,
when the exception is caught, freeMemory() shows that 2MB
are indeed available. Also, immediately retrying the same
allocation now succeeds!

This behavior is so unexpected that I do not see how it
could be considered anything but a bug. I realize that the
VM spec is vague on this subject, but the relevant sentence
is:

"If a computation requires more heap than can be made
available by the automatic storage management system, the
Java virtual machine throws an OutOfMemoryError."

Clearly in the example above, the system COULD have made
sufficient heap available had it tried to do so
synchronously.

Lest you think this is a contrived example, the original
problem causing our application to crash exhibited similar
symptoms. We were able to prevent most of the application
errors by periodically forcing a full GC via System.gc(),
however this is not recommended & should not be necessary.




STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. compile the attached HeapTest.java class
2.java -server -verbose:gc -XX:NewRatio=3 -Xmx100m -Xms100m HeapTest
(different options may or may not reproduce the problem)

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
class HeapTest
{
    static final private int KB = 1024;
    static final private int MB = 1024*1024;

    public static void main( String[] args )
    {
        Runtime rt = Runtime.getRuntime();

        java.util.Vector v = new java.util.Vector(50000);


        int MAXITER = 2000000;
        int BLOCKSIZE = 4*KB;
        int actBlockSize = BLOCKSIZE;

        // determine actual space allocated per block including overhead
        long preFree = rt.freeMemory();
        byte[] ba = new byte[BLOCKSIZE];
        long curFree = rt.freeMemory();
        ba = null;
        actBlockSize = (int)(preFree-curFree);
        System.out.println( "requested allocation size = " + BLOCKSIZE
                                + " actual size = " + actBlockSize + "
bytes" );

        System.gc();
        System.out.println( "--- starting allocations -------" );

        int gCount = 0;

        for( int i=1; i < MAXITER; i++ ) {
            preFree = rt.freeMemory();

            if( preFree < 2*BLOCKSIZE ) {
                System.out.println( i + ": may cause allocation failure. free
heap is " + preFree );
             // System.out.println( i + ": forcing GC" );
             // System.gc();
            }

            try {
                ba = new byte[BLOCKSIZE];
                v.add( ba );
                ba = null;
            }
            catch( java.lang.OutOfMemoryError x ) {
                System.out.println( "" );
                System.out.println( i + ": *** Out of Memory Exception caught
*** (expected " + (gCount*BLOCKSIZE/KB) + "K garbage)" );
                System.out.println( " free heap is now " + rt.freeMemory() + "
bytes" );
                System.out.println( " total blocks allocated = " + v.size() );
                System.out.println( "" );
            }

            curFree = rt.freeMemory();

            long freed = (curFree - preFree) + actBlockSize;
            if( freed > 0 ) {
                System.out.println( i + ": GC freed " + freed/KB + "K"
                    + " (" + (gCount*actBlockSize/KB) + "K expected)" );
                gCount = 0;
            }

            // drop references to the most recently allocated 2 blocks if free
memory is tight
            if( rt.freeMemory() < 1*MB && v.size() > 2 ) {
                v.removeElementAt( v.size()-1 );
                v.removeElementAt( v.size()-1 );
                gCount += 2;
            }
        }
    }

}
---------- END SOURCE ----------
(Review ID: 181286)
======================================================================