FULL PRODUCT VERSION :
All versions (e.g. 6u12)

FULL OS VERSION :
RHEL 4

A DESCRIPTION OF THE PROBLEM :
In the file growableArray.cpp there is a signed integer overflow bug that
causes fatal JVM errors for some large heap cases.

In function "void *GenericGrowableArray::raw_allocate(int elementSize)" the
integer "elementSize" is multiplied by the value of field "_max". Both of these
are signed 32-bit integers. If elementSize=8 then any value over about 250
million for _max will result in signed integer overflow. We see this during
Full GC as an attempt to malloc -1.6 GBytes of memory which of course fails and
results in a "GrET in ... Out of swap?" message.

The fix is simply to cast _max to (size_t) before the multiplication by
elementSize. E.g. "AllocateHeap((size_t)_max*elementSize)".

This problem manifests in any jvm that does a GC on a heap with more than
200 million or so objects in which the identity hash code has been calculated.
The growable array instance that crashes is the overflow mark stack used during
Full GC.

You can trivially replicate the crash with about 10 GBytes of heap by simply
creating a few hundred million instances of Object (calling
System.identityHashcode() on each object) and then forcing a Full GC.

THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: Yes

THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: Yes

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
javac Mark.java (see code below)

java -Xmx10g -Xms10g -verbose:gc Mark 250000000

EXPECTED VERSUS ACTUAL BEHAVIOR :
Should run
ERROR MESSAGES/STACK TRACES THAT OCCUR :
# An unexpected error has been detected by Java Runtime Environment:
#
# java.lang.OutOfMemoryError: requested -1610612736 bytes for GrET in
/BUILD_AREA/jdk6_04/hotspot/src/share/vm/utilities/growableArray.cpp. Out of
swap space?
#
# Internal Error (allocation.inline.hpp:42), pid=32167, tid=1095194944
# Error: GrET in
/BUILD_AREA/jdk6_04/hotspot/src/share/vm/utilities/growableArray.cpp
#
# Java VM: Java HotSpot(TM) 64-Bit Server VM (10.0-b19 mixed mode linux-amd64)
# An error report file with more information is saved as:
# /...
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#
Aborted


REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
public class Mark {
        public static void main(String[] args) throws Exception {
                int arrSz = Integer.parseInt(args[0]);
                Object[] arr = new Object[arrSz];

                for (int i=0; i<arrSz; i++) {
                        arr[i] = new Object();
                        System.identityHashCode(arr[i]);
                }

                while (true) {
                        System.gc();
                }
        }
}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Can't be worked around