On RHEL 4.6 64-bit 4G mem, two instances of jakarta-jmeter with 6000 threads worked fine on 6u3 64-bit.
But after upgrading to 6u6, java processes soon cause memory thrashing and become very slow.

By monitoring virtual and resident size of java processes, 6u4 and later use larger memory. In a case,
6u7 uses 600MB or larger C heap.

By checking malloc related path on gdb, it turned out the most memory allocations come from
ObjectMonitor allocation.

On 6u4, per-thread ObjectMonitor freelist was introdued (_omFreeList). In some test, I saw more than
800 ObjectMonitors in average on 6u7. ObjectMonitor size is 168 bytes on 64-bit JVM for Linux.

6000*800*168=806400000

6u3 can run the same app with 100m to 200m C heap, but 6u4 and later require much larger C heap.

C heap usage levels off at some point, but we cannot know how much memory is really needed to
a specific run of application. Currently there is no clear limit for the ObjectMonitors on
omFreeList.

To suppress the excessive C heap use, we need a switch to turn off the per-thread ObjectMonitor
free list or a good mechanism to control the memory usage (Or simply a better implementation).