FULL PRODUCT VERSION :
java version "1.7.0-ea"
Java(TM) SE Runtime Environment (build 1.7.0-ea-b23)
Java HotSpot(TM) Server VM (build 12.0-b01, mixed mode)


ADDITIONAL OS VERSION INFORMATION :
Linux localhost.localdomain 2.6.20.1 #3 SMP Sun Mar 25 22:45:33 CEST 2007 i686 i686 i386 GNU/Linux


A DESCRIPTION OF THE PROBLEM :
Direct-ByteBuffers use sun.misc.Cleaner to get notified when they have been GCed and java.nio.Bits.reserveMemory() to trigger a GC run when too many dead but uncollected references to DirectByteBuffer-Memory is laying on the heap.

The test-code below stresses the following problem:
The object which references the Direct-ByteBuffer has a finalzer, therefor the DirectByteBuffer cannot be reclamed with only a single GC run. So reserveMemory only triggers a single GC which is enough to discover that the anoynmous instance of runnable can be collected - however the buffer still can be referenced inside of the finalizer - so the GC cannot dereference it.

I would be happy if a fix for the problem would be accepted via OpenJDK, if so I've a working prototype.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Let the test below run for a small amount of time.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
No Exceptions thrown
ACTUAL -
java.lang.OutOfMemoryError: Direct buffer memory

ERROR MESSAGES/STACK TRACES THAT OCCUR :
java.lang.OutOfMemoryError: Direct buffer memory

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------

public static int n=0; //Just global to avoid hotspot optimizations
public static void main(String[] args) throws Exception {
while(true) {
Runnable r = new Runnable() {
ByteBuffer buffer = null;

public void run() {
buffer = ByteBuffer.allocateDirect(32768);
}

public void finalize() {
n++;
}
};
r.run();
}
}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Almost not possible in application code, because you can never know wether some legacy-code uses finalizers (even jdk7's source has plenty of them) and has a reference to the objects created by you.