FULL PRODUCT VERSION :
Every one since 1.4 - including 1.6
ADDITIONAL OS VERSION INFORMATION :
All of them!
A DESCRIPTION OF THE PROBLEM :
When using ByteBuffers et al, there needs to be a way to release the underlying buffer rather than wait for garbage collection to kick in.
Such buffers may, by their very definition (and the JavaDocs own usage recommendations) be very large. These behemoths are 'fronted' by relatively tiny Java classes. The GC may not kick in in any meaningful time-frame. There are already close() (or similar) methods on classes like FileReader, since it is standard and good practice to release operating system resources as soon as possible.
These memory blocks should be considered the same - significant and limited resources.
If you don't want to change the API because it will break existing applications, the easy solution is to create a subclass of Buffer called "FreeableBuffer" or similar with a free() method (or similar) and then make ByteBuffer et al a subclass of FreeableBuffer.
Here's a sample program and output from two different versions of the JDK. This was tested on Solaris 8 in 32-bit mode using JDK 1.4.-b21 and JDK 1.5.0-b64
The program is slightly contrived, but the crux is the previous ByteBuffer(s) have (in effect) been released but in the case of 1.4 GC never kicks in and it barfs.
In the case of 1.5, it's better, GC kicks in, but according to the output
of top (not included) the JVM grew to ~1281M before GC kicked in.
If I could do a bb.release() - or similar - then those resources i.e. the
big memory buffer could be free()-ed, cached and re-used or whatever.
1.4 Output:
# java -Xmx64m -verbose:gc BB
Done 1
Done 2
Done 3
Done 4
Done 5
Done 6
Done 7
Done 8
Done 9
Done 10
Done 11
Done 12
Exception in thread "main" java.lang.OutOfMemoryError
at sun.misc.Unsafe.allocateMemory(Native Method)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:57)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:283)
at BB.main(BB.java:17)
1.5 Output:
# N.B. I had to specify a larger heap size for this test otherwise it ran out of memory immediately.
# java -Xmx1536m -verbose:gc BB
Done 1
Done 2
Done 3
Done 4
[GC 147K->144K(4992K), 0.0149122 secs]
[Full GC 144K->97K(4992K), 0.0202993 secs]
Done 5
Done 6
Done 7
Done 8
[GC 158K->129K(4992K), 0.0024465 secs]
[Full GC 129K->97K(4992K), 0.0260971 secs]
Done 9
Done 10
Done 11
Done 12
[GC 159K->161K(4992K), 0.0008160 secs]
[Full GC 161K->98K(4992K), 0.0166681 secs]
Done 13
Done 14
Done 15
Done 16
[GC 159K->130K(4992K), 0.0011929 secs]
[Full GC 130K->97K(4992K), 0.0278316 secs]
Done 17
Done 18
Done 19
Done 20
[GC 158K->161K(4992K), 0.0020273 secs]
[Full GC 161K->97K(4992K), 0.0181044 secs]
Done 21
Done 22
Done 23
Done 24
[GC 159K->145K(4992K), 0.0041246 secs]
[Full GC 145K->98K(4992K), 0.0210584 secs]
Done 25
Done 26
Done 27
Done 28
[GC 159K->146K(4992K), 0.0058417 secs]
[Full GC 146K->98K(4992K), 0.0204990 secs]
Done 29
Done 30
...<SNIP>... etc...
---------- BEGIN SOURCE ----------
import java.nio.ByteBuffer;
public class BB
{
public static void main(String[] args)
{
// May make these configurable using -Dxxx
final int MAX_BUFFS = 1000;
final int BUFSIZE = 300 * 1024 * 1024;
final int PUT = 1000 * 1000;
final int REPORT = 1;
final boolean store = false;
for(int i = 0; i < MAX_BUFFS; i++)
{
ByteBuffer bb = ByteBuffer.allocateDirect(BUFSIZE);
if (i % REPORT == 0) System.err.println("Done " + (i + 1));
}
}
}
---------- END SOURCE ----------
REPRODUCIBILITY :
This bug can be reproduced always.
###@###.### 2005-07-13 12:21:24 GMT
Every one since 1.4 - including 1.6
ADDITIONAL OS VERSION INFORMATION :
All of them!
A DESCRIPTION OF THE PROBLEM :
When using ByteBuffers et al, there needs to be a way to release the underlying buffer rather than wait for garbage collection to kick in.
Such buffers may, by their very definition (and the JavaDocs own usage recommendations) be very large. These behemoths are 'fronted' by relatively tiny Java classes. The GC may not kick in in any meaningful time-frame. There are already close() (or similar) methods on classes like FileReader, since it is standard and good practice to release operating system resources as soon as possible.
These memory blocks should be considered the same - significant and limited resources.
If you don't want to change the API because it will break existing applications, the easy solution is to create a subclass of Buffer called "FreeableBuffer" or similar with a free() method (or similar) and then make ByteBuffer et al a subclass of FreeableBuffer.
Here's a sample program and output from two different versions of the JDK. This was tested on Solaris 8 in 32-bit mode using JDK 1.4.-b21 and JDK 1.5.0-b64
The program is slightly contrived, but the crux is the previous ByteBuffer(s) have (in effect) been released but in the case of 1.4 GC never kicks in and it barfs.
In the case of 1.5, it's better, GC kicks in, but according to the output
of top (not included) the JVM grew to ~1281M before GC kicked in.
If I could do a bb.release() - or similar - then those resources i.e. the
big memory buffer could be free()-ed, cached and re-used or whatever.
1.4 Output:
# java -Xmx64m -verbose:gc BB
Done 1
Done 2
Done 3
Done 4
Done 5
Done 6
Done 7
Done 8
Done 9
Done 10
Done 11
Done 12
Exception in thread "main" java.lang.OutOfMemoryError
at sun.misc.Unsafe.allocateMemory(Native Method)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:57)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:283)
at BB.main(BB.java:17)
1.5 Output:
# N.B. I had to specify a larger heap size for this test otherwise it ran out of memory immediately.
# java -Xmx1536m -verbose:gc BB
Done 1
Done 2
Done 3
Done 4
[GC 147K->144K(4992K), 0.0149122 secs]
[Full GC 144K->97K(4992K), 0.0202993 secs]
Done 5
Done 6
Done 7
Done 8
[GC 158K->129K(4992K), 0.0024465 secs]
[Full GC 129K->97K(4992K), 0.0260971 secs]
Done 9
Done 10
Done 11
Done 12
[GC 159K->161K(4992K), 0.0008160 secs]
[Full GC 161K->98K(4992K), 0.0166681 secs]
Done 13
Done 14
Done 15
Done 16
[GC 159K->130K(4992K), 0.0011929 secs]
[Full GC 130K->97K(4992K), 0.0278316 secs]
Done 17
Done 18
Done 19
Done 20
[GC 158K->161K(4992K), 0.0020273 secs]
[Full GC 161K->97K(4992K), 0.0181044 secs]
Done 21
Done 22
Done 23
Done 24
[GC 159K->145K(4992K), 0.0041246 secs]
[Full GC 145K->98K(4992K), 0.0210584 secs]
Done 25
Done 26
Done 27
Done 28
[GC 159K->146K(4992K), 0.0058417 secs]
[Full GC 146K->98K(4992K), 0.0204990 secs]
Done 29
Done 30
...<SNIP>... etc...
---------- BEGIN SOURCE ----------
import java.nio.ByteBuffer;
public class BB
{
public static void main(String[] args)
{
// May make these configurable using -Dxxx
final int MAX_BUFFS = 1000;
final int BUFSIZE = 300 * 1024 * 1024;
final int PUT = 1000 * 1000;
final int REPORT = 1;
final boolean store = false;
for(int i = 0; i < MAX_BUFFS; i++)
{
ByteBuffer bb = ByteBuffer.allocateDirect(BUFSIZE);
if (i % REPORT == 0) System.err.println("Done " + (i + 1));
}
}
}
---------- END SOURCE ----------
REPRODUCIBILITY :
This bug can be reproduced always.
###@###.### 2005-07-13 12:21:24 GMT
- duplicates
-
JDK-6749531 ByteBuffer.allocateDirect is still causing OutOfMemoryErrors
- Closed
- relates to
-
JDK-4724038 (fs) Add unmap method to MappedByteBuffer
- Closed