(bf) Allocating direct buffers causes OutOfMemoryError

Name: rmT116609 Date: 06/17/2003


FULL PRODUCT VERSION :
java version "1.4.2-beta"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2-beta-b19)
Java HotSpot(TM) Client VM (build 1.4.2-beta-b19, mixed mode)


FULL OS VERSION :
SunOS pantar 5.7 Generic_106541-17 sun4u sparc SUNW,Ultra-4


A DESCRIPTION OF THE PROBLEM :
Allocating a direct byte buffer may give an OutOfMemoryError, even when plenty of memory is available. Although not used in the test case below, in my application this occurs when reading a socket channel:

 java.lang.OutOfMemoryError
     at java.nio.Bits.reserveMemory(Bits.java:618)
     at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
     at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:285)
     at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:54)
     at sun.nio.ch.IOUtil.read(IOUtil.java:205)
     at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:207)
     at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:171)
     at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:86)


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Compile source below and run it:

javac ByteBufferTest.java

java ByteBufferTest

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
  Program keeps running forever (this is what happens with JDK 1.4.1_02).
ACTUAL -
See exception below.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
Exception in thread "main" java.lang.OutOfMemoryError
        at java.nio.Bits.reserveMemory(Bits.java:618)
        at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
        at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:285)
        at ByteBufferTest.main(ByteBufferTest.java:7)


REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.nio.ByteBuffer;

public class ByteBufferTest {

  public static void main(String[] args) {
    for (;;) {
      ByteBuffer buf = ByteBuffer.allocateDirect(8192);
    }
  }
}

---------- END SOURCE ----------
(Review ID: 188204)
======================================================================

Name: jl125535 Date: 08/20/2003


It seems that the direct buffers should be able to access the 32 bits (or at least 31 bits) that the underlying OS can provide.

For example, with jdk1.4.1 under windows it is possible to map a file into memory that is no larger than 1.6 mb *minus* the amount of memory already allocated to the VM.
This is regardless of the physical memory available.

Why not have access to the full 2-4 GB via a memory mapped file?
Better yet, within the constraints of the os, given that sun.misc.Unsafe is manipulating long pointers anyway, there seems to be no reason that *EACH* MappedByteBuffer should not each have a limit of 4 GB.

JUSTIFICATION :
+Large data-instensive applications can get extreme performance improvements with access to large amounts of data in cheap memory.
+Memory mapped access to large files is not available

Multiple 4GB direct buffers should give most people access to all the memory they need, without the need for 64 bit pointers.
(Review ID: 184727)
======================================================================