FULL PRODUCT VERSION :
JDK build 1.4.2_01-b06, build 1.4.2_03-b02 , and build 1.4.2_04-b05

java version "1.4.2_04"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_04-b05)
Java HotSpot(TM) Client VM (build 1.4.2_04-b05, mixed mode)


ADDITIONAL OS VERSION INFORMATION :
Fails on Linux and Solaris (Sun / Fujitsu)
Seems to work on Windows with no adverse problems

Solaris Info:
SunOS neo6s001 5.8 Generic_108528-27 sun4u sparc SUNW,Sun-Fire-480R
4 CPUs / 16GB of memory

Fujitsu Info:
SunOS neo6f016 5.8 Generic_108528-24 sun4us sparc FJSV,GPUZC-M
4 CPUs / 16GB of memory

Linux Info:
Linux xwing 2.4.20-8 #1 Thu Mar 13 17:54:28 EST 2003 i686 i686 i386 GNU/Linux
with 768MB of memory and 2GB of swap space.


EXTRA RELEVANT SYSTEM CONFIGURATION :
Standard Unix environment.

A DESCRIPTION OF THE PROBLEM :
When the Channel.write method is invoked using a non-direct ByteBuffer in a ByteBuffer array, over time the Native Memory leaks and causes an Out Of Memory error during a large file write.

This is reproducable on Linux and Solaris. Based on analysis in a profiler and thread analyzer along with the stack trace, it appears that in the Channel.write, a new ByteBuffer is created that is Direct. The direct is mapped to Native memory. It appears that it is being pooled and not reused properly.

It takes a long time for this to occur and files processing over 50MB is what caught this problem. We are doing small buffer writes (in the example 400 bytes) continuously to a file. In production it runs with several other threads and combined will cause the OutOfMemory condition no matter what the JVM Heap Size is set to. Adjusting the
Direct Memory size using -XX:MaxDirectMemorySize=8m will cause the problem to occur sooner on systems rather than waiting hours for it to process.




STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
/*

After creating an "output" directory where it will generate between 500MB and 1.6GB of output data, run the following class with:

 java -server -XX:MaxDirectMemorySize=8m IOBugTest -t 4 -i 10000000

This will create 4 threads that will write files to the output directory.
Each file will contain a repeating letter that was assigned to the thread as an ID. This repeats over and over until the iteration is reached.

However after running a period of time, the JVM receives an Out Of Memory error.

*/


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The program should run to completion with no OutOfMemory errors. Assuming you had sufficient disk space, you should have in the "output" directory 4 files containing 400MB of repeating 3 letter characters where each character is the letter assigned to the thread as its ID.


ACTUAL -
One or more threads reach an OutOfMemory condition and abort.
The amount of Direct "native" Memory available will determine how long it takes to reproduce the error. The program only uses about 200MB with Non-Direct ByteBuffers and 11MB with Direct ByteBuffers. It appears to leak slowly over time until the Native memory is exhausted.


ERROR MESSAGES/STACK TRACES THAT OCCUR :
java.lang.OutOfMemoryError
        at java.nio.Bits.reserveMemory(Bits.java:618)
        at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
        at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:285)
        at sun.nio.ch.IOUtil.write(IOUtil.java:134)
        at sun.nio.ch.FileChannelImpl.write0(FileChannelImpl.java:228)
        at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:243)
        at java.nio.channels.FileChannel.write(FileChannel.java:222)
        at IOBugTest.executeIteration(IOBugTest.java:78)
        at IOBugTest.run(IOBugTest.java:46)

I have more info that I can send if you need it. Let me know via email.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
/*

After creating an "output" directory where it will generate between 500MB and 1.6GB of output data, run the following class with:

 java -server -XX:MaxDirectMemorySize=8m IOBugTest -t 4 -i 10000000

This will create 4 threads that will write files to the output directory.
Each file will contain a repeating letter that was assigned to the thread as an ID. This repeats over and over until the iteration is reached.

However after running a period of time, the JVM receives an Out Of Memory error.

*/



public class IOBugTest extends Thread {
    protected static char count = 'A';
    protected static int iterationCount = 0;
    protected char id = count++;
    protected static boolean debug=false;
    protected int repeats=100;
    protected int bufferSize=repeats*4;
    protected ByteBuffer buffer[] = {allocateBuffer(bufferSize)};
 
    public IOBugTest() {
    }
 
    public void putData(ByteBuffer buf, char id) {
      for (int i=0; i<repeats; i++ ){
        buf.put( (byte) id);
        buf.put( (byte) id);
        buf.put( (byte) id);
        buf.put( (byte) '\n');
      }
    }
 
    public void run() {
        final String clazzName=this.getClass().getName();
        final String threadName=clazzName+id ;
        try {
            Thread.currentThread().setName(threadName);
            System.out.println("starting "+threadName);
 
            String fileName = "./output/"+threadName+".out";
            FileOutputStream fos = new FileOutputStream(fileName);
            FileChannel channel = fos.getChannel();
 
            for (int i = 0; i < iterationCount; i++) {
                executeIteration(channel);
                if (debug) {
                  try { Thread.sleep(1000); } catch(Exception ignore){}
                }
            }
            channel.close();
            fos.close();
        } catch (Throwable e) {
            e.printStackTrace();
        } finally {
            System.out.println("leaving "+threadName);
        }
 
    }
 
 
    public void executeIteration(FileChannel channelOut) throws IOException {
 
        putData(buffer[0],id);
 
        // Blob size in binary form
        buffer[0].flip();
 
        /*********************************************************************/
        /** This is where it leaks!!! ****************************************/
        /*********************************************************************/
        /* Does not leak if we use //channelOut.write(buffer[0]); */
        /* Does not leak if we use direct ByteBuffer */
        /* Write blob using Channel write(ByteBuffer[] array) method */
        /* and after some time is will leak and cause Out of Memory error */
        /*********************************************************************/
 
        channelOut.write(buffer); // <=======================================
 
        buffer[0].clear();
    }
 
    public ByteBuffer allocateBuffer(int size) {
        //return ByteBuffer.allocateDirect(size); //this avoids the memory leak
        return ByteBuffer.allocate(size);
    }
 
    public static void main(String args[]) {
        try {
            final String cmdHelp = " java IOBugTest -t <threadCount> -i <iterations>";
 
            if (args == null || args.length < 4) {
                System.out.println(cmdHelp);
                return;
            }
 
            int threadCount = 0;
            iterationCount = 0;
            String parm = null;
            String value = null;
 
            try {
                for (int argc = 0; argc < args.length; argc++) {
                    parm = args[argc];
                    if (parm.indexOf("-t") >= 0) {
                        value = args[argc + 1];
                        System.out.println("using thread count of " + value);
                        threadCount = Integer.parseInt(value);
                    } else if (parm.indexOf("-i") >= 0) {
                        value = args[argc + 1];
                        System.out.println("using iteration count of " + value);
                        iterationCount = Integer.parseInt(value);
                    }
                }
            } catch (Exception e) {
                e.printStackTrace();
                System.out.println(cmdHelp);
                return;
            }
 
            Thread threadArray[] = new Thread[threadCount];
            IOBugTest iot = null;
            System.out.println("staring threads...");
            for (int i = 0; i < threadCount; i++) {
                iot = new IOBugTest();
                threadArray[i] = iot;
                threadArray[i].start();
            }
 
            long begin = System.currentTimeMillis();
            long end = 0;
            long duration = 0;
            String tid = null;
            for (int i = 0; i < threadCount; i++) {
                tid = threadArray[i].getName();
                System.out.println("joining thread id " + tid);
                try {
                    threadArray[i].join();
                } catch (Exception ignore) {
                }
                end = System.currentTimeMillis();
                duration = end - begin;
                System.out.println("thread id " + tid +
                         " join ended at or near " + duration + " ms ");
                //try and free those channels
                System.gc();
                System.runFinalization();
                System.gc();
            }
 
            System.out.println("All threads seem to have joined, review memory...");
            try {
                Thread.sleep(60000);
            } catch (Exception ignore) {
            }
            end = System.currentTimeMillis();
            duration = end - begin;
            System.out.println("all done duration=" + duration + " ms ");
 
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            System.out.println("leaving main!");
        }
    }
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Do not use Non-Direct ByteBuffers in an array and pass it to Channel.write method.

You can use Direct ByteBuffers with no problem.
###@###.### 2005-1-06 07:15:20 GMT