FULL PRODUCT VERSION :
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Linux localhost 2.6.32-5-amd64 #1 SMP Thu Mar 22 17:26:33 UTC 2012 x86_64 GNU/Linux

A DESCRIPTION OF THE PROBLEM :
Our system with a WebApp + Backend architecture, both running on separate hosts in JVMs connected via our proprietary TCP based protocol using java.nio is running into java.lang.OutOfMemoryError: Direct buffer memory on the WebApp server originating by call stack at our java.nio usage. As our protocol is using small direct buffers, a test was done with the following simple code:

import java.nio.ByteBuffer;
public class TestDirectBuffer
{
  public static void main( String args[] ) throws Exception
  {
    while(true) {
      ByteBuffer.allocateDirect(522);
    }
  }
}

When run as follows, a java.lang.OutOfMemoryError: Direct buffer memory is thrown after a few seconds:

java -Xmx200m -Xms200m -XX:+DisableExplicitGC -XX:MaxDirectMemorySize=512m -XX:+UseConcMarkSweepGC -XX:+UseCompressedOops TestDirectBuffer
Exception in thread "main" java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:658)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at TestDirectBuffer.main(TestDirectBuffer.java:8)

Without -XX:+DisableExplicitGC it keeps up continuously without errors and without any leaks into memory, using constantly about 3 GByte virtual memory:

java -Xmx200m -Xms200m -XX:MaxDirectMemorySize=512m -XX:+UseConcMarkSweepGC -XX:+UseCompressedOops TestDirectBuffer

same result in 1.7.0_80, 1.8.0_05, 1.8.0_60 and many more


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
use above test code with JVM flag DisableExplicitGC set


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
no memory leak

ACTUAL -
direct buffer memory is filled until JVM exits with java.lang.OutOfMemoryError: Direct buffer memory

ERROR MESSAGES/STACK TRACES THAT OCCUR :
Exception in thread "main" java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:658)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at TestDirectBuffer.main(TestDirectBuffer.java:8)

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
// java -Xmx200m -Xms200m -XX:+DisableExplicitGC -XX:MaxDirectMemorySize=512m -XX:+UseConcMarkSweepGC -XX:+UseCompressedOops TestDirectBuffer
// runs fine without -XX:+DisableExplicitGC

import java.nio.ByteBuffer;
public class TestDirectBuffer
{
  public static void main( String args[] ) throws Exception
  {
    while(true) {
      ByteBuffer.allocateDirect(522);
    }
  }
}


---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
don't use -XX:+DisableExplicitGC