Summary

Allow users to bound the size of buffers cached in the per-thread buffer caches

Problem

This CCC is submitted on behalf of Tony Printezis, Twitter.

I/O with the channel APIs in java.nio.channels can be used to do I/O with direct or heap ByteBuffers. Direct buffers are backed by memory outside of the java heap, heap buffers are backed by a byte[] in the java heap. The underlying I/O is always done with direct memory (as HotSpot doesn't support object pinning aside from JNI critical sections) and so I/O with heap buffers involves using temporary direct buffers to do the I/O. This mechanism is known in this area as the "temporary buffer cache" and involves a per-thread cache of buffers. The reason for the cache is to avoid a malloc/free per I/O operation.

There are patterns of I/O usage that can result in the temporary buffer cache using a lot of memory. For example, a thread might attempt a read or write with a 4MB buffer but all subsequent I/O operations by that thread might be with a 8K buffer.  This pattern of usage results in the thread caching a 4MB direct buffer that wastes memory. It just takes one outlier to increase the cache, the cache does not discard a large buffer and replace with a smaller buffer based on the pattern of subsequent I/O operations.

The request/proposal from Twitter is to limit the capacity of buffers that can be cached. This is preferable to a more sophisticated solution that would add cycles to I/O operations and would also be harder to understand when monitoring memory usage.

Solution

Add a JDK-specific system property to limit the capacity of buffers that can be held in the temporary buffer cache.

Specification

If the system property jdk.nio.maxCachedBufferSize is set then its value is the maximum capacity (in bytes) of a buffer that is cached. If the value of the property cannot be parsed as a natural number > 0 then the property is ignored.