ADDITIONAL SYSTEM INFORMATION :
Java 1.8 271

A DESCRIPTION OF THE PROBLEM :
The API of InflaterInputStream and DeflaterOutputStream is misleading

Both of this stream classes have plain one-arg ctors that allocate Inflater/Deflater internally, and in that case closing the stream also invokes an "end" method on the Inflater/Deflater objects, which releases native memory immediately


However when trying to control buffer size, we have to call a ternary ctor and allocate Inflater/Deflater externally

e.g.

new InflaterInputStream(in, new Inflater(), 256000)

new DeflaterOutputStream(os, new Deflater(), 32000)

In this case, closing the stream does not invoke the "end" method of Inflater/Deflater, as it assumes no ownership over these objects. Rather the "end" is invoked by the "finalizer" when the objects go through GC. In this case our system went into what seems like native memory leak.

The Inflater/Deflater rely on non-heap native memory, so not terminating them explicitly, defers the release of associated native memory to Full GC event.
This is the #1 reason of reported issues of native memory problems in JVM as reported on the internet.

Suggested remedy:
1. Add a 2-arg ctor to InflaterInputStream and DeflaterOutputStream that accepts an input stream and buffer size - this will prevent falling into the trap of not calling "end" on Deflater/Inflater
2. Document this potential native memory problem on the ternary ctor and in Inflater/Deflater classes

We just spent two weeks on production failures because of this issue which is very hard to diagnose.
Other reports by same cause:
https://medium.com/swlh/native-memory-the-silent-jvm-killer-595913cba8e7
https://www.elastic.co/blog/tracking-down-native-memory-leaks-in-elasticsearch