-
Enhancement
-
Resolution: Unresolved
-
P4
-
None
-
None
-
generic
-
generic
A DESCRIPTION OF THE PROBLEM :
I would like a JEP for a standardized Java lossless compression framework (e.g. java.io.compress). Data compression should be an ideal case for integration into the standard library because:
1. It has a conceptually simple and uniform interface.
2. It is suited for an SPI / plugin interface.
3. Almost all software can benefit from it.
4. The JDK already contains implementations of some compression algorithms, just not behind a uniform interface.
5. Codecs can sometimes be hardware accelerated, which is the sort of thing the JVM is good at. For example see how Intel QAT can be used to speed up zstd.
To be clear, this would not cover archive formats like tar, zip etc. Just the compression part.
As a quick interface proposal, it would need to cover:
1. Streams and buffer oriented usage, with the ability to flush/reset the stream at points (for seeking), e.g. a generic CompressedInputStream/OutputStream pair that can sniff magic bytes, then look up the provider and use them automatically.
2. Some standard way to set quality levels.
3. Standardized access to metadata like:
3a. File extensions
3b. Stream magics
3c. MIME types (for Accept-Encoding headers)
3d. ZIP compression method IDs.
4. A notion of provider priority. For example Intel provide a highly optimized gzip implementation that could either replace the one in the JDK outright, or if not, then be shipped as a plugin that overrides the built in gzip implementation.
plus the usual ServiceProvider stuff.
It would also be nice if such work would encompass updating existing libraries like ZipFile, zipfs, the HTTP libraries etc so they can open files with plugged-in codecs. With careful structuring of the class/module path and HotSpot internals this would allow for e.g. using zstd compressed JARs, as long as the zstd provider was first in the classpath. And of course, for max bonus points, unifying the various places in the JDK where compression is used so things like the modules file or heap dumps can be upgraded in place to use more advanced codecs as they become available, without needing JDK core changes.
I would like a JEP for a standardized Java lossless compression framework (e.g. java.io.compress). Data compression should be an ideal case for integration into the standard library because:
1. It has a conceptually simple and uniform interface.
2. It is suited for an SPI / plugin interface.
3. Almost all software can benefit from it.
4. The JDK already contains implementations of some compression algorithms, just not behind a uniform interface.
5. Codecs can sometimes be hardware accelerated, which is the sort of thing the JVM is good at. For example see how Intel QAT can be used to speed up zstd.
To be clear, this would not cover archive formats like tar, zip etc. Just the compression part.
As a quick interface proposal, it would need to cover:
1. Streams and buffer oriented usage, with the ability to flush/reset the stream at points (for seeking), e.g. a generic CompressedInputStream/OutputStream pair that can sniff magic bytes, then look up the provider and use them automatically.
2. Some standard way to set quality levels.
3. Standardized access to metadata like:
3a. File extensions
3b. Stream magics
3c. MIME types (for Accept-Encoding headers)
3d. ZIP compression method IDs.
4. A notion of provider priority. For example Intel provide a highly optimized gzip implementation that could either replace the one in the JDK outright, or if not, then be shipped as a plugin that overrides the built in gzip implementation.
plus the usual ServiceProvider stuff.
It would also be nice if such work would encompass updating existing libraries like ZipFile, zipfs, the HTTP libraries etc so they can open files with plugged-in codecs. With careful structuring of the class/module path and HotSpot internals this would allow for e.g. using zstd compressed JARs, as long as the zstd provider was first in the classpath. And of course, for max bonus points, unifying the various places in the JDK where compression is used so things like the modules file or heap dumps can be upgraded in place to use more advanced codecs as they become available, without needing JDK core changes.