Summary
Add a new static ofFileChannel(FileChannel channel, long offset, long length)
method to java.net.HttpRequest.BodyPublishers
to provide an HttpClient
request body publisher to upload a certain region of a file.
The new publisher does not modify the state of the passed FileChannel
, streams the file channel bytes as it publishes (i.e., avoids reading the entire file into the memory), and can be leveraged to implement sliced uploads.
Problem
The current set of built-in request body publishers do not help with sliced uploads where different slices of a large file are uploaded in parallel. Sliced uploads can sometimes help improve throughput of uploading file content. Furthermore, sliced uploads also help improve resiliency - if a request failed during publishing a part of the file, then the application would have to only retry publishing that part instead of publishing the entire file again.
Solution
A new request body publisher is introduced that allows uploading a sub-range of bytes from a java.nio.channels.FileChannel
.
This new request body publisher is an out of the box building block that an application could use in its implementation of slice uploads.
An alternative API - HttpRequest.BodyPublishers.ofFile(Path path, long offset, long length)
was also considered. This would have be an overloaded method of the existing HttpRequest.BodyPublishers.ofFile(Path path)
method. It was decided not to introduce this method because the implementation of that would require opening a file descriptor for each call to that method. That would limit the utility of that method if several parts of the same file had to be published concurrently, since it could lead to reaching the file descriptor limit enforced by the system.
On the other hand, a single instance of a FileChannel
can be used concurrently to publish several parts of the same file. The ofFileChannel(FileChannel channel, long offset, long length)
method was thus chosen as the better alternative.
Specification
The following new method is added to java.net.http.HttpRequest.BodyPublishers
:
/**
* {@return a request body publisher whose body is the {@code length}
* content bytes read from the provided file {@code channel} starting
* from the specified {@code offset}}
* <p>
* This method and the returned {@code BodyPublisher} do not modify the
* {@code channel}'s position.
* <p>
* This method does not close the {@code channel}. The caller is
* expected to close the {@code channel} when no longer needed.
*
* @apiNote
* This method can be used to either publish just a region of a file as
* the request body or to publish different regions of a file
* concurrently. A typical usage would be to publish different regions
* of a file by creating a single instance of {@link FileChannel} and
* then send multiple concurrent {@code HttpRequest}s, each of which
* uses a new {@code ofFileChannel BodyPublisher} created from the same
* channel with a different, typically non-overlapping, range of bytes
* specified by offset and length.
*
* @param channel a file channel
* @param offset the offset of the first byte
* @param length the number of bytes to read from the file channel
*
* @throws IndexOutOfBoundsException if the specified byte range is
* found to be {@linkplain Objects#checkFromIndexSize(long, long, long)
* out of bounds} compared with the size of the file referred by the
* channel
*
* @throws IOException if the {@linkplain FileChannel#size() channel's
* size} cannot be determined or the {@code channel} is closed
*
* @throws NullPointerException if {@code channel} is null
*
* @since 26
*/
public static BodyPublisher ofFileChannel(FileChannel channel, long offset, long length) throws IOException { ...
- csr of
-
JDK-8329829 HttpClient: Add a BodyPublishers.ofFileChannel method
-
- In Progress
-