Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8212926

HttpClient does not retrieve files with large sizes over HTTP/1.1

XMLWordPrintable

    • b18
    • Verified

        ADDITIONAL SYSTEM INFORMATION :
        $ java -version
        java version "11.0.1" 2018-10-16 LTS
        Java(TM) SE Runtime Environment 18.9 (build 11.0.1+13-LTS)
        Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.1+13-LTS, mixed mode)

        $ [System.Environment]::OSVersion.Version
        Major Minor Build Revision
        ----- ----- ----- --------
        10 0 17134 0

        A DESCRIPTION OF THE PROBLEM :
        When using the new HttpClient it dosen't download files whose filesize in byte is reported bigger than Integer.MAX_VALUE most of the time if using HTTP 1.1.

        1) A file with size of 2147483648 (= Integer.MAX_VALUE + 1) reported in the content-length header just dosen't invoke any methode besides the onSubscribe methode of a HttpResponse.BodySubscriber.

        2) Interestingly a file with a size of 4294967294 (= Integer.MAX_VALUE - Integer.MIN_VALUE - 1) reported in the content-length header downloads normally.

        3) However a file with a size of 4294967296 (= Integer.MAX_VALUE - Integer.MIN_VALUE + 1) downloads a file with the size of exactly 0 bytes.

        For some reson using HTTP 2 works for at least the first case and downloads that file correctly, I don't have the resources to easily test the other cases using HTTP 2.

        STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
        1. Create Http Client that will use HTTP 1.1 (for the example file I provided this is important as cloudflare will server up HTTP 2 if we don't specificly request HTTP 1.1)
        2. Create a request to download a file with one of the specified sizes (for example: https://map.usbcraft.net/file/usbpc-downloads/java_http_client.fail (content-length = 2147483648))
        3. Try to download from that url into a file with a HttpResponse.BodyHandlers.ofFile()

        EXPECTED VERSUS ACTUAL BEHAVIOR :
        EXPECTED -
        It should download the file normally into the specified file on the hard drive.
        ACTUAL -
        The file on the Hard drive is created, but dosen't get any content and remains at 0 bytes size.
        Looking in the Task Manager shows that java is downloading with full speed, but I don't know what is happening with that data.

        For some reason it also sometimes happend that the CPU usage shot up from ~5% to ~30% and network usage stopped.

        ---------- BEGIN SOURCE ----------
        import java.io.IOException;
        import java.net.URI;
        import java.net.http.HttpClient;
        import java.net.http.HttpRequest;
        import java.net.http.HttpResponse;
        import java.nio.file.Path;

        public class Main {
            public static void main(String[] args) throws IOException, InterruptedException {
                HttpClient client = HttpClient.newBuilder()
                        .version(HttpClient.Version.HTTP_1_1)
                        .build();

                HttpRequest request = HttpRequest.newBuilder()
                        .uri(URI.create("https://map.usbcraft.net/file/usbpc-downloads/java_http_client.fail"))
                        .build();

                client.send(request, HttpResponse.BodyHandlers.ofFile(Path.of("./test.bin")));
            }
        }
        ---------- END SOURCE ----------

        FREQUENCY : always


              michaelm Michael McMahon
              webbuggrp Webbug Group
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: