Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8308364

HttpClientImpl SelectorManager thread never dies if response body not retrieved

XMLWordPrintable

    • generic
    • generic

      ADDITIONAL SYSTEM INFORMATION :
      e.g., Amazon Coretto OpenJDK-derived JDK 17.0.6 on Linux X64

      A DESCRIPTION OF THE PROBLEM :
      HttpClientImpl$SelectorManager will die if it wakes up and finds that its owner (the enclosing HttpClientImpl) instance) is no longer referenced per its isReferenced method. isReferenced returns true if either of these conditions is met: a) if its WeakReference's referent to the HttpClientFacade (which is the HttpClient instance actually published when the HttpClientImplBuilder's build() method is invokoed) referent disappears as a result of having been garbage collected); and b) if its "pending operations" count is non-zero.

      What we've seen is that if, for some reason, a client invokes send(HttpRequest) on the HttpClient, but never invokes the body() method of the HttpResponse, the pending operation count never drops to 0. In that case, the SelectorManager will never die.

      (Reading the source code, I see that SelectorManager threads may be able to be killed via interrupt() requests on the thread; I haven't tried that, since in general, I don't interrupt() strange threads I didn't create, and I don't really have a convenient way to do that in our application.)

      The SelectorManager threads only die when the host JVM is shut down. Or else, in long running applications, you may eventually get an "OutOfMemoryError: Unable to create new native thread".

      Per JDK-8304165, in Java 21, HttpClient is AutoCloseable, and I haven't looked at the source code for that version (which is still under development at the time of writing). But this is not the case in Java 17, and I don't believe there's any indication that those changes are going to be back-ported to earlier releases (or Java 17, in particular, since it is an LTS release).

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      1. Create an HttpClient via HttpClient.newBuilder().create();

      2. Use the send(HttpRequest, HttpResponse.BodyHandler) method to send a request, but do not invoke the body() method on the returned HttpResponse.

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      Eventually the SelectorManager thread will determine that it is safe to end and terminate.
      ACTUAL -
      The SelectorManager thread owned by the HttpClientImpl instance associated with the published HttpClient instance returned by the builder (an HttpClientFacade) never dies.

      CUSTOMER SUBMITTED WORKAROUND :
      Always invoke body() on every HttpResponse, even if the response indicates that the request resulted in, e.g., an HTTP 404 (or some other condition appears in the response that the client's business logic determines means the response is not of interest).

      In our case, since we have a wrapper for this API, anyway, I'm working on a way to track the response instances on the wrapper object, and when its close method is invoked, it will consume the body of any responses for which body() has not yet been invoked. (This amounts to making an HttpClient implementation that is a wrapper for the built-in HttpClient instances, and which itself implements AutoCloseable.)

      FREQUENCY : rarely


        1. HttpClientIssue.java
          7 kB
          Andrew Wang
        2. httpclient-issue-threads.txt
          3 kB
          Andrew Wang
        3. screenshot.png
          281 kB
          Andrew Wang

            Unassigned Unassigned
            webbuggrp Webbug Group
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: