Details
-
Bug
-
Resolution: Won't Fix
-
P4
-
8, 9
-
x86
-
os_x
Description
FULL PRODUCT VERSION :
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
ADDITIONAL OS VERSION INFORMATION :
Discovered on OSX 10.12.6, also reproduced in Windows 10.
A DESCRIPTION OF THE PROBLEM :
Connecting to an Apache (and possibly other) web server(s) with http2 enabled stops http connections being reused. A new connection is made for each request to the web server. If I disabled http2 and switch back to http/1.1 connections are reused - when requesting 5 urls in a row there will be a single connection reused for each request. With http2 enabled a new connection is made for each of the 5 urls.
I see this in the Apache logging to back this up:
[Mon Sep 11 13:31:18.394995 2017] [ssl:info] [pid 15:tid 139731359987456] [client 172.17.0.1:42262] AH01964: Connection to child 66 established (server www.example.com:443)
and when I run with -Djavax.net.debug=all i see this in the http2 case:
main, called close()
main, called closeInternal(true)
rather than the in the good case (http/1.1), I see:
main, setSoTimeout(0) called
I've done some digging around this and it seems to be due to a parsing issue with the Connection header. Apache returns:
Connection: Upgrade, Keep-Alive
However the code in sun.net.www.http.HttpClient in parseHTTPHeader() get's the value of the entire header:
keep = responses.findValue("Connection");
which will be "Upgrade, Keep-Alive"
and only sets the keepAliveConnections flag based on the following check:
if (keep != null && keep.toLowerCase(Locale.US).equals("keep-alive"))
To further verify this I added an extra value to the Connection header on my http/1.1 web sever:
Connection: Keep-Alive, Keep-Alive
which has the same affect as enabling http2 - The extra value in the Connection header prevents http connections being reused.
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Two options
1) Set up an http2 web server, update the attached program to include a few urls on it. Then observe either in the apache log (you need to enable LogLevel warn ssl:info to see the new connections being logged) the new connections being made, or alternatively run with -Djavax.net.debug=all flag set and observer the connections being closed.
2) Setup and http/1.1 web server and manipulate the Connection header to have two values, I did this using the headers_module in Apache and did this:
Header unset Connection
Header set Connection Keep-Alive
Again you can use with the Apache logging or -Djavax.net.debug=all to observer the connections not being reused.
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Connections being reused.
ACTUAL -
Connections are not being reused.
REPRODUCIBILITY :
This bug can be reproduced always.
---------- BEGIN SOURCE ----------
import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.URL;
import java.security.SecureRandom;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import seo.spider.spider.TrustEverythingTrustManger;
public class Http2Test
{
public static void main(
final String[] args)
{
System.out.println(getJavaInfo());
trustAllSSLCerts();
// A few urls on the same webserver
final List<String> urls = Arrays.asList(
"https://www.example.com/url1.html",
"https://www.example.com/url2.html"
);
for (String urlString : urls)
{
try
{
System.out.println("creating url" + urlString);
URL url = new URL(urlString);
System.out.println("url created, creating http connection");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
logHeaders(conn.getHeaderFields(), "RESPONSE HEADERS");
final int responseCode = conn.getResponseCode();
System.out.println("responseCode: " + responseCode);
conn.getInputStream().close();
System.out.println("done closing input stream");
}
catch (IOException e)
{
e.printStackTrace();
}
}
}
private static void trustAllSSLCerts()
{
TrustManager[] trustAllCerts = new TrustManager[] { new TrustEverythingTrustManger() };
// Install the all-trusting trust manager
try
{
SSLContext sc = SSLContext.getInstance("SSL");
sc.init(null, trustAllCerts, new SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
}
catch (Exception e)
{
e.printStackTrace();
}
HttpsURLConnection.setDefaultHostnameVerifier(new HostnameVerifier()
{
@Override
public boolean verify(String hostname, SSLSession session)
{
return true;
}
});
}
private static void logHeaders(
final Map<String, List<String>> headers,
final String message)
{
System.out.println("<-- START " + message + " -->");
for ( Map.Entry<String, List<String>> e : headers.entrySet())
{
for (String value : e.getValue())
{
String key = e.getKey() == null ? "" : e.getKey() + ": ";
System.out.println(key + value);
}
}
System.out.println("<-- END " + message + " -->");
}
private static String getJavaInfo()
{
return "Vendor '" + System.getProperty("java.vendor") + "' " +
"URL '" + System.getProperty("java.vendor.url") + "' " +
"Version '" + System.getProperty("java.version") + "' " +
"Home '" + System.getProperty("java.home") + "'";
}
}
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
I can't think of one.
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
ADDITIONAL OS VERSION INFORMATION :
Discovered on OSX 10.12.6, also reproduced in Windows 10.
A DESCRIPTION OF THE PROBLEM :
Connecting to an Apache (and possibly other) web server(s) with http2 enabled stops http connections being reused. A new connection is made for each request to the web server. If I disabled http2 and switch back to http/1.1 connections are reused - when requesting 5 urls in a row there will be a single connection reused for each request. With http2 enabled a new connection is made for each of the 5 urls.
I see this in the Apache logging to back this up:
[Mon Sep 11 13:31:18.394995 2017] [ssl:info] [pid 15:tid 139731359987456] [client 172.17.0.1:42262] AH01964: Connection to child 66 established (server www.example.com:443)
and when I run with -Djavax.net.debug=all i see this in the http2 case:
main, called close()
main, called closeInternal(true)
rather than the in the good case (http/1.1), I see:
main, setSoTimeout(0) called
I've done some digging around this and it seems to be due to a parsing issue with the Connection header. Apache returns:
Connection: Upgrade, Keep-Alive
However the code in sun.net.www.http.HttpClient in parseHTTPHeader() get's the value of the entire header:
keep = responses.findValue("Connection");
which will be "Upgrade, Keep-Alive"
and only sets the keepAliveConnections flag based on the following check:
if (keep != null && keep.toLowerCase(Locale.US).equals("keep-alive"))
To further verify this I added an extra value to the Connection header on my http/1.1 web sever:
Connection: Keep-Alive, Keep-Alive
which has the same affect as enabling http2 - The extra value in the Connection header prevents http connections being reused.
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Two options
1) Set up an http2 web server, update the attached program to include a few urls on it. Then observe either in the apache log (you need to enable LogLevel warn ssl:info to see the new connections being logged) the new connections being made, or alternatively run with -Djavax.net.debug=all flag set and observer the connections being closed.
2) Setup and http/1.1 web server and manipulate the Connection header to have two values, I did this using the headers_module in Apache and did this:
Header unset Connection
Header set Connection Keep-Alive
Again you can use with the Apache logging or -Djavax.net.debug=all to observer the connections not being reused.
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Connections being reused.
ACTUAL -
Connections are not being reused.
REPRODUCIBILITY :
This bug can be reproduced always.
---------- BEGIN SOURCE ----------
import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.URL;
import java.security.SecureRandom;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import seo.spider.spider.TrustEverythingTrustManger;
public class Http2Test
{
public static void main(
final String[] args)
{
System.out.println(getJavaInfo());
trustAllSSLCerts();
// A few urls on the same webserver
final List<String> urls = Arrays.asList(
"https://www.example.com/url1.html",
"https://www.example.com/url2.html"
);
for (String urlString : urls)
{
try
{
System.out.println("creating url" + urlString);
URL url = new URL(urlString);
System.out.println("url created, creating http connection");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
logHeaders(conn.getHeaderFields(), "RESPONSE HEADERS");
final int responseCode = conn.getResponseCode();
System.out.println("responseCode: " + responseCode);
conn.getInputStream().close();
System.out.println("done closing input stream");
}
catch (IOException e)
{
e.printStackTrace();
}
}
}
private static void trustAllSSLCerts()
{
TrustManager[] trustAllCerts = new TrustManager[] { new TrustEverythingTrustManger() };
// Install the all-trusting trust manager
try
{
SSLContext sc = SSLContext.getInstance("SSL");
sc.init(null, trustAllCerts, new SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
}
catch (Exception e)
{
e.printStackTrace();
}
HttpsURLConnection.setDefaultHostnameVerifier(new HostnameVerifier()
{
@Override
public boolean verify(String hostname, SSLSession session)
{
return true;
}
});
}
private static void logHeaders(
final Map<String, List<String>> headers,
final String message)
{
System.out.println("<-- START " + message + " -->");
for ( Map.Entry<String, List<String>> e : headers.entrySet())
{
for (String value : e.getValue())
{
String key = e.getKey() == null ? "" : e.getKey() + ": ";
System.out.println(key + value);
}
}
System.out.println("<-- END " + message + " -->");
}
private static String getJavaInfo()
{
return "Vendor '" + System.getProperty("java.vendor") + "' " +
"URL '" + System.getProperty("java.vendor.url") + "' " +
"Version '" + System.getProperty("java.version") + "' " +
"Home '" + System.getProperty("java.home") + "'";
}
}
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
I can't think of one.