P4
FULL PRODUCT VERSION :
java version "9-ea"
Java(TM) SE Runtime Environment (build 9-ea+173)
Java HotSpot(TM) 64-Bit Server VM (build 9-ea+173, mixed mode)
ADDITIONAL OS VERSION INFORMATION :
Linux 4.4.0-79-generic #100~14.04.1-Ubuntu SMP Fri May 19 18:36:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
A DESCRIPTION OF THE PROBLEM :
This bug is related to the jdk.incubator.httpclient module.
Opening a websocket for the LXD event stream over TLS ( https://github.com/lxc/lxd/blob/master/doc/rest-api.md#10events ) causes the HttpClient's "SelectorManager" thread to spin indefinitely waiting for new data. The thread keeps receiving data for that web socket and calls onText() nicely, but it keeps the thread in a busy loop doing so. Later regular HTTP requests using the same HttpClient instance don't complete without harming the web socket's ability to keep reading.
The spinning loop is in SSLDelegate.EngineWrapper.recvAndUnwrap(), chan.read always returns 0, SocketChannelImpl.read() always returns -2/IOStatus.UNAVAILABLE, thus EGAIN for the underlying read(). The thread never leaves the recvAndUnwrap method and the target buffer "unwrap_src" is always empty. PlainHttpConnection configured the socket channel in non-blocking mode and no new data is expected, so getting EAGAIN is no surprise.
REGRESSION. Last worked in version 9
ADDITIONAL REGRESSION INFORMATION:
Earlier JDK9 EA releases didn't have this specific issue and partially worked for longer. They however had other problems like overflowing internal buffers later on (JDK-8159911), timeout problems (JDK-8160050) or not receiving any web socket data at all (JDK-8180155/JDK-8178522).
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
I don't have a detached test case, however playing with LXD's API like creating containers, pushing/pulling data to containers, reading lengthy process exec API output etc. has reliably shown bugs in every httpclient version I've tried.
For this specific bug I'm doing the following, all sharing one HttpClient instance:
1. GET /
2. GET /1.0
3. Web socket (wss) for /1.0/events?type=operation,logging with a listener that just prints to stdout and keeps requesting exactly like the default interface methods
4. GET /1.0/containers (doesn't complete)
The http client instance uses a custom SSLContext that supplies a TLS client certificate and skips the usual server verifications. All GET requests use httpClient.sendAsync, String response body processors and a 10s timeout(). The web socket uses httpClient.newWebSocketBuilder, a 10s connectTimeout and buildAsync.
Each step is completed (CompletableFuture.get()) before the next step, for the web socket buildAsync().get(). Step 4 does send the data, but never completes its response future, presumably doesn't receive anything. Delaying step 4 doesn't help either.
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The web sockets keeps waiting for data normally without causing CPU load, follow-up use of the HttpClient instance works normally.
ACTUAL -
The SelectorManager thread uses all its CPU time actively and the HttpClient instance doesn't complete unrelated sendAsync requests.
REPRODUCIBILITY :
This bug can be reproduced always.
java version "9-ea"
Java(TM) SE Runtime Environment (build 9-ea+173)
Java HotSpot(TM) 64-Bit Server VM (build 9-ea+173, mixed mode)
ADDITIONAL OS VERSION INFORMATION :
Linux 4.4.0-79-generic #100~14.04.1-Ubuntu SMP Fri May 19 18:36:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
A DESCRIPTION OF THE PROBLEM :
This bug is related to the jdk.incubator.httpclient module.
Opening a websocket for the LXD event stream over TLS ( https://github.com/lxc/lxd/blob/master/doc/rest-api.md#10events ) causes the HttpClient's "SelectorManager" thread to spin indefinitely waiting for new data. The thread keeps receiving data for that web socket and calls onText() nicely, but it keeps the thread in a busy loop doing so. Later regular HTTP requests using the same HttpClient instance don't complete without harming the web socket's ability to keep reading.
The spinning loop is in SSLDelegate.EngineWrapper.recvAndUnwrap(), chan.read always returns 0, SocketChannelImpl.read() always returns -2/IOStatus.UNAVAILABLE, thus EGAIN for the underlying read(). The thread never leaves the recvAndUnwrap method and the target buffer "unwrap_src" is always empty. PlainHttpConnection configured the socket channel in non-blocking mode and no new data is expected, so getting EAGAIN is no surprise.
REGRESSION. Last worked in version 9
ADDITIONAL REGRESSION INFORMATION:
Earlier JDK9 EA releases didn't have this specific issue and partially worked for longer. They however had other problems like overflowing internal buffers later on (
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
I don't have a detached test case, however playing with LXD's API like creating containers, pushing/pulling data to containers, reading lengthy process exec API output etc. has reliably shown bugs in every httpclient version I've tried.
For this specific bug I'm doing the following, all sharing one HttpClient instance:
1. GET /
2. GET /1.0
3. Web socket (wss) for /1.0/events?type=operation,logging with a listener that just prints to stdout and keeps requesting exactly like the default interface methods
4. GET /1.0/containers (doesn't complete)
The http client instance uses a custom SSLContext that supplies a TLS client certificate and skips the usual server verifications. All GET requests use httpClient.sendAsync, String response body processors and a 10s timeout(). The web socket uses httpClient.newWebSocketBuilder, a 10s connectTimeout and buildAsync.
Each step is completed (CompletableFuture.get()) before the next step, for the web socket buildAsync().get(). Step 4 does send the data, but never completes its response future, presumably doesn't receive anything. Delaying step 4 doesn't help either.
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The web sockets keeps waiting for data normally without causing CPU load, follow-up use of the HttpClient instance works normally.
ACTUAL -
The SelectorManager thread uses all its CPU time actively and the HttpClient instance doesn't complete unrelated sendAsync requests.
REPRODUCIBILITY :
This bug can be reproduced always.