For jtreg b13 one fix was to "use timeoutFactor on the agent communication thread ". This means the SO_TIMEOUT for sockets has been extended based on the timeoutFactor.
However the cleanup timeout was not changed. So the cleanup code does a thread.interrupt() (which won't unblock a thread reading from a socket) and then does a join(timeout) before deciding it was unable to cleanup threads. Because the socket threads are now potentially blocking for much longer before timing out, the cleanup code doesn't wait long enough and so reports error.
Example from a test with a timeoutFactor of 10 applied:
----------System.err:(55/3297)----------
JavaTest Message: Test complete.
JavaTest Message: Problem cleaning up the following threads:
Thread-37
at java.base@12-internal/java.net.SocketInputStream.socketRead0(Native Method)
at java.base@12-internal/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:140)
at java.base@12-internal/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
at java.base@12-internal/java.io.BufferedInputStream.read(BufferedInputStream.java:271)
at Worker.readUntil(MultiThreadTest.java:255)
at Worker.run(MultiThreadTest.java:285)
Thread-38
at java.base@12-internal/java.net.SocketInputStream.socketRead0(Native Method)
at java.base@12-internal/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:140)
at java.base@12-internal/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
at java.base@12-internal/java.io.BufferedInputStream.read(BufferedInputStream.java:271)
at Worker.readUntil(MultiThreadTest.java:255)
at Worker.run(MultiThreadTest.java:285)
Thread-39
at java.base@12-internal/java.net.SocketInputStream.socketRead0(Native Method)
at java.base@12-internal/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:140)
at java.base@12-internal/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
at java.base@12-internal/java.io.BufferedInputStream.read(BufferedInputStream.java:271)
at Worker.readUntil(MultiThreadTest.java:255)
at Worker.run(MultiThreadTest.java:285)
Thread-40
at java.base@12-internal/java.net.SocketInputStream.socketRead0(Native Method)
at java.base@12-internal/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:140)
at java.base@12-internal/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
at java.base@12-internal/java.io.BufferedInputStream.read(BufferedInputStream.java:271)
at Worker.readUntil(MultiThreadTest.java:255)
at Worker.run(MultiThreadTest.java:285)
Thread-41
at java.base@12-internal/java.net.SocketInputStream.socketRead0(Native Method)
at java.base@12-internal/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:140)
at java.base@12-internal/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
at java.base@12-internal/java.io.BufferedInputStream.read(BufferedInputStream.java:271)
at Worker.readUntil(MultiThreadTest.java:255)
at Worker.run(MultiThreadTest.java:285)
----------System.out:(1/42)----------
Connection count = 5 Request count = 100
result: Error. Error while cleaning up threads after test
A better solution, if feasible, would be to close the port/socket that the threads are blocked on so that they will be able to terminate as soon as no longer needed, rather than having to wait for the SO_TIMEOUT to elapse.
However the cleanup timeout was not changed. So the cleanup code does a thread.interrupt() (which won't unblock a thread reading from a socket) and then does a join(timeout) before deciding it was unable to cleanup threads. Because the socket threads are now potentially blocking for much longer before timing out, the cleanup code doesn't wait long enough and so reports error.
Example from a test with a timeoutFactor of 10 applied:
----------System.err:(55/3297)----------
JavaTest Message: Test complete.
JavaTest Message: Problem cleaning up the following threads:
Thread-37
at java.base@12-internal/java.net.SocketInputStream.socketRead0(Native Method)
at java.base@12-internal/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:140)
at java.base@12-internal/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
at java.base@12-internal/java.io.BufferedInputStream.read(BufferedInputStream.java:271)
at Worker.readUntil(MultiThreadTest.java:255)
at Worker.run(MultiThreadTest.java:285)
Thread-38
at java.base@12-internal/java.net.SocketInputStream.socketRead0(Native Method)
at java.base@12-internal/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:140)
at java.base@12-internal/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
at java.base@12-internal/java.io.BufferedInputStream.read(BufferedInputStream.java:271)
at Worker.readUntil(MultiThreadTest.java:255)
at Worker.run(MultiThreadTest.java:285)
Thread-39
at java.base@12-internal/java.net.SocketInputStream.socketRead0(Native Method)
at java.base@12-internal/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:140)
at java.base@12-internal/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
at java.base@12-internal/java.io.BufferedInputStream.read(BufferedInputStream.java:271)
at Worker.readUntil(MultiThreadTest.java:255)
at Worker.run(MultiThreadTest.java:285)
Thread-40
at java.base@12-internal/java.net.SocketInputStream.socketRead0(Native Method)
at java.base@12-internal/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:140)
at java.base@12-internal/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
at java.base@12-internal/java.io.BufferedInputStream.read(BufferedInputStream.java:271)
at Worker.readUntil(MultiThreadTest.java:255)
at Worker.run(MultiThreadTest.java:285)
Thread-41
at java.base@12-internal/java.net.SocketInputStream.socketRead0(Native Method)
at java.base@12-internal/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.base@12-internal/java.net.SocketInputStream.read(SocketInputStream.java:140)
at java.base@12-internal/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
at java.base@12-internal/java.io.BufferedInputStream.read(BufferedInputStream.java:271)
at Worker.readUntil(MultiThreadTest.java:255)
at Worker.run(MultiThreadTest.java:285)
----------System.out:(1/42)----------
Connection count = 5 Request count = 100
result: Error. Error while cleaning up threads after test
A better solution, if feasible, would be to close the port/socket that the threads are blocked on so that they will be able to terminate as soon as no longer needed, rather than having to wait for the SO_TIMEOUT to elapse.
- relates to
-
JDK-8208690 network tests failing due to "JavaTest Message: Problem cleaning up the following threads"
- Resolved