By running EEJS benchmark, I noticed client side throws IOException creating socket.

By looking at the jstack for worker threads in the server side, I noticed a lot of threads are blocked in sun.security.ssl.CipherSuite$BulkCipher.isAvailable(), and some other threads are blocked in
sun.security.ssl.CipherSuite$BulkCipher.clearAvailableCache(). You can find the jstacks in the bottom.

For each cipher suite, the method that checks the availability of the bulk cipher is synchronized, which causes the threads blocked.

In the current implementation, when getting the default cipher suite list, it iterates over the cipher suite list and for each cipher suite, it checks its availability. Inside sun.security.ssl.CipherSuite.java, there is an availableCache, which maps the bulk cipher and its availability.

However when getting the default cipher suite list, this availableCache is cleared each time. Thus when checking the availability of the each bulk cipher, this availableCache is not used at all, and it will execute this synchronized CipherSuite$BulkCipher.isAvaiable() method each time, which hurts performance a lot.

I deleted the clearAvailableCache() call inside getDefaultCipherSuiteList(). I run the benchmark and there is no error and no more blocking. The response time goes down to 0.979s from 1.389s for 1000 users. And with this change, I run the benchmark with 1300 users, the response time is 1.283s.

A part of the jstack is shown like this:
"[ACTIVE] ExecuteThread: '126' for queue: 'weblogic.kernel.Default
(self-tuning)'" #165 daemon prio=5 os_prio=64 tid=0x0000000108654800
nid=0xbe waiting for monitor entry [0xffffffff07efd000]
    java.lang.Thread.State: BLOCKED (on object monitor)
         at
sun.security.ssl.CipherSuite$BulkCipher.isAvailable(CipherSuite.java:542)
         - waiting to lock <0x00000003c43a9320> (a java.lang.Class for
sun.security.ssl.CipherSuite$BulkCipher)
         at
sun.security.ssl.CipherSuite$BulkCipher.isAvailable(CipherSuite.java:527)
         at sun.security.ssl.CipherSuite.isAvailable(CipherSuite.java:194)
         at
sun.security.ssl.SSLContextImpl.getApplicableCipherSuiteList(SSLContextImpl.java:346)
         at
sun.security.ssl.SSLContextImpl.getDefaultCipherSuiteList(SSLContextImpl.java:304)
         - locked <0x00000005c481bf40> (a
sun.security.ssl.SSLContextImpl$TLSContext)
         at sun.security.ssl.SSLSocketImpl.init(SSLSocketImpl.java:614)
         at sun.security.ssl.SSLSocketImpl.<init>(SSLSocketImpl.java:555)
         at
sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:110)
         at
org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:364)


"[ACTIVE] ExecuteThread: '125' for queue: 'weblogic.kernel.Default
(self-tuning)'" #164 daemon prio=5 os_prio=64 tid=0x0000000108655800
nid=0xbd waiting for monitor entry [0xffffffff080fe000]
    java.lang.Thread.State: BLOCKED (on object monitor)
         at
sun.security.ssl.CipherSuite$BulkCipher.clearAvailableCache(CipherSuite.java:537)
         - waiting to lock <0x00000003c43a9320> (a java.lang.Class for
sun.security.ssl.CipherSuite$BulkCipher)
         at
sun.security.ssl.SSLContextImpl.clearAvailableCache(SSLContextImpl.java:386)
         at
sun.security.ssl.SSLContextImpl.getDefaultCipherSuiteList(SSLContextImpl.java:293)
         - locked <0x00000005c44d3328> (a
sun.security.ssl.SSLContextImpl$TLSContext)
         at sun.security.ssl.SSLSocketImpl.init(SSLSocketImpl.java:614)
         at sun.security.ssl.SSLSocketImpl.<init>(SSLSocketImpl.java:555)
         at
sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:110)
         at
org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:364)