This is noticeable on machines with many cores/HW threads, such as 2-socket Xeon Platinum 8176 (112) or 2-socket ThunderX2 (224). It may not be apparent on small systems.

We (primarily Andrey Sudnik) tracked down a performance regression introduced in JDK 9 to the addition ObjectInputFilter support (even when filtering is not enabled).

The issue is the new code in the ObjectInputStream constructor:

     serialFilter = ObjectInputFilter.Config.getSerialFilter();

   public static ObjectInputFilter getSerialFilter() {
            synchronized (serialFilterLock) {
                return serialFilter;
            }
        }
There is a global lock (serialFilterLock) that is held very briefly as part of creating ObjectInputStreams. In SPECjbb this happens when every transaction is received. The filter is never set in SPECjbb.

My theory:
On a single-core machine, there would be essentially never be contention on that lock, so the fastest lock path will be taken. But with more and more cores, at some point there will be contention and the lock gets inflated. In this case, even if contention was unlikely, the lock is taking a slower “fast-path”, and this is increasing the chance of contention. So there’s a performance cliff at some point.

I believe on large-core machines such as ThunderX2 or Xeon Platinum we are falling over that cliff.

Suggested fix:
I don’t think that getSerialFilter() needs to be synchronized after all:
- setSerialFilter() ensures that the value of serialFilter is either null or non-null, and it can only be set once.
- There are not multiple states that need to be kept synchronized.
- An ObjectInputStream created around the time of the setSerialFilter call may or may not see the new value of the serialFilter (with or without current synchronization). If it was important that a new ObjectInputStream would never see a stale null value for serialFilter, a lock would have to be used at a higher level.
- I think making the serialFilter field volatile should be sufficient, so the readers will eventually see a new value.