FULL PRODUCT VERSION :
java version "1.7.0_111"
OpenJDK Runtime Environment (amzn-2.6.7.2.68.amzn1-x86_64 u111-b01)
OpenJDK 64-Bit Server VM (build 24.111-b01, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Darwin Kernel Version 15.6.0

EXTRA RELEVANT SYSTEM CONFIGURATION :
the file handle limit (ulimit -n, or nofile in /etc/security/limits.conf) was set to 64 * 1024 = 65536

A DESCRIPTION OF THE PROBLEM :
The class sun.nio.ch.EPollArrayWrapper initializes an "eventsHigh" HashMap iff the system file descriptor limit exceeds 65536 (see below):

private static final int OPEN_MAX = IOUtil.fdLimit();
private static final int MAX_UPDATE_ARRAY_SIZE = AccessController.doPrivileged(new GetIntegerAction("sun.nio.ch.maxUpdateArraySize", Math.min(OPEN_MAX, 64*1024)));

// eventHigh needed when using file descriptors > 64k
if (OPEN_MAX > MAX_UPDATE_ARRAY_SIZE) {
    eventsHigh = new HashMap<>();
}

However, the method setUpdateEvents invokes isEventsHighKilled, which dereferences eventsHigh for file descriptors >= 65536. This means that in the case that a file descriptor passed to setUpdateEvents is equal to *exactly* 65536 on a system with a ulimit -n of 65536 a NPE is thrown, because eventsHigh has not been initialized. This results in, for example, the following stack trace from our application logs:

09 Sep 2016 16:02:13,809 [WARN] (nioEventLoopGroup-5-1) io.netty.bootstrap.ServerBootstrap: Failed to register an accepted channel: [id: 0x3ae71e56, 0.0.0.0/0.0.0.0:8080]
java.lang.NullPointerException
at sun.nio.ch.EPollArrayWrapper.isEventsHighKilled(EPollArrayWrapper.java:174)
at sun.nio.ch.EPollArrayWrapper.setUpdateEvents(EPollArrayWrapper.java:190)
at sun.nio.ch.EPollArrayWrapper.add(EPollArrayWrapper.java:239)
at sun.nio.ch.EPollSelectorImpl.implRegister(EPollSelectorImpl.java:178)
at sun.nio.ch.SelectorImpl.register(SelectorImpl.java:132)
...




 



private byte getUpdateEvents(int fd) {
    if (fd < MAX_UPDATE_ARRAY_SIZE) {
        return eventsLow[fd];
    } else {
        Byte result = eventsHigh.get(Integer.valueOf(fd));

         // result should never be null
         return result.byteValue();
    }
}

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
-set system ulimit -n to 65536
-instantiate an sun.nio.ch.EPollArrayWrapper
-invoke setUpdateEvents with fd = 65536

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
I expect a NPE when dereferencing eventsHigh
ACTUAL -
I got an NPE when dereferencing eventsHigh

ERROR MESSAGES/STACK TRACES THAT OCCUR :
see description.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
reproducing this is dependent upon system configuration, and cannot be reproduced in unit tests (the class is not written so the system calls that determine file limits can be mocked out or have a supplier injected into them, which is not great)


---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
don't set ulimit -n to exactly 65336