Opening this issue on behalf of Oli Gillespie (see https://mail.openjdk.java.net/pipermail/nio-dev/2022-May/011331.html ).

When registering a SocketChannel with a Selector for the first time,
it's possible to get a CancelledKeyException even though this is the
first register call.

Exception in thread "main" java.nio.channels.CancelledKeyException
    at java.base/sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:75)
    at java.base/sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:104)
    at java.base/sun.nio.ch.SelectorImpl.register(SelectorImpl.java:222)
    at java.base/java.nio.channels.spi.AbstractSelectableChannel.register(AbstractSelectableChannel.java:236)
    at java.base/java.nio.channels.SelectableChannel.register(SelectableChannel.java:260)
    at KeyCancelled.main(KeyCancelled.java:20)

The javadoc states:

@throws CancelledKeyException
    If this channel is currently registered with the given selector
      but the corresponding key has already been cancelled

However in this case the channel is apparently _not_ registered, as shown by
SocketChannel.isRegistered() returning false.

This following sequence triggers this issue:

1. Thread 1 starts SelectableChannel.register
2. A new SelectionKey becomes visible via Selector.keys()
3. Thread 2 iterates Selector.keys() and calls SelectorKey.cancel()
4. Thread 1 (still in the register() invocation) finds that the key is
cancelled and throws CancelledKeyException

Below is a small reproducer which usually exhibits this issue:

import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;

public class KeyCancelled {
    public static void main(String[] args) throws Exception {
        Selector s = Selector.open();

        new Thread(() -> {
            for (int i = 0; i < 100_000; i++) {
                s.keys().forEach(SelectionKey::cancel);
            }
        }).start();

        for (int i = 0; i < 10_000; i++) {
            SocketChannel c = s.provider().openSocketChannel();
            c.configureBlocking(false);
            // Sometimes this throws CancelledKeyException, because
the key is cancelled by
            // the other thread part-way through the register call.
            c.register(s, SelectionKey.OP_READ);
            // c.isRegistered() is false here after the exceptional case
        }
    }
}