FULL PRODUCT VERSION :
java version "9.0.4"
Java(TM) SE Runtime Environment (build 9.0.4+11)
Java HotSpot(TM) 64-Bit Server VM (build 9.0.4+11, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Linux localhost.localdomain 4.15.6-300.fc27.x86_64 #1 SMP Mon Feb 26 18:43:03 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

A DESCRIPTION OF THE PROBLEM :
Deserialization of an object fails under the following conditions:

    * The class is hashable, and the hash function is based on a serializable field.
    * The class has a subclass with a field of any hash container type (e.g. HashMap, HashSet, etc.).
    * The hash container in the subclass contains a circular reference to the object being deserialized.

This issue is very similar to 6208166. The difference being that, in 6208166, the circular reference was stored in the class itself, whereas in this issue, the circular reference is stored in a subclass.

This scenario, while admittedly pathological, worked prior to Java 9. Just as with 6208166, it appears there was an optimization in ObjectInputStream that facilitated this regression. Specifically, in ObjectInputStream#readSerialData(), the following comment suggests that, when possible, field values for *all* classes in a hierarchy are now read before setting the field values:

    // Best effort Failure Atomicity; slotValues will be non-null if field
    // values can be set after reading all field data in the hierarchy.
    // Field values can only be set after reading all data if there are no
    // user observable methods in the hierarchy, readObject(NoData). The
    // top most Serializable class in the hierarchy can be skipped.

In the Java 8 implementation, it appears that all field values for a single class were read, and the field values set, before moving on to the next class in the hierarchy. That permitted the above scenario to succeed because the field used in hashCode() was initialized prior to the self reference being inserted into the hash container.

REGRESSION. Last worked in version 8u162

ADDITIONAL REGRESSION INFORMATION:
java version "1.8.0_162"
Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
See attached test case. The test passes on JDK 1.8.0_162 and errors on JDK 9.0.4 with a NullPointerException.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The object should be deserialized without error.
ACTUAL -
A NullPointerException is thrown during deserialization when the hash container in the subclass calls hashCode() of the superclass at the time the object being deserialized is inserted into the hash container.

Alternatively, if hashCode() is written to avoid the NPE (e.g. using Objects#hash()), deserialization will appear to succeed. However, the object will have been stored in the wrong bucket within the hash container (because the hash code was calculated using a null field value) making future key lookups using that object fail.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
java.lang.NullPointerException
at org.triplea.DeserializationTest$Foo.hashCode(DeserializationTest.java:62)
at java.base/java.util.HashMap.hash(HashMap.java:339)
at java.base/java.util.HashMap.readObject(HashMap.java:1462)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at java.base/java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1160)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2207)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2078)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1585)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2346)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2240)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2078)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1585)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at org.triplea.DeserializationTest.shouldBeAbleToDeserializeWithCircularReferenceInDerivedClass(DeserializationTest.java:30)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:389)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:167)
at org.junit.jupiter.engine.execution.ThrowableCollector.execute(ThrowableCollector.java:40)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:163)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:110)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.lambda$execute$3(HierarchicalTestExecutor.java:83)
at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:77)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.lambda$null$2(HierarchicalTestExecutor.java:92)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:430)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.lambda$execute$3(HierarchicalTestExecutor.java:92)
at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:77)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.lambda$null$2(HierarchicalTestExecutor.java:92)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:430)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.lambda$execute$3(HierarchicalTestExecutor.java:92)
at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:77)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:51)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:43)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:170)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:154)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:90)
at org.eclipse.jdt.internal.junit5.runner.JUnit5TestReference.run(JUnit5TestReference.java:86)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:538)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:760)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:460)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:206)



REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
package org.triplea;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable;
import java.util.HashMap;
import java.util.Map;

import org.junit.jupiter.api.Test;

public final class DeserializationTest {
  @Test
  public void shouldBeAbleToDeserializeWithCircularReferenceInDerivedClass() throws Exception {
    final Bar bar1 = new Bar("one");
    final Bar bar2 = new Bar("two");
    bar1.map.put(bar1, 1);
    bar1.map.put(bar2, 2);

    final ByteArrayOutputStream baos = new ByteArrayOutputStream();
    try (ObjectOutputStream oos = new ObjectOutputStream(baos)) {
      oos.writeObject(bar1);
    }
    final Bar actualBar1;
    try (ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(baos.toByteArray()))) {
      actualBar1 = (Bar) ois.readObject();
    }

    assertEquals(bar1.name, actualBar1.name);
    assertEquals(2, actualBar1.map.size());
    assertTrue(actualBar1.map.containsKey(bar1));
    assertTrue(actualBar1.map.containsKey(bar2));
  }

  static class Foo implements Serializable {
    private static final long serialVersionUID = 6716343853564480063L;

    final String name;

    Foo(final String name) {
      this.name = name;
    }

    @Override
    public boolean equals(final Object obj) {
      if (obj == this) {
        return true;
      } else if (!(obj instanceof Foo)) {
        return false;
      }

      final Foo other = (Foo) obj;
      return name.equals(other.name);
    }

    @Override
    public int hashCode() {
      return name.hashCode();
    }
  }

  static final class Bar extends Foo {
    private static final long serialVersionUID = 9182581686880080478L;

    final Map<Foo, Integer> map = new HashMap<>();

    Bar(final String name) {
      super(name);
    }
  }
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
As the comment in ObjectInputStream#readSerialData() suggests, adding a readObject() method to the superclass forces the framework to set its field values before moving on to the subclass. Adding the following method to Foo in the test case will allow the test to pass on JDK 9:

    private void readObject(final ObjectInputStream in)
        throws IOException, ClassNotFoundException {
      in.defaultReadObject();
    }