Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8277965

Enclosing instance optimization affects serialization

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: P2 P2
    • 18
    • None
    • tools
    • None
    • b27
    • Verified

      JDK-8271623 (omit enclosing instance fields from inner classes that don't use it) has impacts on serialization that were not anticipated and discussed during the review of the CSR (JDK-8271717) and implementation.

      This bug tracks mitigating those affects, either by backing out the change, disabling it for serializable classes, or deciding that the current behaviour is correct.

      The original change causes javac to omit the synthetic `this$0` field from inner classes that do not access their enclosing instance.

      Based on review feedback, the optimization is only applied to serializable classes that define a serialVersionUID, since omitting the synthetic field would affect default serialization:
      https://github.com/openjdk/jdk/pull/4966#issuecomment-967449448

      For classes that define a serialVersionUID, this change affects compatibility if an inner class that does not use this$0 is recompiled to use this$0, and an old form is deserialized, since it would see a null value for this$0. (There's a demo showing a deserialized class with a null this$0 field below.)

       That seems like the kind of incompatibility discussed in the Java Object Serialization Specification, in the note about how serialization of inner classes is 'strongly discouraged' in part because:

      > Synthetic fields generated by javac (or other JavaTM compilers) to implement inner classes are implementation dependent and may vary between compilers; differences in such fields can disrupt compatibility

      The specification also suggests that the synthetic field will exist, although in context (warning that serializing inner classes is a bad idea), this can be read as disclaiming that this can happen, not necessarily guaranteeing it to always happen in a stable and predictable way:

      > inner classes declared in non-static contexts contain implicit non-transient references to enclosing class instances, serializing such an inner class instance will result in serialization of its associated outer class instance as well

      https://docs.oracle.com/en/java/javase/17/docs/specs/serialization/serial-arch.html

      ```
      import java.io.IOException;
      import java.io.InputStream;
      import java.io.ObjectInputStream;
      import java.io.ObjectOutputStream;
      import java.io.OutputStream;
      import java.io.Serializable;
      import java.lang.reflect.Field;
      import java.nio.file.Files;
      import java.nio.file.Paths;

      public class X implements Serializable {

        private static final long serialVersionUID = 0;

        class Inner implements Serializable {

          private static final long serialVersionUID = 0;

          void f() {
            for (Field f : getClass().getDeclaredFields()) {
              f.setAccessible(true);
              try {
                System.err.println(f + ": " + f.get(this));
              } catch (IllegalAccessException e) {
                throw new LinkageError(e.getMessage(), e);
              }
            }
          }
        }

        public static void main(String[] args) throws IOException, ClassNotFoundException {
          Inner i;
          switch (args[0]) {
            case "write":
              try (OutputStream os = Files.newOutputStream(Paths.get("data"));
                  ObjectOutputStream oos = new ObjectOutputStream(os)) {
                i = new X().new Inner();
                oos.writeObject(i);
              }
              break;
            case "read":
              try (InputStream is = Files.newInputStream(Paths.get("data"));
                  ObjectInputStream oos = new ObjectInputStream(is)) {
                i = (Inner) oos.readObject();
              }
              break;
            default:
              throw new AssertionError(args[0]);
          }
          i.f();
        }
      }
      ```

      $ javac --release 11 -XDoptimizeOuterThis=true X.java
      $ java X write
      private static final long X$Inner.serialVersionUID: 0
      $ javac --release 11 X.java
      $ java X read
      private static final long X$Inner.serialVersionUID: 0
      final X X$Inner.this$0: null

      $ javac --release 11 X.java
      $ java X write
      private static final long X$Inner.serialVersionUID: 0
      final X X$Inner.this$0: X@3a03464
      $ javac --release 11 -XDoptimizeOuterThis=true X.java
      $ java X read
      private static final long X$Inner.serialVersionUID: 0

            cushon Liam Miller-Cushon
            cushon Liam Miller-Cushon
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: