Summary

For a corrupted object input stream with a negative array size, ObjectInputStream::readObject() throws a NegativeArraySizeException. This is unexpected because the case of a negative array length is neither mentioned in the "Java Object Serialization Specification" nor in the API documentation of ObjectInputStream::readObject(). Better throw a StreamCorruptedException with appropriate exception message.

Problem

Currently, ObjectInputStream::readObject() doesn't explicitly check for a negative array length in the object input stream. Instead, it calls j.l.r.Array::newInstance(..) with the length read from the stream which results in a NegativeArraySizeException if the length is negative. However, NegativeArraySizeException is an unchecked exception which is neither declared in the signature of ObjectInputStream::readObject() nor mentioned in its API specification. It is therefore not obvious for users of ObjectInputStream::readObject() that they have to handle NegativeArraySizeExceptions. It would therefor be better if a negative array length in the object input stream would be automatically wrapped in an StreamCorruptedException which is a checked exception (derived from IOException via ObjectStreamException) and declared in the signature of ObjectInputStream::readObject().

The current behavior also breaks the contract of ObjectInputFilter.FilterInfo::arrayLength() which is used for deserialization filtering and defined to only return a "non-negative number of array elements" when deserializing an array, because a potential deserialization filter will be applied before j.l.r.Array::newInstance(..) is called and therefor reports the raw array length as read from the object input stream.

Solution

After reading the array length from the object input stream, check for a negative value before applying any filters and throw an StreamCorruptedException with appropriate exception message if the length is negative.

For consistency, also update the private ObjectInputStream::checkArray(Class<?> arrayType, int arrayLength) method to throw a StreamCorruptedException instead of a NegativeArraySizeException if arrayLength is negative. ObjectInputStream::checkArray() is called from various collections classes in their custom readObject() methods to verify that the number of their elements read from the deserialization stream is not negative. Like with ObjectInputStream::readObject() the original behavior can lead to unexpected NegativeArraySizeException during the deserialization of a corrupted object stream.

Specification

No specification changes as the behavior isn't specified neither in the in the "Java Object Serialization Specification" nor in the API documentation of ObjectInputStream::readObject().

The actual code changes are trivial:

--- a/src/java.base/share/classes/java/io/ObjectInputStream.java
+++ b/src/java.base/share/classes/java/io/ObjectInputStream.java
@@ -1423,16 +1423,16 @@ public class ObjectInputStream
      * @param arrayLength the array length
      * @throws NullPointerException if arrayType is null
      * @throws IllegalArgumentException if arrayType isn't actually an array type
-     * @throws NegativeArraySizeException if arrayLength is negative
+     * @throws StreamCorruptedException if arrayLength is negative
      * @throws InvalidClassException if the filter rejects creation
      */
-    private void checkArray(Class<?> arrayType, int arrayLength) throws InvalidClassException {
+    private void checkArray(Class<?> arrayType, int arrayLength) throws ObjectStreamException {
         if (! arrayType.isArray()) {
             throw new IllegalArgumentException("not an array type");
         }

         if (arrayLength < 0) {
-            throw new NegativeArraySizeException();
+            throw new StreamCorruptedException("Array length is negative");
         }

         filterCheck(arrayType, arrayLength);
@@ -2111,7 +2111,9 @@ public class ObjectInputStream

         ObjectStreamClass desc = readClassDesc(false);
         int len = bin.readInt();
-
+        if (len < 0) {
+            throw new StreamCorruptedException("Array length is negative");
+        }                                                                                                                
         filterCheck(arrayType, arrayLength);

The full change together with a jtreg test can be found in the pull request for the corresponding issue: JDK-8306461: ObjectInputStream::readObject() should handle negative array sizes without throwing NegativeArraySizeExceptions.