daniel.indrigo@Canada 1999-03-02

One of our users has a client-server application that is running into server
problems with JDK 117 because of the addition of method readObject() in file
src/share/java/java/math/BigInteger.java.

The problem is that the server fails to verify a valid PublicKey sent to it
from the client. One of the BigIntegers in the PublicKey sent by the client
has a positive bitLength value (eg. 510). But, method BigInteger.readObject()
resets the bitLength field to -1. This difference causes Signature.verify()
to fail. (Signature.verify() calls DSA.engineVerify(), which calls
DSA.generateV(), which calls BigInteger.modPow(). BigInteger.modPow() returns
a different result because of the different value of the BigInteger's bitLength
field.)

Why does BigInteger.readObject() reset the bitCount, bitLength, lowestSetBit,
and firstNonzeroByteNum ? Shouldn't it just keep the values for them that
were read in ?

Below is the source code for BigInteger.readObject(). The problem is caused
by the line: bitCount = bitLength = -1;


    /**
     * Reconstitute the <tt>BigInteger</tt> instance from a stream (that is,
     * deserialize it).
     */
    private synchronized void readObject(java.io.ObjectInputStream s)
        throws java.io.IOException, ClassNotFoundException {
        // Read in all fields
        s.defaultReadObject();

        // Defensively copy magnitude to ensure immutability
        magnitude = (byte[]) magnitude.clone();

        // Validate signum
        if (signum < -1 || signum > 1)
            throw new java.io.StreamCorruptedException(
                        "BigInteger: Invalid signum value");
        if ((magnitude.length==0) != (signum==0))
            throw new java.io.StreamCorruptedException(
                        "BigInteger: signum-magnitude mismatch");

        // Set "cached computation" fields to their initial values
        bitCount = bitLength = -1;
        lowestSetBit = firstNonzeroByteNum = -2;
    }