Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8272541

Incorrect overflow test in Toom-Cook branch of BigInteger multiplication

XMLWordPrintable

    • b13
    • generic
    • generic
    • Verified

        In this code

        if (bitLength(mag, mag.length) +
           bitLength(val.mag, val.mag.length) >
           32L*MAX_MAG_LENGTH) {
           reportOverflow();
        }

        In the above, bitLength() returns 'int', hence the left operand of the '>' comparison is 'int' too, but the right operand of the '>' comparison is 'long' with value == Integer.MAX_VALUE + 1 hence this condition is always false.

              bpb Brian Burkhalter
              bpb Brian Burkhalter
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: