Correct scale handling of BigDecimal.sqrt

XMLWordPrintable

    • Type: CSR
    • Resolution: Approved
    • Priority: P4
    • 26
    • Component/s: core-libs
    • None
    • behavioral
    • low
    • Behavior changed in some cases to return a numerically equal value with a different representation.
    • Java API
    • SE

      Summary

      Change the preferred scale of BigDecimal.sqrt to align with IEEE 754.

      Problem

      Due to an oversight when BigDecimal.sqrt was first added to the platform (JDK-4851777 ), the preferred scale was defined as floor(argumentScale / 2.0) rather than ceil(argumentScale / 2.0).

      IEEE 754 specified preferred exponents where the exponent is the negated scale. Therefore, since IEEE 754 specifies floor(argumentScale / 2.0) the correct translation for BigDecimal is ceil(argumentScale / 2.0). The computations are equivalent for even scales/exponents. In the case of odd scale/exponents, the result differs by one.

      Solution

      Change the specification to require the intended scale.

      This change does have some nonzero behavioral compatibility impact; however, there are a number of mitigating factors:

      • Before and after this change, numerically equal results will be returned. If the results differ, they will only differ in their representation.
      • If the scale of the argument is even, the preferred scale is the same before and after this change.
      • The scale is only preferred and the scale of the result is also a function of other factors, including the precision being used for the computation.

      Specification

      --- a/src/java.base/share/classes/java/math/BigDecimal.java
      +++ b/src/java.base/share/classes/java/math/BigDecimal.java
      @@ -154,7 +154,7 @@
        * <tr><th scope="row">Subtract</th><td>max(minuend.scale(), subtrahend.scale())</td>
        * <tr><th scope="row">Multiply</th><td>multiplier.scale() + multiplicand.scale()</td>
        * <tr><th scope="row">Divide</th><td>dividend.scale() - divisor.scale()</td>
      - * <tr><th scope="row">Square root</th><td>radicand.scale()/2</td>
      + * <tr><th scope="row">Square root</th><td>ceil(radicand.scale()/2.0)</td>
        * </tbody>
        * </table>
        *
      @@ -2113,7 +2113,7 @@
            * with rounding according to the context settings.
            *
            * <p>The preferred scale of the returned result is equal to
      -     * {@code this.scale()/2}. The value of the returned result is
      +     * {@code Math.ceilDiv(this.scale(), 2)}. The value of the returned result is
            * always within one ulp of the exact decimal value for the
            * precision in question.  If the rounding mode is {@link
            * RoundingMode#HALF_UP HALF_UP}, {@link RoundingMode#HALF_DOWN
      

            Assignee:
            Joe Darcy
            Reporter:
            Joe Darcy
            Raffaello Giulietti
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: