JIT/x86 suffers more than one rounding error in FP-strict addition

Name: dkC59003 Date: 09/23/98



JITs for x86 in JDK 1.2fcs-J and -K suffer more than one rounding error when
computing the test_value in the example below.

Correct result for double test_value must be slightly greater than 1. The tiny
delta over 1 vanishes, if test_value is incorrectly calculated using extended
floating-point format suffering second rounding error when rounding test_value
to standard format.

--------------------------------------------------------------------------------

public class fpl352m24r {

    strictfp
    public static void main (String args[]) {

double delta = (double)((1L<<11) + 1)/(1L<<11)/(1L<<53);

double test_value = 1 + delta;

double correct_value = Double.longBitsToDouble(0x3ff0000000000001L);

if (test_value != correct_value) {
System.out.println("test_value = " + test_value);
System.out.println("instead of: " + correct_value +
" == Double.longBitsToDouble(0x3ffe000000000001)");

};
    };
}

---- Prints on both fcs-J and fcs-K under x86:

test_value = 1.0
instead of: 1.0000000000000002 == Double.longBitsToDouble(0x3ffe000000000001)

--------------------------------------------------------------------------------

This test passes whith no JIT (started by: java -Djava.compiler=none ...).
Also it passes under solaris both with and without JIT under both fcs-J & -K.

So, it seems like JITs used by JDK 1.2fcs-J and -K on x86 incorrectly exploit
Intel's 80-bits FPU to calculate some FP-strict expressions.

This bug affects JCK 1.2beta4 tests lang/FP/fpl03502m2 and lang/FP/fpl03502m4,
which check if compile-time constant expressions are treated FP-strict.

======================================================================

Name: dkC59003 Date: 09/23/98



Sample program below adds two double values in strictfp method. Exact sum of ones
has mantissa with rounded part near (but below) ulp, so it must be rounded to low
representable value. Log shows that sum is rounded to upper, possibly because of
double (twice) rounding the Intel 80-bit double-extended result.

On sol/x86 sample fails also with JIT JDK-1.2fcs-K and runs correctly under
JDK 1.2fcsJ and K with interpreter. It also works correctly on Sun SPARC (jit/nojit).

This bug affects the following tests in JCK 1.2beta4:
lang/FP/fpl003/fpl00301m1/fpl00301m1.html
lang/FP/fpl003/fpl00301m2/fpl00301m2.html
lang/FP/fpl003/fpl00301m3/fpl00301m3.html
lang/FP/fpl003/fpl00301m4/fpl00301m4.html
lang/FP/fpl003/fpl00301m5/fpl00301m5.html
lang/FP/fpl003/fpl00301m6/fpl00301m6.html
lang/FP/fpl003/fpl00301m7/fpl00301m7.html
lang/FP/fpl003/fpl00301m8/fpl00301m8.html
lang/FP/fpl003/fpl00301m9/fpl00301m9.html
lang/FP/fpl003/fpl00301m10/fpl00301m10.html
lang/FP/fpl003/fpl00301m11/fpl00301m11.html
lang/FP/fpl003/fpl00301m12/fpl00301m12.html
lang/FP/fpl003/fpl00301m13/fpl00301m13.html
lang/FP/fpl003/fpl00301m14/fpl00301m14.html
lang/FP/fpl003/fpl00301m15/fpl00301m15.html
lang/FP/fpl003/fpl00301m16/fpl00301m16.html
lang/FP/fpl003/fpl00301m17/fpl00301m17.html
lang/FP/fpl003/fpl00301m18/fpl00301m18.html
lang/FP/fpl003/fpl00301m19/fpl00301m19.html
lang/FP/fpl003/fpl00301m20/fpl00301m20.html
lang/FP/fpl003/fpl00301m21/fpl00301m21.html
lang/FP/fpl003/fpl00301m22/fpl00301m22.html
lang/FP/fpl003/fpl00301m23/fpl00301m23.html
lang/FP/fpl003/fpl00301m24/fpl00301m24.html
lang/FP/fpl003/fpl00301m25/fpl00301m25.html
lang/FP/fpl003/fpl00301m26/fpl00301m26.html


========================================= fpl00301m11.java

import java.io.PrintStream;

public strictfp class fpl00301m11 {
   
    public static strictfp void main(String argv[]) {
         double dx = (1L<<53)-1; // 1-53-1.0 ; full mantissa
         double dy = dx/(1L<<54); // 0.01-53-1 ; <0.5 -> sum must be rounded to dx
         if ( (dx + dy) - dx != 0 ) {
             System.out.println("FP-strict sum of double diffs by "+((dx + dy) - dx));
             return;
         }
         System.out.println("Pass. " );
    }
}

========================================= log
___ /export/ld14/java/dest/jdk1.2fcsJ/x86/bin/java fpl00301m11
FP-strict sum of double diffs by 1.0

___ /export/ld14/java/dest/jdk1.2fcsJ/x86/bin/java -version
java version "1.2fcs"
Classic VM (build JDK-1.2fcs-J, green threads, sunwjit)
___ uname -a
SunOS novo91 5.6 Generic i86pc i386 i86pc

=========================================

======================================================================