20% regression in sci_32bit-lpages_Monte in build s10u9_01
sci_32bit-lpages_Monte :

--------------------------------------------------------------------------------
Historical sci_32bit-lpages_Monte results on oaf403_64
--------------------------------------------------------------------------------
Build Result Status Description Related Bugs
--------------------------------------------------------------------------------
s10u9_01 57.6 RED 20% regression
s10u8_08a 71.97 GREEN unchanged
s10u8_08
s10u8_07 71.98 GREEN unchanged
s10u8_06
s10u8_05 71.96 GREEN unchanged
s10u8_04 71.97 GREEN unchanged
s10u8_03 71.96 GREEN unchanged
s10u8_02 ~ YELLOW Comparison not statistically significant6858135
s10u8_01 ~ YELLOW Comparison not statistically significant
s10u7_08 72 GREEN unchanged
s10u7_07 72.02 GREEN unchanged
s10u7_06 72 GREEN unchanged
s10u7_05 71.99 GREEN unchanged
s10u7_04 72.01 GREEN unchanged
s10u7_03
s10u7_02 71.96 GREEN unchanged
s10u7_01
s10u6_07b 71.97 GREEN unchanged
s10u6_07a 71.95 GREEN unchanged
s10u6_07
s10u6_06
s10u6_05 71.93 GREEN unchanged
s10u6_04
s10u6_03
s10u6_02 71.94 GREEN unchanged
s10u6_01
s10u5_10 71.98 GREEN unchanged
s10u5_09
s10u5_08
s10u5_07 71.97 GREEN unchanged
s10u5_06
s10u5_05 71.93 GREEN unchanged
s10u5_05 71.93 GREEN unchanged
s10u5_04
s10u5_03 71.96 First Result

Using s10u8_08a as baseline
13 baselines. Average is 71.969 PctStDevis 0.00%
Plot_mean: 57.60 , overall change: 19.97 %
Iteration Result Change Colour
1 57.591 -19.98 % RED
2 57.579 -20.00 % RED
3 57.604 -19.96 % RED
4 57.604 -19.96 % RED
5 57.604 -19.96 % RED
6 57.604 -19.96 % RED
7 57.604 -19.96 % RED
Number of runs: 7
With all numbers shown, overall change will be -19.97%
Previous build:s10u8_08a 71.974 0.01 % YELLOW
The found results are: 1 YELLOW, 7 RED including the old result
Std. Deviation as % from Mean is 0.02
TTest against baseline: 0.0000000000

Outliers
=========
Can't find the machine in /etc/bootparams.
Benchmark res_range: 10:-1 (not ship)
Driver Version changes...
  -scsi_vhci SCSI VHCI Driver 1.57 (1.57 -> 1.60)
  -cmlb Common Labeling module 1.7 (1.7 -> 1.8)
  -consconfig console configuration 1.35 (1.35 -> 1.36)
  -consconfig_dacf Consconfig DACF 1.36 (1.36 -> 1.37)
  -mpt MPT HBA Driver v1.92 (1.92 -> 1.96)
  -sd SCSI Disk Driver 1.547 (1.547 -> 1.550)
  -st SCSI tape Driver 1.255 (1.255 -> 1.256)


Detailed statistical comparison of results is available at:
http://perfwww.ireland/cgi-bin/detail.cgi?rig=oaf403_64&benchmark=sci_32bit-lpages_Monte&build=s10u9_01&base=s10u8_08a&clean=FALSE

Using s10_74L2a as baseline
No Baselines
Plot_mean: 57.60 , overall change: 100.00 %
Warning: plot_per is 0.00, but overall change is 100.00. Will update later.
Iteration Result Change Colour
1 57.591
2 57.579
3 57.604
4 57.604
5 57.604
6 57.604
7 57.604
Number of runs: 7
With all numbers shown, overall change will be 0.00%
Previous build:s10u8_08a 71.974
Std. Deviation as % from Mean is 0.02
TTest against baseline: 1.0000000000

Outliers
=========
Can't find the machine in /etc/bootparams.
Benchmark res_range: 10:-1 (not ship)

Detailed statistical comparison of results is available at:
http://perfwww.ireland/cgi-bin/detail.cgi?rig=oaf403_64&benchmark=sci_32bit-lpages_Monte&build=s10u9_01&base=s10_74L2a&clean=FALSE

http://perfwww.ireland/cgi-bin/publictools/shipII.cgi?&BUILD=s10u9&BNAMES=sci_32bit-lpages_Monte&tSysteM=oaf403_64

The machine is a :X3220 1soc 4 x 2.4GHz 4 GB ram
The results are in for a run of s10u8_08a with the java packages from s10u9_01.
The regression in s10u9_01 is being introduced by the upgrade of the default java version
from 1.5.0_20 to 1.5.0_23.

mean change% num stddev% build
71.97 0.01% 13 s10u8_08a
57.60 -19.97% 7 0.02 s10u9_01
56.46 -21.55% 3 0.01 s10u8_08a-6919117-java-s10u9_01

Baseline used: s10u8_08a


he s10u8_08a-6919117-java-s10u9_01 run here is a fresh install of s10u8_08a with these packages
from s10u9_01: SUNWj6rt SUNWj6rtx SUNWj6cfg SUNWj5rt SUNWj5rtx SUNWj5cfg
So it looks like this regression is being introduced by the upgrade in the java version.
Looking at the other scimark metrics we are seeing regressions in

s10u8_08a -> s10u9_01
Mean Change #-res StdDev% Metric
57.59 -20.01% 7 0.02 sci_32bit_Monte
57.60 -19.97% 7 0.02 sci_32bit-lpages_Monte
47.41 -13.53% 7 0.39 sci_32bit_FFT
47.32 -13.28% 7 1.13 sci_32bit-lpages_FFT
535.73 -2.18% 7 2.06 sci_64bit_Sparse
513.95 -2.12% 7 0.07 sci_32bit-lpages_Sparse
51.21 -1.42% 8 0.71 sci_64bit-lpages_FFT
466.42 -1.36% 7 0.07 sci_32bit-lpages_Composite
466.11 -1.12% 7 0.07 sci_32bit_Composite
51.42 -1.11% 7 0.28 sci_64bit_FFT
513.91 -0.65% 7 0.09 sci_32bit_Sparse
807.03 -0.23% 7 0.11 sci_32bit_LU
71.75 0.01% 8 0.01 sci_64bit-lpages_Monte
71.75 0.02% 7 0.01 sci_64bit_Monte
808.26 0.04% 7 0.11 sci_32bit-lpages_LU
904.99 0.04% 7 0.12 sci_32bit-lpages_SOR
904.61 0.07% 7 0.11 sci_32bit_SOR
901.19 0.09% 8 0.07 sci_64bit-lpages_SOR
902.63 0.14% 7 0.09 sci_64bit_SOR
540.03 0.21% 8 0.11 sci_64bit-lpages_Sparse
406.69 0.47% 8 2.93 sci_64bit-lpages_Composite
407.38 1.41% 7 3.18 sci_64bit_Composite

s10u8_08a -> s10u8_08a with JVM 1.5.0_23
Mean Change #-res StdDev% Metric
56.46 -21.55% 3 0.01 sci_32bit-lpages_Monte
57.60 -20.00% 3 0.01 sci_32bit_Monte
46.75 -14.74% 3 0.22 sci_32bit_FFT
46.97 -13.92% 3 0.62 sci_32bit-lpages_FFT
426.35 -7.50% 3 0.08 sci_64bit-lpages_LU
508.61 -3.14% 3 0.03 sci_32bit-lpages_Sparse
50.67 -2.47% 3 0.88 sci_64bit-lpages_FFT
464.83 -1.69% 3 0.04 sci_32bit-lpages_Composite
398.64 -1.52% 3 0.04 sci_64bit-lpages_Composite
51.23 -1.47% 3 0.63 sci_64bit_FFT
539.94 -1.41% 3 0.11 sci_64bit_Sparse
467.27 -0.87% 3 0.40 sci_32bit_Composite
807.27 -0.08% 3 0.12 sci_32bit-lpages_LU
808.33 -0.07% 3 0.00 sci_32bit_LU
71.74 -0.01% 3 0.03 sci_64bit-lpages_Monte
71.74 0.00% 3 0.02 sci_64bit_Monte
904.83 0.02% 3 0.09 sci_32bit-lpages_SOR
901.72 0.04% 3 0.00 sci_64bit_SOR
904.38 0.05% 3 0.13 sci_32bit_SOR
540.23 0.25% 3 0.03 sci_64bit-lpages_Sparse
519.30 0.39% 3 1.88 sci_32bit_Sparse
904.21 0.42% 3 0.06 sci_64bit-lpages_SOR


The rig seeing this regression is:
X3220:oaf403_64:4 x 2.4GHz:4 GB:

The regession was in sci_32bit-lpages_Monte which can be found:
http://perfwww.ireland/Benchmarks/Java/scimark_specific.tgz
(using RUN_specific_jvm_tuned)

Explorer:
http://perfwww.ireland/cgi-bin/internaltools/machineStats.cgi?s10u9_01:oaf403_64

Statis before regression:
http://perfwww.ireland/Results9/s10u8_08a.oaf403_64/sci_32bit-lpages_Monte/9/statit.bin.txt
http://perfwww.ireland/Results9/s10u8_08a.oaf403_64/sci_32bit-lpages_Monte/3/statit.bin.txt
http://perfwww.ireland/Results9/s10u8_08a.oaf403_64/sci_32bit-lpages_Monte/4/statit.bin.txt

Statits after regression:
http://perfwww.ireland/Results9/s10u8_08a-6919117-java-s10u9_01.oaf403_64/sci_32bit-lpages_Monte/2/statit.bin.txt
http://perfwww.ireland/Results9/s10u8_08a-6919117-java-s10u9_01.oaf403_64/sci_32bit-lpages_Monte/1/statit.bin.txt
http://perfwww.ireland/Results9/s10u8_08a-6919117-java-s10u9_01.oaf403_64/sci_32bit-lpages_Monte/3/statit.bin.txt