When running SPECjEnterprise2010 on a T series system, I've noticed a degradation in performance between JDK7u10 and JDK8. With JDK7 the system can get a clean run of IR=2700 with roughly 85% CPU utilization. When running with JDK8 the system can only manage IR=2500 with roughly 89% utilization. Dave recommended looking at GC logs and that's where I noticed increased heap pressure when running with JDK8. This maybe due to the permgen removal.

Flags:
JAVA_OPTIONS="-d64 -XX:+UseCompressedOops -Xms26g -Xmx26g -Xmn20g -Xss256k
-XX:MetaspaceSize=512m -XX:MaxMetaspaceSize=512m -XX:ParallelGCThreads=16
-XX:-TraceClassUnloading -XX:+UseParallelOldGC -XX:TargetSurvivorRatio=90 -XX:MaxJavaStack
TraceDepth=2 -XX:-UseAdaptiveSizePolicy -XX:+UseLargePages -XX:-UseBiasedLocking"

Replace MetaspaceSize with PermSize for JDK7. The GC stats look like:

JDK8
ajiva@azeem-t410 ~ $ bin/PrintGCStats -v cpus=64 jdk8.txt

what count total mean max stddev
gen0t(s) 272 105.486 0.38782 1.191 0.2173
gen1t(s) 16 25.280 1.58000 3.379 0.5934
GC(s) 288 130.766 0.45405 3.379 0.3712
alloc(MB) 272 3945900.903 14506.98862 15371.562 3035.2559
promo(MB) 272 3582.777 13.17198 742.631 46.8062
used0(MB) 272 4063146.655 14938.03917 16741.043 3074.5571
used1(MB) 16 26910.056 1681.87848 3117.555 506.6787
used(MB) 288 4565945.023 15853.97578 19098.888 4548.3592
commit0(MB) 272 4874240.000 17920.00000 17920.000 0.0000
commit1(MB) 272 1671168.000 6144.00000 6144.000 0.0000
commit(MB) 272 6545408.000 24064.00000 24064.000 0.0000

alloc/elapsed_time = 3945900.903 MB / 12777.304 s = 308.821 MB/s
alloc/tot_cpu_time = 3945900.903 MB / 817747.456 s = 4.825 MB/s
alloc/mut_cpu_time = 3945900.903 MB / 809378.433 s = 4.875 MB/s
promo/elapsed_time = 3582.777 MB / 12777.304 s = 0.280 MB/s
promo/gc0_time = 3582.777 MB / 105.486 s = 33.964 MB/s
gc_seq_load = 8369.023 s / 817747.456 s = 1.023%
gc_conc_load = 0.000 s / 817747.456 s = 0.000%
gc_tot_load = 8369.023 s / 817747.456 s = 1.023%

JDK7:
ajiva@azeem-t410 ~ $ bin/PrintGCStats -v cpus=64 jdk7.txt

what count total mean max stddev
gen0t(s) 236 71.859 0.30449 1.221 0.1924
gen1t(s) 3 8.997 2.99903 3.170 0.1578
GC(s) 239 80.856 0.33831 3.170 0.3566
alloc(MB) 236 3593924.473 15228.49353 15360.000 1190.6400
promo(MB) 236 4164.562 17.64645 695.195 46.5540
used0(MB) 236 3706116.513 15703.88353 16696.315 1215.6062
used1(MB) 3 8086.154 2695.38477 3255.923 752.3770
used(MB) 239 4191167.555 17536.26592 19018.354 2124.3230
commit0(MB) 236 4229120.000 17920.00000 17920.000 0.0000
commit1(MB) 236 1449984.000 6144.00000 6144.000 0.0000
commit(MB) 236 5679104.000 24064.00000 24064.000 0.0000

alloc/elapsed_time = 3593924.473 MB / 5201.546 s = 690.934 MB/s
alloc/tot_cpu_time = 3593924.473 MB / 332898.944 s = 10.796 MB/s
alloc/mut_cpu_time = 3593924.473 MB / 327724.159 s = 10.966 MB/s
promo/elapsed_time = 4164.562 MB / 5201.546 s = 0.801 MB/s
promo/gc0_time = 4164.562 MB / 71.859 s = 57.955 MB/s
gc_seq_load = 5174.785 s / 332898.944 s = 1.554%
gc_conc_load = 0.000 s / 332898.944 s = 0.000%
gc_tot_load = 5174.785 s / 332898.944 s = 1.554%