Customer reported a series of crashes that showed exactly identical stack trace:

 libjvm.so:symbolOopDesc::as_klass_external_name()
 libjvm.so:InterpreterRuntime::throw_ClassCastException()
 

hs_err_pid9521.log

Stack: [0xab21b000,0xab23c000], sp=0xab23ae9c, free space=127k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x59062b] _ZNK13symbolOopDesc22as_klass_external_nameEv+0xb
V [libjvm.so+0x364ab9] _ZN18InterpreterRuntime24throw_ClassCastExceptionEP10JavaThreadP7oopDesc+0x49
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter

hs_err_pid25705.log

Stack: [0xaafb1000,0xaafd2000], sp=0xaafd0720, free space=125k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x59ca8b] _ZNK13symbolOopDesc22as_klass_external_nameEv+0xb
V [libjvm.so+0x36e499] _ZN18InterpreterRuntime24throw_ClassCastExceptionEP10JavaThreadP7oopDesc+0x49
j com.firsthop.common.util.data.BinaryHeap.getEntry(I)Lcom/firsthop/common/util/data/BinaryHeap$HeapEntry;+14
j com.firsthop.common.util.data.BinaryHeap.getValue(I)Ljava/lang/Comparable;+2
j com.firsthop.common.util.data.BinaryHeap.percolateUp(Ljava/lang/Object;)Z+35

hs_err_pid28579.log

Stack: [0x67dbf000,0x67de0000], sp=0x67ddeca4, free space=127k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x59ca8b] _ZNK13symbolOopDesc22as_klass_external_nameEv+0xb
V [libjvm.so+0x36e499] _ZN18InterpreterRuntime24throw_ClassCastExceptionEP10JavaThreadP7oopDesc+0x49
j com.firsthop.common.scheduler.task.TimedTask.compareTo(Ljava/lang/Object;)I+1
j com.firsthop.common.util.data.BinaryHeap.percolateUp(Ljava/lang/Object;)Z+38

hs_err_pid4132.log

Stack: [0x6ab8a000,0x6abab000], sp=0x6aba9ac0, free space=126k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x59ca8b] _ZNK13symbolOopDesc22as_klass_external_nameEv+0xb
V [libjvm.so+0x36e499] _ZN18InterpreterRuntime24throw_ClassCastExceptionEP10JavaThreadP7oopDesc+0x49
j com.firsthop.common.util.data.BinaryHeap.getEntry(I)Lcom/firsthop/common/util/data/BinaryHeap$HeapEntry;+14
j com.firsthop.common.util.data.BinaryHeap.getValue(I)Ljava/lang/Comparable;+2
j com.firsthop.common.util.data.BinaryHeap.percolateUp(Ljava/lang/Object;)Z+35

hs_err_pid17806.log

Stack: [0xab19e000,0xab1bf000], sp=0xab1bdc00, free space=127k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x59ca8b] _ZNK13symbolOopDesc22as_klass_external_nameEv+0xb
V [libjvm.so+0x36e499] _ZN18InterpreterRuntime24throw_ClassCastExceptionEP10JavaThreadP7oopDesc+0x49
j com.firsthop.common.util.data.BinaryHeap.getEntry(I)Lcom/firsthop/common/util/data/BinaryHeap$HeapEntry;+14
j com.firsthop.common.util.data.BinaryHeap.getValue(I)Ljava/lang/Comparable;+2
j com.firsthop.common.util.data.BinaryHeap.compare(II)I+2
j com.firsthop.common.util.data.BinaryHeap.percolateDown(Ljava/lang/Object;)Z+49


hs_err_pid19940.log

Stack: [0xaa255000,0xaa276000], sp=0xaa2748fc, free space=126k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x59ca8b] _ZNK13symbolOopDesc22as_klass_external_nameEv+0xb
V [libjvm.so+0x36e499] _ZN18InterpreterRuntime24throw_ClassCastExceptionEP10JavaThreadP7oopDesc+0x49
j com.firsthop.common.util.data.BinaryHeap.getEntry(I)Lcom/firsthop/common/util/data/BinaryHeap$HeapEntry;+14
j com.firsthop.common.util.data.BinaryHeap.getValue(I)Ljava/lang/Comparable;+2
j com.firsthop.common.util.data.BinaryHeap.compare(II)I+2
j com.firsthop.common.util.data.BinaryHeap.percolateDown(Ljava/lang/Object;)Z+49

hs_err_pid24415.log

Stack: [0xab1f4000,0xab215000], sp=0xab213c9c, free space=127k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x59ca8b] _ZNK13symbolOopDesc22as_klass_external_nameEv+0xb
V [libjvm.so+0x36e499] _ZN18InterpreterRuntime24throw_ClassCastExceptionEP10JavaThreadP7oopDesc+0x49
j com.firsthop.common.util.data.BinaryHeap.getEntry(I)Lcom/firsthop/common/util/data/BinaryHeap$HeapEntry;+14
j com.firsthop.common.util.data.BinaryHeap.getValue(I)Ljava/lang/Comparable;+2
j com.firsthop.common.util.data.BinaryHeap.compare(II)I+2
j com.firsthop.common.util.data.BinaryHeap.percolateDown(Ljava/lang/Object;)Z+49


 The crashes were observed using 1.6.0_06, and 1.6.0_10 resp.

% grep vm_info *
hs_err_pid17806.log:vm_info: Java HotSpot(TM) Server VM (11.0-b15) for linux-x86 JRE (1.6.0_10-b33), built on Sep 26 2008 01:05:20 by "java_re" with gcc 3.2.1-7a (J2SE release)
hs_err_pid19940.log:vm_info: Java HotSpot(TM) Server VM (11.0-b15) for linux-x86 JRE (1.6.0_10-b33), built on Sep 26 2008 01:05:20 by "java_re" with gcc 3.2.1-7a (J2SE release)
hs_err_pid24415.log:vm_info: Java HotSpot(TM) Server VM (11.0-b15) for linux-x86 JRE (1.6.0_10-b33), built on Sep 26 2008 01:05:20 by "java_re" with gcc 3.2.1-7a (J2SE release)
hs_err_pid25705.log:vm_info: Java HotSpot(TM) Server VM (11.0-b15) for linux-x86 JRE (1.6.0_10-b33), built on Sep 26 2008 01:05:20 by "java_re" with gcc 3.2.1-7a (J2SE release)
hs_err_pid28579.log:vm_info: Java HotSpot(TM) Server VM (11.0-b15) for linux-x86 JRE (1.6.0_10-b33), built on Sep 26 2008 01:05:20 by "java_re" with gcc 3.2.1-7a (J2SE release)
hs_err_pid4132.log:vm_info: Java HotSpot(TM) Server VM (11.0-b15) for linux-x86 JRE (1.6.0_10-b33), built on Sep 26 2008 01:05:20 by "java_re" with gcc 3.2.1-7a (J2SE release)
hs_err_pid9521.log:vm_info: Java HotSpot(TM) Server VM (10.0-b22) for linux-x86 JRE (1.6.0_06-b02), built on Mar 25 2008 00:26:44 by "java_re" with gcc 3.2.1-7a (J2SE release)
%

Operating system was:
Red Hat Enterprise Linux AS release 4 (Nahant Update 4)

on MP systems:
% grep CPU *
hs_err_pid17806.log:CPU:total 16 (2 cores per cpu, 2 threads per core) family 15 model 6 stepping 8, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ht
hs_err_pid19940.log:CPU:total 16 (2 cores per cpu, 2 threads per core) family 15 model 6 stepping 8, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ht
hs_err_pid24415.log:CPU:total 16 (2 cores per cpu, 2 threads per core) family 15 model 6 stepping 8, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ht
hs_err_pid25705.log:CPU:total 16 (2 cores per cpu, 2 threads per core) family 15 model 6 stepping 8, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ht
hs_err_pid28579.log:CPU:total 8 (2 cores per cpu, 2 threads per core) family 15 model 6 stepping 8, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ht
hs_err_pid4132.log:CPU:total 8 (2 cores per cpu, 2 threads per core) family 15 model 6 stepping 8, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ht
hs_err_pid9521.log:CPU:total 16 (2 cores per cpu, 2 threads per core) family 15 model 6 stepping 8, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ht

-XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing was used in each of the incidents.