reliability: vm crashes in libs reliability test on Sol sparc 10

JDK : tiger b49
Platforms[s] : Sol Sparc 10 (not on RHAS3, WinXPpro)
testbase : /net/cady/export/dtf/unified/knight-ws/suites/libs_reliability/testbase

test machines:
===============
SunOS jtg-s130 5.10 s10_57 sun4u sparc SUNW,Ultra-60
SunOS norma 5.10 s10_56 sun4u sparc SUNW,Ultra-60


How to reproduce:
==================
1. modify run.sh (as attached or /net/cady/export/dtf/unified/knight-ws/suites/libs_reliability/run.sh)
to have correct JAVA_HOME, RESULT_DIR

2. sh run.sh to start libs reliability tests. This test will run for 72 hours (TEST_HRS=72). In 1.5 hour, I got vm coredump and the test stops.


hs_err_pid1293.log: (complete log is attached)

#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
# SIGSEGV (0xb) at pc=0xfed93224, pid=1293, tid=11920
#
# Java VM: Java HotSpot(TM) Client VM (1.5.0-beta2-b49 mixed mode, sharing)
# Problematic frame:
# V [libjvm.so+0x193224]
#

--------------- T H R E A D ---------------

Current thread (0x00196848): JavaThread "Thread-10298" [_thread_in_vm, id=11920]

siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0x00000010

Registers:
 O0=0xf8dde388 O1=0x0000678c O2=0x00000000 O3=0xfece480c
 O4=0x00006400 O5=0xff0b2000 O6=0xc517eac8 O7=0xfece3cbc
 G1=0x001de89c G2=0x00000000 G3=0x00000006 G4=0x00000001
 G5=0xd540ade8 G6=0x00000000 G7=0xdc8c8800 Y=0x00000000
 PC=0xfed93224 nPC=0xfed93228


Top of Stack: (sp=0xc517eac8)
0xc517eac8: 00000000 f8dde894 fed93154 00000000
0xc517ead8: 00000000 00000000 ff0cab0c fed74e5c
0xc517eae8: c517ec28 f8dde388 c517f768 fed93220
0xc517eaf8: c517f858 c517ec2c c517eb28 fece40a0
0xc517eb08: 00000000 00000000 c517f858 f8c70448
0xc517eb18: 00000000 00000000 00000000 00000000
0xc517eb28: c517ec40 fece3c3c fece3c3c 00000000
0xc517eb38: 00000000 c517ef48 f8c0c01c c517eb58

Instructions: (pc=0xfed93224)
0xfed93214: 81 e8 00 00 81 c7 e0 08 81 e8 00 00 d4 02 20 2c
0xfed93224: d2 02 a0 10 90 0a 61 00 80 a0 00 08 81 c3 e0 08

Stack: [0xc5100000,0xc5180000), sp=0xc517eac8, free space=506k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x193224]
V [libjvm.so+0xe40a8]
V [libjvm.so+0x121e18]
V [libjvm.so+0x1212dc]
V [libjvm.so+0x1211b4]
C [libjava.so+0xee40] Java_java_lang_Throwable_fillInStackTrace+0x14
j java.lang.Throwable.fillInStackTrace()Ljava/lang/Throwable;+709726520
j java.lang.Throwable.fillInStackTrace()Ljava/lang/Throwable;+0
j java.lang.Throwable.<init>()V+10
j java.lang.Error.<init>()V+1
j java.lang.LinkageError.<init>()V+1
j java.lang.IncompatibleClassChangeError.<init>()V+1
v ~StubRoutines::call_stub
V [libjvm.so+0xd2538]
V [libjvm.so+0x121068]
V [libjvm.so+0x120f48]
V [libjvm.so+0x28a258]
V [libjvm.so+0x28a2ec]
V [libjvm.so+0x3e4dc8]
V [libjvm.so+0x23ace0]
v ~RuntimeStub::throw_incompatible_class_change_error Runtime1 stub
J 0xf8dde89c
J java.util.concurrent.DelayQueue.poll()Ljava/lang/Object;
J java.util.concurrent.DelayQueue.remove()Ljava/lang/Object;
j Consumer.run()V+30
v ~StubRoutines::call_stub
V [libjvm.so+0xd2538]
V [libjvm.so+0xfa670]
V [libjvm.so+0xfa4cc]
V [libjvm.so+0xfa350]
V [libjvm.so+0xcbfec]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j java.lang.Throwable.fillInStackTrace()Ljava/lang/Throwable;+0
j java.lang.Throwable.<init>()V+10
j java.lang.Error.<init>()V+1
j java.lang.LinkageError.<init>()V+1
j java.lang.IncompatibleClassChangeError.<init>()V+1
v ~StubRoutines::call_stub
v ~RuntimeStub::throw_incompatible_class_change_error Runtime1 stub
J 0xf8dde89c
J java.util.concurrent.DelayQueue.poll()Ljava/lang/Object;
J java.util.concurrent.DelayQueue.remove()Ljava/lang/Object;
j Consumer.run()V+30
v ~StubRoutines::call_stub

.....
Other Threads:
  0x000d72c8 VMThread [id=2]
  0x000e4a10 WatcherThread [id=8]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
 def new generation total 7040K, used 155K [0xccc00000, 0xcd310000, 0xcd310000)
  eden space 6848K, 2% used [0xccc00000, 0xccc26fb0, 0xcd2b0000)
  from space 192K, 0% used [0xcd2b0000, 0xcd2b0000, 0xcd2e0000)
  to space 192K, 0% used [0xcd2e0000, 0xcd2e0000, 0xcd310000)
 tenured generation total 2376K, used 1361K [0xcd310000, 0xcd562000, 0xd0c00000)
   the space 2376K, 57% used [0xcd310000, 0xcd464448, 0xcd464600, 0xcd562000)
 compacting perm gen total 8192K, used 1866K [0xd0c00000, 0xd1400000, 0xd4c00000)
   the space 8192K, 22% used [0xd0c00000, 0xd0dd2800, 0xd0dd2800, 0xd1400000)
    ro space 8192K, 64% used [0xd4c00000, 0xd51247c8, 0xd5124800, 0xd5400000)
    rw space 12288K, 44% used [0xd5400000, 0xd5963fb0, 0xd5964000, 0xd6000000)

Dynamic libraries:
0x00010000 /usr/jdk/j2sdk1.5.0/bin/java
0xff3a8000 /lib/libthread.so.1
0xff3a0000 /lib/libdl.so.1
0xff280000 /lib/libc.so.1
0xff398000 /platform/SUNW,Ultra-60/lib/libc_psr.so.1
0xfec00000 /usr/jdk/j2sdk1.5.0/jre/lib/sparc/client/libjvm.so
0xff360000 /lib/libsocket.so.1
0xff390000 /usr/lib/libsched.so.1
0xff230000 /usr/lib/libCrun.so.1
0xff210000 /lib/libm.so.1
0xff100000 /lib/libnsl.so.1
0xfeb00000 /lib/libm.so.2
0xff1f0000 /lib/libmd5.so.1
0xff1d0000 /platform/SUNW,Ultra-60/lib/libmd5_psr.so.1
0xfebe0000 /lib/libmp.so.2
0xfeab0000 /usr/jdk/j2sdk1.5.0/jre/lib/sparc/native_threads/libhpi.so
0xfea60000 /usr/jdk/j2sdk1.5.0/jre/lib/sparc/libverify.so
0xfea20000 /usr/jdk/j2sdk1.5.0/jre/lib/sparc/libjava.so
0xfe9f0000 /usr/jdk/j2sdk1.5.0/jre/lib/sparc/libzip.so
0xfe2b0000 /usr/jdk/j2sdk1.5.0/jre/lib/sparc/libj2pkcs11.so
0xfe290000 /usr/lib/libpkcs11.so
0xfe1e0000 /usr/lib/libcryptoutil.so.1
0xfe1c0000 /lib/libdoor.so.1
0xfe160000 /usr/lib/security/pkcs11_softtoken.so
0xf8780000 /usr/jdk/j2sdk1.5.0/jre/lib/sparc/libawt.so
0xf8600000 /usr/jdk/j2sdk1.5.0/jre/lib/sparc/libmlib_image.so
0xfb880000 /usr/jdk/j2sdk1.5.0/jre/lib/sparc/motif21/libmawt.so
0xf8380000 /usr/dt/lib/libXm.so.4
0xfbbd0000 /usr/openwin/lib/libXp.so.1
0xfac80000 /usr/openwin/lib/libXt.so.4
0xfbb90000 /usr/openwin/lib/libXext.so.0
0xfbac0000 /usr/openwin/lib/libXtst.so.1
0xf8280000 /usr/openwin/lib/libX11.so.4
0xfba90000 /usr/openwin/lib/libSM.so.6
0xfb9c0000 /usr/openwin/lib/libICE.so.6

VM Arguments:
java_command: CustomClassLoader /net/cady/export/dtf/unified/knight-ws/suites/libs_reliability/testbase/run/src_classes/ tl 72

Environment Variables:
CLASSPATH=/net/cady/export/dtf/unified/knight-ws/suites/libs_reliability/testbase/HelperClasses/classes:/net/cady/export/dtf/unified/knight-ws/suites/libs_reliability/testbase/HelperClasses/runAll:.
PATH=/usr/sbin:/usr/bin
LD_LIBRARY_PATH=/usr/jdk/j2sdk1.5.0/jre/lib/sparc/client:/usr/jdk/j2sdk1.5.0/jre/lib/sparc:/usr/jdk/j2sdk1.5.0/jre/../lib/sparc
SHELL=/bin/csh
DISPLAY=vpn-129-150-17-3.SFBay.Sun.COM:0.0


--------------- S Y S T E M ---------------

OS: Solaris 10 s10_57 SPARC
           Copyright 2004 Sun Microsystems, Inc. All Rights Reserved.
                        Use is subject to license terms.
                             Assembled 14 April 2004

uname:SunOS 5.10 s10_57 sun4u (T2 libthread)
rlimit: STACK 8192k, CORE infinity, NOFILE 65536, AS infinity
load average:0.00 0.00 0.01

CPU:total 2 has_v8, has_v9, has_vis1

Memory: 8k page, physical 524288k(243072k free)

vm_info: Java HotSpot(TM) Client VM (1.5.0-beta2-b49) for solaris-sparc, built on Apr 28 2004 03:44:10 by unknown with unknown Workshop:0x550



also fails with tiger b48 on jtg-s130:

#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
# SIGSEGV (0xb) at pc=0xfed936d0, pid=1506, tid=11152
#
# Java VM: Java HotSpot(TM) Client VM (1.5.0-beta2-b48 mixed mode)
# Problematic frame:
# V [libjvm.so+0x1936d0]
#


###@###.### 2004-04-29