The following JVM crashes reported by GlassFish QA when running stress test with SSL with GlassFish 3.1 3-instance cluster. Each instance was crashed (not at same time) after the test running ~4 days normally.

Java Runtime: 1.6.0_23 Sun Microsystems Inc.
Windows Server 2008 R2 6.1 amd64 (4 cpu)

The following are from the 3 JVM crashes, attached hs_err files.
 
====================================
On DAS + inst3 - asqe-oblade-15 : There was a JVM crash on 1/18/11 at 1:36 pm.Please see jvm crash log under file C:\export\glassfish3\glassfish\nodes\node3\in3\config

--------------- T H R E A D ---------------

Current thread (0x00000000090bf000): JavaThread "TransactionReaper" [_thread_in_Java, id=2716, stack(0x000000000d140000,0x000000000d160000)]

siginfo: ExceptionCode=0xc0000005, reading address 0x000000000066006e

Registers:
RAX=0x00000000c7a1e3e8, RBX=0x00000000d3274840, RCX=0x00000000d4209be0, RDX=0x00000000f170afd8
RSP=0x000000000d15f4c0, RBP=0x00000000f170afd8, RSI=0x0000000000000000, RDI=0x00000000f1769cc8
R8=0x00000000f1709698, R9=0x0000000000660036, R10=0x0000000000000000, R11=0x00000000f1709698
R12=0x0000000000000000, R13=0x0000000000000000, R14=0x000000000d15f558, R15=0x00000000090bf000
RIP=0x0000000002af57c2, EFLAGS=0x0000000000010202

Register to memory mapping:

RAX=0x00000000c7a1e3e8
{instance class}
 - klass: {other class}

RBX=0x00000000d3274840
java.util.ArrayList
 - klass: 'java/util/ArrayList'

RCX=0x00000000d4209be0
[Ljava.lang.Object;
 - klass: 'java/lang/Object'[]
 - length: 38

RDX=0x00000000f170afd8

[error occurred during error reporting (printing registers, top of stack, instructions near pc), id 0xe0000000]

Stack: [0x000000000d140000,0x000000000d160000], sp=0x000000000d15f4c0, free space=125k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
J com.sun.messaging.jmq.util.UID.equals(Ljava/lang/Object;)Z

====================================
On in2 bigapp-oblade-2 : There is a JVM crash on 1/19/11 1:22 am. Please see jvm crash log under file C:\export\glassfish3\glassfish\nodes\node1\in1\config

--------------- T H R E A D ---------------

Current thread (0x0000000006ac4800): JavaThread "TransactionReaper" daemon [_thread_in_Java, id=3928, stack(0x000000000ca00000,0x000000000ca20000)]

siginfo: ExceptionCode=0xc0000005, reading address 0x0000000000650085

Registers:
RAX=0x00000000c40c4b20, RBX=0x0000000000000003, RCX=0x00000000f0953400, RDX=0x00000000c76fe1f8
RSP=0x000000000ca1f640, RBP=0x00000000d2dbd440, RSI=0x00000000c7a1d728, RDI=0x59d92d9d95057f07
R8=0x0000000000000001, R9=0x00000000d4249428, R10=0x0000000000000004, R11=0x000000000065004d
R12=0x0000000000000000, R13=0x00000000c78f3488, R14=0x00000000c40121b0, R15=0x0000000006ac4800
RIP=0x0000000001d782c6, EFLAGS=0x0000000000010286

Register to memory mapping:

RAX=0x00000000c40c4b20
{instance class}
 - klass: {other class}

RBX=0x0000000000000003
0x0000000000000003 is pointing to unknown location

RCX=0x00000000f0953400

[error occurred during error reporting (printing registers, top of stack, instructions near pc), id 0xe0000000]

Stack: [0x000000000ca00000,0x000000000ca20000], sp=0x000000000ca1f640, free space=125k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
J com.sun.messaging.jmq.jmsserver.data.TransactionReaper.run()V

====================================
On in1 - bigapp-oblade-1 : There is a JVM crash on 1/19/11 10:08 AM. Here is the reason for the crash. Please see jvm crash log under file C:\export\glassfish3\glassfish\nodes\node1\in1\config

--------------- T H R E A D ---------------

Current thread (0x0000000008aae000): JavaThread "Grizzly-kernel-thread(1)" daemon [_thread_in_vm, id=3572, stack(0x0000000008360000,0x0000000008380000)]

siginfo: ExceptionCode=0xc0000005, reading address 0x0000000000000000

Registers:
RAX=0x0000000000000000, RBX=0x0000000008aae000, RCX=0x0000000000000e00, RDX=0x0000000000000000
RSP=0x000000000837f2b0, RBP=0x000000000a85bc50, RSI=0x0000000000000000, RDI=0x0000000000000000
R8=0x0000000000000032, R9=0x000000000a85bc50, R10=0x0000000000000000, R11=0x0000000000000246
R12=0x0000000000000032, R13=0x00000000c53922d8, R14=0x000000000837f448, R15=0x0000000008aae000
RIP=0x000000006dadf79e, EFLAGS=0x0000000000010202

Register to memory mapping:

RAX=0x0000000000000000
0x0000000000000000 is pointing to unknown location

RBX=0x0000000008aae000
"Grizzly-kernel-thread(1)" daemon prio=10 tid=0x0000000008aae000 nid=0xdf4 runnable [0x000000000837f000]
   java.lang.Thread.State: RUNNABLE

RCX=0x0000000000000e00
0x0000000000000e00 is pointing to unknown location

RDX=0x0000000000000000
0x0000000000000000 is pointing to unknown location

RSP=0x000000000837f2b0
0x000000000837f2b0 is pointing into the stack for thread: 0x0000000008aae000
"Grizzly-kernel-thread(1)" daemon prio=10 tid=0x0000000008aae000 nid=0xdf4 runnable [0x000000000837f000]
   java.lang.Thread.State: RUNNABLE

RBP=0x000000000a85bc50
0x000000000a85bc50 is pointing to unknown location

RSI=0x0000000000000000
0x0000000000000000 is pointing to unknown location

RDI=0x0000000000000000
0x0000000000000000 is pointing to unknown location

R8 =0x0000000000000032
0x0000000000000032 is pointing to unknown location

R9 =0x000000000a85bc50
0x000000000a85bc50 is pointing to unknown location

R10=0x0000000000000000
0x0000000000000000 is pointing to unknown location

R11=0x0000000000000246
0x0000000000000246 is pointing to unknown location

R12=0x0000000000000032
0x0000000000000032 is pointing to unknown location

R13=0x00000000c53922d8
{constMethod}
 - klass: {other class}
 - method: 0x00000000c53922e0 {method} 'accept0' '(Ljava/io/FileDescriptor;Ljava/io/FileDescriptor;[Ljava/net/InetSocketAddress;)I' in 'sun/nio/ch/ServerSocketChannelImpl'
 - exceptions: 0x00000000c4001ef8

R14=0x000000000837f448
0x000000000837f448 is pointing into the stack for thread: 0x0000000008aae000
"Grizzly-kernel-thread(1)" daemon prio=10 tid=0x0000000008aae000 nid=0xdf4 runnable [0x000000000837f000]
   java.lang.Thread.State: RUNNABLE

R15=0x0000000008aae000
"Grizzly-kernel-thread(1)" daemon prio=10 tid=0x0000000008aae000 nid=0xdf4 runnable [0x000000000837f000]
   java.lang.Thread.State: RUNNABLE


Top of Stack: (sp=0x000000000837f2b0)
0x000000000837f2b0: 0000000008aae000 0000000000002048
0x000000000837f2c0: 000000000837f438 000000000837f438
0x000000000837f2d0: 00000000000009a0 000000000837f430
0x000000000837f2e0: 0000000000000000 000000000837f438
0x000000000837f2f0: 0000000008aae1c8 00000000c4832e18
0x000000000837f300: 0000000000000000 000000006d6c3215
0x000000000837f310: 000000006dee6520 000000000837f430
0x000000000837f320: 0000000008aae1c8 00000000c4149600
0x000000000837f330: 0000000000000001 0000000000000001
0x000000000837f340: 0000000000000010 50bc850a8fa60002
0x000000000837f350: 0000000000000000 0000b74226c117d6
0x000000000837f360: 0000000000000001 00000000012258fa
0x000000000837f370: 00000000c4149600 000000000837f410
0x000000000837f380: 00000000c53922e0 00000000012312a0
0x000000000837f390: 0000000000000001 00000000c53c1f98
0x000000000837f3a0: 0000000008aae000 000000006da91dbd

Instructions: (pc=0x000000006dadf79e)
0x000000006dadf78e: 83 38 02 00 00 06 00 00 00 80 3d 2a ca 41 00 00
0x000000006dadf79e: 48 8b 16 74 17 44 8b 4a 08 0f b6 0d 0a b0 40 00


Stack: [0x0000000008360000,0x0000000008380000], sp=0x000000000837f2b0, free space=124k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [jvm.dll+0x24f79e]

[error occurred during error reporting (printing native stack), id 0xc0000005]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j sun.nio.ch.ServerSocketChannelImpl.accept0(Ljava/io/FileDescriptor;Ljava/io/FileDescriptor;[Ljava/net/InetSocketAddress;)I+0
j sun.nio.ch.ServerSocketChannelImpl.accept()Ljava/nio/channels/SocketChannel;+94
j com.sun.grizzly.TCPSelectorHandler.acceptWithoutRegistration(Ljava/nio/channels/SelectionKey;)Ljava/nio/channels/SelectableChannel;+11
j com.sun.enterprise.v3.services.impl.monitor.MonitorableSelectorHandler.acceptWithoutRegistration(Ljava/nio/channels/SelectionKey;)Ljava/nio/channels/SelectableChannel;+2
j com.sun.grizzly.http.SelectorThreadHandler.onAcceptInterest(Ljava/nio/channels/SelectionKey;Lcom/sun/grizzly/Context;)Z+2
J com.sun.grizzly.SelectorHandlerRunner.handleSelectedKey(Ljava/nio/channels/SelectionKey;Lcom/sun/grizzly/SelectorHandler;Lcom/sun/grizzly/NIOContext;)Z
J com.sun.grizzly.SelectorHandlerRunner.doSelect(Lcom/sun/grizzly/SelectorHandler;Lcom/sun/grizzly/NIOContext;)Z
J com.sun.grizzly.SelectorHandlerRunner.run()V
j java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Ljava/lang/Runnable;)V+59
j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+28
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub