Dynamo is an application server running a stack of layered applications. The
error seemed less prevalent with JDK 1.3.0x.

Prior to HotSpot actually crashing the AWT-EventQueue thread disappears. The
UI hangs (screen doesn't redraw, doesn't respond to input) then at some
point after the hang begins we often get the following:

#
# HotSpot Virtual Machine Error, Internal Error
# Please report this error at
# http://java.sun.com/cgi-bin/bugreport.cgi
#
# Error ID: 4349254E560E43505000F4
#
# Problematic Thread: prio=5 tid=0x816148 nid=0x708 runnable
#

We don't always see the crash after the hang. However, this may simply be
due to impatience. On a number of occasions I have allowed the system to
continue after the hang for 10 to 15 minutes, sometimes viewing pages via
Dynamo's internal web server, and eventually got the same HotSpot crash.

The problem is seen in both Win and Solaris platforms using JDK 1.3.1 and
1.3.1_01 test builds.

1.3.1-b24

bash-2.04$ bin/startDynamo.bat -m Portal.demo-channels-lite
Dynamo Application Server 5.5
java version "1.3.1"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.1-b24)

I've been able to reproduce the error twice since I installed VC++. We're
using the default (Client) version of HotSpot with JDK/JRE 1.3.1-b24. One
caveat: I'm not able to attach to the process the way you described (through
the menus). Rather I can only attach through the Task Manager (right
clicking on the java process offers a debug option.) or, when VM goes to
crash and the dialog window comes up saying "Fatal: unhandled ci exception.
Do you want to debug the problem?" I can say "Yes" and VC++ comes up
automatically.

Assuming this is all okay, here's the results from the two runs:

#1:
call stack:

JVM! 6d48c372()
JVM! 6d48c437()
JVM! 6d4b1d38()
JVM! 6d498ffc()
JVM! 6d49ba19()
JVM! 6d49c5b5()
JVM! 6d49877b()
JVM! 6d4b4470()
JVM! 6d4912f2()
JVM! 6d4910ff()
JVM! 6d482c10()
JVM! 6d482be0()
MSVCRT! 7800248d()
KERNEL32! 77e837cd()

thread dump: I was running Dynamo from emacs and following the crash, the
sigquit stopped working. Thus I couldn't get the java thread dump.

#2:

I ran from a cmd window this time but still wasn't able to get a thread
dump. Ctrl-break didn't do anything.

The call stack in VC++:

JVM! 6d48bf70()
JVM! 6d48c1d2()
JVM! 6d48c207()
JVM! 6d48cd41()
JVM! 6d48c4bc()
JVM! 6d48c53f()
JVM! 6d4b1d81()
JVM! 6d49a9a6()
JVM! 6d49bef4()
JVM! 6d49c5b5()
JVM! 6d49877b()
JVM! 6d4b4470()
JVM! 6d4912f2()
JVM! 6d4910ff()
JVM! 6d482c10()
JVM! 6d482be0()
MSVCRT! 7800248d()
KERNEL32! 77e837cd()

I hope this is useful to you all.

I suspect that having this available to you will be useful since the abnormal
death of the AWT-EventQueue thread which precedes the crash (it never exits
it's run() method but once the UI hangs a thread dump shows that it has
disappeared) may be the start of the problem that eventually causes HotSpot
to crash.

Testcase:
---------
- install AppDAP5.5 (Dynamo + all applications)
- start the appropriate database
- run the Channel Portal demo
- point your browser at http://localhost:8840/portal/cam
-login as airto/airto
-from the "Community Contacts" block click on "Michael Brooks"
-click on the "Work Desk" link to go back
-click on "Administer"
-from the "Page Template" drop down menu change the selection to "Phoenix
Page Template"
-click "Community Members"
-click the "Guests" tab
-click "+ Add / Remove Organizations"
-click on "Phoenix" from the box on the left; it should reappear in the box
on the right
-click the "Done" button
-click on "Community Pages"
-click the "Edit" link for Work Desk
-click the "Next" button
-select "Contracts" from the "Narrow" list then click the "->" arrow that
highlights; "Contracts" should move to the "Wide" list.
-click the "Done" button

-point your browser to http://localhost:8830/
-login as admin/admin
-click the "Admin ACC" link
-click the "Start ACC" button
-login to the UI as admin/admin
-In the UI window there's a series of boxes, click the one called "Portals"
-From the sub list that expand from Portals click "Document Repositories"

The UI should hang at this point. Doing a thread dump should reveal that the
AWT-EventQueue thread has disappeared. If you go back to
http://localhost:8830/portal/cam and continue browsing pages you may help
the VM to crash. Or it may crash by itself. Or it may not crash and continue
to hang.
Java HotSpot(TM) Client VM (build 1.3.1-b24, mixed mode)
java full version "1.3.1-b24"

----

Thread dump:
---------------
I have attached the thread dump that I took right after our UI hung and before we get the HotSpot crash.