FULL PRODUCT VERSION :
java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
Java HotSpot(TM) Client VM (build 1.5.0_06-b05, mixed mode, sharing)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows XP [Version 5.1.2600]

A DESCRIPTION OF THE PROBLEM :
I have a stock trading program which is schedulaed to start automatically every weekday morning at 9:25 am. Today, it crashed immediately. The following text was found in a crash file named hs_err_pid2540.log that was automatically generated:


#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x6d9093b9, pid=2540, tid=2556
#
# Java VM: Java HotSpot(TM) Server VM (1.5.0_06-b05 mixed mode)
# Problematic frame:
# V [jvm.dll+0x893b9]
#

--------------- T H R E A D ---------------

Current thread (0x00a36a40): ConcurrentMarkSweepThread [id=2556]

siginfo: ExceptionCode=0xc0000005, reading address 0x00330040

Registers:
EAX=0x01000000, EBX=0x4955fb38, ECX=0x00000018, EDX=0x0033003c
ESP=0x4955fa50, EBP=0x46f00000, ESI=0x00a3665c, EDI=0x035200e0
EIP=0x6d9093b9, EFLAGS=0x00010206

  Top of Stack: (sp=0x4955fa50)
0x4955fa50: 4955fb38 035200e0 00055880 034b0e00
0x4955fa60: 6d8f5a70 035200e0 034b0e00 00055880
0x4955fa70: 00000ab1 00a36628 6db788a0 00a58200
0x4955fa80: 6d9068f4 034b0e00 00055880 4955fb38
0x4955fa90: 00000000 00a36628 00000000 00000001
0x4955faa0: 000000b4 03607000 12ad0000 034b0e00
0x4955fab0: 00055880 033ca600 03dc1680 034b0e00
0x4955fac0: 00055880 00a366bc 00a3671c 00a36af8

Instructions: (pc=0x6d9093b9)
0x6d9093a9: c1 e8 02 5f 5e 5d 5b c2 0c 00 8b 57 04 83 c2 08
0x6d9093b9: 8b 42 04 85 c0 7f 25 7d 1b 83 c9 ff 2b c8 8b 47


Stack: [0x49520000,0x49560000), sp=0x4955fa50, free space=254k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [jvm.dll+0x893b9]


--------------- P R O C E S S ---------------

Java Threads: ( => current thread )
  0x496f4ba8 JavaThread "ConsoleUtil_Checker" [_thread_blocked, id=2620]
  0x496a03e0 JavaThread "pool-1-thread-1" [_thread_in_Java, id=2616]
  0x49702848 JavaThread "RoboTrader_StdinReader" [_thread_in_native, id=2612]
  0x495a1988 JavaThread "MemoryMonitor_Watcher" [_thread_blocked, id=2600]
  0x49593f48 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=2592]
  0x00acdc00 JavaThread "CompilerThread1" daemon [_thread_blocked, id=2588]
  0x4958bda8 JavaThread "CompilerThread0" daemon [_thread_in_native, id=2584]
  0x00acfbc0 JavaThread "AdapterThread" daemon [_thread_blocked, id=2580]
  0x4958da80 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=2576]
  0x49589008 JavaThread "Surrogate Locker Thread (CMS)" daemon [_thread_blocked, id=2572]
  0x4958e5e8 JavaThread "Finalizer" daemon [_thread_blocked, id=2568]
  0x49580c88 JavaThread "Reference Handler" daemon [_thread_blocked, id=2564]
  0x000371c8 JavaThread "main" [_thread_in_Java, id=2544]

Other Threads:
  0x00acfad0 VMThread [id=2560]
  0x49590560 WatcherThread [id=2596]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
 par new generation total 8128K, used 12326K [0x02ad0000, 0x032d0000, 0x032d0000)
  eden space 8064K, 152% used [0x02ad0000, 0x036d9908, 0x032b0000)
  from space 64K, 0% used [0x032b0000, 0x032b0000, 0x032c0000)
  to space 64K, 0% used [0x032c0000, 0x032c0000, 0x032d0000)
 concurrent mark-sweep generation total 253952K, used 0K [0x032d0000, 0x12ad0000, 0x42ad0000)
 concurrent-mark-sweep perm gen total 16384K, used 3606K [0x42ad0000, 0x43ad0000, 0x46ad0000)

Dynamic libraries:
0x00400000 - 0x0040c000 C:\Program Files\Java\jdk1.5.0_06\bin\java.exe
0x7c900000 - 0x7c9b0000 C:\WINDOWS\system32\ntdll.dll
0x7c800000 - 0x7c8f4000 C:\WINDOWS\system32\kernel32.dll
0x77dd0000 - 0x77e6b000 C:\WINDOWS\system32\ADVAPI32.dll
0x77e70000 - 0x77f01000 C:\WINDOWS\system32\RPCRT4.dll
0x77c10000 - 0x77c68000 C:\WINDOWS\system32\MSVCRT.dll
0x6d880000 - 0x6dc31000 C:\Program Files\Java\jdk1.5.0_06\jre\bin\server\jvm.dll
0x77d40000 - 0x77dd0000 C:\WINDOWS\system32\USER32.dll
0x77f10000 - 0x77f57000 C:\WINDOWS\system32\GDI32.dll
0x76b40000 - 0x76b6d000 C:\WINDOWS\system32\WINMM.dll
0x76390000 - 0x763ad000 C:\WINDOWS\system32\IMM32.DLL
0x629c0000 - 0x629c9000 C:\WINDOWS\system32\LPK.DLL
0x74d90000 - 0x74dfb000 C:\WINDOWS\system32\USP10.dll
0x6d2f0000 - 0x6d2f8000 C:\Program Files\Java\jdk1.5.0_06\jre\bin\hpi.dll
0x76bf0000 - 0x76bfb000 C:\WINDOWS\system32\PSAPI.DLL
0x6d6b0000 - 0x6d6bc000 C:\Program Files\Java\jdk1.5.0_06\jre\bin\verify.dll
0x6d370000 - 0x6d38d000 C:\Program Files\Java\jdk1.5.0_06\jre\bin\java.dll
0x6d6d0000 - 0x6d6df000 C:\Program Files\Java\jdk1.5.0_06\jre\bin\zip.dll
0x6d530000 - 0x6d543000 C:\Program Files\Java\jdk1.5.0_06\jre\bin\net.dll
0x71ab0000 - 0x71ac7000 C:\WINDOWS\system32\WS2_32.dll
0x71aa0000 - 0x71aa8000 C:\WINDOWS\system32\WS2HELP.dll
0x6d550000 - 0x6d559000 C:\Program Files\Java\jdk1.5.0_06\jre\bin\nio.dll
0x009b0000 - 0x009b6000 C:\Program Files\DTN\IQFeed\iq_jni.dll
0x49ea0000 - 0x49eba000 C:\Progra~1\DTN\IQFeed\iq32.DLL
0x71ad0000 - 0x71ad9000 C:\WINDOWS\system32\WSOCK32.dll
0x7c9c0000 - 0x7d1d5000 C:\WINDOWS\system32\SHELL32.dll
0x77f60000 - 0x77fd6000 C:\WINDOWS\system32\SHLWAPI.dll
0x774e0000 - 0x7761d000 C:\WINDOWS\system32\ole32.dll
0x77120000 - 0x771ac000 C:\WINDOWS\system32\OLEAUT32.dll
0x76b20000 - 0x76b31000 C:\WINDOWS\system32\ATL.DLL
0x76080000 - 0x760e5000 C:\WINDOWS\system32\MSVCP60.dll
0x76c90000 - 0x76cb8000 C:\WINDOWS\system32\IMAGEHLP.dll
0x773d0000 - 0x774d2000 C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.2600.2180_x-ww_a84f1ff9\comctl32.dll
0x5d090000 - 0x5d127000 C:\WINDOWS\system32\comctl32.dll

VM Arguments:
jvm_args: -enableassertions -enablesystemassertions -XX:+UseConcMarkSweepGC -XX:CMSTriggerRatio=20 -XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing -Xms256m -Xmx1024m -XX:OnError=userdump.exe %p -Xverify:all
java_command: bb.finance.trading.RoboTrader -directorySecurity E:/dataFinancial/stock_securityInfo -directorySymbolsToSubscribe ../output/SymbolsToSubscribe
Launcher Type: SUN_STANDARD

Environment Variables:
CLASSPATH=;..\lib\elliptic.jar;..\lib\bb.jar;C:\Progra~1\Jts\api.jar;C:\Progra~1\Jts\jts.jar;C:\Progra~1\DTN\IQFeed;"C:\Program Files\Java\jaf-1.0.2"\activation.jar;"C:\Program Files\Java\javamail-1.3.3_01"\mail.jar
PATH=C:\Program Files\Java\jdk1.5.0_06\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\Microsoft OEM Support Tools\userdump;C:\Progra~1\DTN\IQFeed
USERNAME=BrentAndLily
OS=Windows_NT
PROCESSOR_IDENTIFIER=x86 Family 15 Model 2 Stepping 9, GenuineIntel



--------------- S Y S T E M ---------------

OS: Windows XP Build 2600 Service Pack 2

CPU:total 2 family 15, cmov, cx8, fxsr, mmx, sse, sse2, ht

Memory: 4k page, physical 2095852k(1381508k free), swap 4038636k(3234544k free)

vm_info: Java HotSpot(TM) Server VM (1.5.0_06-b05) for windows-x86, built on Nov 10 2005 10:53:00 by "java_re" with MS VC++ 6.0



STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
After I noticed the above error, I tried manually executing the .bat file which runs my program. It crashed with what appeared to be the same error (well, it generated another hs_err_pidXXX.log file, but I did not have time to compare the file contents with the first one since I was frantically trying to get my program running before the stock market opened.)

I then tried to manually execute the program a second time. This time it worked just fine.

Obviously, the program did not change between all these 3 attempts, nor has it changed in the last couple of weeks (and it has worked all the other days), so I have no idea why it failed today the first 2 times.


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
I expected my program to run with no problems.
ACTUAL -
It crashed for some mysterious low level reason.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
My .bat file which executes the program has the following:

set errorHandling=-XX:OnError="userdump.exe %%p"
:: To understand the above, see
:: Section 3.2.4 starting at page 40 and 121 of http://java.sun.com/j2se/1.5/pdf/jdk50_ts_guide.pdf
:: (CRITICAL: must escape the % char if use it in an env var as above)
:: http://support.microsoft.com/?kbid=241215
:: http://download.microsoft.com/download/win2000srv/Utility/3.0/NT45/EN-US/Oem3sr2.zip
:: http://www-128.ibm.com/developerworks/java/library/j-tiger03175.html


The call to java to execute this program uses the above errorHandling env var, which means that when the program crashed, a fairly massive 350 MB windows core dump file was generated.

If you want access to that core file for analysis, you need to let me know how to ftp it to you or something.

REPRODUCIBILITY :
This bug can be reproduced occasionally.

---------- BEGIN SOURCE ----------
I think that the problem is not with my user level code but with the jvm or something else, as explained above.
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Try multiple time to launch my program, including possibly rebooting my box.

But this is fairly bad for a critical application like stock trading.

I would really like to determine why this error occurs and see if there is any way to prevent it.
*** NOTE ****

Synopsis changed from the overtly general:

Jvm crashed at [jvm.dll+0x893b9] with an EXCEPTION_ACCESS_VIOLATION

to the more specific:

iCMS crashes with Eden top past end

This will hopefully prevent falase and unintended aliasing of
developer votes for this bug. Please read the evaluation section
carefully before casting your ballot.