I am running an application on linux-cinnabar having an ATI Radeon 9600XT with OGL flag. This application displays all the available display modes and the user can select one from them and set it to the GraphicsDevice. I see 4 display modes being listed there. I am selecting one by one and setting it to the GD. After 4-5 times, I am seeing a VM crash on the console.

This is reproducible only on the PIT build and not reproducible on Mustang-b37 since FullScreen and Display Mode features are not available in the last promoted build. There is no VM crash when I don't specify OGL flag.

Here is the PIT build:
java version "1.6.0-2d.pit-jcg-win-03"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.6.0-2d.pit-jcg-win-03-java2d_26_may_2005_00_46-b00)
Java HotSpot(TM) Client VM (build 1.6.0-ea-b37, mixed mode)

I have attached a sample test. Execute the sample test on cinnabar with OGL flag. YOu would see a JTable with available display modes. Select the display modes one by one and click 'Set Display'. You would see a VM crash after 4-5 times.

Here is the crash info:
#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
# SIGSEGV (0xb) at pc=0x4e5f3264, pid=12139, tid=1308486576
#
# Java VM: Java HotSpot(TM) Client VM (1.6.0-ea-b37 mixed mode)
# Problematic frame:
# C [fglrx_dri.so+0x614264]
#

--------------- T H R E A D ---------------

Current thread (0x083d6048): JavaThread "Java2D Queue Flusher" daemon [_thread_in_native, id=12149]

siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0x000000ad

Registers:
EAX=0x09465160, EBX=0x09465160, ECX=0x00000000, EDX=0x00000000
ESP=0x4dfde380, EBP=0x4dfde3a8, ESI=0x09668060, EDI=0x00000011
EIP=0x4e5f3264, CR2=0x000000ad, EFLAGS=0x00010206

Top of Stack: (sp=0x4dfde380)
0x4dfde380: 082f9608 08dabcf8 4dfde3a8 4df3587f
0x4dfde390: 4002baf7 00000000 082f9608 09410848
0x4dfde3a0: 082f9608 02a00088 4dfde3d8 4df33dff
0x4dfde3b0: 082f9608 00000000 09465160 083d6048
0x4dfde3c0: 083d6048 00000000 900003e8 4dcace88
0x4dfde3d0: 08d84e50 08dabcf8 4dfde3f8 4dca1df0
0x4dfde3e0: 082f9608 09410848 083d60fc 4dcace88
0x4dfde3f0: 0834d0e8 083ce010 4dfde418 4dca1eb8

Instructions: (pc=0x4e5f3264)
0x4e5f3254: 00 00 8b 73 10 85 f6 74 11 8b 7e 4c 85 ff 74 0a
0x4e5f3264: 8b 8f 9c 00 00 00 85 c9 75 46 8b 73 14 89 1c 24

Stack: [0x4df5e000,0x4dfdf000), sp=0x4dfde380, free space=512k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [fglrx_dri.so+0x614264]
C [libGL.so.1+0x7adff]
C [libmawt.so+0x26df0]
C [libmawt.so+0x26eb8]
C [libmawt.so+0x24f04] Java_sun_java2d_opengl_OGLRenderQueue_flushBuffer+0x944
j sun.java2d.opengl.OGLRenderQueue.flushBuffer(Ljava/nio/ByteBuffer;I)V+0
j sun.java2d.opengl.OGLRenderQueue.flushBuffer()V+24
j sun.java2d.opengl.OGLRenderQueue.access$300(Lsun/java2d/opengl/OGLRenderQueue;)V+1
j sun.java2d.opengl.OGLRenderQueue$QueueFlusher.run()V+87
v ~StubRoutines::call_stub
V [libjvm.so+0x170742]
V [libjvm.so+0x257438]
V [libjvm.so+0x16ffa5]
V [libjvm.so+0x17003e]
V [libjvm.so+0x1d7894]
V [libjvm.so+0x2c0e63]
V [libjvm.so+0x257f0f]
C [libpthread.so.0+0x59ed]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j sun.java2d.opengl.OGLRenderQueue.flushBuffer(Ljava/nio/ByteBuffer;I)V+0
j sun.java2d.opengl.OGLRenderQueue.flushBuffer()V+24
j sun.java2d.opengl.OGLRenderQueue.access$300(Lsun/java2d/opengl/OGLRenderQueue;)V+1
j sun.java2d.opengl.OGLRenderQueue$QueueFlusher.run()V+87
v ~StubRoutines::call_stub

--------------- P R O C E S S ---------------

Java Threads: ( => current thread )
  0x087451b8 JavaThread "TimerQueue" daemon [_thread_blocked, id=12153]
  0x0805d000 JavaThread "DestroyJavaVM" [_thread_blocked, id=12139]
  0x0868c818 JavaThread "AWT-EventQueue-0" [_thread_blocked, id=12152]
  0x0868c218 JavaThread "AWT-Shutdown" [_thread_blocked, id=12151]
  0x08668e80 JavaThread "AWT-XAWT" daemon [_thread_blocked, id=12150]
=>0x083d6048 JavaThread "Java2D Queue Flusher" daemon [_thread_in_native, id=12149]
  0x080bed18 JavaThread "Java2D Disposer" daemon [_thread_blocked, id=12148]
  0x080c88f0 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=12146]
  0x080c73b8 JavaThread "CompilerThread0" daemon [_thread_blocked, id=12145]
  0x080c6460 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=12144]
  0x080c5828 JavaThread "Attach Listener" daemon [_thread_blocked, id=12143]
  0x080b9500 JavaThread "Finalizer" daemon [_thread_blocked, id=12142]
  0x080b8858 JavaThread "Reference Handler" daemon [_thread_blocked, id=12141]

Other Threads:
  0x080b41b0 VMThread [id=12140]
  0x080c9e20 WatcherThread [id=12147]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
 def new generation total 2048K, used 78K [0x45590000, 0x457c0000, 0x45a70000)
  eden space 1856K, 4% used [0x45590000, 0x455a3bc0, 0x45760000)
  from space 192K, 0% used [0x45760000, 0x45760000, 0x45790000)
  to space 192K, 0% used [0x45790000, 0x45790000, 0x457c0000)
 tenured generation total 25980K, used 18659K [0x45a70000, 0x473cf000, 0x49590000)
   the space 25980K, 71% used [0x45a70000, 0x46ca8c38, 0x46ca8e00, 0x473cf000)
 compacting perm gen total 8192K, used 6976K [0x49590000, 0x49d90000, 0x4d590000)
   the space 8192K, 85% used [0x49590000, 0x49c60310, 0x49c60400, 0x49d90000)
No shared spaces configured.

Dynamic libraries:
08048000-08057000 r-xp 00000000 00:20 1667152 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/bin/java
08057000-08059000 rw-p 0000e000 00:20 1667152 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/bin/java
08059000-09c2a000 rwxp 08059000 00:00 0
40000000-40016000 r-xp 00000000 03:02 15167 /lib/ld-2.3.3.so
40016000-40017000 rw-p 00016000 03:02 15167 /lib/ld-2.3.3.so
40017000-40018000 rw-p 40017000 00:00 0
40025000-40032000 r-xp 00000000 03:02 15195 /lib/tls/libpthread.so.0
40032000-40033000 rw-p 0000d000 03:02 15195 /lib/tls/libpthread.so.0
40033000-40035000 rw-p 40033000 00:00 0
40035000-40037000 r-xp 00000000 03:02 15176 /lib/libdl.so.2
40037000-40038000 rw-p 00002000 03:02 15176 /lib/libdl.so.2
40038000-40141000 r-xp 00000000 03:02 15193 /lib/tls/libc.so.6
40141000-40149000 rw-p 00109000 03:02 15193 /lib/tls/libc.so.6
40149000-4014d000 rw-p 40149000 00:00 0
4014d000-40480000 r-xp 00000000 00:20 1666507 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/client/libjvm.so
40480000-4049c000 rw-p 00333000 00:20 1666507 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/client/libjvm.so
4049c000-408b7000 rw-p 4049c000 00:00 0
408b7000-408b8000 r--p 408b7000 00:00 0
408b8000-408b9000 rwxp 408b8000 00:00 0
408b9000-408bf000 r-xp 00000000 00:20 1666500 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/native_threads/libhpi.so
408bf000-408c0000 rw-p 00006000 00:20 1666500 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/native_threads/libhpi.so
408c4000-408e5000 r-xp 00000000 03:02 15194 /lib/tls/libm.so.6
408e5000-408e6000 rw-p 00020000 03:02 15194 /lib/tls/libm.so.6
408e6000-408ee000 rw-s 00000000 03:02 173013 /tmp/hsperfdata_pm143956/12139
408f3000-40905000 r-xp 00000000 03:02 15179 /lib/libnsl.so.1
40905000-40906000 rw-p 00011000 03:02 15179 /lib/libnsl.so.1
40906000-40908000 rw-p 40906000 00:00 0
40908000-4090f000 r-xp 00000000 03:02 15180 /lib/libnss_compat.so.2
4090f000-40910000 rw-p 00006000 03:02 15180 /lib/libnss_compat.so.2
40910000-40918000 r-xp 00000000 03:02 15184 /lib/libnss_nis.so.2
40918000-40919000 rw-p 00007000 03:02 15184 /lib/libnss_nis.so.2
40919000-40921000 r-xp 00000000 03:02 15182 /lib/libnss_files.so.2
40921000-40922000 rw-p 00008000 03:02 15182 /lib/libnss_files.so.2
40922000-4092d000 r-xp 00000000 00:20 1666513 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/libverify.so
4092d000-4092e000 rw-p 0000b000 00:20 1666513 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/libverify.so
4092e000-4094f000 r-xp 00000000 00:20 1666514 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/libjava.so
4094f000-40951000 rw-p 00020000 00:20 1666514 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/libjava.so
40951000-40960000 r-xp 00000000 00:20 1666516 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/libzip.so
40960000-40962000 rw-p 0000e000 00:20 1666516 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/libzip.so
40962000-4306c000 r--s 00000000 00:20 1666563 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/rt.jar
4306c000-43095000 rw-p 4306c000 00:00 0
43095000-430da000 r--s 00000000 00:20 1666562 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/jsse.jar
430da000-430ee000 r--s 00000000 00:20 1666554 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/jce.jar
430ee000-43501000 r--s 00000000 00:20 1667061 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/charsets.jar
43501000-43609000 rwxp 43501000 00:00 0
43609000-45501000 rwxp 43609000 00:00 0
45501000-45506000 rwxp 45501000 00:00 0
45506000-45581000 rwxp 45506000 00:00 0
45581000-45583000 rw-s e16d4000 03:02 27655 /dev/dri/card0
45583000-45585000 rw-s e16d6000 03:02 27655 /dev/dri/card0
45585000-45587000 rw-s e16d8000 03:02 27655 /dev/dri/card0
45587000-4558a000 rw-s e262d000 03:02 27655 /dev/dri/card0
45590000-457c0000 rwxp 45590000 00:00 0
457c0000-45a70000 rwxp 457c0000 00:00 0
45a70000-473cf000 rwxp 45a70000 00:00 0
473cf000-49590000 rwxp 473cf000 00:00 0
49590000-49d90000 rwxp 49590000 00:00 0
49d90000-4d590000 rwxp 49d90000 00:00 0
4d590000-4d592000 rwxp 4d590000 00:00 0
4d592000-4d5a0000 rwxp 4d592000 00:00 0
4d5a0000-4d5b0000 rwxp 4d5a0000 00:00 0
4d5b0000-4d5b4000 rwxp 4d5b0000 00:00 0
4d5b4000-4d5d0000 rwxp 4d5b4000 00:00 0
4d5d0000-4d5de000 rwxp 4d5d0000 00:00 0
4d5de000-4d5ef000 rwxp 4d5de000 00:00 0
4d5ef000-4d5f4000 rwxp 4d5ef000 00:00 0
4d5f4000-4d610000 rwxp 4d5f4000 00:00 0
4d610000-4d611000 ---p 4d610000 00:00 0
4d611000-4d691000 rwxp 4d611000 00:00 0
4d691000-4d694000 ---p 4d691000 00:00 0
4d694000-4d712000 rwxp 4d694000 00:00 0
4d712000-4d715000 ---p 4d712000 00:00 0
4d715000-4d793000 rwxp 4d715000 00:00 0
4d793000-4d7c6000 r--p 00000000 03:02 18516 /usr/lib/locale/en_US.utf8/LC_CTYPE
4d7c6000-4d7cc000 r--s 00000000 03:02 15931 /usr/lib/gconv/gconv-modules.cache
4d7cc000-4d7cf000 ---p 4d7cc000 00:00 0
4d7cf000-4d84d000 rwxp 4d7cf000 00:00 0
4d84d000-4d850000 ---p 4d84d000 00:00 0
4d850000-4d8ce000 rwxp 4d850000 00:00 0
4d8ce000-4d8d1000 ---p 4d8ce000 00:00 0
4d8d1000-4d94f000 rwxp 4d8d1000 00:00 0
4d94f000-4d952000 ---p 4d94f000 00:00 0
4d952000-4d9d0000 rwxp 4d952000 00:00 0
4d9d0000-4d9d1000 ---p 4d9d0000 00:00 0
4d9d1000-4da51000 rwxp 4d9d1000 00:00 0
4da51000-4da77000 r--s 00000000 00:20 1666494 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/ext/sunjce_provider.jar
4da77000-4daa2000 r--s 00000000 00:20 1666495 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/ext/sunpkcs11.jar
4daa2000-4daa5000 r--s 00000000 00:20 1666496 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/ext/dnsns.jar
4daa5000-4db14000 r--s 00000000 00:20 1666497 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/ext/localedata.jar
4db14000-4db89000 r-xp 00000000 00:20 1666529 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/libawt.so
4db89000-4db90000 rw-p 00074000 00:20 1666529 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/libawt.so
4db90000-4dbb4000 rw-p 4db90000 00:00 0
4dbb4000-4dc7a000 r-xp 00000000 00:20 1666528 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/libmlib_image.so
4dc7a000-4dc7b000 rw-p 000c5000 00:20 1666528 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/libmlib_image.so
4dc7b000-4dcaa000 r-xp 00000000 00:20 1666533 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/xawt/libmawt.so
4dcaa000-4dcae000 rw-p 0002e000 00:20 1666533 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/xawt/libmawt.so
4dcae000-4dcbc000 r-xp 00000000 03:02 24608 /usr/X11R6/lib/libXext.so.6.4
4dcbc000-4dcbd000 rw-p 0000d000 03:02 24608 /usr/X11R6/lib/libXext.so.6.4
4dcbd000-4ddb5000 r-xp 00000000 03:02 24597 /usr/X11R6/lib/libX11.so.6.2
4ddb5000-4ddb9000 rw-p 000f7000 03:02 24597 /usr/X11R6/lib/libX11.so.6.2
4ddb9000-4de29000 r-xp 00000000 00:20 1666538 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/libfontmanager.so
4de29000-4de33000 rw-p 00070000 00:20 1666538 /net/sqindia/export/disk05/2d-SQE/PIT/2005-05-25.mustang/linux-i586/jre/lib/i386/libfontmanager.so
4de33000-4de38000 rw-p 4de33000 00:00 0
4de38000-4de3b000 ---p 4de38000 00:00 0
4de3b000-4deb9000 rwxp 4de3b000 00:00 0
4deb9000-4df56000 r-xp 00000000 03:02 172813 /usr/X11R6/lib/libGL.so.1.2
4df56000-4df5b000 rw-p 0009c000 03:02 172813 /usr/X11R6/lib/libGL.so.1.2
4df5b000-4df5e000 rw-p 4df5b000 00:00 0
4df5e000-4df61000 ---p 4df5e000 00:00 0
4df61000-4dfdf000 rwxp 4df61000 00:00 0
4dfdf000-4e800000 r-xp 00000000 03:02 172816 /usr/X11R6/lib/modules/dri/fglrx_dri.so
4e800000-4e822000 rw-p 00820000 03:02 172816 /usr/X11R6/lib/modules/dri/fglrx_dri.so
4e822000-4e87e000 rw-p 4e822000 00:00 0
4e87e000-4e881000 rw-s e3512000 03:02 27655 /dev/dri/card0
4e88b000-4e92d000 r-xp 00000000 03:02 19230 /usr/lib/libstdc++.so.5.0.6
4e92d000-4e943000 rw-p 000a1000 03:02 19230 /usr/lib/libstdc++.so.5.0.6
4e943000-4e948000 rw-p 4e943000 00:00 0
4e948000-4e94f000 r-xp 00000000 03:02 15197 /lib/tls/librtkaio.so.1
4e94f000-4e950000 rw-p 00006000 03:02 15197 /lib/tls/librtkaio.so.1
4e950000-4e957000 r-xp 00000000 03:02 18895 /lib/libgcc_s.so.1
4e957000-4e958000 rw-p 00007000 03:02 18895 /lib/libgcc_s.so.1
4e958000-56958000 rw-s e0000000 03:02 27655 /dev/dri/card0
56958000-5695a000 rw-s e284e000 03:02 27655 /dev/dri/card0
5695a000-5696a000 rw-s fcff0000 03:02 27655 /dev/dri/card0
5696a000-5696b000 rw-s c3764000 03:02 27655 /dev/dri/card0
5696b000-56a6b000 rw-s f0001000 03:02 27655 /dev/dri/card0
56a6b000-56d27000 rw-s f0101000 03:02 27655 /dev/dri/card0
56d27000-56e67000 rw-s f03bd000 03:02 27655 /dev/dri/card0
56e67000-56f03000 rw-p 56e67000 00:00 0
56f03000-56f06000 rw-s e1101000 03:02 27655 /dev/dri/card0
56f06000-56f3c000 rw-p 56f06000 00:00 0
56f3c000-56f3d000 rw-s e1104000 03:02 27655 /dev/dri/card0
56f3d000-56f3e000 rw-s e1105000 03:02 27655 /dev/dri/card0
56f3e000-56f3f000 rw-s e1106000 03:02 27655 /dev/dri/card0
56f4c000-5714d000 rw-p 56f4c000 00:00 0
5714d000-57150000 rw-s e1107000 03:02 27655 /dev/dri/card0
57150000-57186000 rw-p 57150000 00:00 0
57186000-57187000 r-xp 00000000 03:02 24505 /usr/X11R6/lib/X11/locale/lib/common/xlcUTF8Load.so.2
57187000-57188000 rw-p 00000000 03:02 24505 /usr/X11R6/lib/X11/locale/lib/common/xlcUTF8Load.so.2
57189000-57689000 rw-s e0701000 03:02 27655 /dev/dri/card0
57689000-57b89000 rw-s e7b00000 03:02 27655 /dev/dri/card0
57b89000-57bae000 rw-p 57b89000 00:00 0
57bae000-57bb6000 r-xp 00000000 03:02 107100 /usr/X11R6/lib/libXcursor.so.1.0.2
57bb6000-57bb7000 rw-p 00007000 03:02 107100 /usr/X11R6/lib/libXcursor.so.1.0.2
57bb7000-57bbe000 r-xp 00000000 03:02 24630 /usr/X11R6/lib/libXrender.so.1.2.2
57bbe000-57bbf000 rw-p 00006000 03:02 24630 /usr/X11R6/lib/libXrender.so.1.2.2
57bbf000-57bc2000 ---p 57bbf000 00:00 0
57bc2000-57c40000 rwxp 57bc2000 00:00 0
57c40000-57c43000 ---p 57c40000 00:00 0
57c43000-57cc1000 rwxp 57c43000 00:00 0
57cc1000-57cc4000 ---p
I installed the latest driver from ATI (8.20.8) on Linux and tried this scenario and the crash is still reproducible. But the log file shows a slightly different trace now with the new driver. I have attached the log file to this bug report.