Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-6306367

multistress scenarios crashes with JVMTIagent attached

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: P3 P3
    • 6
    • 6
    • hotspot
    • 6
    • b79
    • generic
    • generic
    • Verified

      RAS VM test framework crashes with nsk JVMTIagent attached. Exact point of crash differ from time to time. See attachment for details.

      PS sources of JVMTIAgent located inside src/nsk/share in VM testbase
      In October 2005 when I took a look into this bug I observed several failure modes.

      1. One of them looked like a compiler issue, but now it is not reproducible anymore.
         I guess, this problem has been already fixed.
      #
      # An unexpected error has been detected by HotSpot Virtual Machine:
      #
      # Internal Error (/net/prt-solsparc-q1-16/tmp/PrtBuildDir/workspace/src/share/vm/code/nmethod.cpp, 1514 [ Patched ]), pid=12802, tid=1199
      #
      # Java VM: Java HotSpot(TM) Server VM (20051013170524.lewy.hotspot-debug compiled mode)
      #
      # Error: unhandled implicit exception in compiled code

      --------------- T H R E A D ---------------

      Current thread (0x001dff00): JavaThread "Thread-1187" [_thread_in_Java, id=1199]

      Stack: [0xf0e40000,0xf0e80000), sp=0xf0e7f058, free space=252k
      Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
      V [libjvm.so+0xf9f224];; void VMError::report_and_die()+0x7c8
      V [libjvm.so+0x475f44];; void report_fatal(const char*,int,const char*)+0x6c
      V [libjvm.so+0xc40fe0];; unsigned char*nmethod::continuation_for_implicit_exception(unsigned char*)+0x230
      V [libjvm.so+0xddadec];; unsigned char*SharedRuntime::continuation_for_implicit_exception(JavaThread*,unsigned char*,SharedRuntime::ImplicitExceptionKind)+0x2d4
      V [libjvm.so+0xc8aaec];; JVM_handle_solaris_signal+0x750
      C [libc.so.1+0xbc534];; __sighndlr+0x14
      C [libc.so.1+0xb19a0];; call_user_handler+0x3c0
      J multistress.WorkThread.run()V
      v ~StubRoutines::call_stub
      V [libjvm.so+0x645a20];; void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0xaf4
      V [libjvm.so+0x643acc];; void JavaCalls::call_virtual(JavaValue*,KlassHandle,symbolHandle,symbolHandle,JavaCallArguments*,Thread*)+0x438
      V [libjvm.so+0x643c7c];; void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*)+0xfc
      V [libjvm.so+0x825bac];; void thread_entry(JavaThread*,Thread*)+0x2c8
      V [libjvm.so+0xeaf9a0];; void JavaThread::thread_main_inner()+0x228
      V [libjvm.so+0xc7ef00];; java_start+0x144

      2. The second failure mode with this stress test case looked like a JNI handle
         issue (below). Now it is not reproducible as well.

      #
      # An unexpected error has been detected by HotSpot Virtual Machine:
      #
      # SIGSEGV (0xb) at pc=0xfd9c94c4, pid=10073, tid=2984
      #
      # Java VM: Java HotSpot(TM) Server VM (20051013144526.lewy.e-debug mixed mode)
      # Problematic frame:
      # V [libjvm.so+0x5c94c4]
       ....
      Stack: [0xef080000,0xef17c000), sp=0xef17bb90, free space=1006k
      Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
      V [libjvm.so+0x5c94c4];; oop CollectedHeap::obj_allocate(KlassHandle,int,Thread*)+0x1f8
      V [libjvm.so+0x5aa528];; instanceOop instanceKlass::allocate_instance(Thread*)+0x1f8
      V [libjvm.so+0x2337ec];; instanceHandle instanceKlass::allocate_instance_handle(Thread*)+0x2c
      V [libjvm.so+0xeacf38];; void JavaThread::allocate_threadObj(Handle,char*,bool,Thread*)+0x49c
      V [libjvm.so+0x74bf88];; jni_AttachCurrentThread+0x4c0
      C [libnative.so+0x5cc];; zzz+0x64
      [error occurred during error reporting, step 120, id 0xb]

      3. The third failure mode is a naked oop issue in the JVM_InvokeMethod function.
         Now it is the only failure mode left.
         It is reproducible on Solaris sparc/x86 with both Client and Server VM.
         This stack trace below:

      #
      # An unexpected error has been detected by Java Runtime Environment:
      #
      # SIGSEGV (0xb) at pc=0xddf74be5, pid=20269, tid=3373
      #
      # Java VM: Java HotSpot(TM) Client VM (1.6.0-internal-debug mixed mode)
      # Problematic frame:
      # V [libjvm.so+0x374be5]
      #
      # If you would like to submit a bug report, please visit:
      # http://java.sun.com/webapps/bugreport/crash.jsp
      #

      --------------- T H R E A D ---------------

      Current thread (0x082e4c00): JavaThread "Thread-3361" [_thread_in_vm, id=3373]

      siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0xbaadbac6;;
      ;; si_signo=11 SIGSEGV
      ;; si_code=1 SEGV_MAPERR /* Address not mapped to object. */

      Registers:
      EAX=0x00000008, EBX=0xdece1bf0, ECX=0xbaadbabe, EDX=0x000000ab
      ESP=0xd290fb94, EBP=0xd290fbb8, ESI=0x082e4c00, EDI=0xd290fc60
      EIP=0xddf74be5, EFLAGS=0x00010296

      Top of Stack: (sp=0xd290fb94)
      0xd290fb94: ded1170f 082e4c00 dece1bf0 d76442b0
      0xd290fba4: d7400d60 ddf84248 dece1bf0 d290fc48
      0xd290fbb4: d290fc74 d290fc74 de09c03d d290fc60
      0xd290fbc4: 082e4c00 d762b6d8 def72000 d290fd90
      0xd290fbd4: db815d94 d290fdbc d290fd90 db815d94
      0xd290fbe4: ded25cac ded2e640 ded117b7 d2910d94
      0xd290fbf4: 00001000 00000001 00000000 de3f0300
      0xd290fc04: ded193f8 d34dc6e8 d34dc6e8 fefefefe

      Instructions: (pc=0xddf74be5)
      0xddf74bd5: d0 d6 00 8b 7d 08 8b 0f 8b 83 9c 25 00 00 8b 00
      0xddf74be5: 8b 04 01 89 45 f4 8b b3 20 00 00 00 80 3e 00 75
      #
      # An unexpected error has been detected by Java Runtime Environment:
      #
      # SIGSEGV (0xb) at pc=0xddf74be5, pid=20269, tid=3373
      #
      # Java VM: Java HotSpot(TM) Client VM (1.6.0-internal-debug mixed mode)
      # Problematic frame:
      # V [libjvm.so+0x374be5]
      #
      # If you would like to submit a bug report, please visit:
      # http://java.sun.com/webapps/bugreport/crash.jsp
      #

      --------------- T H R E A D ---------------

      Current thread (0x082e4c00): JavaThread "Thread-3361" [_thread_in_vm, id=3373]

      siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0xbaadbac6;;
      ;; si_signo=11 SIGSEGV
      ;; si_code=1 SEGV_MAPERR /* Address not mapped to object. */

      Registers:
      EAX=0x00000008, EBX=0xdece1bf0, ECX=0xbaadbabe, EDX=0x000000ab
      ESP=0xd290fb94, EBP=0xd290fbb8, ESI=0x082e4c00, EDI=0xd290fc60
      EIP=0xddf74be5, EFLAGS=0x00010296

      Top of Stack: (sp=0xd290fb94)
      0xd290fb94: ded1170f 082e4c00 dece1bf0 d76442b0
      0xd290fba4: d7400d60 ddf84248 dece1bf0 d290fc48
      0xd290fbb4: d290fc74 d290fc74 de09c03d d290fc60
      0xd290fbc4: 082e4c00 d762b6d8 def72000 d290fd90
      0xd290fbd4: db815d94 d290fdbc d290fd90 db815d94
      0xd290fbe4: ded25cac ded2e640 ded117b7 d2910d94
      0xd290fbf4: 00001000 00000001 00000000 de3f0300
      0xd290fc04: ded193f8 d34dc6e8 d34dc6e8 fefefefe

      Instructions: (pc=0xddf74be5)
      0xddf74bd5: d0 d6 00 8b 7d 08 8b 0f 8b 83 9c 25 00 00 8b 00
      0xddf74be5: 8b 04 01 89 45 f4 8b b3 20 00 00 00 80 3e 00 75
      ;; ddf74bd5 d0 d6 rcl %dh
      ;; ddf74bd7 00 8b 7d 08 8b 0f add %cl,0xf8b087d(%ebx)
      ;; ddf74bdd 8b 83 9c 25 00 00 mov 0x259c(%ebx),%eax
      ;; ddf74be3 8b 00 mov (%eax),%eax
      ;; ---------------
      ;; ddf74be5 8b 04 01 mov (%ecx,%eax,1),%eax
      ;; ddf74be8 89 45 f4 mov %eax,0xfffffff4(%ebp)
      ;; ddf74beb 8b b3 20 00 00 00 mov 0x20(%ebx),%esi
      ;; ddf74bf1 80 3e 00 cmpb $0x0,(%esi)
      ;; ddf74bf4 75 ff jne 0xddf74bf5
      ;;
      Stack: [0xd28c1000,0xd2910d94), sp=0xd290fb94, free space=314k
      Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
      V [libjvm.so+0x374be5];; bool java_lang_Class::is_primitive(oop)+0x25
      V [libjvm.so+0x49c03d];; JVM_InvokeMethod+0xb31
      C [libjava.so+0xe81b] Java_sun_reflect_NativeMethodAccessorImpl_invoke0+0x27;; Java_sun_reflect_NativeMethodAccessorImpl_invoke0+0x27
      j sun.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+0
      j sun.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+87
      j sun.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+6
      j java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+111
      j multistress.Tester_2.test(I)V+76
      v ~StubRoutines::call_stub
      V [libjvm.so+0x36f176];; void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x68a
      V [libjvm.so+0x707407];; void os::os_exception_wrapper(void(*)(JavaValue*,methodHandle*,JavaCallArguments*,Thread*),JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x27
      V [libjvm.so+0x36eac0];; void JavaCalls::call(JavaValue*,methodHandle,JavaCallArguments*,Thread*)+0x40
      V [libjvm.so+0x76117d];; oop Reflection::invoke(instanceKlassHandle,methodHandle,Handle,bool,objArrayHandle,BasicType,objArrayHandle,bool,Thread*)+0x2a7d
      V [libjvm.so+0x76eb73];; oop Reflection::invoke_method(oop,Handle,objArrayHandle,Thread*)+0x947
      V [libjvm.so+0x49bdec];; JVM_InvokeMethod+0x8e0
      C [libjava.so+0xe81b] Java_sun_reflect_NativeMethodAccessorImpl_invoke0+0x27;; Java_sun_reflect_NativeMethodAccessorImpl_invoke0+0x27
      j sun.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+0
      j sun.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+87
      j sun.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+6
      j java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+111
      j multistress.RecursiveClassLoading.run()V+67
      j multistress.WeightThread.run()V+9
      v ~StubRoutines::call_stub
      V [libjvm.so+0x36f176];; void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x68a
      V [libjvm.so+0x707407];; void os::os_exception_wrapper(void(*)(JavaValue*,methodHandle*,JavaCallArguments*,Thread*),JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x27
      V [libjvm.so+0x36eac0];; void JavaCalls::call(JavaValue*,methodHandle,JavaCallArguments*,Thread*)+0x40
      V [libjvm.so+0x36dde2];; void JavaCalls::call_virtual(JavaValue*,KlassHandle,symbolHandle,symbolHandle,JavaCallArguments*,Thread*)+0x2b2
      V [libjvm.so+0x36dff4];; void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*)+0xd0
      V [libjvm.so+0x4764ad];; void thread_entry(JavaThread*,Thread*)+0x211
      V [libjvm.so+0x81b9b2];; void JavaThread::thread_main_inner()+0x16e
      V [libjvm.so+0x81b75c];; void JavaThread::run()+0x348
      V [libjvm.so+0x7020bb];; java_start+0xe3

      Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
      j sun.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+0
      j sun.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+87
      j sun.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+6
      j java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+111
      j multistress.Tester_2.test(I)V+76
      v ~StubRoutines::call_stub
      j sun.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+0
      j sun.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+87
      j sun.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+6
      j java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+111
      j multistress.RecursiveClassLoading.run()V+67
      j multistress.WeightThread.run()V+9
      v ~StubRoutines::call_stub

      --------------- P R O C E S S ---------------

      Java Threads: ( => current thread )
      =>0x082e4c00 JavaThread "Thread-3361" [_thread_in_vm, id=3373]
        0x081bb000 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=10]
        0x081b8c00 JavaThread "CompilerThread0" daemon [_thread_in_native, id=9]
        0x081b7000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=8]
        0x0819c400 JavaThread "Finalizer" daemon [_thread_blocked, id=7]
        0x0819b000 JavaThread "Reference Handler" daemon [_thread_blocked, id=6]
        0x08087800 JavaThread "main" [_thread_blocked, id=4]

      Other Threads:
        0x08195c00 VMThread [id=5]
        0x081bd400 WatcherThread [id=11]

      VM state:not at safepoint (normal execution)

      VM Mutex/Monitor currently owned by a thread: None

      Heap
      ...

      Dynamic libraries:
      0x08050000 /net/tomsk.sfbay/export/home/ss45998/1.5/hs_jref.b72/i386/jdk1.6.0/bin/java
      0xdfb60000 /usr/lib/libthread.so.1
      0xdfb40000 /net/tomsk.sfbay/export/home/ss45998/1.5/hs_jref.b72/i386/jdk1.6.0/bin/../jre/lib/i386/jli/libjli.so
      0xdfb30000 /usr/lib/libdl.so.1
      0xdfa70000 /usr/lib/libc.so.1
      0xddc00000 /net/tomsk.sfbay/export/home/ss45998/1.5/hs_jref.b72/i386/jdk1.6.0/jre/lib/i386/client/libjvm.so
      0xdfa30000 /usr/lib/libsocket.so.1
      0xdfa00000 /usr/lib/libsched.so.1
      0xdf9d0000 /usr/lib/libCrun.so.1
      0xdf9b0000 /usr/lib/libm.so.1
      0xdf990000 /usr/lib/libdoor.so.1
      0xdf8e0000 /usr/lib/libnsl.so.1
      0xdf8c0000 /usr/lib/libmp.so.2
      0xdefe0000 /net/tomsk.sfbay/export/home/ss45998/1.5/hs_jref.b72/i386/jdk1.6.0/jre/lib/i386/native_threads/libhpi.so
      0xdefa0000 libs/solaris-i586/libJVMTIagent.so
      0xdef80000 /net/tomsk.sfbay/export/home/ss45998/1.5/hs_jref.b72/i386/jdk1.6.0/jre/lib/i386/libverify.so
      0xdef40000 /net/tomsk.sfbay/export/home/ss45998/1.5/hs_jref.b72/i386/jdk1.6.0/jre/lib/i386/libjava.so
      0xdef10000 /net/tomsk.sfbay/export/home/ss45998/1.5/hs_jref.b72/i386/jdk1.6.0/jre/lib/i386/libzip.so

      VM Arguments:
      jvm_args: -agentlib:JVMTIagent
      java_command: multistress.Main --multistress.Main.TIMEOUT=60000
      Launcher Type: SUN_STANDARD

      Environment Variables:
      JAVA_HOME=/usr/j2se
      CLASSPATH=/usr/j2se/lib:.
      PATH=/usr/ccs/bin:/usr/bin:/usr/sbin:/home/ss45998/bin:/home/ss45998/bin/intel-S2:/usr/dt/bin:/usr/dist/exe:/usr/openwin/bin:/java/devtools/i386/SUNWspro/SS11/bin:/java/devtools/i386/SUNWspro/SS11/bin:/java/devtools/i386/SUNWspro/SS10/bin:/java/devtools/i386/SUNWspro/SOS8/bin:/java/devtools/i386/bin:/usr/dist/share/forte_dev_i386/SUNWspro/bin:/net/jano.sfbay/export/disk20/serviceability/tools/bin:/usr/lib:/usr/ucb:/sbin:/etc:.
      LD_LIBRARY_PATH=/net/tomsk.sfbay/export/home/ss45998/1.5/hs_jref.b72/i386/jdk1.6.0/jre/lib/i386/client:/net/tomsk.sfbay/export/home/ss45998/1.5/hs_jref.b72/i386/jdk1.6.0/jre/lib/i386:/net/tomsk.sfbay/export/home/ss45998/1.5/hs_jref.b72/i386/jdk1.6.0/jre/../lib/i386:libs/solaris-i586
      SHELL=/bin/csh
      DISPLAY=bratsk:1.0
      ...

      --------------- S Y S T E M ---------------

      OS: Solaris 8 2/02 s28x_u7wos_08a INTEL
                 Copyright 2002 Sun Microsystems, Inc. All Rights Reserved.
                                 Assembled 18 December 2001

      uname:SunOS 5.8 Generic_117351-32 i86pc (T1 libthread)
      rlimit: STACK 8480k, CORE 0k, NOFILE 1024, AS infinity
      load average:5.48 4.59 4.47

      CPU:total 2 family 15, cmov, cx8, fxsr, mmx, mmxext, 3dnowext, 3dnow

      Memory: 4k page, physical 2096700k(1673968k free)

      vm_info: Java HotSpot(TM) Client VM (1.6.0-internal) for solaris-x86, built on Mar 3 2006 14:29:58 by "ss45998" with unknown Workshop:0x580

            sspitsyn Serguei Spitsyn
            azhmursunw Anatoly Zhmur (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved:
              Imported:
              Indexed: