Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-6392004

Self-suspension holding Heap_lock blocks the use of FollowReferences in agent thread

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: P3 P3
    • 6
    • 6
    • hotspot
    • None
    • generic
    • generic

      The attached script, runnbx.sh, runs NetBeans 4.1 under a simple debugger.
      The debugger polls the keyboard for input. When 'Enter' is pressed,
      the debugger calls JDI methods:
        - VirtualMachine.suspend()
        - VirtualMachine.allClasses()
        - VirutalMachine.instanceCounts on all the classes

      Rarely, a hang occurs here in the NetBeans process:

      ----------------- t@7 -----------------
      0xff29f760 ___lwp_mutex_lock + 0x8
      0xfe8858cc void Mutex::lock() + 0x144
      0xfecc3f64 void JvmtiTagMap::follow_references(int,KlassHandle,_jobject*,const _jvmtiHeapCallbacks*,const void*) + 0xd0
      0xfec8e4d0 jvmtiError JvmtiEnv::FollowReferences(int,_jclass*,_jobject*,const _jvmtiHeapCallbacks*,const void*) + 0x200
      0xfec12008 jvmti_FollowReferences + 0x38c
      0xfe7e3698 classInstanceCounts + 0x22c
      0xfe7bc47c instanceCounts + 0x12c
      0xfe7c3960 debugLoop_run + 0x1bc
      0xfe7da108 connectionInitiated + 0xf8
      0xfe7da418 attachThread + 0x64
      0xfecb2600 void JvmtiAgentThread::call_start_function() + 0xe0
      0xfedca3d0 void JavaThread::thread_main_inner() + 0x38
      0xfed3e2ec java_start + 0x138
      0xff3857b4 _lwp_start

      See the attached file that contains the stack dumps for all threads.

      I believe that this thread is waiting for the Heap_lock.

      A full gcore dump is in
        /net/jano.sfbay/export/disk20/serviceability/ws/regressionTestsInWaiting/core.followReferences
      Finally reproduced this hang with Mustang-B75. I've attached
      a thread dump for my hang as threads.log.17708. Here are the
      two interesting threads:

      THREAD t@9

      This is our debugger threads waiting for a lock:

      t@9(l@11) stopped in ___lwp_mutex_lock at 0xff29f990
      0xff29f990: ___lwp_mutex_lock+0x0008: ta 8
      current thread: t@9
      =>[1] ___lwp_mutex_lock(0x31200, 0xa, 0xff296ed4, 0x2b9f8, 0xff130000, 0xfe756d38), at 0xff29f990
        [2] Mutex::wait_for_lock_blocking_implementation(0x2c7c8, 0x104c00, 0x1ffc, 0x1, 0xfe72e000, 0x20980), at 0xfe1a6564
        [3] Mutex::lock(0x2c7c8, 0x104c00, 0xed5, 0x76ac0e, 0xfe752fdc, 0xfe72e000), at 0xfe0859ac
        [4] JvmtiTagMap::follow_references(0x1b8e680, 0x20, 0xed581774, 0x0, 0xed581878, 0xed5818b8), at 0xfe4c3ff4
        [5] JvmtiEnv::FollowReferences(0x1c593f0, 0x20, 0x36c, 0x0, 0xed581878, 0xed5818b8), at 0xfe48e540
        [6] jvmti_FollowReferences(0x1c593f0, 0x20, 0x0, 0x0, 0xed581878, 0xed5818b8), at 0xfe412078
        [7] classInstanceCounts(0xfe7face0, 0x1c593f0, 0x0, 0x0, 0xfe7face0, 0x0), at 0xfe7e3698
        [8] instanceCounts(0xed581b00, 0xed5819a0, 0x1dda370, 0x104ce4, 0xfe7fa714, 0x400), at 0xfe7bc47c
        [9] debugLoop_run(0x400, 0x0, 0xfe7fa714, 0x1, 0xfe7face0, 0xfe7bc350), at 0xfe7c3960
        [10] connectionInitiated(0xfde22bf0, 0xa4c, 0x1, 0xfe7fa714, 0xfe7fb160, 0x800), at 0xfe7da108
        [11] attachThread(0xfe7fac68, 0x554, 0xfde22bf0, 0xfe7fa714, 0x2035c, 0x400), at 0xfe7da418
        [12] JvmtiAgentThread::call_start_function(0x104c00, 0x29714, 0x2b800, 0xfe72e000, 0x29400, 0x7), at 0xfe4b2694
        [13] JavaThread::thread_main_inner(0x104c00, 0xed825ec0, 0x1043a8, 0xfe72e000, 0x11f018, 0x0), at 0xfe5ca358
        [14] java_start(0x104c00, 0x5, 0x452c, 0xfe72e000, 0xfe6a3b89, 0x29400), at 0xfe53e374

      Several other threads are self-suspend, but this one is interesting:

      THREAD t@26

      This thread has self-suspended while waiting for a VM operation to
      complete. I'm guessing since there is a GenCollectorPolicy::mem_allocate_work()
      call on the stack, that this thread is holding the Heap_lock.

      t@26(l@28) stopped in ___lwp_cond_wait at 0xff29fa08
      0xff29fa08: ___lwp_cond_wait+0x0004: ta 8
      current thread: t@26
      =>[1] ___lwp_cond_wait(0x4e0880, 0x4e0868, 0x0, 0x0, 0x0, 0x0), at 0xff29fa08
        [2] _lwp_cond_wait(0x4e0880, 0x4e0868, 0xfe767630, 0x1406f40, 0x0, 0x1), at 0xff296f4c
        [3] Monitor::wait(0x6516b0, 0x4e0400, 0x0, 0xfe72e000, 0x4e0840, 0x0), at 0xfe52e04c
        [4] os::pd_self_suspend_thread(0x4e0400, 0x28e40, 0xec0, 0x1, 0x80000000, 0xfe72e000), at 0xfe53eb5c
        [5] JavaThread::java_suspend_self(0x40000000, 0x1, 0x4e0400, 0x1406f40, 0x0, 0xdeac), at 0xfe5cbea4
        [6] Monitor::wait(0x30e78, 0x4e0400, 0x4e09e0, 0xfe72e000, 0x30ea8, 0x4e0400), at 0xfe52e668
        [7] VMThread::execute(0xec000b1c, 0x35c00, 0xfe75afe8, 0x4e0400, 0x1, 0xfe72e000), at 0xfe1797cc
        [8] GenCollectorPolicy::mem_allocate_work(0x34ec8, 0x804, 0x67, 0xec000bd3, 0xec000b58, 0xfe72e000), at 0xfe2d7ab0
        [9] instanceKlass::allocate_objArray(0xfe340954, 0x69d7d0, 0x800, 0x804, 0x4e0400, 0x2d93c), at 0xfe0982c0
        [10] InterpreterRuntime::anewarray(0x4e0400, 0xf7761d70, 0x8, 0x800, 0x6, 0xfe72e000), at 0xfe098a5c
        [11] 0xfbc2065c(0xee4eb200, 0xb7, 0xf78c9cd0, 0xfbc1f2a0, 0x71a, 0xec000cc0), at 0xfbc2065b
        [12] 0xfbc05a30(0xee4eb200, 0xf124bbb8, 0xf78c9c78, 0xfbc1f9e8, 0x86d, 0xec000d60), at 0xfbc05a2f
        [13] 0xfbe2d43c(0xf0f86bf8, 0xf124bbb8, 0xee530010, 0x1, 0x974, 0xec000e30), at 0xfbe2d43b
        [14] 0xfbc058c0(0xf0f8fa58, 0xb6, 0xf78a4c24, 0xfbc1c5c8, 0xa1, 0xec000ef8), at 0xfbc058bf
        [15] 0xfbc058c0(0xf0f86850, 0xb7, 0xf786f2b8, 0xfbc1f6c0, 0xad, 0xec000fb8), at 0xfbc058bf
        [16] 0xfbc05a30(0xf0f86850, 0xb7, 0xf786ef60, 0xfbc1f9a0, 0xac, 0xec001078), at 0xfbc05a2f
        [17] 0xfbc05a30(0xf0f86850, 0xb7, 0xf786ef60, 0xfbc1f9a0, 0xa7, 0xec001138), at 0xfbc05a2f
        [18] 0xfbc05a30(0xf0f86850, 0xb7, 0xf786ef60, 0xfbc1f9a0, 0x6e, 0xec0011f8), at 0xfbc05a2f
        [19] 0xfbc05a30(0xf0f86850, 0xb7, 0xf786ef60, 0xfbc1f9a0, 0xedd349e0, 0xec0012b8), at 0xfbc05a2f
        [20] 0xfbc05a30(0xedcbf498, 0xb6, 0x0, 0xfbc1f9f0, 0xedcc38a0, 0xec001378), at 0xfbc05a2f
        [21] 0xfbc058c0(0xf06a7f40, 0xb6, 0x0, 0xfbc1f9a0, 0xf06a7f40, 0xec001450), at 0xfbc058bf
        [22] 0xfbc058c0(0xf06a7f40, 0xb7, 0x0, 0xfbc1c5c8, 0xf6773ad8, 0xec001520), at 0xfbc058bf
        [23] 0xfbc05a30(0xf06a7f40, 0x80000000, 0x0, 0xfbc1f9e8, 0x29714, 0xec0015d0), at 0xfbc05a2f
        [24] 0xfbc05a30(0xeda19f20, 0xf582ea50, 0x0, 0xfbc1f710, 0x6, 0xec001658), at 0xfbc05a2f
        [25] 0xfbc05f50(0xeda19f48, 0xb7, 0x0, 0xfbc1fd90, 0xf06b0120, 0xec0016d8), at 0xfbc05f4f
        [26] 0xfbc05a30(0xeda19f48, 0xb6, 0x0, 0xfbc1f9a0, 0x80000000, 0xec001758), at 0xfbc05a2f
        [27] 0xfbc05a30(0xec001d38, 0xfe756d38, 0x0, 0xfbc1f6c0, 0xee076128, 0xec001800), at 0xfbc05a2f
        [28] 0xfbc0021c(0xec001908, 0xec001bb0, 0xa, 0xf61ffdd8, 0xfbc0f700, 0xec001a98), at 0xfbc0021b
        [29] JavaCalls::call_helper(0x1, 0xf61ffdd8, 0xec001a90, 0x4e0400, 0x69d7b0, 0xec0018e8), at 0xfe0dd270
        [30] JavaCalls::call_virtual(0xec001ba8, 0x69d7b4, 0xfe767f88, 0x69d7c0, 0xec001a90, 0x4e0400), at 0xfe378f10
        [31] JavaCalls::call_virtual(0xec001ba8, 0xec001ba4, 0xec001ba0, 0xec001b9c, 0xec001b98, 0x69d7b4), at 0xfe1884b4
        [32] thread_entry(0xf58320b8, 0x4e0400, 0x39800, 0xfe768068, 0xfe767f88, 0xfe767a08), at 0xfe19b648
        [33] JavaThread::thread_main_inner(0x4e0400, 0xee076128, 0x59eaf8, 0xfe72e000, 0x4ddc40, 0x0), at 0xfe5ca358
        [34] java_start(0x4e0400, 0x16, 0x452c, 0xfe72e000, 0xfe6a3b89, 0x29400), at 0xfe53e374

            dcubed Daniel Daugherty
            jjh James Holmlund (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Created:
              Updated:
              Resolved:
              Imported:
              Indexed: