A CU found out JVM jumps to 0 address and terminates abnormally.

CONFIGURATION :
JDK : JDK5u14 server
OS : Solaris 10

PHENOMENA:
The following is the stack trace when the abort occurs.

#0 0xfe33092c in JVM_handle_solaris_signal
#1 <signal handler called>
#2 0x00000000 in ??
#3 0xf880c1d8 in %%% sun.misc.GC#maxObjectInspectionAge()J
#4 0xf8805904 in %%% sun.misc.GC$Daemon#run()V
#5 0xf8800220 in === BufferBlob[StubRoutines (1)] @ 0xf8800108 ===
#6 0xfe1a2e74 in void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)
#7 0xfe37f424 in void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*)
#8 0xfe39f9f0 in void thread_entry(JavaThread*,Thread*)
#9 0xfe39b2e4 in void JavaThread::run()
#10 0xfe7a6968 in void*_start(void*)
#11 0xfeac8c70 in _lwp_start (0x0, 0x0, 0x0, 0x0, 0x0, 0x0) from root/usr/lib/libc.so.1

(%%% means interpreter is running.)

JVM jumps to 0 address in "call" instruction at the following line# 0xf880c224.
This is the code corresponding to
  ...
method entry point (kind = native) [0xf880c070, 0xf880c4c0] 1104 bytes not safepoint safe
  ...
in interpreter codelet

0xf880c1d0: call 0xfe33fcc8 <void InterpreterRuntime::prepare_native_call(JavaThread*,methodOopDesc*)>
 0xf880c1d4: mov %g2, %o0
 0xf880c1d8: mov %l7, %g2
 0xf880c1dc: clr [ %g2 + 0x120 ]
 0xf880c1e0: clr [ %g2 + 0x124 ]
 0xf880c1e4: clr [ %g2 + 0x12c ]
 0xf880c1e8: ld [ %g2 + 4 ], %g5
 0xf880c1ec: tst %g5
 0xf880c1f0: be %icc, 0xf880c200
 0xf880c1f4: nop
 0xf880c1f8: call 0xf8800140
 0xf880c1fc: nop
 0xf880c200: ld [ %l2 + 0x4c ], %g3
 0xf880c204: st %l2, [ %sp + 8 ]
 0xf880c208: mov %l3, %o1
 0xf880c20c: add %fp, -24, %o2
 0xf880c210: sub %sp, %fp, %o3
 0xf880c214: save %sp, %o3, %sp
 0xf880c218: ld [ %fp + 8 ], %l2
 0xf880c21c: mov %i1, %l3
 0xf880c220: mov %i2, %l6
 0xf880c224: call %g3 // NOTE !!

signature_handler of methodOop should have been set to %3 at the line 0xf880c224,
but actually 0 is set.
signature_handler should have been set in InterpreterRuntime::prepare_native_call at the line 0xf880c1d0
but have not.

The code of SignatureHandlerLibrary::add called from InterpreterRuntime::prepare_native_call is as follows.

    if (method->signature_handler() == NULL) {
    ...
      int handler_index = -1;
    ...
      if (UseFastSignatureHandlers && method->size_of_parameters() <= Fingerprinter::max_size_of_parameters) {
    ...
           MutexLocker mu(SignatureHandlerLibrary_lock);
    ...
           uint64_t fingerprint = Fingerprinter(method).fingerprint();
           handler_index = _fingerprints->find(fingerprint);
    ...
           address handler = set_handler(buffer);
    ...
           _fingerprints->append(fingerprint);
           _handlers->append(handler); // *1
    ...
      }
    ...
      method->set_signature_handler(_handlers->at(handler_index)); // *2
    }

The program is locked until the line *1 by MutexLocker mu, but not at the line *2.
When a thread is accessing to the line *2, another thread executes the line *1
and GrowableArray::grow is called, GrowableArray::_data might be overwritten.
The thread which is executing _handlers->at(handler_index) at *2
possibly refers to different data.

The following code is extarcted from GrowableArray::grow,

  for (int i = 0; i < _len; i++) newData[i] = _data[i];
  ...
  _data = newData;

The above program set new data to _data variable after copying
old _data. Some memory ordering code seems needed here.

When the crash occurs, the status at *2 seems,
  new _data can be seen but copied elements can not be seen yet.
(This is similar to 6704010.)


Please see "comment" section also.