The following nsk/jvmti tests failed with SIGBUS once in nightly (2007.06.02):
  nsk/jvmti/RawMonitorEnter/rawmonenter003
  nsk/jvmti/RawMonitorExit/rawmonexit003
  nsk/jvmti/RawMonitorNotify/rawmnntfy003
  nsk/jvmti/RawMonitorNotifyAll/rawmnntfyall003
  nsk/jvmti/RawMonitorWait/rawmnwait003

The jvmti RawMonitor magic check expect the RawMonitor pointer passed to
the jvmti function from the agent code to be aligned.
This causes VM to crash with SIGBUS.

The following C-program demonstrates the alignement problem in jvmti:
% cat sigbus.c
int main() {
  int magic = (int)(('T' << 24) | ('I' << 16) | ('R' << 8) | 'M');
  char arr[] = "a";
  char str[] = "invalid raw monitor is very bad";
  int *ptr = (int*) str;
  if (ptr[0] != magic) {
     return 0;
  }
  return 99;
}

% cc -xarch=v9 sigbus.c
% a.out
Bus Error


This is the nightly failures analysis:

---------------------------------------------------------------------------
I think you are right.
Most likely a C string can be not aligned to the 32-bit word boundary.
Theb the is_valid() code is going to crash at the comparison instruction:
   _magic == JVMTI_RM_MAGIC

class JvmtiRawMonitor : public ObjectMonitor {
private:
  int _magic;
  char * _name;
  // JVMTI_RM_MAGIC is set in contructor and unset in destructor.
  enum { JVMTI_RM_MAGIC = (int)(('T' << 24) | ('I' << 16) | ('R' << 8) | 'M') };

public:
  ...
  bool is_valid() { return _magic == JVMTI_RM_MAGIC; }
};

If the above is true then it is a bug in the is_valid() code
where bytes can be compared instead of int's.
I'm puzzled why we never saw this problem before.

Sure, you can take this item if you want. :)

Thanks,
Serguei

Daniel D. Daugherty wrote:

> I suspect memory alignment is the key. A char array doesn't have
> alignment restrictions. Depending on where the char array aligns
> I suspect we'll get SIGBUS.
>
> I suspect if the test is whacked to cycle through the first 8
> bytes in a loop, then it will always blow up at some point, i.e.,
>
> &bad_bug[0] .. &bad_bug[7]
>
> Evil, but interesting...
> Dan
---------------------------------------------------------------------------

First, all 5 tests got SIGBUS when they pass bad monitor pointers like this:
  static char bad_buf[] = "this is a bad raw monitor";
  static jrawMonitorID bad_monitor = (jrawMonitorID)bad_buf;

and expecting to get the JVMTI_ERROR_INVALID_MONITOR error code
from the RawMonitor* functions:
    err = (*jvmti)->RawMonitorEnter(jvmti, bad_monitor);
    if (err != JVMTI_ERROR_INVALID_MONITOR) {
        printf("Error expected: JVMTI_ERROR_INVALID_MONITOR,\n");
        printf("\tactual: %s (%d)\n", TranslateError(err), err);
        result = FAILED;
    }

hs_err logs show that SIGBUS really happen in the up-level jvmti jvmtiEnter.cpp
functions which should catch the condition "!rmonitor->is_valid()":

jvmti_RawMonitorEnter(jvmtiEnv* env,
            jrawMonitorID monitor) {
  ...

    JvmtiRawMonitor *rmonitor = (JvmtiRawMonitor *)monitor;
  if (rmonitor == NULL) {
      return JVMTI_ERROR_INVALID_MONITOR;
  }
  if (!rmonitor->is_valid()) {
      return JVMTI_ERROR_INVALID_MONITOR;
  }

The test never failed before, but only on 2007.06.02
and on several 64-bit plaforms at once:

nsk/jvmti/RawMonitorExit/rawmonexit003 Servicability New Solaris-AMD64 2007-06-02
nsk/jvmti/RawMonitorExit/rawmonexit003 Servicability New 64BITWIN-AMD64 2007-06-02
nsk/jvmti/RawMonitorExit/rawmonexit003 Servicability New Solaris-SparcV9 2007-06-02

Thanks,
Serguei


Daniel D. Daugherty wrote:

> Greetings,
>
> This nightly was a %$#@ train wreck. It took 3+ hours to analyze this
> mess. There are couple of possible real issues, but most of this is
> just infrastructure noise.
>
> New:

...

> Serguei, please see the RawMonitor failures on 2007.06.02. This
> nightly was a train wreck so these failures may not be real.
> I didn't find anything obvious, but...

> New nsk.jvmti failures (from 2007.06.02)
> * nsk/jvmti/RawMonitorEnter/rawmonenter003
> * nsk/jvmti/RawMonitorExit/rawmonexit003
> * nsk/jvmti/RawMonitorNotify/rawmnntfy003
> * nsk/jvmti/RawMonitorNotifyAll/rawmnntfyall003
> * nsk/jvmti/RawMonitorWait/rawmnwait003
> These tests failed due to SIGBUS on Solaris SPARC-64 Server VM
> machine nyny). Other tests passed before, after and interleaved
> with these failing tests. I didn't find anything suspicious in
> /var/adm/messages during the time the tests were run.
>
> Note: the subsuite page identifies the config as Solaris SPARC
> Server VM, but the hs_err_pid file correctly shows the config
> as Solaris SPARC-64 Server VM.
>
...
> URL: http://gtee.sfbay/gtee/results/MUSTANG/NIGHTLY/VM-MAIN/2007-06-02/Serv_Baseline/
> VM Build: 20070531193122.dcubed.service_hs_b14_merge.2 (same VM as previous run)
-----------------------------------------------------------------------------------

This is one of the hs_err logs below:

ss45998@nyny hs_err --jvm=/home/ss45998/libjvm.so hs_err_pid28405.log | c++filt
;; Using jvm: /home/ss45998/libjvm.so
#
# An unexpected error has been detected by Java Runtime Environment:
#
# SIGBUS (0xa) at pc=0xffffffff7bdb802c, pid=28405, tid=4
#
# Java VM: Java HotSpot(TM) 64-Bit Server VM (20070531193122.dcubed.service_hs_b14_merge.2-fastdebug compiled mode solaris-sparc)
# Problematic frame:
# V [libjvm.so+0x9b802c]
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

--------------- T H R E A D ---------------

Current thread (0x000000010013ec00): JavaThread "main" [_thread_in_native, id=4, stack(0xffffffff7a202000,0xffffffff7a301c50)]

siginfo:si_signo=SIGBUS: si_errno=0, si_code=1 (BUS_ADRALN), si_addr=0xffffffff79d15dcd;;

Registers:
 O0=0x0000000000000001 O1=0xffffffff7b8a1720 O2=0x0000000000000001 O3=0x0000000000000000
 O4=0x0000000000000000 O5=0xffffffff7b8a1588 O6=0xffffffff7a300691 O7=0xffffffff7bdb8008
 G1=0x007ff0000042efad G2=0x007fffffffbd180e G3=0x00000ffffffff7a3 G4=0x00000000000acb28
 G5=0x00000000000ac800 G6=0x0000000000000000 G7=0xffffffff7a301c50 Y=0x0000000000000000
 PC=0xffffffff7bdb802c nPC=0xffffffff7bdb803c


Top of Stack: (sp=0xffffffff7a300e90)
0xffffffff7a300e90: ffffffff7d474e20 ffffffff7d41eb31
0xffffffff7a300ea0: 0000000000075e51 0000000000075c00
0xffffffff7a300eb0: 000000010013ec00 ffffffff7a301c50
0xffffffff7a300ec0: 0000000000000d68 ffffffff7d455808
0xffffffff7a300ed0: 000000010011a3e8 ffffffff79d15d25
0xffffffff7a300ee0: 0000000000000064 00000000000001ad
0xffffffff7a300ef0: ffffffff7d474e20 ffffffff7d3a8ce0
0xffffffff7a300f00: ffffffff7a300741 ffffffff79c0d480
0xffffffff7a300f10: 0000000000000000 000000010013ec00
0xffffffff7a300f20: 000000010013ec00 0000000000000001
0xffffffff7a300f30: 0000000000000000 0000000010000000
0xffffffff7a300f40: 0000000000000000 0000000000000004
0xffffffff7a300f50: ffffffff7d41f220 0000000000000003
0xffffffff7a300f60: 0000000000000000 ffffffff7a301450
0xffffffff7a300f70: 000000010013f918 0000000000000000
0xffffffff7a300f80: 000000010011a3f0 ffffffff79d15d18

Instructions: (pc=0xffffffff7bdb802c)
0xffffffff7bdb801c: b0 10 20 73 81 c7 e0 08 81 e8 20 00 2a c6 40 05
0xffffffff7bdb802c: e2 06 60 a8 b0 10 20 32 81 c7 e0 08 81 e8 20 00
;; Using jvm: /home/ss45998/libjvm.so
#
# An unexpected error has been detected by Java Runtime Environment:
#
# SIGBUS (0xa) at pc=0xffffffff7bdb802c, pid=28405, tid=4
#
# Java VM: Java HotSpot(TM) 64-Bit Server VM (20070531193122.dcubed.service_hs_b14_merge.2-fastdebug compiled mode solaris-sparc)
# Problematic frame:
# V [libjvm.so+0x9b802c]
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

--------------- T H R E A D ---------------

Current thread (0x000000010013ec00): JavaThread "main" [_thread_in_native, id=4, stack(0xffffffff7a202000,0xffffffff7a301c50)]

siginfo:si_signo=SIGBUS: si_errno=0, si_code=1 (BUS_ADRALN), si_addr=0xffffffff79d15dcd;;

Registers:
 O0=0x0000000000000001 O1=0xffffffff7b8a1720 O2=0x0000000000000001 O3=0x0000000000000000
 O4=0x0000000000000000 O5=0xffffffff7b8a1588 O6=0xffffffff7a300691 O7=0xffffffff7bdb8008
 G1=0x007ff0000042efad G2=0x007fffffffbd180e G3=0x00000ffffffff7a3 G4=0x00000000000acb28
 G5=0x00000000000ac800 G6=0x0000000000000000 G7=0xffffffff7a301c50 Y=0x0000000000000000
 PC=0xffffffff7bdb802c nPC=0xffffffff7bdb803c


Top of Stack: (sp=0xffffffff7a300e90)
0xffffffff7a300e90: ffffffff7d474e20 ffffffff7d41eb31
0xffffffff7a300ea0: 0000000000075e51 0000000000075c00
0xffffffff7a300eb0: 000000010013ec00 ffffffff7a301c50
0xffffffff7a300ec0: 0000000000000d68 ffffffff7d455808
0xffffffff7a300ed0: 000000010011a3e8 ffffffff79d15d25
0xffffffff7a300ee0: 0000000000000064 00000000000001ad
0xffffffff7a300ef0: ffffffff7d474e20 ffffffff7d3a8ce0
0xffffffff7a300f00: ffffffff7a300741 ffffffff79c0d480
0xffffffff7a300f10: 0000000000000000 000000010013ec00
0xffffffff7a300f20: 000000010013ec00 0000000000000001
0xffffffff7a300f30: 0000000000000000 0000000010000000
0xffffffff7a300f40: 0000000000000000 0000000000000004
0xffffffff7a300f50: ffffffff7d41f220 0000000000000003
0xffffffff7a300f60: 0000000000000000 ffffffff7a301450
0xffffffff7a300f70: 000000010013f918 0000000000000000
0xffffffff7a300f80: 000000010011a3f0 ffffffff79d15d18

Instructions: (pc=0xffffffff7bdb802c)
0xffffffff7bdb801c: b0 10 20 73 81 c7 e0 08 81 e8 20 00 2a c6 40 05
0xffffffff7bdb802c: e2 06 60 a8 b0 10 20 32 81 c7 e0 08 81 e8 20 00
;;
Stack: [0xffffffff7a202000,0xffffffff7a301c50], sp=0xffffffff7a300e90, free space=1019k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x9b802c];; jvmti_RawMonitorWait+0x144
C [librawmnwait003.so+0xd488];; Inject+0x1e0
J nsk.jvmti.RawMonitorWait.rawmnwait003.check()I
J nsk.jvmti.RawMonitorWait.rawmnwait003.run([Ljava/lang/String;Ljava/io/PrintStream;)I
v ~BufferBlob::Interpreter
v ~BufferBlob::StubRoutines (1)
V [libjvm.so+0x6fef58];; void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x9a8
V [libjvm.so+0x6fe55c];; void JavaCalls::call(JavaValue*,methodHandle,JavaCallArguments*,Thread*)+0x74
V [libjvm.so+0x749604];; void jni_invoke_static(JNIEnv_*,JavaValue*,_jobject*,JNICallType,_jmethodID*,JNI_ArgumentPusher*,Thread*)+0xa94
V [libjvm.so+0x7a2d64];; jni_CallStaticVoidMethod+0xaa4
C [java+0x3ed0];; JavaMain+0x1928

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J nsk.jvmti.RawMonitorWait.rawmnwait003.check()I
J nsk.jvmti.RawMonitorWait.rawmnwait003.run([Ljava/lang/String;Ljava/io/PrintStream;)I
v ~BufferBlob::Interpreter
v ~BufferBlob::StubRoutines (1)

--------------- P R O C E S S ---------------

Java Threads: ( => current thread )
  0x0000000100271000 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=11, stack(0xffffffff6a602000,0xffffffff6a701c50)]
  0x000000010026dc00 JavaThread "CompilerThread1" daemon [_thread_blocked, id=10, stack(0xffffffff6a802000,0xffffffff6a901c50)]
  0x000000010026b400 JavaThread "CompilerThread0" daemon [_thread_blocked, id=9, stack(0xffffffff6aa02000,0xffffffff6ab01c50)]
  0x0000000100269800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=8, stack(0xffffffff6ac02000,0xffffffff6ad01c50)]
  0x0000000100243800 JavaThread "Finalizer" daemon [_thread_blocked, id=7, stack(0xffffffff6ae02000,0xffffffff6af01c50)]
  0x000000010023c800 JavaThread "Reference Handler" daemon [_thread_blocked, id=6, stack(0xffffffff6b002000,0xffffffff6b101c50)]
=>0x000000010013ec00 JavaThread "main" [_thread_in_native, id=4, stack(0xffffffff7a202000,0xffffffff7a301c50)]

Other Threads:
  0x0000000100235800 VMThread [stack: 0xffffffff6b202000,0xffffffff6b301c50] [id=5]
  0x0000000100284000 WatcherThread [stack: 0xffffffff6a402000,0xffffffff6a501c50] [id=12]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

...