after upgrading from Java 5.0 to 6_u5 customer started seeing crashes once or twice a day making it unstable enough in their eyes to revert back to 5.0
- OS: RedHat Enterprise Workstation, kernel 2.6.18-8.el5
- java version "1.6.0_05"
Java(TM) SE Runtime Environment (build 1.6.0_05-b13)
Java HotSpot(TM) Server VM (build 10.0-b19, mixed mod
It failed in:
frame frame::sender(RegisterMap* map) const {
// Default is we done have to follow them. The sender_for_xxx will
// update it accordingly
map->set_include_argument_oops(false);
if (is_entry_frame()) return sender_for_entry_frame(map);
if (is_interpreted_frame()) return sender_for_interpreter_frame(map);
assert(_cb == CodeCache::find_blob(pc()),"Must be the same");
if (_cb != NULL) {
return sender_for_compiled_frame(map);
}
// Must be native-compiled frame, i.e. the marshaling code for native
// methods that exists in the core system.
return frame(sender_sp(), link(), sender_pc());
}
The code:
0x6318e67 <_ZNK5frame6senderEP11RegisterMap+135>: mov 0x10(%esi),%eax // esi is map -> map offset 16 should be some register value
0x6318e6a <_ZNK5frame6senderEP11RegisterMap+138>: sub $0xc,%esp
0x6318e6d <_ZNK5frame6senderEP11RegisterMap+141>: mov (%eax),%esi // the value is NULL
0x6318e6f <_ZNK5frame6senderEP11RegisterMap+143>: lea 0x8(%eax),%ecx
The hs_erro log contained this
#
# An unexpected error has been detected by Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x06318e6d, pid=31401, tid=2133318544
#
# Java VM: Java HotSpot(TM) Server VM (10.0-b19 mixed mode linux-x86)
# Problematic frame:
# V [libjvm.so+0x318e6d]
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#
--------------- T H R E A D ---------------
Current thread (0x7d7f9400): JavaThread "ConnThreadDef:98.199.73.10:49570:28637465" [_thread_in_vm, id=10803, stack(0x7f25d000,0x7f27e000)]
siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x00000000
Registers:
EAX=0x00000000, EBX=0x7f27c958, ECX=0x087519e0, EDX=0x8050e800
ESP=0x7f27c8cc, EBP=0x7f27c8f0, ESI=0x80304724, EDI=0x7f27c978
EIP=0x06318e6d, CR2=0x00000000, EFLAGS=0x00010216
Top of Stack: (sp=0x7f27c8cc)
0x7f27c8cc: 7f27ca98 80304634 7f27c918 00000000
0x7f27c8dc: 7f27c8f8 00000000 7f27c958 00000000
0x7f27c8ec: 7f27c908 7f27c940 0631635d 7f27c958
0x7f27c8fc: 80304724 7f27c978 00000000 00000000
0x7f27c90c: 7f009f50 7f27ca98 8050e800 00000000
0x7f27c91c: 00000000 00000000 b04f7110 852526c8
0x7f27c92c: 7f009f48 851e0950 7f27c978 00000000
0x7f27c93c: 7f27c958 7f27ca50 065e5d71 7f27c958
Instructions: (pc=0x06318e6d)
0x06318e5d: f4 89 d8 5b 5e 5f 5d c2 04 00 8b 46 10 83 ec 0c
0x06318e6d: 8b 30 8d 48 08 8b 40 04 89 73 10 89 0b 89 4b 14
Stack: [0x7f25d000,0x7f27e000], sp=0x7f27c8cc, free space=126k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x318e6d]
V [libjvm.so+0x31635d]
V [libjvm.so+0x5e5d71]
V [libjvm.so+0x5e5e72]
V [libjvm.so+0x1c04f7]
V [libjvm.so+0x1bfe52]
V [libjvm.so+0x1bf4df]
V [libjvm.so+0x592025]
V [libjvm.so+0x365a1e]
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
J com.pogo.serv.thrserver.ConnThreadDef$RunThread.run()V
v ~BufferBlob::StubRoutines (1)
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x80505800 JavaThread "ConnThreadDef:12.34.184.146:1542:22472734" [_thread_in_native, id=10813, stack(0x7bdf0000,0x7be11000)]
0x80307800 JavaThread "ConnThreadDef:216.152.189.225:4441:4799109" [_thread_in_native, id=10812, stack(0x7ac4c000,0x7ac6d000)]
=>0x7d7f9400 JavaThread "ConnThreadDef:98.199.73.10:49570:28637465" [_thread_in_vm, id=10803, stack(0x7f25d000,0x7f27e000)]
0x8050e800 JavaThread "ConnThreadDef:98.199.73.10:49568:6373458" [_thread_in_native, id=10801, stack(0x7efea000,0x7f00b000)]
I've attached a perl script called hs_err that helps us decode the hs_err_pid* files. From decoding the information attached, the call stack looks like below. This looks like another bug that we have.
V [libjvm.so+0x318e6d]
;; _ZNK5frame6senderEP11RegisterMap+0x8d
V [libjvm.so+0x31635d]
;; _ZNK5frame11real_senderEP11RegisterMap+0x1d
V [libjvm.so+0x5e5d71]
;; _ZNK6vframe6senderEv+0x81
V [libjvm.so+0x5e5e72]
;; _ZNK6vframe11java_senderEv+0x32
V [libjvm.so+0x1c04f7]
;; _Z27get_or_compute_monitor_infoP10JavaThread+0x127
V [libjvm.so+0x1bfe52]
;; _Z11revoke_biasP7oopDescbbP10JavaThread+0x1a2
V [libjvm.so+0x1bf4df]
;; _ZN13BiasedLocking17revoke_and_rebiasE6HandlebP6Thread+0x19f
V [libjvm.so+0x592025]
;; _ZN18ObjectSynchronizer10fast_enterE6HandleP9BasicLockbP6Thread+0x35
V [libjvm.so+0x365a1e]
;; _ZN18InterpreterRuntime12monitorenterEP10JavaThreadP15BasicObjectLock+0x6e
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
- OS: RedHat Enterprise Workstation, kernel 2.6.18-8.el5
- java version "1.6.0_05"
Java(TM) SE Runtime Environment (build 1.6.0_05-b13)
Java HotSpot(TM) Server VM (build 10.0-b19, mixed mod
It failed in:
frame frame::sender(RegisterMap* map) const {
// Default is we done have to follow them. The sender_for_xxx will
// update it accordingly
map->set_include_argument_oops(false);
if (is_entry_frame()) return sender_for_entry_frame(map);
if (is_interpreted_frame()) return sender_for_interpreter_frame(map);
assert(_cb == CodeCache::find_blob(pc()),"Must be the same");
if (_cb != NULL) {
return sender_for_compiled_frame(map);
}
// Must be native-compiled frame, i.e. the marshaling code for native
// methods that exists in the core system.
return frame(sender_sp(), link(), sender_pc());
}
The code:
0x6318e67 <_ZNK5frame6senderEP11RegisterMap+135>: mov 0x10(%esi),%eax // esi is map -> map offset 16 should be some register value
0x6318e6a <_ZNK5frame6senderEP11RegisterMap+138>: sub $0xc,%esp
0x6318e6d <_ZNK5frame6senderEP11RegisterMap+141>: mov (%eax),%esi // the value is NULL
0x6318e6f <_ZNK5frame6senderEP11RegisterMap+143>: lea 0x8(%eax),%ecx
The hs_erro log contained this
#
# An unexpected error has been detected by Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x06318e6d, pid=31401, tid=2133318544
#
# Java VM: Java HotSpot(TM) Server VM (10.0-b19 mixed mode linux-x86)
# Problematic frame:
# V [libjvm.so+0x318e6d]
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#
--------------- T H R E A D ---------------
Current thread (0x7d7f9400): JavaThread "ConnThreadDef:98.199.73.10:49570:28637465" [_thread_in_vm, id=10803, stack(0x7f25d000,0x7f27e000)]
siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x00000000
Registers:
EAX=0x00000000, EBX=0x7f27c958, ECX=0x087519e0, EDX=0x8050e800
ESP=0x7f27c8cc, EBP=0x7f27c8f0, ESI=0x80304724, EDI=0x7f27c978
EIP=0x06318e6d, CR2=0x00000000, EFLAGS=0x00010216
Top of Stack: (sp=0x7f27c8cc)
0x7f27c8cc: 7f27ca98 80304634 7f27c918 00000000
0x7f27c8dc: 7f27c8f8 00000000 7f27c958 00000000
0x7f27c8ec: 7f27c908 7f27c940 0631635d 7f27c958
0x7f27c8fc: 80304724 7f27c978 00000000 00000000
0x7f27c90c: 7f009f50 7f27ca98 8050e800 00000000
0x7f27c91c: 00000000 00000000 b04f7110 852526c8
0x7f27c92c: 7f009f48 851e0950 7f27c978 00000000
0x7f27c93c: 7f27c958 7f27ca50 065e5d71 7f27c958
Instructions: (pc=0x06318e6d)
0x06318e5d: f4 89 d8 5b 5e 5f 5d c2 04 00 8b 46 10 83 ec 0c
0x06318e6d: 8b 30 8d 48 08 8b 40 04 89 73 10 89 0b 89 4b 14
Stack: [0x7f25d000,0x7f27e000], sp=0x7f27c8cc, free space=126k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x318e6d]
V [libjvm.so+0x31635d]
V [libjvm.so+0x5e5d71]
V [libjvm.so+0x5e5e72]
V [libjvm.so+0x1c04f7]
V [libjvm.so+0x1bfe52]
V [libjvm.so+0x1bf4df]
V [libjvm.so+0x592025]
V [libjvm.so+0x365a1e]
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
J com.pogo.serv.thrserver.ConnThreadDef$RunThread.run()V
v ~BufferBlob::StubRoutines (1)
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x80505800 JavaThread "ConnThreadDef:12.34.184.146:1542:22472734" [_thread_in_native, id=10813, stack(0x7bdf0000,0x7be11000)]
0x80307800 JavaThread "ConnThreadDef:216.152.189.225:4441:4799109" [_thread_in_native, id=10812, stack(0x7ac4c000,0x7ac6d000)]
=>0x7d7f9400 JavaThread "ConnThreadDef:98.199.73.10:49570:28637465" [_thread_in_vm, id=10803, stack(0x7f25d000,0x7f27e000)]
0x8050e800 JavaThread "ConnThreadDef:98.199.73.10:49568:6373458" [_thread_in_native, id=10801, stack(0x7efea000,0x7f00b000)]
I've attached a perl script called hs_err that helps us decode the hs_err_pid* files. From decoding the information attached, the call stack looks like below. This looks like another bug that we have.
V [libjvm.so+0x318e6d]
;; _ZNK5frame6senderEP11RegisterMap+0x8d
V [libjvm.so+0x31635d]
;; _ZNK5frame11real_senderEP11RegisterMap+0x1d
V [libjvm.so+0x5e5d71]
;; _ZNK6vframe6senderEv+0x81
V [libjvm.so+0x5e5e72]
;; _ZNK6vframe11java_senderEv+0x32
V [libjvm.so+0x1c04f7]
;; _Z27get_or_compute_monitor_infoP10JavaThread+0x127
V [libjvm.so+0x1bfe52]
;; _Z11revoke_biasP7oopDescbbP10JavaThread+0x1a2
V [libjvm.so+0x1bf4df]
;; _ZN13BiasedLocking17revoke_and_rebiasE6HandlebP6Thread+0x19f
V [libjvm.so+0x592025]
;; _ZN18ObjectSynchronizer10fast_enterE6HandleP9BasicLockbP6Thread+0x35
V [libjvm.so+0x365a1e]
;; _ZN18InterpreterRuntime12monitorenterEP10JavaThreadP15BasicObjectLock+0x6e
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
- duplicates
-
JDK-6676175 BigApps crash JVM Client VM in BiasedLocking::revoke_and_rebias() walking wrong stack
-
- Closed
-
- relates to
-
JDK-6735499 crash in 1.6.0_04 in libjvm.so:revoke_bias()
-
- Closed
-