-
Bug
-
Resolution: Won't Fix
-
P4
-
None
-
8u131
-
x86_64
-
linux
FULL PRODUCT VERSION :
-bash-4.1$ java -version
java version "1.8.0_31"
Java(TM) SE Runtime Environment (build 1.8.0_31-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode)
FULL OS VERSION :
OS:Scientific Linux release 6.6 (Carbon)
uname:Linux 2.6.32-696.6.3.el6.x86_64 #1 SMP Tue Jul 11 14:22:34 CDT 2017 x86_64
libc:glibc 2.12 NPTL 2.12
rlimit: STACK 10240k, CORE 0k, NPROC 8192, NOFILE
EXTRA RELEVANT SYSTEM CONFIGURATION :
This is pretty standard Tomcat web application configured for 20G of memory running on server with 128G of RAM.
A DESCRIPTION OF THE PROBLEM :
Problem occurs approximately once / week on each of our production servers. It generates hs_err_pid???.log file with detailed information.
Typical outcome is like below:
# Out of Memory Error (os_linux.cpp:2671), pid=92184, tid=139837919295232
#
# JRE version: Java(TM) SE Runtime Environment (8.0_31-b13) (build 1.8.0_31-b13)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.31-b07 mixed mode linux-amd64 compressed oops)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
--------------- T H R E A D ---------------
Current thread (0x00007f2cd0011000): JavaThread "HandshakeCompletedNotify-Thread" daemon [_thread_new, id=105289, stack(0x00007f2e8d797000,0x00007f2e8d818000)]
Stack: [0x00007f2e8d797000,0x00007f2e8d818000], sp=0x00007f2e8d816850, free space=510k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0xa79e0a] VMError::report_and_die()+0x2ca
V [libjvm.so+0x4e57bb] report_vm_out_of_memory(char const*, int, unsigned long, VMErrorType, char const*)+0x8b
V [libjvm.so+0x8e5a23] os::Linux::commit_memory_impl(char*, unsigned long, bool)+0x103
V [libjvm.so+0x8e5aec] os::pd_commit_memory(char*, unsigned long, bool)+0xc
V [libjvm.so+0x8dd9aa] os::commit_memory(char*, unsigned long, bool)+0x2a
V [libjvm.so+0x8e3f0f] os::pd_create_stack_guard_pages(char*, unsigned long)+0x7f
V [libjvm.so+0xa22c5e] JavaThread::create_stack_guard_pages()+0x5e
V [libjvm.so+0xa2aa34] JavaThread::run()+0x34
V [libjvm.so+0x8e8158] java_start(Thread*)+0x108
C [libpthread.so.0+0x7aa1]
. . . skipped . . .
--------------- S Y S T E M ---------------
OS:Scientific Linux release 6.6 (Carbon)
uname:Linux 2.6.32-696.6.3.el6.x86_64 #1 SMP Tue Jul 11 14:22:34 CDT 2017 x86_64
libc:glibc 2.12 NPTL 2.12
rlimit: STACK 10240k, CORE 0k, NPROC 8192, NOFILE 65535, AS infinity
load average:6.18 2.54 1.57
/proc/meminfo:
MemTotal: 132052000 kB
MemFree: 7943300 kB
Buffers: 37753872 kB
Cached: 36042640 kB
SwapCached: 98408 kB
Active: 75104352 kB
Inactive: 39757380 kB
Active(anon): 38212544 kB
Inactive(anon): 2856684 kB
Active(file): 36891808 kB
Inactive(file): 36900696 kB
Unevictable: 4194304 kB
Mlocked: 4194304 kB
SwapTotal: 10485756 kB
SwapFree: 8860776 kB
Dirty: 16140 kB
Writeback: 0 kB
THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: Did not try
THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: Did not try
REPRODUCIBILITY :
This bug can be reproduced often.
-bash-4.1$ java -version
java version "1.8.0_31"
Java(TM) SE Runtime Environment (build 1.8.0_31-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode)
FULL OS VERSION :
OS:Scientific Linux release 6.6 (Carbon)
uname:Linux 2.6.32-696.6.3.el6.x86_64 #1 SMP Tue Jul 11 14:22:34 CDT 2017 x86_64
libc:glibc 2.12 NPTL 2.12
rlimit: STACK 10240k, CORE 0k, NPROC 8192, NOFILE
EXTRA RELEVANT SYSTEM CONFIGURATION :
This is pretty standard Tomcat web application configured for 20G of memory running on server with 128G of RAM.
A DESCRIPTION OF THE PROBLEM :
Problem occurs approximately once / week on each of our production servers. It generates hs_err_pid???.log file with detailed information.
Typical outcome is like below:
# Out of Memory Error (os_linux.cpp:2671), pid=92184, tid=139837919295232
#
# JRE version: Java(TM) SE Runtime Environment (8.0_31-b13) (build 1.8.0_31-b13)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.31-b07 mixed mode linux-amd64 compressed oops)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
--------------- T H R E A D ---------------
Current thread (0x00007f2cd0011000): JavaThread "HandshakeCompletedNotify-Thread" daemon [_thread_new, id=105289, stack(0x00007f2e8d797000,0x00007f2e8d818000)]
Stack: [0x00007f2e8d797000,0x00007f2e8d818000], sp=0x00007f2e8d816850, free space=510k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0xa79e0a] VMError::report_and_die()+0x2ca
V [libjvm.so+0x4e57bb] report_vm_out_of_memory(char const*, int, unsigned long, VMErrorType, char const*)+0x8b
V [libjvm.so+0x8e5a23] os::Linux::commit_memory_impl(char*, unsigned long, bool)+0x103
V [libjvm.so+0x8e5aec] os::pd_commit_memory(char*, unsigned long, bool)+0xc
V [libjvm.so+0x8dd9aa] os::commit_memory(char*, unsigned long, bool)+0x2a
V [libjvm.so+0x8e3f0f] os::pd_create_stack_guard_pages(char*, unsigned long)+0x7f
V [libjvm.so+0xa22c5e] JavaThread::create_stack_guard_pages()+0x5e
V [libjvm.so+0xa2aa34] JavaThread::run()+0x34
V [libjvm.so+0x8e8158] java_start(Thread*)+0x108
C [libpthread.so.0+0x7aa1]
. . . skipped . . .
--------------- S Y S T E M ---------------
OS:Scientific Linux release 6.6 (Carbon)
uname:Linux 2.6.32-696.6.3.el6.x86_64 #1 SMP Tue Jul 11 14:22:34 CDT 2017 x86_64
libc:glibc 2.12 NPTL 2.12
rlimit: STACK 10240k, CORE 0k, NPROC 8192, NOFILE 65535, AS infinity
load average:6.18 2.54 1.57
/proc/meminfo:
MemTotal: 132052000 kB
MemFree: 7943300 kB
Buffers: 37753872 kB
Cached: 36042640 kB
SwapCached: 98408 kB
Active: 75104352 kB
Inactive: 39757380 kB
Active(anon): 38212544 kB
Inactive(anon): 2856684 kB
Active(file): 36891808 kB
Inactive(file): 36900696 kB
Unevictable: 4194304 kB
Mlocked: 4194304 kB
SwapTotal: 10485756 kB
SwapFree: 8860776 kB
Dirty: 16140 kB
Writeback: 0 kB
THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: Did not try
THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: Did not try
REPRODUCIBILITY :
This bug can be reproduced often.