Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8279657

jvm suddely went down in production environment

XMLWordPrintable

    • gc
    • x86_64
    • linux

      ADDITIONAL SYSTEM INFORMATION :
      linux/jdk1.8.0_311

      A DESCRIPTION OF THE PROBLEM :
      jvm suddely went down in production environment

      REGRESSION : Last worked in version 8u311

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      suddenly jvm went down

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      jvm should not crash and work as expected
      ACTUAL -
      jvm went down

      ---------- BEGIN SOURCE ----------
      hs_err_pid1921152.log
      ========================================
      OS:
      ----------------------------------------
      Version: Red Hat Enterprise Linux release 8.4 (Ootpa)
      ARCH: X86_64
      CPUs (cpu x cpu cores x hyperthreading): 8
      Memory: 94391M
      Memory Free: 25835M (27%)
      Memory Available: 54458M (58%)
      Swap: 2048M
      Swap Free: 2048M (100%)
      ========================================
      JVM:
      ----------------------------------------
      Version: 1.8.0_162-b12
      Vendor: ORACLE
      Application: JBOSS_EAP7
      JVM User: jboss
      VM State: at safepoint (normal execution)
      Crash Date: Fri Nov 26 07:32:17 2021
      Run Time: 6d 7h 10m 52s
      Garbage Collector(s): PARALLEL_SCAVENGE, PARALLEL_OLD
      Heap Max: 6144M
      Heap Allocation: 6130M (100% Heap Max)
      Heap Used: 763M (12% Heap Allocation)
      Compressed oops mode: UNKNOWN
      Metaspace Max: 512M
      Metaspace Allocation: 254M (50% Metaspace Max)
      Metaspace Used: 230M (91% Metaspace Allocation)
      Thread Stack Size: 1024K
      Thread Stack Memory: 627M
      Code Cache Max: 420M
      JVM Memory Max: >7703M (8% Available Memory)
      ========================================
      Threads:
      ----------------------------------------
      Current thread: VMThread [stack: 0x000014909f1f3000,0x000014909f2f3000] [id=1921168]
      VM operation: ParallelGCSystemGC, mode: safepoint, requested by thread 0x0000149034c5c800
      # Java threads: 627
      ========================================
      Error(s):
      ----------------------------------------
      # SIGSEGV (0xb) at pc=0x00001490d19a80a6, pid=1921152, tid=0x000014909f2f2700
      # V [libjvm.so+0x98c0a6] PSParallelCompact::IsAliveClosure::do_object_b(oopDesc*)+0x36
      ========================================
      Stack:
      ----------------------------------------
      Stack: [0x000014909f1f3000,0x000014909f2f3000], sp=0x000014909f2f11f0, free space=1016k
      Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
      V [libjvm.so+0x98c0a6] PSParallelCompact::IsAliveClosure::do_object_b(oopDesc*)+0x36
      V [libjvm.so+0x8c8f6c] MethodData::clean_extra_data(BoolObjectClosure*)+0x1dc
      V [libjvm.so+0x648cf6] InstanceKlass::clean_weak_instanceklass_links(BoolObjectClosure*)+0x56
      V [libjvm.so+0x7d0d9e] Klass::clean_weak_klass_links(BoolObjectClosure*, bool)+0x18e
      V [libjvm.so+0x98dea1] PSParallelCompact::marking_phase(ParCompactionManager*, bool, ParallelOldTracer*)+0x4e1
      V [libjvm.so+0x99303e] PSParallelCompact::invoke_no_policy(bool)+0x40e
      V [libjvm.so+0x9938b3] PSParallelCompact::invoke(bool)+0x63
      V [libjvm.so+0xad16e8] VM_ParallelGCSystemGC::doit()+0xa8
      ...
      ========================================
      ANALYSIS:
      ----------------------------------------
      error
      ----------------------------------------
      *JVM crash inside libjvm.so. Reference: https://access.redhat.com/solutions/442903.
      *The crash occurred dereferencing a null pointer (si_addr: 0x0000000000000000).
      *Duplicate jvm options: -XX:MetaspaceSize=96m -XX:MetaspaceSize=256m.
      *Duplicate jvm options: -XX:MaxMetaspaceSize=256m -XX:MaxMetaspaceSize=512m.
      ----------------------------------------
      warn
      ----------------------------------------
      *MaxMetaspaceSize is less than CompressedClassSpaceSize. MaxMetaspaceSize includes CompressedClassSpaceSize, so MaxMetaspaceSize should be larger than CompressedClassSpaceSize. If MaxMetaspaceSize is set smaller than CompressedClassSpaceSize, the JVM auto adjusts CompressedClassSpaceSize as follows: CompressedClassSpaceSize = MaxMetaspaceSize - (2 * InitialBootClassLoaderMetaspaceSize).
      *Number of GC log files is defined (-XX:NumberOfGCLogFiles), yet GC log file rotation is disabled. Either remove -XX:NumberOfGCLogFiles or enable GC log file rotation with -XX:+UseGCLogFileRotation.
      *The gc log file has a static name and will be overwritten on JVM startup. Enable log file rotation and/or include process id or datestamp in the file name (e.g. -Xloggc:gc_%p_%t.log).
      *GC log file size (-XX:GCLogFileSize=N[K|M|G]) is small (< 5M). Data needed for troubleshooting could be lost due to excessive log rotation. Consider increasing the size.
      *The gc log file has a static name and will be overwritten on JVM startup. Enable log file rotation and/or include process id or datestamp in the file name (e.g. -Xloggc:gc_%p_%t.log).
      ----------------------------------------
      info
      ----------------------------------------
      *JDK does not appear to be a Red Hat build.
      *Signal number SIGSEGV: Segmentation fault. Accessing valid memory in an invalid way.
      *Signal code SI_KERNEL: Sent by the kernel.
      *Initial and/or max metaspace size is being set. This is generally not recommended. Reference: https://access.redhat.com/solutions/1489263.
      *Consider adding -XX:+HeapDumpOnOutOfMemoryError, a standard recommended option to generate a heap dump when the first thread throws OutOfMemoryError. It does not impact performance (until the heap is actually written out) and generally should always be used, as it provides critical information in case of a memory error.
      *Metaspace includes class metadata plus compressed class space.
      *Instrumentation is being used: -javaagent:/opt/ca_introscope/rh_jboss/wily/Agent.jar.
      *GC log file rotation is disabled (-XX:-UseGCLogFileRotation). Consider enabling rotation to protect disk space.
      ========================================


      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      restart the jvm

      FREQUENCY : often


            rsurianaraya Rajagopal Surianarayanan
            webbuggrp Webbug Group
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: