Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8073711

Tomcat Memory high utilization error

    XMLWordPrintable

    Details

    • Subcomponent:
    • CPU:
      x86
    • OS:
      linux_redhat_6.0

      Description

      FULL PRODUCT VERSION :


      FULL OS VERSION :
      CUCM Product Ver : 7.1.5.34096-1 Platform Ver : 2.0.0.1-1


      EXTRA RELEVANT SYSTEM CONFIGURATION :
      #
      # A fatal error has been detected by the Java Runtime Environment:
      #
      # SIGSEGV (0xb) at pc=0x014071e0, pid=16178, tid=7748496
      #
      # JRE version: 6.0_41-b02
      # Java VM: Java HotSpot(TM) Server VM (20.14-b01 mixed mode linux-x86 )
      # Problematic frame:
      # V [libjvm.so+0x40c1e0] unsigned+0xd0
      #
      # If you would like to submit a bug report, please visit:
      # http://java.sun.com/webapps/bugreport/crash.jsp
      #

      --------------- T H R E A D ---------------

      Current thread (0x09ffc400): JavaThread "http-8443-1" daemon [_thread_in_vm, id=16400, stack(0x00714000,0x00764000)]

      siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x00000000

      Registers:
      EAX=0x00000000, EBX=0x017ffcc8, ECX=0x00000000, EDX=0x00000ffc
      ESP=0x00762894, EBP=0x007628f8, ESI=0x09ffc400, EDI=0x00000880
      EIP=0x014071e0, EFLAGS=0x00010212, CR2=0x00000000

      Top of Stack: (sp=0x00762894)
      0x00762894: 7b715830 00000007 01407129 09ffc400
      0x007628a4: 00000000 7b715830 007628b0 09ffc400
      0x007628b4: 00000000 00762908 008c3534 09ffc400
      0x007628c4: 0a065478 007628f8 0087a368 09ffc400
      0x007628d4: 0a065470 09f4f64c 0076290c 006ac018
      0x007628e4: 09f00c18 00000019 008c3534 0a065470
      0x007628f4: 0a065478 007629b8 008785f0 09ffc520
      0x00762904: 00000000 00000000 00000000 7ce80420

      Instructions: (pc=0x014071e0)
      0x014071c0: 75 c8 89 45 b4 89 75 b8 c7 45 bc 00 00 00 00 8b
      0x014071d0: 4e 04 85 c9 0f 85 36 01 00 00 8b 45 0c 83 ec 0c
      0x014071e0: 8b 38 57 e8 78 74 fd ff 8d 70 01 89 34 24 e8 3d
      0x014071f0: fa 1c 00 83 c4 10 85 c0 89 45 b0 0f 84 e8 00 00

      Register to memory mapping:

      EAX=0x00000000 is an unknown value
      EBX=0x017ffcc8: <offset 0x804cc8> in /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so at 0x00ffb000
      ECX=0x00000000 is an unknown value
      EDX=0x00000ffc is an unknown value
      ESP=0x00762894 is pointing into the stack for thread: 0x09ffc400
      EBP=0x007628f8 is pointing into the stack for thread: 0x09ffc400
      ESI=0x09ffc400 is a thread
      EDI=0x00000880 is an unknown value


      Stack: [0x00714000,0x00764000], sp=0x00762894, free space=314k
      Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
      V [libjvm.so+0x40c1e0] unsigned+0xd0
      C [libGenericAlarmJNI.so+0x2c5f0] Java_com_cisco_alarmutil_GenericAlarmJNI_sendAlarmwithBigBuffer__Ljava_lang_String_2I_3Ljava_lang_Object_2Ljava_lang_String_2+0x22c
      j com.cisco.alarmutil.GenericAlarmJNI.sendAlarmwithBigBuffer(Ljava/lang/String;I[Ljava/lang/Object;Ljava/lang/String;)I+0
      j com.cisco.alarmutil.GenericAlarms.sendAuditAlarm(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;IILjava/lang/String;I)I+290
      j com.cisco.platform.valve.AuthenticationValve.writeAuditlog(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;IIILjava/lang/String;)V+62
      j com.cisco.platform.valve.BasicAuthenticationValve.authenticate(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;Lorg/apache/catalina/deploy/LoginConfig;)Z+695
      j org.apache.catalina.authenticator.AuthenticatorBase.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+559
      j org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+64
      j org.apache.catalina.valves.ErrorReportValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+6

      Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
      j com.cisco.alarmutil.GenericAlarmJNI.sendAlarmwithBigBuffer(Ljava/lang/String;I[Ljava/lang/Object;Ljava/lang/String;)I+0
      j com.cisco.alarmutil.GenericAlarms.sendAuditAlarm(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;IILjava/lang/String;I)I+290
      j com.cisco.platform.valve.AuthenticationValve.writeAuditlog(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;IIILjava/lang/String;)V+62
      j com.cisco.platform.valve.BasicAuthenticationValve.authenticate(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;Lorg/apache/catalina/deploy/LoginConfig;)Z+695
      j org.apache.catalina.authenticator.AuthenticatorBase.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+559
      j org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+64
      j org.apache.catalina.valves.ErrorReportValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+6
      J org.apache.catalina.valves.AccessLogValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V
      j org.apache.catalina.authenticator.SingleSignOn.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+244
      j org.apache.catalina.core.StandardEngineValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+42
      j org.apache.catalina.connector.CoyoteAdapter.service(Lorg/apache/coyote/Request;Lorg/apache/coyote/Response;)V+158
      j org.apache.coyote.http11.Http11Processor.process(Ljava/net/Socket;)V+468
      j org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Ljava/net/Socket;)Z+82
      j org.apache.tomcat.util.net.JIoEndpoint$Worker.run()V+41
      j java.lang.Thread.run()V+11
      v ~StubRoutines::call_stub

      A DESCRIPTION OF THE PROBLEM :
      Tomcat process fails due to Java error. Error message in logs below. Failure removes existing configuration in CUCM


      #
      # A fatal error has been detected by the Java Runtime Environment:
      #
      # SIGSEGV (0xb) at pc=0x014071e0, pid=16178, tid=7748496
      #
      # JRE version: 6.0_41-b02
      # Java VM: Java HotSpot(TM) Server VM (20.14-b01 mixed mode linux-x86 )
      # Problematic frame:
      # V [libjvm.so+0x40c1e0] unsigned+0xd0
      #
      # If you would like to submit a bug report, please visit:
      # http://java.sun.com/webapps/bugreport/crash.jsp
      #

      --------------- T H R E A D ---------------

      Current thread (0x09ffc400): JavaThread "http-8443-1" daemon [_thread_in_vm, id=16400, stack(0x00714000,0x00764000)]

      siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x00000000

      Registers:
      EAX=0x00000000, EBX=0x017ffcc8, ECX=0x00000000, EDX=0x00000ffc
      ESP=0x00762894, EBP=0x007628f8, ESI=0x09ffc400, EDI=0x00000880
      EIP=0x014071e0, EFLAGS=0x00010212, CR2=0x00000000

      Top of Stack: (sp=0x00762894)
      0x00762894: 7b715830 00000007 01407129 09ffc400
      0x007628a4: 00000000 7b715830 007628b0 09ffc400
      0x007628b4: 00000000 00762908 008c3534 09ffc400
      0x007628c4: 0a065478 007628f8 0087a368 09ffc400
      0x007628d4: 0a065470 09f4f64c 0076290c 006ac018
      0x007628e4: 09f00c18 00000019 008c3534 0a065470
      0x007628f4: 0a065478 007629b8 008785f0 09ffc520
      0x00762904: 00000000 00000000 00000000 7ce80420

      Instructions: (pc=0x014071e0)
      0x014071c0: 75 c8 89 45 b4 89 75 b8 c7 45 bc 00 00 00 00 8b
      0x014071d0: 4e 04 85 c9 0f 85 36 01 00 00 8b 45 0c 83 ec 0c
      0x014071e0: 8b 38 57 e8 78 74 fd ff 8d 70 01 89 34 24 e8 3d
      0x014071f0: fa 1c 00 83 c4 10 85 c0 89 45 b0 0f 84 e8 00 00

      Register to memory mapping:

      EAX=0x00000000 is an unknown value
      EBX=0x017ffcc8: <offset 0x804cc8> in /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so at 0x00ffb000
      ECX=0x00000000 is an unknown value
      EDX=0x00000ffc is an unknown value
      ESP=0x00762894 is pointing into the stack for thread: 0x09ffc400
      EBP=0x007628f8 is pointing into the stack for thread: 0x09ffc400
      ESI=0x09ffc400 is a thread
      EDI=0x00000880 is an unknown value


      Stack: [0x00714000,0x00764000], sp=0x00762894, free space=314k
      Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
      V [libjvm.so+0x40c1e0] unsigned+0xd0
      C [libGenericAlarmJNI.so+0x2c5f0] Java_com_cisco_alarmutil_GenericAlarmJNI_sendAlarmwithBigBuffer__Ljava_lang_String_2I_3Ljava_lang_Object_2Ljava_lang_String_2+0x22c
      j com.cisco.alarmutil.GenericAlarmJNI.sendAlarmwithBigBuffer(Ljava/lang/String;I[Ljava/lang/Object;Ljava/lang/String;)I+0
      j com.cisco.alarmutil.GenericAlarms.sendAuditAlarm(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;IILjava/lang/String;I)I+290
      j com.cisco.platform.valve.AuthenticationValve.writeAuditlog(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;IIILjava/lang/String;)V+62
      j com.cisco.platform.valve.BasicAuthenticationValve.authenticate(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;Lorg/apache/catalina/deploy/LoginConfig;)Z+695
      j org.apache.catalina.authenticator.AuthenticatorBase.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+559
      j org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+64
      j org.apache.catalina.valves.ErrorReportValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+6

      Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
      j com.cisco.alarmutil.GenericAlarmJNI.sendAlarmwithBigBuffer(Ljava/lang/String;I[Ljava/lang/Object;Ljava/lang/String;)I+0
      j com.cisco.alarmutil.GenericAlarms.sendAuditAlarm(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;IILjava/lang/String;I)I+290
      j com.cisco.platform.valve.AuthenticationValve.writeAuditlog(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;IIILjava/lang/String;)V+62
      j com.cisco.platform.valve.BasicAuthenticationValve.authenticate(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;Lorg/apache/catalina/deploy/LoginConfig;)Z+695
      j org.apache.catalina.authenticator.AuthenticatorBase.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+559
      j org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+64
      j org.apache.catalina.valves.ErrorReportValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+6
      J org.apache.catalina.valves.AccessLogValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V
      j org.apache.catalina.authenticator.SingleSignOn.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+244
      j org.apache.catalina.core.StandardEngineValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+42
      j org.apache.catalina.connector.CoyoteAdapter.service(Lorg/apache/coyote/Request;Lorg/apache/coyote/Response;)V+158
      j org.apache.coyote.http11.Http11Processor.process(Ljava/net/Socket;)V+468
      j org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Ljava/net/Socket;)Z+82
      j org.apache.tomcat.util.net.JIoEndpoint$Worker.run()V+41
      j java.lang.Thread.run()V+11
      v ~StubRoutines::call_stub

      THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: No

      THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: No

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      Peak out the memory usage on the tomcat process in cucm.

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      Single number reach configurations remove themselves.
      ERROR MESSAGES/STACK TRACES THAT OCCUR :
      #
      # A fatal error has been detected by the Java Runtime Environment:
      #
      # SIGSEGV (0xb) at pc=0x014071e0, pid=16178, tid=7748496
      #
      # JRE version: 6.0_41-b02
      # Java VM: Java HotSpot(TM) Server VM (20.14-b01 mixed mode linux-x86 )
      # Problematic frame:
      # V [libjvm.so+0x40c1e0] unsigned+0xd0
      #
      # If you would like to submit a bug report, please visit:
      # http://java.sun.com/webapps/bugreport/crash.jsp
      #

      --------------- T H R E A D ---------------

      Current thread (0x09ffc400): JavaThread "http-8443-1" daemon [_thread_in_vm, id=16400, stack(0x00714000,0x00764000)]

      siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x00000000

      Registers:
      EAX=0x00000000, EBX=0x017ffcc8, ECX=0x00000000, EDX=0x00000ffc
      ESP=0x00762894, EBP=0x007628f8, ESI=0x09ffc400, EDI=0x00000880
      EIP=0x014071e0, EFLAGS=0x00010212, CR2=0x00000000

      Top of Stack: (sp=0x00762894)
      0x00762894: 7b715830 00000007 01407129 09ffc400
      0x007628a4: 00000000 7b715830 007628b0 09ffc400
      0x007628b4: 00000000 00762908 008c3534 09ffc400
      0x007628c4: 0a065478 007628f8 0087a368 09ffc400
      0x007628d4: 0a065470 09f4f64c 0076290c 006ac018
      0x007628e4: 09f00c18 00000019 008c3534 0a065470
      0x007628f4: 0a065478 007629b8 008785f0 09ffc520
      0x00762904: 00000000 00000000 00000000 7ce80420

      Instructions: (pc=0x014071e0)
      0x014071c0: 75 c8 89 45 b4 89 75 b8 c7 45 bc 00 00 00 00 8b
      0x014071d0: 4e 04 85 c9 0f 85 36 01 00 00 8b 45 0c 83 ec 0c
      0x014071e0: 8b 38 57 e8 78 74 fd ff 8d 70 01 89 34 24 e8 3d
      0x014071f0: fa 1c 00 83 c4 10 85 c0 89 45 b0 0f 84 e8 00 00

      Register to memory mapping:

      EAX=0x00000000 is an unknown value
      EBX=0x017ffcc8: <offset 0x804cc8> in /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so at 0x00ffb000
      ECX=0x00000000 is an unknown value
      EDX=0x00000ffc is an unknown value
      ESP=0x00762894 is pointing into the stack for thread: 0x09ffc400
      EBP=0x007628f8 is pointing into the stack for thread: 0x09ffc400
      ESI=0x09ffc400 is a thread
      EDI=0x00000880 is an unknown value


      Stack: [0x00714000,0x00764000], sp=0x00762894, free space=314k
      Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
      V [libjvm.so+0x40c1e0] unsigned+0xd0
      C [libGenericAlarmJNI.so+0x2c5f0] Java_com_cisco_alarmutil_GenericAlarmJNI_sendAlarmwithBigBuffer__Ljava_lang_String_2I_3Ljava_lang_Object_2Ljava_lang_String_2+0x22c
      j com.cisco.alarmutil.GenericAlarmJNI.sendAlarmwithBigBuffer(Ljava/lang/String;I[Ljava/lang/Object;Ljava/lang/String;)I+0
      j com.cisco.alarmutil.GenericAlarms.sendAuditAlarm(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;IILjava/lang/String;I)I+290
      j com.cisco.platform.valve.AuthenticationValve.writeAuditlog(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;IIILjava/lang/String;)V+62
      j com.cisco.platform.valve.BasicAuthenticationValve.authenticate(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;Lorg/apache/catalina/deploy/LoginConfig;)Z+695
      j org.apache.catalina.authenticator.AuthenticatorBase.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+559
      j org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+64
      j org.apache.catalina.valves.ErrorReportValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+6

      Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
      j com.cisco.alarmutil.GenericAlarmJNI.sendAlarmwithBigBuffer(Ljava/lang/String;I[Ljava/lang/Object;Ljava/lang/String;)I+0
      j com.cisco.alarmutil.GenericAlarms.sendAuditAlarm(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;IILjava/lang/String;I)I+290
      j com.cisco.platform.valve.AuthenticationValve.writeAuditlog(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;IIILjava/lang/String;)V+62
      j com.cisco.platform.valve.BasicAuthenticationValve.authenticate(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;Lorg/apache/catalina/deploy/LoginConfig;)Z+695
      j org.apache.catalina.authenticator.AuthenticatorBase.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+559
      j org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+64
      j org.apache.catalina.valves.ErrorReportValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+6
      J org.apache.catalina.valves.AccessLogValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V
      j org.apache.catalina.authenticator.SingleSignOn.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+244
      j org.apache.catalina.core.StandardEngineValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+42
      j org.apache.catalina.connector.CoyoteAdapter.service(Lorg/apache/coyote/Request;Lorg/apache/coyote/Response;)V+158
      j org.apache.coyote.http11.Http11Processor.process(Ljava/net/Socket;)V+468
      j org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Ljava/net/Socket;)Z+82
      j org.apache.tomcat.util.net.JIoEndpoint$Worker.run()V+41
      j java.lang.Thread.run()V+11
      v ~StubRoutines::call_stub

      REPRODUCIBILITY :
      This bug can be reproduced rarely.

      CUSTOMER SUBMITTED WORKAROUND :
      Restarting cisco tomcat

        Attachments

          Activity

            People

            Assignee:
            aroy Abhijit Roy (Inactive)
            Reporter:
            webbuggrp Webbug Group
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: