Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8237544

Failed to create heap dump file with "jhsdb jmap"

    XMLWordPrintable

Details

    • x86_64
    • linux

    Description

      ADDITIONAL SYSTEM INFORMATION :
      CentOS release 6.9 (Final)

      openjdk 11.0.6 2020-01-14
      OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.6+10)
      OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.6+10, mixed mode)


      A DESCRIPTION OF THE PROBLEM :
      Failed to create heap dump with "jhsdb jmap". See exact commands and errors in "Actual Result".
      It failed often (but sometimes works). It failed in both cases: connect to pid and connect to core dump file.
      Interesting that same command is failing with 3 different errors (probably depends on logic that executed in application at specific time):
      "java.lang.IndexOutOfBoundsException: bad SID 0", "sun.jvm.hotspot.types.WrongTypeException: No suitable match for type of address 0x0000000100118c40", "sun.jvm.hotspot.utilities.AssertionFailure: can not get class data for java/lang/invoke/LambdaForm$MH0x00000001010bbc40"

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      - Run java app
      - Create heap dump with jhsdb (core file mode and pid mode)

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      Heap dump created.
      ACTUAL -
      Error1:
      $ /opt/app/java/bin/jhsdb jmap --binaryheap --dumpfile hd.hprof --exe /opt/app/java/bin/java --core core.16506
      Attaching to core core.16506 from executable /opt/app/java/bin/java, please wait...
      Debugger attached successfully.
      Server compiler detected.
      JVM version is 11.0.6+10
      Exception in thread "main" java.lang.IndexOutOfBoundsException: bad SID 0
      at jdk.hotspot.agent/sun.jvm.hotspot.runtime.vmSymbols.symbolAt(vmSymbols.java:58)
      at jdk.hotspot.agent/sun.jvm.hotspot.oops.InstanceKlass.getFieldName(InstanceKlass.java:353)
      at jdk.hotspot.agent/sun.jvm.hotspot.oops.InstanceKlass.<init>(InstanceKlass.java:153)
      at jdk.internal.reflect.GeneratedConstructorAccessor4.newInstance(Unknown Source)
      at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
      at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VMObjectFactory.newObject(VMObjectFactory.java:58)
      at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VirtualBaseConstructor.instantiateWrapperFor(VirtualBaseConstructor.java:104)
      at jdk.hotspot.agent/sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(Metadata.java:73)
      at jdk.hotspot.agent/sun.jvm.hotspot.oops.java_lang_Class.asKlass(java_lang_Class.java:67)
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.HeapHprofBinWriter.writeClass(HeapHprofBinWriter.java:608)
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.AbstractHeapGraphWriter$1.doObj(AbstractHeapGraphWriter.java:83)
      at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.iterateLiveRegions(ObjectHeap.java:355)
      at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.iterate(ObjectHeap.java:173)
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.AbstractHeapGraphWriter.write(AbstractHeapGraphWriter.java:52)
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.HeapHprofBinWriter.write(HeapHprofBinWriter.java:440)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.writeHeapHprofBin(JMap.java:182)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.run(JMap.java:97)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:223)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176)
      at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:369)
      at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:538)

      Error2:
      $ /opt/app/java/bin/jhsdb jmap --binaryheap --dumpfile hd.hprof --exe /opt/app/java/bin/java --core core.16506
      Attaching to core core.16506 from executable /opt/app/java/bin/java, please wait...
      Debugger attached successfully.
      Server compiler detected.
      JVM version is 11.0.6+10
      Exception in thread "main" sun.jvm.hotspot.types.WrongTypeException: No suitable match for type of address 0x0000000100118c40
      at jdk.hotspot.agent/sun.jvm.hotspot.runtime.InstanceConstructor.newWrongTypeException(InstanceConstructor.java:62)
      at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VirtualBaseConstructor.instantiateWrapperFor(VirtualBaseConstructor.java:109)
      at jdk.hotspot.agent/sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(Metadata.java:73)
      at jdk.hotspot.agent/sun.jvm.hotspot.oops.java_lang_Class.asKlass(java_lang_Class.java:67)
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.HeapHprofBinWriter.writeClass(HeapHprofBinWriter.java:608)
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.AbstractHeapGraphWriter$1.doObj(AbstractHeapGraphWriter.java:83)
      at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.iterateLiveRegions(ObjectHeap.java:355)
      at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.iterate(ObjectHeap.java:173)
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.AbstractHeapGraphWriter.write(AbstractHeapGraphWriter.java:52)
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.HeapHprofBinWriter.write(HeapHprofBinWriter.java:440)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.writeHeapHprofBin(JMap.java:182)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.run(JMap.java:97)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:223)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176)
      at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:369)
      at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:538)

      Error3:
      $ /opt/app/java/bin/jhsdb jmap --binaryheap --dumpfile b1.hprof --exe /opt/app/java/bin/java --core core_b1.25274
      Attaching to core core_b1.25274 from executable /opt/app/java/bin/java, please wait...
      Debugger attached successfully.
      Server compiler detected.
      JVM version is 11.0.5+10
      Exception in thread "main" sun.jvm.hotspot.utilities.AssertionFailure: can not get class data for java/lang/invoke/LambdaForm$MH0x00000001010bbc40
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.Assert.that(Assert.java:32)
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.HeapHprofBinWriter.writeInstance(HeapHprofBinWriter.java:940)
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.AbstractHeapGraphWriter$1.doObj(AbstractHeapGraphWriter.java:96)
      at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.iterateLiveRegions(ObjectHeap.java:355)
      at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.iterate(ObjectHeap.java:173)
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.AbstractHeapGraphWriter.write(AbstractHeapGraphWriter.java:52)
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.HeapHprofBinWriter.write(HeapHprofBinWriter.java:440)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.writeHeapHprofBin(JMap.java:182)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.run(JMap.java:97)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:223)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176)
      at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:329)
      at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:488)

      Same error when connect to pid:
      $ /opt/app/java/bin/jhsdb jmap --binaryheap --dumpfile hd1.hprof --pid 16506
      Attaching to process ID 16506, please wait...
      Debugger attached successfully.
      Server compiler detected.
      JVM version is 11.0.6+10
      Exception in thread "main" sun.jvm.hotspot.types.WrongTypeException: No suitable match for type of address 0x0000000100118c40
      at jdk.hotspot.agent/sun.jvm.hotspot.runtime.InstanceConstructor.newWrongTypeException(InstanceConstructor.java:62)
      at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VirtualBaseConstructor.instantiateWrapperFor(VirtualBaseConstructor.java:109)
      at jdk.hotspot.agent/sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(Metadata.java:73)
      at jdk.hotspot.agent/sun.jvm.hotspot.oops.java_lang_Class.asKlass(java_lang_Class.java:67)
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.HeapHprofBinWriter.writeClass(HeapHprofBinWriter.java:608)
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.AbstractHeapGraphWriter$1.doObj(AbstractHeapGraphWriter.java:83)
      at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.iterateLiveRegions(ObjectHeap.java:355)
      at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.iterate(ObjectHeap.java:173)
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.AbstractHeapGraphWriter.write(AbstractHeapGraphWriter.java:52)
      at jdk.hotspot.agent/sun.jvm.hotspot.utilities.HeapHprofBinWriter.write(HeapHprofBinWriter.java:440)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.writeHeapHprofBin(JMap.java:182)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.run(JMap.java:97)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:223)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
      at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176)
      at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:369)
      at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:538)


      ---------- BEGIN SOURCE ----------
      It is big application. Any idea how to troubleshot it and understand what java code lead to those errors? then I can build simple app for reproduce problem.
      ---------- END SOURCE ----------

      FREQUENCY : often


      Attachments

        Issue Links

          Activity

            People

              fmatte Fairoz Matte
              webbuggrp Webbug Group
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: