-
Enhancement
-
Resolution: Unresolved
-
P4
-
17, 21
-
Fix Understood
Currently, heap dumps can be taken by the SA tools from a frozen process or core file or directly from a running process with jcmd, jconsole & JMX, jmap, etc. If the heap of a running process is dumped, this happens at a safepoint (see VM_HeapDumper). Because the time to produce a heap dump is roughly proportional to the size and fill ratio of the heap, this leads to safepoint times which can range from ~100ms for a 100mb heap to ~1s for a 1gb heap up to 15s and more for a 8gb heap (measured on my Core i7 laptop with SSD).
One possibility to decrease the safepoint time is to offload the dump work to an asynchronous process. On Linux (and probably any other OS which supports fork()) this can be achieved by forking and offloading the dump process to child process. Forking still needs to happen at a safepoint, but forking is considerably faster compared to the dumping process itself. The fork performance is still proportional to the size of the original Java process. Although fork won't copy any memory pages, the kernel still needs to duplicate the page table entries of the process.
Linux uses a “copy-on-write” technique for the creation of a forked child process. This means that right after creation, the child process will have exactly the same memory image like its parent process. But at the same time, the child process won’t use any additional physical memory, as long as it doesn’t change (i.e. writes into) its memory. Since heap dumping only reads the child process's memory and then exits immediately, this technique can be applied even if the Java process already uses almost the whole free physical memory.
I've created a quick POC for Linux (see http://cr.openjdk.java.net/~simonis/webrevs/2020/8252768/) which decreases the aforementioned ~100ms, ~1s and 15s for a 100mb, 1gb and 8gb heap to ~3ms, ~15ms and ~60ms on my laptop which I think is significant. You can try it out by using the new "-async" or "-async=true" option of the "GC.heap_dump" jcmd command.
Of course this will require a CSR for the additional jcmd GC.heap_dump "-async" option which I'll be happy to create if there's any interest in this enhancement.
One possibility to decrease the safepoint time is to offload the dump work to an asynchronous process. On Linux (and probably any other OS which supports fork()) this can be achieved by forking and offloading the dump process to child process. Forking still needs to happen at a safepoint, but forking is considerably faster compared to the dumping process itself. The fork performance is still proportional to the size of the original Java process. Although fork won't copy any memory pages, the kernel still needs to duplicate the page table entries of the process.
Linux uses a “copy-on-write” technique for the creation of a forked child process. This means that right after creation, the child process will have exactly the same memory image like its parent process. But at the same time, the child process won’t use any additional physical memory, as long as it doesn’t change (i.e. writes into) its memory. Since heap dumping only reads the child process's memory and then exits immediately, this technique can be applied even if the Java process already uses almost the whole free physical memory.
I've created a quick POC for Linux (see http://cr.openjdk.java.net/~simonis/webrevs/2020/8252768/) which decreases the aforementioned ~100ms, ~1s and 15s for a 100mb, 1gb and 8gb heap to ~3ms, ~15ms and ~60ms on my laptop which I think is significant. You can try it out by using the new "-async" or "-async=true" option of the "GC.heap_dump" jcmd command.
Of course this will require a CSR for the additional jcmd GC.heap_dump "-async" option which I'll be happy to create if there's any interest in this enhancement.