The full description is in the bug https://youtrack.jetbrains.com/issue/IDEA-305777/Memory-leak-from-com.jetbrains.jdi.VirtualMachineImpl-when-debugging-programs.
In short: if the program is creating a lot of short living threads and just being run under the debugger with enabled threadStart and threadDeath events enabled, jdi will consume more and more memory until OOM.
The problem is split into two places:
1. VirtualMachineImpl.processBatchedDisposes is invoked only inside waitForTargetReply, so if no commands are sent to the debugger it will never be called and batchedDisposeRequests may grow too big
2. unreachable listeners are removed from VMState.listeners only when something is happening with the debugger (VMState.processVMAction is called), which is not happening in our case, so listeners list grows and grows

We've fixed this in IntelliJ jdi fork this way: https://github.com/JetBrains/intellij-deps-jdi/commit/ca1b639d9fd648dd075f258517fdd1635d0eec2d
https://github.com/JetBrains/intellij-deps-jdi/commit/0b6b28d83e4ceaf3a550ebbf1eaeaf23b650f1c7

probably worth fixing in openjdk as well