Recently (JDK-8373096) I had a customer issue with a stack overflow in a non-java thread.
For issues like these, we don't get an hs-err file, obviously, since we don't have a stack to run the error handler on. So the JVM crashes right away. If things go somewhat well, we will at least have a core dump. If not (e.g., running in the cloud with core file limit set to 0), we don't get anything; the JVM is just gone.
I dislike cases like these, since these are often not even investigated. If they are, the analyst often has to fight the logistics of the installation, which is time-consuming.
I think an abridged version of the java thread stack overflow protection would help here: protect a set of pages at the end of the stack, unprotect them on hit and then proceed to normal error reporting. The set of pages should be large enough to run the error handler, but I don't think we need that much space; maybe one or two 4K pages would be enough. We also don't need the full hs-err file. If hs-err reporting even gives us a glimpse of the call stack, we are fine - stack overflows are easy to solve once you see the call stack.
It would be nice if one could piggyback on the existing StackOverflow mechanism.
If doing this for every non-Java thread hits opposition, we should at least do it for the VMThread, since that one executes the widest variety of code in HotSpot and is most prone to problems from recursive code.
Possible follow-up/related issues:
- Maybe make callstack dumping limited and/or more intelligent to deal with huge call stacks. A simple solution could be to print the top n and bottom n frames, and leave out anything in the middle (for stack size > n*2).
- Maybe flush the hs-err file after every error reporting STEP, to make sure that we see the output of successful STEPs even if we don't get to the point where we close() the file. This may be a good idea to do in any case.
For issues like these, we don't get an hs-err file, obviously, since we don't have a stack to run the error handler on. So the JVM crashes right away. If things go somewhat well, we will at least have a core dump. If not (e.g., running in the cloud with core file limit set to 0), we don't get anything; the JVM is just gone.
I dislike cases like these, since these are often not even investigated. If they are, the analyst often has to fight the logistics of the installation, which is time-consuming.
I think an abridged version of the java thread stack overflow protection would help here: protect a set of pages at the end of the stack, unprotect them on hit and then proceed to normal error reporting. The set of pages should be large enough to run the error handler, but I don't think we need that much space; maybe one or two 4K pages would be enough. We also don't need the full hs-err file. If hs-err reporting even gives us a glimpse of the call stack, we are fine - stack overflows are easy to solve once you see the call stack.
It would be nice if one could piggyback on the existing StackOverflow mechanism.
If doing this for every non-Java thread hits opposition, we should at least do it for the VMThread, since that one executes the widest variety of code in HotSpot and is most prone to problems from recursive code.
Possible follow-up/related issues:
- Maybe make callstack dumping limited and/or more intelligent to deal with huge call stacks. A simple solution could be to print the top n and bottom n frames, and leave out anything in the middle (for stack size > n*2).
- Maybe flush the hs-err file after every error reporting STEP, to make sure that we see the output of successful STEPs even if we don't get to the point where we close() the file. This may be a good idea to do in any case.
- relates to
-
JDK-8373096 JFR leak profiler: path-to-gc-roots search should be non-recursive
-
- In Progress
-