Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8023720

(hotspot) setjmp/longjmp changes the process signal mask on OS X

XMLWordPrintable

    • svc
    • b49
    • os_x
    • Not verified

        This bug originated from the following report: http://mail.openjdk.java.net/pipermail/serviceability-dev/2013-August/011318.html

        That report has a reproducer that causes attach to stop working on OS X. The reason is that the process ends up with a signal mask that blocks SIGQUIT:

        - The VM sets the signal mask for all threads (except the internal VMThread) to mask out SIGQUIT using pthread_sigmask(). So the process will still respond to SIGQUIT, but only one thread.
        - The verifier code calls setjmp() to save the calling context. On OS X this also saves the signal mask.
        - The example code causes a verification error somewhere which causes the verifier to call longjmp().
        - longjmp will restore the signal mask using sigprocmask() which sets the signal mask for the _process_.
        - Now the process has a signal mask that masks out SIGQUIT and attach stops working.

        This only happens on OS X because setjmp/longjmp does not save and restore the signal mask on other platforms. There are functions _setjmp/_longjmp on OS X to skip the signal mask save/restore and I think this is what we need to use in the verifier (also need to check other uses of setjmp/longjmp in the JDK).

              sla Staffan Larsen (Inactive)
              sla Staffan Larsen (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: