Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8023720

(hotspot) setjmp/longjmp changes the process signal mask on OS X

    XMLWordPrintable

Details

    • svc
    • b49
    • os_x
    • Not verified

    Backports

      Description

        This bug originated from the following report: http://mail.openjdk.java.net/pipermail/serviceability-dev/2013-August/011318.html

        That report has a reproducer that causes attach to stop working on OS X. The reason is that the process ends up with a signal mask that blocks SIGQUIT:

        - The VM sets the signal mask for all threads (except the internal VMThread) to mask out SIGQUIT using pthread_sigmask(). So the process will still respond to SIGQUIT, but only one thread.
        - The verifier code calls setjmp() to save the calling context. On OS X this also saves the signal mask.
        - The example code causes a verification error somewhere which causes the verifier to call longjmp().
        - longjmp will restore the signal mask using sigprocmask() which sets the signal mask for the _process_.
        - Now the process has a signal mask that masks out SIGQUIT and attach stops working.

        This only happens on OS X because setjmp/longjmp does not save and restore the signal mask on other platforms. There are functions _setjmp/_longjmp on OS X to skip the signal mask save/restore and I think this is what we need to use in the verifier (also need to check other uses of setjmp/longjmp in the JDK).

        Attachments

          Issue Links

            Activity

              People

                sla Staffan Larsen (Inactive)
                sla Staffan Larsen (Inactive)
                Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                  Created:
                  Updated:
                  Resolved: