Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8023956

Provide a work-around to broken Linux 32 bit "Exec Shield" using CS for NX emulation (crashing with SI_KERNEL)



    • b53
    • x86
    • linux



        JDK-8015837 contains several crashes on 32 bit Linux due to a bug a patch from RedHat, part of "Exec Shield".

        RHEL 5 & 6 are affected, and earlier Ubuntu releases 10.04 (LTS), 11.04, and 12.04 have also taken in the patch. Whilst earlier Exec Shield patches had a switch, current incantations have no work-around, no way to disable.

        Details of the problem see RH bug filed: https://bugzilla.redhat.com/show_bug.cgi?id=996149

        To summarize: to emulate NX feature on X86_32 code segment is used to limit execution to the highest executable VA. There is a tiny race on SMP MM invalidation code which can cause the lazy CS update code in trap handling to think a general protection fault wasn't cause by itself. This results in sending the JVM a useless SIGSEGV with si_code:SI_KERNEL, results in JVM signal handling forcing a dump.

        The suggested work around (limited to 32 bit Linux): is to enable execution (PROT_EXEC) on a high address and execute some code.

        The memory layout on these distros is similar: TASK_LIMIT set to 3G (0xC0000000), and the main stack typically grows down from around here, so:
        1) using a page in main stack at JVM start up, test for said layout (don't bother if its not what we expect, i.e. exotic Linux config/distro)
        2) enable execution (mprotect)
        3) Place a single "ret" instruction and call it.

        This will cause CS to be updated to a high enough address to basically disable any further updates (due to code generation), effectively disabling the feature (which is already of questionable value, NX is the way to go).


          Issue Links



                dsimms David Simms
                dsimms David Simms
                0 Vote for this issue
                4 Start watching this issue