Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8134396

Check upper limit of flags setting numbers of stack protection pages.

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: P4 P4
    • 9
    • 9
    • hotspot
    • None
    • 9

      TestOptionsWithRanges shows that illegal values for red and yellow pages are possible.

      In JavaThread::create_stack_guard_pages(),
      setting StackRedPages=92233720368547753 and StackYellowPages=1 yields

      len = (92233720368547753 + 1) * os::vm_page_size()
          = 0x8000000000000000 * os::vm_page_size()
          = 0

      Thus not protecting any pages of the stack.

      The check in os::init_2() succeeds, as there the Shadow pages are considered, too, so that the overflowed value is > 0.

      On linux, the VM stops with the misleading message
      "OpenJDK 64-Bit Server VM warning: Attempt to allocate stack guard pages failed."
      On aix, we get assertions and SIGSEGVs.

            goetz Goetz Lindenmaier
            goetz Goetz Lindenmaier
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: