Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8358957

[ubsan]: The assert in layout_helper_boolean_diffbit() in klass.hpp needs UB to fail

XMLWordPrintable

    • Icon: Enhancement Enhancement
    • Resolution: Unresolved
    • Icon: P4 P4
    • 26
    • None
    • hotspot

      In layout_helper_boolean_diffbit(), inside the loop that is calculating the result, there is an assert that diffbit (the in-progress calculated result) is not zero. But it can never be zero in the absence of UB. It is initialized to 1, and then left shifted 1 for each iteration. That bit could be shifted all the way to the sign bit, so that the value is INT_MIN. But another left shift, which naively would result in zero, is actually UB.

            azafari Afshin Zafari
            azafari Afshin Zafari
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: