C2: CmpUNode::sub is not monotonic

XMLWordPrintable

    • b13
    • 26
    • b07

        The implementation of CmpUNode::sub returns CC_LE if the first input is a constant 0, and returns CC_NE if the inputs do not intersect. These can result in monotonicity violations. For example. given r = CmpU(x, y)

        At the first iteration, type(x) = {0}, type(y) = {-1, 1}. CmpUNode::sub returns CC_LE.

        At the next iteration, type(x) = {0, 2}, type(y) = {-1, 1}. CmpUNode::sub returns CC_NE.

        As CC_LE is not a subset of CC_NE, monotonicity is violated.

          1. Test.java
            0.5 kB
            Quan Anh Mai

              Assignee:
              Quan Anh Mai
              Reporter:
              Quan Anh Mai
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: