Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8314148

Fix variable scope in SunMSCAPI

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: P4 P4
    • 22
    • 22
    • security-libs
    • None

      Found when compiling with Clang for Windows; the following tests crash:
      sun/security/mscapi/InteropWithSunRsaSign.java
      sun/security/mscapi/ShortRSAKeyWithinTLS.java

      The produced stack trace points inside NCryptSignHash, which is called from Java_sun_security_mscapi_CSignature_signCngHash.

      The problem here is that the 2 local variables:
          BCRYPT_PKCS1_PADDING_INFO pkcs1Info;
          BCRYPT_PSS_PADDING_INFO pssInfo;
      are scoped to the "switch" statement, but used later. Clang optimizes away writes to these variables, which in turn leads to crashes. It works just fine with the MSVC compiler, presumably its optimizer is less aggressive.

      The fix involves extending the scope of these variables to the entire method.

            djelinski Daniel Jelinski
            djelinski Daniel Jelinski
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: