Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8314148

Fix variable scope in SunMSCAPI

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • P4
    • 22
    • 22
    • security-libs
    • None

    Description

      Found when compiling with Clang for Windows; the following tests crash:
      sun/security/mscapi/InteropWithSunRsaSign.java
      sun/security/mscapi/ShortRSAKeyWithinTLS.java

      The produced stack trace points inside NCryptSignHash, which is called from Java_sun_security_mscapi_CSignature_signCngHash.

      The problem here is that the 2 local variables:
          BCRYPT_PKCS1_PADDING_INFO pkcs1Info;
          BCRYPT_PSS_PADDING_INFO pssInfo;
      are scoped to the "switch" statement, but used later. Clang optimizes away writes to these variables, which in turn leads to crashes. It works just fine with the MSVC compiler, presumably its optimizer is less aggressive.

      The fix involves extending the scope of these variables to the entire method.

      Attachments

        Issue Links

          Activity

            People

              djelinski Daniel Jelinski
              djelinski Daniel Jelinski
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: