Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8223940

Private key not supported by chosen signature algorithm

XMLWordPrintable

    • b23
    • x86_64
    • windows_7
    • Not verified

        A DESCRIPTION OF THE PROBLEM :
        When using a PKCS11 keystore (using the SunPKCS11 Provider) for client authentication in TLSv1.2, the client chooses a signature algorithm which does not support the P11PrivateKey.

        The server sends a list of supported SignatureSchemes for use in client authentication. The method sun.security.ssl.SignatureScheme.getPreferableAlgorithm(List<SignatureScheme>, PrivateKey, ProtocolVersion) chooses which SignatureScheme to use in client authentication. It can choose any algorithm for which a provider exists, even if the algorithm's java.security.Provider$Service.supportsParameter(privateKey) == false. In this case it chooses RSASSA-PSS from the provider sun.security.rsa.SunRsaSign.

        Because of this, signing with P11PrivateKey fails (with a misleading message stating no providers support the key, when it's a question of the <algorithm name> implementations not supporting the key):

        Caused by: javax.net.ssl.SSLHandshakeException: Cannot produce CertificateVerify signature
        at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:128)
        at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)
        at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
        at java.base/sun.security.ssl.CertificateVerify$T12CertificateVerifyMessage.<init>(CertificateVerify.java:590)
        at java.base/sun.security.ssl.CertificateVerify$T12CertificateVerifyProducer.produce(CertificateVerify.java:740)
        at java.base/sun.security.ssl.SSLHandshake.produce(SSLHandshake.java:436)
        at java.base/sun.security.ssl.ServerHelloDone$ServerHelloDoneConsumer.consume(ServerHelloDone.java:173)
        at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
        at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
        at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421)
        at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:178)
        at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164)
        at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1152)
        at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1063)
        at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402)
        at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:567)
        at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1356)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1331)
        at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:241)
        <snip>
        Caused by: java.security.InvalidKeyException: No installed provider supports this key: sun.security.pkcs11.P11Key$P11PrivateKey
        at java.base/java.security.Signature$Delegate.chooseProvider(Signature.java:1163)
        at java.base/java.security.Signature$Delegate.engineInitSign(Signature.java:1204)
        at java.base/java.security.Signature.initSign(Signature.java:546)
        at java.base/sun.security.ssl.SignatureScheme.getSignature(SignatureScheme.java:473)
        at java.base/sun.security.ssl.CertificateVerify$T12CertificateVerifyMessage.<init>(CertificateVerify.java:580)
        ... 26 more

        The problem cannot be worked around by adding RSASSA-PSS to the java.security configuration jdk.tls.disabledAlgorithms property.

        STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
        Regular Java TLS client using PKCS11 KeyStore from the SunPKCS11 provider for client authentication.

        TLSv1.2 server with client authentication, with supported_signature_algorithms preferring RSASSA-PSS (not supported by PKCS11 provider) over RSA_PKCS1_SHA256 (supported by PKCS11 provider).

        Open connection from client to server.

        EXPECTED VERSUS ACTUAL BEHAVIOR :
        EXPECTED -
        The client chooses the RSA_PKCS1_SHA256 signature algorithm. The TLS connection is established with PKCS11 for client authentication.
        ACTUAL -
        The client chooses the RSASSA-PSS signature algorithm. The client cannot complete TLS handshake because it cannot use the P11PrivateKey in the signature algorithm.

        CUSTOMER SUBMITTED WORKAROUND :
        Configure server so that supported_signature_algorithms prefers signature algorithms supported by the SunPKCS11 provider (RSA_PKCS1_SHA256, RSA_PKCS1_SHA384, RSA_PKCS1_SHA_512, RSA_SHA224, RSA_PKCS1_SHA1).

        FREQUENCY : always


            [JDK-8223940] Private key not supported by chosen signature algorithm

            Yuri Nesterenko added a comment -
            Fix request (13u):
            applicable to 13u as well and should be downported. Patch applies clean at this moment; all relevant regtests do pass.

            Yuri Nesterenko added a comment - Fix request (13u): applicable to 13u as well and should be downported. Patch applies clean at this moment; all relevant regtests do pass.

            Andrew Hughes added a comment -
            Fix Request [8u]:

            Fix is needed now that 8u272 has the TLSv1.3 stack from 11u. If a client prefers RSA-PSS, but the provider does not support it, it will fail as above rather than falling back on other providers.

            Regarding the workaround, I could not see any obvious way for the server side code to just be configured to reject RSA-PSS. In fact, the bug states that signatures can't be added to jdk.tls.disabledAlgorithms.

            The fix applies cleanly once automated shuffling is applied with the 11u & 9u scripts.

            Andrew Hughes added a comment - Fix Request [8u]: Fix is needed now that 8u272 has the TLSv1.3 stack from 11u. If a client prefers RSA-PSS, but the provider does not support it, it will fail as above rather than falling back on other providers. Regarding the workaround, I could not see any obvious way for the server side code to just be configured to reject RSA-PSS. In fact, the bug states that signatures can't be added to jdk.tls.disabledAlgorithms. The fix applies cleanly once automated shuffling is applied with the 11u & 9u scripts.

            Goetz Lindenmaier added a comment -
            (11u)
            RFR is available:
            http://mail.openjdk.java.net/pipermail/jdk-updates-dev/2020-June/003257.html

            Goetz Lindenmaier added a comment - (11u) RFR is available: http://mail.openjdk.java.net/pipermail/jdk-updates-dev/2020-June/003257.html

            Severin Gehwolf added a comment -
            [~mbaesken] Given that there is no patch and we are already in ramp down for July and there is a work-around - configure server so that it prefers signature algorithms supported by the SunPKCS11 provider. It seems a corner case. I'm setting jdk11u-critical-no. Please request jdk11u-fix-request instead.

            Severin Gehwolf added a comment - [~mbaesken] Given that there is no patch and we are already in ramp down for July and there is a work-around - configure server so that it prefers signature algorithms supported by the SunPKCS11 provider. It seems a corner case. I'm setting jdk11u-critical-no. Please request jdk11u-fix-request instead.

            Matthias Baesken added a comment -
            jdk11 backport request
            I would like to have the patch in openjdk11 as well (for better parity with 11.0.8_oracle). The patch does not apply cleanly, RFR follows.

            Matthias Baesken added a comment - jdk11 backport request I would like to have the patch in openjdk11 as well (for better parity with 11.0.8_oracle). The patch does not apply cleanly, RFR follows.

            HG Updates added a comment -
            URL: https://hg.openjdk.java.net/jdk/jdk/rev/d6e682e8fcc3
            User: xuelei
            Date: 2019-11-06 17:45:19 +0000

            HG Updates added a comment - URL: https://hg.openjdk.java.net/jdk/jdk/rev/d6e682e8fcc3 User: xuelei Date: 2019-11-06 17:45:19 +0000

            Xuelei Fan added a comment -
            This bug may have compatibility impact, please consider backport to supported releases.

            Xuelei Fan added a comment - This bug may have compatibility impact, please consider backport to supported releases.

            Xuelei Fan added a comment -
            The TLS code could fall back to use the next preferred signature algorithm for the specific key. However, it could be effective if the provider could check if the specific algorithms and parameters are supported.

            The idea to use java.security.Provider$Service.supportsParameter(privateKey) does not actually work as the signature parameters are expected to work with multiple providers and keys. It is not possible to have the signature parameters provider know the key provider details. So SignatureParameter.provider.service.supportsParameter(privateKey) may not work as expected.

            Maybe, privateKey.provider.service.supportsParameter(RSASSA-PSSParamter) could work. However, the current provider does not support to check signature algorithm parameters. It may be a potential improvement.

            Xuelei Fan added a comment - The TLS code could fall back to use the next preferred signature algorithm for the specific key. However, it could be effective if the provider could check if the specific algorithms and parameters are supported. The idea to use java.security.Provider$Service.supportsParameter(privateKey) does not actually work as the signature parameters are expected to work with multiple providers and keys. It is not possible to have the signature parameters provider know the key provider details. So SignatureParameter.provider.service.supportsParameter(privateKey) may not work as expected. Maybe, privateKey.provider.service.supportsParameter(RSASSA-PSSParamter) could work. However, the current provider does not support to check signature algorithm parameters. It may be a potential improvement.

            Valerie Peng added a comment -
            Re-assigning this to Xuelei so he can evaluate if TLS code can detect this issue and switch to a different signature impl or algorithm.

            Valerie Peng added a comment - Re-assigning this to Xuelei so he can evaluate if TLS code can detect this issue and switch to a different signature impl or algorithm.

            Valerie Peng added a comment - - edited
            Yes, this does look like a duplicate of JDK-8217611. However, I wonder if the TLS code can somehow fall back to use the next preferred signature algorithm if the specified key cannot be used by the desired signature implementation. Or is it too late to fall back to a different signature algorithm at that point?

            I didn't close this as a duplicate of JDK-8217611 because if such scenario can be detected by TLS code and properly handled, then it can be addressed here,

            Valerie Peng added a comment - - edited Yes, this does look like a duplicate of JDK-8217611 . However, I wonder if the TLS code can somehow fall back to use the next preferred signature algorithm if the specified key cannot be used by the desired signature implementation. Or is it too late to fall back to a different signature algorithm at that point? I didn't close this as a duplicate of JDK-8217611 because if such scenario can be detected by TLS code and properly handled, then it can be addressed here,

              xuelei Xuelei Fan
              webbuggrp Webbug Group
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

                Created:
                Updated:
                Resolved: