PEM Encodings of Cryptographic Objects (Final)

XMLWordPrintable

    • Type: JEP
    • Resolution: Unresolved
    • Priority: P3
    • None
    • Component/s: security-libs
    • None
    • Anthony Scarpino
    • Feature
    • Open
    • SE
    • security dash dev at openjdk dot org
    • S
    • S

      Summary

      Introduce an API for encoding objects that represent cryptographic keys, certificates, and certificate revocation lists into the widely-used Privacy-Enhanced Mail (PEM) transport format, and for decoding from that format back into objects. This is a preview API.

      History

      The PEM API was proposed as a preview feature by JEP 470 and delivered in JDK 25. To allow time for feedback and to get more experience, they were proposed as a preview feature again by JEP 524 and delivered in JDK 26. This JEP proposes to finalize the PEM API in JDK 27 with the following changes from JDK 25.

      Changes since the first preview:

      Goals

      • Ease of use — Define a concise API that converts between PEM text and objects representing keys, certificates, and certificate revocation lists.

      • Support standards — Support conversions between PEM text and cryptographic objects that have standard representations in the binary formats PKCS#8 (for private keys), X.509 (public keys, certificates, and certificate revocation lists), and PKCS#8 v2.0 (encrypted private keys and asymmetric keys).

      Motivation

      The Java Platform API has rich support for cryptographic objects such as public keys, private keys, certificates, and certificate revocation lists. Developers use these objects to sign and verify signatures, verify network connections secured by TLS, and perform other cryptographic operations.

      Applications often send and receive representations of cryptographic objects, whether via user interfaces, over the network, or to and from storage devices. The Privacy-Enhanced Mail (PEM) format, defined by RFC 7468, is often used for this purpose.

      This textual format was originally designed for sending cryptographic objects via e-mail, but over time it has been used and extended for other purposes. Certificate authorities issue certificate chains in the PEM format. Cryptographic libraries such as OpenSSL provide operations for generating and converting PEM-encoded cryptographic objects. Security-sensitive applications such as OpenSSH store communication keys in the PEM format. Hardware authentication devices such as Yubikeys ingest and dispense PEM-encoded cryptographic objects.

      Here is an example of a PEM-encoded cryptographic object, in this case an elliptic curve public key:

      -----BEGIN PUBLIC KEY-----
      MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEi/kRGOL7wCPTN4KJ2ppeSt5UYB6u
      cPjjuKDtFTXbguOIFDdZ65O/8HTUqS/sVzRF+dg7H3/tkQ/36KdtuADbwQ==
      -----END PUBLIC KEY-----
      

      A PEM text contains a Base64-encoded representation of the key's binary representation surrounded by a header and footer containing the words BEGIN and END, respectively. The remaining text in the header and the footer identifies the type of the cryptographic object, in this case a PUBLIC KEY. Details of the key, such as its algorithm and content, can be obtained by parsing the Base64-encoded binary representation.

      The Java Platform does not include an easy-to-use API for decoding and encoding text in the PEM format. This pain point was validated by the Java Cryptographic Extensions Survey in April 2022. While each cryptographic object provides a method to return its binary-encoded representation, and the Base64 API can be used to convert it to text, the rest of the work is left to developers:

      • Encoding a public key is straightforward, if tedious.

      • Decoding a PEM-encoded key requires careful parsing of the source PEM text, determining the factory to use to create the key object, and determining the key's algorithm.

      • Encrypting and decrypting a private key requires over a dozen lines of code.

      Surely, we can do better.

      Description

      We introduce a new interface and three new classes in the java.security package:

      • The BinaryEncodable interface is implemented by Java Platform API classes representing cryptographic objects with binary-encodable key or certificate material.

      • The PEMEncoder and PEMDecoder classes are for encoding to and decoding from the PEM format. Instances of these classes are immutable and reusable, i.e., they do not retain information from the previously encoded or decoded cryptographic object.

      • The PEM class, which implements BinaryEncodable, is for encoding and decoding PEM text representing cryptographic objects for which no Java Platform API exists.

      Binary-encodable cryptographic objects

      PEM is a textual format for binary data. To encode a cryptographic object into PEM text, or to decode PEM text into a cryptographic object, we need a way to convert such objects to and from binary data. Fortunately, the Java APIs for cryptographic keys, certificates, and certificate revocation lists all provide the means to convert their instances to and from byte arrays in the Distinguished Encoding Rules (DER) format. Unfortunately, these APIs are not hierarchically related, and the manner in which they expose their conversions is not uniform.

      We thus introduce a new interface, BinaryEncodable, to identify the cryptographic APIs that provide such conversions and whose instances can therefore be encoded to, and decoded from, the PEM format. This empty interface is sealed; its permitted classes and interfaces are AsymmetricKey, X509Certificate, X509CRL, KeyPair, EncryptedPrivateKeyInfo, PKCS8EncodedKeySpec, X509EncodedKeySpec, and PEM:

      public sealed interface BinaryEncodable
          permits AsymmetricKey, KeyPair,
                  PKCS8EncodedKeySpec, X509EncodedKeySpec,
                  EncryptedPrivateKeyInfo, X509Certificate, X509CRL, PEM
      { }
      

      We make corresponding adjustments to some of the permitted classes and interfaces:

      public non-sealed interface AsymmetricKey { ... }
      public non-sealed class PKCS8EncodedKeySpec { ... }
      public non-sealed class X509EncodedKeySpec { ... }
      public non-sealed class EncryptedPrivateKeyInfo { ... }
      public non-sealed abstract class X509Certificate { ... }
      public non-sealed abstract class X509CRL { ... }
      

      Encoding

      The PEMEncoder class declares methods for encoding BinaryEncodable objects into PEM text:

      public final class PEMEncoder {
      
          public static PEMEncoder of();
      
          public byte[] encode(BinaryEncodable so);
          public String encodeToString(BinaryEncodable so);
      
          public PEMEncoder withEncryption(char[] password);
      
      }
      

      To encode a BinaryEncodable object, first obtain a PEMEncoder instance by calling of(). The returned instance is thread-safe and reusable, so its encode methods can be used repeatedly.

      There are two methods for encoding. One method returns PEM text in a byte array containing characters encoded in the ISO-8859-1 charset; for example, to encode a private key:

      PEMEncoder pe = PEMEncoder.of();
      byte[] pem = pe.encode(privateKey);
      

      The other encoding method returns PEM text as a string; for example, to encode a public/private key pair into a string:

      String pem = pe.encodeToString(new KeyPair(publicKey, privateKey));
      

      If you are encoding a PrivateKey then you can encrypt it via the withEncryption method, which takes a password and returns a new immutable PEMEncoder instance configured to encrypt the key with that password:

      String pem = pe.withEncryption(password).encodeToString(privateKey);
      

      A PEMEncoder configured in this way can encode PrivateKey, KeyPair, and PKCS8EncodedKeySpec objects. It uses a default encryption algorithm and throws CryptoException on encryption errors. To use non-default encryption parameters, or to encrypt with a different encryption provider, use an EncryptedPrivateKeyInfo object (see below).

      Decoding

      The PEMDecoder class declares methods for decoding PEM text to BinaryEncodable objects:

       public final class PEMDecoder {
      
           public static PEMDecoder of();
      
           public BinaryEncodable decode(String str);
           public BinaryEncodable decode(InputStream is) throws IOException;
           public <S extends BinaryEncodable> S decode(String string, Class<S> cl);
           public <S extends BinaryEncodable> S decode(InputStream is, Class<S> cl)
               throws IOException;
      
           public PEMDecoder withDecryption(char[] password);
           public PEMDecoder withFactory(Provider provider);
      
       }
      

      To decode PEM text, first obtain a PEMDecoder instance by calling of(). The returned instance is thread-safe and reusable, so its decode methods can be used repeatedly.

      There are four methods for decoding; they each return a BinaryEncodable object. You can use pattern matching with the instanceof operator or a switch statement to identify the type of cryptographic object returned. For example, to decode PEM text that you expect to encode either a public key or a private key:

      PEMDecoder pd = PEMDecoder.of();
      switch (pd.decode(pem)) {
          case PublicKey publicKey -> ...;
          case PrivateKey privateKey -> ...;
          default -> throw new IllegalArgumentException(...);
      }
      

      If you know the type of the encoded cryptographic object in advance then you can pass the corresponding class to one of the decode methods that takes a Class argument, avoiding the need to pattern-match on, or else check and then cast to, the type of the method's result. For example, if you know that the type is ECPublicKey:

      ECPublicKey key = pd.decode(pem, ECPublicKey.class);
      

      With these methods, if the class is incorrect then a ClassCastException is thrown.

      If the input PEM text encodes an encrypted private key then you can decrypt it via the withDecryption method, which takes a password and returns a new PEMDecoder instance configured to decrypt the key into a PrivateKey object. A PEMDecoder configured in this way can still decode unencrypted objects. For example, to decrypt an ECPrivateKey:

      ECPrivateKey eckey = pd.withDecryption(password)
                             .decode(pem, ECPrivateKey.class);
      

      If you decode PEM text that encodes a private key, but do not provide a password, then the decode methods return an EncryptedPrivateKeyInfo instance which can be used to decrypt and produce a PrivateKey object (see below).

      In some situations, you may need to use a specific cryptographic provider when decoding PEM text. The withFactory method returns a new PEMDecoder instance that uses the specified provider to produce cryptographic objects. For example, to decode a Certificate with a specific provider:

      PEMDecoder d = pd.withFactory(providerFactory);
      Certificate c = d.decode(pem, X509Certificate.class);
      

      If the provider cannot produce the required cryptographic object type, an IllegalArgumentException is thrown.

      When decoding PEM text to a cryptographic object, any data preceding the PEM header in the input string or byte stream is ignored. If you need that data, you can obtain it by decoding to a PEM object.

      An llegalArgumentException is thrown if the PEM input cannot be parsed. Decryption failures result in a CryptoException. Bytes read from input streams are interpreted as ISO-8859-1 charset.

      The PEM class

      The PEM class implements BinaryEncodable. Its instances can hold any type of PEM data. It thus enables you to encode and decode PEM representing cryptographic objects for which no Java Platform API exists such as, e.g., PKCS#10 certification requests.

      public class PEM implements BinaryEncodable
      {
          public PEM(String type, String base64Content);
          public PEM(String type, String base64Content, byte[] leadingData)
          public PEM(String type, byte[] binaryContent);
          public PEM(String type, byte[] binaryContent, byte[] leadingData)
          String type();           // Cryptographic object type, from the header text
                                   // (e.g., "PRIVATE KEY")
          byte[] content();        // binary-encoded content
          byte[] leadingData();    // Any content preceding the PEM header
      }
      

      A PEMDecoder instance decodes PEM text into a PEM object when there is no Java Platform API for the text’s PEM type:

      BinaryEncodable d = PEMDecoder.of().decode(pem);
      if (d instanceof PEM pr) {
          throw new IllegalArgumentException("Unhandled PEM type: " + pr.type()
                                             + "; data: " + pr.content());
      }
      

      <a id="decode-to-pemrecord"/> If you need access to the leading data of a PEM text, or if you want to handle the text’s content yourself, you can specifically request a PEM when decoding:

      PEM pr = PEMDecoder.of().decode(pem, PEM.class);
      

      A PEMEncoder instance encodes a PEM object into PEM text without validating its content.

      The EncryptedPrivateKeyInfo class

      The existing EncryptedPrivateKeyInfo class represents an encrypted private key. To make it easier to use with the PEMEncoder and PEMDecoder classes, we have added seven methods to it:

       EncryptedPrivateKeyInfo {
      
           ...
      
           public static EncryptedPrivateKeyInfo
               encrypt(BinaryEncodable key, char[] password);
           public static EncryptedPrivateKeyInfo
               encrypt(BinaryEncodable key, char[] password,
                          String algorithm, AlgorithmParameterSpec params,
                          Provider provider);
           public static EncryptedPrivateKeyInfo 
               encrypt(BinaryEncodable key, Key encKey,
                          String algorithm, AlgorithmParameterSpec params,
                          Provider provider, SecureRandom random);
      
           public PrivateKey getKey(char[] password) 
               throws NoSuchAlgorithmException, InvalidKeyException;
           public PrivateKey getKey(Key decryptKey)
               throws NoSuchAlgorithmException, InvalidKeyException;
      
           public KeyPair getKeyPair(char password) 
               throws NoSuchAlgorithmException, InvalidKeyException;
           public KeyPair getKeyPair(Key decryptKey)
               throws NoSuchAlgorithmException, InvalidKeyException;
      
       }
      

      The three new static encrypt methods encrypt the given BinaryEncodable with the given password. The BinaryEncodable must be a PrivateKey, KeyPair, or a PKCS8EncodedKeySpec. For advanced usage, the second and third encrypt methods allow additional cryptographic parameters to be specified if the defaults are not sufficient. The returned EncryptedPrivateKeyInfo instance can then be passed to a PEMEncoder to encode to PEM text:

      var epki = EncryptedPrivateKeyInfo.encryptKey(privateKey, password);
      byte[] pem = PEMEncoder.of().encode(epki);
      

      The new getKey methods decrypt the private key from an EncryptedPrivateKeyInfo instance, accepting either a password or a Key and returning a PrivateKey:

      EncryptedPrivateKeyInfo epki = PEMDecoder.of().decode(pem);
      PrivateKey key = epki.getKey(password);
      

      The new getKeyPair methods decrypt an EncryptedPrivateKeyInfo instance into a KeyPair if the encoding contains both a public and a private key. If a public key is not present, an IllegalArgumentException is thrown.

      The default password-based encryption (PBE) algorithm used when encrypting a BinaryEncodable with either PEMEncoder or EncryptedPrivateKeyInfo is defined in the default security properties file. The jdk.epkcs8.defaultAlgorithm security property defines the default algorithm to be "PBEWithHmacSHA256AndAES_128". The default algorithm might change in the future, but this will not affect PEM text created today since the data encoded in that text contains the algorithm name and all other parameters necessary for decryption.

      The CryptoException class

      This exception represents a general cryptographic error that occurs during processing. It is intended for unrecoverable failures related to GeneralSecurityException in contexts where checked exceptions are not desired.

      Alternatives

      A PEM API is a bridge between Base64 and cryptographic objects. We rejected many other potential designs because they did not fit well with the existing cryptographic APIs. While some of the alternatives might have been adequate, we chose the proposed API for its similarity to the HexFormat API and the nested Encoder and Decoder classes of the Base64 API. We wanted to have immutability, thread safety, and distinct paths through the API for encoding and decoding.

      Some of the alternatives we considered include:

      • Extend the EncodedKeySpec API — This API encapsulates binary-encoded key data for KeyFactory instances and other cryptographic classes. A new PEMEncodedKeySpec subclass could type-identify encapsulated PEM text while providing encoding and decoding operations between PEM text and the appropriate private or public key EncodedKeySpec.

        This design had several deficiencies. First, the PEMEncodedKeySpec class would be used for translation, which is not the purpose of its superclass, EncodedKeySpec. Second, EncodedKeySpec is key-centric and thus cannot support the encoding of certificates or certificate revocation lists to PEM text. Finally, a new EncodedKeySpec subclass would raise compatibility risks and ease-of-use issues with existing third-party cryptographic providers.

      • Enhance the CertificateFactory and KeyFactory APIs — The CertificateFactory API already supports the decoding of PEM certificate and certificate revocation list data, so adding encoding methods to CertificateFactory and KeyFactory would be consistent with the existing design.

        CertificateFactory makes this approach look easy, since there is one industry standard encoding for certificates. KeyFactory, by contrast, would have to support different encoding formats. To make matters worse, providers of KeyFactory instances need not support all known types of asymmetric keys. Provider maintainers, moreover, may balk at having to be responsible for PEM encoding and also for handling encrypted private keys. This makes enhancing KeyFactory a difficult solution.

      • Static methods — Static methods are good for immutability and thread safety, but encrypted private keys present a usability issue. Conversions of encrypted private keys require a password, while conversions of other types of cryptographic objects do not. Thus, with static methods, for encrypted private keys we would need unpleasant solutions such as additional overloaded methods that take encryption parameters, or the mandatory use of EncryptedPrivateKeyInfo instances. Having encoders and decoders store the encryption password results in a better user experience.

      • Intermediate PEM object API — We could introduce a wrapper class whose instances would contain a key, a certificate, a certificate revocation list, or some PEM text. This class could either declare encoding and decoding methods, or a separate API could perform operations on its instances.

        While this approach would provide an independent representation of PEM text, too much flexibility is a negative. A class whose instances can wrap both cryptographic objects and PEM text could be confusing, since these are fundamentally two different kinds of things. Having distinct encoding and decoding paths from given data provides a more guided user experience.

      • Single-class API — A single PEM class could perform both encoding and decoding, but, as with the static-method and intermediate-PEM-object approaches, it would lack distinct operational paths for encoding and decoding. Separating encoding and decoding into their own classes results in an API that is easier to use since each class presents just the operations needed.

      • Create cryptographic provider service support for encodings — We considered enabling cryptographic providers to support services for converting between textual and binary representations of cryptographic objects. Binary formats are already used internally by providers for importing and exporting cryptographic objects, and adding conversion services could be useful beyond PEM.

        This approach would, however, require a lot of infrastructure but result in little additional service. At the same time, it would put compatibility at risk for existing providers and complicate the use of the API.

      • Introduce a general-purpose cryptographic encoding and decoding API — We considered a generic API that could be used with many textual formats. We rejected this because these formats do not all have the same features, differing in their support for keys, certificate chains, compression, and other options. Many formats in one API, where some methods are format-specific, would be confusing.

      Testing

      Tests will include:

      • Verifying that all supported BinaryEncodable classes can encode and decode PEM;
      • Verifying that RSA, EC, ML-KEM, and EdDSA cryptographic objects can be encoded and decoded;
      • Reading PEM text generated by third-party applications and vice versa; and
      • Negative testing with bad PEM text.

            Assignee:
            Anthony Scarpino
            Reporter:
            Anthony Scarpino
            Anthony Scarpino Anthony Scarpino
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: