Support endpoint identification algorithm in RFC 6125

Summary

Make the JDK TLS implementation fully compliant with RFC 6125.

Problem

RFC 6125 [1] "specifies recommended procedures for representing and verifying application service identity in certificates intended for use in application protocols employing TLS." For the most part, we are fully compliant except for one case involving wildcards [2]:

1. The client SHOULD NOT attempt to match a presented identifier in which the wildcard character comprises a label other than the left-most label (e.g., do not match bar.*.example.net).

However, this is inconsistent with RFC 2818 [3] which allows this behavior:

Names may contain the wildcard character * which is considered to match
any single domain name component or component fragment. E.g., *.a.com
matches foo.a.com but not bar.foo.a.com. f*.com matches foo.com but not
bar.com.

Currently, we also allow wildcards to be in components other than the left-most component.

It should be rare and in general is not good practice to allow a wildcard in a non-leftmost component. Also worth noting is that the CABForum Baseline Requirements [4] only allows a wildcard character in the left most component of a domain name (see the definition of a Wildcard Domain Name). In general, our implementation has deferred to RFC 6125 as it provides a more secure and robust set of rules for domain names in certificates.

[1] https://datatracker.ietf.org/doc/html/rfc6125#section-1.1

[2] https://datatracker.ietf.org/doc/html/rfc6125#section-6.4.3

[3] https://datatracker.ietf.org/doc/html/rfc2818#section-3.1

[4] https://cabforum.org/wp-content/uploads/CA-Browser-Forum-BR-1.8.0.pdf

Solution

Modify the implementation to be RFC 6125 compliant by disallowing wildcard characters other than in the leftmost component of the domain name.

Specification

N/A.