ADDITIONAL SYSTEM INFORMATION :
OS: Windows 10 64 bit, but it fail on Linux too

java version "1.8.0_301" Java(TM) SE Runtime Environment (build 1.8.0_301-b09) Java HotSpot(TM) 64-Bit Server VM (build 25.301-b09, mixed mode)


A DESCRIPTION OF THE PROBLEM :
I have a sample page sample.html

and a regex pattern

<html.*(?:</html|</body)

For java 1.8.0_301 and a lot of older versions:

java -version

I get:

java.lang.StackOverflowError
at java.util.regex.Pattern$Curly.match0(Pattern.java:4264)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
The source code and the sample html page
## compile
`mvn clean compile`

## run
`mvn exec:java`

## output
`java.lang.StackOverflowError
        at java.util.regex.Pattern$CharProperty.match(Pattern.java:3790)
        at java.util.regex.Pattern$Curly.match0(Pattern.java:4264)
        at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
        at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
        at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
        at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
        at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
        at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
        at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)`

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
No exception printed.
ACTUAL -
java.lang.StackOverflowError
at java.util.regex.Pattern$Curly.match0(Pattern.java:4264)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4277)

---------- BEGIN SOURCE ----------
import java.io.IOException;
import java.net.URISyntaxException;
import java.net.URL;
import java.nio.file.Paths;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import org.apache.commons.io.FileUtils;

public class Sample
{
  public static void main(String[] args) throws IOException, URISyntaxException
  {
    URL resource = Sample.class.getClassLoader().getResource("sample.html");
    String content = FileUtils.readFileToString(Paths.get(resource.toURI()).toFile(), "utf-8");

    String regEx = "<html.*(?:</html|</body)";

    Pattern pattern = Pattern.compile(regEx, Pattern.CASE_INSENSITIVE | Pattern.DOTALL);
    Matcher matcher = pattern.matcher(content);
    try
    {
      matcher.find();
    }
    catch (Throwable t)
    {
      t.printStackTrace();
    }
  }
}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
changed the regex to: <html.*?(?:</html|</body)