JDK-8222737 introduced tier1-like testing to 8u. It does report non-zero exit code on test failures.

Unfortunately, that is only because the summary_tier1 target specifically aggregates the per-suite test reports. So, "make test" does not exit with non-zero exit code, when a suite is run separately. This affects (jdk|langtools|hotspot)_tier1 suites and other predefined test groups. It is easy to demonstrate by defining the jdk_adhoc with known-failing test:

$ head jdk/test/TEST.groups
jdk_adhoc = \
    sun/text/IntHashtable/Bug4170614Test.sh

$ make test TEST=jdk_adhoc
...
FAILED: sun/text/IntHashtable/Bug4170614Test.sh
TEST STATS: name=jdk_adhoc run=1 pass=0 fail=1
EXIT CODE: 2
EXIT CODE: 2
make[3]: *** [Makefile:310: jtreg_tests] Error 2
make[3]: Leaving directory '/home/shade/trunks/jdk8u-dev/jdk/test'
make[2]: *** [Makefile:261: jdk_adhoc] Error 2
make[2]: Leaving directory '/home/shade/trunks/jdk8u-dev/jdk/test'
make[1]: *** [Makefile:80: jdk_adhoc] Error 2
## Finished test (build time 00:00:05)

$ echo $?
0

So, CIs and scripts that look for that non-zero exit code would happily be green even when tests are failing.

I believe that is because the added code looks at the wrong path for exitcode.txt:
https://hg.openjdk.java.net/jdk8u/jdk8u/rev/ef851705e3e1#l2.7

There are actually two path patterns for exitcode.txt:

$ find | grep exitcode
./build/linux-x86_64-normal-server-fastdebug/testoutput/jdk_adhoc/exitcode.txt
./build/linux-x86_64-normal-server-fastdebug/testoutput/tier1_exitcode.txt
./build/linux-x86_64-normal-server-fastdebug/testoutput/langtools_tier1/exitcode.txt
./build/linux-x86_64-normal-server-fastdebug/testoutput/hotspot_tier1/exitcode.txt
./build/linux-x86_64-normal-server-fastdebug/testoutput/jdk_tier1/exitcode.txt

Existing pattern -- ${TEST}_exitcode.txt -- seems to look for tier1_exitcode.txt, which is fine for complete tier1 runs.
Individual suites store exit codes in ${TEST}/exitcode.txt -- and so non-tier1 runs still omit the exit code.