Some while ago, the top level test-make target was refactored into the RunTest.gmk framework. I understand the rationale behind this, but it had some rather server consequences.

1. RunTest.gmk does not return a failure if a test fails. Since test-make is considered a "build" task in our distributed testing framework, there is no test parsing happening. A build failure is expected to return non zero. This means we would not see failures in CI runs of testmake today.

2. The "special" tests in RunTest.gmk do not have any results parsing, so it's completely hidden from the user even running locally if a make test fails unless the build log is carefully inspected.

3. Since more makefile bootstrapping and setting up is run through before even getting to the tests, it's harder to experiment with basic make functionality and run the tests as unit tests while developing.

I just hit all of these pretty hard while investigating JDK-8237879. I would like all 3 to be addressed:

1. An option to fail on test failure is needed in RunTest.gmk.

2. We should add a very rudimentary parsing of results for test-make, so that it at least shows that there is 1 failure if EXITCODE is non zero.

3. I would like to readd a basic "test-make-raw" target that just plainly calls TestMake.gmk like before.