[testbug] Some unit tests call System.exit

Calling System.exit from a JUnit test is an anti-pattern that will cause the test to exit abruptly in a way that will interfere with correct reporting. For system tests, it can cause that test to either be silently skipped or report confusing results. For tests in other modules, which can run several tests in the same VM, it will cause the entire rest of the test suite to be skipped.

I have attached a simple example test. If you drop that in:

tests/system/src/test/java/test/SystemExitTest.java

you will see the problems that can arise.

If you run all of :systemTests:test then, the test won't show up in the report and it will look like the whole test suite passes! If you run just that one test, as follows, you will get a confusing error message that says there are no tests to run:

$ gradle --info -PFULL_TEST=true cleanTest :systemTests:test --tests test.SystemExitTest
...
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':systemTests:test'.
> No tests found for given includes: [**/*.*](include rules) [test/robot/**, test/robot/com/sun/glass/ui/monocle/**](exclude rules) [test.SystemExitTest](--tests filter)


I grepped through our sources, and have found the following two lists of classes that call System.exit.

1. The following classes call System.exit and do not need any action:

tests/system/src/test/java/test/javafx/scene/shape/meshmanagercacheleaktest/MeshManagerCacheLeakApp.java
tests/system/src/test/java/test/shutdowntest/ShutdownHookApp.java
tests/system/src/test/java/test/javafx/css/imagecacheleaktest/ImageCacheLeakApp.java
tests/system/src/test/java/test/sandbox/app/*.java
tests/system/src/test/java/test/launchertest/Test*.java
tests/system/src/test/java/test/launchertest/PlatformExitApp.java

These classes are not themselves JUnit tests, but are applications that are forked/execed by a Junit test, so the call to System.exit is expected.


2. The following classes call System.exit and need some remediation:

modules/javafx.fxml/src/test/java/test/com/oracle/javafx/fxml/test/TestLoadPerformance.java
ACTION: delete this or else move it to a directory under tests/performace; it is not a unit test and is never executed

modules/javafx.graphics/src/test/java/test/com/sun/javafx/iio/ImageStorageTest.java.DISABLED
ACTION: delete this file

modules/javafx.graphics/src/test/java/test/com/sun/javafx/pgstub/StubToolkit.java
ACTION: change the System.exit to throw UnsupportedOperationException (it would fail the entire test suite if any test ever called that stub method). This one might warrant adding a test in javafx.graphics to verify it.

tests/system/src/test/java/test/com/sun/javafx/application/StaticStartupTest.java
ACTION: fix this test to throw an error in the case where it currently calls System.exit

tests/system/src/test/java/test/com/sun/javafx/image/impl/ImageRaceTest.java
ACTION: change the System.exit to Assert.fail

tests/system/src/test/java/test/javafx/embed/swing/JFXPanelEmbeddedWindowTest.java
tests/system/src/test/java/test/robot/javafx/embed/swing/JFXPanelTest.java
ACTION: remove the call, possibly by removing the main method; it is never executed by JUnit and is not needed