Details
-
Enhancement
-
Resolution: Unresolved
-
P3
-
None
-
None
-
None
Description
There are may occasions - eg fastdebug builds testing - where only crashes is what tester is interest in.
In current implementation of jtreg/jtharness it is left on downstream user to post process results, and to conisder as failed tests only those who failed, and crashed.
jtreg/jtharness should contain and tool chain, where the jvm crashes will be considered, before reporting.
The basic use case shoudl some --crashesOnly switch, which will switch jtreg/jtharness to mode, where only thest which failed, and crashed together will be reported as pass or skipped (configurable?). Another configurable part may be to consider passed test with crash as failure, as although there are many tests intentionally crashing jvm and passing, some other tests with crash may be hidden false positives.
It is not a goal of this bug track implemention of this on junit/testng level, but an attempt to implement similar feature is not an bad idea.
In current implementation of jtreg/jtharness it is left on downstream user to post process results, and to conisder as failed tests only those who failed, and crashed.
jtreg/jtharness should contain and tool chain, where the jvm crashes will be considered, before reporting.
The basic use case shoudl some --crashesOnly switch, which will switch jtreg/jtharness to mode, where only thest which failed, and crashed together will be reported as pass or skipped (configurable?). Another configurable part may be to consider passed test with crash as failure, as although there are many tests intentionally crashing jvm and passing, some other tests with crash may be hidden false positives.
It is not a goal of this bug track implemention of this on junit/testng level, but an attempt to implement similar feature is not an bad idea.
Attachments
Issue Links
- links to
-
Review openjdk/jtreg/164