I have a test that crashes but the hs_err_pid.log file is gone. They should be retained by default.

It appears that all the jtreg tests are run in JTwork/scratch directory so you shouldn't keep the hs_err_pid.log files there. If every test created it's own scratch directory, like for test xxx then remove the artifacts of the test if it passes and retain the artifacts of the failed test, then it would be easy to find and analyze the results. The error report from hotspot has the hs_err file location which is handy so it would be nice if it doesn't move.