SA core file tests sometimes time out on OSX with "java.io.IOException: App waiting timeout"

The following test failed in the JDK16 CI:

serviceability/sa/ClhsdbFindPC.java#id3
serviceability/sa/ClhsdbFindPC.java#id1

Both tests report a SIGSEGV in Java_jdk_test_lib_apps_LingeredApp_crash
but I think that crash is intentionally caused by the test.

Here's a snippet from the ClhsdbFindPC.java#id3 log file:

#section:main
----------messages:(4/189)----------
command: main ClhsdbFindPC false true
reason: User specified action: run main/othervm/timeout=480 ClhsdbFindPC false true
Mode: othervm [/othervm specified]
elapsed time (seconds): 445.85
----------configuration:(0/0)----------
----------System.out:(9/1828)----------
Starting the ClhsdbFindPC test
Command line: ['/scratch/mesos/jib-master/install/jdk-16+6-168/macosx-x64-debug.jdk/jdk-16/fastdebug/bin/java' '-XX:MaxRAMPercentage=12' '-Djava.io.tmpdir=/scratch/mesos/slaves/52628e90-e5e7-4ef3-af97-10d8776d10db-S992/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/1ca523ef-9cef-4c75-97f6-121eaacfa189/runs/130452c2-8592-4adc-9789-a0fa066e52a0/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/tmp' '-Xint' '-XX:+CreateCoredumpOnCrash' '-Djava.library.path=/scratch/mesos/jib-master/install/jdk-16+6-168/macosx-x64-debug.test/hotspot/jtreg/native' '-cp' '/scratch/mesos/slaves/52628e90-e5e7-4ef3-af97-10d8776d10db-S992/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/1ca523ef-9cef-4c75-97f6-121eaacfa189/runs/130452c2-8592-4adc-9789-a0fa066e52a0/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/0/serviceability/sa/ClhsdbFindPC_id3.d:/scratch/mesos/slaves/52628e90-e5e7-4ef3-af97-10d8776d10db-S992/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/1ca523ef-9cef-4c75-97f6-121eaacfa189/runs/130452c2-8592-4adc-9789-a0fa066e52a0/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/0/test/lib' 'LingeredAppWithTrivialMain' '3bf7de6b-8edf-411d-8bf7-7014944a9bd0.lck' 'forceCrash']
[2020-07-09T08:13:08.986673Z] Gathering output for process 13944
[2020-07-09T08:13:09.053042Z] Waiting for completion for process 13944
[2020-07-09T08:13:09.053225Z] Waiting for completion finished for process 13944
Output and diagnostic info for process 13944 was saved into 'pid-13944-output.log'
[2020-07-09T08:13:09.247708Z] Waiting for completion for process 13944
[2020-07-09T08:13:09.247843Z] Waiting for completion finished for process 13944
Run test with ulimit -c: unlimited
----------System.err:(21/1339)----------
java.lang.RuntimeException: Test ERROR java.io.IOException: App waiting timeout
at ClhsdbFindPC.testFindPC(ClhsdbFindPC.java:155)
at ClhsdbFindPC.main(ClhsdbFindPC.java:167)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.io.IOException: App waiting timeout
at jdk.test.lib.apps.LingeredApp.waitAppReady(LingeredApp.java:264)
at jdk.test.lib.apps.LingeredApp.waitAppReady(LingeredApp.java:294)
at jdk.test.lib.apps.LingeredApp.startAppExactJvmOpts(LingeredApp.java:416)
at jdk.test.lib.apps.LingeredApp.startApp(LingeredApp.java:431)
at ClhsdbFindPC.testFindPC(ClhsdbFindPC.java:90)
... 7 more

JavaTest Message: Test threw exception: java.lang.RuntimeException: Test ERROR java.io.IOException: App waiting timeout
JavaTest Message: shutting down test

STATUS:Failed.`main' threw exception: java.lang.RuntimeException: Test ERROR java.io.IOException: App waiting timeout
----------rerun:(33/6540)*----------


Here's a snippet from the ClhsdbFindPC.java#id1 log file:

#section:main
----------messages:(4/188)----------
command: main ClhsdbFindPC true true
reason: User specified action: run main/othervm/timeout=480 ClhsdbFindPC true true
Mode: othervm [/othervm specified]
elapsed time (seconds): 457.343
----------configuration:(0/0)----------
----------System.out:(9/1829)----------
Starting the ClhsdbFindPC test
Command line: ['/scratch/mesos/jib-master/install/jdk-16+6-168/macosx-x64-debug.jdk/jdk-16/fastdebug/bin/java' '-XX:MaxRAMPercentage=12' '-Djava.io.tmpdir=/scratch/mesos/slaves/52628e90-e5e7-4ef3-af97-10d8776d10db-S992/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/1ca523ef-9cef-4c75-97f6-121eaacfa189/runs/130452c2-8592-4adc-9789-a0fa066e52a0/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/tmp' '-Xcomp' '-XX:+CreateCoredumpOnCrash' '-Djava.library.path=/scratch/mesos/jib-master/install/jdk-16+6-168/macosx-x64-debug.test/hotspot/jtreg/native' '-cp' '/scratch/mesos/slaves/52628e90-e5e7-4ef3-af97-10d8776d10db-S992/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/1ca523ef-9cef-4c75-97f6-121eaacfa189/runs/130452c2-8592-4adc-9789-a0fa066e52a0/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/1/serviceability/sa/ClhsdbFindPC_id1.d:/scratch/mesos/slaves/52628e90-e5e7-4ef3-af97-10d8776d10db-S992/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/1ca523ef-9cef-4c75-97f6-121eaacfa189/runs/130452c2-8592-4adc-9789-a0fa066e52a0/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/1/test/lib' 'LingeredAppWithTrivialMain' '87856dbf-33fd-454d-b89e-01435a05e80c.lck' 'forceCrash']
[2020-07-09T08:12:57.448369Z] Gathering output for process 13928
[2020-07-09T08:12:57.580371Z] Waiting for completion for process 13928
[2020-07-09T08:12:57.580672Z] Waiting for completion finished for process 13928
Output and diagnostic info for process 13928 was saved into 'pid-13928-output.log'
[2020-07-09T08:12:57.657282Z] Waiting for completion for process 13928
[2020-07-09T08:12:57.657499Z] Waiting for completion finished for process 13928
Run test with ulimit -c: unlimited
----------System.err:(21/1339)----------
java.lang.RuntimeException: Test ERROR java.io.IOException: App waiting timeout
at ClhsdbFindPC.testFindPC(ClhsdbFindPC.java:155)
at ClhsdbFindPC.main(ClhsdbFindPC.java:167)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.io.IOException: App waiting timeout
at jdk.test.lib.apps.LingeredApp.waitAppReady(LingeredApp.java:264)
at jdk.test.lib.apps.LingeredApp.waitAppReady(LingeredApp.java:294)
at jdk.test.lib.apps.LingeredApp.startAppExactJvmOpts(LingeredApp.java:416)
at jdk.test.lib.apps.LingeredApp.startApp(LingeredApp.java:431)
at ClhsdbFindPC.testFindPC(ClhsdbFindPC.java:88)
... 7 more

JavaTest Message: Test threw exception: java.lang.RuntimeException: Test ERROR java.io.IOException: App waiting timeout
JavaTest Message: shutting down test

STATUS:Failed.`main' threw exception: java.lang.RuntimeException: Test ERROR java.io.IOException: App waiting timeout
----------rerun:(33/6541)*----------

A curious observation: both tests specify /timeout=480
and both tests timeout before 480 seconds. That's weird
and that timeout value should have been multiplied by
the timeoutFactor which is usually 4.