Consider the following test:
/*
* @test
* @run testng/othervm -Ddataproviderthreadcount=4 ParallelTestNg
*/
public class ParallelTestNg {
@DataProvider(parallel = true)
public static Iterator<Object[]> integers() {
List<Object[]> args = new LinkedList<>();
for (int i = 0; i < 10; i++) {
args.add(new Object[]{i});
}
return args.iterator();
}
@Test(dataProvider = "integers")
public void test(Integer arg) throws InterruptedException {
TimeUnit.MILLISECONDS.sleep(100);
fail(String.valueOf(arg));
}
}
"dataproviderthreadcount=4" passed to TestNG in conjunction with "parallel =
true" element passed to @DataProvider annotation instructs TestNG to dispatch
invocations of "test" to a thread pool of size 4.
This was designed to speed up executions of tests with data supplied from data
providers.
However, when this test is run (jtreg -verbose:all ... ), it reports different
numbers of tests failed each time:
TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.Exception: failures: 6
or
TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.Exception: failures: 9
etc. In the same output TestNG consistently reports the same correct number:
===============================================
ParallelTestNg.java
Total tests run: 10, Failures: 10, Skips: 0
===============================================
/*
* @test
* @run testng/othervm -Ddataproviderthreadcount=4 ParallelTestNg
*/
public class ParallelTestNg {
@DataProvider(parallel = true)
public static Iterator<Object[]> integers() {
List<Object[]> args = new LinkedList<>();
for (int i = 0; i < 10; i++) {
args.add(new Object[]{i});
}
return args.iterator();
}
@Test(dataProvider = "integers")
public void test(Integer arg) throws InterruptedException {
TimeUnit.MILLISECONDS.sleep(100);
fail(String.valueOf(arg));
}
}
"dataproviderthreadcount=4" passed to TestNG in conjunction with "parallel =
true" element passed to @DataProvider annotation instructs TestNG to dispatch
invocations of "test" to a thread pool of size 4.
This was designed to speed up executions of tests with data supplied from data
providers.
However, when this test is run (jtreg -verbose:all ... ), it reports different
numbers of tests failed each time:
TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.Exception: failures: 6
or
TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.Exception: failures: 9
etc. In the same output TestNG consistently reports the same correct number:
===============================================
ParallelTestNg.java
Total tests run: 10, Failures: 10, Skips: 0
===============================================
- links to
-
Commit(master) openjdk/jtreg/3750f096
-
Review(master) openjdk/jtreg/216