-
Enhancement
-
Resolution: Unresolved
-
P4
-
21
We are using more and more IR tests. That is good, we get better coverage and regression tests.
However, the IR tests are significantly slower than non-IR tests.
Currently, the IR framework goes through the tests linearly, scenario by scenario. We can parallelise this.
A first step would be to run each scenario in a separate TestVM, and run all scenarios concurrently. This should work without modifying the tests.
A more fine-grained level of parallelism could be reached by parallelizing at the @Test level. However, this requires independence between the tests - it should not introduce race-conditions. Maybe this can be disabled by default, and enabled for tests that do not have any shared resources.
However, the IR tests are significantly slower than non-IR tests.
Currently, the IR framework goes through the tests linearly, scenario by scenario. We can parallelise this.
A first step would be to run each scenario in a separate TestVM, and run all scenarios concurrently. This should work without modifying the tests.
A more fine-grained level of parallelism could be reached by parallelizing at the @Test level. However, this requires independence between the tests - it should not introduce race-conditions. Maybe this can be disabled by default, and enabled for tests that do not have any shared resources.