This has arisen out of CODETOOLS-7902065. This is the root problem for that issue, whereas the issue itself describes a suggested solution.

CachingTestFilter uses a HashTable<TestDescription,Boolean> to cache the results of a filter. This caches based on the *content* of the test description, and not the location of the test within the test suite. While this is OK for most filters, it means that filters based on the location of the test (specifically, ExcludeListFilter) will not be cached correctly.

Although it will not cache as effectively, it would be better to cache results based on the location of the test, i.e. its rootRelativeURL, or use an IdentityHashMap.