Recently there were some odd failures with some tests throwing IllegalAccessErrors running "tier1 linux-x64" tests on Mach5 production. I find these errors odd in that they don't happen very often and looking at the tests, they shouldn't happen at all.

Taking a look at the jtr file , it failed with with this IllegalAccessError which shouldn't seem possible

java.lang.IllegalAccessError: tried to access class Foo from class BoxedForeach
at BoxedForeach.foolist(BoxedForeach.java:83)
at BoxedForeach.f(BoxedForeach.java:76)
at BoxedForeach.main(BoxedForeach.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:502)
at com.sun.javatest.regtest.MainAction$SameVMRunnable.run(MainAction.java:776)
at java.lang.Thread.run(Thread.java:745)
since Foo is defined in BoxedForeach.java

I notice in the jtr file the classpath for the run is odd. I think normally it would have javatest.jar:jtreg.jar:JTworkclasses:testsrcdir.
Instead it has javatest.jar:jtreg.jar:jtreg.jar::jtreg.jar:JTworkclasses:testsrcdir
That's 3 jtreg.jar files and an empty classpath element.

-classpath /scratch/jenkins/workspace/9-dev-linux-openonly-build/jdkbuildwrapper/.data/unpacked/jpg/infra/builddeps/jtreg-4.1/1.0/jtreg/lib/javatest.jar:
/scratch/jenkins/workspace/9-dev-linux-openonly-build/jdkbuildwrapper/.data/unpacked/jpg/infra/builddeps/jtreg-4.1/1.0/jtreg/lib/jtreg.jar:
/scratch/jenkins/workspace/9-dev-linux-openonly-build/jdkbuildwrapper/.data/unpacked/jpg/infra/builddeps/jtreg-4.1/1.0/jtreg/lib/jtreg.jar::
/scratch/jenkins/workspace/9-dev-linux-openonly-build/jdkbuildwrapper/.data/unpacked/jpg/infra/builddeps/jtreg-4.1/1.0/jtreg/lib/jtreg.jar:
/scratch/jenkins/workspace/9-dev-linux-openonly-build/JTwork/langtools_test/classes/17/tools/javac/boxing:
/scratch/jenkins/workspace/9-dev-linux-openonly-build/langtools/test/tools/javac/boxing \\

This run had several of these types of failures.
Build job: http://java.se.oracle.com/mach5/job/9-dev-linux-openonly-build/709/
-->Test job: http://java.se.oracle.com/mach5/view/9-dev-test/job/9-dev-tier1-linux-x64-openonly/324/
   -->history listing: http://java.se.oracle.com/mach5/view/9-dev-test/job/9-dev-tier1-linux-x64-openonly-failuresHistory/Failure_History/ (see the 6 $324 failures)

There is no evidence of an invocation problem by jenkins and the initial command looks correct. Is it possible in the reuse of a vm the env is not clean?