Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8212028

Use run-test makefile framework for testing in Oracle's Mach5



    • b16



        In order to fully adopt the new run-test framework, we need to switch over the automated and distributed testing system at Oracle to the new framework.

        To get this to work, there are number of issues that need to be fixed in run-test:

        * Need JAVA_OPTIONS parameter in addition VM_OPTIONS on global level.
        * The current TIMEOUT option is better named TIMEOUT_FACTOR as that is what it is used as.
        * The default number of test jobs need to be consistently calculated, the same as the old test makefiles. The calculated number also needs to be modifiable by factors, from different sources.
        * The default setting of test specific parameters from the TEST_OPTS parameters need to happen even if the TEST_OPTS parameters are set like TEST_OPTS_TIMEOUT_FACTOR=. Combining options into a single TEST_OPTS=OPTION1=,OPTION2= is just inconvenient for Mach5.
        * Need support for extra problemlists for jtreg
        * Langtools default memory size needs to be 768m
        * The ignore keyword is set by default but really not needed. Lets not set default keywords.
        * When setting keywords, since those strings contain shell sensitive characters (! &), try to handle quoting of them as flexible as possible.

        For run-test-prebuilt target specifically
        * Create the OUTPUTDIR if not present.
        * Provide a fallback if NUM_CORES could not be determined.
        * On Solaris use gnu tools

        * For greater flexibility, try to detect if running in a full source dir or in a src.conf dir and only require the full source to be downloaded in the latter case. This makes it much simpler to run tests using Jib locally.

        Test timeouts
        The current default timeoutFactor in the makefiles is 4. However, the old Mach5 executor overrides that to 10. I don't think it should dabble with such things and leave it to the makefiles, so with the new run-test executor, it no longer does. This means many tests now have a much shorter effective timeout. Because of this, we need to increase the timeout on some that are now prone to timing out. I have run tier1-5 a few times to try and find these and added /timeout=300 (which will result in the same effective timeout as before) when specific tests seemed problematic.

        This test spawns a child process and tries to locate it using the attach api, by looking for a unique token in the command line string of the spawned JVM. The problem is that the command line string it gets from the attach api is truncated and the token is last on the command line. This normally works well, but the arguments before it are 3 files, with full absolute paths inside the jtreg work directory. With Mach5 we have pretty deep work directories, and with run-test, we make them even deeper. This unfortunately trips the limit and the test fails. I have fixed this by reordering the arguments to the child process.

        Work is ongoing in the jdk-sandbox repo, branch ihse-runtestprebuilt-branch.


          Issue Links



                erikj Erik Joelsson
                erikj Erik Joelsson
                0 Vote for this issue
                3 Start watching this issue