Uploaded image for project: 'Code Tools'
  1. Code Tools
  2. CODETOOLS-7900034

optionally disallow default behavior with no @run tag present

    XMLWordPrintable

Details

    • Enhancement
    • Resolution: Fixed
    • P2
    • None
    • None
    • tools
    • None

    Description

      Jtreg has a feature where a default behavior is assumed if no @run tag is present in the defining file of a test. This behavior is convenient and saves typing, but it can lead to problems where a test appears to be passing when in fact it isn't being run at all.

      For example, suppose we have a test that has these tags in a file named MyTest.java:

          @test
          @bug 1234567
          @summary My simple test.
          @author Fred

      Since there are no @run tags, jtreg will implicitly act as if "@run main MyTest" were present. OK so far.

      Now suppose we modify the test to use a helper class of some sort. We'd need to add an @build tag to ensure this class is built:

          @test
          @bug 1234567
          @summary My simple test.
          @author Fred
          @build Helper

      Since @build is a shorthand for "@run build", there is implicitly an @run tag in this file, and the default behavior does not apply. "Running" this test will then consist of building the MyTest and Helper classes but not actually executing the test.

      This kind of problem has occurred in practice when tests are modified or maintained. It's fairly easy for a test to end up in this state, and it's fairly difficult to detect, as the test "passes" when run, but one has to inspect the log carefully in order to determine that the test hasn't actually been run. The JDK regression suite has encountered this problem.

      It would be nice to be able to require that every test have an @run tag. Since @build counts as an @run tag, the above example will still "pass" even though nothing was executed. The rule would have to be finer-grained, perhaps requiring that a @run tag with a test execution action (e.g., "main", "shell", "applet", "testng") be present in every test.

      Of course, this is an incompatible change. It may be necessary to introduce this requirement via a command-line option or a property in TEST.ROOT.

      There may be tests that need to be executed solely for their @compile or @build side effects. (The compiler test suite may be an example of this.) It may be sufficient not to have the option enabled such suites. Or, if a single suite has a mixture of execution and non-execution tests, it may be necessary to introduce a new action like "@run noop" that satisfies the test execution requirement, but that explicitly indicates that the test performs no action. (It's easy to write a no-op Java or shell test, but it seems like this ought to be built into jtreg.)

      Attachments

        Issue Links

          Activity

            People

              jjg Jonathan Gibbons (Inactive)
              smarks Stuart Marks
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: