Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8166893

Need a standard to define test pass or fail when the test environment not meet the execution preconditions

XMLWordPrintable

      Few tests need precondition to run ,
      As JDK-8164322 mentioned.
      some testcase using PKCS11Test.java to run need below conditions:
      1) The OS need to support NSS.
      2) The test machine need to be installed NSS.

      When the testcae execute on the OS not support NSS or OS not installed NSS, the case is skipping and passing now, it is not help to track the issue whether from OS ,machine or NSS.
      If make they failed, it could help to find the OS or machine issue , but the testcase was not really run and can't respond the issue from testcase or product. If a lot of this kind of issue, it may need spend much time to track the root reason one by one.

      Jtreg have @requires tag to keep test execute system being tested , but in this case it's can't check whether the NSS was installed on the machine.

      It may have a agreement to define if the test environment not meet the condition the testcase keep passing/skipping or show failures.

            jjiang John Jiang
            tidu Tim Du (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: