A test plan ...

• extends the “Testing” part of the corresponding JEP and provides all details on how the functionality is to be tested
• is a document to capture and share thought process on how the JEP functionality should be tested
• is a place for collaboration on testing methodology and requirements
• is owned by the team which owns the corresponding JEP
• should contain some sub-tasks defining test success criteria

Test success criteria

• Test success criteria are exit criteria for testing
• Test success criteria help to see if the testing is successful
• Test success criteria are handled as sub-tasks of a test plan
• Test success criteria sub-tasks are closed when satisfied
• See sub-tasks of this plan template for examples of test success criteria
• Criteria suggested below are only examples, only those which suitable should be used
• Any number of additional criteria could be used to better ensure test success

Depending on the changes introduced by the JEP, it is expected by the test plan to specify relevant criteria. If the JEP is adding ...

• new code - block, method code coverage
• public API - public API coverage
• options - options coverage
• specification, javadoc - specification coverage
• etc.

Please see individual criteria descriptions for more information.

Test plan handling and lifecycle

• A test plan should be a "JEP Task", blocking the JEP
• A test plan should be moved to closed state before the JEP is integrated
• A test plan can only be closed once test methodology is explained and test success criteria are reached
• It is recommended that you create a new test plan for a JEP by cloning this template
• Portions of description in italic are comments and supposed to be removed
• Optional sections of the test plan can be removed
• Additional sections or sub-sections should be added as needed
• All non-applicable test success criteria sub-tasks should be dropped
• Additional criteria sub-tasks should be added as needed

See linked issues for examples of previously created test plans.

1 Test Methodology

Extends "Testing" section of the JEP. This section could go as deep as necessary explaining what kind of testing should be done for the JEP and why.

2 Test Inventory (optional)

In some cases it helps to spell which tests are to be reused or added to test the feature. In some cases some tests are not longer relevant and should be removed/deprecated. This section can be used to list specific tests, test groups or test types such as unit, functional, integration, manual/automated, negative, etc.

This section should also be used to point to which test groups, tiers, etc, the new tests will be added.

Existing Tests

New Tests

Deprecated Tests

3 Test Configurations (optional)

It is often important to make sure that correct sets of tests be executed in the correct set of configurations, which could be set of platforms or VM flag combinations. Notice that simply repeating JDK supported platforms is redundant, so this section could be used when there are some specific requirements.