Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8264527

Test Plan for JEP 408: Simple Web Server

    XMLWordPrintable

Details

    • JEP Task
    • Status: Closed
    • P4
    • Resolution: Rejected
    • None
    • other-libs
    • None

    Description

      1 Test Methodology

      Simple Web Server Test Plan

      Objective

      To specify a range of test, and the success metrics, required to qualify the release of the functionality provided by the Simple Web Server Tool ( https://bugs.openjdk.java.net/browse/JDK-8260510)

      The range of testing will cover functional tests, reliability tests, stress (stability), scale, and security tests. Additionally, elementary performance analysis and code coverage analysis will be carried.

      Testing Scope

      What's in scope

      What's out of scope

      • Formal JCK testing. However, there will be equivalent API level tests.

      2 Test Inventory

      Regression Tests Inventory

      1. Functional tests

      Component Functional test cases

      tests Count
      ????
      ????

      Command line test cases

      tests Count
      ????
      ????

      2. API tests

      The SWST has an API, and this should be extensively tested for functional correctness and operational rubustness.

      JCK equivalent tests

      tests Count
      ???? ??
      ???? ??

      3. SPI tests

      SPI tests

      tests Count
      ???? ??
      ???? ??

      4. HTTP Response Conformance tests

      Establish that the server returns the correct HTTP responses for relevant

      HTTP Resposne tests

      tests Count
      ???? ??
      ???? ??

      5. HTML Response Conformance tests

      Validate that any HTML response bodies are well formed and correct HTML validation

      HTML Response Validation tests

      tests Count
      ???? ??
      ???? ??

      6. HTML Response WCAG Conformance tests

      Validate that any HTML response bodies are valid as per WCAG (Web content accessibility guidelines - e.g. screenreaders)

      HTML Response Validation tests

      tests Count
      ???? ??
      ???? ??

      Other Test Inventory

      Stability, Reliability and Scale Tests

      Reliability is defined as the probability that a product, system, or service will perform its intended function adequately for a specified period of time, or will operate in a defined environment without failure.

      Stability is a complex term but for the purposes of this use case can be defined as reliability at a given scale

      The main objective is to establish that the SWST performs reliably and stable under known conditions. Also, that a know set of metrics are establish in terms of scale, performance and throughput. In performing reliably it should be able to provide continuous service of a defined number of days, and should not exhibit any appreciable or excessive resource retention, in particular excessive memory growth. Tests to establish the number of connections and number of requests per second the SWST will handle

      Reliability tests scenarios

      tests Count
      ??? ???

      Stability tests scenarios

      tests Count
      ??? ???

      Scale Metrics tests

      tests Count
      ??? ???

      Performance Metrics Tests

      Stress tests

      tests Count
      ??? ???

      Request Performance Metrics tests

      tests Count
      ??? ???

      Microbenchmark Tests

      TBD

      tests Count
      ??? ???

      Interoperability Tests

      TBD

      • Java HttpClient
      • Apache HttpClient
      • curl
      • Basic Common Browser testing

      4. Others

      investigate basic security and vulnerability tests - OWASP Zed Attack Proxy tests

      OWASP ZAP Tests

      tests Count
      ??? ???

      Success criteria

      Regression Tests

      Criteria Planned Status Result Comments
      Public API coverage 100% 100%
      Block coverage of new code ??% ??%
      Test stability SBR runs and ATR make sure there are no intermittent failures. All good in ATR and SBR runs
      Open test issues makes sure all the tests are ready
      Pass Rate 100%

      Test Configurations

      N/A

      Attachments

        Issue Links

          Activity

            People

              isipka Ivan Sipka
              msheppar Mark Sheppard
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: