Roles and Activities > Tester > Evaluate Test

Purpose
  • To evaluate the test results and log change requests.
  • To calculate and deliver the key measures of test.
  • To generate the test evaluation summary.
Steps
Input Artifacts: Resulting Artifacts:
Role: Tester

Workflow Details:

 

Analyze Test Results and Submit Change Requests To top of page

Purpose
  • To identify and assess the differences between the expected results and actual results
  • To enter change request information into a tracking tool for assessment, management, and resolution

In the Test Execution activity, the results of test were reviewed to ensure that the testing executed to completion, and the reported test results were not influenced by non-target-of-test factors. 

In this activity, the test results are analyzed to identify the details regarding the differences between the expected test results and the actual test results. Differences indicate potential defects in the target-of-test and should be entered into a tracking system as change requests, and the appropriate next corrective actions taken.

Review each reported test failure and identify the necessary information needed to enter a change request into the appropriate tracking system. The information entered should be accurate, appropriate, and meet accepted change request tracking standards or guidelines.

Evaluate Requirements-based Test Coverage To top of page

Purpose
  • To determine if the required (or appropriate) requirements-based test coverage has been achieved

To evaluate requirements-based test coverage, you need to review the test results and determine:

  • The ratio between how many requirement-based tests (test cases) have been performed in this iteration and a total number of tests for the target for test.
  • The ratio of successfully performed test cases.

The objective is to ensure that 100 % of the requirements-based tests targeted for this iteration have been executed successfully. If this is not possible or feasible, a different test coverage criteria should be identified, based upon:

  • Risk or priority
  • Acceptable coverage percent

Document the results in a Test Evaluation Report for this iteration.

Evaluate Code-based Test CoverageTo top of page

Purpose
  • To determine if the required (or appropriate) code-based test coverage has been achieved.

To evaluate code-based test coverage, you need to review the test results and determine:

  • The ratio between the code that has been executed during test (such as lines or statements) in this iteration and the total code in the test target.

The objective is to ensure that 100 % of the code targeted for this iteration phase has been executed successfully. If this is not possible or feasible, a different test coverage criteria should be identified, based upon:

  • Risk or priority
  • Acceptable coverage percent

Document the results in a Test Evaluation Report for this iteration.

Analyze Defects To top of page

Purpose
  • To evaluate the defects and recommend the appropriate follow-on activity
  • To produce objective reports communicating the results of testing

To analyze defects, you need to review and analyze the measures chosen as part of your defect analysis strategy. The most common defect measures used include the following different measures (often displayed in the form of a graph):

  • Defect Density - the number of defects are shown as a function of one or two defect attributes (such as status or severity).
  • Defect Trend - the defect count is shown as a function over time.
  • Defect Aging - a special defect density report in which the defect counts are shown as a function of the length of time a defect remained in a given status (open, new, waiting-for-test, etc.)

Compare the measures from this iteration to the results from the analysis of previous iterations, to get a view of defect trends.

It is advisable to show the results in diagram form.

Test Completion and Success Criteria To top of page

Purpose
  • To determine if testing has been executed completely and acceptably
  • To identify the appropriate follow-on test activity

Review the test strategy defined in the test plan. It should present test criteria in terms of test coverage and, or defect evaluation. Examine the test results, defects, and the analysis of the defects, and determine if the criteria have been met.

If the criteria have not been met, there are several alternatives:

Gather additional information:

  • Produce different reports, such as different defect density reports
  • Investigate the process to determine if unexpected conditions contributed to the deviation from the stated test criteria and re-assess the criteria based upon this new knowledge

Suggest scheduling of additional testing:

  • Implement new tests to add depth to the test cases
  • Implement new tests to add greater test coverage

Modify the test criteria:

  • Review and assess the risk of changing the criteria POST testing
  • Identify a subset of the software that satisfies the test criteria and decide whether it can be deployed.

Generate the Test Evaluation Report To top of page

Purpose
  • To produce an objective report communicating the results of testing, key measures, and test recommendations

The last step in the Evaluate Test activity is to generate the test evaluation report.  This is accomplished by assembling the above information into a single report and distributing it to the appropriate roles for review.