Execute Test Procedures
- To execute the test procedures (or test scripts if testing is
To execute the tests, the following steps should be followed:
- Set-up the test environment to ensure that all the needed components
(hardware, software, tools, data, etc.) have been implemented and are in the
- Initialize the test environment to ensure all components are in the
correct initial state for the start of testing.
- Execute the test procedures.
Note: executing the test procedures will vary dependent upon whether
testing is automated or manual.
- Automated testing: The test scripts created during the
Implement Test activity are executed.
- Manual execution: The structured test procedures
developed during the Design Test activity are used to manually execute
Evaluate Execution of Test
- To determine whether testing executed to completion or halted
- To determine if corrective action is required
The execution of testing ends or terminates in one of two conditions:
- Normal: all the test procedures (or scripts) execute as intended and to
If testing terminates normally, then continue with Verify
- Abnormal or premature: the test procedures (or scripts) did not execute
completely or as intended. When testing ends abnormally, the test results
may be unreliable. The cause of the abnormal / premature termination needs
to be identified, corrected, and the tests re-executed before any additional
test activities are performed.
If testing terminates abnormally, continue with Recover
From Halted Tests.
Verify Test Results
- To determine if the test results are reliable
- To identify appropriate corrective action the test results indicate
flaws in the test effort or artifacts
Upon the completion of testing, the test results should be reviewed to ensure
that the test results are reliable and reported failures, warnings, or
unexpected results were not caused by external influences (to the
target-of-test), such as improper set-up or data.
The most common failures reported when test procedures and test scripts
execute completely, and their corrective actions are given below:
- Test verification failures - this occurs when the actual result and the
expected result do not match. Verify that the verification method(s) used
focus only on the essential items and / or properties and modify if
- Unexpected GUI windows - this occurs for several reasons. The most common
is when a GUI window other than the expected one is active or the number of
displayed GUI windows is greater than expected. Ensure that the test
environment has been set-up and initialized as intended for proper test
- Missing GUI windows - this failure is noted when a GUI window is expected
to be available (but not necessarily active) and is not. Ensure that the
test environment has been set-up and initialized as intended for proper test
execution. Verify that the actual missing windows are / were removed from
If the reported failures are due to errors identified in the test artifacts,
or due to problems with the test environment, the appropriate corrective action
should be taken and the testing re-executed. For additional information,
see "Recover From Halted Tests"
If the test results indicate the failures are genuinely due to the
target-of-test, then the Execute Test Activity is complete and typically either the Activity:
Submit Change Request or Activity: Fix a Defect
should now be performed.
Recover From Halted Tests
- To determine the appropriate corrective action to recover from a
- To correct the problem, recover, and re-execute the tests
There are two major types of halted tests:
- Fatal errors - the system fails (network failures, hardware crashes, etc.)
- Test Script Command Failures - specific to automated testing, this is when
a test script cannot execute a command (or line of
Both types of abnormal termination to testing may exhibit the same symptoms:
- many unexpected actions, windows, or events occur while the test script is
- test environment appears unresponsive or in an undesirable state (such as
hung or crashed).
To recover from halted tests, do the following:
- determine the actual cause of the problem
- correct the problem
- re-set-up test environment
- re-initialize test environment
- re-execute tests