Defining the System Test Plan
The system test plan is a more formal and more comprehensive document than the component test plans we have been using. In many cases it will be reviewed by the customers. In Figure 3.14 we listed the sections in the IEEE test plan format. We are not going to discuss every section, but we will cover some of the sections and we will complete some of the sections for the Brickles application. We will touch on a couple of the items here, and then, in the following sections, we will expand on the more important ones. In Figure 9.1 we provide an abbreviated system test plan for Brickles using the format that we have used in previous chapters.
Figure 9.1. A system test plan for Brickles
Features Tested and Not Tested
Some amount of validation testing is performed as each increment is delivered. Rather than having sections on which features are tested, we have a schedule that is a copy of the project's increment plan combined with dates by which each increment will be tested. This increment plan is usually defined in terms of use cases and so is the test plan.
For Brickles we created a plan that called for three increments, as shown in Figure 3.2. First we developed the basic infrastructure, the “Move Paddle” and “Lose Puck” use cases. The second increment contained the “Break Brick” and “Win/Lose” use cases. The third increment provided the “Pause” use case. The test plan schedules a system validation at the end of each increment.
Figure 9.2. A Brickles state diagram
Test Suspension Criteria and Resumption Requirements
Since system testers are neither debuggers nor developers, if the system being tested contains too many defects, testing may need to be suspended before all planned tests have been run. Typically we would begin with tests of one of the use cases scheduled for the current increment and then move on to the next use case. If we run a test against a use case and it fails, then we move to tests for the next use case. Testing is suspended if there is no use case for which we can successfully complete a test. Testing is resumed when sufficient development effort has been expended to cause a significant percentage of the use cases to pass tests.