Conclusion

Maintaining a suite of automated test procedures can become a full-time job in itself. This is true because even simple module/interface testing can require hundreds of test scripts. We recently tested an error routine using a driver module that presented a dialog box with a severity icon, an error message, and several different sets of command. Testing this small routine took just under a hundred test scripts.

Test suite maintenance begins far in advance of the actual testing. It is always advisable to start designing and constructing test scripts and test cases as soon as you have enough information to begin. The problem that you will encounter is that the test scripts and test cases must be revised each time the developers change the software. If there is little or no control over how developers insert changes during the analysis and design processes, it will drive you crazy trying to keep the automated test scripts up to date with the changes.

A compromise solution is to begin designing, but not constructing, test scripts and test cases as soon as possible. This eliminates 50% of the maintenance burden. You have to update only the design, and you do not have to rerecord or reprogram the test scripts. At some point as the actual test date nears, you have to begin construction, but the longer you can put it off, the less pretest maintenance you will have to do. In addition, constructing paper-and-pencil test scripts, which are easily modified when necessary, can enhance this approach.

This approach also addresses another problem that occurs frequently. Developers customarily do not deliver the functional software modules until right before the test date, and yet you are still expected to perform testing on time. But how are you supposed to construct automated test scripts without a working prototype of the software? Well, if you have the test cases designed and you have kept the design as up-to-date as possible, and if you have constructed paper-and-pencil test scripts, you will probably still be able to construct the test scripts in time to do the testing.