6.1. Testing in Context

Quality control deals with ensuring that the quality of the end product meets a given specification. In the manufacturing domain, quality control deals with checking the nuts and bolts on goods produced for their adherence to given standards. In the case of software, quality control deals with whether the software product functions correctly, according to specifications, and in conjunction with other software products and databases. The ultimate aim of software quality control (or testing, as it is routinely known), is to “break” the system long before it reaches the user in real life.

Ideally, therefore, if a tester manages to pass an illegal transaction through the system being tested, or is somehow able to crash the system, then that test is hailed as a pass or success. This is because the tester has successfully managed to discover an error under controlled test conditions, which if gone undetected, may have the potential to bring a real system (and probably a real business) to a halt.

However, testing is an operational and tactical effort. A tester is bent on finding as many errors as possible. This is different from the role of a quality analyst, who is interested in reducing the occurrences of errors in the first place—primarily by ensuring compliance with a process discipline and modeling. A quality manager is responsible for creating the right environment—with the right technology, methodology, and sociology conducive to the quality software development. Figure 6.1 shows the three major areas of work related to quality and their strategic versus tactical nature. Quality management is shown to influence quality assurance, which in turn influences quality control or testing. The responsibilities of the three major areas of work shown in Figure 6.1 are summarized as follows:

• Quality management, together with project management, deals with the overall approach to project quality. Creating a quality environment with a good environmental factor, creating and managing teams, and handling the overall sociology within the project are some of the responsibilities of quality management.

• Quality assurance ensures that the models and processes within software development are followed in a manner that results in good software. By following a quality process, one expects to find fewer errors and bugs and more satisfactory acceptance tests, resulting from the fact that the software development has taken place in response to a well-defined problem. Quality assurance provides all the necessary techniques (such as the detailed syntax, semantics, and aesthetic checks in the three modeling spaces) and the process (as described in Chapter 3). It provides the templates for deliverables and verifies the compliance of the deliverables to the standards within the project. This results in the overall quality of models and processes, which is reassuring to the stakeholders in terms of the expected and accepted level of system quality.

• Quality control or testing does the policing work on the software system. The following sections discuss the best way to approach this testing work so that maximum errors are found during the last stage before the system is released into the real world. Testing is mentioned as operational in nature in the context of the overall project. On its own, however, testing requires careful planning and an approach that suits the system being tested. The type of project, as discussed in Chapter 1, influences the creation of the test plan. For example, in integration projects, testing of new interfaces to the existing legacy application requires extensive testing, whereas the testing needs of a data warehousing project simply focus on the accuracy and relevance of the data being used. The project size also influences the testing approach. For example, large projects need a much bigger system-testing lifecycle and have to factor in the costs and times for the necessary rework resulting from testing. Small projects, on the other hand, may focus on black box acceptance tests (described in the next section) right from the beginning, as opposed to the detailed walk throughs and inspections needed on large projects. Project criticality is also important in quality control. Although one would like to test every possible alternative, in practice, after the basic system testing has been accomplished, the scope of testing should be narrowed to focus on the critical needs of the system. For example, preferably, a submarine project should undergo extensive zero-tolerance testing, but a name and address system for an insurance mail list may be released in production if it is “good enough.” It is not always possible to get this balancing act right. The quality manager, together with the project manager, should ensure that testing is not reduced under the guise of less criticality, but is also not “done to death” at the cost of missing business opportunities. Experience helps in getting this balance right.