Notice that these definitions are not equivalent, although they define similar notions.
A product may conform perfectly to its specifications but serve no useful purpose what-
soever. In a less extreme case, a program may conform to its specification but not be as
useful as planned because the environment changed. It is also possible that the specifi-
cation did not contemplate all aspects.
Some of the pioneers in quality also wrestled with the definition of quality. Juran
and Godfrey (1999) defined quality as fitness to use. Crosby (1979) posed quality as con-
formance to requirements and pushed the concept of prevention of error and of zero
defects.
Corresponding to these two notions of quality, we have the following two activities:
n
Verification, which is the act of checking that a software product conforms to its
requirements and specifications. The requirements for a software product are traced
through the development phases, and the transformation of software from one phase
to another is “verified.”
n
Validation, which is the act of checking that a finished software product meets users’
requirements and specifications. This can usually be done only at the end of a project,
with a complete software system.
The three definitions of fault, failure, and error presented here will delineate the differ-
ences of a problem found by users from the source of the problem. We usually identify a
fault as the cause of a failure, but it is not always possible to identify
one specific fault as the cause of a failure. Oftentimes, faults may exist
without being detected or observed in the form of failures. During
testing, failures that reveal the presence of faults are observed.
Note that this distinction is important for all software docu-
ments, not just running programs. For example, a requirement
that is hard to understand is a fault. It becomes a failure only if
it leads to the misunderstanding of a requirement, which in turn
causes an error in design and code that manifests itself in the
form of software failure. A human error creates a fault, and fault
may cause a failure. Not all defects turn into failures, especially
those that stay dormant because the specific code logic was never executed.
When deciding how a program should behave, you need to be aware of all the
explicit and implicit requirements and specifications. An explicit requirement needs to
be mentioned in the requirement documents, and an explicit specification is recognized
as authoritative by the software team. Notice that specifications are not always produced
for all projects, but they can be generic and included by reference. For example, there
may be a requirement such as “conforms to the Human Interface Guidelines of its plat-
form,” which makes those guidelines an explicit specification.
In many cases, there are also implicit specifications. These are not authoritative, but
they are good references and should be followed whenever possible. When inspecting
or reviewing a software product or planning test cases, these implicit specifications need
to be made explicit, even if only as guidelines.
You must also distinguish between the severity of a fault, which is a measure of the
impact or consequences it may have to the users or the organization, and the priority,
Fault/defect A condition that may cause a
failure in a system. It is caused by an error
made by the software engineer. A fault is
also called a bug.
Failure/problem The inability of a system
to perform a function according to its speci-
fications. It is a result of the defect in the
system.
Error A mistake made by a software engi-
neer or programmer.
10.1 Introduction to Testing and Quality Assurance
201
91998_CH10_Tsui.indd 201 1/10/13 10:59:27 AM