Quality must be defined and measured if improvement is to be achieved. Yet, a major problem in quality engineering and management is that the term quality is ambiguous, such that it is commonly misunderstood. The confusion may be attributed to several reasons. First, quality is not a single idea, but rather a multidimensional concept. The dimensions of quality include the entity of interest, the viewpoint on that entity, and the quality attributes of that entity. Second, for any concept there are levels of abstraction; when people talk about quality, one party could be referring to it in its broadest sense, whereas another might be referring to its specific meaning. Third, the term quality is a part of our daily language and the popular and professional uses of it may be very different.
In this chapter we discuss the popular views of quality, its formal definitions by quality experts and their implications, the meaning and specific uses of quality in software, and the approach and key elements of total quality management.
A popular view of quality is that it is an intangible trait—it can be discussed, felt, and judged, but cannot be weighed or measured. To many people, quality is similar to what a federal judge once commented about obscenity: “I know it when I see it.” Terms such as good quality, bad quality, and quality of life exemplify how people talk about something vague, which they don’t intend to define. This view reflects the fact that people perceive and interpret quality in different ways. The implication is that quality cannot be controlled and managed, nor can it be quantified. This view is in vivid contrast to the professional view held in the discipline of quality engineering that quality can, and should, be operationally defined, measured, monitored, managed, and improved.
Another popular view is that quality connotes luxury, class, and taste. Expensive, elaborate, and more complex products are regarded as offering a higher level of quality than their humbler counterparts. Therefore, a Cadillac is a quality car, but a Chevrolet is not, regardless of reliability and repair records; or, a surround-sound hi-fi system is a quality system, but a single-speaker radio is not. According to this view, quality is restricted to a limited class of expensive products with sophisticated functionality and items that have a touch of class. Simple, inexpensive products can hardly be classified as quality products.
The misconceptions and vagueness of the popular views do not help the quality improvement effort in the industries. To that end, quality must be described in a workable definition. Crosby (1979) defines quality as “conformance to requirements” and Juran and Gryna (1970) define it as “fitness for use.” These two definitions are related and consistent, as we will see later. These definitions of quality have been adopted and used by many quality professionals.
“Conformance to requirements” implies that requirements must be clearly stated such that they cannot be misunderstood. Then, in the development and production process, measurements are taken regularly to determine conformance to those requirements. The nonconformances are regarded as defects—the absence of quality. For example, one requirement (specification) for a certain radio may be that it must be able to receive certain frequencies more than 30 miles away from the source of broadcast. If the radio fails to do so, then it does not meet the quality requirements and should be rejected. By the same token, if a Cadillac conforms to all the requirements of a Cadillac, then it is a quality car. If a Chevrolet conforms to all the requirements of a Chevrolet, then it is also a quality car. The two cars may be very different in style, performance, and economy. But if both measure up to the standards set for them, then both are quality cars.
The “fitness for use” definition takes customers’ requirements and expectations into account, which involve whether the products or services fit their uses. Since different customers may use the products in different ways, it means that products must possess multiple elements of fitness for use. According to Juran, each of these elements is a quality characteristic and all of them can be classified into categories known as parameters for fitness for use. The two most important parameters are quality of design and quality of conformance.
Quality of design in popular terminology is known as grades or models, which are related to the spectrum of purchasing power. The differences between grades are the result of intended or designed differences. Using the example of cars again, all automobiles provide to the user the service of transportation. However, models differ in size, comfort, performance, style, economy, and status. In contrast, quality of conformance is the extent to which the product conforms to the intent of the design. In other words, quality of design can be regarded as the determination of requirements and specifications and quality of conformance is conformance to requirements.
The two definitions of quality (conformance to requirements and fitness for use), therefore, are essentially similar. The difference is that the fitness for use concept implies a more significant role for customers’ requirements and expectations.
The role of the customer, as it relates to quality, can never be overstated. From a customer’s standpoint, quality is the customer’s perceived value of the product he or she purchased, based on a variety of variables such as price, performance, reliability, and satisfaction. In Guaspari’s book I Know It When I See It (1985 p.77), he discusses quality in the customers’ context as follows:
Your customers are in a perfect position to tell you about quality, because that’s all they’re really buying. They’re not buying a product. They’re buying your assurances that their expectations for that product will be met.
And you haven’t really got anything else to sell them but those assurances. You haven’t really got anything else to sell but quality.
From a concept’s high-level definition to a product’s operational definition, many steps are involved, each of which may be vulnerable to shortcomings. For example, to achieve the state of conformance to requirements, the customers’ requirements must be first gathered and analyzed, specifications to meet those requirements must be produced, and the product must be developed and manufactured accordingly. In each phase of the process, errors can occur that will affect the quality of the finished product. The requirements may be erroneous (this is especially the case for software development), the development and manufacturing process may be subject to variables that induce defects, and so forth. From the customer’s perspective, satisfaction after the purchase of the product is the ultimate validation that the product conforms to requirements and is fit to use. From the producer’s perspective, once requirements are specified, developing and producing the product in accordance with the specifications is the path to achieving quality. Usually, for product quality, the lack of defects and good reliability are the most basic measures.
Because of the two perspectives on quality (customer satisfaction as the ultimate validation of quality and producer’s adherence to requirements to achieve quality), the de facto definition of quality consists of two levels. The first is the intrinsic product quality, often operationally limited to the product’s defect rate and reliability. This narrow definition is referred to as the “small q” (q for quality). The broader definition of quality includes product quality, process quality, and customer satisfaction, and it is referred to as the “big Q.” This two-level approach to the definition of quality is being used in many industries, including the automobile industry, the computer industry (both software and hardware), and the consumer electronics industry.
The two-level concept of quality is supposed to form a closed-loop cycle: customer’s wants and needs → requirements and specifications → products designed, developed, and manufactured in accordance with the requirements, and with continuous focus on process improvement → excellent product quality, plus good distribution and service processes → total customer satisfaction. However, this was not always the case in many industries, especially before the late 1980s when the modern quality era began. Product requirements were often generated without customer input, and customer satisfaction was not always a factor in business decision making. Although the final products conformed to requirements, they may not have been what the customers wanted. Therefore, the customers’ role should be explicitly stated in the definition of quality: conformance to customers’ requirements.
In software, the narrowest sense of product quality is commonly recognized as lack of “bugs” in the product. It is also the most basic meaning of conformance to requirements, because if the software contains too many functional defects, the basic requirement of providing the desired function is not met. This definition is usually expressed in two ways: defect rate (e.g., number of defects per million lines of source code, per function point, or other unit) and reliability (e.g., number of failures per n hours of operation, mean time to failure, or the probability of failure-free operation in a specified time). Customer satisfaction is usually measured by percent satisfied or nonsatisfied (neutral and dissatisfied) from customer satisfaction surveys. To reduce bias, techniques such as blind surveys (the interviewer does not know the customer and the customer does not know the company the interviewer represents) are usually used. In addition to overall customer satisfaction with the software product, satisfaction toward specific attributes is also gauged. For instance, IBM monitors satisfaction with its software products in levels of CUPRIMDSO (capability [functionality], usability, performance, reliability, installability, maintainability, documentation/information, service, and overall). Hewlett-Packard focuses on FURPS (functionality, usability, reliability, performance, and serviceability). Other compa-nies use similar dimensions of software customer satisfaction. Juran calls such attributes quality parameters, or parameters for fitness for use.
To increase overall customer satisfaction as well as satisfaction with various quality attributes, the quality attributes must be taken into account in the planning and design of the software. However, these quality attributes are not always congruous with each other. For example, the higher the functional complexity of the software, the harder it becomes to achieve maintainability. Depending on the type of software and customers, different weighting factors are needed for different quality attributes. For large customers with sophisticated networks and real-time processing, performance and reliability may be the most important attributes. For customers with standalone systems and simple operations, on the other hand, ease of use, installability, and documentation may be more important. Figure 1.1 shows the possible relationships of some quality attributes. Some relationships are mutually supportive, some are negative, and yet others are not clear, depending on the types of customers and applications. For software with a diverse customer set, therefore, setting goals for various quality attributes and to meet customers’ requirements is not easy.
In view of these discussions, the updated definition of quality (i.e., conformance to customers’ requirements) is especially relevant to the software industry. It is not surprising that requirements errors constitute one of the major problem categories in software development. According to Jones (1992), 15% or more of all software defects are requirements errors. A development process that does not address requirements quality is bound to produce poor-quality software.
Yet another view of software quality is that of process quality versus end-product quality. From customer requirements to the delivery of software products, the development process is complex and often involves a series of stages, each with feedback paths. In each stage, an intermediate deliverable is produced for an intermediate user—the next stage. Each stage also receives an intermediate deliverable from the preceding stage. Each intermediate deliverable has certain quality attributes that affect the quality of the end product. For instance, Figure 1.2 shows a simplified representation of the most common software development process, the waterfall process.
Intriguingly, if we extend the concept of customer in the definition of quality to include both external and internal customers, the definition also applies to process quality. If each stage of the development process meets the requirements of its intermediate user (the next stage), the end product thus developed and produced will meet the specified requirements. This statement, of course, is an oversimplification of reality, because in each stage numerous factors exist that will affect that stage’s ability to fulfill its requirements. To improve quality during development, we need models of the development process, and within the process we need to select and deploy specific methods and approaches, and employ proper tools and technologies. We need measures of the characteristics and quality parameters of the development process and its stages, as well as metrics and models to ensure that the development process is under control and moving toward the product’s quality objectives. Quality metrics and models are the focus of this book.
The term Total quality management (TQM) was originally coined in 1985 by the Naval Air Systems Command to describe its Japanese-style management approach to quality improvement. The term has taken on a number of meanings, depending on who is interpreting it and how they are applying it. In general, however, it represents a style of management aimed at achieving long-term success by linking quality and customer satisfaction. Basic to the approach is the creation of a culture in which all members of the organization participate in the improvement of processes, products, and services. Various specific methods for implementing the TQM philosophy are found in the works of Crosby (1979), Deming (1986), Feigenbaum (1961, 1991), Ishikawa (1985), and Juran and Gryna (1970).
Since the 1980s, many U.S. companies have adopted the TQM approach to quality. The Malcolm Baldrige National Quality Award (MBNQA), established by the U.S. government in 1988, highlights the embracing of such a philosophy and management style. The adoption of ISO 9000 as the quality management standard by the European Community and the acceptance of such standards by the U.S. private sector in recent years further illustrates the importance of the quality philosophy in today’s business environments. In the computer and electronic industry, examples of successful TQM implementation include Hewlett-Packard’s Total Quality Control (TQC), Motorola’s Six Sigma Strategy, and IBM’s Market Driven Quality. In fact, Motorola won the first MBNQA award (in 1988) and IBM’s AS/400 Division in Rochester, Minnesota, won it in 1990.
Hewlett-Packard’s TQC focuses on key areas such as management commitment, leadership, customer focus, total participation, and systematic analysis. Each area has strategies and plans to drive the improvement of quality, efficiency, and responsiveness, with the final objective being to achieve success through customer satisfaction (Shores, 1989). In software development, the Software Quality and Productivity Analysis (SQPA) program (Zimmer, 1989) is one of the approaches to improve quality.
Motorola’s Six Sigma strategy focuses on achieving stringent quality levels in order to obtain total customer satisfaction. Cycle time reduction and participative management are among the key initiatives of the strategy (Smith, 1989). Six Sigma is not just a measure of the quality level; inherent in the concept are product design improvements and reductions in process variations (Harry and Lawson, 1992). Six Sigma is applied to product quality as well as everything that can be supported by data and measurement.
“Customer is the final arbiter” is the key theme of IBM’s Market Driven Quality strategy. The strategy comprises four initiatives: defect elimination, cycle time reduction, customer and business partner satisfaction, and adherence to the Baldrige assessment discipline.
Despite variations in its implementation, the key elements of a TQM system can be summarized as follows:
Customer focus: The objective is to achieve total customer satisfaction. Customer focus includes studying customers’ wants and needs, gathering customers’ requirements, and measuring and managing customers’ satisfaction.
Process: The objective is to reduce process variations and to achieve continuous process improvement. This element includes both the business process and the product development process. Through process improvement, product quality will be enhanced.
Human side of quality: The objective is to create a companywide quality culture. Focus areas include leadership, management commitment, total participation, employee empowerment, and other social, psychological, and human factors.
Measurement and analysis: The objective is to drive continuous improvement in all quality parameters by the goal-oriented measurement system.
Furthermore, an organization that practices TQM must have executive leadership, must focus on infrastructure, training, and education, and must do strategic quality planning.
Figure 1.3 is a schematic representation of the key elements of TQM. Clearly, measurement and analysis are the fundamental elements for gauging continuous improvement.
Various organizational frameworks have been proposed to improve quality that can be used to substantiate the TQM philosophy. Specific examples include Plan-Do-Check-Act (Deming, 1986; Shewhart, 1931), Quality Improvement Paradigm/ Experience Factory Organization (Basili, 1985, 1989; Basili and Rombach, 1987, 1988; Basili et al., 1992), Software Engineering Institute (SEI) Capability Maturity Model (Humphrey, 1989; Radice et al., 1985), and Lean Enterprise Management (Womack et al., 1990).
Plan-Do-Check-Act is based on a feedback cycle for optimizing a single process or production line. It uses techniques, such as feedback loops and statistical quality control, to experiment with methods for improvement and to build predictive models of the product. Basic to the assumption is that a process is repeated multiple times, so that data models can be built that allow one to predict results of the process.
The Quality Improvement Paradigm/Experience Factory Organization aims at building a continually improving organization, based on its evolving goals and an assessment of its status relative to those goals. The approach uses internal assessments against the organization’s own goals and status (rather than process areas) and such techniques as Goal/Question/Metric (GQM), model building, and qualitative/ quantitative analysis to improve the product through the process. The six fundamental steps of the Quality Improvement Paradigm are (1) characterize the project and its environment, (2) set the goals, (3) choose the appropriate processes, (4) execute the processes, (5) analyze the data, and (6) package the experience for reuse. The Experience Factory Organization separates the product development from the experience packaging activities. Basic to this approach is the need to learn across multiple project developments.
The SEI Capability Maturity Model is a staged process improvement, based on assessment of key process areas, until you reach level 5, which represents a continuous process improvement. The approach is based on organizational and quality management maturity models developed by Likert (1967) and Crosby (1979), respectively. The goal of the approach is to achieve continuous process improvement via defect prevention, technology innovation, and process change management.
As part of the approach, a five-level process maturity model is defined based on repeated assessments of an organization’s capability in key process areas. Improvement is achieved by action plans for poor process areas. Basic to this approach is the idea that there are key process areas and attending to them will improve your software development.
Lean Enterprise Management is based on the principle of concentration of production on “value-added” activities and the elimination or reduction of “not-value-added” activities. The approach has been used to improve factory output. The goal is to build software with the minimum necessary set of activities and then to tailor the process to the product’s requirements. The approach uses such concepts as technology management, human-centered management, decentralized organization, quality management, supplier and customer integration, and internationalization/ regionalization. Basic to this approach is the assumption that the process can be tailored to classes of problems.
This chapter discusses the definition of quality from both popular and professional views, and describes total quality management (TQM) as it relates to software quality. From the popular view, quality is some type of thing that cannot be quantified: I know it when I see it. Quality and grade (or class) are often confused. From the professional view, quality must be defined and measured for improvement and is best defined as “conformance to customers’ requirements.” In software as well as other industries, the de facto operational definition of quality consists of two levels: the intrinsic product quality (small q) and customer satisfaction (big Q).
The TQM philosophy aims at long-term success by linking quality and customer satisfaction. Despite variations in its implementation, a TQM system comprises four key common elements: (1) customer focus, (2) process improvement, (3) human side of quality, and (4) measurement and analysis.
It is not surprising that the professional definition of quality fits perfectly in the TQM context. That definition correlates closely with the first two of the TQM elements (customer focus and process improvement). To achieve good quality, all TQM elements must definitely be addressed, with the aid of some organizational frameworks. In this book, our key focus is on metrics, measurements, and quality models as they relate to software engineering. In the next chapter we discuss various software development models and the process maturity framework.