3.2.1 Initial Model Proposed by McCall

McCall and his colleagues have been attributed with the original idea for a software quality model [MCC 77]. This model was developed in the 1970s for the United States Air Force and was designed for use by software designers and engineers. It proposes three perspectives for the user (see Figure 3.1) and primarily promotes a product-based view of the software product:

• – Operation: during its use;
• – Revision: during changes made to it over the years;
• – Transition: for its conversion to other environments when the time comes to migrate to a new technology.

Each perspective is broken down into a number of quality factors. The model proposed by McCall and his colleagues lists 11 quality factors.

Each quality factor can be broken down into several quality criteria (see Figure 3.2). In general, the quality factors considered by McCall are internal attributes that fall under the responsibility and control of software engineers during software development. Each quality factor (on the left side of Figure 3.2) is associated with two or more quality criteria (which are not directly measurable in the software). Each quality criterion is defined by a set of measures. For example, the quality factor “reliability” is divided into two criteria: accuracy and error tolerance. The right side of Figure 3.2 presents the measurable properties (called “quality criteria”), which can be evaluated (through observation of the software) to assess quality. McCall proposes a subjective evaluation scale of 0 (minimum quality) to 10 (maximum quality).

The McCall quality model was primarily aimed at software product quality (i.e., the internal perspective) and did not easily tie in with the perspective of the user who is not concerned with technical details. Take, for example, a car owner who is not concerned with the metals or alloys used to make the engine. He expects the car to be well designed so as to minimize frequent and expensive maintenance costs. Another criticism regarding this model was that it involved far too many measurable properties (approximately 300).

3.2.2 The First Standardized Model: IEEE 1061

The IEEE 1061 standard, that is, the Standard for a Software Quality Metrics Methodology [IEE 98b], provides a framework for measuring software quality that allows for the establishment and identification of software quality measures based on quality requirements in order to implement, analyze, and validate software processes and products. This standard claims to adapt to all business models, types of software, and all of the stages of the software life cycle. The IEEE 1061 standard presents examples of measures without formally prescribing specific measures.

Figure 3.3 illustrates the structure of the major concepts proposed in this quality model. At the top tier, we can see that software quality requires prior specification of a certain number of quality attributes, which serve to describe the final quality desired in the software. The attributes desired by clients and users allow for the definition of the software quality requirements. There must be a consensus with respect to these requirements within the project team, and their definitions should be clearly written in both the project and technical specifications.

Quality factors suggested by this standard are assigned attributes at the next tier. At the tier below that, and only if necessary, subfactors can then be assigned to each quality factor. Lastly, measures are associated with each quality factor, allowing for a quantitative evaluation of the quality factor (or subfactor).

This model is interesting since it provides defined steps to use quality measures in the following situations:

• – Software program acquisition: In order to establish a contractual commitment regarding quality objectives for client-users and verify whether they were met by allowing their measurement when adapting and releasing the software.
• – Software development: In order to clarify and document quality characteristics on which designers and developers must work in order to respect the customer's quality requirements.
• – Quality assurance/quality control/audit: In order to enable those outside the development team to evaluate the software quality.
• – Maintenance: Allow the maintainer to understand the level of quality and service to maintain when making changes or upgrades to the software.
• – Client/user: Allow users to state quality characteristics and evaluate their presence during acceptance tests (i.e., if the software does not meet the specifications agreed upon by the developer, the client can negotiate to his advantage conditions to have them met; for example, the client could negotiate free software maintenance for a specific period of time or the addition of functions at no additional cost).

The following steps are proposed under the IEEE 1061 [IEE 98b] standard:

• – Start by identifying the list of non-functional (quality) requirements for the software as of the beginning of the specifications elicitation. To define these requirements, contractual stipulations, standards, and company history must be taken into account. Set a priority for these requirements and try not to resolve any conflicting quality requirements at this time. Make sure that all participants can share their opinion when collecting information.
• – Make sure that you meet everybody involved to discuss the factors that should be considered.
• – Make a list and make sure to resolve any conflicting points of view.
• – Quantify each quality factor. Identify the measures to evaluate this factor and the desired objective in order to meet the threshold and level of quality expected.
• – Have measures and thresholds approved. This step is important, since it will identify the possibility of carrying out these measurements within your organization. The standard suggests that you customize and document Figure 3.4 for your specific project.
• – Perform a cost–benefit study to identify the costs of implementing the measures for the project. This may be necessary due to:

• additional costs to enter information, automate calculations, interpret, and present the results;
• costs to modify support software;
• costs for software assessment specialists;
• the purchase of specialized software to measure the software application;
• training required to apply the measurement plan.

• – Implement the measurement method: Define the data collection procedure, describing storage, responsibilities, training, etc. Prototype the measurement process. Choose which part of the software on which to apply the measures. Use the result to improve the cost-benefit analysis. Collect data and calculate values observed from quality factors.
• – Analyze the results: Analyze the differences between the measurements obtained and the expected values. Analyze significant differences. Identify measures outside the expected limits for further analysis. Make decisions based on quality (redo or continue). Use validated measures to make predictions during development. Use measures to validate the quality during tests.
• – Validate the measures: It is necessary to identify measures that can predict the value of quality factors, which are numeric representations of quality requirements. Validation is not universal, but must be done for each project. To validate measures, the standard recommends using the following techniques:

• Linear correlation: If there is a positive correlation, the measure can be used as a substitute for the factor.
• Identification of variations: If a factor goes from F₁ (time t₁) to F₂ (time t₂), the measure must change in the same way. This criterion ensures that the measure chosen can detect changes in quality.
• Consistency: If F₁ > F₂ > F₃, then M₁ > M₂ > M₃. This allows us to sort products based on quality.
• Foreseeability: (F_a − F_p)/F_p < A (F_a actual factor at time t, F_p anticipated factor at time t, A constant). This factor evaluates whether the measurement formula can predict a quality factor with the desired accuracy (A).
• Power of discrimination: The measures must be able to distinguish between high-quality software and low-quality software.
• Reliability: This measure must show that in P%, the correlation, identification, consistency, and foreseeability are valid.

The IEEE 1061 [IEE 98b] standard has allowed us to put measurement into practice and to link product measures with client-user requirements. Since this standard was classified as a guide, it was not very popular outside of the military for several reasons:

• – seen as being too expensive;
• – some did not see its usefulness;
• – the industry was not ready to use it;
• – suppliers did not want to be measured in this way by their clients.

This American standard influenced international software quality standards. The ISO 25000 [ISO 14a] standard is presented next.

3.2.3 Current Standardized Model: ISO 25000 Set of Standards

It was during the eighth international conference in 1980, at The Hague in the Netherlands, that Japan proposed setting up an ISO committee to standardize an internationally recognized software quality model. A work group was created, workgroup 6, and it was assigned to Professor Motoei Azuma (an Emeritus Professor at Waseda University, Tokyo, Japan) who, in turn, asked for help from the international community to study the proposals and possible solutions.

The international standardization of a software quality model, the ISO 9126 standard [ISO 01], was published for the first time in 1991. As we can see by the terminology used by McCall et al. (1977) [MCC 77] and the IEEE 1061 [IEE 98b] standard, there are a number of definitions and terms that were reused. The ISO 9126 [ISO 01] standard has attempted to promote the systematic implementation of software quality measures since 1991. However, it is little known and not often used in industry and was replaced by the ISO/IEC 25000 standard [ISO 14a]. We continue to see that manufacturers, suppliers, and major consulting firms avoid the formal use of the standard in their services. The main reason is that it creates requirements for quality and for guarantees that they are trying to avoid. Gradually and inevitably, this standard will become essential for software professionals.

On February 11, 1993, the Treasury Board of Canada issued an internal directive, Directive NCTTI 26 entitled “Software Evaluation—Quality Characteristics of Software and Usage Guidelines” [CON 93]. This directive proposes the practical use of the International Organization for Standardization (ISO) standard. This internal standard for information technology at the Treasury Board supports the government's policy for improving the management of information technology, which requires the adoption of quality management practices, including control systems, in order to promote the acquisition, use and management of information technology resources in an innovative and cost effective manner. Since the software industry had gained a certain level of maturity and that software was now an essential component of a number of government products and services, it was necessary to be concerned with their quality. They went on to say that given the increasing demands for quality and safety, in the future, evaluation of software quality should be completed using a quality model proposed by ISO.

The Treasury Board concluded that there were basically two ways to determine the quality of a software product: (1) assess the quality of the development process, and (2) assess the quality of the final product. The ISO 25000 [ISO 14a] standard allows for the evaluation of the quality of the final software product.

In some cases, ministries and agencies may decide not to use this standard, particularly when it is much more advantageous in regards to performance or cost, it is for the general advantage of the Government of Canada, or if the ministry or agency has a contractual obligation or is a partner of an international treaty (such as NATO).

Since 2005, ISO 25000 [ISO 14a] has provided a series of standards for the evaluation of software quality. The purpose of this standard is to provide a framework and references for defining the quality requirements of software and the way in which these requirements will be assessed.

The ISO 25000’s series of standards recommends the following four steps [ISO 14a]:

• – set quality requirements;
• – establish a quality model;
• – define quality measures;
• – conduct evaluations.

Note that the ISO 25000 [ISO 14a] standard was selected by the Software Engineering Institute (SEI) as a useful reference for the improvement of performance processes described in the Capability Maturity Model Integration (CMMI^®) model. The CMMI model, which describes software engineering models and standards, will be presented in Chapter 4.

The ISO 25010 standard identifies eight quality attributes for software, as illustrated in Figure 3.4.

To illustrate how this standard is used, we will describe the characteristic of maintainability, which has five sub-characteristics: modularity, reusability, analyzability, modifiability, and testability (see Table 3.1). Maintainability is defined, under ISO 25010 [ISO 11i], as being the level of efficiency and efficacy with which software can be modified. Changes may include software corrections, improvements, or adaptation to changes in the environment, requirements, or functional specifications.

Two different perspectives, the internal and external points of view, of the maintainability of the software are often presented in software engineering publications [LAG 96]. Approached from an external point of view, maintainability attempts to measure the effort required to troubleshoot, analyze, and make changes to specific software. From an internal point of view, maintainability usually involves measuring the attributes of the software that influence this change effort. The internal measurement of maintainability is not a direct measurement, that is, a single measurement for software cannot be used and multiple attributes must be measured to draw conclusions about its internal maintainability [PRE 14].

Note that the ISO 25010 standard proposes a wide range of measures for maintainability: size (e.g., number of lines of modified code), time (internal—in terms of software execution, and external—as perceived by the client), effort (individual or for a task), units (e.g., number of production failures, number of attempts to correct a production failure), rating (results of formulas, percentages, or ratios for several characteristics or types of measures, e.g., correlation between the complexity of the software and the testing effort when modifying the software).

From an internal point of view, in order for software to be maintainable, designers must pay special attention to its architecture and internal structure. Architecture and software structure measures are generally extracted from the source code through observations of the characteristics represented in the form of a graph¹ describing its classes, methods, programs, and functions of the source code of programs. Graph studies help to determine the level of complexity of the software. Even today, there are a large number of publications focused on the static and dynamic evaluation of source code. These studies are inspired by those performed in the 1970s by McCabe (1976) [MCC 76], Halstead (1978) [HAL 78], and Curtis (1979) [CUR 79].

Today we see a large number of commercial software programs and open-source, such as Lattix, Cobertura, and SonarQube, which can measure the internal characteristics of source code. These products contain ready-to-use measures and also enable the user to design new measures based on their specific requirements. Boloix [BOL 95] specified that it is the interpretation of these measures that is difficult, since they are very specialized, and there are not many mechanisms to summarize information for decision making. Software and SQA practitioners often end up with highly technical measures without a lot of added value that can be communicated directly to management or their clients.

Professor April [APR 00] describes the experience Cable & Wireless Inc. had using three commercial programs used to measure the internal quality of software (Insure++, Logiscope, and Codecheck) and how certain characteristics of maintainability can be quantified from source code. In this way, a well-designed software can be built with independent, modular, and concealed components with clear boundaries and standardized interfaces.

By using the principle of concealing information when designing the software, greater benefits can be obtained when changes are needed during tests or after release. This technique has the added advantage of protecting the other parts of the application when changes are needed. Any “side effects” can thus be reduced as a result of the change [BOO 94].

Routine coupling measures, for their part, help determine whether the software components are independent or not. It is specifically this code measure that helps to identify whether there will be “side effects” upon modifying the source code. A measure of the internal documentation of a given software may also be automatically carried out using these and similar tools. The internal documentation of software helps the maintenance programmer to understand, in more detail, the meaning of the variables and the logic behind a group of instructions. Some measures point to the necessity of using a simple programming style. Certain measures will evaluate whether the programmers upheld programming standards over time.

A substantial part of maintenance costs goes toward functional adaptations, which become necessary given the changes required by users. “According to our observations and data, we also see that well-structured software makes it easier to make changes than with poorly designed software” [FOR 92].

We know that structured programming techniques are based on breaking down a complex problem into simpler parts and force each component to have a single input and a single output. The most common source code measures evaluate complexity and size, and help programmers form an opinion on the number of sets of test cases required as well as the complexity of the decisions in the source code.

Note that certain non-functional requirements can have a negative interaction with each other. For example, for usability and efficiency characteristics, the additional code and the time required to execute this code to increase usability will take up more storage space and entail greater processing times, potentially negatively affecting the efficiency of the code.

Table 3.2 illustrates, for other quality characteristics, the positive (+), negative (−), or neutral (0) interactions. It is important to explain to users that they have choices to make, and that each choice will have implications.

In conclusion, all software quality models have a similar structure with the same goals. The presence of more or fewer factors, however, is not indicative of a good or bad model. The value of a model of software quality is revealed in its practical use. It is important to completely understand how it works as well as the interaction between factors. Today, the model to use is the ISO model, since it represents an international consensus.

What is important for software developers and SQA is using a standardized model on which they can rely. It is important to use the definitions of the model proposed by ISO 25010 with suppliers. In this way, it is not the personal proposal of a specialist, but the use of a model and definitions that are published internationally and are indisputable.

3.3.1 Specifying Quality Requirements: The Process

Quality is often specified or described informally in RFPs, a requirements document, or in a systems requirement document. The software designer must interpret each functional and non-functional requirement to prepare quality requirements from these documents. To do this, he must follow a process. The quality requirements specifications process will allow for:

• – correctly describing quality requirements;
• – verifying whether the practices in place will allow for the development of software that will meet the client's needs and expectations;
• – verifying or assessing that the software developed meets the quality requirements.

To identify quality requirements, the software engineer will have to carry out the steps described in Figure 3.6. These steps can be done at the same time as defining the functional requirements.

A non-functional requirement is something required but that does not add business rules in a business process. Some examples of a non-functional requirement include the number of users and transactions supported, transaction response time, disaster recovery time, and security. Therefore, all ISO 25010 quality characteristics and sub-characteristics fall under non-functional or performance requirements.

We will focus here on describing the quality aspects expected by clients that would not necessarily be discussed by business analysts during business requirements discussions.

The importance of non-functional requirements must not be underestimated. When developing the requirements of a system, greater importance is often given to business and functional requirements and describing the processing that the system should provide so that users can complete their tasks. For example, an unstable system or a system whose interface is greatly lacking but that fully meets functional requirements cannot be deemed a success.

It is the responsibility of the software engineer to look at each quality characteristic in the ISO 25010 model and discuss whether it should be taken into account in the project. Quality requirements should also be verifiable and stated clearly in the same way as functional requirements. If the quality requirements are not verifiable, they could be interpreted, applied, and evaluated differently by different stakeholders. If the software engineer does not clarify non-functional requirements, they could be overlooked during software development.

The first activity in Figure 3.6 describes identifying stakeholders. In fact, stakeholders are any person or organization that has a legitimate interest in the quality of the software. A certain number of stakeholders will express different needs and hopes based on their own perspective. These needs and hopes may change during the system life cycle and must be checked when there is any change.

Stakeholders rarely specify their non-functional requirements since they only have a vague understanding of what quality really is in a software product. In practice, they perceive them more as general costs. It is important that you remember to include a representative from the infrastructure group, security group, and any other functional group within the company.

For subcontracted projects, quality requirements will often appear as part of the contractual agreement between the acquirer and the representative.

The acquirer may also require a specific evaluation of the product quality when a certain level of software criticality is required and human lives could be endangered.

The second activity in Figure 3.6 involves developing the questionnaire that presents the external quality characteristics in terms that are easy to understand for managers. This questionnaire introduces the quality characteristics and asks the respondent to choose a certain number of them and identify their importance (see Table 3.3).

The third activity consists of meeting with stakeholders to explain to them what is involved in identifying quality characteristics. The next activity consists of consolidating the different quality requirements put forth and incorporating the decisions and descriptions into a requirements document or a project quality plan. Quality characteristics may be presented in a summary table by specifying their importance as, for example, “Indispensable,” “Desirable,” or “Non-applicable.”

Next, for each characteristic, the quality measure must be described in detail and include the following information (see also the cases described in this chapter):

• – quality characteristic;
• – quality sub-characteristic;
• – measure (i.e., formula);
• – objectives (i.e., target);
• – example.

The last step in defining quality requirements involves having these requirements authorized through consensus. After the quality requirements have been accepted, at different milestones of the project, typically when assessing a project step, these measures will be evaluated and compared with fixed and agreed upon objectives in the specifications.

Of course, in order to do this, the measurement must be implemented. Challenges to implementing a measurement program will be described in a later chapter.

3.7 Further Reading

1. GALIN D. Software Quality: Concepts and Practice. Wiley-IEEE Computer Society Press, Hoboken, New Jersey, 2017, 726 p.