2
Software Quality Attributes

When one buys a car, one may be interested in any number of quality attributes, including purchase price, maintenance cost, reliability, safety, fuel efficiency (mpg), engine size/power, acceleration (0–60 mph), comfort, roominess, looks, noise level, trunk size, environmental impact, and so on. Likewise, a software product may feature many quality attributes, which we review briefly in this chapter, focusing in particular on those that may be of interest with respect to software testing. We distinguish between five categories of software quality attributes which are as follows:

  • Functional attributes, which characterize the input/output behavior of the software product
  • Operational attributes, which characterize the operational conditions of the software product
  • Usability attributes, which characterize the extent to which the software product can be used and adapted to user needs
  • Business attributes, which characterize the cost of developing, using, and evolving the software product
  • Structural attributes, which characterize the internal structure of the software product

We consider these families of attributes in turn, in this chapter. For each attribute, we will present a definition and, possibly, a quantification (i.e., a way to measure the attribute quantitatively). This classification is not perfectly orthogonal, and many attributes may be listed in more than one category; but it helps us to define some structure in the set of attributes.

2.1 FUNCTIONAL ATTRIBUTES

Functional attributes characterize the input/output behavior of software products. We distinguish between two broad categories of functional attributes: those that are of a Boolean nature (a software product has them or does not have them) and those that are of a statistical nature (a software product has them to a smaller or larger extent). The functional attributes of a software product depend on the existence of a specification, which describes a set of situations the product is intended to face, along with a prescription of correct program behaviors for each situation. We refer to the set of relevant situations as the domain of the specification.

2.1.1 Boolean Attributes

We recognize two attributes of a Boolean nature in a software product, which are as follows:

  • Correctness, which is the property that the software product behaves according to its specification for all possible situations in the domain of the specification.
  • Robustness, which is the property that the software product behaves according to its specification for all possible situations in the domain of the specification and behaves reasonably for situations outside the domain of the specification. Of course, reasonable behavior is not a well-defined condition, hence robustness is only partially defined; but it generally refers to behavior that alerts the user to the anomaly of the situation and acts prudently and conservatively (avoiding irreversible operations, avoiding irretrievable losses of information, etc.).

As defined, robustness logically implies correctness: whereas correctness refers solely to the behavior of the software product within the domain of the specification, robustness also refers to the behavior of the product outside the domain of the specification. Conversely, we can argue that robustness is not distinguishable from correctness since it is merely correctness with respect to a stronger specification (one that specifies the behavior of candidate programs inside the original specification’s domain, as well as outside it); nevertheless, for a given specification, these are distinct properties.

2.1.2 Statistical Attributes

Correctness and a fortiori robustness are notoriously difficult to establish for software products of any realistic size; hence we introduce statistical attributes, which measure (over a continuum) how close a software product is to being correct or robust. We distinguish between two broad families of statistical attributes, depending on whether the obstacles to correctness and robustness are inadvertent (product complexity, programmer incompetence, etc.) or voluntary (malicious attempts to cause product failure).

  • Dependability: Dependability is the probability that the system behaves according to its specifications for a period of operation time. We recognize two attributes within dependability, namely, reliability and safety, that differ by the stakes attached to satisfying the specification.
    • Reliability reflects the probability that the software product operates for a given amount of time without violating its specification.
    • Safety reflects the probability that the software product operates for a given amount of time without causing a catastrophic failure.
    • Both reliability and safety are related to the product’s ability to operate according to its specification; but whereas reliability reflects the product’s ability to adhere to all the clauses of its specification, safety focuses particularly on high-stakes clauses, whose violation causes a catastrophic loss, in terms of human lives, mission success, high financial stakes, etc. Safe systems are sometimes referred as fail-safe, in the sense that it may fail to satisfy its specification but still satisfy the high-stakes requirements of its specification. A system may be reliable but unsafe (fails seldom but causes a catastrophic loss whenever it fails); and a system may be safe but unreliable (fails often, but causes low-stakes losses, and never causes catastrophic losses).
    • The most commonly used metric to quantify reliability is the mean time to failure, which is the mean of the random variable that represents the operation time until the next system failure; the same metric can be used to quantify safety, if we just replace failure by catastrophic failure. Older metrics include the mean time between failures, which is the mean of the random variable that represents the time between two successive failures. More recent metrics include the mean failure cost, which measures the mean of the random variable that measures the loss of a stakeholder as a result of possible system failures.
    • Security: Whereas dependability refers to failures that result from system design flaws, security refers primarily to voluntary actions by malicious perpetrators, although one can argue that these actions are rendered possible by system vulnerabilities, which also stem from system design flaws. We find four attributes that can be considered as aspects of security, which are as follows:
      • Confidentiality: Confidentiality refers to a system’s ability to prevent unauthorized access to confidential data entrusted to its custody.
      • Integrity: Integrity refers to a system’s ability to prevent loss or damage to critical data entrusted to its custody.
      • Authentication: Authentication refers to a system’s ability to properly identify each user that gains access to its resources and to grant users access privileges according to their rightful status.
      • Availability: Availability refers to a system’s ability to continue delivering service to its user community; it can be measured as a percentage. This attribute is usually the casualty of denial of service attacks: when the system is under attack, its ability to deliver services to its legitimate users suffers.
      • There is no widely accepted measure of system security. To the extent that security attacks result from system vulnerabilities, it is possible to quantify all dimensions of security (including availability) by mean time to detection (MTTD), which is the mean of the random variable that measures the time it takes perpetrators to uncover system vulnerabilities, and by mean time to exploitation (MTTE), which is the mean of the random variable that represents the time it takes perpetrators to find a way to exploit discovered system vulnerabilities.

2.2 OPERATIONAL ATTRIBUTES

Whereas functional attributes characterize the functions/services that a software product delivers to its users, operational attributes characterize the conditions under which these services are delivered. We find four attributes that can be considered as operational attributes, which are as follows:

  • Latency: Latency (or response time) is the time that elapses between the submission of a query to the system and the response to the query; this attribute is relevant for interactive systems and can be measured in seconds. Because the time between a query and its response varies, of course, from query to query, and varies according to the system workload at the time the query is submitted, this attribute is understood to be an average over many queries, for an average workload.
  • Throughput: Throughput is the volume of processing that the system can deliver per unit of operation time; this attribute is relevant for batch systems, such as the program that a bank runs in the middle of the night to update all the transactions of the day, and can be measured in transactions per second. Though latency and throughput appear to be related, as they both reflect the speed with which the system can process transactions, they are fairly orthogonal, in the sense that a system can have short (good) latency and small (bad) throughput (if it spends much of its time switching between queries), and long (bad) latency and large (good) throughput if its scheduler favors keeping the processor busy over the concern of fairness.
  • Efficiency: The efficiency of a software product is its ability to deliver its functions and services with minimal computing resources, such as CPU cycles, memory space, disk space, and network bandwidth. It is difficult to quantify this attribute for a given software product in a way that prorates resource consumption to the services rendered by the product, but very easy to use this attribute to compare candidate software products for a given application.
  • Capacity: The capacity of a system is the number of simultaneous users that a system can sustain while preserving a degree of quality of service (in terms of response time, timeliness, precision, size of data, etc.). This definition is ambiguous in a number of ways (what does it means to be simultaneous? At what level of granularity are we defining simultaneity? What degree of quality are we considering? etc.), but is a useful metric nevertheless.
  • Scalability: The scalability of a system reflects its ability to continue delivering adequate service when its workload exceeds its original capacity. This attribute is sometimes referred to as graceful degradation: of course, as the workload increases beyond the system’s preplanned capacity, one expects some degradation of the quality of service; scalability consists in ensuring that this degradation remains graceful (i.e., a progressive, continuous function of the workload).

2.3 USABILITY ATTRIBUTES

Whereas functional attributes measure the services rendered by the software product to the end user, and operational attributes reflect the operational conditions under which these services are rendered, usability attributes reflect the extent to which the software product is easy to use, and to customize to the end user’s needs and circumstances. We identify five such attributes, which are as follows:

  • Ease of Use: Ease of use is important especially in systems whose user community is broad, diverse, heterogeneous and possibly unskilled; this includes, for example, an online system in a public space. Qualities that support ease of use include simplicity of system interactions, uniformity of interaction patterns, availability of help menus, use of simple vocabulary, tolerance to misuse, and so on.
  • Ease of Learning: Ease of learning is important especially in systems whose user community is homogeneous in terms of skill level (e.g., all the clerical staff of an organization) and who envisions to use the software product on a long-term basis (e.g., an accounting application). Qualities that support ease of learning include intuitiveness of system interactions, consistency of interaction protocols, uniformity of system outputs, and so on.
  • Customizability: The customizability of a software product is its ability to be tuned to specific functional requirements of the end user, by the end user. The more control the end user has over the functionality of the software product, the better the customizability.
  • Calibrability: The calibrability of a software product is its ability to be tuned to specific operational requirements of the end user, by the end user. The more control the end user has over the operational attributes of the software product, the better the calibrability.
  • Interoperability: The interoperability of a software product is its ability to work in conjunction with other applications; it is difficult to give a general quantitative metric for this attribute, but it can be qualitatively characterized by the range of applications with which it can collaborate (for example, the breadth of file formats it can analyze and process, or by the range of file formats in which it can produce its output); it can also be qualitatively characterized by the range of technologies with which it is compatible.

The functional attributes, the operational attributes, and the usability attributes of a software product constitute all the attributes of the product that are of interest to the end user; in the remainder of this chapter, we consider other stakeholders of a software product, and review attributes that are of interest to them.

2.4 BUSINESS ATTRIBUTES

Broadly speaking, business attributes reflect the stakes of a software manager in the development, operation and evolution of a software product. They include the following attributes:

  • Development Cost: Development cost is clearly an important attribute, and a business attribute at that. We usually quantify it in person-months invested in the development of the software product, from its requirements analysis to its acceptance testing.
  • Maintainability: The maintainability of a software product is the amount of effort invested in the maintenance of the product during its operation phase (post delivery). We can quantify it in absolute terms, by means of person-months of maintenance effort per year, or in relative terms, by means of person-months of maintenance effort par year per line of code. Since most maintenance costs are typically spent on adaptive maintenance, which is driven by user demands rather than by any intrinsic attribute of the software product, we may want to normalize the measure of maintainability to the volume of adaptive maintenance, or exclude adaptive maintenance altogether from the person-month calculation of maintenance effort.
  • Portability: The portability of a software product is the average cost of porting the product from one hardware/software platform to another. Portability is enhanced by reducing the platform-dependent functionality as much as possible, or by confining it to a single component of the product (hence confining changes to this component in the event of a migration of the product from one platform to another). Though it is usually thought of as a qualitative attribute, portability can be quantified by the average cost (in person-months) of a migration.
  • Reusability: Whereas the three previous attributes reflect product cost, this attribute reflects potential benefits. The reusability of a software product is its ability to be reused, in whole or in part, in the design and development of other software products within the product’s application domain. Reusability is the aggregate of two orthogonal properties, which are as follows:
    • Usefulness, which is the extent to which the product (or component) is widely needed in the product’s application domain
    • Usability, which is the ease (lower cost) with which it is possible to adapt the product (or component) to the requirements of an application within the domain
    Whereas usefulness is a property of the product’s (or component’s) specification, usability is a property of its design: well-designed components can be adapted to related requirements at low cost, by such devices as genericity and parameterization.

2.5 STRUCTURAL ATTRIBUTES

Like business attributes, structural attributes are of interest to system custodians/developers/maintainers/operators; but whereas business attributes deal with economic aspects of system management, structural attributes deal with their technical aspects. In other words, whereas business attributes are relevant to software managers, structural attributes are of interest to engineers, designers and other technical personnel. We identify four structural attributes, which are as follows:

  • Design Integrity: The quality of a design is easier to recognize than to define, which is in turn much easier than to quantify. Qualities of a good design include simplicity, orthogonality (the quality of a design that results from a set of independent decisions), economy of concept, cohesiveness of the design rationale, consistency of design rules, adherence to a simple design discipline, and so on.
  • Modularity: Modularity can be defined in terms of a single design principle: information hiding. The main characteristic of a modular design is that each component of the system hides a design decision that other components need not know about (i.e., be influenced by). Hence one of the main features of a modular design is the separation between the specification of a module and its implementation. Modularity can be quantified by two attributes, which are as follows:
    • Cohesion: The cohesion of a component represents the volume of information flow within the component, and can be quantified using information theory.
    • Coupling: The coupling between two components represents the bandwidth of information interchange that takes place between the components, and can be quantified by the entropy of the random variable that represents the flow of information interchange.
  • Testability: The testability of a software product reflects the extent to which one can test the system or components thereof to an arbitrary level of thoroughness. Testability can be quantified by means of two attributes, which are as follows:
    • Controllability: The controllability of a component in a system is the bandwidth (breadth) of input values we can submit (as test data) to the component by controlling system inputs. This can be quantified by the conditional entropy of the system’s input given an input value of the component.
    • Observability: The observability of a component is the extent to which we can infer the output produced by the component by observing the system output. This can be quantified by the conditional entropy of the component’s output, given the system’s output.
  • Adaptability: The adaptability of a system is the ease with which it can be modified to satisfy changing requirements. This attribute sounds similar to customizability, in that they both deal with adjusting the software product to meet different sets of requirements. In fact they are different in two crucial ways, which are as follows:
    • Whereas customizability refers to changes carried out by the end user (hence accessible to her/him, by design), adaptability refers to changes carried out by the software engineer.
    • Whereas customizability refers to changes that are planned for within the design of the system, adaptability refers to changes in the system requirements.

In many instances, we find that structural attributes support business attributes; for example, modularity supports maintainability. But we consider them as distinct, on the grounds that one is a business attribute while the other is a structural/technical attribute. Also, there is no one-to-one correspondence between them: modularity supports not only maintainability but also development cost (which it reduces); also, maintainability is supported not only by modularity but also by testability.

2.6 CHAPTER SUMMARY

This chapter provides us a vocabulary for discussing software properties and enables us to explore the effects of software testing on achieving or measuring these properties, as we will do in subsequent chapters. We have presented five classes of software quality attributes, which we divide broadly into three categories, which are as follows:

  • Attributes that are relevant to software users: These include functional attributes, operational attributes, and usability attributes and reflect the overall quality of service delivered by the software product.
  • Attributes that are relevant to software operators: These include business attributes and reflect the cost of developing, maintaining, and evolving the software product.
  • Attributes that are relevant to software engineers: These include structural attributes and reflect the engineering qualities of the product and their impact on analyzing and working with the software product for the duration of its lifecycle.

2.7 EXERCISES

  1. 2.1. Your colleague Joe argues that since reliability is the probability that the product operates free of failure for some unit of time and safety is the probability that the product operates free of catastrophic failures, and since catastrophic failures are failures, then a reliable system is necessarily safe. Do you agree with Joe? Why or why not?
  2. 2.2. Can a system be reliable but not secure? Can a system be secure but not reliable?
  3. 2.3. Can a system have high reliability but low availability? Can a system have high availability but low reliability?
  4. 2.4. Consider a queue simulation module M that uses an implementation of an first in/first out (FIFO) queue Q. Using information theory, propose a formula for the cohesion of M and the coupling of M and Q. What information do you need about the implementation of M and Q to compute the cohesion of M and the coupling of M and Q.

2.8 BIBLIOGRAPHIC NOTES

Some of the terminology introduced in Section 2.1, pertaining to functional attributes, is inspired by Avizienis et al. ( 2004 ); because the definitions of Avizienis et al. are too detailed for our purposes, we have not adopted them wholesale but have simplified them a little. The quantification of security by means of MTTD and MTTE is due to Nicol et al. ( 2004 ). For more information on information theory, consult Csiszar and Koerner ( 2011 ).

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.217.107.229