Appendix A. Other Issues

“This sentence contradicts itself—or rather—well, no, actually it doesn’t!”

Douglas Hofstadter

We’re done with acceptance testing. No, not really. Here are a few more issues that didn’t seem to fit in the other chapters.

Context

When developing Sam’s system, the requirements and tests—check-out and check-in—were created first. The existence of CDs and customers were a given. Those tests form a context for other requirements and tests. Because entities such as CDs and customers are needed, there is an implicit requirement for a means to get them into the system.¹ The existence of those entities implies that we probably need Create-Read-Update-Delete (CRUD) functionality for each of them, as shown in Chapter 12, “Development Review.” Because the form of acceptance tests for this functionality is fairly consistent, they may be created by just the developer or tester and reviewed by the others. However, a higher level of business rules may need to be applied to the operations. For example, the customer may want to limit access to the operations. The CRUD functionality gives the context for these limitations. A table can show the permissions for every user type, such as:

Another approach would be to create an entity, as a CD; determine its security requirements; and then do the CRUD functionality. Then repeat the sequence for the other entities.

Customer Examples

All examples that a customer creates can become tests [Marick01]. However, not all tests are created directly from examples. Some, like the previous entity tests, are indirectly developed. If the number of tests starts to cause confusion, you can separate test cases into those that come from customer-provided examples and those created by the tester or developer to check other states or calculations.

Fuzzy Acceptance Tests

Suppose Sam wants to suggest CDs that a user might rent. This could be both a user benefit and a sales tool. It’s unclear exactly what CDs should be suggested. This is the case of a story that does not have absolute results. It’s fuzzy.

You can come up with algorithms for determining CDs to suggest. You can test the implementation of these algorithms to see that they come up with the expected results. That is a definitive test.

However, it’s difficult, if not impossible, to determine whether the resulting suggestions help meet Sam’s expectations of more sales. Whether a suggestion grabs a customer enough to rent the CD cannot be measured with the types of acceptance tests we’ve been describing. It’s too fuzzy to test.² Another means for measuring the results is needed.³

Acceptance Test Detail

A customer such as Sam may know what he wants but not be able to specify all the detail. For example, he might want the check-out to be faster than his competitors, but he doesn’t have the information on how fast that is. You can still specify acceptance tests without all the detail. You can fill it in later. Or you can use comparative results as shown in Chapter 17, “Decouple with Interfaces,” as the expected outcome.

Requirements and Acceptance Tests

According to the Institute for Electrical and Electronic Engineers (IEEE), a requirement is a “condition or capability needed by a user to solve a problem or achieve an objective” or a “condition or capability met or possessed by a system component to satisfy a contract, standard, specification or regulation.” A requirements document usually should not include ways to implement the requirements or a specific manifestation. Requirements should be testable/verifiable, modifiable, and prioritized. They should be clear, unambiguous, and complete (at the time of implementation). Creating acceptance tests aids in creating requirements that meet these characteristics.

The “manifestation” is how a system appears either externally or internally; it is a result of design and implementation. Requirements may include constraints on manifestation. Examples of external constraints are “The user interface shall follow the corporate web standard” or “The interface to the merchant bank shall follow its standards.” Internal constraints may include specific implementation or design manifestations, such as “code in Java J2EE,” “use JavaServer Faces,” or “employ corporate architectural framework.” Internal constraints are usually instituted to reduce the number of technologies that must be maintained. You may create acceptance tests for these constraints. However, tests are often more a subjective measure of how well a manifestation meets the constraints.

Documenting Requirements and Tests

You can use traditional applications—such as Microsoft Word, Rational Requisite Pro, or Hewlett-Packard Quality Center—for keeping requirements and associated tests. Alternatively, you can keep the requirements and tests in an easily editable online format, such as a wiki.⁴ All members of the triad can update the requirements and tests in a more collaborative environment. In either case, you should link the objectives to the features, the features to the stories, and the stories to the acceptance tests. As an application becomes larger, there becomes a greater need to keep the documentation well organized and cross-referenced.

Decoupling Requirements

Decoupling tests from each other helps to decouple requirements from each other. This decoupling follows along the same lines as Larry Constantine’s [Constantine02] decoupling of program modules that makes for higher quality code. Decoupled requirements are easier to test. There is always a possibility that requirements are coupled to each other. Implementing one requirement may break the implementation of another requirement unless this dependency is recognized in advance. An example of coupled requirements was given in the highly available disk storage systems in Chapter 12.

Separation of Issues

Creating customer data that matches a particular business rule can be difficult. The initial discount example in Chapter 4, “An Introductory Acceptance Test,” had discount levels based on whether a customer was good or excellent. There was no specification given for what determines a good or excellent customer. Suppose that a business rule using past order history determined the customer rating. (This rule might be simpler than Sam’s rules in Chapter 13, “Simplification by Separation,” for determining whether a customer could reserve.)

Without worrying about how a good/excellent customer is determined, you can easily verify the discount business rule. You can then create good, excellent, and regular customers with the history required to match the customer rating rule and run the business rule against those customers. Then you should run one or more customers through the full test (the first test variation in Chapter 4).

Tom might come up with other tests for the customer rating rule. For example, he might run it against the entire production set of customers to see how many customers matched each rating. That test could suggest to the business representative in Chapter 4 that a report of that type might come in handy.

Testing Systems with Random Events

Many systems have random events to which they respond. For example, the sequence of check-outs and check-ins can vary dramatically. You can test the functionality of the response to each event. You can also test the correct functionality of a particular sequence of events. However, the testing of any random sequence is more difficult to test. In particular, testing an implementation in which timing of the events may expose a defect is a quality attribute issue. The types of tests to uncover defects caused by errors in technical areas such as threading or locking are beyond the scope of this book. See [IBM02] for more information.

The Power of Three

The triad represents one manifestation of the power of three. The number three occurs often in the world. Jerry Weinberg suggests that you should create at least three ways to implement a requirement so that you can make a design between them. You could explore at least three forms of tables to see which one is most suitable for the customer.

The number three appears in many solutions in the non-software world. For example, there are three ways to save on transport costs if you are shipping a product. You can ship the product in larger batches; you can send it by a slower method; or you can build it closer to your customers. With at least three alternatives, you have an opportunity to compare, contrast, and pick the best or merge the three options into a better one.

Summary

• Requirements and tests form a context for other requirements and tests.

• Separation of issues and decoupling requirement makes for easier testing.