Introduction

“Context is all.”

Margaret Atwood, The Handmaid’s Tale

The context for the tale is introduced. A brief background of acceptance test–driven development (ATDD) is presented.

Testable Requirements

Developing software with testable requirements is the theme of this book. A testable requirement is one with an acceptance test. Acceptance tests drive the development of the software. As many development groups have experienced, creating acceptance tests prior to implementing requirements decreases defects and improves productivity. (See the Epilogue for examples.) A triad—the customer/business analyst, developer, and tester—collaborates on producing these tests to clarify what is to be done. In creating a high-quality product, ATDD is as much about this clarification as it is about the actual testing.

As an example, do you have criteria in mind as to whether this book will meet your needs? If you finish this book, how will you know whether it has met those criteria? This book represents an implementation of something that should meet your needs. Because you are reading this book after its completion, you don’t have an opportunity to influence the acceptance criteria. But let me list the criteria here and see if this is what you are after.

In English classes, the teacher emphasized that a story should contain a who, what, when, where, why, and how. So I’ve made that the goal of this book. It explains

• Who creates acceptance tests

• What acceptance tests are

• When the acceptance tests should be created

• Where the acceptance tests are used

• Why acceptance test-driven development is beneficial

• How the acceptance tests are created

By the end of this book, the expectation is that you should understand how testable requirements can make the software development process more enjoyable (or at least less painful) and help in producing higher-quality products., Let’s begin with a brief discussion on the why, what, where, and who issues.

Why ATDD Is Beneficial

Let’s start with the answer to the why question. Jeff Sutherland, the cocreator of Scrum, has metrics on software productivity [Sutherland01]. He has found that adding a quality assurance person to the team and creating acceptance tests prior to implementation doubles the team’s productivity. Your actual results may vary, but teams adopting ATDD have experienced productivity and quality increases. Mary Poppendieck says that creating tests before writing code is one of the two most effective and efficient process changes for producing quality code. (The other is frequent feedback.) [Poppendieck01] Customer-developer-tester collaboration reduces unnecessary loops in the development process. As Jerry Weinberg and Don Gause wrote, “Surprising, to some people, one of the most effective ways of testing requirements is with test cases very much like those for testing the completed system” [Weinberg01].

If you are going to test something and document those tests, it costs no more to document the tests up front than it does to document them at the end. But these are more than just tests. As stated in Chapter 3, “Testing Strategy,” “The tests clarify and amplify the requirements.” An acceptance test is “an authoritative and reliable source of what the software should do functionally” [Adzic01].

What Are Acceptance Tests?

Acceptance tests, as used in this book, are defined by the customer in collaboration with the developer and tested and created prior to implementation. They are not the traditional user acceptance tests [Cimperman01], which are performed after implementation “by the end user to determine if the system is working according to the specification in the contract.” [Answers01] They are also not system tests which are usually independently written by testers by reading the requirements to ensure that the system meets those requirements. [Answers02] All three are related in that they are all black box tests—that is, they are independent of the implementation. It is the time and manner of creation in which they differ.

Where Are Acceptance Tests Used?

The concept of an acceptance test is defined by the intent of the test, not its implementation. You can apply an acceptance test at the unit, integration, or user interface level. You can use it as a validation test, which allows input to or produces outputs from an application installed in the customer’s environment. Further, you can use it as a design verification test that ensures a unit or component meets it intended responsibility. In either case, the test makes certain the application is acceptable to the customer.

Who Creates the Acceptance Tests?

This book refers to a triad: the customer, developer, and tester. The power of three people working together [Crispin01] can create the bests acceptance tests.

If the triad writes the tests together, the distinction between user acceptance tests and system tests is practically eliminated. As will be shown, the three roles of customer, developer, and tester may be played by different individuals or by the same individual with different focuses.

What Types of Software Are Covered?

The acceptance tests covered in this book revolve mainly around requirements that have determinable results. These results are typical in business situations. You place an order, and the order total is determinable. On the other hand, you have a requirement to find the shortest possible path that goes through a number of points. For example, you want to determine the shortest driving trip that travels over every road in the United States. For a small number of roads (such as the interstate highways), the result is determinable by brute force. However, for a large number of roads, the answer is not determinable. You can have a test that checks the output of one way of solving the problem against the output of another way. But that does not guarantee that the shortest solution has been found.

How Will We Get to ATDD?

The answers to how and when the acceptance tests should be created are shown by a continuous example throughout this book. Each step in their creation and use is covered. Some books are devoted entirely to a single step and go into much greater detail than does this book. In particular, the references offer links for tools to automate the acceptance tests, to the agile process itself, to requirement elicitation, and to testing the other qualities of a software system (usability, performance, and so on).

The continuous example for Sam’s CD Rental Store follows Sam’s story in Prefactoring—Extreme Abstraction, Extreme Separation, Extreme Readability. That book used the tale as the context for examples of good design. Prefactoring covered some of the aspects of developer-customer interaction, because a good design requires understanding the customer’s needs. Prefactoring’s focus was on the internal software quality. This book’s focus is on externally visible quality. The two books complement each other.

Organization

The material is presented in six parts. The first part documents the tale of the triad members—customer, developer, tester—as they create a software system. It shows how acceptance testing permeates the entire process, from the project charter to individual stories. The second part covers details in acceptance testing, as simplification by separation. The third part explores general subjects, such as test presentation and valuation. The fourth part includes case studies from real-life situations. In some instances, the studies have been simplified to show only the relevant parts. The fifth part involves more technical issues, as how to handle test setup. The sixth part offers the appendices, which give additional information on topics as business value and test automation. For those who want to get the quick summary of ATDD and its benefits, read Chapter 25, “Retrospective and Perspective.” Those who want to read the experiences of others, see the Epilogue.

Example Tables

The book presents tests with examples in tables rather than in narrative form. These tables follow the concepts of David Parnas, who states, “The tables constitute a precise requirements document” [Parnas01]. Some people prefer free text over tables. Those who prefer the narrative can easily convert tables to this form. The reverse is usually more difficult. Tables are familiar to spreadsheet users. Many business rules have conditions that are more easily tested with a table. From an analysis point of view, you can often find missing conditions by examining the values in a table’s columns.

Automation After Communication

I emphasize acceptance tests as customer-developer-tester communication. If you don’t have an acceptance test, you have nothing to automate. I do not advocate a particular test automation framework. When you automate an acceptance test that includes its accompanying requirement, you get what many term an executable specification [Melnik02], [Melnik03].

Acceptance tests can be manual. But if they are automated, you can use them as regression tests to ensure that future changes to the system do not affect previously implemented requirements. So the most effective use of the tests is as an executable specification. Appendix C, “Test Framework Examples,” shows examples of test automation using several frameworks. The code for the examples is available online at http://atdd.biz.

ATDD Lineage

A Chinese proverb says, “There are many paths to the top of the mountain, but the view is always the same.” And many of the paths share the same trail for portions of the journey. Although acceptance testing has been around for a long time, it was reinvigorated by extreme programming [Jefferies01]. Its manifestations include ATDD as described in this book, example-driven development (EDD) by Brian Marick [Marick01], behavior-driven development (BDD) by Dan North [Chelimsky01], story test-driven development (SDD) by Joshua Kerievsky of Industrial Logic [Kerievsky01], domain-driven design (DDD) by Eric Evans [Evans01], and executable acceptance test-driven development (EATDD) [EATDD01]. All these share the common goal of producing high-quality software. They aid developers and testers in understanding the customer’s needs prior to implementation and customers being able to converse in their own domain language.

Many aspects are shared among the different approaches. ATDD in this book encompasses aspects of these other approaches. I’ve documented the parts that come specifically from the other driven developments (DDs), including Brian Marick’s examples, Eric Evan’s ubiquitous language, and Dan North’s given-when-then template. The most visible differences are that the tests here are presented in table format rather than in a more textual format, such as BDD’s Cucumber language, and they concentrate on functionality instead of the user interface. This book’s version of ATDD matches closely that described by Lasse Koskela [Koskela01] and Gojko Adzic [Adzic01] and follows the testing recommendations of Jim Coplien [Coplien01].

One of the most well-known DDs is test-driven development (TDD) by Kent Beck [Beck01]. TDD encompasses the developer’s domain and tests the units or modules that comprise a system. TDD has the same quality goal as ATDD. The two interrelate because the acceptance tests can form a context in which to derive the tests for the units. TDD helps creates the best design for an application. A TDD design issue would be assigning responsibilities to particular modules or classes to pass all or part of an acceptance test.

Summary

• Testable requirements have acceptance tests associated with them.

• ATDD involves developing requirement tests prior to implementation.

• ATDD can improve productivity.

• Acceptance tests are developed collaboratively between the customer, developer, and tester.