Unit Tests, Integration Tests, and Acceptance Tests

TDD is a programmer practice to help you incrementally drive the design of code. You’ve learned how to use it to verify small bits of C++ logic by writing unit tests, which in turn allows you to continually shape the design of the code at will.

For the purposes of this book, unit means a small piece of isolated logic that affects some systematic behavior. The word isolated in the definition suggests you can execute the logic independently. This requires decoupling the logic from dependencies on things such as service calls, APIs, databases, and the file system. (Technically, independent code should also be decoupled from any other code, but a pragmatic approach suggests that it’s not necessary for all code to be purely isolated.) As emphasized elsewhere in this book (for example, ​Fast Tests, Slow Tests, Filters, and Suites​), the important aspect of unit tests for purposes of doing TDD is that they’re darn fast.

By definition, unit tests are inadequate. Since they verify small isolated pieces of code, they can’t demonstrate the correctness of an end-to-end deployed-and-configured solution. In addition to unit tests, your system requires tests that will provide high confidence that you are shipping a high-quality product. Depending on the shop, these tests include what might be called system tests, customer tests, acceptance tests, load tests, performance test, usability tests, functional tests, and scalability tests, to name a few (some of these are more or less the same thing). All of these tests verify against an integrated software product and are thus integration tests.

Who defines the integration tests varies depending on the circumstance. Typically, they are defined by people in one or more of three roles: tester, programmer, and customer.

Per the Agile community, customer tests are any tests defined to demonstrate that the software meets business needs. In an Agile process, these tests are defined before development in order to provide a specification of sorts to the development team—a close analog to the TDD process. Agile proponents will often refer to customer tests defined up front as acceptance tests (ATs). If the development team builds software that gets all the test to pass, the customer agrees to accept the delivery of the software.

How Do TDD and Acceptance Testing Relate?

Driving the development of a system by defining ATs up front is quite similar to driving the development of bits of logic using TDD. In fact, many teams employing the use of ATs in this manner refer to the process as Acceptance Test--Driven Development (ATDD).

You can read volumes about how acceptance tests might fit into your development process, what kind of tools are used, what the tests should look like, and so on. If you understand TDD, though, you already have most of the mentality needed to understand what ATDD is all about and how to succeed in its practice.

The important distinction between TDD and ATDD is who defines the tests (and therefore who consumes the tests). With TDD, programmers are responsible for defining unit tests in a programmatic language. As such, there’s no expectation that anyone but programmers will read or otherwise consume the unit tests. (That doesn’t mean you can slouch on their readability, however!)

With ATDD, customers (which might include people like product owners or business analysts) are responsible for defining ATs based on business needs. They rarely do this in a vacuum. Insights and information from everyone else on the team, including testers and programmers, are required to build robust acceptance tests. Everyone consumes tests created using ATDD.

You will find a number of books on ATDD, including Specification by Example [Adz11] and ATDD By Example [Gä12]. When searching, you may also find relevant information using the search phrase specification by example.

Programmer-Defined Integration Tests

As a programmer, you always have the option to create programmer-facing integration tests. A few well-chosen integration tests can be invaluable, and your shop might even require them. You might include tests that exercise your data access layer directly to get more immediate information and more direct failure messages about discrepancies between the code and the data store’s definition. Or, you might use a series of smoke tests to quickly determine whether a deployment isn’t configured properly.

However, integration tests are difficult to maintain. Since they deal with software deployed to a configured environment that must interact with the wild world of external services and data stores, integration tests are brittle. Keeping the tests up-to-date and running across all your deployment environments is a sizable challenge.

You’re best off if you code only as many integration tests as you absolutely need, and no more. As a strategy, either seek to position an integration test as a customer test (that your customer is willing to take on) or remove its dependencies and use it as a unit test that demonstrates code-level logic.

If your team has already produced a number of tests using a unit testing tool like Google Test, you may find that many of them are really integration tests pretending to be unit tests. These tests attempt to verify bits of code logic but have many unfortunate dependencies that relegate them to the realm of slow and brittle integration tests.

Find the time to triage your integration tests. For each, determine one of the following four actions:

• Clean up the integration test and sell it to the business as an AT.

• Convert it to a fast unit test by removing its dependencies.

• Retain it as a rare nonacceptance integration test.

• Delete it.

Immediately remove any remaining integration tests from the fast unit test suite.

Overlap Between TDD and ATDD

One of the bigger sources of anxiety for a team practicing both ATDD and TDD is that some of the tests will inevitably appear to overlap, particularly if everything gets properly test-driven. Most acceptance tests usually represent functional interests and will demonstrate how the system works from the perspective of an actor interacting with the system. When doing TDD, you’ll test-drive this user interface layer also. Is this duplicate effort?

Indeed, the tests at the interface layer will appear similar. However, remember that the audiences and goals for each set of tests are different. No one else will ever read your programmer tests. The acceptance tests, in contrast, are designed to be read by anyone.

The tests that a customer defines will provide broad coverage against a piece of end-to-end functionality. Since they will typically run quite slowly, such tests cannot hope to have high levels of coverage against the countless possible combinations and permutations. You can code TDD-produced unit tests, however, to rapidly slam through a much larger number of combinations and permutations.

From a design stance, the amount of overlap should be low. In a well-designed system, the interface layer is thin and largely a delegator to business domain classes (and there should be many more non-interface-layer classes than interface-layer classes). Thus, most of your unit tests against the interface layer need to simply prove that work was delegated appropriately.

From a pure testing standpoint, the benefit of having both unit tests and acceptance tests provides you with an extra layer of protection. Inevitably, your unit tests will miss covering an important scenario. Having a layer of tests above unit level that are defined by a different set of brains will provide you with an invaluable safety net. You’ll write some tests to represent scenarios the business didn’t think about, and vice versa. (But don’t forget that nets are comprised of lots of gaping holes.)

Defects represent an incorrect or missing unit test. You correct defects by following a test-driven approach. First write a test that demonstrates the existence of the defect by failing. Then write the code that gets the unit test (and any corresponding acceptance tests) to pass.