Characterization Tests

In his excellent book Working Effectively with Legacy Code [Fea04], Michael Feathers talks about two different types of tests, which he calls specification tests and characterization tests.

Specification tests are the ones we’ve been talking about in the rest of this book. They check that the code does what it’s supposed to. Ideally you write them before you write the code itself and use them as a guide to help you get the code into the right shape.

Characterization tests are different. You can think of them more like a science experiment, where you test the properties of a mysterious substance by boiling it or mixing it with other substances to see how it reacts. With characterization tests, the aim is just to understand what the system currently does.

Characterizations tests apply perfectly to legacy code that has limited or no tests and where the code is hard to read and understand. Here is our recipe, adapted from Michael Feathers’s, for creating a Cucumber characterization test:

1. Write a scenario that exercises some interesting—but mysterious—behavior of your system.

2. Write a Then step that you know will fail.

3. Wire up the step definitions and run the scenario. Let the failure in the Then step tell you what the actual behavior is.

4. Change the failing Then step so that it describes the actual behavior of the system.

5. Repeat.

As an example, let’s suppose we’re making some changes to the checkout system for a supermarket. We’ve been asked to add a new special offer to the system, which was developed a couple of years ago by a firm of expensive management consultants, who are sadly no longer with us. We’ve grabbed the source code, managed to get the system to spin up on our development machine, and have been poking around through the user interface. The code is pretty gnarly and it’s hard to tell exactly what’s going on, but we noticed something about shampoo. It looks like there might be an existing special offer on shampoo bottles already baked into the code, but it’s hard to tell exactly what the rules are. Let’s write a test:

We know that’s going to fail: they’re not going to give us the shampoo for free, are they? (Steps 1 and 2.)

So, we wire up the step definitions to the system and run the scenario. Here’s what happens:

A-ha! Now we know what price the system is charging for two $1.99 bottles of shampoo: $1.99. (Step 3.)

Now we update the Then step of our scenario to reflect our new understanding of what the system does (step 4):

We run the scenario again, and this time it passes. Great! We’ve added our first characterization test. It’s not much, but it’s a start, and we know that whatever we do to the code from now on, we’ll always have this scenario to tell us whether we break this particular aspect of its behavior.

It looks like there’s a buy-one-get-one-free offer on shampoo, but we can’t be sure from this single example. Following the recipe, we now need to repeat steps 1 to 4 and add some more scenarios to give us some more clues as to what the system is doing.

We can refactor the scenario into a scenario outline (see Chapter 5, ​Expressive Scenarios​) to allow us to try different amounts and prices:

We can now add examples into the table, one at a time. Each time we start with a silly value for the total price and then run the scenario and let it tell us what the real total is. Then we update the scenario to document that behavior.