Test-Driving vs. Testing

We need to test-drive more of the consonant conversion logic in order to generalize our solution. Should we add an assertion to ReplacesConsonantsWithAppropriateDigits, or should we create an additional test?

The rule of thumb for TDD is one assert per test (see ​One Assert per Test​ for more information on this guideline). It’s a good idea that promotes focusing on the behavior of the tests, instead of centering tests around functions. We will follow this rule most of the time.

An assertion that represents encoding a second consonant doesn’t seem like distinct behavior. Were we to create a new test, how would we name it? ReplacesBWith1, ReplacesCWith2, and so on...yuk!

We make the rare choice of adding a second assertion, representing a discrete test case, to the test. We’d prefer that if one assertion fails, the others still execute. To accomplish that goal, we use the EXPECT_THAT macro provided by Google Mock, instead of ASSERT_THAT.

The second consonant drives in the need for an if statement to handle the special case.

We add a third data case.

The need for a third consonant makes it clear that we need to replace the if with a hash-based collection.

Now we need to code support for the rest of the consonant conversions. The question is, do we need to test-drive each one?

A mantra surfaced in the early TDD days that says, “Test everything that can possibly break.” This is a glib response to the oft-asked question, “What do I have to test?” Realistically, coding encodings map is a low-risk activity. It’s unlikely we’ll break anything in doing so.

A counterargument is that you can break just about anything, no matter how simple (and I’ve done it). As the tedium of entering repetitive data increases, so does our likelihood to make a mistake and not even notice it. Having tests would decrease the chance that we create a defect.

Tests would provide a clear document of the conversions (though you could argue that the table itself is the clearest document). On the flip side, were we to create a table with hundreds of elements, having a test for each would be ridiculous.

What’s the right answer? Maybe the most important consideration is that we are test-driving, not testing. “Is there a difference?” you ask. Yes. Using a testing technique, you would seek to exhaustively analyze the specification in question (and possibly the code) and devise tests that exhaustively cover the behavior. TDD is instead a technique for driving the design of the code. Your tests primarily serve the purpose of specifying the behavior of what you will build. The tests in TDD are almost a by-product of the process. They provide you with the necessary confidence to make subsequent changes to the code.

The distinction between test-driving and testing may seem subtle. The important aspect is that TDD represents more of a sufficiency mentality. You write as many tests as you need to drive in the code necessary and no more. You write tests to describe the next behavior needed. If you know that the logic won’t need to change any further, you stop writing tests.

Of course, real experience provides the best determinant. Test-driving for confidence works great until you ship a defect. When you do, remind yourself to take smaller, safer steps.

We choose to test-drive. We complete the conversion table.

What about the tests? Do we need three assertions in ReplacesConsonantsWithAppropriateDigits? To answer that question, we ask ourselves whether having the additional assertions provides increased understanding of how the feature works. We answer ourselves: probably not. We eliminate two assertions, change the remaining one to use ASSERT_THAT, and choose a different encoding just to bolster our confidence a little.