Getting Green on Red

The first rule of TDD requires you to first demonstrate test failure before you can write any code. In spirit, this is a simple rule to follow. If you write only just enough code to make a test to pass, another test for additional functionality should fail automatically. In practice, however, you’ll find yourself sometimes writing tests that pass right off the bat. I refer to these undesired events as premature passes.

You might experience a premature pass for one of many reasons:

• Running the wrong tests

• Testing the wrong code

• Unfortunate test specification

• Invalid assumptions about the system

• Suboptimal test order

• Linked production code

• Overcoding

• Testing for confidence

Running the Wrong Tests

The first thing to do when sitting down is run your test suite. How many tests do you have? Each time you add a new test, anxiously await the new count when your run your test suite. “Hmm...why am I getting a green? Well, I should have forty-three tests now...but wait! There are only forty-two. The test run doesn’t include my new test.”

Tracking your test count helps you quickly determine when you’ve made one of the following silly mistakes:

• You ran the wrong suite.

• Your Google Mock filter (see ​Running a Subset of the Tests​) omits the new test.

• You didn’t compile or link in the new test.

• You disabled (​Disabling Tests​) the new test.

If you don’t first observe test failure, things get worse. You write the test, skip running the test suite, write the code, and run the tests. You think they’re all passing because you wrote the right code, but in reality they pass because the test run doesn’t include the new test. You might blissfully chug along for a good while before you discover that you’ve coded a broken heap of sludge.

Keeping to the TDD cycle can prevent you from wasting a lot of time.

Testing the Wrong Code

Similar to running the wrong tests, you can test the wrong code. Usually it’s related to a build error but not always. Here are a few reasons why you might end up running the wrong code:

• You forgot to save or build, in which case the “wrong” code is the last compiled version that doesn’t incorporate your new changes. Ways to eliminate this mistake include making the unit test run dependent upon compilation or running tests as part of a postbuild step.

• The build failed and you didn’t notice, thinking it had passed.

• The build script is flawed. Did you link the wrong object module? Is another object module with a same-named module clashing?

• You are testing the wrong class. If you’re doing interesting things with test doubles, which allow you to use polymorphic substitution in order to make testing easier, your test might be exercising a different implementation than you think.

Unfortunate Test Specification

You accidentally coded a test to assert one thing when you meant another.

Suppose we code a test as follows:

Well, no. Portfolios should be empty when created, per the test name; therefore, we should expect the call to isEmpty to return true, not false. Oops.

On a premature pass, always reread your test to ensure it specifies the proper behavior.

Invalid Assumptions About the System

Suppose you write a test and it passes immediately. You affirm you’re running the right tests and testing the right code. You reread the test to ensure it does what you want. That means the system already has the behavior you just specified in the new test. Hmmm.

You wrote the test because you assumed the behavior didn’t already exist (unless you were testing out of insecurity—see Testing for Confidence). A passing test tells you that your assumption was wrong. The behavior is already in the system! You must stop and analyze the system in the light of the behavior you thought we were adding, until you understand the circumstances enough to move on.

Getting a passing test in this case represents a good thing. You’ve been alerted to something important. Perhaps it is your misunderstanding of how a third-party component behaves. Taking the time to investigate may well save you from shipping a defect.

Suboptimal Test Order

The interface for RetweetCollection requires size and a convenience member function called isEmpty. After getting our first two tests for these concepts to pass, we refactor the implementation of isEmpty to delegate to size so that we don’t have variant algorithms for two related concepts.

To expand on the concept of emptiness, we write a subsequent test to ensure a retweet collection isn’t empty once tweets are added to it.

(For now, we don’t care about the content of a tweet, so all we need to supply for the Tweet class definition in Tweet.h is simply class Tweet {};.)

Getting IsNoLongerEmptyAfterTweetAdded to pass is a simple matter of introducing a member variable to track the collection’s size.

But we now have a problem. If we want to explore the behavior of size after adding a tweet, it will pass immediately.

What might we have done to avoid writing a test that passed?

Before answering that question, maybe we need a different perspective. Did we need to even write this test? TDD is about confidence, not exhaustive testing. Once we have an implementation we are confident is sufficient, we can stop writing tests. Perhaps we should delete HasSizeOfOneAfterTweetAdded and move on. Or, perhaps we should retain it for its documentation value.

Otherwise, any time we get a premature pass, we look at the code we wrote to pass the prior test. Did we write too much code? In this case, no, the code was as simple as it could have been. But what if we had explored the emptiness behavior further before introducing the size behavior? With tests in the order IsEmptyWhenCreated, IsNoLongerEmptyAfterTweetAdded, and HasSizeZeroWhenCreated, we would have ended up with a slightly different scenario.

We can’t link size to isEmpty at this point, since we have no tests that define how size behaves after a purchase. Thus, we can’t refactor size; it remains as is with a hard-coded return. Now when we code HasSizeOfOneAfterTweetAdded, it will fail. The simplest implementation to get it to pass is a little odd, but that’s OK!

It’s easy to be lazy as a programmer. You’ll likely resist reverting your changes and starting over in order to find a path that avoids a premature pass. Yet you’ll learn some of the most valuable lessons in the rework. If you choose not to, at least rack your brain and think how you might have avoided the premature pass, which might help you avoid the problem next time.

Linked Production Code

In ​Suboptimal Test Order​, isEmpty is a convenience method designed to make client code clearer. We know it’s linked to the concept of size (we coded it!). The collection is empty when the size is zero and not empty when the size is greater than zero.

Adding a convenience method such as isEmpty creates duplication on the side of the code’s interface. It represents a different way for clients to interface with behavior that we’ve already test-driven. That means a test against isEmpty will pass automatically. We still want to clearly document its behavior, though.

As we add new functionality to RetweetCollection, such as combining similar tweets, we’ll want to verify that the new behavior properly impacts the collection’s size and emptiness. We have several ways to address verifying both attributes.

The first option is for every size-related assertion, add a second assertion around emptiness. This choice produces unnecessarily repetition and cluttered tests.

The second option is for every test around size, write a second test for emptiness. While keeping in line with the notion of one assert per test, this choice would also produce too much duplication.

The third option is to create a helper method or custom assertion. Since the code contains a conceptual link between size and emptiness, perhaps we should link the concepts in the assertions.

The MATCHER_P macro in Google Mock defines a custom matcher that accepts a single argument. See https://code.google.com/p/googlemock/wiki/CheatSheet for further information. You can use a simple helper method if your unit testing tool doesn’t support custom matchers.

The fourth option is to create tests that explicitly document the link between the two concepts. In a sense, these tests are a form of documenting invariants. From there on out, we can stop worrying about testing for emptiness in any additional tests.

(In the expression Gt(0), Gt stands for greater than.)

The asserts that relate to checking size in these two tests act as precondition assertions. They are technically unnecessary but in this case serve to bolster the relationship between the two concepts.

I will usually prefer the fourth option, though sometimes find the third useful. TDD is not an exact science. It provides room for many different approaches, meaning that you must stop and assess the situation from time to time and choose a solution that works best in context.

Overcoding

If you’ve programmed for any length of time, you have a normal tendency to jump right with what you know is needed for an end solution. “I know we’re gonna need a dictionary data structure, so let’s just code in the map now.” Or, “Yeah, that could throw an exception that we need to handle. Let’s code for that case and log the error in the catch block and then rethrow.”

These inclinations and intuitions about what the code needs to do are what makes a good programmer. You don’t have to discard these thoughts that come to mind. Knowing about hash-based structures can help guide you to better solutions, and knowing what errors can arise is important.

But to succeed with TDD, you must ensure that you introduce these concepts incrementally and with tests. Keeping to the red-green-refactor rhythm provides many safeguards and also helps grow the number and coverage of unit tests.

Introducing a map prematurely can present you with premature passes for some time, since it will immediately cover many of the cases your code needs to handle. You might omit a number of useful tests as a result. You can instead incrementally grow the underlying data structure, using failing tests as proof for a need to generalize the solution.

Sometimes you’ll realize that you don’t even need a map. Resisting a more-than-necessary implementation allows you to keep the code simpler in the meantime. You’ll also avoid permanent over-complexity when there’s a simpler end solution.

Writing tests for things like exception handling means that client programmers will have an easier time understanding how to interact with your class. You force yourself to clearly understand and document the scenarios in which problems can occur.

Learning to produce just enough code is one of the more challenging facets of TDD, but it can pay off dramatically. Sticking to red-green-refactor can help reinforce the incremental approach of TDD.

Testing for Confidence

Sometimes you just don’t know what the code does in certain cases. Test-driving generated what you believe to be a complete solution, but you’re not sure. “Does our algorithm handle this case?” You write the corresponding test as a probe of the solution. If it fails, you’re still in the red-green-refactor cycle, with a failing test prodding you to write the code that handles the new case.

If your probe passes, great! The system works as you hoped. You can move on. But should you keep or discard the new test? Your answer should reflect its documentation value. Does it help a future consumer or developer to understand something important? Does it help better document why another test fails? If so, retain the test. Otherwise, remove it.

Writing a test to probe behavior represents an assumption you’re making about the system that you want to verify. You might liken testing for confidence to ​Invalid Assumptions About the System​. The distinction is that every time you write a test, you should know whether you expect it to fail or pass. If you expect a test to fail and you’re surprised when it doesn’t, you’ve made an invalid assumption. If you write a test that you expect to pass, you’re testing for confidence.

Stop and Think

Premature passes should be fairly rare. But they’re significant, particularly when you’re learning to practice TDD. Each time one occurs, ask yourself these questions: What did I miss? Did I take too large a step? Did I make a dumb mistake? What might I have done differently? Like compiler warnings, you always want to listen to what premature passes are telling you.