Assertions

Assertions turn a test into an automated test. In the absence of an assertion, Google Mock simply executes a block of code. If you want to verify that the block of code worked, you’d need to manually inspect the results (perhaps by step-debugging or printing the contents of some variables).

You don’t have time for manual verification. TDD generates automated unit tests, which self-verify (see ​Tests Come FIRST​), saving you from the risky and tedious work of visually inspecting results. There’s hardly anything more wasteful than manually bringing up the GUI for an application, time and time again, in order to verify something that a unit test could have verified automatically.

When your test framework runs an individual test, it executes statements from the top to the bottom of the test’s code block. Each assertion encountered represents an attempt to verify that something occurred as expected. If the assertion’s condition is not met, the test framework aborts execution of the remainder of the test. (This is often accomplished by having the assertion throw an exception that the framework traps.) The framework stores information about the test failure, runs any teardown logic, and moves on to another test.

Some tools, Google Mock included, provide an alternate assertion mechanism that allows a test to continue executing even if the assertion fails. These assertions are known as nonfatal assertions (in contrast to fatal assertions that abort the test).

Your production tests should use fatal assertions. A test should abort once an assertion failure occurs. It makes little sense to continue executing a test once an assumption verified by an assertion no longer holds true. If you strive for one assert per test (​One Assert per Test​), you’ll rarely have code that executes after that sole assertion.

When designing tests or deciphering test failures, you might use additional assertions as probes. Is a member variable initialized to a specific value? Does setup put things into the state I expect? If a nonfatal assertion fails, the rest of the test still runs, perhaps providing you with additional useful knowledge. You would normally remove such probes when you’re ready to check in code.

Classic Form Assertions

Most, if not all, frameworks support what I’ll refer to as classic-form asserts. They follow a pattern first set out with SUnit, a Smalltalk unit testing framework built in the 1990s. Nothing is wrong with using this form, but you might also consider the more literary form of assertions known as Hamcrest (see ​Hamcrest Assertions​). You’ll want to learn both, since you’ll encounter plenty of code using classic assertions.

The table here shows the two core workhorse assertions for Google Mock. Your framework will use similar verbiage.

Google Mock, like most C++ unit-testing frameworks, uses macros to implement the behavior for assertions. The assertion implementations for equality comparisons are generally overloaded to support comparison of all primitive types.

Most frameworks provide several additional assertion forms designed to improve expressiveness. For example, Google Mock supports ASSERT_FALSE (assert that an expression returns false) plus a series of relational assertions such as ASSERT_GT (assert that a first argument is greater than a second). You would express the first assertion in the preceding table as ASSERT_LT(4, 7).

Hamcrest Assertions

Your unit testing tool provides a small fixed set of classic-form assertions. The most prevalent assertion, comparing two values for equality (ASSERT_EQ in Google Mock), is idiomatic. You have to remember that the expected value comes first. Were you to read an assertion out loud, it would sound like “Assert equals the expected value of 5 to the actual value of x.” It’s not a big deal, because you’ll be reading thousands of asserts over time, but the awkward phrasing slightly detracts from the readability of a test. Also, it’s easier for newbies to mistakenly invert the order of actual and expected values.

Hamcrest assertions were introduced into unit testing tools several years ago in an attempt to improve the expressiveness of tests, the flexibility of creating complex assertions, and the information supplied by failure messages. Hamcrest uses matchers (“Hamcrest” is an anagram of “matchers”) to declare comparisons against actual results. Matchers can be combined to build complex yet easily understood comparison expressions. You can also create custom matchers.

A few simple examples are worth a thousand words, or at least the number of words in the preceding paragraph.

The assertion reads left to right: assert that the actual value is Equal to the string "alpha". For simple equality comparisons, the improved readability comes at the low cost of a few extra characters.

Hamcrest can seem like overkill initially. But the wealth of matchers should make it clear that Hamcrest provides ample opportunities for high levels of expressiveness in your tests. Many of the matchers can actually reduce the amount of code you write at the same time they increase the abstraction level in your tests.

A list of the provided matchers appears in the Google Mock documentation.^[11]

You’ll want to add a using declaration to your test file.

Otherwise, the assertions that you so dearly want to be expressive will be filled with clutter.

The value of using Hamcrest increases even more with the improved readability of the failure messages.

The ability to combine matchers can reduce multiple lines needed to assert something into a single-line assertion.

In this example, AllOf indicates that all of the matcher arguments must succeed in order for the entire assert to pass. Thus, the actual string must start with "al", end with "ha", and not be equal to "aloha".

Most developers eschew the use of Hamcrest for assertions against boolean values and prefer to use the classic-form assertion instead.

Finally, if the set of matchers Google Mock provides is insufficient for your needs, you can create custom matchers.^[12]

Choosing the Right Assertion

The goal of an assertion in your test is to succinctly describe the expected outcome of preceding statements executed in the test. Usually you want the assertion to be specific. If you know that the value of a variable should be 3, use an equality assertion form and assert exactly that.

Weaker comparisons can at times be more expressive, but most of the time you want to avoid them.

Most of your assertions should use the equality form. You could technically use the ASSERT_TRUE form for everything, but the equality form assertions (Hamcrest or not) provide a better message when an assertion does fail. When the following assertion fails...

the failure message provides minimal information.

However, when you use an equality form...

the failure message tells you what the assertion expected plus what was actually received.

The wording of the failure message is why you always want to specify the expected and actual values in the correct order. Unintentionally reversing the order creates a confusing failure message that will take additional time to decipher. If you use ASSERT_THAT, the actual value comes first; if you use ASSERT_EQ, the expected value comes first.

Comparing Floats

Floating-point numbers are imprecise binary representations of real numbers. As such, the result of a float-based calculation might not exactly match another float value, even though it appears as if they should. Here’s an example:

On my machine, I receive the following failure when I execute the previous assertion:

Google Mock and other tools provide special assertion forms that allow you to compare two floating-point quantities using a tolerance. If the two quantities differ by an amount greater than the tolerance, the assertion fails. Google Mock provides a simpler comparison by using units in the last place (ULPs) as the default tolerance.

(In a forthcoming version of Google Test, you can also specify the tolerance value itself if you are daring enough—ULPs and comparing floating-point numbers are complex topics (http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm).

Exception-Based Tests

As a good programmer, you know the possibilities for failure that could arise from executing code. You know when to throw exceptions and also when you might need to protect your application by introducing try-catch blocks. Your intimate knowledge of the code paths is what allows you to know when you need to worry about handling exceptions.

When doing TDD, you drive code to handle these concerns into your system by first writing a failing test. The result is a test that documents to other developers—who don’t have your intimate knowledge of the code paths—what could go wrong and what should happen when it does. A client developer armed with this knowledge can consume your classes with much higher confidence.

Suppose you are testing a bit of code that throws an exception under certain circumstances. Your job as a professional is to document that case in a test.

Some unit testing frameworks allow you to declare that an exception should be thrown and fail the test if an exception is not thrown. Using Google Mock, we code the following:

The ASSERT_ANY_THROW macro fails the test if the expression it encloses does not throw an exception. We run all the tests and await the failure of this test.

Here is corresponding code that gets the test to pass:

(The implementation of isValid is sufficient for the one new test we added to TweetTest. Its implementation assumes that Tweet’s constructor is passed a nonempty string for the user argument. So, what other test do we need to write?)

If you know the type of the exception that will be thrown, you can specify it.

The failure message tells you when the expected exception type does not match what was actually thrown.

If your framework does not support a single-line declarative assert that ensures an exception is thrown, you can use the following structure in your test:

We could code this in a number of ways, but I prefer the technique shown here. It’s the generally preferred idiom in the TDD community. You might also need to use the try-catch structure if you must verify any postconditions after the exception is thrown. For example, you may want to verify the text associated with the thrown exception object.

Note the use of ASSERT_STREQ. Google Mock supplies four assertion macros (ASSERT_STREQ, ASSERT_STRNE, ASSERT_STRCASEEQ, and ASSERT_STRCASENE) designed to support C-style (null-terminated) strings, in other words, char* variables.