5

Adding More Confirm Types

The previous chapter introduced confirmations and showed you how to use them to verify that the bool values within your tests match what you expect them to be. The chapter did this with some exploratory code based on a school grading example. We’re going to change the grading example to better fit with a test library and add additional types that you will be able to use in your confirms.

In this chapter, we will cover the following main topics:

• Fixing the bool confirms
• Confirming equality
• Changing the code to fix a problem that line numbers are causing with test failures
• Adding more confirm types
• Confirming string literals
• Confirming floating-point values
• How to write confirms

The additional types add some new twists to confirms that, in this chapter, you’ll learn how to work around. By the end of this chapter, you’ll be able to write tests that can verify any result you need to be tested.

Technical requirements

All of the code in this chapter uses standard C++ that builds on any modern C++ 17, or later, compiler and standard library. The code is based on and continues from the previous chapters.

You can find all the code for this chapter at the following GitHub repository:

https://github.com/PacktPublishing/Test-Driven-Development-with-CPP

Fixing the bool confirms

The previous chapter explored what it means to confirm a value. However, it left us with some temporary code that we need to fix. Let’s start by fixing the code in Confirm.cpp so that it no longer refers to school grades. We want confirms to work with types such as bool. That’s why the confirm macros we have now are called CONFIRM_TRUE and CONFIRM_FALSE. The mention of true and false in the macro names are the expected values. Additionally, the macros accept a single parameter, which is the actual value.

Instead of a test about passing grades, let’s replace it with a test about bools like this:

TEST("Test bool confirms")

{

    bool result = isNegative(0);

    CONFIRM_FALSE(result);

    result = isNegative(-1);

    CONFIRM_TRUE(result);

}

The new test is clear about what it tests and needs a new helper function called isNegative instead of the previous function that determined whether a grade was passing or not. I wanted something that is simple and can be used to generate a result with an obvious expected value. The isNegative function replaces the previous isPassingGrade function and looks like this:

bool isNegative (int value)

{

    return value < 0;

}

This is a simple change that removes the exploratory code based on grades and gives us something that now fits in the test library. Now, in the next section, we can continue with confirms that test for equality.

Confirming equality

In a way, the bool confirms do test for equality. They ensure that the actual bool value is equal to the expected value. This is what the new confirms that are introduced in this chapter will do, too. The only difference is that the CONFIRM_TRUE and CONFIRM_FALSE confirms don’t need to accept a parameter for the expected value. Their expected value is implied in their name. We can do this for bool types because there are only two possible values.

However, let’s say that we want to verify that an actual int value equals 1. Do we really want a macro that’s called CONFIRM_1? We would need billions of macros for each possible 32-bit int and even more for a 64-bit int. And verifying text strings to make sure they match expected values becomes impossible with this approach.

Instead, all we need to do is modify the macros for the other types to accept both an expected value and an actual value. If the two values are not equal to each other, then the macros should result in the test failing with an appropriate message that explains what was expected and what was actually received.

Macros are not designed to resolve different types. They perform simple text replacement only. We’ll need real C++ functions to work properly with the different types we’ll be checking. Additionally, we might as well change the existing bool macros to call a function instead of defining the code directly inside the macro. Here are the existing bool macros, as we defined them in the previous chapter:

#define CONFIRM_FALSE( actual ) 

if (actual) 

{ 

    throw MereTDD::BoolConfirmException(false, __LINE__); 

}

#define CONFIRM_TRUE( actual ) 

if (not actual) 

{ 

    throw MereTDD::BoolConfirmException(true, __LINE__); 

}

What we need to do is move the if and throw statements into a function. We only need one function for both true and false, and it will look like this:

inline void confirm (

    bool expected,

    bool actual,

    int line)

{

    if (actual != expected)

    {

        throw BoolConfirmException(expected, line);

    }

}

This function can be placed in Test.h inside the MereTDD namespace right before TestBase is defined. The function needs to be inline and no longer needs to qualify the exception with the namespace since it’s now inside the same namespace.

Also, you can see better that this is an equality comparison even for bool values. The function checks to make sure that the actual value is equal to the expected value, and if not, then it throws an exception. The macros can be simplified to call the new function like this:

#define CONFIRM_FALSE( actual ) 

    MereTDD::confirm(false, actual, __LINE__)

#define CONFIRM_TRUE( actual ) 

    MereTDD:: confirm(true, actual, __LINE__)

Building and running show that all of the tests pass, and we are ready to add additional types to confirm. Let’s start with a new test in Confirm.cpp for int types like this:

TEST("Test int confirms")

{

    int result = multiplyBy2(0);

    CONFIRM(0, result);

    result = multiplyBy2(1);

    CONFIRM(2, result);

    result = multiplyBy2(-1);

    CONFIRM(-2, result);

}

Instead of a bools, this code tests int values. It uses a new helper function that should be simple to understand, which just multiplies a value by 2. We need the new helper function to be declared at the top of the same file like this:

int multiplyBy2 (int value)

{

    return value * 2;

}

The test won’t build yet. That’s okay because, when using a TDD approach, we want to focus on the usage first. This usage seems good. It will let us confirm that any int value is equal to whatever we expect it to be. Let’s create the CONFIRM macro and place it right after the two existing macros that confirm true and false like this:

#define CONFIRM_FALSE( actual ) 

    MereTDD::confirm(false, actual, __LINE__)

#define CONFIRM_TRUE( actual ) 

    MereTDD:: confirm(true, actual, __LINE__)

#define CONFIRM( expected, actual ) 

    MereTDD::confirm(expected, actual, __LINE__)

Changing the macros to call a function is really paying off now. The CONFIRM macro needs an extra parameter for the expected value, and it can call the same function name. How can it call the same function, though? Well, that’s because we’re going to overload the function. What we have now only works for bool values. This is why we switched to a design that can make use of data types. All we need to do is provide another implementation of confirm that is overloaded to work with ints like this:

inline void confirm (

    int expected,

    int actual,

    int line)

{

    if (actual != expected)

    {

        throw ActualConfirmException(expected, actual, line);

    }

}

This is almost identical to the existing confirm function. It takes ints for the expected and actual parameters instead of bools and will throw a new exception type. The reason for the new exception type is so that we can format a failure message that will display both the expected and actual values. The BoolConfirmException type will only be used for bools and will format a message that only mentions what was expected. Additionally, the new ActualConfirmException type will format a message that mentions both the expected and actual values.

The new exception type looks like this:

class ActualConfirmException : public ConfirmException

{

public:

    ActualConfirmException (int expected, int actual, int line)

    : mExpected(std::to_string(expected)),

      mActual(std::to_string(actual)),

      mLine(line)

    {

        formatReason();

    }

private:

    void formatReason ()

    {

        mReason =  "Confirm failed on line ";

        mReason += std::to_string(mLine) + "
";

        mReason += "    Expected: " + mExpected + "
";

        mReason += "    Actual  : " + mActual;

    }

    std::string mExpected;

    std::string mActual;

    int mLine;

};

You might be wondering why the new exception type stores the expected and actual values as strings. The constructor accepts ints and then converts the ints into strings before formatting the reason. This is because we’ll be adding multiple data types, and we don’t really need to do anything different. Each type just needs to display a descriptive message based on strings when a test fails.

We don’t need to use the expected or actual values for any calculations. They just need to be formatted into a readable message. Additionally, this design will let us use a single exception for all the data types other than bool. We could use this new exception for bools too, but the message doesn’t need to mention the actual value for bools. So, we’ll keep the existing exception for bools and use this new exception type for everything else.

By storing the expected and actual values as strings, all we need is an overloaded constructor for each new data type we want to support. Each constructor can convert the expected and actual values into strings that can then be formatted into a readable message. This is better than having an IntActualConfirmException class, a StringActualConfirmException class, and more.

We can build and run the tests again. The results for both the bool and int tests look like this:

---------------

Test bool confirms

Passed

---------------

Test int confirms

Passed

---------------

So, what happens if the confirms fail? Well, we’ve already seen in the previous chapter what a failed bool confirm looks like. But we don’t yet have any tests for failure cases. We should add them and make them expected failures so that the behavior can be captured. Even a failure should be tested to make sure it remains a failure. It would be bad if, in the future, we made some changes to the code that turned a failure into a success. That would be a breaking change because a failure should be expected. Let’s add a couple of new tests to Confirm.cpp like this:

TEST("Test bool confirm failure")

{

    bool result = isNegative(0);

    CONFIRM_TRUE(result);

}

TEST("Test int confirm failure")

{

    int result = multiplyBy2(1);

    CONFIRM(0, result);

}

We get the expected failures, and they look like this:

---------------

Test bool confirm failure

Failed

Confirm failed on line 41

    Expected: true

---------------

Test int confirm failure

Failed

Confirm failed on line 47

    Expected: 0

    Actual  : 2

---------------

The next step is to set the expected failure messages so that these tests pass instead of fail. However, there’s a problem. The line number is part of the error message. We want the line number to be displayed in the test results. But that means we also have to include the line number in the expected failure message in order to treat the failures as passing. Why is this a problem? Well, that’s because every time a test is moved or even when other tests are added or removed, the line numbers will change. We don’t want to have to change the expected error messages for something that is not really part of the error. The line number tells us where the error happened and should not be part of the reason for why it happened.

In the next section, we’ll fix the line numbers with some refactoring.

Decoupling test failures from line numbers

We need to remove the line number from the confirm failure reason so that tests can be given an expected failure reason that won’t change as the test is moved or shifted to different locations in the source code file.

This type of change is called refactoring. We’re not going to make any changes that cause different or new behaviors to appear in the code. At least, that’s the goal. Using TDD will help you refactor your code because you should already have tests in place for all of the important aspects.

Refactoring with proper tests lets you verify that nothing has changed. Many times, refactoring without TDD is avoided because the risk of introducing new bugs is too great. This tends to make problems bigger, as the refactoring is delayed or avoided entirely.

We have a problem with the line numbers. We could ignore the problem and just update the tests with new line numbers in the expected failure messages anytime a change is made. But that is not right and will only lead to more work and brittle tests. As more tests are added, the problem will only get worse. We really should fix the problem now. Because we’re following TDD, we can feel confident that the changes we are about to make will not break anything that has already been tested. Or, at least, if it does break, we’ll know about it and can fix any breaks right away.

The first step is to add line number information to the ConfirmException base class in Test.cpp:

class ConfirmException

{

public:

    ConfirmException (int line)

    : mLine(line)

    { }

    virtual ~ConfirmException () = default;

    std::string_view reason () const

    {

        return mReason;

    }

    int line () const

    {

        return mLine;

    }

protected:

    std::string mReason;

    int mLine;

};

Then, in the runTests function, we can get the line from the confirm exception and use it to set the failure location in the test like this:

        try

        {

            test->runEx();

        }

        catch (ConfirmException const & ex)

        {

            test->setFailed(ex.reason(), ex.line());

        }

Even though we are not starting with a test, notice how I’m still following a TDD approach to writing the code, as I’d like it to be used before implementing it fully. This is a great example because I originally thought about adding a new method to the test class. It was called setFailedLocation. But that made the existing setFailed method seem strange. I almost renamed setFailed to setFailedReason, which would have meant that it would need to be changed in the other places it’s called. Instead, I decided to add an extra parameter for the line number to the existing setFailed method. I also decided to give the parameter a default value so that the other code would not need to be changed. This makes sense and lets the caller set the failed reason by itself or with a line number if known.

We need to add a line number data member to the TestBase class. The line number will only be known for confirms, so it will be called mConfirmLocation like this:

    std::string mName;

    bool mPassed;

    std::string mReason;

    std::string mExpectedReason;

    int mConfirmLocation;

};

The new data member needs to be initialized in the TestBase constructor. We’ll use the value of -1 to mean that the line number location is not applicable:

    TestBase (std::string_view name)

    : mName(name), mPassed(true), mConfirmLocation(-1)

    { }

We need to add the line number parameter to the setFailed method like this:

    void setFailed (std::string_view reason,          int confirmLocation = -1)

    {

        mPassed = false;

        mReason = reason;

        mConfirmLocation = confirmLocation;

    }

Additionally, we need to add a new getter method for the confirm location like this:

    int confirmLocation () const

    {

        return mConfirmLocation;

    }

This will let the runTests function set the line number when it catches a confirm exception, and the test will be able to remember the line number. At the end of runTests, where the failure message is sent to the output, we need to test confirmLocation and change the output if we have a line number or not, as follows:

        else

        {

            ++numFailed;

            if (test->confirmLocation() != -1)

            {

                output << "Failed confirm on line "

                    << test->confirmLocation() << "
";

            }

            else

            {

                output << "Failed
";

            }

            output << test->reason()

                << std::endl;

        }

This will also fix a minor problem with confirms. Previously, the test results printed a line that said the test failed and then another line that said a confirm failed. The new code will only display either a generic failed message or a confirm failed message with a line number.

We’re not done yet. We need to change both derived exception class constructors to initialize the base class line number and to stop including the line number as part of the reason. The constructor for BoolConfirmException looks like this:

    BoolConfirmException (bool expected, int line)

    : ConfirmException(line)

    {

        mReason += "    Expected: ";

        mReason += expected ? "true" : "false";

    }

Additionally, the ActualConfirmException class needs to be changed throughout. The constructor needs to initialize the base class with the line number, the formatting needs to change, and the line number data member can be removed since it’s now in the base class. The class looks like this:

class ActualConfirmException : public ConfirmException

{

public:

    ActualConfirmException (int expected, int actual, int line)

    : ConfirmException(line),

      mExpected(std::to_string(expected)),

      mActual(std::to_string(actual))

    {

        formatReason();

    }

private:

    void formatReason ()

    {

        mReason += "    Expected: " + mExpected + "
";

        mReason += "    Actual  : " + mActual;

    }

    std::string mExpected;

    std::string mActual;

};

We can build again and running still shows the expected failures. The failure reasons are formatted slightly differently than before and look like this:

---------------

Test bool confirm failure

Failed confirm on line 41

    Expected: true

---------------

Test int confirm failure

Failed confirm on line 47

    Expected: 0

    Actual  : 2

---------------

It looks almost the same, which is good. Now we can set the expected failure messages without needing to worry about the line numbers like this:

TEST("Test bool confirm failure")

{

    std::string reason = "    Expected: true";

    setExpectedFailureReason(reason);

    bool result = isNegative(0);

    CONFIRM_TRUE(result);

}

TEST("Test int confirm failure")

{

    std::string reason = "    Expected: 0
";

    reason += "    Actual  : 2";

    setExpectedFailureReason(reason);

    int result = multiplyBy2(1);

    CONFIRM(0, result);

}

Notice that the expected failure reason needs to be formatted to exactly match what the test displays when it fails. This includes spaces used to indent and new lines. Once the expected failure reasons are set, all of the tests pass again like this:

---------------

Test bool confirm failure

Expected failure

    Expected: true

---------------

Test int confirm failure

Expected failure

    Expected: 0

    Actual  : 2

---------------

Both tests are expected failures and are treated as passing. Now we can continue adding more confirm types.

Adding more confirm types

Currently, we can confirm bool and int values inside the tests. We need more than this, so what should we add next? Let’s add support for the long type. It’s similar to an int and, on many platforms, will effectively be the same. Even though it may or may not use the same number of bits as an int, to the C++ compiler, it is a different type. We can begin by adding a basic test to Confirm.cpp that tests the long type like this:

TEST("Test long comfirms")

{

    long result = multiplyBy2(0L);

    CONFIRM(0L, result);

    result = multiplyBy2(1L);

    CONFIRM(2L, result);

    result = multiplyBy2(-1L);

    CONFIRM(-2L, result);

}

The test calls the same multiplyBy2 helper function, which performs extra conversions because it’s not working with longs throughout. We start with long literal values by adding the L suffix. These get converted into ints in order to be passed to multiplyBy2. The return value is also an int, which gets converted into a long in order to be assigned to result. Let’s prevent all of this extra conversion by creating an overloaded version of multiplyBy2 that accepts a long type and returns a long type:

long multiplyBy2 (long value)

{

    return value * 2L;

}

If we try to build right now, there will be an error because the compiler doesn’t know which overload of confirm to call. The only available choices are to either convert the long expected and actual values into ints or bools. Neither choice is a match, and the compiler sees the call as ambiguous. Remember that the CONFIRM macro gets transformed into a call to the overloaded confirm function.

We can fix this by adding a new overloaded version of confirm that uses long parameters. However, a better solution is to change the existing version of confirm that uses int parameters into a template like this:

template <typename T>

void confirm (

    T const & expected,

    T const & actual,

    int line)

{

    if (actual != expected)

    {

        throw ActualConfirmException(

            std::to_string(expected),

            std::to_string(actual),

            line);

    }

}

We still have the version of confirm that uses a bool parameter. The template will match both int and long types. Additionally, the template will match types that we don’t yet have tests for. The new templated confirm method also does the conversion into std::string when creating the exception to be thrown. In Chapter 12, Creating Better Test Confirmations, you’ll see that there is a problem with how we convert the expected and actual values into strings. Or, at least, there is a better way. What we have does work but only for numeric types that can be passed to std::to_string.

Let’s update the ActualConfirmException constructor to use strings that we will now be calling std::to_string from within the confirm function. The constructor looks like this:

    ActualConfirmException (

        std::string_view expected,

        std::string_view actual,

        int line)

    : ConfirmException(line),

      mExpected(expected),

      mActual(actual)

    {

        formatReason();

    }

Everything builds, and all the tests pass again. We can add a new test in Confirm.cpp for a long failure like this:

TEST("Test long confirm failure")

{

    std::string reason = "    Expected: 0
";

    reason += "    Actual  : 2";

    setExpectedFailureReason(reason);

    long result = multiplyBy2(1L);

    CONFIRM(0L, result);

}

The failure reason string is the same as for an int even though we are testing a long type. The test result for the new test looks like this:

---------------

Test long confirm failure

Expected failure

    Expected: 0

    Actual  : 2

---------------

Let’s try a type that will show something different. A long long type can definitely hold numeric values that are bigger than an int. Here is a new test in Confirm.cpp that tests long long values:

TEST("Test long long confirms")

{

    long long result = multiplyBy2(0LL);

    CONFIRM(0LL, result);

    result = multiplyBy2(10'000'000'000LL);

    CONFIRM(20'000'000'000LL, result);

    result = multiplyBy2(-10'000'000'000LL);

    CONFIRM(-20'000'000'000LL, result);

}

With a long long type, we can have values greater than a maximum 32-bit signed value. The code uses single quote marks to make the larger numbers easier to read. The compiler ignores the single quote marks, but they help us to visually separate every group of thousands. Also, the suffix, LL, tells the compiler to treat the literal value as a long long type.

The result for this passing test looks like the others:

---------------

Test long long confirms

Passed

---------------

We need to look at a long long failure test result to see the larger numbers. Here is a failure test:

TEST("Test long long confirm failure")

{

    std::string reason = "    Expected: 10000000000
";

    reason += "    Actual  : 20000000000";

    setExpectedFailureReason(reason);

    long long result = multiplyBy2(10'000'000'000LL);

    CONFIRM(10'000'000'000LL, result);

}

Because we’re not formatting the output with separators, we need to use the unadorned numbers in text format without any commas. This is probably for the best anyway because some locales use commas and some use dots. Note that we don’t try to do any formatting, so the expected failure message also uses no formatting.

Now we can see that the failure description does indeed match the larger numbers and looks like this:

---------------

Test long long confirm failure

Expected failure

    Expected: 10000000000

    Actual  : 20000000000

---------------

I want to highlight one important point about failure tests. They are purposefully using incorrect expected values to force a failure. You will not do this in your tests. But then you also will not need to write tests that you want to fail. We want these tests to fail so that we can verify that the test library is able to properly detect and handle any failures. Because of this, we treat the failures as passes.

We could keep going and add tests for shorts, chars, and all of the unsigned versions. However, this is becoming uninteresting at this point because all we are doing is testing that the template function works properly. Instead, let’s focus on types that use non-template code that has been written to work properly.

Here is a simple test for the string type:

TEST("Test string confirms")

{

    std::string result = "abc";

    std::string expected = "abc";

    CONFIRM(expected, result);

}

Instead of writing a fake helper method that returns a string, this test simply declares two strings and will use one as the actual value and the other as the expected value. By initializing both strings with the same text, we expect them to be equal, so we call CONFIRM to make sure they are equal.

When you are writing a test, you will want to assign result a value that you get from the function or method that you are testing. Our goal here is to test that the CONFIRM macro and the underlying test library code work properly. So, we can skip the function being tested and go straight to the macro with two string values where we know what to expect.

This seems like a reasonable test. And it is. But it doesn’t compile. The problem is that the confirm template function tries to call std::to_string on the values provided. This doesn’t make sense when the values are already strings.

What we need is a new overload of confirm that uses strings. We’ll actually create two overloads, one for string views and one for strings. The first overload function looks like this:

inline void confirm (

    std::string_view expected,

    std::string_view actual,

    int line)

{

    if (actual != expected)

    {

        throw ActualConfirmException(

            expected,

            actual,

            line);

    }

}

This first function takes string views, which will be a better match than the template method when working with string views. Then, it passes the strings given to the ActualConfirmException constructor without trying to call std::to_string because they are already strings.

The second overloaded function looks like this:

inline void confirm (

    std::string const & expected,

    std::string const & actual,

    int line)

{

    confirm(

        std::string_view(expected),

        std::string_view(actual),

        line);

}

This second function takes constant string references, which will also be a better match than the template method when working with strings. Then, it converts the strings into string views and calls the first function.

Now we can add a string failure test like this:

TEST("Test string confirm failure")

{

    std::string reason = "    Expected: def
";

    reason += "    Actual  : abc";

    setExpectedFailureReason(reason);

    std::string result = "abc";

    std::string expected = "def";

    CONFIRM(expected, result);

}

The test result after building and running the tests looks like this:

---------------

Test string confirm failure

Expected failure

    Expected: def

    Actual  : abc

---------------

There’s one more important aspect to consider about strings. We need to consider string literals that are really constant char pointers. We’ll explore pointers followed by string literals in the next section.

Confirming string literals

A string literal might look like a string, but the C++ compiler treats a string literal as a pointer to the first of a set of constant chars. The set of constant chars is terminated with a null character value, which is the numeric value of zero. That’s how the compiler knows how long the string is. It just keeps going until it finds the null. The reason that the chars are constant is that the data is normally stored in memory that is write protected, so it cannot be modified.

When we try to confirm a string literal, the compiler sees a pointer and has to decide which overloaded confirm function to call. Before we get too far with our exploration of string literals, what other problems can we get into with pointers?

Let’s start with the simple bool type and see what kinds of problems we run into if we try to confirm bool pointers. This will help you to understand string literal pointers by, first, understanding a simpler example test for bool pointers. You don’t need to add this test to the project. It is included here just to explain what happens when we try to confirm a pointer. The test looks like this:

TEST("Test bool pointer confirms")

{

    bool result1 = true;

    bool result2 = false;

    bool * pResult1 = &result1;

    bool * pResult2 = &result2;

    CONFIRM_TRUE(pResult1);

    CONFIRM_FALSE(pResult2);

}

The preceding test actually compiles and runs. But it fails with the following result:

---------------

Test bool pointer confirms

Failed confirm on line 86

    Expected: false

---------------

Line 86 is the second confirm in the test. So, what is going on? Why does the confirm think that pResult2 points to a true value?

Well, remember that the confirm macro just gets replaced with a call to one of the confirm methods. The second confirm deals with the following macro:

#define CONFIRM_FALSE( actual ) 

    confirm(false, actual, __LINE__)

And it tries to call confirm with a hardcoded false bool value, the bool pointer that was passed to the macro, and the int line number. There is no exact match for a bool, a bool pointer, or an int for any version of confirm, so something either has to be converted or the compiler will generate an error. We know there was no error because the code compiled and ran. So, what got converted?

This is a great example of the TDD process, as explained in Chapter 3, The TDD Process, to write the code first as you want it to be used and compile it even if you expect the build to fail. In this case, the build did not fail, and that gives us insight that we might have otherwise missed.

The compiler was able to convert the pointer value into a bool and that was seen as the best choice available. In fact, I didn’t even get a warning about the conversion. The compiler silently made the decision to convert the pointer to a bool into a bool value. This is almost never what you want to happen.

So, what does it even mean to convert a pointer into a bool? Any pointer with a valid nonzero address will get converted into true. Additionally, any null pointer with a zero address will get converted into false. Because we have the real address of result2 stored in the pResult2 pointer, the conversion was made to a true bool value.

You might be wondering what happened to the first confirm and why it did not fail. Why did the test proceed to the second confirm before it failed? Well, the first confirm went through the same conversion for a bool, bool pointer, and int. Both conversions resulted in a true bool value because both pointers held valid addresses.

The first confirm called confirm with true, true, and the line number, which passed. But the second confirm called confirm with false, true, and the line number, which failed.

To solve this, we either need to add support for pointers of all types or remember to dereference the pointers before confirming them. Adding support for pointers might seem like a simple solution until we get to string literals, which are also pointers. It’s not as simple as it seems and is not something we need to do now. Let’s keep the test library as simple as possible. Here is how you can fix the bool confirm test shown earlier:

TEST("Test bool pointer dereference confirms")

{

    bool result1 = true;

    bool result2 = false;

    bool * pResult1 = &result1;

    bool * pResult2 = &result2;

    CONFIRM_TRUE(*pResult1);

    CONFIRM_FALSE(*pResult2);

}

Notice that the tests dereference the pointers instead of passing the pointers directly to the macros. This means that the test is really just testing bool values, and that’s why I said that you really don’t need to add the test.

String literals are frequently found in the source code. They are an easy way to represent an expected string value. The problem with string literals is they are not strings. They are a pointer to a constant char. And we can’t just dereference a string literal pointer as we did for a bool pointer. That would result in a single char. We want to confirm the whole string.

Here is a test that shows what will likely be the major usage of string literals. The most common usage will be comparing a string literal with a string. The test looks like this:

TEST("Test string and string literal confirms")

{

    std::string result = "abc";

    CONFIRM("abc", result);

}

This works because one of the argument types that ends up getting passed to the confirm function is std::string. The compiler doesn’t find an exact match for both arguments; however, because one is a string, it decides to convert the string literal into a string, too.

Where we run into problems is when we try to confirm two string literals for both the expected and actual values. The compiler sees two pointers and has no clue that they should both be converted into strings. This is not a normal situation that you will need to verify in a test. Additionally, if you ever do need to compare two string literals, it’s easy to wrap one of them into a std::string argument type before confirming.

Also, in Chapter 12, Creating Better Test Confirmations, you’ll see how you can get around the problem of confirming two string literals. We’ll be improving the whole design used to confirm the test results. The design we have now is often called the classic way to confirm values. Chapter 12 will introduce a new way that is more extensible, easier to read, and more flexible.

We’ve come a long way in terms of adding support for different types, and you also understand how to work with string literals. However, I’ve stayed away from the two floating-point types, float and double, because they need some special consideration. They will be explained next.

Confirming floating point values

At the most basic level, confirms work by comparing an expected value with an actual value and throwing an exception if they are different. This works for all the integral types such as int and long, bool types, and even strings. The values either match or don’t match.

This is where things get difficult for the float and double floating point types – because it’s not always possible to accurately compare two floating-point values.

Even in the decimal system that we are used to from grade school, we understand there are some fractional values that can’t be accurately represented. A value such as 1/3 is easy to represent as a fraction. But writing it in a floating-point decimal format looks like 0.33333 with the digit 3 continuing forever. We can get close to the true value of 1/3, but at some point, we have to stop when writing 0.333333333... And no matter how many 3s we include, there are always more.

In C++, floating-point values use a binary number system that has similar accuracy issues. But the accuracy issues in binary are even more common than in decimal.

I won’t go into all the details because they are not important. However, the main cause of the extra issues in binary is caused by there being fewer factors of 2 than there are of 10. With the base 10 decimal system, the factors are 1, 2, 5, and 10. While in binary, the factors of 2 are only 1 and 2.

So, why are the factors important? Well, it’s because they determine which fractions can be accurately described and which cannot. A fraction such as 1/3 causes trouble for both systems because 3 is not a factor in either. Another example is 1/7. These fractions are not very common, though. The fraction of 1/10 is very common in decimal. Because 10 is a factor, this means that values such as 0.1, 0.2, 0.3, and more can all be accurately represented in decimal.

Additionally, because 10 is not a factor in binary base 2, these same values that are widely used have no representation with a fixed number of digits as they do in decimal.

So, what all of this means is that if you have a binary floating-point value that looks like 0.1, it is close to the actual value but can’t quite be exact. It might be displayed as 0.1 when converted into a string but that also involves a little bit of rounding.

Normally, we don’t worry about the computer’s inability to accurately represent values that we are used to being exact from grade school – that is, until we need to test one floating-point value to see if it equals another.

Even something as simple as 0.1 + 0.2 that looks like 0.3 will probably not equal 0.3.

When comparing computer floating-point values, we always have to allow for a certain amount of error. As long as the values are close, we can assume they are equal.

However, the ultimate problem is that there is no good single solution that can determine whether two values are close. The amount of error we can represent changes depending on how big or how small the values are. Floating-point values change drastically when they get really close to 0. And they lose the ability to represent small values as they get larger. Because floating-point values can get really large, the amount of accuracy that is lost with large values can also be large.

Let’s imagine if a bank used floating-point values to keep track of your money. Would you be happy if your bank could no longer track anything less than a thousand dollars just because you have billions? We’re no longer talking about losing a few cents. Or maybe you only have 30 cents in your account and you want to withdraw all 30 cents. Would you expect the bank to deny your withdrawal because it thinks 30 cents is more than the 30 cents you have? These are the types of problems that floating-point values can lead to.

Because we’re following a TDD process, we’re going to start out simple with floating point values and include a small margin of error when comparing either float, double, or long double values to see whether they are equal. We’re not going to get fancy and try to adjust the margin depending on how big or small the values are.

Here is a test that we will use for the float values:

TEST("Test float confirms")

{

    float f1 = 0.1f;

    float f2 = 0.2f;

    float sum = f1 + f2;

    float expected = 0.3f;

    CONFIRM(expected, sum);

}

The test for float types actually passes on my computer.

So, what happens if we create another test for double types? The new double test looks like this:

TEST("Test double confirms")

{

    double d1 = 0.1;

    double d2 = 0.2;

    double sum = d1 + d2;

    double expected = 0.3;

    CONFIRM(expected, sum);

}

This test is almost identical, yet it fails on my computer. And the crazy part is that the failure description makes no sense unless you understand that values can be printed as text, which has been adjusted to appear like a nice round number when it really is not. Here is what the failure message shows on my computer:

---------------

Test double confirms

Failed confirm on line 122

    Expected: 0.300000

    Actual  : 0.300000

---------------

Looking at the message, you might ask how it is possible that 0.300000 does not equal 0.300000. The reason is that neither the expected nor the actual values are exactly 0.300000. They have both been adjusted slightly so that they will display these round-looking values.

The test for long doubles is almost the same as for doubles. Only the types have been changed, as follows:

TEST("Test long double confirms")

{

    long double ld1 = 0.1;

    long double ld2 = 0.2;

    long double sum = ld1 + ld2;

    long double expected = 0.3;

    CONFIRM(expected, sum);

}

The long double test also fails on my machine for the same reason as the test with doubles. We can fix all of the floating-point confirms by adding special overloads for all three of these types.

Here is an overloaded confirm function that uses a small margin of error when comparing float values:

inline void confirm (

    float expected,

    float actual,

    int line)

{

    if (actual < (expected - 0.0001f) ||

        actual > (expected + 0.0001f))

    {

        throw ActualConfirmException(

            std::to_string(expected),

            std::to_string(actual),

            line);

    }

}

We need almost the same overload for doubles as for floats. Here is the double overload that does the comparison with a margin of error that is plus or minus the expected value:

inline void confirm (

    double expected,

    double actual,

    int line)

{

    if (actual < (expected - 0.000001) ||

        actual > (expected + 0.000001))

    {

        throw ActualConfirmException(

            std::to_string(expected),

            std::to_string(actual),

            line);

    }

}

Other than the type changes from float to double, this method uses a smaller margin of error and leaves off the f suffix from the literal values.

The overload function for long doubles is similar to the one for doubles, as follows:

inline void confirm (

    long double expected,

    long double actual,

    int line)

{

    if (actual < (expected - 0.000001) ||

        actual > (expected + 0.000001))

    {

        throw ActualConfirmException(

            std::to_string(expected),

            std::to_string(actual),

            line);

    }

}

After adding these overloads for floats, doubles, and long doubles, all the tests pass again. We’ll be revisiting the problem of comparing floating-point values again in Chapter 13, How to Test Floating-Point and Custom Values. The comparison solution we have is simple and will work for now.

We have also covered all of the confirm types we’ll be supporting at this time. Remember the TDD rule to do only what is necessary. We can always enhance the design of confirmations later, and that’s exactly what we’ll be doing in Chapter 12, Creating Better Test Confirmations.

Before ending this chapter, I have some advice on writing confirmations. It’s nothing that we haven’t already been doing, but it does deserve mentioning so that you are aware of the pattern.

How to write confirms

Usually, there are many different ways you can write your code and your tests. What I’ll share here is based on years of experience, and while it’s not the only way to write tests, I hope you learn from it and follow a similar style. Specifically, I want to share guidance on how to write confirms.

The most important thing to remember is to keep your confirms outside of the normal flow of your tests but still close to where they are needed. When a test runs, it performs various activities that you want to ensure work as expected. You can add confirms along the way to make sure the test is making progress as expected. Or maybe you have a simple test that does one thing and needs one or more confirms at the end to make sure everything worked. All of this is good.

Consider the following three examples of test cases. They each do the same thing, but I want you to focus on how they are written. Here is the first example:

TEST("Test int confirms")

{

    int result = multiplyBy2(0);

    CONFIRM(0, result);

    result = multiplyBy2(1);

    CONFIRM(2, result);

    result = multiplyBy2(-1);

    CONFIRM(-2, result);

}

This test is one that was used earlier to make sure that we can confirm int values. Notice how it performs an action and assigns the result to a local variable. Then, that variable is checked to make sure its value matches what is expected. If so, the test proceeds to perform another action and assign the result to the local variable. This pattern continues, and if all the confirms match the expected values, the test passes.

Here is the same test written in a different form:

TEST("Test int confirms")

{

    CONFIRM(0, multiplyBy2(0));

    CONFIRM(2, multiplyBy2(1));

    CONFIRM(-2, multiplyBy2(-1));

}

This time, there is no local variable to store the result of each action. Some people would consider this an improvement. It is shorter. But I feel this hides what is being tested. I find it better to think about confirms as something that can be removed from a test without changing what a test does. Of course, if you do remove a confirm, then the test might miss a problem that the confirm would have caught. I’m talking about mentally ignoring confirms to get a feel for what a test does, and then thinking about what makes sense to verify along the way. Those verification points become the confirms.

Here is another example:

TEST("Test int confirms")

{

    int result1 = multiplyBy2(0);

    int result2 = multiplyBy2(1);

    int result3 = multiplyBy2(-1);

    CONFIRM(0, result1);

    CONFIRM(2, result2);

    CONFIRM(-2, result3);

}

This example avoids putting the test steps inside the confirms. However, I feel that it goes too far to separate the test steps from the confirms. There’s nothing wrong with sprinkling confirms into your test steps. Doing so lets you catch problems right away. This example puts all the confirms at the end, which means that it also has to wait until the end to catch any problems.

And then there’s the problem of the multiple result variables needed so that each can be checked later. This code looks too forced to me – like a programmer who took the long way to reach a goal when there was a simple path available instead.

The first example shows the style of tests written so far in this book, and now you can see why they have been written in this manner. They use confirms where needed and as close to the point of verification as possible. And they avoid placing actual test steps inside the confirms.

Summary

This chapter took us past the simple ability to confirm true and false values. You can now verify anything you need to make sure it matches what you expect.

We simplified the confirm macros by putting the code into overloaded functions with a templated version to handle other types. You saw how to confirm simple data types and work with pointers by dereferencing them first.

The code needed to be refactored, and you saw how TDD helps when you need to make design changes to your code. I could have written the code in this book to make it seem like the code was written perfectly from the very beginning. But that wouldn’t help you because nobody writes perfect code from the beginning. As our understanding grows, we sometimes need to change the code. And TDD gives you the confidence to make those changes as soon as they become known instead of waiting – because problems that you delay have a tendency to get bigger instead of going away.

And you should be gaining an understanding of how to write your tests and the best way to incorporate confirms into your tests.

Up until now, we’ve been working with the C++ features and capabilities found in C++ 17. There is an important new feature found in C++ 20 that will help us get line numbers from the compiler. The next chapter will add this C++ 20 feature and explore some alternate designs. Even if we stay with the same overall design we have now, the next chapter will help you to understand how other testing libraries might do things differently.