15
How to Test With Multiple Threads
Multi-threading is one of the most difficult aspects of writing software. Something that’s often overlooked is how we can test multiple threads. And can we use TDD to help design software that uses multiple threads? Yes, TDD can help and you’ll find useful and practical guidance in this chapter that will show you how to use TDD with multiple threads.
The main topics in this chapter are as follows:
- Using multiple threads in tests
- Making the logging library thread-safe
- The need to justify multiple threads
- Changing the service return type
- Making multiple service calls
- How to test multiple threads without sleep
- Fixing one last problem detected with logging
First, we’ll examine what problems you’ll find when using multiple threads in your tests. You’ll learn how to use a special helper class in the testing library to simplify the extra steps needed when testing with multiple threads.
Once we can use multiple threads inside of a test, we’ll use that ability to call into the logging library from multiple threads at the same time and see what happens. I’ll give you a hint: some changes will need to be made to the logging library to make the library behave well when called from multiple threads.
Then, we’ll go back to the simple service we developed in the previous chapter and you’ll learn how to use TDD to design a service that uses multiple threads in a way that can support reliable testing.
We’ll be working with each project in turn in this chapter. First, we will be using the testing library project. Then, we’ll switch over to the logging library project. Finally, we’ll use the simple service project.
Technical requirements
All the code in this chapter uses standard C++, which builds on any modern C++ 20 or later compiler and standard library. The code in this chapter uses all three projects developed in this book: the testing library from Part 1, Testing MVP, the logging library from Part 2, Logging Library, and the simple service from the previous chapter.
You can find all the code for this chapter in this book’s GitHub repository: https://github.com/PacktPublishing/Test-Driven-Development-with-CPP.
Using multiple threads in tests
Adding multiple threads to your tests presents challenges that you need to be aware of. I’m not talking about running the tests themselves in multiple threads. The testing library registers and runs the tests and it will remain single-threaded. What you need to understand are the problems that can arise when multiple threads are created inside of a test.
To understand these problems, let’s create a test that uses multiple threads so that you can see exactly what happens. We’ll be working with the unit test library project in this section so, first, add a new test file called Thread.cpp. The project structure should look like this after you’ve added the new file:
MereTDD project root folder Test.h tests folder main.cpp Confirm.cpp Creation.cpp Hamcrest.cpp Setup.cpp Thread.cppInside the Thread.cpp file, add the following code:
#include "../Test.h"#include <atomic>#include <thread>using namespace MereTDD;TEST("Test can use additional threads"){ std::atomic<int> count {0}; std::thread t1([&count]() { for (int i = 0; i < 100'000; ++i) { ++count; } CONFIRM_THAT(count, NotEquals(100'001)); }); std::thread t2([&count]() { for (int i = 0; i < 100'000; ++i) { --count; } CONFIRM_THAT(count, NotEquals(-100'001)); }); t1.join(); t2.join(); CONFIRM_THAT(count, Equals(0));}The preceding code includes atomic so that we can safely modify a count variable from multiple threads. We need to include thread to bring in the definition of the thread class. The test creates two threads. The first thread increments count, while the second thread decrements the same count. The final result should return count to zero because we increment and decrement the same number of times.
If you build and run the test application, everything will pass. The new test causes no problem at all. Let’s change the third CONFIRM_THAT macro so that we can try to confirm that count is not equal to 0 at the end of the test, like so:
t1.join(); t2.join(); CONFIRM_THAT(count, NotEquals(0));With this change, the test fails with this result:
------- Test: Test can use additional threadsFailed confirm on line 30 Expected: not 0 Actual : 0So far, we have a test that uses multiple threads and it works as expected. We added some confirmations that can detect and report when a value does not match the expected value. You might be wondering what problems multiple threads can cause when the threads seem to be working okay so far.
Here’s the quick answer: creating one or more threads inside of a test causes no problem at all – that is, assuming that the threads are managed correctly such as making sure they are joined before the test ends. Confirmations work as expected from the main test thread itself. You can even have confirmations inside the additional threads. One type of problem comes when a confirmation inside one of the additional threads fails. To see this, let’s put the final confirmation back to Equals and change the first confirmation to Equals too, like so:
for (int i = 0; i < 100'000; ++i) { ++count; } CONFIRM_THAT(count, Equals(100'001));count should never reach 100'001 because we only increment 100'000 times. The confirmation always passed before this change, which is why it did not cause a problem. But with this change, the confirmation will fail right away. If this was a confirmation in the main test thread, then the failure would cause the test to fail with a summary message that describes the problem. But we’re not in the main test thread now.
Remember that failed confirmations throw exceptions and that an unhandled exception inside of a thread will terminate an application. When we confirm that the count equals 100'001, we cause an exception to be thrown. The main test thread is managed by the testing library and the main thread is ready to catch any confirmation exceptions so that they can be reported. However, our additional thread inside the test lambda has no protection against thrown exceptions. So, when we build and run the test application, it terminates like this:
------- Test: Test can use additional threadsterminate called after throwing an instance of 'MereTDD::ActualConfirmException'Abort trap: 6You might get a slightly different message, depending on what computer you’re using. What you won’t get is a test application that runs and reports the results of all the tests. The application terminates soon after the confirmation inside the additional thread fails and throws an exception.
Other than confirmations inside a thread failing and throwing exceptions, are there any other problems with using multiple threads inside of a test? Yes. Threads need to be managed properly – that is, we need to make sure they are either joined or detached before going out of scope. You’re unlikely to need to detach a thread that was created in a test, so you’re left with making sure that all the threads created inside of a test are joined before the test ends. Notice that the test we’re using manually joins both threads.
If the test has other confirmations, then you need to be sure that a failed confirmation doesn’t cause the test to skip the thread joins. This is because leaving a test without joining will also cause the application to terminate. Let’s see this by putting the first confirmation back to using NotEquals so that it will not cause any problems. Then, we will add a new confirmation that will fail before the joins:
CONFIRM_TRUE(false); t1.join(); t2.join(); CONFIRM_THAT(count, Equals(0));The confirmations inside the additional threads no longer cause any problems. However, the new CONFIRM_TRUE confirmation will cause the joins to be skipped. The result is another termination:
------- Test: Test can use additional threadsterminate called without an active exceptionAbort trap: 6We’re not going to do anything to help solve this second type of termination. You’ll need to make sure that any threads that are created are joined properly. You might want to use the new jthread in C++20, which will make sure that the threads are joined. Alternatively, you might just need to be careful about where you put confirmations in the main test thread to make sure that all the joins happen first.
We can remove the CONFIRM_TRUE confirmation now so that we can focus on fixing the first problem of confirmations failing inside the threads.
What can we do to fix this problem? We could put a try/catch block in the thread, which would at least stop the termination:
TEST("Test can use additional threads"){ std::atomic<int> count {0}; std::thread t([&count]() { try { for (int i = 0; i < 100'000; ++i) { ++count; } CONFIRM_THAT(count, NotEquals(100'001)); } catch (...) { } }); t.join(); CONFIRM_THAT(count, Equals(100'000));}To simplify the code, I removed the second thread. The test now uses a single additional thread to increment the count. The result after the thread finishes is that count should be equal to 100'000. At no point should count reach 100'001, which is confirmed inside the thread. Let’s say we change the confirmation inside the thread so that it will fail:
CONFIRM_THAT(count, Equals(100'001));Here, the exception is caught and the test fails normally and reports the result. Or does it? Building and running this code shows that all the tests pass. The confirmation inside the thread is detecting the mismatched values but the exception has no way to be reported back to the main test thread. We can’t throw anything inside the catch block because that will just terminate the application again.
We know that we can avoid the test application terminating by catching the confirmation exception. And we also know from the first threading test that a confirmation that doesn’t throw is also okay. The bigger problem we need to solve is how to let the main test thread know about any confirmation failures in the additional threads that have been created. Maybe we can inform the main thread in the catch block by using a variable passed to the thread.
I want to emphasize this point. If you’re creating threads inside of a test simply to divide the work and speed up a test and don’t need to confirm anything inside the threads, then you don’t need to do anything special. All you need to manage is the normal thread concerns, such as making sure you join all threads before the test ends and that none of the threads have unhandled exceptions. The only reason to use the following guidance is when you want to put confirmations inside of the additional threads.
After trying out a few alternatives, here is what I came up with:
TEST("Test can use additional threads"){ ThreadConfirmException threadEx; std::atomic<int> count {0}; std::thread t([&threadEx, &count]() { try { for (int i = 0; i < 100'000; ++i) { ++count; } CONFIRM_THAT(count, Equals(100'001)); } catch (ConfirmException const & ex) { threadEx.setFailure(ex.line(), ex.reason()); } }); t.join(); threadEx.checkFailure(); CONFIRM_THAT(count, Equals(100'000));}This is the TDD style. Modify the test until you’re happy with the code and then get it working. The test assumes a new exception type called ThreadConfirmException and it creates a local instance called threadEx. The threadEx variable is captured by reference in the thread lambda so that the thread can access threadEx.
The thread can use all the normal confirmations it wants, so long as everything is inside a try block with a catch block that is looking for the ConfirmException type. If a confirmation fails, then it will throw an exception that will be caught. We can use the line number and reason to set a failure mode in the threadEx variable.
Once the thread has finished and we’re back in the main thread, we can call another method to check for a failure in the threadEx variable. If a failure was set, then the checkFailure method should throw an exception, just like how a regular confirmation throws an exception. Because we’re back in the main test thread, any confirmation exception that gets thrown will be detected and reported in the test summary report.
Now, we need to implement the ThreadConfirmException class in Test.h, which can go right after the ConfirmException base class, like this:
class ThreadConfirmException : public ConfirmException{public: ThreadConfirmException () : ConfirmException(0) { } void setFailure (int line, std::string_view reason) { mLine = line; mReason = reason; } void checkFailure () const { if (mLine != 0) { throw *this; } }};If we build and run now, then the confirmation inside the thread will detect that count does not equal 100'001 and the failure will be reported in the summary results, like this:
------- Test: Test can use additional threadsFailed confirm on line 20 Expected: 100001 Actual : 100000The question now is, is there any way to simplify the test? The current test looks like this:
TEST("Test can use additional threads"){ ThreadConfirmException threadEx; std::atomic<int> count {0}; std::thread t([&threadEx, &count]() { try { for (int i = 0; i < 100'000; ++i) { ++count; } CONFIRM_THAT(count, Equals(100'001)); } catch (ConfirmException const & ex) { threadEx.setFailure(ex.line(), ex.reason()); } }); t.join(); threadEx.checkFailure(); CONFIRM_THAT(count, Equals(100'000));}Here, we have a new ThreadConfirmException type, which is good. However, the test author still needs to pass an instance of this type to the thread function, similar to how threadEx is captured by the lambda. The thread function still needs a try/catch block and needs to call setFailure if an exception is caught. Finally, the test needs to check for a failure once it’s back in the main test thread. All of these steps are shown in the test.
We might be able to use a few macros to hide the try/catch block, but this seems fragile. The test author will likely have slightly different needs. For example, let’s go back to two threads and see what the test will look like with multiple threads. Change the test so that it looks like this:
TEST("Test can use additional threads"){ std::vector<ThreadConfirmException> threadExs(2); std::atomic<int> count {0}; std::vector<std::thread> threads; for (int c = 0; c < 2; ++c) { threads.emplace_back( [&threadEx = threadExs[c], &count]() { try { for (int i = 0; i < 100'000; ++i) { ++count; } CONFIRM_THAT(count, Equals(200'001)); } catch (ConfirmException const & ex) { threadEx.setFailure(ex.line(), ex.reason()); } }); } for (auto & t : threads) { t.join(); } for (auto const & ex: threadExs) { ex.checkFailure(); } CONFIRM_THAT(count, Equals(200'000));}This test is different than the original two-thread test at the beginning of this section. I wrote the test differently to show that there are lots of ways to write a multi-threaded test. Because we have more code inside the thread to handle the confirmation exceptions, I made each thread similar. Instead of one thread incrementing the count while another thread decrements, both threads now increment. Also, instead of naming each thread t1 and t2, the new test puts the threads in a vector. We also have a vector of ThreadConfirmException with each thread getting a reference to its own ThreadConfirmException.
One thing to notice about this solution is that while each thread will fail its confirmation and both ThreadConfirmationException instances will have a failure set, only one failure will be reported. In the loop at the end of the test that goes through the threadExs collection, the moment one ThreadConfirmationException fails the check, an exception will be thrown. I thought about extending the testing library to support multiple failures but decided against the added complexity.
If you have a test with multiple threads, then they will likely be working with different sets of data. If there happens to be an error that causes multiple threads to fail in the same test run, then only one failure will be reported in the test application. Fixing that failure and running again may then report the next failure. It’s a little tedious to fix problems one after another but not a likely scenario that justifies the added complexity to the testing library.
The new test structure with two threads highlights the difficulty of creating reasonable macros that can hide all the thread confirmation handling. So far, all three versions of the test have been different. There doesn’t seem to be a common way to write multi-threaded tests that we would be able to wrap up in some macros. I think we’ll stick with what we have now – a ThreadConfirmException type that can be passed to a thread. The thread will need to catch the ConfirmException type and call setFailure. The main test thread can then check each ThreadConfirmException, which will throw if the failure was set. Before we move on, let’s change the confirmation inside the thread lambda so that it tests for a count not equal to 200'001, like this:
CONFIRM_THAT(count, NotEquals(200'001));The NotEquals confirmation will let the test pass again.
With the understanding you’ve gained from this section, you’ll be able to write tests that use multiple threads inside the test. You can continue to use the same CONFIRM and CONFIRM_THAT macros to verify the results. The next section will use multiple threads to log messages so that we can make sure that the logging library is thread-safe. You’ll also learn what it means for code to be thread-safe.
Making the logging library thread-safe
We don’t know if a project that uses the logging library will be trying to log from multiple threads or a single thread. With an application, we’re in full control and can choose to use multiple threads or not. But a library, especially a logging library, often needs to be thread-safe. This means that the logging library needs to behave well when an application uses the library from multiple threads. Making code thread-safe adds some extra overhead to the code and is not needed if the library will only be used from a single thread.
What we need is a test that calls log from multiple threads that are all running at the same time. Let’s write a test with the code we have now and see what happens. We’re going to be using the logging project in this section and adding a new file to the tests folder called Thread.cpp. The project structure will look like this with the new file added:
MereMemo project root folder MereTDD folder Test.h MereMemo folder Log.h tests folder main.cpp Construction.cpp LogTags.h Tags.cpp Thread.cpp Util.cpp Util.hInside the Thread.cpp file, let’s add a test that calls log from several threads, like so:
#include "../Log.h"#include "Util.h"#include <MereTDD/Test.h>#include <thread>TEST("log can be called from multiple threads"){ // We'll have 3 threads with 50 messages each. std::vector<std::string> messages; for (int i = 0; i < 150; ++i) { std::string message = std::to_string(i); message += " thread-safe message "; message += Util::randomString(); messages.push_back(message); } std::vector<std::thread> threads; for (int c = 0; c < 3; ++c) { threads.emplace_back( [c, &messages]() { int indexStart = c * 50; for (int i = 0; i < 50; ++i) { MereMemo::log() << messages[indexStart + i]; } }); } for (auto & t : threads) { t.join(); } for (auto const & message: messages) { bool result = Util::isTextInFile(message, "application.log"); CONFIRM_TRUE(result); }}This test does three things. First, it creates 150 messages. We’ll get the messages ready before we start the threads so that the threads will be able to call log as quickly as possible many times in a loop.
Once the messages are ready, the test starts 3 threads, and each thread will log part of the messages that have already been formatted. The first thread will log messages 0 to 49. The second thread will log messages 50 to 99. Finally, the third thread will log messages 100 to 149. We don’t do any confirmations in the threads.
Once everything has been logged and the threads have been joined, then the test confirms that all 150 messages appear in the log file.
Building and running this will almost certainly fail. This type of test goes against one of the points that makes a good test, as explained in Chapter 8, What Makes A Good Test? The reason this is not the best type of test is that the test is not completely reproducible. Each time the test application is run, you’ll get a slightly different result. You might even find that this test causes other tests to fail!
Even though we’re not basing the behavior of the test on random numbers, we’re using threads. And thread scheduling is unpredictable. The only way to make this test mostly reliable is to log many messages like we’re doing already. The test does everything it can to set the threads up for conflicts. This is why the messages are preformatted. I wanted the threads to immediately go into a loop of logging messages and not spend any extra time formatting messages.
When the test fails, it’s because the log file is jumbled. One portion of the log file looks like this for one of my test runs:
2022-08-16T04:54:54.635 100 thread-safe message 40492022-08-16T04:54:54.635 100 thread-safe message 40492022-08-16T04:54:54.635 0 thread-safe message 88662022-08-16T04:54:54.637 101 thread-safe message 82712022-08-16T04:54:54.637 1 thread-safe message 32052022-08-16T04:54:54.637 102 thread-safe message 75142022-08-16T04:54:54.637 51 thread-safe message 74052022-08-16T04:54:54.637 2 thread-safe message 57232022-08-16T04:54:54.637 52 thread-safe message 44682022-08-16T04:54:54.637 52 thread-safe message 4468I removed the color and log_level tags so that you can see the messages better. The first thing you’ll notice is that some messages are repeated. Number 100 appears twice, and number 50 seems to be missing completely.
To be honest, I expected the log file to be even more jumbled than it is. The interleaving between message groups 0-49 and 50-99 and 100-149 is to be expected. We do have three threads running at the same time. For example, once message number 51 is logged, we should expect to have already seen number 50.
Let’s fix the logging code to get the test to pass. It still won’t be the best test but it will have a good chance of finding a bug if the logging library is not thread-safe.
The fix is simple: we need a mutex and then we need to lock the mutex. First, let’s include the mutex standard header at the top of Log.h, like this:
#include <algorithm>#include <chrono>#include <ctime>#include <filesystem>#include <fstream>#include <iomanip>#include <map>#include <memory>#include <mutex>#include <ostream>#include <sstream>#include <string>#include <string_view>#include <vector>Then, we need a place to place a global mutex. Since the logging library is a single header file, we can’t declare a global variable without getting a linker error. We might be able to declare a global mutex as inline. This is a new feature in C++ that I haven’t used that lets you declare inline variables, just like how we can declare inline functions. I’m more comfortable with a function that uses a static variable. Add the following function to the top of Log.h, right after the opening namespace of MereMemo:
inline std::mutex & getLoggingMutex (){ static std::mutex m; return m;}Now, we need to lock the mutex at the proper spot. At first, I added a lock to the log function, but that had no effect. This is because the log function returns a LogStream without actually doing any logging. So, the log function obtained the lock and then released the lock before any logging happened. The logging is done in the LogStream destructor, so that’s where we need to put the lock:
~LogStream () { if (not mProceed) { return; } const std::lock_guard<std::mutex> lock(getLoggingMutex()); auto & outputs = getOutputs(); for (auto const & output: outputs) { output->sendLine(this->str()); } }The lock tries to obtain the mutex and will block if another thread already owns the mutex. Only one thread at a time can proceed after the lock and the lock is released after the text is sent to all the outputs.
If we build and run, the threading problem will be fixed. However, when I ran the test application, one of the tests failed. At first, I thought there was still a problem with the threads, but the failure was in another test. This is the test that failed:
TEST("Overridden default tag not used to filter messages"){ MereTDD::SetupAndTeardown<TempFilterClause> filter; MereMemo::addFilterLiteral(filter.id(), info); std::string message = "message "; message += Util::randomString(); MereMemo::log(debug) << message; bool result = Util::isTextInFile(message, "application.log"); CONFIRM_FALSE(result);}This test has nothing to do with the multiple thread test. So, why did it fail? Well, the problem is that this test is confirming that a particular message does not appear in the log file. But the message is just the word "message ", followed by a random number string. We just added an extra 150 logged messages, which all have the same text followed by a random number string.
We have a problem with the tests themselves. The tests can sometimes fail due to random numbers. The problem wasn’t noticed when we had a few log messages but it’s more noticeable now that we have many more chances for duplicate random numbers.
We could either increase the size of the random number strings added to each log message or make the tests more specific so that they all use a different base message string.
At this point, you might be wondering why my test has a simple base message when we’ve been using unique messages in each test ever since the logging library was first created in Chapter 9, Using Tests. That’s because the code starting in Chapter 9, Using Tests, originally did have simple, common log messages. I could have left those common messages as-is and waited until now to have you go back and change all of them. However, I edited the chapters to fix the problem from the beginning. It seems like a waste to go through all the tests now just to change a string. Therefore, I added an explanation to Chapter 9, Using Tests. We don’t need to change any of the test messages now because they’ve already been fixed.
Okay, back to the threading – the new test passes now and the sample from the log file looks much better:
2022-08-16T06:20:36.807 0 thread-safe message 62692022-08-16T06:20:36.807 50 thread-safe message 18092022-08-16T06:20:36.807 100 thread-safe message 62972022-08-16T06:20:36.808 1 thread-safe message 8482022-08-16T06:20:36.808 51 thread-safe message 41032022-08-16T06:20:36.808 101 thread-safe message 55702022-08-16T06:20:36.808 2 thread-safe message 61562022-08-16T06:20:36.809 102 thread-safe message 42132022-08-16T06:20:36.809 3 thread-safe message 6646Again, this sample has been modified to remove the color and log_level tags. This change makes each line shorter so that you can see the messages better. The messages within each thread are ordered, even though the messages are mixed between threads – that is, message number 0 is followed at some point by message number 1 and then by number 2; message number 50 is followed later by number 51, and message number 100 is followed by number 101. Each following numbered message might not immediately follow the previous message. This sample looks better because there are no duplicates and no missing messages.
One final thought is about the thread-safety of the logging library. We tested that multiple threads can all safely call log without worrying about problems. But we didn’t test if multiple threads can manage default tags or filtering, or add new outputs. The logging library will likely need more work to be fully thread-safe. It will work for our purposes for now.
Now that the logging library is mostly thread-safe, the next section will go back to the SimpleService project and begin exploring how to test code that uses multiple threads.
The need to justify multiple threads
So far in this chapter, you’ve learned how to write tests that use multiple threads and how to use these extra threads to test the logging library. The logging library doesn’t use multiple threads itself, but we needed to make sure that the logging library is safe to use with multiple threads.
The remainder of this chapter will provide some guidance on how to test code that does use multiple threads. To test multi-threaded code, we need some code that uses multiple threads. For this, we’ll use the SimpleService project from the previous chapter.
We need to modify the simple service so that it uses multiple threads. Right now, the simple service is an example of a greeting service that responds to a greeting request with a reply based on the user making the request being identified. There’s not much of a need for multiple threads in a greeting service. We’re going to need something different.
This brings us to the first guidance: we need to make sure there is a valid need for multiple threads before we try to add multiple threads. Writing multi-threaded code is hard and should be avoided if only a single thread is needed. If you only need a single thread, then make sure that you follow the advice from the previous section and make your code thread-safe if it will be used by multiple threads.
What you want to do is write as much of your code as possible so that it’s single-threaded. If you can identify a particular way to calculate a result that only needs some input data to arrive at an output, then make that a single-threaded calculation if possible. If the amount of input data is large and can be divided and calculated separately, then break up the input and pass smaller pieces to your calculation. Keep the calculation single-threaded and focused on working with the input provided. Then, you can create multiple threads where each thread is given a portion of the input data to calculate. This will separate your multi-threaded code from your calculations.
Isolating your single-threaded code will let you design and test the code without you having to worry about thread management. Sure, you might need to make sure the code is thread-safe, but that’s easier when thread-safety is all you need to worry about.
Testing multiple threads is harder because of the randomness of the thread scheduling. If possible, try to avoid clunky methods such as sleeping to coordinate tests. You want to avoid putting actual code threads to sleep to coordinate the order between threads. When a thread goes to sleep, it stops running for a while, depending on how long of a delay is specified in the sleep call. Other threads that are not sleeping can then be scheduled to run.
We’ll design the code in this chapter to let the test control the thread’s synchronization so that we can remove the randomness and make the tests predictable. Instead of starting this section with a test, let’s look at a modified service that has a reason to use multiple threads. The modified handleRequest method looks like this:
std::string SimpleService::Service::handleRequest ( std::string const & user, std::string const & path, std::string const & request){ MereMemo::log(debug, User(user), LogPath(path)) << "Received: " << Request(request); std::string response; if (request == "Calculate") { response = "token"; } if (request == "Status") { response = "result"; } else { response = "Unrecognized request."; } MereMemo::log(debug, User(user), LogPath(path)) << "Sending: " << Response(response); return response;}When following TDD, you’ll normally want to start with tests first. So, why am I showing you a modified service first? Because our goal is to test multi-threaded code. In your projects, you should avoid the desire to use some technology without having a good reason. Our reason is that we need an example to learn from. So, we’re starting with a backward need to use multi-threading.
I tried to think of a good reason for a greeting service to use multiple threads and nothing came to mind. So, we’re going to change the service to something a little more complicated; I want to explain this new idea before we begin writing tests.
The new service is still as simple as I can make it. We’ll continue ignoring all the networking and message routing. We’ll need to change the request and response types to structs and we’ll continue to ignore serializing the data structs for transmission to and from the service.
The new service will simulate the calculation of a difficult problem. One valid reason to create a new thread is to let the new thread perform some work while the original thread continues what it was doing. The idea of the new service is that a Calculate request can take a long time to complete and we don’t want the caller to time out while waiting for the result. So, the service will create a new thread to perform the calculation and immediately return a token to the caller. The caller can use this token to call back into the service with a different Status request, which will check on the progress of the calculation that was just begun. If the calculation is not done yet, then the response to the Status request will let the caller know approximately how much has been completed. If the calculation is done, then the response will contain the answer.
We now have a justification for multiple threads and can write some tests. Let’s take care of an unrelated test that should have been added already. We want to make sure that anybody calling the service with an unrecognized request will get an unrecognized response. Put the following test in the Message.cpp file in the tests folder of the SimpleService project:
TEST_SUITE("Unrecognized request is handled properly", "Service 1"){ std::string user = "123"; std::string path = ""; std::string request = "Hello"; std::string expectedResponse = "Unrecognized request."; std::string response = gService1.service().handleRequest( user, path, request); CONFIRM_THAT(response, Equals(expectedResponse));}I put this test at the top of Message.cpp. All it does is send the previous greeting request but with an unrecognized expected response.
Let’s also change the name of the test suite to "Calculation Service" like this in SetupTeardown.cpp:
MereTDD::TestSuiteSetupAndTeardown<ServiceSetup>gService1("Calculation Service", "Service 1");Now, let’s remove the greeting test and add the following simple test, which makes sure we get something other than the unrecognized response:
TEST_SUITE("Calculate request can be sent and recognized", "Service 1"){ std::string user = "123"; std::string path = ""; std::string request = "Calculate"; std::string unexpectedResponse = "Unrecognized request."; std::string response = gService1.service().handleRequest( user, path, request); CONFIRM_THAT(response, NotEquals(unexpectedResponse));}This test is the opposite of the unrecognized test and makes sure that the response is something other than unrecognized. Normally, it’s better to confirm that a result matches what you expect to happen instead of confirming that a result is not what you don’t expect. A double negative is not only harder to think about, but can lead to problems because it’s not possible to catch all the ways something can go wrong. By confirming what you want to happen, you can eliminate all the possible error conditions, which are too many to catch individually.
This test is a little different, though. We’re not interested in the response. The test only intends to confirm that the request was recognized. Confirming that the response is not unrecognized is appropriate, even though it seems similar to the double negative trap we just described.
Building and running this code shows that the unrecognized test passes but the Calculate request fails:
Running 1 test suites--------------- Suite: Service 1------- Setup: Calculation ServicePassed------- Test: Unrecognized request is handled properlyPassed------- Test: Calculate request can be sent and recognizedFailed confirm on line 30 Expected: not Unrecognized request. Actual : Unrecognized request.------- Teardown: Calculation ServicePassed-----------------------------------Tests passed: 3Tests failed: 1It seems that we’re getting an unrecognized response for a request that should be valid. This is the value of adding simple tests at the beginning of a project. The tests help catch simple errors right away. The problem is in the handleRequest method. I added the second check for a valid request by copying the first check and forgot to change the if statement to an else if statement. The fix for this is as follows:
if (request == "Calculate") { response = "token"; } else if (request == "Status") { response = "result"; } else { response = "Unrecognized request."; }To continue further, we’re going to send and receive more than strings. When we send a Calculate request, we should get back a token value that we can pass to the Status request. The Status response should then contain either the answer or an estimate of how much progress has been made. Let’s take this one step at a time and define the Calculate request and response structures. Add the following two struct definitions to the top of Service.h inside the SimpleService namespace:
struct CalculateRequest{ int mSeed;};struct CalculateResponse{ std::string mToken;};This will let us pass some initial value to be calculated; in return, we will get a token that we can use to eventually get the answer. But we have a problem. If the Calculate request is changed to return a struct, then that will break the existing test, which expects a string. We should change the tests so that they use the structs, but that leads to another problem: most of the time, we need to return the correct response struct. And we need to return an error response for error cases.
What we need is a response that can represent both a good response and an error response. Since we’re going to have a response that can serve multiple purposes, why not let it also handle a struct for the Status response? This means we’ll have a single response type that can be either an error response, a calculate response, or a status response. And since we have a multi-purpose response type, why not create a multi-purpose request type? Let’s change the tests.
We’re going to use std::variant to hold the different types of requests and responses. We can remove the test that sent a request string that was not valid. We can still get an invalid request but only with mismatched service versions between the caller and the service. That’s a little more involved, so we’ll ignore the possibility that a service can be called with a different idea of what requests are available than the service knows about. If you’re writing a real service, then this is a possibility that needs to be addressed and tested. You’ll probably want to use something different than a variant too. A good choice would be something such as Google’s Protocol Buffers, where the service would accept Protocol Buffer messages. While using Protocol Buffers is a better choice than simple structs, the design is also a lot more complicated and would make this explanation much longer.
We’ll have a single test in Message.cpp that will look like this:
TEST_SUITE("Calculate request can be sent", "Service 1"){ std::string user = "123"; std::string path = ""; SimpleService::RequestVar request = SimpleService::CalculateRequest { .mSeed = 5 }; std::string emptyResponse = ""; std::string response = gService1.service().handleRequest( user, path, request); CONFIRM_THAT(response, NotEquals(emptyResponse));}This test focuses on the request type first and leaves the response type as a string. We’ll make the changes one step at a time. This is especially good advice when working with std::variant because it can be challenging if you’re not familiar with variants. We’ll have a variant type called RequestVar that can be initialized with a specific request type. We’re initializing the request with a CalculateRequest and using the designated initializer syntax to set the mSeed value. The designated initializer syntax is fairly new to C++. It lets us set data member values based on the name by putting a dot in front of the data member’s name.
Now, let’s define the request types in Service.h:
#ifndef SIMPLESERVICE_SERVICE_H#define SIMPLESERVICE_SERVICE_H#include <string>#include <variant>namespace SimpleService{struct CalculateRequest{ int mSeed;};struct StatusRequest{ std::string mToken;};using RequestVar = std::variant< CalculateRequest, StatusRequest >;Note that we need to include the standard variant header file. The RequestVar type can now only be either a CalculateRequest or a StatusRequest. We need to make one more change in Service.h to the handleRequest method in the Service class:
class Service{public: void start (); std::string handleRequest (std::string const & user, std::string const & path, RequestVar const & request);};The Service.cpp file needs to be changed so that it updates the handleRequest method, like this:
std::string SimpleService::Service::handleRequest ( std::string const & user, std::string const & path, RequestVar const & request){ std::string response; if (auto const * req = std::get_ if<CalculateRequest>(&request)) { MereMemo::log(debug, User(user), LogPath(path)) << "Received Calculate request for: " << std::to_string(req->mSeed); response = "token"; } else if (auto const * req = std::get_ if<StatusRequest>(&request)) { MereMemo::log(debug, User(user), LogPath(path)) << "Received Status request for: " << req->mToken; response = "result"; } else { response = "Unrecognized request."; } MereMemo::log(debug, User(user), LogPath(path)) << "Sending: " << Response(response); return response;}The updated handleRequest method continues to check for an unknown request type. All the responses are strings that will need to change. We’re not looking at the seed or token values yet, but we have enough that can be built and tested.
Now that the single test passes, in the next section, we will look at the responses and use structs instead of response strings.
Changing the service return type
We’ll be making a similar change in this section to move away from strings and use a struct in the service request handling. The previous section changed the service request type; this section will change the service return type. We need to make these changes so that we can get the service to a level of functionality where it can support the need for an additional thread.
The SimpleService project that we’re using started as a greeting service and I could not think of any reason for such a simple service to need another thread. We started adapting the service to a calculation service in the previous section; now, we need to modify the return types that the service returns when handling requests.
First, let’s define the return type structs in Service.h, which come right after the request types. Add the following code to Service.h:
struct ErrorResponse{ std::string mReason;};struct CalculateResponse{ std::string mToken;};struct StatusResponse{ bool mComplete; int mProgress; int mResult;};using ResponseVar = std::variant< ErrorResponse, CalculateResponse, StatusResponse >;These structs and the variant are following the same pattern that was used for the requests. One small difference is that we now have an ErrorResponse type, which will be returned for any errors. We can modify the test in Message.cpp so that it looks like this:
TEST_SUITE("Calculate request can be sent", "Service 1"){ std::string user = "123"; std::string path = ""; SimpleService::RequestVar request = SimpleService::CalculateRequest { .mSeed = 5 }; auto const responseVar = gService1.service().handleRequest( user, path, request); auto const response = std::get_if<SimpleService::CalculateResponse>(&responseVar); CONFIRM_TRUE(response != nullptr);}This test will call the service as it did previously with a calculate request; the response that comes back is tested to see if it is a calculate response.
For the code to compile, we need to change the handleRequest declaration in Service.h so that it returns the new type, like this:
class Service{public: void start (); ResponseVar handleRequest (std::string const & user, std::string const & path, RequestVar const & request);};Then, we need to change the implementation of handleRequest in Service.cpp:
SimpleService::ResponseVar SimpleService::Service::handleRequest ( std::string const & user, std::string const & path, RequestVar const & request){ ResponseVar response; if (auto const * req = std::get_ if<CalculateRequest>(&request)) { MereMemo::log(debug, User(user), LogPath(path)) << "Received Calculate request for: " << std::to_string(req->mSeed); response = SimpleService::CalculateResponse { .mToken = "token" }; } else if (auto const * req = std::get_ if<StatusRequest>(&request)) { MereMemo::log(debug, User(user), LogPath(path)) << "Received Status request for: " << req->mToken; response = SimpleService::StatusResponse { .mComplete = false, .mProgress = 25, .mResult = 0 }; } else { response = SimpleService::ErrorResponse { .mReason = "Unrecognized request." }; } return response;}The code is getting a little more complicated. I removed the log at the end, which was used to log the response before returning. We could put the log back in but that would require the ability to convert a ResponseVar into a string. Alternatively, we would need to log the response in multiple places like the code does for the request. That’s a detail that we can skip.
The new handleRequest method does almost the same things it used to do except that it now initializes a ResponseVar type instead of returning a string. This allows us to return different types with more detailed information than before when we were returning a string for both the requests and the error.
To add a test for an unrecognized request, we would need to add a new request type to RequestVar but ignore the new request type in the if statements inside the handleRequest method. We’re going to skip that test too because we really should be using something other than a std::variant.
The only reason we’re using std::variant for this example is to avoid extra complexity. We’re trying to get the code ready to support another thread.
In the next section, we will add a test that uses both request types. The first request will begin a calculation, while the second request will check the status of the calculation and get the result when the calculation is complete.
Making multiple service calls
If you’re considering using multiple threads to speed up a calculation, then I recommend that you get the code tested and working with a single thread before taking on the additional complexity of multiple threads.
For the service we’re working on, the reason to add a second thread is not to increase the speed of anything. We need to avoid a timeout for a calculation that might take a long time. The additional thread we’re going to add is not designed to make the calculation any faster. Once we get the calculation working with one additional thread, we can consider adding more threads to speed up the calculation.
The need to create a thread to do some work while the original thread continues with something else is common. This is not an optimization that should be done later. This is part of the design and the additional thread should be included from the very beginning.
Let’s begin by adding a new test to Message.cpp that looks like this:
TEST_SUITE("Status request generates result", "Service 1"){ std::string user = "123"; std::string path = ""; SimpleService::RequestVar calcRequest = SimpleService::CalculateRequest { .mSeed = 5 }; auto responseVar = gService1.service().handleRequest( user, path, calcRequest); auto const calcResponse = std::get_if<SimpleService::CalculateResponse> (&responseVar); CONFIRM_TRUE(calcResponse != nullptr); SimpleService::RequestVar statusRequest = SimpleService::StatusRequest { .mToken = calcResponse->mToken }; int result {0}; for (int i = 0; i < 5; ++i) { responseVar = gService1.service().handleRequest( user, path, statusRequest); auto const statusResponse = std::get_if<SimpleService::StatusResponse> (&responseVar); CONFIRM_TRUE(statusResponse != nullptr); if (statusResponse->mComplete) { result = statusResponse->mResult; break; } } CONFIRM_THAT(result, Equals(50));}All the code is already in place for this new test to compile. Now, we can run the tests to see what happens. The test will fail, as follows:
Running 1 test suites--------------- Suite: Service 1------- Setup: Calculation ServicePassed------- Test: Calculate request can be sentPassed------- Test: Status request generates resultFailed confirm on line 62 Expected: 50 Actual : 0------- Teardown: Calculation ServicePassed-----------------------------------Tests passed: 3Tests failed: 1What does the test do? First, it creates a calculate request l and gets back a hardcoded token value. There is no calculation for when the service begins yet, so when we make a status request with the token, the service responds with a hardcoded response that says the calculation is not done yet. The test is looking for a status response that says the calculation is complete. The test tries making a status request five times before giving up, which causes the confirmation at the end of the test to fail because we didn’t get the expected result. Note that even trying multiple times is not the best way to proceed. Threads are unpredictable and your computer may make all five attempts before the service can complete the request. You might need to increase the number of attempts if your test continues to fail or wait for a reasonable amount of time. Our calculation will eventually multiply the seed by 10. So, when we give an initial seed of 5, we should expect a final result of 50.
We need to implement the calculation and status request handling in the service so that we can use a thread to get the test to pass. The first thing we need to do is include mutex, thread, and vector at the top of Service.cpp. We also need to add an unnamed namespace, like this:
#include "Service.h"#include "LogTags.h"#include <MereMemo/Log.h>#include <mutex>#include <thread>#include <vector>namespace{}We’re going to need some locking so that we don’t try to read the calculation status while the status is being updated by a thread. To do the synchronization, we’ll use a mutex and a lock, as we did in the logging library. There are other designs you might want to explore, such as locking data for different calculation requests separately. We’re going to use a simple approach and have a single lock for everything. Add the following function inside the unnamed namespace:
std::mutex & getCalcMutex () { static std::mutex m; return m; }We need something to keep track of the completion status, the progress, and the result for each calculation request. We’ll create a class to hold this information called CalcRecord inside the unnamed namespace, right after the getCalcMutex function, like this:
class CalcRecord { public: CalcRecord () { } CalcRecord (CalcRecord const & src) { const std::lock_guard<std::mutex> lock(getCalcMutex()); mComplete = src.mComplete; mProgress = src.mProgress; mResult = src.mResult; } void getData (bool & complete, int & progress, int & result) { const std::lock_guard<std::mutex> lock(getCalcMutex()); complete = mComplete; progress = mProgress; result = mResult; } void setData (bool complete, int progress, int result) { const std::lock_guard<std::mutex> lock(getCalcMutex()); mComplete = complete; mProgress = progress; mResult = result; } CalcRecord & operator = (CalcRecord const & rhs) = delete; private: bool mComplete {false}; int mProgress {0}; int mResult {0}; };It looks like there’s a lot more to this class, but it’s fairly simple. The default constructor doesn’t need to do anything because the data members already define their default values. The only reason we need a default constructor is that we also have a copy constructor. And the only reason we need a copy constructor is so that we can lock the mutex before copying the data members.
Then, we have a method to get the data members all at once and another method to set the data members. Both the getter and the setter need to acquire the lock before proceeding.
There should be no need to assign one CalcRecord to another, so the assignment operator has been deleted.
The last thing we need in the unnamed namespace is a vector of CalcRecord, like this:
std::vector<CalcRecord> calculations;We’re going to add a CalcRecord to the calculations collection every time a calculation request is made. A real service would want to clean up or reuse CalcRecord entries.
We need to modify the request handling in Service.cpp so that a thread gets created to use a new CalcRecord every time we get a calculation request, like this:
if (auto const * req = std::get_ if<CalculateRequest>(&request)) { MereMemo::log(debug, User(user), LogPath(path)) << "Received Calculate request for: " << std::to_string(req->mSeed); calculations.emplace_back(); int calcIndex = calculations.size() - 1; std::thread calcThread([calcIndex] () { calculations[calcIndex].setData(true, 100, 50); }); calcThread.detach(); response = SimpleService::CalculateResponse { .mToken = std::to_string(calcIndex) }; }What happens when we get a calculation request? First, we add a new CalcRecord to the end of the calculations vector. We’ll use the index of CalcRecord as the token that gets returned in the response. This is the simplest design I could think of to identify a calculation request. A real service would want to use a more secure token. The request handler then starts a thread to do the calculation and detaches from the thread.
Most threading code that you’ll write will create a thread and then join the thread. It’s not very common to create a thread and then detach from the thread. Alternatively, you can use a pool of threads when you want to do some work and not worry about joining. The reason for detaching is that I wanted the most simple example without bringing in thread pools.
The thread itself is very simple because it immediately sets CalcRecord to complete with a progress of 100 and a result of 50.
We can build and run the test application now, but we will get the same failure we did previously. That’s because the status request handling still returns a hardcoded response. We need to modify the request handler like this for the status request:
else if (auto const * req = std::get_ if<StatusRequest>(&request)) { MereMemo::log(debug, User(user), LogPath(path)) << "Received Status request for: " << req->mToken; int calcIndex = std::stoi(req->mToken); bool complete; int progress; int result; calculations[calcIndex].getData(complete, progress, result); response = SimpleService::StatusResponse { .mComplete = complete, .mProgress = progress, .mResult = result }; }With this change, the status request converts the token into an index that it uses to find the correct CalcRecord. Then, it gets the current data from CalcRecord to be returned in the response.
You may also want to consider adding sleep to the test loop that attempts five service call requests so that the total time given to the service is reasonable. The current test will fail if all five attempts are made quickly before the service has time to complete even a simple calculation.
All the tests pass after building and running the test application. Are we done now? Not yet. All of these changes let the service calculate a result in a separate thread while continuing to handle requests on the main thread. The whole point of adding another thread is to avoid timeouts due to calculations that take a long time. But our calculation is very quick. We need to slow the calculation down so that we can test the service with a reasonable response time.
How will we slow down the thread? And what amount of time should the calculation require to complete? These are the questions that we’ve been building code to answer in this chapter. The next section will explain how you can test services that use multiple threads. And now that we have a service that uses another thread for the calculation, we can explore the best way to test this situation.
I’d also like to clarify that what the next section does is different than adding a delay to the five service call attempts. A delay in the test loop will improve the reliability of the test we have now. The next section will remove the loop completely and show you how to coordinate a test with another thread so that both the test and the thread proceed together.
How to test multiple threads without sleep
Earlier in this chapter, in the The need to justify multiple threads section, I mentioned that you should try to do as much work as possible with single threads. We’re going to follow this advice now. In the current request handling for the calculate request, the code creates a thread that does a simple calculation, like this:
std::thread calcThread([calcIndex] () { calculations[calcIndex].setData(true, 100, 50); });Okay, maybe a simple calculation is the wrong way to describe what the thread does. The thread sets the result to a hardcoded value. We know this is temporary code and that we’ll need to change the code to multiply the seed value by 10, which is what the tests expect.
Where should the calculation be done? It would be easy to do the calculation in the thread lambda, but that would go against the advice of doing as much work as possible with a single thread.
What we want to do is create a calculation function that the thread can call. This will let us test the calculation function separately without worrying about any threading issues and make sure that the calculation is correct.
And here’s the really interesting part: creating a function to do the calculation will help us test the thread management too! How? Because we’re going to create two calculation functions.
One function will be the real calculation function, which can be tested independently of any threads. For our project, the real calculation will still be simple and fast. We’re not going to try to do a lot of work to slow down the calculation and we’re not going to put the thread to sleep either. And we’re not going to write a bunch of tests to make sure the calculation is correct. This is just an example of a pattern that you can follow in your projects.
The other function will be a test calculation function and will do some fake calculations designed to match the real calculation result. The test calculation function will also contain some thread management code designed to coordinate the thread’s activity. We’ll use the thread management code in the test calculation function to slow down the thread so that we can simulate a calculation that takes a long time.
What we’re doing is mocking the real calculation with code that is less focused on the calculation and more focused on the thread’s behavior. Any test that wants to test the real calculation can use the real calculation function, while any test that wants to test the thread timing and coordination can use the test calculation function.
First, we’ll declare the two functions in Service.h right before the Service class, like this:
void normalCalc (int seed, int & progress, int & result);void testCalc (int seed, int & progress, int & result);You can define your calculation functions in your projects to do whatever you need. Your functions will likely be different. The main point to understand is that they should have the same signature so that the test function can be substituted for the real function.
The Service class needs to be changed so that one of these functions can be injected into the service. We’ll set up the calculation function in the constructor and use the real function as the default, like this:
class Service{public: using CalcFunc = void (*) (int, int &, int &); Service (CalcFunc f = normalCalc) : mCalc(f) { } void start (); ResponseVar handleRequest (std::string const & user, std::string const & path, RequestVar const & request);private: CalcFunc mCalc;};The Service class now has a member function pointer that will point to one of the calculation functions. Which one will be called is determined when the Service class is created.
Let’s implement the two functions in Service.cpp, like this:
void SimpleService::normalCalc ( int seed, int & progress, int & result){ progress = 100; result = seed * 10;}void SimpleService::testCalc ( int seed, int & progress, int & result){ progress = 100; result = seed * 10;}At the moment, both functions are the same. We’ll take this one step at a time. Each function just sets progress to 100 and result to seed times 10. We’re going to leave the real or normal function as-is. Eventually, we’ll change the test function so that it controls the thread.
Now, we can change the calculate request handler in Service.cpp so that it uses the calculation function, like this:
if (auto const * req = std::get_ if<CalculateRequest>(&request)) { MereMemo::log(debug, User(user), LogPath(path)) << "Received Calculate request for: " << std::to_string(req->mSeed); calculations.emplace_back(); int calcIndex = calculations.size() - 1; int seed = req->mSeed; std::thread calcThread([this, calcIndex, seed] () { int progress; int result; mCalc(seed, progress, result); calculations[calcIndex].setData(true, progress, result); }); calcThread.detach(); response = SimpleService::CalculateResponse { .mToken = std::to_string(calcIndex) }; }In the thread lambda, we call mCalc instead of setting progress and result to hardcoded values. Which calculation function is called depends on which function mCalc points to.
If we build and run the test application, we’ll see that the tests pass. But there’s something wrong with how we’re calling mCalc. We want to get intermediate progress so that a caller can make status requests and see the progress increasing until the calculation is finally complete. By calling mCalc once, we only give the function one chance to do something. We should be calling the mCalc function in a loop until progress reaches 100 percent. Let’s change the lambda code:
std::thread calcThread([this, calcIndex, seed] () { int progress {0}; int result {0}; while (true) { mCalc(seed, progress, result); if (progress == 100) { calculations[calcIndex].setData(true, progress, result); break; } else { calculations[calcIndex].setData(false, progress, result); } } });This change does not affect the tests because the mCalc function currently sets progress to 100 on the first call; therefore, the while loop will only run once. We don’t want the thread to take too long to run without some synchronization with the tests because we’ll never join with the thread. If this was a real project, we would want to use threads from a thread pool and wait for the threads to complete before stopping the service.
Making a change that does not affect the tests is a great way to verify changes. Take small steps instead of trying to do everything in one giant set of changes.
Next, we’re going to duplicate the test that generates a result except we will use the test calculation function in the duplicate test. The test will need to be modified slightly so that it can use the test calculation function. But for the most part, the test should remain almost identical. The new test goes in Message.cpp and looks like this:
TEST_SUITE("Status request to test service generates result", "Service 2"){ std::string user = "123"; std::string path = ""; SimpleService::RequestVar calcRequest = SimpleService::CalculateRequest { .mSeed = 5 }; auto responseVar = gService2.service().handleRequest( user, path, calcRequest); auto const calcResponse = std::get_if<SimpleService::CalculateResponse> (&responseVar); CONFIRM_TRUE(calcResponse != nullptr); SimpleService::RequestVar statusRequest = SimpleService::StatusRequest { .mToken = calcResponse->mToken }; int result {0}; for (int i = 0; i < 5; ++i) { responseVar = gService2.service().handleRequest( user, path, statusRequest); auto const statusResponse = std::get_if<SimpleService::StatusResponse> (&responseVar); CONFIRM_TRUE(statusResponse != nullptr); if (statusResponse->mComplete) { result = statusResponse->mResult; break; } } CONFIRM_THAT(result, Equals(40));}The only changes are to give the test a different name so that it uses a new test suite called "Service 2", and then use a different global service called gService2. Here, we expect a slightly different result. We’ll be changing this test soon so that it will eventually contribute more value than it does now, and we’ll be removing the loop that tries to make the request five times. Making these changes in small steps will let us verify that we don’t break anything major. And expecting a slightly different result will let us verify that we are using a different calculation function.
To build the project, we need to define gService2, which will use a new setup and teardown class. Add the following code to SetupTeardown.h:
class TestServiceSetup{public: TestServiceSetup () : mService(SimpleService::testCalc) { } void setup () { mService.start(); } void teardown () { } SimpleService::Service & service () { return mService; }private: SimpleService::Service mService;};extern MereTDD::TestSuiteSetupAndTeardown<TestServiceSetup>gService2;The TestServiceSetup class defines a constructor that initializes the mService data member with the testCalc function. The gService2 declaration uses TestServiceSetup. We need to make a small change in SetupTeardown.cpp for gService2, like so:
#include "SetupTeardown.h"MereTDD::TestSuiteSetupAndTeardown<ServiceSetup>gService1("Calculation Service", "Service 1");MereTDD::TestSuiteSetupAndTeardown<TestServiceSetup>gService2("Calculation Test Service", "Service 2");The SetupTeardown.cpp file is short and only needs to define instances of gService1 and gService2.
We need to change the testCalc function so that it will multiply by 8 to give an expected result of 40 instead of 50. Here are both calculation functions in Service.cpp:
void SimpleService::normalCalc ( int seed, int & progress, int & result){ progress = 100; result = seed * 10;}void SimpleService::testCalc ( int seed, int & progress, int & result){ progress = 100; result = seed * 8;}Building and running the test application shows that all the tests pass. We now have two test suites. The output looks like this:
Running 2 test suites--------------- Suite: Service 1------- Setup: Calculation ServicePassed------- Test: Calculate request can be sentPassed------- Test: Status request generates resultPassed------- Teardown: Calculation ServicePassed--------------- Suite: Service 2------- Setup: Calculation Test ServicePassed------- Test: Status request to test service generates resultPassed------- Teardown: Calculation Test ServicePassed-----------------------------------Tests passed: 7Tests failed: 0Here, we introduced a new service that uses a slightly different calculation function and can use both services in the tests. The tests pass with minimal changes. Now, we’re ready to make more changes to coordinate the threads. This is a better approach than jumping directly into the thread management code and adding the new service and calculation function.
When following TDD, the process is always the same: get the tests to pass, make small changes to the tests or add new tests, and get the tests to pass again.
The next step will complete this section. We’re going to control the speed at which the testCalc function works so that we can make multiple status requests to get a complete result. We’ll wait inside the test calculation function so that the test has time to verify that the progress does indeed increase over time until the result is finally calculated once the progress reaches 100%.
Let’s start with the test. We’re going to signal the calculation thread from within the test thread so that the calculation thread will progress in-step with the test. This is what I meant by testing multiple threads without using sleep. Sleeping within a thread is not a good solution because it’s not reliable. You might be able to get a test to pass only to have the same test fail later when the timing changes. The solution you’ll learn here can be applied to your testing.
All you need to do is create a test version of part of your code that can be substituted for the real code. In our case, we have a testCalc function that can be substituted for the normalCalc function. Then, you can add one or more condition variables to your test and wait on those condition variables from within the test version of your code. A condition variable is a standard and supported way in C++ to let one thread wait until a condition is met before proceeding. The test calculation function will wait on the condition variable. The test will notify the condition variable when it’s ready for the calculation to continue. Notifying the condition variable will unblock the waiting calculation thread at exactly the right time so that the test can verify the proper thread behavior. Then, the test will wait until the calculation has been completed before continuing. We’ll need to include condition_variable at the top of Service.h, like this:
#ifndef SIMPLESERVICE_SERVICE_H#define SIMPLESERVICE_SERVICE_H#include <condition_variable>#include <string>#include <variant>Then, we need to declare a mutex, two condition variables, and two bools in Service.h so that they can be used by the test calculation function and by the test. Let’s declare the mutex, condition variables, and the bools right before the test calculation function, like this:
void normalCalc (int seed, int & progress, int & result);extern std::mutex service2Mutex;extern std::condition_variable testCalcCV;extern std::condition_variable testCV;extern bool testCalcReady;extern bool testReady;void testCalc (int seed, int & progress, int & result);Here is the modified test in Message.cpp:
TEST_SUITE("Status request to test service generates result", "Service 2"){ std::string user = "123"; std::string path = ""; SimpleService::RequestVar calcRequest = SimpleService::CalculateRequest { .mSeed = 5 }; auto responseVar = gService2.service().handleRequest( user, path, calcRequest); auto const calcResponse = std::get_if<SimpleService::CalculateResponse> (&responseVar); CONFIRM_TRUE(calcResponse != nullptr); // Make a status request right away before the service // is allowed to do any calculations. SimpleService::RequestVar statusRequest = SimpleService::StatusRequest { .mToken = calcResponse->mToken }; responseVar = gService2.service().handleRequest( user, path, statusRequest); auto statusResponse = std::get_if<SimpleService::StatusResponse> (&responseVar); CONFIRM_TRUE(statusResponse != nullptr); CONFIRM_FALSE(statusResponse->mComplete); CONFIRM_THAT(statusResponse->mProgress, Equals(0)); CONFIRM_THAT(statusResponse->mResult, Equals(0)); // Notify the service that the test has completed the first // confirmation so that the service can proceed with the // calculation. { std::lock_guard<std::mutex> lock(SimpleService::service2Mutex); SimpleService::testReady = true; } SimpleService::testCV.notify_one(); // Now wait until the service has completed the calculation. { std::unique_lock<std::mutex> lock(SimpleService::service2Mutex); SimpleService::testCalcCV.wait(lock, [] { return SimpleService::testCalcReady; }); } // Make another status request to get the completed result. responseVar = gService2.service().handleRequest( user, path, statusRequest); statusResponse = std::get_if<SimpleService::StatusResponse> (&responseVar); CONFIRM_TRUE(statusResponse != nullptr); CONFIRM_TRUE(statusResponse->mComplete); CONFIRM_THAT(statusResponse->mProgress, Equals(100)); CONFIRM_THAT(statusResponse->mResult, Equals(40));}The test is a bit longer than it used to be. We’re no longer making status requests in a loop while looking for a completed response. This test takes a more deliberate approach and knows exactly what it expects at each step. The initial calculation request and calculation response are the same. The test knows that the calculation will be paused, so the first status request will return an uncompleted response with zero progress.
After the first status request has been confirmed, the test notifies the calculation thread that it can continue and then the test waits. Once the calculation is complete, the calculation thread will notify the test that the test can continue. At all times, the test and the calculation thread are taking turns, which lets the test confirm each step. There is a small race condition in the test calculation thread that I’ll explain after you’ve seen the code. A race condition is a problem where two or more threads can interfere with each other and the result is not completely predictable.
Let’s look at the other half now – the test calculation function. We need to declare the mutex, condition variables, and the bools too. The variables and the test calculation function should look like this:
std::mutex SimpleService::service2Mutex;std::condition_variable SimpleService::testCalcCV;std::condition_variable SimpleService::testCV;bool SimpleService::testCalcReady {false};bool SimpleService::testReady {false};void SimpleService::testCalc ( int seed, int & progress, int & result){ // Wait until the test has completed the first status request. { std::unique_lock<std::mutex> lock(service2Mutex); testCV.wait(lock, [] { return testReady; }); } progress = 100; result = seed * 8; // Notify the test that the calculation is ready. { std::lock_guard<std::mutex> lock(service2Mutex); testCalcReady = true; } testCalcCV.notify_one();}The first thing that the test calculation function does is wait. No calculation progress will be made until the test has a chance to confirm the initial status. Once the test calculation thread is allowed to proceed, it needs to notify the test before returning so that the test can make another status request.
The most important thing to understand about this process is that the test calculation function should be the only code interacting with the test. You shouldn’t put any waits or notifications in the main service response handler or even in the lambda that is defined in the response handler. Only the test calculation function that gets swapped out for the real calculation function should have any awareness that a test is being run. In other words, you should put all the waiting and condition variable notifications in testCalc. This is the source of the race condition that I mentioned. When the testCalc function notifies the test thread that the calculation is complete, it’s not completely correct. The calculation is only complete when setData finishes updating CalcRecord. However, we don’t want to send the notification after calling setData because that would put the notification outside of the testCalc function.
Ideally, we would change the design so that the calculation function is called one additional time after completing the calculation. We could say that this gives the calculation function a chance to clean up any resources used during the calculation. Or maybe we can create another set of functions for cleaning up. One cleanup function could be the normal cleanup, while the other function could be substituted for test cleanup. Either approach would let us notify the test that the calculation has finished, which would eliminate the race condition.
Building and running these tests shows that all the tests continue to pass. We’re almost done. We’ll leave the race condition as-is because fixing it would only add extra complexity to this explanation. The only remaining task is to fix a problem that I noticed in the log file. I’ll explain more about this new problem in the next section.
Fixing one last problem detected with logging
There’s a big reason why I choose to build a logging library in Part 2, Logging Library, of this book. Logging can be a huge help when debugging known problems. Something that’s often overlooked is the benefit that logging provides when looking for bugs that haven’t been detected yet.
I’ll often look at the log file after running tests to make sure the messages match what I expect. After making the enhancements in the previous section for the thread coordination between the test and the test calculation thread, I noticed something strange in the log file. The log file looks like this:
2022-08-27T05:00:50.409 Service is starting.2022-08-27T05:00:50.410 user="123" Received Calculate request for: 52022-08-27T05:00:50.411 user="123" Received Calculate request for: 52022-08-27T05:00:50.411 user="123" Received Status request for: 12022-08-27T05:00:50.411 Service is starting.2022-08-27T05:00:50.411 Service is starting.2022-08-27T05:00:50.411 user="123" Received Calculate request for: 52022-08-27T05:00:50.411 user="123" Received Calculate request for: 52022-08-27T05:00:50.411 user="123" Received Status request for: 22022-08-27T05:00:50.411 user="123" Received Status request for: 22022-08-27T05:00:50.411 user="123" Received Status request for: 22022-08-27T05:00:50.411 user="123" Received Status request for: 2I removed the log_level and logpath tags just to shorten the messages so that you can see the important parts better. The first strange thing that I noticed is that the service was started three times. We only have gService1 and gService2, so the service should only have been started twice.
The first four lines in the log file make sense. We start gService1 and then run a simple test that requests a calculation and checks that the response is of the proper type. Then, we run another test that makes a status request up to five times while looking for a complete response. The first status request finds the complete response, so no additional status requests are needed. The token for the first status request is 1.
Line 5 in the log file, which is where the service is started for the second time, is where the log file begins to look strange. We should only need to start the second service, make a single additional request, and then make two status requests. It looks like the log file is getting duplicate messages from line 5 until the end.
After a little debugging and the hint that we’re duplicating log messages, I found the problem. When I originally designed the service, I configured the logging in the Service::start method. I should have kept the logging configuration in the main function. Everything worked until we needed to create and start a second service so that the second service could be configured to use a test calculation function. Well, the second service was also configuring the logging when it started, and it added another file output. The second file’s output caused all the log messages to be sent to the log file twice. The solution is simple: we need to configure the logging in main like this:
#include <MereMemo/Log.h>#include <MereTDD/Test.h>#include <iostream>int main (){ MereMemo::FileOutput appFile("logs"); MereMemo::addLogOutput(appFile); return MereTDD::runTests(std::cout);}Then, we need to remove the logging configuration from the service start method so that it looks like this:
void SimpleService::Service::start (){ MereMemo::log(info) << "Service is starting.";}With these changes, the tests still pass and the log file looks better. Again, I removed some tags to shorten the log message lines. Now, the content of the log file is as follows:
2022-08-27T05:35:30.573 Service is starting.2022-08-27T05:35:30.574 user="123" Received Calculate request for: 52022-08-27T05:35:30.574 user="123" Received Calculate request for: 52022-08-27T05:35:30.574 user="123" Received Status request for: 12022-08-27T05:35:30.574 Service is starting.2022-08-27T05:35:30.574 user="123" Received Calculate request for: 52022-08-27T05:35:30.574 user="123" Received Status request for: 22022-08-27T05:35:30.575 user="123" Received Status request for: 2While the problem ended up being a mistake in how the logging was configured, the point I wanted to make is to remind you to look through the log files periodically and make sure the log messages make sense.
Summary
This is the last chapter of this book and it explained one of the most confusing and difficult aspects of writing software: how to test multiple threads. You’ll find a lot of books that explain multi-threading but fewer will give you advice and show you effective ways to test multiple threads.
Because the target customer of this book is a microservices C++ developer who wants to learn how to use TDD to design better software, this chapter tied everything in this book together to explain how to test multi-threaded services.
First, you learned how to use multiple threads in your tests. You need to make sure you handle exceptions inside tests that start additional threads. Exceptions are important because the testing library uses exceptions to handle failed confirmations. You also learned how to use a special helper class to report failed confirmations that arise in additional threads.
Threads must also be considered when writing and using libraries. You saw how to test a library to make sure it’s thread-safe.
Finally, you learned how to test multi-threaded services in a fast and reliable manner that avoids putting threads to sleep in an attempt to coordinate the actions of multiple threads. You learned how to refactor your code so that you can test as much as possible in a single-threaded manner and then how to substitute the normal code for special test-aware code that works with a test. You can use this technique any time you need a test and multi-threaded code to work together so that the test can take specific and reliable steps and confirm your expectations along the way.
Congratulations on reaching the end of this book! This chapter visited all the projects we’ve been working on. We enhanced the unit testing library to help you use multiple threads in your tests. We also made the logging library thread-safe. Finally, we enhanced the service so that it can coordinate multiple threads between the service and the tests. You now have all the skills you’ll need to apply TDD to your projects.