11
Managing Dependencies
Identifying dependencies and implementing your code around common interfaces that the dependencies use will help you in many ways. You’ll be able to do the following things:
- Avoid waiting for another team or even yourself to finish a complicated and necessary component
- Isolate your code and make sure it works, even if there are bugs in other code that you use
- Achieve greater flexibility with your designs so that you can change the behavior by simply changing dependent components
- Create interfaces that clearly document and highlight essential requirements
In this chapter, you’ll learn what dependencies are and how to design your code to use them. By the end of this chapter, you’ll learn how to finish writing your code faster and prove that it works, even if the rest of the project is not ready for it yet.
You don’t need to be using TDD to design and use dependencies. But if you are using TDD, then the whole process becomes even better because you’ll also be able to write better tests that can focus on specific areas of code without worrying about extra complexity and bugs coming from outside the code.
This chapter will cover the following main topics:
- Designing with dependencies
- Adding multiple logging outputs
Technical requirements
All code in this chapter uses standard C++ that builds on any modern C++ 20 or later compiler and standard library. The code uses the testing library from Part 1, Testing MVP, of this book and continues the development of a logging library started in the previous chapters.
You can find all the code for this chapter in the following GitHub repository:
https://github.com/PacktPublishing/Test-Driven-Development-with-CPP
Designing with dependencies
Dependencies are not always obvious. If a project uses a library, such as how the logging project uses the unit test library, then that’s an easy dependency to spot. The logging project depends on the unit test library to function correctly. Or in this case, only the logging tests depend on the unit test library. But that’s enough to form a dependency.
Another easy dependency to spot is if you need to call another service. Even if the code checks to see if the other service is available first before making a call, the dependency still exists.
Libraries and services are good examples of external dependencies. You have to do extra work to get a project to use the code or services of another project, which is why an external dependency is so easy to spot.
Other dependencies are harder to spot, and these are usually internal dependencies within the project. In a way, almost all the code in a project depends on the rest of the code doing what it’s supposed to do. So let’s refine what we mean by a dependency. Normally, when a dependency is mentioned, as it relates to code design, we refer to something that can be exchanged.
This might be easiest to understand with the external service dependency example. The service operates on its own with a well-defined interface. You make a request to a service based on its location or address using the interface that the service defines. You could instead call a different service for the same request if the first service is not available. Ideally, the two services would use the same interface so that the only thing your code needs to change is the address.
If the two services use different interfaces, then it might make sense to create a wrapper for each service that knows how to translate between what each service expects and a common interface that your code will use. With a common interface, you can swap one service for another without changing your code. Your code depends on the service interface definition more than any specific service.
If we look at internal design decisions, maybe there is a base class and a derived class. The derived class definitely depends on the base class, but this is not the type of dependency that can be changed without rewriting the code to use a different base class.
We get closer to a dependency that can be swapped when considering the tags that the logging library defines. New tags can be defined and used without changing existing code. And the logging library can use any tag without worrying about what each tag does. But are we really swapping out tags? To me, the tags were designed to solve the problem of logging key=value elements in the log file in a consistent manner that does not depend on the data type of the value. Even though the logging library depends on tags and the interface they use, I wouldn’t classify the tag design as the same type of dependency as the external service.
I mentioned early on when thinking about the logging library that we will need the ability to send the log information to different destinations, or maybe even multiple destinations. The code uses the log function and expects it to either be ignored or to go somewhere. The ability to send a log message to a specific destination is a dependency that the logging library needs to rely on. The logging library should let the project doing the logging decide on the destination.
And this brings us to another aspect of dependencies. A dependency is often something that is configured. What I mean is that we can say that the logging library depends on some component to perform the task of sending a message to a destination. The logging library can be designed to choose its own destination, or the logging library can be told what dependency to use. When we let other code control the dependencies, we get something called dependency injection. You get a more flexible solution when you let the calling code inject dependencies.
Here’s some initial code that I put into the main function to configure a component that knows how to send log messages to a file and then inject the file component into the logger so that the logger will know where to send the log messages:
int main (){ MereMemo::FileOutput appFile("application.log"); appFile.maxSize() = 10'000'000; appFile.rolloverCount() = 5; MereMemo::addLogOutput(appFile); MereMemo::addDefaultTag(info); MereMemo::addDefaultTag(green); return MereTDD::runTests(std::cout);}The idea is to create a class called FileOutput and give it the name of the file to write log messages. Because we don’t want log files to get too big, we should be able to specify a maximum size. The code uses 10 million bytes for the maximum size. What do we do when a log file reaches the maximum size? We should stop writing to that file and create a new file. We should be able to specify how many log files to create before we start deleting old log files. The code sets the maximum number of log files to five.
Once the FileOutput instance is created and configured the way we want, it is injected into the logging library by calling the addLogOutput function.
Will this code meet our needs? Is it intuitive and easy to understand? Even though this is not a test, we’re still following TDD by concentrating on the usage of a new feature before writing the code to implement the new feature.
As for meeting our needs, that’s not really the right question to ask. We need to ask if it will meet the needs of our target customer. We’re designing the logging library to be used by a micro-services developer. There might be hundreds of services running on a server computer and we really should place the log files in a specific location. The first change we’ll need is to let the caller specify a path where the log files should be created. The path seems like it should be separate from the filename.
And for the filenames, how will we name multiple log files? They can’t all be called application.log. Should the files be numbered? They will all be placed in the same directory and the only requirement that the filesystem needs is that each file has a unique name. We need to let the caller provide a pattern for the log filenames instead of a single filename. A pattern will let the logging library know how to make the name unique while still following the overall naming style that the developer wants. We can change the initial code to look like this instead:
MereMemo::FileOutput appFile("logs"); appFile.namePattern() = "application-{}.log"; appFile.maxSize() = 10'000'000; appFile.rolloverCount() = 5; MereMemo::addLogOutput(appFile);When designing a class, it’s a good idea to make the class work with reasonable defaults after construction. For file output, the bare minimum we need is the directory to create the log files. The other properties are nice but not required. If the name pattern is not provided, we can default to a simple unique number. The max size can have an unlimited default, or at least a really big number. And we only need a single log file. So, the rollover count can be some value that tells us to use a single file.
I decided to use simple open and close curly braces {} for the placeholder in the pattern where a unique number will be placed. We’ll just pick a random three-digit number to make the log filename unique. That will give us up to a thousand log files, which should be more than enough. Most users will only want to keep a handful and delete older files.
Because the output is a dependency that can be swapped or even have multiple outputs at the same time, what would a different type of output look like? We’ll figure out what the output dependency component interface will be later. For now, we just want to explore how to use different outputs. Here is how output can be sent to the std::cout console:
MereMemo::StreamOutput consoleStream(std::cout); MereMemo::addLogOutput(consoleStream);The console output is an ostream so we should be able to create a stream output that can work with any ostream. This example creates an output component called consoleStream, which can be added to the log output just like the file output.
When using TDD, it’s important to avoid interesting features that may not really be needed by the customer. We’re not going to add the ability to remove outputs. Once an output is added to the logging library, it will remain. In order to remove an output, we’d have to return some sort of identifier that can be used to later remove the same output that was added. We did add the ability to remove filter clauses because that ability seemed likely to be needed. Removing outputs is something that seems unlikely for most customers.
In order to design a dependency that can be swapped for another, we’ll need a common interface class that all outputs implement. The class will be called Output and goes in Log.h right before the LogStream class, like this:
class Output{public: virtual ~Output () = default; Output (Output const & other) = delete; Output (Output && other) = delete; virtual std::unique_ptr<Output> clone () const = 0; virtual void sendLine (std::string const & line) = 0; Output & operator = (Output const & rhs) = delete; Output & operator = (Output && rhs) = delete;protected: Output () = default;};The only methods that are part of the interface are the clone and the sendLine methods. We’ll follow a similar cloning pattern as the tags, except we’re not going to use templates. The sendLine method will be called whenever a line of text needs to be sent to the output. The other methods make sure that nobody can construct instances of Output directly or copy or assign one Output instance to another. The Output class is designed to be inherited from.
We’ll keep track of all the outputs that have been added with the next two functions, which go right after the Output class like this:
inline std::vector<std::unique_ptr<Output>> & getOutputs (){ static std::vector<std::unique_ptr<Output>> outputs; return outputs;}inline void addLogOutput (Output const & output){ auto & outputs = getOutputs(); outputs.push_back(output.clone());}The getOutputs function uses a static vector of unique pointers and returns the collection when requested. The addLogOutput function adds a clone of the given output to the collection. This is all similar to how the default tags are handled.
One interesting use of dependencies that you should be aware of is their ability to swap out a real component for a fake component. We’re adding two real components to manage the logging output. One will send output to a file and the other to the console. But you can also use dependencies if you want to make progress on your code and are waiting for another team to finish writing a needed component. Instead of waiting, make the component a dependency that you can swap out for a simpler version. The simpler version is not a real version, but it should be faster to write and let you continue making progress until the real version becomes available.
Some other testing libraries take this fake dependency ability a step further and let you create components with just a few lines of code that respond in various ways that you can control. This lets you isolate your code and make sure it behaves as it should because you can rely on the fake dependency to always behave as specified, and you no longer have to worry about bugs in the real dependency affecting the results of your tests. The common term for these fake components is mocks.
It doesn’t matter if you are using a testing library that generates a mock for you with a few lines of code or if you are writing your own mock. Anytime you have a class that imitates another class, you have a mock.
Other than isolating your code from bugs, a mock can also help speed up your tests and improve collaboration with other teams. The speed is improved because the real code might need to spend time requesting or calculating a result, while the mock can return quickly without the need to do any real work. Collaboration with other teams is improved because everybody can agree to simple mocks that are quick to develop and can be used to communicate design changes.
The next section will implement the file and stream output classes based on the common interface. We’ll be able to simplify the LogStream class and the log function to use the common interface, which will document and make it easier to understand what is really needed to send log messages to an output.
Adding multiple logging outputs
A good way to validate that a design works with multiple scenarios is to implement solutions for each scenario. We have a common Output interface class that defines two methods, clone and sendLine, and we need to make sure this interface will work for sending log messages to a log file and to the console.
Let’s start with a class called FileOutput that inherits from Output. The new class goes in Log.h right after the getOutputs and the addLogOutput functions, like this:
class FileOutput : public Output{public: FileOutput (std::string_view dir) : mOutputDir(dir), mFileNamePattern("{}"), mMaxSize(0), mRolloverCount(0) { } FileOutput (FileOutput const & rhs) : mOutputDir(rhs.mOutputDir), mFileNamePattern(rhs.mFileNamePattern), mMaxSize(rhs.mMaxSize), mRolloverCount(rhs.mRolloverCount) { } FileOutput (FileOutput && rhs) : mOutputDir(rhs.mOutputDir), mFileNamePattern(rhs.mFileNamePattern), mMaxSize(rhs.mMaxSize), mRolloverCount(rhs.mRolloverCount), mFile(std::move(rhs.mFile)) { } ~FileOutput () { mFile.close(); } std::unique_ptr<Output> clone () const override { return std::unique_ptr<Output>( new FileOutput(*this)); } void sendLine (std::string const & line) override { if (not mFile.is_open()) { mFile.open("application.log", std::ios::app); } mFile << line << std::endl; mFile.flush(); }protected: std::filesystem::path mOutputDir; std::string mFileNamePattern; std::size_t mMaxSize; unsigned int mRolloverCount; std::fstream mFile;};The FileOutput class follows the usage that was determined in the previous section, which looks like this:
MereMemo::FileOutput appFile("logs"); appFile.namePattern() = "application-{}.log"; appFile.maxSize() = 10'000'000; appFile.rolloverCount() = 5; MereMemo::addLogOutput(appFile);We give the FileOutput class a directory in the constructor where the log files will be saved. The class also supports a name pattern, a max log file size, and a rollover count. All the data members need to be initialized in the constructors and we have three constructors.
The first constructor is a normal constructor that accepts the directory and gives default values to the other data members.
The second constructor is the copy constructor, and it initializes the data members based on the values in the other instance of FileOutput. Only the mFile data member is left in a default state because we don’t copy fstreams.
The third constructor is the move copy constructor, and it looks almost identical to the copy constructor. The only difference is that we now move the fstream into the FileOutput class being constructed.
The destructor will close the output file. This is actually a big improvement over what was done up to this point. We used to open and close the output file each time a log message was made. We’ll now open the log file and keep it open until we need to close it at a later time. The destructor makes sure that the log file gets closed if it hasn’t already been closed.
Next is the clone method, which calls the copy constructor to create a new instance that gets sent back as a unique pointer to the base class.
The sendLine method is the last method, and it needs to check whether the output file has been opened already or not before sending the line to the file. We’ll add the ending newline here after each line gets sent to the output file. We also flush the log file after every line, which helps to make sure that the log file contains everything written to it in case the application doing the logging crashes suddenly.
The last thing we need to do in the FileOutput class is to define the data members. We’re not going to fully implement all the data members though. For example, you can see that we’re still opening a file called application.log instead of following the naming pattern. We have the basic idea already and skipping the data members will let us test this part to make sure we haven’t broken anything. We’ll need to comment out the configuration in the main function, so it looks like this for now:
MereMemo::FileOutput appFile("logs"); //appFile.namePattern() = "application-{}.log"; //appFile.maxSize() = 10'000'000; //appFile.rolloverCount() = 5; MereMemo::addLogOutput(appFile);We can always come back to the configuration methods and the directory later once we get the multiple outputs working in a basic manner. This follows the TDD practice of doing as little as possible each step along the way. In a way, what we’re doing is creating a mock for the ultimate FileOutput class.
I almost forgot to mention that because we’re using filesystem features, such as path, we need to include filesystem at the top of Log.h, like this:
#include <algorithm>#include <chrono>#include <ctime>#include <filesystem>#include <fstream>#include <iomanip>#include <map>#include <memory>#include <ostream>#include <sstream>#include <string>#include <string_view>#include <vector>We’ll make use of filesystem more once we start rolling log files over into new files instead of always opening the same file each time.
Next is the StreamOutput class, which can go in Log.h right after the FileOutput class and looks like this:
class StreamOutput : public Output{public: StreamOutput (std::ostream & stream) : mStream(stream) { } StreamOutput (StreamOutput const & rhs) : mStream(rhs.mStream) { } std::unique_ptr<Output> clone () const override { return std::unique_ptr<Output>( new StreamOutput(*this)); } void sendLine (std::string const & line) override { mStream << line << std::endl; }protected: std::ostream & mStream;};The StreamOutput class is simpler than the FileOutput class because it has fewer data members. We only need to keep track of an ostream reference that gets passed in the constructor in main. We also don’t need to worry about a specific move copy constructor because we can easily copy the ostream reference. The StreamOutput class was already added in main like this:
MereMemo::StreamOutput consoleStream(std::cout); MereMemo::addLogOutput(consoleStream);The StreamOutput class will hold a reference to std::cout that main passes to it.
Now that we’re working with the output interface, we no longer need to manage a file in the LogStream class. The constructors can be simplified to no longer worry about an fstream data member, like this:
LogStream () : mProceed(true) { } LogStream (LogStream const & other) = delete; LogStream (LogStream && other) : std::stringstream(std::move(other)), mProceed(other.mProceed) { }The destructor of the LogStream class is where all the work happens. It no longer needs to send the message directly to a file that the class manages. The destructor now gets all the outputs and sends the message to each one using the common interface, like this:
~LogStream () { if (not mProceed) { return; } auto & outputs = getOutputs(); for (auto const & output: outputs) { output->sendLine(this->str()); } }Remember that the LogStream class inherits from std::stringstream and holds the message to be logged. If we are to proceed, then we can get the fully formatted message by calling the str method.
The end of LogStream no longer needs the mFile data member and only needs the mProceed flag, like this:
private: bool mProceed;};Because we removed the LogStream constructor parameters for the filename and open mode, we can simplify how the LogStream class gets created in the log function like this:
inline LogStream log (std::vector<Tag const *> tags = {}){ auto const now = std::chrono::system_clock::now(); std::time_t const tmNow = std::chrono::system_clock::to_time_t(now); auto const ms = duration_cast<std::chrono::milliseconds>( now.time_since_epoch()) % 1000; LogStream ls; ls << std::put_time(std::gmtime(&tmNow), "%Y-%m-%dT%H:%M:%S.") << std::setw(3) << std::setfill('0') << std::to_string(ms.count());We can now construct the ls instance without any arguments, and it will use all the outputs that have been added.
Let’s check the test application by building and running the project. The output to the console looks like this:
Running 1 test suites--------------- Suite: Single Tests------- Test: Message can be tagged in log2022-07-24T22:32:13.116 color="green" log_level="error" simple 7809Passed------- Test: log needs no namespace when used with LogLevel2022-07-24T22:32:13.118 color="green" log_level="error" no namespacePassed------- Test: Default tags set in main appear in log2022-07-24T22:32:13.118 color="green" log_level="info" default tag 9055Passed------- Test: Multiple tags can be used in log2022-07-24T22:32:13.118 color="red" log_level="debug" size="large" multi tags 7933Passed------- Test: Tags can be streamed to log2022-07-24T22:32:13.118 color="green" log_level="info" count=1 1 type 32472022-07-24T22:32:13.118 color="green" log_level="info" id=123456789012345 2 type 64802022-07-24T22:32:13.118 color="green" log_level="info" scale=1.500000 3 type 68812022-07-24T22:32:13.119 color="green" log_level="info" cache_hit=false 4 type 778Passed------- Test: Tags can be used to filter messages2022-07-24T22:32:13.119 color="green" log_level="info" filter 1521Passed------- Test: Overridden default tag not used to filter messagesPassed------- Test: Inverted tag can be used to filter messagesPassed------- Test: Tag values can be used to filter messages2022-07-24T22:32:13.119 color="green" count=101 log_level="info" values 8461Passed------- Test: Simple message can be logged2022-07-24T22:32:13.120 color="green" log_level="info" simple 9466 with more text.Passed------- Test: Complicated message can be logged2022-07-24T22:32:13.120 color="green" log_level="info" complicated 9198 double=3.14 quoted="in quotes"Passed-----------------------------------Tests passed: 11Tests failed: 0You can see that the log messages did indeed go to the console window. The log messages are included in the console alongside the test results. What about the log file? It looks like this:
2022-07-24T22:32:13.116 color="green" log_level="error" simple 78092022-07-24T22:32:13.118 color="green" log_level="error" no namespace2022-07-24T22:32:13.118 color="green" log_level="info" default tag 90552022-07-24T22:32:13.118 color="red" log_level="debug" size="large" multi tags 79332022-07-24T22:32:13.118 color="green" log_level="info" count=1 1 type 32472022-07-24T22:32:13.118 color="green" log_level="info" id=123456789012345 2 type 64802022-07-24T22:32:13.118 color="green" log_level="info" scale=1.500000 3 type 68812022-07-24T22:32:13.119 color="green" log_level="info" cache_hit=false 4 type 7782022-07-24T22:32:13.119 color="green" log_level="info" filter 15212022-07-24T22:32:13.119 color="green" count=101 log_level="info" values 84612022-07-24T22:32:13.120 color="green" log_level="info" simple 9466 with more text.2022-07-24T22:32:13.120 color="green" log_level="info" complicated 9198 double=3.14 quoted="in quotes"The log file contains only the log messages, which are the same log messages that were sent to the console window. This shows that we have multiple outputs! There’s no good way to verify that the log messages are being sent to the console window, such as how we can open the log file and search for a specific line.
But we could add yet another output using the StreamOutput class that is given std::fstream instead of std::cout. We can do this because fstream implements ostream, which is all that the StreamOutput class needs. This is also dependency injection because the StreamOutput class depends on an ostream and we can give it any ostream we want it to use, like this:
#include <fstream>#include <iostream>int main (){ MereMemo::FileOutput appFile("logs"); //appFile.namePattern() = "application-{}.log"; //appFile.maxSize() = 10'000'000; //appFile.rolloverCount() = 5; MereMemo::addLogOutput(appFile); MereMemo::StreamOutput consoleStream(std::cout); MereMemo::addLogOutput(consoleStream); std::fstream streamedFile("stream.log", std::ios::app); MereMemo::StreamOutput fileStream(streamedFile); MereMemo::addLogOutput(fileStream); MereMemo::addDefaultTag(info); MereMemo::addDefaultTag(green); return MereTDD::runTests(std::cout);}We’re not going to make this change. It’s just for demonstration purposes only. But it shows that you can open a file and pass that file to the StreamOutput class to use instead of the console output. If you do make this change, then you’ll see that the stream.log and application.log files are the same.
Why would you want to consider using StreamOutput as if it was FileOutput? And why do we need FileOutput if StreamOutput can also write to a file?
First of all, FileOutput is specialized for files. It will eventually know how to check the current file size to make sure it doesn’t get too big and roll over to a new log file whenever the current log file approaches the maximum size. There is a need for file management that StreamOutput will not even be aware of.
The StreamOutput class is simpler though because it doesn’t need to worry about files at all. You might want to use StreamOutput to write to a file in case the FileOutput class takes too long to create. Sure, we created a simplified FileOutput without all the file management features, but another team might not be so willing to give you a partial implementation. You might find it better to use a mock solution while you wait for a full implementation.
The ability to swap one implementation for another is a big advantage you get with properly managed dependencies.
In fact, this book will leave the current implementation of FileOutput as it is now because finishing the implementation would take us into topics that have little to do with learning about TDD.
Summary
We not only added a great new feature to the logging library that lets it send log messages to multiple destinations but we also added this ability using an interface. The interface helps document and isolate the idea of sending lines of text to a destination. This helped uncover a dependency that the logging library has. The logging library depends on the ability to send text somewhere.
The destination could be a log file or the console, or somewhere else. Until we identified this dependency, the logging library was making assumptions in many places that it was working with a log file only. We were able to simplify the design and, at the same time create a more flexible design.
We were also able to get the file logging working without a complete file logging component. We created a mock of the file logging component that leaves out all the additional file management tasks that a full implementation will need. While useful, the additional capabilities are not needed right now, and the mock will let us proceed without them.
The next chapter will go back to the unit testing library and will show you how to enhance the confirmations to a new style that is extensible and easier to understand.