In Chapter 6, we discussed the Release/Reuse Equivalence principle. It’s the first principle of package cohesion: it tells an important part of the story about which classes belong together in a package, namely those that you can properly release and maintain as a package. You need to take care of delivering a package that is a true product.
If you follow all the advice given in the previous chapter, you will have a well-behaving package. It has great usability and it’s easily available, so it will be quickly adopted by other developers. But even when a package behaves well as a package, it may at the same time not be very useful.
Let’s say you have a nice collection of very useful classes, implementing several interesting features. When you group those classes into packages, there are two extremes that need to be avoided. If you release all the classes as one package, you force your users to pull the entire package into their project, even if they use just a very small part of it. This is quite a maintenance burden for them.
On the other hand, if you put every single class in a separate package, you will have to release a lot of packages. This increases your own maintenance burden. At the same time, users have a hard time managing their own list of dependencies and keeping track of all the new versions of those tiny packages.
Classes that are used together are packaged together.
So when you design a package you should put classes in it that are going to be used together. This may seem a bit obvious; you wouldn’t put completely unrelated classes that are never used together in one package. But things become somewhat less obvious when we consider the other side of this principle—you should not put classes in a package that are not used together. This includes classes that are likely not to be used together (which leaves the user with irrelevant code imported into their project).
In this chapter, we look at some packages that obviously violate this rule: they contain all sorts of classes that are not used together—either because those classes implement isolated features or because they have different dependencies. Sometimes the package maintainer puts those classes in the same package because they have some conceptual similarity. Some may think it enhances the usability of the package for them or its users.
At the end of this chapter, we try to formulate the principle in a more positive way and we discuss some guiding questions that can be used to make your packages conform to the Common Reuse principle.
There are many signs, or “smells,” by which you can recognize a package that violates the Common Reuse principle. I’ll discuss some of these signs, using some real-world packages as examples. A quick word before we continue—in no way do I want to dispute the greatness of these packages. They are well-established packages, created by expert developers, and used by many people all over the world. However, I don’t agree with some of the package design choices that were made. So you should take my comments not as angry criticism, but as a gesture toward what I think is the ideal package design approach.
Feature Strata
A layer of material, naturally or artificially formed, often one of a number of parallel layers, one upon another. 2
I’d like to define “feature strata” as features existing together in the same package, but not dependent on each other. This means that you would be able to use feature A without feature B, but adding feature B is possible without disturbing feature A. It also means that afterward, disabling feature B is no problem and won’t cause feature A to break. Feature A and B don’t touch; they work in parallel.
In the context of packages, feature strata often manifest themselves as classes that belong together because they implement some specific feature. But then after one feature has been implemented, the maintainer of the package kept adding new features, consisting of conglomerates of classes to the same package. In most cases, this happens because the features are conceptually, but not materially, related.
Obvious Stratification
The Directory Tree of the symfony/security Package
Classes from these major namespaces don’t have to be used together at the same time: classes in Acl only depend on Core, classes in Http depend on classes in Core and optionally on classes in Csrf. Not the other way around. This means that if someone were to install this package to use the Csrf classes , they would only use a small subset of this package. So if we think about what the Common Reuse principle was all about (“Classes that are used together are packaged together”), then this is a clear violation of this principle.
As you can see, some of the major namespaces are split up into minor namespaces. And as it turns out, many of the classes from the minor namespaces can be used separately from classes in the other namespaces. This again indicates that the Common Reuse principle has been violated, and that the package should be split. Fortunately, the package maintainers already decided to split this package into several other packages, so now at least the major namespaces have their own package definition file and can be installed separately.
Obfuscated Stratification
In many other cases, a package has feature strata that aren’t so easy to spot. The classes that are grouped around a certain functionality are not separated by their namespace, but by some other principle of division, for instance by the type of the class. Take a look at the nelmio/security-bundle package,5 which contains NelmioSecurityBundle,6 which can be used in Symfony projects to add some specific security measures that are not provided by the framework itself. Because of the nature of the Symfony HttpKernel,7 everything related to security can be implemented as an event listener, which hooks into the process of converting a request to a response. Some security-related event listeners will prevent the kernel from further handling the current request (for instance, based on the protocol used), and some listeners modify the final response (e.g., encrypt cookies or session data).
The Directory Structure of the nelmio/security-bundle Package
The Available Configuration Options for the nelmio/security-bundle Package
It becomes clear that all of these listeners represent a different functionality that can be configured on its own and any of the listeners can be disabled, while the other one will still keep working. This forces us to conclude that when you use one class from this package, you will not always use all the other (nor even most of the other) classes inside this package. And thus, the package violates the Common Reuse principle in a very clear way. Somebody who wants to use only one of the features provided by this bundle (and they can!) still has to install the entire bundle.
HTTPS Handling Can’t Be Configured to be “Forced” and “Flexible” at the Same Time
This means that when you use one particular class in this package, i.e. the ForcedSslListener , you will definitely not use all other classes of this package. In fact, you will with certainty not use FlexibleSslListener. Of course, this definitely asks for a package split, so users would not have to be concerned with this exclusiveness.
Classes That Can Only Be Used When … Is Installed
It should be noted that the previous examples were about packages that provide separate feature strata, but each of those features has the same dependencies (in this case, other Symfony components or the entire Symfony framework). The following examples will be about feature strata within packages that have different dependencies themselves.
List of Files in the monolog/monolog Package (Abbreviated)
A developer can install this package and start instantiating handler classes for any storage facility that is already available in their development environment. As a user you don’t need to think about which package you should install; it’s always the main package. This would seem to have a high usability factor: isn’t this easy? Whatever your situation is, just install the monolog/monolog package and you can use it right away (this is known as the “batteries included” approach).
Optional Dependencies for the monolog/monolog Package
Why didn’t the maintainer of monolog/monolog add the ext-mongo dependency to the list of required packages? Well, imagine a developer who only wants to use the StreamHandler, which just appends log messages to a file on the local filesystem. They don’t need a Mongo database nor the mongo extension to be available. If ext-mongo would be a required dependency, installing the monolog/monolog package would force them to also install the mongo extension, even though the code that really needs the extension will never be executed in their environment. This is why ext-mongo is just a suggested dependency.
Manually Adding ext-mongo as a Project Dependency
The first issue for me is that I don’t know which version of the mongo PHP extension I need to install to be able to use the MongoDBHandler . There’s no way to find that out, other than to just try installing the latest stable version and hope that it’s supported by MongoDBHandler. I will only know this for sure if the MongoDBHandler comes with tests that fully specify its behavior. Then I could run the tests and see if they pass with the specific version of ext-mongo that I just installed. This is my first objection to the design choice of making the MongoDBHandler part of the core Monolog package.
The second objection to this approach is that it forces me to add the mongo extension to the list of dependencies of my own project. This is wrong, since it’s not a dependency of my project but of the monolog/monolog package. It’s a dependency of a class inside that package. My own project might not contain any code related to MongoDB at all, yet I have to require the mongo extension in my project because I want to use the MongoDBHandler class.
So the monolog/monolog package doesn’t handle its dependencies well, and I have to do this myself. But this doesn’t really match with the idea of a package manager, which I instruct to install each of my own dependencies and any dependencies required by these dependencies. I want all the packages I install to fully take care of their own dependencies. It’s not my responsibility to know the dependencies of each class inside a package and to guess which ones I need to manually install to be able to use them. Furthermore, I should not be the one who needs to find out which version of a dependency is compatible with the code in the package. This is the task of the package maintainer.
This reasoning applies to each of the specific handlers that the monolog/monolog package supplies. And nearly all handlers will be used exclusively by any particular user, which means that the package violates the Common Reuse principle an equal number of times. For any handler in the package, the other handlers and their optional dependencies will probably never be used at the same time.
Suggested Refactoring
The Definition File for the monolog/mongo-db-handler Package
monolog packages with explicit dependencies
Suggested Dependencies for the monolog/monolog Package
A Package Should Be “Linkable”
Let’s take one last look at MongoDBHandler, as it’s currently still a part of the monolog/monolog package. We already concluded that after installing the package, you would not be able to use this class without also installing the mongo PHP extension first. If we don’t do this, and we try to use this specific handler, we would get all kinds of errors, in particular errors related to classes that were not found. The code inside the MongoDBHandler is syntactically correct, it just doesn’t work in the context of this project.
We need to make a conceptual division here when it comes to correctness, a division that is not often made by PHP developers. In many programming languages, problems with the code can occur at compile time or at link time. Compiling code means checking its syntax, building an abstract syntax tree, and converting the higher-level code to lower-level code. The result of the compile step are object files, which need to be linked together, in order to be executable. During the link process, references to classes and functions will be verified. If a function or class is not defined in any of the object files, the linker produces an error.
One of the characteristics of PHP is that it has no link process. It compiles code, yes, but if a class or a function does not exist, it will only be noticed at runtime, and even then in most cases, at the very last moment.
I strongly believe that even though PHP allows us to be very flexible in this regard, we must teach ourselves to think more in a “compiled language” way. We have to ask ourselves: would this code compile? It should, definitely, otherwise it would just be malformed code. And regarding every explicit, non-optional dependency of my package: would this code “link”? The answer would be “No” if the MongoDBHandler stays inside the monolog/monolog package, which has the mongo extension as a suggested dependency only. If we move the MongoDBHandler to its own monolog/mongo-db-handler package, which has an explicit dependency on ext-mongo, the answer will be “Yes,” as it should be.
Static Analysis Tools as Substitutes for a Real Compiler
Over the last few years, PHP as a programming language has moved further and further into the direction of being a statically typed language. PHP code will remain a dynamic language with compilation at runtime. But the language has better type checking with every release. Besides, PHP developers compensate for what the language doesn’t offer by using all kinds of static analysis tools. These tools become more and more popular, since they help catch a lot of potential issues with the code, before the code runs on some server.
In the context of our discussion about compiling and linking, PHPStan deserves a first mention here:
PHPStan moves PHP closer to compiled languages in the sense that the correctness of each line of the code can be checked before you run the actual line. 10
Comparable tools are Psalm11 and Phan.12
There’s another tool worth mentioning here. It’s called Composer Require Checker13 and it checks whether a package uses imported symbols (classes, interfaces, etc.) that aren't part of its direct dependencies. This is very useful, since it will prevent the scenario where your package uses a class that’s inside a package that is only indirectly a dependency of your package. In other words, if Package A depends on Package B and B depends on C, then if A also depends on C, it needs to make this dependency explicit. Otherwise, if one day B stops depending on C, this will break A. If you use PhpStorm as your IDE, the PHP Inspections plugin can also tell you about implicit dependencies.
Cleaner Releases
There’s one more characteristic of the monolog/monolog package that I’d like to discuss here, which again points us in the direction of creating separate packages for each of the specific handlers.
The MongoDBHandler
The first constructor parameter $mongo has no explicit type. Instead, the validity of the argument is being checked inside the constructor and this validation step allows us to use two different kinds of $mongo objects. It should be either an instance of MongoClient or an instance of Mongo. Both are classes from the mongo PHP extension, but the Mongo class is deprecated since version 1.3.0 of the extension.
The MongoDBHandler with a Strictly Typed Constructor Argument
But if we remove the if clause, this class would be useless for people who have an older version of mongo installed on their system.
The only way to solve this dilemma is to create extra branches in the version control repository of the monolog/monolog package—a branch for version ranges of MongoDB that should receive special treatment, e.g. monolog/monolog@mongo_db_older_than_1_3_0 and monolog/monolog@mongo_db. Of course, this will soon end in a big mess. The monolog/monolog package has many more handlers that may require such treatment.
The Definition File for the monolog/mongo-db-handler Package (Revisited)
Adding Version 1.2 of monolog/monolog-db-handler as a Dependency
Someone who already has the latest version of the mongo extension would simply choose "~1.3" as the version constraint.
Splitting the package based on its (optional) dependencies is advantageous not only to the user of the package. It will also help the maintainer a lot. They can let someone else maintain the MongoDB-specific handler package, someone who already keeps a close eye on the mongo extension releases. This person doesn’t have to be able to modify the main monolog/monolog package. This package automatically becomes a more stable package, because it has fewer reasons to change (see also Chapter 8 about the Common Closure principle). This is by itself a good thing for its users, who don’t need to keep track of every new version that is released because of a change in one of the handlers they don’t use.
The Cost of Splitting
When discussing the package design principles and applying them to real-world packages, the advice is usually to split the package into smaller ones. While we’re still in the middle of discussing all the reasons for doing so, it’s good to mention that there’s a trade-off involved in splitting packages. The smaller your packages are, the more you will have of them, the more work you have to put into making new releases, managing the repositories, their issues and pull requests, etc. The larger your packages are, the more complicated they are to work with from a user perspective. The package will need to be upgraded often, and the user will pull in lots of code and lots of dependencies they don’t need.
It's good to keep this in mind when you’re a package developer. You need to find that golden middle between too many small packages, and too few large packages.
A technical solution that could be helpful is the so-called “mono-repo”. It means that the code for all your packages will be hosted in one repository. Any change to any of the packages will be committed to that repository. To make it possible for users to install every package separately, there will be a read-only repository for each of the subpackages inside the mono-repo. These sub-repositories will be updated upon every change to the main repository. For Git, this process is called “subtree split”. There’s no need to manually set it up, since there are automated solutions, like Git Subtree Split as a Service.14
Bonus Features
The Abbreviated Directory Structure of the matthiasnoback/microsoft-translator Package
While I was developing this library, I realized that my application might make many duplicate calls to the Microsoft Translator API. For instance, it would ask the translation service several times to translate the word “Submit” to Dutch. And even though this would trigger a new HTTP request every time, the response from the API would always be the same. In order to prevent these duplicate requests, I decided to add a caching layer to the package and I thought it would be a good idea to do this by wrapping the Buzz browser client in a CachedClient19 class. The CachedClient class would analyze each incoming request and look in the cache to see if it had made this request before. If so, the cached response would be returned; otherwise, the request would be forwarded to the real Buzz client, and afterwards the fresh response would be stored in the cache.
Dependency diagram of the microsoft-translator package
Though at the time I thought the design of this library was pretty good, I would now try to eliminate the dependency on Buzz. There’s nothing so special about Buzz that this package would really need it. In fact, all it needs is “some HTTP client”. Considering the need for abstraction, I would introduce an interface for HTTP clients (e.g., HttpClientInterface) and then create a separate package, called matthiasnoback/microsoft-translator-buzz, which would provide an implementation of my own HttpClientInterface that uses Buzz. Even better, I could rely on an existing abstraction for HTTP clients (e.g., HTTPlug20) or some otherwise standardized and widely supported interface, like the upcoming PHP Standards Recommendation (PSR) 18.21
But there’s some other thing that’s wrong with the design of this library: it contains this smart little CachedClient class. Since it’s indeed such a useful class, every user of this package will likely use this CachedClient class together with the other classes in the package. So there’s no immediate violation of the Common Reuse principle here. However, suppose that your project already depends on the kriswallsmith/buzz package and you need a cache implementation for the Buzz browser client. The matthiasnoback/microsoft-translator package contains such an implementation, so you would simply install it and use only its CachedClient class , and no other class from the same package. Now this makes it crystal-clear that the package does indeed violate the Common Reuse principle. If you use a class from this package, you will not use all the other classes.
Suggested Refactoring
Dependency diagram of the microsoft-translator package after moving the CachedClient class out
Wrapping the Translator and Caching Return Values of the translate() Method
The CachedClient class we just discussed was a nice example of a “bonus feature” that consists of only a single class. It seems overkill to install an entire package with many classes in your project, just to use one class. But of course, this is a sliding scale. Which number or percentage of classes would be acceptable for a package to remain intact and not be split into multiple other packages?
Guiding Questions
Does the class introduce a dependency?
If it does, is it an optional/suggested dependency? Then you have to create a new package containing this class, explicitly mentioning its dependency.
Would the class be useful without the rest of the package?
If it would, then you have to create a new package to enable users to pull in only the code they want to use.
Asking yourself these two questions, and following the advice they give, will automatically divide your packages according to their dependencies and the functionality they provide. These aspects are also the main reasons why people will select a certain package and not another one—whether or not it introduces too much or too little of the functionality they need, and whether or not its dependencies are compatible with the dependencies of their own projects. If either of these aspects don’t match with the requirements of the user, they won’t install the package.
When to Apply the Principle
You can apply the Common Reuse principle at different moments in the package development lifecycle, for instance when you’re creating a new package for existing classes. The first thing you will need to do is group the classes that are always used together and put them in a package. When you need class A and it always needs class B, it would be a bad idea to put these classes in different packages, package-a and package-b. Separating these classes would require a developer who would like to use class A to install both package-a and package-b.
But the Common Reuse principle should also constantly be applied when you’re adding new classes to an existing package. You need to check if the new class you’re about to add will always be used together with all the other classes in the package. Chances are that by adding a new class to a package you’re adding features to the package that aren’t used by everybody who would require the package as a dependency of their project.
When to Violate the Principle
As we see in the next chapter, the Common Reuse principle is a principle that can only be maximized. You cannot always follow it perfectly. There are times when you may choose to put two classes in one package that can be used separately. Strictly speaking, you would thereby violate the principle, but there may be good reasons to do so. First of all, for convenience. Every package you create needs some extra care. It requires time and energy from you as the package maintainer and there’s a limit to how many packages you can or want to maintain.
Another reason to violate the principle is that you may know all about your target audience. When you know that almost all of the developers who use your package are using it inside a project built using the Laravel framework, you can follow the Common Reuse principle less strictly and add some classes to your package, which may only make sense when someone uses the full-stack Laravel framework. These classes would normally belong in a separate package because they could in theory be used separately, but in practice they will always be used together, which allows you to put them in the same package.
Why Not to Violate the Principle
In most cases, however, there’s the following good reason to not violate the Common Reuse principle: every class that’s part of your package is susceptible to change. Maybe one of its methods contains a bug, or some of its functionality needs to be changed. Maybe its interface needs to be modified and a backward compatibility break will be introduced. As a user of the package, you need to follow all the changes and decide if and when to upgrade to a new version. This takes time, because after upgrading a package, you need to check all the parts of your project that use classes from the package. Maybe you have some automated tests for this, maybe not.
However, you can’t choose not to upgrade. You need to take care that your project depends on the latest stable versions of all packages, to prevent (future) problems like depending on deprecated or even abandoned packages. If the package you depend on is big, contains lots of classes, and is related to all kinds of things, upgrading it will be quite a problem. There are many points of contact between the code in your project and the code inside the package, which means that changes will have many side effects.
On the contrary, when the package you depend on is small, it will be easier to track the changes and less painful to upgrade the package. There will be fewer points of contact, and the chance that an upgrade will break your project is consequently much smaller.
So you greatly help the users of your package when you keep your package small and only put classes in it that they will actually use. Then they will be able to upgrade their dependencies fearlessly.
Conclusion
It’s coherent: All the classes it contains are about the same thing. Users don’t need to install a large package just to use one class or a small group of classes.
It has no “optional” dependencies: all its dependencies are true requirements; they are mentioned explicitly and have sensible version ranges. Users don’t need to manually add extra dependencies to their project.
They use dependency inversion to make dependencies abstract instead of concrete.
As an effect, they are open for extension and closed for modification. Adding or modifying an alternative implementation doesn’t mean opening the package, but creating an extra package.