A Conceptual Framework for Testing
For conceptual guidance, I use two principles to guide the overall shape of my testing efforts. First and foremost, I focus on the purpose of the software I am testing. Second, I actively work to reduce the degree of coupling introduced by the tests. Let’s consider purpose now; we will discuss coupling in the next chapter.
The purpose of code applies differently at different levels. At a system level, the purpose of the code is the reason the software exists: its features, requirements, and use cases. Critical evaluation of the purpose helps to constrain your test cases and acceptance criteria. Is it important that a user interface element is a particular color or size or that it is aligned a particular way? You may not need to verify particular database fields if the value of the application is simply that the data or state is persisted and can be retrieved; exercising the retrieval and verifying the results may be sufficient.
At a module or component level, purpose refers to a function in the overall system. Integration tests satisfy the needs at this level. A module may overtly implement a feature, directly bearing the responsibility for the functionality the users expect. Alternatively, a module may implement an underlying, enabling design component. Either way, the module serves a role in the overall system. That role should be clear, and it needs to be tested.
At a unit- or isolation-test level, I like to think of the purpose as the value added by the code. Those who live in countries like Mexico or the European Union with a value-added tax (VAT) may find this concept clear. A company only pays a VAT on the amount of value it adds to a product. Raw materials or component parts have a value when you receive them that is subtracted out of the value of the product you sell for purpose of taxation. Similarly, your code takes the libraries and collaborators on which it is built and adds an additional level of functionality or purpose for its consumers: the value added by that code.
Defining a unit test has caused considerable debate, but the value-added perspective gives us an alternative. Many have used definitions like that put forth by Michael Feathers3 stating what a unit test does not do. I feel this merely codifies some test-design heuristics as rules and prefer to use the value-added concept for an inclusive definition of a unit test.
3. See his blog entry on the topic at www.artima.com/weblogs/viewpost.jsp?thread=126923. See also [WEwLC].
A unit test is a test that verifies the value added by the code under test. Any use of independently testable collaborators is simply a matter of convenience.
With this definition, use of an untestable method in the same class falls within the value added by the code under test. A testable method called from the code under test should have independent tests and therefore need not be part of the verification except to the extent that it adds value to the code under test. Use of other classes can be mocked, stubbed, or otherwise test doubled because they are independently testable. You can use databases, networks, or file systems at the expense of test performance, although I would not recommend it.
Every subset of the system, from the smallest method to the entire system itself, should have a well-defined purpose. If you find yourself struggling to test the subset, you may not know its purpose. If you do not know the purpose of the subset, it probably has some implementation, design, or architectural issues. You have just diagnosed current or future problems maintaining and extending your software simply by looking at it through a particular lens in order to test it.