Understand Your Scope

The first question you need to ask yourself is, “What am I trying to test?” This broad question includes the need to answer several more detailed questions, such as the following.

• Which use cases am I trying to verify?

• Are you trying to test the full stack, an integrated subset, or a unit?

• What technologies are you trying to verify?

• What architectural layers are you trying to verify?

• Are you testing new code, clean, well-written code, or are you rescuing a legacy hair ball?

• Can you decompose the testing problem into usefully smaller, more tractable pieces?

A full stack test gives you the ultimate verification of the fully integrated system, but at a cost. Full stack tests often run slowly due to the presence of all of the component interactions, including network, database, and file system. Full stack tests often have difficulty distinguishing nuanced behaviors and can require complicated recipes to induce error conditions because of the larger number of conditions that you must manage to achieve those results. Finally, full stack tests may be fragile or difficult to implement because of user interface and timing interactions. The smaller the scope of the test you tackle, the less these factors will have significance.

The technologies and architectural layers will naturally relate to your scope, but will steer the testing tools you use and influence your options for test doubles.¹ The testing frameworks you use to test a web interface are different from those for a ReST service, which are different from those for a Java class. Targeting the business layer may give you substitution options for the database, whereas testing stored procedures or ORM mappings may not.

1. Meszaros suggests the term “test double” as the blanket to cover the variety of stubs, fakes, mocks, etc. that can be used to support tests [xTP].

The maturity and cleanliness of the code will guide your approach significantly. If you are starting a new project, you have the power to create the future. An existing exemplary code base makes your life easy, perhaps even boring, as you focus on the task at hand instead of reverse engineering and debugging. An unfamiliar spaghetti mess requires patience, insight, and a host of techniques from Michael Feathers [WEwLC]—such as characterization testing and seam² identification—before you can even start cleaning it up and adding features to it with confidence.

2. The term seam is strongly intuitive, but to be precise, Michael Feathers defines a seam as “a place where you can alter behavior in your program without editing in that place” (p. 31).

The more independent the parts of your system, the more likely you will feel confident testing those parts separately. Just like system decomposition, breaking your testing task into multiple subtasks will simplify your work and reduce your overall coupling.