7.1. Unit tests vs. integration tests vs. functional tests
Let’s start with a brief review of the types of tests useful to an enterprise application. This knowledge isn’t specific to Spock, so if you already know the theory,[1] feel free to skip ahead to the Spock implementation.
For more information, see Test Driven by Lasse Koskela (Manning, 2007).
Each time you want to create a new unit test, you have to decide on its scope. An automated test can focus on a single class, multiple classes, a single module, or even the whole system. The breadth of the tested area will affect several factors of your unit test, from the time it takes to complete (the more you’re trying to test, the bigger the unit test execution) to the readability and effort it takes to write it (a unit test that needs to set up several modules needs more preparation).
At one end of the spectrum, you have “pure” unit tests that focus on a single class. These are easy to write, run quickly, and depend only on the production code. At the opposite end are functional tests (also called acceptance tests) that examine the system as a whole, emulating user behavior and even interacting with the graphical user interface (GUI). A functional test sends a request to the system and expects a response without any other knowledge of the inner workings of the system.
In the middle of these extremes, tests can examine either a code module or a code service. These are the integration tests (because they examine how individually tested classes integrate into modules). Figure 7.1 shows the scope examined by these categories of tests.
Figure 7.1. Unit tests focus on a single class, integration tests cover multiple modules, and functional tests cover end-to-end testing from the web interface to the database.
The example in this figure is the e-shop application mentioned multiple times in the previous chapters. The image shows the following test types:
- Unit tests always examine a single class. All other classes are mocked/stubbed so that they don’t interfere with the result of the test. A unit test, for example, would verify that the basket class correctly calculates the weight of products it contains.
- Integration tests focus on multiple classes. Mocks/stubs are rarely used, as you’re interested in both the code and the way communication happens between modules. An integration test, for example, would verify the communication between the warehouse inventory (which is backed by a database) and the products contained in the basket. When the customer attempts to check out, the basket will show which products are in stock and which aren’t (after querying the inventory, which queries the database).
- Functional tests assume that the whole system is a black box. They test end-to-end interactions starting from the user interface (or the network API), and pass through the whole system. Functional testing usually requires a clone or duplicate of the real system. A functional test, for example, would be an automated test that opens a browser on its own, selects products by emulating the clicking of buttons on the web pages, checks out, enters a credit card, and expects to “see” onscreen a tracking number of the order shipped.
The distinction between these three categories isn’t always clear. After all, concepts such as module may mean different things to different people. Don’t get consumed by terminology.
7.1.1. Characteristics of the test categories
A well-tested application needs tests from all three categories. I sometimes imagine that a well-designed software product is like a well-designed car. If you’re a car manufacturer, you need to test the individual screws, bolts, and frames of a car (unit tests); test how these are assembled (integration tests); and in the end, perform tests on the final product by driving it in a controlled environment[2] (functional tests).
Or perform crash tests with dummies, which is much more fun.
It would be unrealistic to release a car without making sure that all screws are correctly assembled, and it would also be foolish to release a car without testing it as a whole on the road. I’m still puzzled when I see software organizations that either have only functional tests or only integration tests, and at the same time don’t understand why more bugs than expected are found in production.
The challenge of these three categories of tests is that they have different requirements and need different accommodations in the software lifecycle of a project. Table 7.1 briefly outlines the differences among them.
Table 7.1. Test categories in an enterprise application must be handled differently.
Table 7.1 is intended as a rough guide and is geared toward large enterprise projects.[3] Your project might be different, but the general principles still apply. You can write pure unit tests with Spock with no additional external library. But if you need to write a test that launches a web browser and starts pressing buttons in an automated manner, Spock isn’t enough on its own.
Think of a code base of 500K lines of code, a team of 20 people, a dedicated QA department, requirements that resemble a small book when printed—you get the picture.
7.1.2. The testing pyramid
So far, all tests you’ve seen in the previous chapters are mainly unit tests (pure tests). You might be wondering why I devoted three whole chapters (chapters 4, 5, and 6) for basic unit tests and left only a single chapter for both integration and functional tests.
The reason is that although all three categories of tests are essential, pure unit tests have a larger weight. This is best illustrated by a testing pyramid[4] that shows the percentage of tests from each category that compose your whole testing suite, as shown in figure 7.2.
See “Just Say No to More End-to-End Tests” by Mike Wacker on the Google Testing Blog for more details (http://googletesting.blogspot.co.uk/2015/04/just-say-no-to-more-end-to-end-tests.html).
Figure 7.2. Breakdown of total test count for each test category in a large enterprise application: the testing pyramid
Pure unit tests are your first line of defense. They’re the foundation that other tests build upon. It makes no sense to start creating complex integration tests if you’re not sure about the quality of the individual Java classes that compose them.
Only after you have enough unit tests can you start writing integration tests. Integration tests should be focused on testing things that pure unit tests can’t detect. Typical examples are transactions, security, and other cross-cutting concerns in your application. Integration tests are often used to ensure correct functionality between the prior code base and new modules added to a project.
Finally, when you’re happy with the number of integration tests, it’s time to create functional tests. These view the whole system as a black box and should be used as a way to catch serious runtime or graphical errors that slip through the rest of the tests.
If you look back at the pyramid, you can imagine that any other shape is an antipattern. A project that has no unit tests is clearly missing the foundations of the pyramid.[a] A project that has too many functional tests is also problematic (the pyramid will fall under its own weight).
For more information on the test pyramid, see http://martinfowler.com/bliki/TestPyramid.html.
7.1.3. Spock support for integration and functional testing
Unfortunately, unlike pure unit tests (which can be covered using just Spock), integration tests require a different infrastructure depending on the Java framework you use. Given the number of Java frameworks present, covering all possible cases would be difficult.
This section covers integration testing with Spock and Spring (https://spring.io/) and gives you pointers for Java EE[5] and Guice (https://github.com/google/guice). I chose Spring because of its popularity at the time of this writing. This section also shows how to test back-end applications that use REST services (powered by HTTP/JSON). I assume that your application has a web interface and explain that you can use Spock and Geb together to automate the web browser for effective functional tests. Finally, I complete the puzzle by covering Maven configuration and some advice on the build server setup.
Java EE can be tested with the help of Arquillian. See http://arquillian.org/.
If you’re writing an exotic Java application that doesn’t match this profile, I apologize in advance. You need to do some research on your own. Either a Spock extension already exists for what you need or you can use Spock’s compatibility with JUnit and attempt to use a tool from the JUnit world.
Covering integration testing for all kinds of Java applications in a single chapter would be impossible. I’d need a series of books for that. I know that everybody has a favorite testing tool or way to do integration testing, and I can’t cover them all. This chapter covers mainly Spring applications because they seem to be more popular and Spock has built-in support for them. But I’ll give you helpful pointers on what to do if your application isn’t based on Spring. The main theme of the chapter is that in Spock you can use your favorite Java testing libraries and learn new tricks with Groovy-based testing utilities. The testing tools I show here are my personal selection. I tried to find simple examples that anybody can understand. All the examples are contrived. If you want to learn more about integration testing for Java applications in general, you should consult books that focus on the specific technology of your application.
7.1.4. Source code organization of the examples
Unlike other chapters, the source code for this chapter is organized into three distinct projects, as shown in figure 7.3.
Figure 7.3. Source code contains three projects
Each project is a mini application on its own. Showing all source files in the book as code listings would be unmanageable. Instead, I focus on only important classes, so feel free to consult the GitHub code at https://github.com/kkapelon/java-testing-with-spock/tree/master/chapter7 while reading the book. The projects were created strictly for illustration purposes, so take notice of the Spock tests instead of the “production” code.