Chapter 8. In-container testing

The secret of success is sincerity. Once you can fake that you’ve got it made.

Jean Giraudoux

This chapter covers

• The drawbacks of mock objects
• In-container testing
• Comparing stubs, mock objects, and in-container testing

This chapter examines one approach to unit testing components in an application container: in-container unit testing, or integration unit testing. We discuss in-container testing pros and cons and show what can be achieved using the mock objects approach introduced in chapter 7, where mock objects fall short, and how in-container testing enables you to write integration unit tests. Finally, we compare the stubs, mock objects, and in-container approaches we’ve already covered in this second part of this book.

8.1. Limitations of standard unit testing

Let’s start with the example servlet in listing 8.1, which implements the HttpServlet method isAuthenticated, the method we want to unit test.

Listing 8.1. Servlet implementing isAuthenticated

[...]
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpSession;

public class SampleServlet extends HttpServlet {
public boolean isAuthenticated(HttpServletRequest request) {
      HttpSession session = request.getSession(false);
if (session == null) {
return false;
      }
      String authenticationAttribute =
         (String) session.getAttribute("authenticated");
return Boolean.valueOf(authenticationAttribute).booleanValue();
   }
}

This servlet, although simple enough, allows us to show the limitation of standard unit testing. In order to test the method isAuthenticated, we need a valid HttpServletRequest. Because HttpServletRequest is an interface, we can’t just call a new HttpServletRequest. The HttpServletRequest lifecycle and implementation are provided by the container (in this case, a servlet container.) The same is true for other server-side objects like HttpSession. JUnit alone isn’t enough to write a test for the isAuthenticated method and for servlets in general.

 

Definition

Component and container—In this chapter, a component executes in a container. A container offers services for the components it’s hosting, such as lifecycle, security, transaction, distribution, and so forth.

 

In the case of servlets and JSPs, the container is a servlet container like Jetty or Tomcat. There are other types of containers, for example, EJB, database, and OSGi containers.

As long as a container creates and manages objects at runtime, we can’t use standard JUnit techniques to test those objects.

8.2. The mock objects solution

We look at several solutions for in-container testing. The first solution for unit testing the isAuthenticated method (listing 8.1) is to mock the HttpServletRequest class using the approach described in chapter 7. Although mocking works, we need to write a lot of code to create a test. We can achieve the same result more easily using the open source EasyMock^[1] framework (see chapter 7), as listing 8.2 demonstrates.

¹http://easymock.org/

Listing 8.2. Testing a servlet with EasyMock

We start by importing the necessary classes and methods using Java 5 static imports . We use the EasyMock class extensively, which has similar syntax to the JUnit Hamcrest matchers. Next, we declare instance variables for the objects we want to mock, HttpServletRequest and HttpSession. The setUp method annotated with @Before runs before each call to @Test methods; this is where we instantiate all of our mock objects. Next, we implement our tests following this pattern:

1. Set our expectations using the EasyMock API .
2. Invoke the replay method to finish declaring our expectations .
3. Assert test conditions on the servlet .

Finally, at the @After method (executed after each @Test method) calls the EasyMock verify API to check whether the mocked objects met all of our programmed expectations.

Mocking a minimal portion of a container is a valid approach to testing components. But mocking can be complicated and require a lot of code. As with other kinds of tests, when the servlet changes, the test expectations must change to match. Next, we look at easing this task.

8.3. In-container testing

Another approach to testing the SampleServlet is to run the test cases where the HttpsServletRequest and HttpSession objects live, in the container itself. This avoids the need to mock any objects; we access the objects and methods we need in the real container.

For our example, we need HttpServletRequest and HttpSession to be real objects managed by the container. Using a mechanism to deploy and execute our tests in a container, we have in-container testing. Next, we see what options are available to implement in-container tests.

8.3.1. Implementation strategies

We have two architectural choices to drive in-container tests: server-side and client-side. As we stated previously, we can drive the tests directly by controlling the server-side container and the unit tests. Alternatively, we can drive the tests from the client side, as shown in figure 8.1.

Figure 8.1. Lifecycle of a typical in-container test

Once the tests are packaged and deployed in the container and to the client, the JUnit test runner executes the test classes on the client (1). A test class opens a connection via a protocol like HTTP(S) and calls the same test case on the server side (2). The server-side test case operates on server-side objects, which are normally available (such as HttpServletRequest, HttpServletResponse, HttpSession, BundleContext, and so on), and tests our domain objects (3). The server returns the result from the tests back to the client (4), which an IDE or Ant or Maven can gather.

8.3.2. In-container testing frameworks

As you just saw, in-container testing is applicable when code interacts with a container and tests can’t create valid container objects (HttpServletRequest in the previous example).

Our example uses a servlet container, but there are many different types of containers: servlet, database, OSGi, SIP, and the like. In all of these cases, we can apply the in-container testing strategy. In the third part of the book, we present different open source projects that use this strategy. Table 8.1 lists the types of containers and the testing frameworks we use in later chapters.

Table 8.1. Containers and testing frameworks

Container type	In-container testing framework to use	Chapter with detailed description
HTTP container	Cactus (for testing servlets, JSPs, tag libraries, filters, and EJBs)	Chapter 14
HTTP container	JSFUnit (for testing JSF components)	Chapter 15
OSGi container	JUnit4OSGi (for testing OSGi modules)	Chapter 16
Database container	DBUnit	Chapter 17

Although the open source world offers other projects, we cover the more mature projects listed in the table. Next, we compare stubs, mock objects, and in-container testing.

8.4. Comparing stubs, mock objects, and in-container testing

In this section we compare^[2] the different approaches we presented to test components. This section draws from the many questions in forums and mailing lists asking about the pros and cons of stubs, mock objects, and in-container testing.

² For an in-depth comparison of the stubs and mocks technology, see “Mocks Aren’t Stubs” by Martin Fowler: http://martinfowler.com/articles/mocksArentStubs.html.

8.4.1. Stubs pros and cons

We introduced stubs as our first out-of-container testing technique in chapter 6. Stubs work well to isolate a given class for testing and asserting the state of its instances. For example, stubbing a servlet container allows us to track how many requests were made, what the state of the server is, or what URLs where requested.

Using mocks, however, we can test the state of the server and its behavior. When using mock objects, we code and verify expectations; we check at every step to see whether tests execute domain methods and how many times tests call these methods.

One of the biggest advantages of stubs over mocks is that stubs are easier to understand. Stubs isolate a class with little extra code compared to mock objects, which require an entire framework to function. The drawback of stubs is that they rely on external tools and hacks, and they don’t track the state objects they fake.

Going back to chapter 6, we easily faked a servlet container with stubs; doing so with mock objects would be much harder because we’d need to fake container objects with state and behavior. Here’s a summary of stubs pros and cons.

Pros:

• They’re fast and lightweight.
• They’re easy to write and understand.
• They’re powerful.
• Tests are coarser grained.

Cons:

• Specialized methods are required to verify state.
• They don’t test the behavior of faked objects.
• They’re time consuming for complicated interactions.
• They require more maintenance when the code changes.

8.4.2. Mock objects pros and cons

The biggest advantage of mock objects^[3] over in-container testing is that mocks don’t require a running container in order to execute tests. Tests can be set up quickly and run quickly. The main drawback is that the tested components don’t run in the container in which you’ll deploy them. The tests can’t verify the interaction between components and container. The tests also don’t test the interaction between the components themselves as they run in the container.

³ For an in-depth comparison of the mocks and in-container testing, see “Mock Objects vs. In-Container Testing”: http://jakarta.apache.org/cactus/mock_vs_cactus.html.

You still need a way to perform integration tests. Writing and running functional tests could achieve this. The problem with functional tests is that they’re coarse grained and test only a full use case—you lose the benefits of fine-grained unit testing. You won’t be able to test as many different cases with functional tests as you will with unit tests.

There are other disadvantages to mock objects. For example, you may have many mock objects to set up, which may prove to be considerable overhead. Obviously, the cleaner the code (small and focused methods), the easier tests are to set up.

Another important drawback to mock objects is that in order to set up a test, you usually must know exactly how the mocked API behaves. It’s easy to know the behavior of your own API, but it may not be so easy for another API, such as the Servlet API.

Even though containers of a given type all implement the same API, not all containers behave the same way. For example, consider the following servlet method:

public void doGet(HttpServletRequest request,
                     HttpServletResponse response) {
   response.setContentType("text/xml");
}

This example seems simple enough, but if you run it in Tomcat (http://tomcat.apache.org/) and in Orion prior to version 1.6.0 (http://www.orionserver.com/), you’ll notice different behaviors. In Tomcat, the returned content type is text/xml, but in Orion, it’s text/html.

This is an extreme example; all servlet containers implement the Servlet API in pretty much the same way. But the situation is far worse for the various Java EE APIs—especially the EJB API. Implementers can interpret an API specification differently, and a specification can be inconsistent, making it difficult to implement. In addition, containers have bugs; the previous example may have been a bug in Orion 1.6.0. Although it’s unfortunate, you’ll have to deal with bugs, tricks, and hacks for various third-party libraries in any project.

To wrap up this section, we summarize the drawbacks of unit testing with mock objects:

• This approach doesn’t test interactions with the container or between the components. It doesn’t test the deployment of components.
• It requires excellent knowledge of the API to mock, which can be difficult (especially for external libraries).
• It doesn’t provide confidence that the code will run in the target container.
• It’s more fine grained, which may lead you to being swamped with interfaces.
• Like stubs, it requires maintenance when code changes.

8.4.3. In-container testing pros and cons

So far, we’ve described the advantages of in-container unit testing. But there are also a few disadvantages, which we now discuss.

Specific tools required

A major drawback is that although the concept is generic, the tools that implement in-container unit testing are specific to the tested API, such as Apache Jakarta Cactus for Java EE testing. If you wish to write integration unit tests for another component model, chances are such a framework exists. With mock objects, because the concept is generic, you can test almost any API.

No good IDE support

A significant drawback of most of the in-container testing frameworks is the lack of good IDE integration. In most cases, you can use Ant or Maven to execute tests, which also provides the ability to run a build in a continuous integration server (CIS, see chapter 11). Alternatively, IDEs can execute tests that use mock objects as normal JUnit tests.

We strongly believe that in-container testing falls in the category of integration testing. This means that you don’t need to execute your in-container tests as often as normal unit tests and will most likely run them in a CIS, alleviating the need for IDE integration.

Longer execution time

Another issue is performance. For a test to run in a container, you need to start and manage the container, which can be time consuming. The overhead in time (and memory) depends on the container: Jetty can start in less than 1 second, Tomcat in 5 seconds, and WebLogic in 30 seconds. The startup overhead isn’t limited to the container. For example, if a unit test hits a database, the database must be in an expected state before the test starts (see database application testing in chapter 17). In terms of execution time, integration unit tests cost more than mock objects. Consequently, you may not run them as often as logic unit tests.

Complex configuration

The biggest drawback to in-container testing is that tests are complex to configure. Because the application and its tests run in a container, your application must be packaged (usually as a WAR or EAR file) and deployed to the container. You must then start the container and run the tests.

On the other hand, because you must perform these exact same tasks for production, it’s a best practice to automate this process as part of the build and reuse it for testing purposes. As one of the most complex tasks of a Java EE project, providing automation for packaging and deployment becomes a win-win situation. The need to provide in-container testing will drive the creation of this automated process at the beginning of the project, which will also facilitate continuous integration.

To further this goal, most in-container testing frameworks include support for build tools like Ant and Maven. This will help hide the complexity involved in building various runtime artifacts as well as with running tests and gathering reports.

8.4.4. In-container versus out-of-container testing

A standard design goal is to separate presentation from business layers. For example, you should implement the code for a tag that retrieves a list of customers from a database with two classes. One class implements the tag and depends on the Taglib API, whereas the other implements the database access and depends not on the Taglib API but on database classes like JDBC.

The separation-of-concerns strategy not only permits the reuse of classes in more than one context, but it also simplifies testing. You can test the business logic class with JUnit and mock objects and use in-container testing to validate the tag class.

In-container testing requires more setup than mock objects but is well worth the effort. You may not run the in-container tests as often, but they can confirm that your tags will work in the target environment. Although you may be tempted to skip testing a component, such as a taglib, reasoning that functional tests will eventually test the tag as a side effect, we recommend that you fight this temptation. All components benefit from unit testing. We outline these benefits here:

• Fine-grained tests can be run repeatedly and tell you when, where, and why your code breaks.
• You have the ability to test completely your components, not only for normal behavior but also for error conditions. For example, when testing a tag accessing a database, you should confirm that the tag behaves as expected when the connection with the database is broken. This would be hard to test in automated functional tests, but it’s easy to test when you combine in-container testing and mock objects.

8.5. Summary

When it comes to unit testing an application in a container, we’ve shown that standard JUnit tests come up short. Although testing with mock objects works, it misses some scenarios like integration tests to verify component interactions in and with a container. In order to verify behavior in a container, we need a technique that addresses testing from an architectural point of view: in-container testing.

Although complex, in-container testing addresses these issues and provides developers with the confidence necessary to change and evolve their applications. This chapter provided the foundation for the last part of this book, where we continue to explore such testing frameworks.

Next, we start the third part of this book by integrating JUnit into the build process, a tenet of test-driven development.