Chapter 13. Testing

Testing is critical to the robustness and longevity of your application, but unless you make it part of your application development process, you’re bound to find it painful. Somewhat surprisingly, however, teams and individuals who do integrate testing as part of standard development practices find that they almost can’t live without it. Why? There are two reasons. First, testing itself demands testable code, which in turn implies better-structured code because of the need for cleanly interfacing units. Second, a well-built test suite is essentially machine-interpretable documentation that not only prevents regressions but also gives developers insight into the intent of the code, no matter its condition.

Merb, as a framework and application development platform, has always encouraged testing. However, the specifics of testing continue to be intensely debated. For this reason, Merb has attempted to remain agnostic toward the tools used in building test suites. Nonetheless, a decision had to be made, and RSpec was picked both internally and at a stack level as the testing framework of choice.

In this chapter we will cover the use of RSpec for testing models and requests, but following the precedent set by all the other chapters we’re more intent on revealing how testing has been integrated within the Merb framework than describing its precise use. That said, let’s start off with an exploration of tasks most commonly used with Merb when testing.

13.1 Rake tasks

All rake tasks related to testing within Merb are contained within the task name-space :spec. Here’s the beginning of spectasks.rb contained within the Merb core:

Observe that the default spec task is effectively created by passing in the string 'default' to Spec::Rake::SpecTask.new. The SpecTask class has been designed to simplify RSpec rake task creation. The default Merb spec, furthermore, sets two configuration variables through the block passed in to the initializer. These are the spec options and the spec files. Application developers should be aware that appending TASK='spec/some_spec.rb' to the rake spec command runs only one particular spec file. Otherwise, all files ending in _spec.rb and contained within any level subdirectory of spec/ are used.

The two non-default tasks, model and request, are meant to handle model and request specs. Their rake task construct is not very different from the default task, but developers should remember that MODEL and REQUEST are the environment variables for specifying the running of only particular spec files. Merb also creates a couple more tasks for view and controller testing, but most Merb developers frown upon the use of view and controller testing. The reasons are different in each case. View testing tends to be the testing of bluntly self-descriptive templates, whereas controller testing is a form of unit testing that, removed from the simulation of the request-response cycle, does not test context-genuine behavior. If you find yourself asking why the testing of context-genuine behavior is so important, you’ll definitely raise a substantially contentious question.

Leaving opinion aside, the findings of many have been that the development of a test suite that does test context-genuine behavior alone quickly becomes a crufty mess whenever application code refactoring is attempted. Said differently, the isolation of particular application code through tests via excessive stubbing and mocking tends to reveal tests that possess uncanny knowledge of implementation and not simply interface. Our advice is consequently that you stick to the bread and butter of model and request specs, and if you truly think you need a controller spec, then you should probably refactor some code out of your indubitably overburdened controller.

Two other spec tasks exists. These are html and coverage. The former simply outputs spec results in HTML, and the other uses rcov to determine the coverage of the test suite. Rake task in hand, let’s move on to producing some actual spec files within a Merb application.

13.2 Spec files

For the rest of this chapter we’re going to be working with a fake application called 0g. Honestly, though, the only critical bit of information about the application is that there is a Ship model and a Ships controller. To create the files for both of these classes, we used merb-gen resource ship. The only interesting bit is that it also gave us two additional files:

Before we jump in and manipulate these spec files, let’s first look at the code for a file that both require, spec/spec_helper.rb:

The most important method here is Merb.start_environment, which starts up Merb if it has not yet been started. Note, however, that it has been requested that Merb be started in testing mode. This prevents any Merb workers from being created, and though the application will be up and running, it won’t be accessible from any local port. Also note the inclusion of the test helper modules within the Spec::Runner.configure blocks; this makes them available for all the tests.

13.3 Model specs

The spec file for our ship model is located at spec/models/ship_spec.rb. Without any modification it appears as shown below.

Because the first spec description has egged us on, we should probably flesh out the file:

Through the specifications above we have set expectations about how a ship should behave. The methods before, it, and should are all out of the scope of this chapter, but if you need more familiarity with them please check the RSpec documentation. Let’s run our model specs and the result:

Obviously, the code has not written itself, and so we have ended up with a failure. Let’s modify our model as follows:

Now, running the model specs again, our specifications pass:

Realistically, this has been only a taste of what model testing is like, but given that domain-level testing is more dependent upon knowing the chosen testing framework than any extensions that may be built on top of it (Merb in fact comes with no explicit model test helpers), we’ll just leave it at that.

13.4 Request specs

Request specifications are the preferred way to test a Merb application’s full stack behavior. The concept, simple enough, is to fake a request in, and then set expectations on the response. Let’s open up the file controller spec file for ships. Since Ships was generated as a resource controller, a number of specifications establishing the basis of a RESTful interface are automatically provided:

If you’re new to specs with Merb, the given will throw you off. This method, an extension to RSpec, takes a block that can be run before spec groups. We’ll see this usage later on, but for now, just recognize that it’s a great way to keep our tests DRY. Also notice the use of the method request, which creates a request and sends it to the Merb test server. Its first parameter is a URL, here given using the resource URL helper. The second and optional parameter is a hash of request parameters. Above, the request method and request params have been specified for the creation of a ship.

Notice how the describes blocks are nested one inside the other. The first is for resource(:ships) and the second more specifically for how it behaves with GET applied.

There’s nothing magical about these names, and as we can see in the before(:each) block, a generic request to '/ships' is made. The return value of the request is saved as an instance variable so that we can set expectations on it within the individual it blocks. The first of these tests the most basic but essential expectation of the request: It should be successful; that is, it should come back to us with a successful status code without having raised any errors.

The next it block does something more specific: It tests that the response contains a list of ships. To do this, it makes use of the have_xpath helper, which tests for the presence of an element given an XPath. Now because the application may not respond with a list (maybe you need a table instead, or just a bunch of divs) and because a more thorough testing of the response is desirable, the pending method has been used to alert us to its incompleteness. The second describe block comes with a given that will run the previously described given block before each contained example. Otherwise the difference between this describe and the other are minuscule—a list element is also expected now that we know one ship is around. Having tested essentially the index action, let’s move on to more curious HTTP methods.

As we see, POST and DELETE are used to send out requests to different URLs. The first goes to '/ships' and the second presumably goes to '/ships/1'. Both examples, however, set the expectation that we are redirected to some other URL using the method redirect_to. The great thing about RSpec and Ruby testing frameworks in general is that they read so naturally that they just make sense.

The two example groups, for /ships/new and /ships/1/edit, are barely filled in. When you get down to specifying your own application, you may want to make sure particular form elements exist on each of these pages.

Closing up the resource-generated request spec file, we see the testing of /ships/1. The first describe block tests the success of the retrieval of the page, and the second makes sure that updating the ship object redirects us to its page. For the rest of this chapter we will describe in detail the Merb test helper methods, some of which were shown above.

13.5 Request helper

The request method which we saw before takes two parameters, the first a URL and the second an optional hash of all the environment variables related to the request. Let’s open up the source to see how it works:

Above we see the request method taking the uri given and translating it to the full and proper uri to test with. Note that we can pass in a symbol and it is automatically wrapped in the method url. We advise against this, however, for the sake of explicitness within specs. If the URL method is POST, then the method pushes all the params as a query string into env[:input]. We can do this ourselves, test-side, using the hash option :input, but there are few cases when it’s desirable to do so. Finally, if the request method is not POST, all params are inserted as query params.

The request method also makes sending and receiving cookies a possibility. To do so we can pass in a cookie jar with the option :jar. This jar must be constructed using the class Merb::Test::CookieJar. Because statefulness through cookies is so important on the web, if we do not pass in a cookie jar a default one is passed in for us.

And here’s the real magic. Using Rack::MockRequest, the request is sent off and handled by our application. Note how the rack response is restructured for our ease in testing. The methods status, helpers, body, url, and original_env all get us to the bits and pieces of the response we want to test. Finally, the dispatcher work queue is emptied, so we can have a clean room for testing our next request no matter what happened.

13.6 Request matchers

Among the spec matchers Merb provides, several simply test the application’s responsiveness to the request. We’ve already seen a few of them, so now let’s peek inside to see what they’re made of:

Created via the class method Spec::Matchers.create (a method Merb provides as an extension to RSpec), be_successful and respond_successfully are two matchers that test if the status code returned was somewhere in the range of 200 to 207. Note the use of the message to return failure messages.

The matchers be_missing and be_client_error are just the same, but for status codes 400 through 417.

The match have_body is expected to be used with both should and should_not. As a consequence, its failure messages are bifurcated into failure message and negative failure message. Additionally, note the second block parameter with matches. This is the parameter on the have_body matcher method. If they do not match (or, in the negative case, do match), then the test will fail.

The code for the rest of the matchers is very similar to what we’ve already seen, so we’ll leave you to venture into it if you so desire. The remaining request matchers are of use to application developers, however, so let’s describe them more briefly. The match have_content_type takes in a MIME symbol (for example, :html) that the request should match. Finally, the matchers redirect and redirect_to test whether a response has resulted in redirect and, in the case of redirect_to, if it was redirected to a specific location.

13.7 RSpec extensions

There are a few helper extensions Merb makes to RSpec directly. Let’s take a look at one we’ve already encountered. Remember the given method from the request specs? Here it is:

One should always be wary of Kernel methods, but for the sake of testing this method is added when our application runs in test mode. Interestingly enough, the given block must push itself into a describe block and then instance evaluate the code that should be run as early as possible before each example. Noting how the example group has been set to be shared, it’s hard to keep from showing you how they actually hook into later describe blocks. Let’s take a look at the relevant section of Merb’s extension of ExampleGroup:

There you have it: a module_eval (certainly a rarity) that uses the RSpec method it_should_behave_like. In other words, given is virtually an alias for it_should_behave_like that tightens up its usage.

It’s a shame that we have to end this section on a sour note, but Merb makes two further extension to RSpec:

Out of convenience you cannot make your examples fail outright by using the helpers fail and fails_with. Use the second if you need a message to tag along.

13.8 Miscellaneous extensions

A few other methods are added onto primitives so that your tests can read nicely. For strings the methods contain? and match? have been added. These are also added to Hpricot, so you can use them with either. Below is the code that adds these methods and their aliases to String.

Note that contain? really just delegates to include?, and match? to match. The first takes a string, the second a regular expression.

13.9 Conclusion

Merb’s use of RSpec both internally and within applications themselves exhibits the framework’s belief in specifying behavior during development. The availability of other frameworks for use during application development, however, reveals a willingness to suppress opinions for the sake of developer comfort. Nonetheless, as we saw, Merb has adopted the preference of request specification over that of unit testing controllers and views. Though this may seem like a small difference at first, it is truly a fundamental one that insists upon the specification of only genuine behavior.