8.1. Using additional Spock features for enterprise tests
Chapter 4 covered all Spock blocks in detail as well as the general structure of a Spock test. Spock offers several complementary features in the form of annotations that further enhance the expressiveness of your tests.
The Spock tests demonstrated here are based on the e-shop example introduced in chapter 6. They revolve around placing products in an electronic basket and paying via credit card.
8.1.1. Testing the (non)existence of exceptions: thrown() and notThrown()
In all Spock tests that I’ve shown you so far, the expected result is either a set of assertions or the verification of object interactions. But in some cases, the “expected” result is throwing an exception. If you’re developing a library framework, for example, you have to decide what exceptions will be thrown to the calling code and verify this decision with a Spock test. The next listing demonstrates the capturing of an exception.
Listing 8.1. Expecting an exception in a Spock test
Here you design the Warehouse class so that it throws an exception when it can’t find the name of a product. When this test runs, you explicitly tell Spock that the when: block will throw an exception. The test will fail if an exception isn’t thrown, as shown in figure 8.1.
Figure 8.1. The test will fail if an exception isn’t thrown in the when: block.
In this case, you use an existing exception as offered by Java, but the same syntax works with any kind of exception. (You could create a custom exception class called ProductNotFoundException instead.)
It’s also possible to “capture” the exception thrown and perform further assertions in order to make the test stricter. The following listing provides an example of a message of an exception that’s checked.
Listing 8.2. Detailed examination of an expected exception
This listing further enhances the code of listing 8.1 by checking both the type of the exception and its message. Here you examine the built-in message property that’s present in all Java exceptions, but again, you could examine any property of a custom-made exception instead (the last statement in listing 8.2 is a standard Groovy assertion).
Finally, it’s possible to define in a Spock test that you don’t expect an exception for an operation in the when: block, as the following listing shows. I admit that the semantics of this syntax are subtle, but the capability is there if you need it.
Listing 8.3. Explicit declaration that an exception shouldn’t happen
I believe that the notThrown() syntax is intended as a hint to the human reader of the test and not so much to the test framework itself.
8.1.2. Mapping Spock tests to your issue-tracking system: @Issue
In chapter 4, you saw the @Subject, @Title, and @Narrative annotations that serve as metadata for the Spock test. These annotations are particularly useful to nontechnical readers of the tests (for example, business analysts) and will show their value when reporting tools use them for extra documentation.
Any nontrivial enterprise application has a product backlog or issue tracker that serves as a central database of current bugs and future features. Spock comes with an @Issue annotation that allows you to mark a test method that solves a specific issue with the code, as shown in the following listing.
Listing 8.4. Marking a test method with the issue it solves
Notice that the annotation has a strictly informational role. At least at the time of writing, no automatic connection to any external system exists (in this example, to JIRA, available at www.atlassian.com/software/jira). In fact, the value inside the annotation is regarded as free text by Spock. The next listing shows another example using a full URL of a Redmine tracker (www.redmine.org).
Listing 8.5. Using the URL of an issue solved by a Spock test
Finally, a common scenario is having multiple issue reports that stem from the same problem. Spock has you covered, and you can use multiple issues, as shown in the following listing.
Listing 8.6. Marking a Spock test with multiple issues
The @Issue annotation is also handy when you practice test-driven development, as you can use it to mark Spock tests for product features before writing the production code.
8.1.3. Failing tests that don’t finish on time: @Timeout
Chapter 7 covered integration and functional tests and how they differ from pure unit tests. A common characteristic of integration tests is their slow execution time because of real databases, web services, and external systems that are often used as part of the test.
Getting quick feedback from a failed unit test should be one of your primary goals when writing integration tests. The external systems used in integration tests can affect the execution time in a nondeterministic way, as their response time is affected by their current load or other environmental reasons.
Spock comes with an @Timeout annotation that unconditionally fails a test if its execution time passes the given threshold. The following listing shows an example.
Listing 8.7. Declaring a test time-out
The reasoning behind the @Timeout annotation is that it helps you quickly isolate environmental problems in your integration tests. If a service is down, there’s no point in waiting for the full time-out of your Java code (which could be 30 minutes, for example) before moving to the next unit test.
Using the @Timeout annotation, you can set your own bounds on the “expected” runtime of an integration test and have Spock automatically enforce it. The default unit is seconds, as shown in the previous listing, but you can override it with your own setting, as shown in the next listing.
Listing 8.8. Declaring a test time-out—custom unit
The importance of the @Timeout annotation is evident in the case of multiple long tests that take a long time to finish. I’ve seen build jobs that typically take minutes but because of a misconfiguration can take hours if time-outs aren’t used correctly.
8.1.4. Ignoring certain Spock tests
A large enterprise application can have thousands of unit tests. In an ideal world, all of them would be active at any given time. In real life, this is rarely the case.
Test environments that get migrated, features that wait to be implemented, and business requirements that aren’t yet frozen are common reasons that force some tests to be skipped. Fortunately, Spock offers several ways to skip one or more tests deliberately so your tests don’t fail while these restructurings and developments are taking place.
Ignoring a single test: @Ignore
Spock allows you to knowingly skip one or more tests and even provides you with the ability to give a reason for skipping that test (see the next listing).
Listing 8.9. Ignoring a single test
The primary purpose of skipping a test is so that the rest of your test suite is run successfully by your build server. An ignored test should always be a temporary situation because you’re vulnerable to code changes that would normally expose a bug verified by that test.
The human-readable description inside the @Ignore annotation should give a hint about why this test is ignored (the value is free text, as far as Spock is concerned). More often than not, the original developer who marks a test as ignored doesn’t always remove the @Ignore annotation, so it’s essential to document inside the source code the reason why the test was skipped in the first place.
You can place @Ignore on a single test method or on a whole class if you want all its test methods to be skipped.
Ignoring all but one test: @IgnoreRest
If you’re also lucky, and you want to ignore all but one test in a Spock specification, you can use the @IgnoreRest annotation. Assume that you have a set of integration tests that contact a credit card external service in a staging environment (it doesn’t actually charge cards). The service is down for maintenance. To keep your tests running, you could ignore tests selectively, as shown in the following listing.
Listing 8.10. Ignoring all tests except one
Running the Spock test shown in this listing produces the output in figure 8.2.
Figure 8.2. Only the test marked with @IgnoreRest runs.
Again, I admit that this Spock annotation is specialized, and you might never need to use it.
Ignoring Spock tests according to the runtime environment: @IgnoreIf part 1
The @Ignore annotations shown in the previous paragraph are completely static. A test is either skipped or not, and that decision is made during compile time.
Spock offers a set of smarter @Ignore annotations that allow you to skip tests dynamically (by examining the runtime environment). As a first step, Spock allows a test to query the following:
- The current environment variables
- The JVM system properties
- The operating system
Spock then decides whether the test will run, depending on that result. An example of skipping tests is shown in the next listing.
Listing 8.11. Skipping Spock tests according to the environment
Running this listing on my Windows system with JDK 7 and no extra JVM properties produces the output shown in figure 8.3.
Figure 8.3. A test is skipped because the current OS is Windows.
I won’t list all possible options supported by Spock. You can find the full details in its source code.[2] Ignoring tests depending on environment variables enables you to split your tests into separate categories/groups, which is a well-known technique. As an example, you could create “fast” and “slow” tests and set up your build server with two jobs for different feedback lifecycles.
Ignoring certain Spock tests with preconditions: @IgnoreIF part 2
To obtain the maximum possible flexibility from @IgnoreIf annotations, you need to define your own custom conditions. You can do this easily in Spock because the @IgnoreIf annotation accepts a full closure. The closure will be evaluated and the test will be skipped if the result is false.
The following listing shows a smarter Spock test that runs only if the CreditCardService is up and running.
Listing 8.12. Skipping a Spock test based on a dynamic precondition
This listing assumes that the Java class representing the external credit card system has a built-in method called online() that performs a “ping” on the remote host. Spock runs this method, and if it gets a negative result, it skips the test (there’s no point in running it if the service is down).
The contents of the closure passed as an argument in the @IgnoreIf annotation can be any custom code you write. If, for example, the built-in online() method wasn’t present, you could create your own Java (or Groovy) class that performs an HTTP request (or something appropriate) to the external system and have that inside the closure.
Reversing the Boolean condition of IgnoreIf: @Requires
If for some reason you find yourself always reverting the condition inside the @IgnoreIf annotation (as seen in listing 8.12, for example), you can instead use the @Requires annotation, as the following listing shows.
Listing 8.13. Requires is the opposite of IgnoreIf
The @Requires annotation has the same semantics as @IgnoreIf but with the reverse behavior. The test will be skipped by Spock if the code inside the closure does not evaluate to true. The option to use one or the other annotation comes as a personal preference.
8.1.5. Automatic cleaning of resources: @AutoCleanup
Chapter 4 showed you the cleanup: block as a way to release resources (for example, database connections) at the end of a Spock test regardless of its result. An alternative way to achieve the same thing is by using the @AutoCleanup annotation, as shown in the following listing.
Listing 8.14. Releasing resources with AutoCleanup
If you mark a resource with the @AutoCleanup annotation, Spock makes sure that the close() method will be called on that resource at the end of the test (even if the test fails). You can use the annotation on anything you consider a resource in your tests. Database connections, file handles, and external services are good candidates for the @AutoCleanup annotation.
You can override the method name that will be called by using it as an argument in the annotation, as done in listing 8.14. In that example, the shutdown() method will be called instead (Spock will call close() by default).
I prefer to use the cleanup: block and cleanup()/cleanupSpec() methods as explained in chapter 4 (especially when multiple resources must be released), but if you’re a big fan of annotations, feel free to use @AutoCleanup instead.[3] As you might guess, @AutoCleanup works both with instance fields and objects marked with the @Shared annotation shown in chapter 4.
You can also ignore exceptions during cleanup if you use the annotation like @AutoCleanup(quiet = true), but I don’t endorse this practice unless you know what you’re doing.
This concludes the additional Spock annotations,[4] and we can now move to refactoring of big Spock tests.
Yes, I know that expecting exceptions does not happen via annotations. Thanks for catching it!