Chapter 5. Isolation (mock object) frameworks
- Understanding isolation frameworks
- Defining fake objects
- Using Rhino Mocks to create stubs and mocks
- Surveying advanced use cases for mocks and stubs
- Exploring arrange-act-assert and record-and-replay syntax
- Avoiding common misuses of isolation frameworks
In the previous chapter, we looked at writing mocks and stubs manually and saw the challenges involved. In this chapter, we’ll look at some elegant solutions for these problems in the form of a mock object framework—a reusable library that can create and configure stub and mock objects at runtime. These objects are usually referred to as dynamic stubs and dynamic mocks.
We’ll start this chapter off with an overview of mock object frameworks (or isolation frameworks—the word mock is too overloaded already) and what they can do for us. We’ll then take a closer look at one specific framework: Rhino Mocks. We’ll see how we can use it to test various things and to create stubs, mocks, and other interesting things.
Later in this chapter, we’ll contrast Rhino Mocks with other frameworks currently available to .NET developers, and we’ll finish with a list of things to watch out for when using such frameworks in your tests.
Let’s start at the beginning: What are isolation frameworks?
5.1. Why use isolation frameworks?
We’ll start with a basic definition.
Definition
An isolation framework is a set of programmable APIs that make creating mock and stub objects much easier. Isolation frameworks save the developer from the need to write repetitive code to test or simulate object interactions.
This definition may sound a bit bland, but it needs to be generic in order to include the various isolation frameworks out there.
Isolation frameworks exist for most languages that have a unit-testing framework associated with them. For example, C++ has mockpp and other frameworks, and Java has jMock and EasyMock, among others. .NET has NMock, Moq, Typemock Isolator, and Rhino Mocks.
Using isolation frameworks instead of writing mocks and stubs manually, as in previous chapters, has several advantages that make developing more elaborate and complex tests easier, faster, and less error-prone.
The best way to understand the value of an isolation framework is to see a problem and solution. One problem that might occur when using handwritten mocks and stubs is repetitive code.
Assume you have an interface a little more complicated than the ones shown so far:
|
public interface IComplicatedInterface |
Creating a handwritten stub or mock for this interface may be time-consuming, because we’d need to remember the parameters on a per-method basis, as listing 5.1 shows.
Listing 5.1. Implementing complicated interfaces with handwritten stubs
class MytestableComplicatedInterface:IComplicatedInterface
{
public string meth1_a;
public string meth1_b,meth2_b;
public bool meth1_c,meth2_c,meth3_c;
public int meth1_x,meth2_x,meth3_x;
public int meth1_0,meth2_0,meth3_0;
public void Method1(string a,
string b, bool c,
int x, object o)
{
meth1_a = a;
meth1_b = b;
meth1_c = c;
meth1_x = x;
meth1_0 = 0;
}
public void Method2(string b, bool c, int x, object o)
{
meth2_b = b;
meth2_c = c;
meth2_x = x;
meth2_0 = 0;
}
public void Method3(bool c, int x, object o)
{
meth3_c = c;
meth3_x = x;
meth3_0 = 0;
}
}
Not only is this test time-consuming and cumbersome to write, what happens if we want to test that a method is called many times? Or if we want it to return a specific value based on the parameters it receives, or to remember all the values for all the method calls on the same method (the parameter history)? The code gets ugly fast.
Using an isolation framework, the code for doing this becomes trivial, readable, and much shorter, as you’ll see when you create your first dynamic mock object.
Note
The word fake was introduced in chapter 4 as a generic term for either a stub or a mock.
5.2. Dynamically creating a fake object
Let’s define dynamic fake objects, and how they’re different from regular mocks.
Definition
A dynamic fake object is any stub or mock that’s created at runtime without needing to use a handwritten implementation of an interface or subclass.
Using dynamic fakes removes the need to hand-code classes that implement interfaces or that derive from other classes, because this can be generated for the developer at runtime by a simple line of code.
5.2.1. Introducing Rhino Mocks into your tests
In this chapter, we’ll use Rhino Mocks, an isolation framework that’s open source and freely downloadable from http://ayende.com. It’s simple and quick to use, with little overhead in learning how to use the API. I’ll walk you through a few examples, and then we’ll explore other frameworks and discuss the differences among them.
The only thing you’ll need to do to use Rhino Mocks, after downloading and unzipping it, (assuming you have NUnit installed on your machine) is to add a reference to the Rhino.Mocks.Dll. In the Add Reference dialog of the test project, click Browse, and locate the downloaded DLL file (which you can get from http://www.ayende.com/projects/rhino-mocks/downloads.aspx).
Rhino Mocks allows us to create and use fakes in two different ways. The first one is the record-and-replay model, and is the one you’ll be seeing most in this chapter. The other is the arrange-act-assert model, which I’ll be discussing near the end of this chapter.
Rhino Mocks contains in its API a class called MockRepository that has special methods for creating mocks and stubs. The first one we’ll look at is StrictMock(). The method is called with a generic parameter matching the type of an interface or class that you’d like to fake, and it dynamically creates a fake object that adheres to that interface at runtime. You don’t need to implement that new object in real code.
5.2.2. Replacing a handwritten mock object with a dynamic one
As an example, let’s look at a handwritten mock object used to check whether a call to the log was performed correctly. Listing 5.2 shows the test class and the handwritten mock.
Listing 5.2. Asserting against a handwritten mock object
[TestFixture]
public class LogAnalyzerTests
{
[Test]
public void Analyze_TooShortFileName_CallsWebService()
{
ManualMockService mockService = new ManualMockService ();
LogAnalyzer log = new LogAnalyzer(mockService);
string tooShortFileName="abc.ext";
log.Analyze(tooShortFileName);
Assert.AreEqual("Filename too short:abc.ext",
mockService.LastError);
}
}
public class ManualMockService:IWebService
{
public string LastError;
public void LogError(string message)
{
LastError = message;
}
}
The parts of the code in bold are the parts that will change when we start using dynamic mock objects.
We’ll now create a dynamic mock object, and eventually replace the earlier test. Listing 5.3 shows a simple piece of code that creates a simulated object based on an interface using the record-and-replay syntax in Rhino Mocks.
Listing 5.3. Creating a dynamic mock object using Rhino Mocks
A couple of lines rid us of the need to use a handwritten stub or mock, because they generate one dynamically
. The simulatedService object instance is a dynamically generated object that implements the IWebService interface, but there’s no implementation inside any of the IWebService methods.
Next, we set the simulated service object into a “record” state. This is a special state where each method call on the simulated
object is recorded as an expectation that something should happen to our simulated object
. These expectations will later be verified by calling mockEngine.VerifyAll() or preferably by calling mockEngine.Verify(mockObject). In this case, we only have one expectation—that LogError() will be called with the exact input of "file name was too short".
Expectations on mocks and stubs
An expectation on a fake object is an ad hoc rule that’s set on the object.
- Expectations on mocks— The rule will usually tell the object that a specific method call on that object is expected to happen later. It may also define how many times it should be called, whether or not the object should throw an exception when that call arrives, or perhaps that the call to this method should never be expected. Expectations are usually set on mock objects during the recording phase of the object, and they’re verified at the end of the test where the mock object lives. You set expectations on mocks using the static method of MockRepository.StrictMock<T> or MockRespository.DynamicMock<T>.
- Expectations on stubs— This wording may feel unintuitive at first, but the rule can tell the stub what value to return based on expected method calls, whether to throw exceptions, and so on. These expectations aren’t to be verified at the end of the test. They’re there so that you can run the test and simulate some alternative reality for your code under test. You create stubs and set expectations on them using MockRepository.GenerateStub<T> or the instance method of MockRepository.Stub<T>.
Then we invoke the object under test—our LogAnalyzer—by injecting it with our mock object and sending in a short filename that should make it invoke the logger internally
.
The last step in this test is to do some sort of assert. In this case, we’ll need to find a way to assert that all the expectations
have been met by our mock object (that LogError was indeed called with the correct string). We can do this by using mocks.Verify(simulatedService)
. The Verify(mock) method will go through each of the expectations that we’ve set on our mock object and make sure they’re true (called correctly).
What happens if the implementation of log.Analyze() contains an unexpected call to the service, either through an unexpected parameter value or an unexpected method call that we never recorded?
|
public void Analyze(string fileName) |
The test will fail, but it won’t fail when calling the Verify() method. It will fail before that, during the test run when the call to LogError() is executed. The test will never get to the Verify() line because an exception will be thrown before that. To understand why, we’ll review the idea of strict and nonstrict mocks.
5.3. Strict versus nonstrict mock objects
Let’s discuss what strict and nonstrict mocks mean, and why I consider nonstrict mocks better for most tests.
5.3.1. Strict mocks
A strict mock object can only be called by methods that were explicitly set via expectations. Any call that differs either by the parameter values defined or by the method name will usually be handled by throwing an exception. The test will fail on the first unexpected method call to a strict mock object. I say “usually” because whether or not the mock throws an exception depends on the implementation of the isolation framework. Some frameworks allow you to define whether to delay all exceptions until calling verify() at the end of the test.
This means that a strict mock can fail in two ways: when an unexpected method is called on it, or when expected methods aren’t called on it (which is determined by calling Verify()).
In Rhino Mocks, strict mocks are created by calling the StrictMock<T> method. Unexpected method call exceptions will always be thrown, even if your test contains a global try-catch clause, which you’d think would catch such an exception thrown from the isolation framework.
5.3.2. Nonstrict mocks
Most of the time, nonstrict mocks make for less brittle tests. A nonstrict mock object will allow any call to be made to it, even if it was not expected. As long as the call doesn’t require a return value, it will do what’s necessary for everything in the test to work out.
If a method that needs to return a value is called, and you did not set up a return value when you set up that mock object, a Rhino Mocks nonstrict mock or stub object can return the default value for that method’s return type (0 or null usually). Other frameworks may take different approaches and may throw an exception if the method isn’t configured to return anything.
A nonstrict mock can only fail a test if an expected method was not called. You have to call the Verify(mock) method to find out if such a call is missing from the interaction, or the test will pass.
The example in listing 5.3 uses a strict mock approach, which is why running the test fails mid-test instead of when calling Verify(). By calling MockRepository.DynamicMock<type>() instead of MockRepository.StrictMock<Type>(), you’ll get a test that only fails on the last line.
Listing 5.4 shows how the test from listing 5.3 would look if we used a nonstrict mock object with Rhino Mocks.
Listing 5.4. Creating a nonstrict mock
[Test]
public void Analyze_TooShortFileName_ErrorLoggedToService()
{
MockRepository mocks = new MockRepository();
IWebService simulatedService =
MockRespository.DynamicMock<IWebService>();
using(mocks.Record())
{
//we expected "Filename too short:abc.ext"
simulatedService.LogError("bad string");
}
LogAnalyzer log = new LogAnalyzer(simulatedService);
string tooShortFileName="abc.ext";
log.Analyze(tooShortFileName);
mocks.VerifyAll();
}
Mocks created with an isolation framework can also be used as stubs. We can tell them to return simulated values and create other interesting effects. The next section shows how to do that.
5.4. Returning values from fake objects
To return values from fake objects, you’ll almost always want to create a stub, and not a mock object. That means you can use either the static MockRepository.GenerateStub<T> method or the Stub<T> instance method. You’ll see this in the next section.
You won’t often need to, but you can also return values using mock objects, so let’s look at how to do that. You can instruct the mock object to return a value based on a method call by using a special class called LastCall.
Listing 5.5 shows how we can return a value from a mock object when the interface method has a nonvoid return value. For this example, we’ll add an IGetResults interface into the system. During the recording stage, we use the LastCall class to set the return value for that method call when that specific input (a, in this case) is sent to the method.
Listing 5.5. Returning a value from a mock object using the LastCall class
As you can see in listing 5.5, there are three expectations set on the mock object, and after each one we set the result to be returned from these method calls. Our mock object will be smart enough to return the correct value based on the input that was set in the expectation. If the same input is set with different return values, they will be returned in the order the code has added them.
You’ll notice that, after the recording stage
, we call GetSomeNumber with the b input, but the test will still pass, which means the order of the calls doesn’t matter.
Note
If we wanted the order to matter, we could use a concept called ordered mocks, which are used to define the correct order in which calls and return values should be executed. You can find out more about ordered mocks on the Rhino Mocks website.
If we change the order of the last two asserts in the test (which both input a), the test will fail because the recording order matters when the input is the same for the expectations.
You can also use LastCall to set the method call to throw a specific exception:
LastCall.Throw(Exception)
Or you can even execute your own delegate:
LastCall.Do(yourdelegatehere)
Again, it’s usually a bad idea to tell a mock to return a value and also to verify mock expectations. In this case, your test may be overspecified; it checks too many things and can break easily. A fake object should either be used as a mock or as a stub, not both.
Stubs are usually more appropriate for returning fake values than mock objects, and isolation frameworks can also create them.
5.5. Creating smart stubs with an isolation framework
A stub returns the appropriate responses when called and is never used to see if a test passes or not. Calling VerifyAll() or Verify(stub) won’t verify anything against stub objects—only against mocks. Most isolation frameworks contain the semantic notion of a stub, and Rhino Mocks is no exception.
5.5.1. Creating a stub in Rhino Mocks
Listing 5.6 shows how you create a stub object in Rhino Mocks.
Listing 5.6. Creating a stub is remarkably similar to creating a mock object
[Test]
public void ReturnResultsFromStub()
{
MockRepository mocks = new MockRepository();
IGetResults resultGetter = mocks.Stub<IGetResults>();
using(mocks.Record())
{
resultGetter.GetSomeNumber("a");
LastCall.Return(1);
}
int result = resultGetter.GetSomeNumber("a");
Assert.AreEqual(1, result);
}
The syntax for creating a stub is almost the same as for mock objects. But consider what happens if you run a test that doesn’t call an expected method on the stub, but still verifies expectations. This is shown in listing 5.7.
Listing 5.7. Verifying expectations on a stub object can’t fail a test
The test in listing 5.7 will still pass because the stub, by definition, is incapable of breaking a test. Any expectations set on it are purely to determine the return value or the exceptions they should throw.
Rhino Mocks contains a handy feature that isn’t supported by most frameworks (except Typemock Isolator). For simple properties on stub objects, get and set properties are automatically implemented and can be used as if the stub were a real object. You can still set up a return value for a property, but you don’t need to do it for each and every property on a stub object. Here’s an example:
|
ISomeInterfaceWithProperties stub = |
We can also simulate an exception using expectations on a stub. Listing 5.8 shows how you would simulate an OutOfMemoryException.
Listing 5.8. Faking an exception using the LastCall class
[Test]
public void StubSimulatingException()
{
MockRepository mocks = new MockRepository();
IGetResults resultGetter = mocks.Stub<IGetResults>();
using(mocks.Record())
{
resultGetter.GetSomeNumber("A");
LastCall.Throw(
new OutOfMemoryException("The system is out of memory!")
);
}
resultGetter.GetSomeNumber("A");
}
This test will fail due to a nasty out-of-memory exception. That’s how easy it is.
In the next section, we’ll use this ability to simulate errors when testing a more complex scenario.
5.5.2. Combining dynamic stubs and mocks
We’ll use the example from listing 4.2 in chapter 4, where we talked about LogAnalyzer using a MailSender class and a WebService class. This is shown in figure 5.1.
Figure 5.1. The web service will be stubbed out to simulate an exception, and the email sender will be mocked to see if it was called correctly. The whole test will be about how LogAnalyzer interacts with other objects.
We want to make sure that, if the service throws an exception, LogAnalyzer will use MailSender to send an email to an administrator.
Listing 5.9 shows what the logic looks like with all the tests passing.
Listing 5.9. The method under test and a test that uses handwritten mocks and stubs
public void Analyze(string fileName)
{
if(fileName.Length<8)
{
try
{
service.LogError("Filename too short:" + fileName);
}
catch (Exception e)
{
email.SendEmail("a","subject",e.Message);
}
}
//..other logic
}
[Test]
public void Analyze_WebServiceThrows_SendsEmail()
{
StubService stubService = new StubService();
stubService.ToThrow= new Exception("fake exception");
MockEmailService mockEmail = new MockEmailService();
LogAnalyzer2 log = new LogAnalyzer2();
//we use setters instead of
//constructor parameters for easier coding
log.Service = stubService;
log.Email=mockEmail;
string tooShortFileName="abc.ext";
log.Analyze(tooShortFileName);
Assert.AreEqual("a",mockEmail.To);
Assert.AreEqual("fake exception",mockEmail.Body);
Assert.AreEqual("subject",mockEmail.Subject);
}
}
public class StubService:IWebService
{
...
}
public class MockEmailService:IEmailService
{
...
}
Listing 5.10 shows what the test could look like if we used Rhino Mocks.
Listing 5.10. Converting the previous test into one that uses dynamic mocks and stubs
The nice thing about this test is that it requires no handwritten mocks, and it’s still readable for the developer.
You might notice a line in listing 5.10 that you haven’t come across yet
. That’s a parameter constraint that makes sure the exception is thrown no matter what we send the stub as a parameter. Parameter
constraints are the topic of the next section.
5.6. Parameter constraints for mocks and stubs
Isolation frameworks enable us to easily test the value of parameters being passed into our mock objects. In the coming sections, we’ll look at the various ways you can check parameters, such as strings, properties, and full object models, using very little code.
5.6.1. Checking parameters with string constraints
Consider the following scenario. We’d like to test that our LogAnalyzer sends a message of this nature to the error log:
"[Some GUID] Error message"
"33DFCC9D-D6C5-45ea-A520-A6018C88E490 Out of memory"
In our test, we don’t care about the GUID (globally unique identifier) at the beginning, but we care what the error message is. In fact, we don’t really have control over the GUID. (We could gain control by creating some sort of IGuidCreator interface and stubbing it out in the test, but that might prove a little too much work for what we need.)
Parameter constraints allow us to specify demands or rules to our mocks and stubs for each specific parameter to a method call. Isolation frameworks allow you to create these parameter constraints, and each framework has its own syntax for doing so.
The simplest way to use constraints, as shown in listing 5.11, is by using the LastCall class in conjunction with one of the constraints classes. In our case, it would be the Contains class, which takes as a constructor the inner string to search for.
Listing 5.11. Using a string constraint in a test
[Test]
public void SimpleStringConstraints()
{
MockRepository mocks = new MockRepository();
IWebService mockService = mocks.CreateMock<IWebService>();
using (mocks.Record())
{
mockService.LogError("ignored string");
LastCall.Constraints(new Contains("abc"));
}
mockService.LogError(Guid.NewGuid() + " abc");
mocks.VerifyAll();
}
Using the LastCall.Constraints() method, we can send in a constraint object (which has to inherit from AbstractConstraint, defined as part of Rhino Mocks) for each parameter the method expects. There are four major “helper” classes for constraints in Rhino Mocks, listed in table 5.1. For string-related constraints, we have Contains, EndsWith, Like, and StartsWith. All of these classes take a string at the constructor. To help you use these constraints, Rhino Mocks includes a helper class called Text that has static methods to return these constraint objects.
Table 5.1. The four types of constraints in Rhino Mocks
|
Helper class |
Description |
Methods |
|---|---|---|
|
Text |
Checks string-related constraints |
Contains(string) |
|
List |
Checks collection-related constraints |
Count(constraint) |
|
Is |
Checks the direct value of parameters passed in |
Anything() |
|
Property |
Checks the value of a specific property on an object that’s passed in as a parameter |
IsNull() |
|
And, Or |
Allows combining multiple constraints into a single logical constraint |
|
|
Callback |
Allows triggering a custom delegate whenever the method is called |
Here’s the same test using the Text class:
LastCall.Constraints(Text.Contains("abc"));
Some of the more interesting constraints are Property constraints, And and Or constraints, and Callback constraints. Let’s review them one by one.
5.6.2. Checking parameter object properties with constraints
Assume that the IWebService interface has a method that expects to take in a TraceMessage object, which contains specific rules about its properties. We could easily check the values of the passed-in TraceMessage object properties by using the Property-related constraints. This is shown in listing 5.12.
Listing 5.12. Using the Property constraints by using the Property static class
Notice the use of the && operators here.
Combining constraints with AND and OR
The && operators in listing 5.12 are overloaded to use a special And constraint, which requires all of these constraints to be true for the test to pass. You could also use the || overload to set a special Or constraint, which only requires one of the constraints to be true.
The And and Or constraints both take two AbstractConstraint objects in their constructor. The previous example in listing 5.12 combines two And constraints and could have been written as in listing 5.13.
Listing 5.13. Combining constraints with And and Or
And combined1 =
new And(
Property.Value("Message", "expected msg" ),
Property.Value("Severity", 100));
And combined2 =
new And(combined1,
Property.Value("Source", "Some Source"));
LastCall.Constraints(combined2);
As you can see, this sort of syntax can get messy pretty fast, and I don’t have much use for it. This is where using a manual stub or a callback can make things much clearer, instead of using the And/Or syntax.
Comparing full objects against each other
If we’re going to test things in the simplest way, we could compare two objects. We could send the “expected” object with all the expected properties set as part of the recording process (no need for constraints), and then call verify(), as shown in listing 5.14.
Listing 5.14. Comparing full objects
[Test]
public void TestingObjectPropertiesWithObjects()
{
MockRepository mocks = new MockRepository();
IWebService mockservice = mocks.CreateMock<IWebService>();
using (mocks.Record())
{
mockservice.LogError(
new TraceMessage("Some Message",100,"Some Source"));
}
mockservice.LogError(new TraceMessage("",1,"Some Source"));
mocks.VerifyAll(); //this should fail the test
}
Testing full objects only works for cases where
- it’s easy to create the object with the expected properties.
- you want to test all the properties of the object in question.
- you know the exact values of each constraint.
- the Equals() method is implemented correctly on the two objects being compared. (It’s usually bad practice to rely on the out-of-the-box implementation of object.Equals().)
5.6.3. Executing callbacks for parameter verification
The Is.Matching<T>(Predicate<T>) constraint is a powerful feature that allows the developer to test whatever he wants against the passed-in parameter, and return true or false based on complex rules.
For example, assume that the IWebService interface has a method that expects to take in a TraceMessage object, which in turn has a property that holds an object you’d like to check. If we had a ComplexTraceMessage class with an InnerMessage property and a complex verification on it, it might look like listing 5.15.
Listing 5.15. Using an anonymous delegate to verify a parameter
LastCall.Constraints(
Is.Matching<ComplexTraceMessage>(
delegate(ComplexTraceMessage msg)
{
if (msg.InnerMessage.Severity < 50
&& msg.InnerMessage.Message.Contains("a"))
{
return false;
}
return true;
}));
In listing 5.15, we’re creating a delegate that holds the logic to verify the complex parameter structure.
Instead of using a delegate, we could create a method with the same signature that does the same thing, as shown in listing 5.16.
Listing 5.16. Using a regular method instead of an anonymous delegate
[Test]
public void ComplexConstraintsWithCallbacks()
{
...
using (mocks.Record())
{
mockservice.LogError(new TraceMessage("", 0, ""));
LastCall.Constraints(
Is.Matching<ComplexTraceMessage>(verifyComplexMessage));
}
...
}
private bool verifyComplexMessage(ComplexTraceMessage msg)
{
if (msg.InnerMessage.Severity < 50
&& msg.InnerMessage.Message.Contains("a"))
{
return false;
}
return true;
}
Rhino Mocks has a simpler syntax if you’re only testing a method that accepts a single parameter. Instead of using this syntax,
LastCall.Constraints(
Is.Matching<ComplexTraceMessage>(verifyComplexMessage));
you can write this:
LastCall.Callback(verifyComplexMessage);
Next, we’ll see how to test whether objects raise events properly, or whether other objects have registered for an event properly.
5.7. Testing for event-related activities
Testing event-related actions has always been one of the gaping holes in isolation frameworks and has required manual workarounds to test things such as whether an object registered to get an event from another object, or whether an object triggered an event when it should have. Rhino Mocks has facilities to help in these areas as well.
The first scenario we’ll tackle is checking whether an object registered to receive an event from another object.
5.7.1. Testing that an event has been subscribed to
Let’s assume we have a Presenter class that needs to register to the Load event of a view class it receives. The code for presenter might look like listing 5.17.
Listing 5.17. Testing that an event was registered properly
During the recording stage, we overload the Load event
. Then we make sure we ignore the arguments in the call, and make sure the call happened
.
Some people find that testing whether an event was subscribed to is helpful, but knowing that someone has registered to an event doesn’t mean she did something meaningful with it. It’s not a real functional requirement that’s tested here. If I were doing a test review, I’d say that this was not needed. Instead, you should test that something happened in response to the event being triggered.
To test this scenario as part of a functional requirement, we could say that, upon getting the Load event, the presenter’s production code will do something that we can see from our test (write to a log, for example). To get that something to happen, we need to find a way to trigger the event from within the stub object and see if the presenter does that action in response. That’s what the next section is about.
5.7.2. Triggering events from mocks and stubs
Listing 5.18 shows a test that makes sure Presenter writes to a log upon getting an event from our stub. It also uses a class called EventRaiser, which triggers the event from the interface.
Listing 5.18. Triggering an event via the EventRaiser class in Rhino Mocks
Another way of getting an EventRaiser object is by using the recording mechanism:
|
IEventRaiser eventer; |
Notice in listing 5.18 that we’re using a stub
to trigger the event, and a mock
to check that the service was written to. The EventRaiser takes a stub or a mock and the name of the event to raise from that stub or mock. The Raise() method of EventRaiser takes a params object[] array that requires you to send the number of parameters that the event signature requires. The verification of whether the
message was received happened against the mock service.
Now, let’s take a look at the opposite end of the testing scenario. Instead of testing the subscriber, we’d like to make sure that the event source triggers the event at the right time. The next section shows how we can do that.
5.7.3. Testing whether an event was triggered
There are two basic approaches to testing that an event was triggered. One is simple, but only works in C#, and the other takes a bit more work, but will work in VB.NET. First, let’s look at the simplest way—using a handwritten anonymous method.
Testing event firing inline
A simple way to test the event is by manually registering to it inside the test method using an anonymous delegate. Listing 5.19 shows a simple example.
Listing 5.19. Using an anonymous delegate to register to an event
[Test]
public void EventFiringManual()
{
bool loadFired = false;
SomeView view = new SomeView();
view.Load+=delegate
{
loadFired = true;
};
view.TriggerLoad(null, EventArgs.Empty);
Assert.IsTrue(loadFired);
}
The delegate simply records whether the event was fired or not. You could also have it record the values, and they could later be asserted as well. That code could become quite cumbersome, but most of the time it’s quite a workable solution, and I recommend it if you use C#. Unfortunately, this won’t work in VB.NET because VB.NET currently doesn’t support inline anonymous delegates that don’t return a value. (It does support nonvoid anonymous delegates and will support the void ones in version 10.)
With VB.NET, the solution requires a bit more work. You need to send in the address of a full method in the class, and have that method set a class scope variable flag telling our test whether the event was fired or not. It’s not as clean as I’d like, but it works.
The next section shows a less cumbersome way to test the event triggering and the values that are passed in if we can’t use anonymous delegates.
Using EventsVerifier for event testing
Another approach is to use a class called EventsVerifier (not part of Rhino Mocks), which will dynamically register against the required delegate and verify the values that are passed in by the fired event. EventsVerifier can be downloaded from http://weblogs.asp.net/rosherove/archive/2005/06/13/EventsVerifier.aspx. Listing 5.20 shows an example of its use.
Listing 5.20. Using the EventsVerifier class to test for event values
[Test]
public void EventFiringWithEventsVerifier()
{
EventsVerifier verifier = new EventsVerifier();
SomeView view = new SomeView();
verifier.Expect(view, "Load",null,EventArgs.Empty);
view.TriggerLoad(null, EventArgs.Empty);
verifier.Verify();
}
This test assumes we have a class that implements IView, and that the class has a method that triggers the event. The verifier takes the object to test against, the name of the event, and any parameter values that are passed in as part of the event signature. Rhino Mocks currently doesn’t have a decent enough API to test event verification the way EventsVerifier does.
We’ve covered the basic techniques with Rhino Mocks, so let’s look at a different syntax that it supports: arrange-act-assert.
5.8. Arrange-act-assert syntax for isolation
The record-and-replay model for setting expectations on stubs and mocks has always been a polarizing feature. Some people found it easy to grasp, and some people found it unnatural and cumbersome to write. It also makes tests less readable if you have lots of stubs and expectations in a single test.
The Moq framework changed the landscape in the .NET isolation arena by offering a simplified and concise syntax for setting expectations. Rhino Mocks and Typemock Isolator soon followed.
The idea is based on the way we structure our unit tests today: we arrange objects, act on them, and assert that something is true or false (arrange-act-assert, or AAA). It would be nice if we could use isolation frameworks similarly—to arrange mocks and stubs, set their default behavior, and, only at the end of the test, verify whether a call to the mock object took place.
Listing 5.21 shows a simple test with the record-and-replay syntax, followed by the same test in AAA-style syntax using Typemock Isolator and Rhino Mocks.
Listing 5.21. Record-and-replay versus AAA-style isolation
Setting up Typemock Isolator
To use Typemock Isolator, you’ll need to first download it from Typemock.com and install it. It’s a commercial product with a 21-day (extendable) evaluation period. Unlike the other frameworks, you can’t just reference a DLL and start using it. It requires a Visual Studio plugin to work, and a special runner to run tests in command-line mode.
To add a reference to Typemock, right-click on the test project in Solution Explorer, and select Add Reference. From there, go to the .NET tab and select both “Typemock Isolator” and “Typemock Isolator—C#” or “Typemock Isolator—VB”. (The product has a special VB-friendly API that solves some VB-specific issues with regard to anonymous delegate usage.)
The main difference between the record-and-replay model and the AAA model is that in AAA we don’t need to record what we expect will happen. We just need to check at the end of the test that something did happen correctly. This makes the test more readable, but you’ll also notice the use of .NET 3.5 lambda syntax in the AAA-style tests. This is an essential part of these new APIs and what makes these syntax changes technically possible.
If you’re not comfortable using lambdas yet, you might be better off using the record-and-replay style until you get used to lambdas, and they’re not an obstacle to understanding the code.
Listing 5.22 shows how you’d use AAA-style stubs with Rhino Mocks and Typemock Isolator.
Listing 5.22. Stubs in AAA-style isolation
These tests will fail because we’re throwing a fake exception
from the web service stub. The Expect() method that magically appears on the IWebService interface
when using Rhino Mocks is due to extension methods that are used in .NET 3.5. With Typemock Isolator, there’s a single point
of entry to the API
, so no extension methods are necessary.
Personally, I find that the AAA syntax for Typemock Isolator is more readable than in Rhino Mocks. There are many other facets to the new AAA syntax. You can learn more about it at the websites for the various frameworks.
It’s now time to compare Rhino Mocks to other isolation frameworks in the .NET world.
5.9. Current isolation frameworks for .NET
Rhino Mocks is certainly not the only isolation framework around. But in an informal poll held March 2009, I asked my blog readers, “Which isolation framework do you use?” Close to 46 percent of the more than 600 people who responded reported using Rhino Mocks, 20 percent were using Moq, and 7 percent were using Typemock. (See figure 5.2.)
Figure 5.2. Usage of isolation frameworks among my blog readers
What follows is a short review of the current isolation frameworks in .NET. It’s usually a good idea to pick one and stick with it as much as possible, for the sake of readability and to lower the learning curve for team members. The information that follows should help you make a choice, but note that each of the frameworks mentioned (especially the top three) can add new features at an alarming pace, so the choice of what’s best for your team will seem to be in a constant state of flux.
5.9.1. NUnit.Mocks
NUnit.Mocks is an internal side project of the NUnit framework. It was originally created to provide a simple, lightweight isolation framework that could be used to test NUnit itself, without having to rely on external libraries and version dependencies. Part of NUnit is open source, but it was never regarded as public, so there’s little or no documentation about using it on the web today. Charlie Poole, the current maintainer of NUnit, has said that he is considering either removing it completely from the distribution of NUnit or making it public in version 3.0 of NUnit.
Here are a few of the limitations of NUnit.Mocks:
- It doesn’t support stub syntax.
- It requires strings to expect calls on method names.
- It has no support for testing or firing events.
- It doesn’t support parameter constraints (expected parameter values that are hardcoded in the test).
5.9.2. NMock
NMock is a port of the jMock framework from the Java language. As such, it has been around quite a long time and has many users. It has been largely unmaintained since early 2008 because the developers have gone to work on something bigger and better: NMock2. NMock is open source.
NMock supports the stub syntax but still requires strings for method name expectations. It has no event-raise or test support, but it does support parameter constraints.
5.9.3. NMock2
NMock2 is a large leap forward from NMock. The APIs have changed greatly to accommodate a more fluent calling interface. NMock2, unfortunately, at the time of this writing, has been largely unmaintained since early 2008 and has only come back into some sort of update cycle in 2009, which has driven many people away from using it. NMock2 is open source.
NMock2 supports most, if not all, of the features that Rhino Mocks has, with the main difference being that method expectations are string-based in NMock2, whereas in Rhino Mocks they’re call-based. (You call the method as part of the recording process.) NMock2 also features parameter constraints, event-related assertions, and callback abilities.
5.9.4. Typemock Isolator
Typemock Isolator is a commercial isolation framework, although there’s a free edition with the same features for use in open source project development. Because it’s commercial, it also has good documentation that’s always up to date, a support program, and continually updated versions. Isolator is a perfect fit for testing not only new code but also legacy code (untested, existing code) where testing can be impossible in many situations.
Typemock Isolator builds on top of the abilities of the other frameworks, and it also allows mock (called “fake” in the Isolator API) classes that have private constructors, static methods, and much more. It does this by attaching to the .NET profiler APIs—a set of APIs that allow you to intercept a call to anything, anywhere, including private members, statics, and events. Anything that goes on in the .NET runtime can be intercepted. Typemock Isolator has raised quite a stir in the unit-testing and isolation world of .NET. Some people claim that it may be too powerful, because it makes it easy to simulate and break the dependencies of any object in your existing code. In that way, it doesn’t force you to think about how your design might need to change.
Others feel that it provides a sort of bridge for getting started with testing even if the design is untestable, allowing you to learn better design as you go, instead of having to refactor and learn better design skills before testing. If you can’t mock an object in your code, it can mean that the design could be improved to be more decoupled. That’s why many people like to use their tests to flush out design problems. Appendix A discusses this issue.
5.9.5. Rhino Mocks
Rhino Mocks was first released in June 2005, and has gained a massive user base already. It’s open source, is continuously being worked upon, and has frequent releases. Currently, it’s maintained by a single developer (who seems to have no life whatsoever). It has powerful APIs, and one of the things it’s most noted for is avoiding the use of strings inside tests. To understand why strings in tests are bad, see the sidebar.
Why method strings are bad inside tests
The best way to explain this is to look at an example of using NUnit.Mocks and Rhino Mocks to do the same thing. We’ll see the differences in using strings for method names and using the isolation framework syntax.
We’ll mock the following interface:
interface ILogger
{
void LogError(string msg, int level, string location);
}First, we’ll look at how we’d mock this interface using NUnit.Mocks:
The Rhino Mocks code looks different:
Notice how lines
With Rhino Mocks, changing the name would not be a problem, because we’re invoking the method API as part of our recording stage. Any method changes would keep the test from compiling, and we’d know immediately that there was a problem with the test.
With automated refactoring tools like those in Visual Studio 2005 and 2008, renaming a method is easier, but most refactorings will still ignore strings in the source code. (ReSharper for .NET is an exception. It also corrects strings, but that’s only a partial solution that may prove problematic in some scenarios.)
5.9.6. Moq
Moq is a relatively new framework. It requires using .NET 3.5 because it uses lambda constructs to work its magic. It’s simple to use if you’re comfortable with lambda syntax, but it will only work on interface types and classes that are nonsealed with virtual methods. Unlike Rhino Mocks, it does allow you to fake protected methods.
Let’s recap the advantages of using isolation frameworks over handwritten mocks. Then we’ll discuss things to watch out for when using isolation frameworks.
5.10. Advantages of isolation frameworks
From what we’ve covered in this chapter, we can see some distinct advantages to using isolation frameworks:
- Easier parameter verification— Using handwritten mocks to test that a method was given the correct parameter values can be a tedious process, requiring time and patience. Most isolation frameworks make checking the values of parameters passed into methods a trivial process even if there are many parameters.
- Easier verification of multiple method calls— With manually written mocks, it can be difficult to check that multiple method calls on the same method were made correctly with each having appropriate different parameter values. As we’ll see later, this is a trivial process with isolation frameworks.
- Easier fakes creation— Isolation frameworks can be used for creating both mocks and stubs more easily.
Note
When we create mock objects, we establish expectations as to what calls will be made against our mock object, and we define any return values necessary. With isolation frameworks, an expectation is an essential part of the work process, and it’s an integral part of the isolation syntax. This makes it far easier to write multiple expectations on a mock instance while keeping the test readable.
5.11. Traps to avoid when using isolation frameworks
Although there are many advantages to using isolation frameworks, there are some possible dangers too. Some examples are overusing an isolation framework when a manual mock object would suffice, making tests unreadable because of overusing mocks in a test, or not separating tests well enough.
Here’s a simple list of things to watch out for:
- Unreadable test code
- Verifying the wrong things
- Having more than one mock per test
- Overspecifying the tests
Let’s look at each of these in more depth.
5.11.1. Unreadable test code
Using a mock in a test already makes the test a little less readable, but still readable enough that an outsider can look at it and understand what’s going on. Having many mocks, or many expectations, in a single test can ruin the readability of the test so it’s hard to maintain or even to understand what’s being tested.
If you find that your test becomes unreadable or hard to follow, consider removing some mocks or some mock expectations, or separating the test into several smaller tests that are more readable.
5.11.2. Verifying the wrong things
Mock objects allow us to verify that methods were called on our interfaces, but that doesn’t necessarily mean that we’re testing the right thing. Testing that an object subscribed to an event doesn’t tell us anything about the functionality of that object. Testing that when the event is raised something meaningful happens is a better way to test that object.
5.11.3. Having more than one mock per test
It’s considered good practice to only test one thing per test. Testing more than one thing can lead to confusion and problems maintaining the test. Having two mocks in a test is the same as testing several things. If you can’t name your test because it does too many things, it’s time to separate it into more than one test.
5.11.4. Overspecifying the tests
If your test has too many expectations, you may create a test that breaks down with even the lightest of code changes, even though the overall functionality still works. Consider this a more technical way of not verifying the right things. Testing interactions is a double-edged sword: test it too much, and you start to lose sight of the big picture—the overall functionality; test it too little, and you’ll miss the important interactions between objects.
Here are some ways to balance this effect:
- Use nonstrict mocks when you can. The test will break less often because of unexpected method calls. This helps when the private methods in the production code keep changing.
- Use stubs instead of mocks when you can. You only need to test one scenario at a time. The more mocks you have, the more verifications will take place at the end of the test, but only one of them will usually be the important one. The rest will be noise against the current test scenario.
- Avoid using stubs as mocks. Use a stub only for faking return values into the program under test, or to throw exceptions. Don’t verify that methods were called on stubs. Use a mock only for verifying that some method was called on it, but don’t use it to return values into your program under test. If you can’t avoid this situation, you should probably be using a stub and testing something other than what the mock object receives.
- Don’t repeat logic in your tests. If you’re asserting that some calculation is correct in your code, make sure your test doesn’t repeat the calculation in the test code, or the bug might be duplicated and the test will magically pass.
- Don’t use “magic” values. Try to always use hardcoded, known return values to assert against production code, and don’t create expected values dynamically. That would significantly increase the chances for an unreadable test or a bug in the test.
Overspecification is a common form of test abuse. Make sure you keep your eyes on this by doing frequent test reviews with your peers.
5.12. Summary
Dynamic mock objects are pretty cool, and you should learn to use them at will. But it’s important to lean toward state-based testing (as opposed to interaction testing) whenever you can, so that your tests assume as little as possible about internal implementation details. Mocks should be used only when there’s no other way to test the implementation, because they eventually lead to tests that are harder to maintain if you’re not careful.
Learn how to use the advanced features of an isolation framework such as Rhino Mocks or Typemock Isolator, and you can pretty much make sure that anything happens or doesn’t happen in your tests. All you need is for your code to be testable.
You can also shoot yourself in the foot by creating overspecified tests that aren’t readable or will likely break. The art lies in knowing when to use dynamic versus handwritten mocks. My guideline is that, when the code using the isolation framework starts to look ugly, it’s a sign that you may want to simplify things. Use a handwritten mock, or change your test to test a different result that proves your point but is easier to test.
When all else fails and your code is hard to test, you have three choices: use a “super” framework like Typemock Isolator, change the design, or quit your job.
Isolation frameworks can help make your testing life much easier and your tests more readable and maintainable. But it’s also important to know when they might hinder your development more than they help. In legacy situations, for example, you might want to consider using a different framework based on its abilities. It’s all about picking the right tool for the job, so be sure to look at the big picture when considering how to approach a specific problem in testing.
That’s it! We’ve covered the core techniques for writing unit tests. The next part of the book deals with managing test code, arranging tests, and patterns for tests that you can rely on, maintain easily, and understand clearly.