Understanding different kinds of software testing
Creating and running unit tests
Introducing stubs and mocks
Using the Object Test Bench
Exploring unit-testing frameworks
So you're plugging along coding like crazy. You'll have this set of requirements knocked out in no time. But how do you know when you're done? If you're unit testing your code while you go, the answer is when your code passes all your tests. If you're not unit testing while you go, you'll probably code until you feel like you're done and then pass the code off for somebody else to test.
Testing is an important part of the software development life cycle. Many kinds of tests are performed to ensure an application fulfills its requirements. Developers are usually responsible for writing unit tests to test the code they write. In Visual Studio 2010 Professional Edition, testing is more of a coding practice than a feature. Microsoft provides upgraded editions of Visual Studio that provide extensive support for all kinds of testing.
This chapter discusses the importance of testing, shows you how to write simple unit tests, and introduces you to some of the Visual Studio tools you can use to support your testing efforts.
Testing means different things, depending on the context in which it's used. Developers are usually responsible for testing the code they write — unit testing. After code passes unit tests, it's usually checked into a source code control repository. At some point, the entire system under development is compiled, and quality assurance testers perform even more testing.
The kinds of tests you might encounter include
Unit tests: These programs are written by developers to test the code they write.
Integration tests: These test units of code after they're integrated with each other.
System tests: During system tests, the entire integrated system is tested. Depending on the kind of software being tested, system tests might include user interface testing, regression testing, and load testing.
Acceptance tests: During acceptance tests, any or all of the system's stakeholders might participate in testing the software in a lab.
When software progresses through testing, tests become less focused on the inner workings of the code. As a result, testing becomes less automated and requires more user interaction. In the case of acceptance testing, a test lab is often set up where end users come in and bang away at the system for weeks at a time, uncovering bugs and functional shortcomings.
As a developer, you're actively involved in writing and running unit tests. Your interaction with the testing process beyond unit testing depends on several factors, including the extent to which your code provides any of the system's core functionality.
Of course, use of unit testing doesn't mean that no bugs exist. Rather, the implicit understanding is that when you "check in" your code, you're telling other developers that the code works at some basic level of functionality. Code that breaks every time that it's called by another developer shows a pretty good indication that you're either not unit testing at all or not doing enough unit testing.
If you've worked as a tester, you can likely spot those developers who never run unit tests. Not only does their code usually not conform to the requirements, but it usually blows up as soon you try to test it. Other developers, however, are very conscientious about unit testing their code. As a result, you won't need to interact with those developers as much. In other words, if you don't want quality assurance and your fellow developers on your back, unit test your code.
By definition, unit testing is about writing code. When you write a unit test, you're writing code to test code. You write a unit test to test a single unit of your code. For example, assume that your project includes code for a Customer object and an Address object. You should have one unit test for each object. The unit test contains multiple tests to test the various methods and properties of your object. Figure 8-1 shows an example.
You'll find tons of rules about how you should write your tests. However, they're not really rules: They're more like opinions. And everybody has an opinion on what it means to unit test. Some good guidelines to follow are
All tests should be automated and run as part of a suite.
All your code should be tested.
You should write your tests before you write your code.
Note
Getting started with unit testing can be quite overwhelming. Although these approaches are valid and ones to which you should aspire, it's better to
Write imperfect tests than to write no tests at all.
Run tests manually before you check in your code than run no tests at all. Test the riskiest parts of your code first until you get the hang of unit testing.
Unit testing is an important good coding practice. While you're coding, you should write tests that make sure your code works as you expect. Unit tests aren't supposed to test every aspect of the system. Rather, unit tests are sanity checks you use to make sure that your code works. For example, if your requirement states that the function should return an integer, you might create a unit test to test that the value returned by your function is indeed an integer.
When used properly, unit tests help you achieve the following important goals:
Lots of patterns and frameworks are available for writing unit tests. At its simplest, a unit test simply tests that your code does what it says it does. Unit tests either pass or fail. There's no in-between.
Before you can write a unit test, you need code to test. This section uses a very simple Hello World example with the following methods:
CreateMessage(): Creates aHello Worldmessage, depending on whether the user supplies a name.SayHello(): Displays aHello Worldmessage in a message box.
These methods are used in a Windows Form that displays a Hello World message. If the user enters a name in a text box on the form, the user's name appears in the message.
Listing 8-1 shows the code to be unit tested. Notice the two methods to test.
Example 8-1. Sample Unit Test Code
private string CreateMessage(string name)
{
if (name == null)
{
throw new ArgumentNullException("name");
}
else if (name == string.Empty)
{
throw new ArgumentException("Parameter must not be empty.", "name");
}
return ("Hello" + " " + name + "!");
}
private void SayHello(string name)
{
if (name == null)
{
throw new ArgumentNullException("name");
}
else if (name == string.Empty)
{
throw new ArgumentException("Parameter must not be empty.", "name");
}
MessageBox.Show(CreateMessage(name), "Sample Application",
MessageBoxButtons.OK, MessageBoxIcon.Information);
}To test this code, you write a set of tests that makes sure that the output is as expected. For example, the SayHello() method is supposed to return Hello World. The unit test should test the return value. Visual Studio 2010 automates this unit testing for you.
To create unit tests for these two methods:
Select the methods in the code editor.
Right-click and choose Create Unit Tests from the contextual menu, shown in Figure 8-2.
Visual Studio displays the Create Unit Tests dialog box, shown in Figure 8-3.
Click OK.
Visual Studio creates a new Visual C# project in your solution that creates a test harness to unit tests for the methods you selected in Figure 8-3. Figure 8-4 shows this project in the Solution Explorer.
Each test you create is a test case. Some functions might require multiple test cases. Visual Studio automatically creates unit tests for you, as shown in Listing 8-2.
Example 8-2. Code to Test the Set of Code
namespace TestProject1
{
/// <summary>
///This is a test class for Form1Test and is intended
///to contain all Form1Test Unit Tests///</summary>
[TestClass()]
public class Form1Test
{
private TestContext testContextInstance;
/// <summary>
///Gets or sets the test context which provides
///information about and functionality for the current test run.
///</summary>
public TestContext TestContext
{
get
{
return testContextInstance;
}
set
{
testContextInstance = value;
}
}
#region Additional test attributes
//
//You can use the following additional attributes as you write your
tests:
//
//Use ClassInitialize to run code before running the first test in the
class
//[ClassInitialize()]
//public static void MyClassInitialize(TestContext testContext)
//{
//}
//
//Use ClassCleanup to run code after all tests in a class have run
//[ClassCleanup()]
//public static void MyClassCleanup()
//{
//}
//
//Use TestInitialize to run code before running each test
//[TestInitialize()]
//public void MyTestInitialize()
//{
//}
//
//Use TestCleanup to run code after each test has run
//[TestCleanup()]
//public void MyTestCleanup()
//{
//}
//
#endregion
/// <summary>
///A test for CreateMessage
///</summary>
[TestMethod()][DeploymentItem("WindowsFormsApplication1.exe")]
public void CreateMessageTest()
{
Form1_Accessor target = new Form1_Accessor(); // TODO: Initialize to
an appropriate value
string name = string.Empty; // TODO: Initialize to an appropriate
value
string expected = string.Empty; // TODO: Initialize to an appropriate
value
string actual;
actual = target.CreateMessage(name);
Assert.AreEqual(expected, actual);
Assert.Inconclusive("Verify the correctness of this test method.");
}
/// <summary>
///A test for SayHello
///</summary>
[TestMethod()]
[DeploymentItem("WindowsFormsApplication1.exe")]
public void SayHelloTest()
{
Form1_Accessor target = new Form1_Accessor(); // TODO: Initialize to
an appropriate value
string name = string.Empty; // TODO: Initialize to an appropriate
value
target.SayHello(name);
Assert.Inconclusive("A method that does not return a value cannot be
verified.");
}
}
}The Assert class in the Microsoft.VisualStudio.TestTools.UnitTesting namespace provides a set of static methods that provide you the conditions for your unit tests. Methods on the Assert class throw the AssertFailedException if the condition specified isn't true. Table 8-1 lists the methods on the Assert class.
Table 8-1. Assert Class Static Methods
Method | Description |
|---|---|
| Verifies that specified values are equal |
| Verifies that specified values aren't equal |
| Verifies that specified object variables refer to different objects |
| Verifies that specified objects refer to the same object |
Fails an assertion without checking any conditions | |
| Indicates that an assertion can't be proven true or false. Also used to indicate an assertion that hasn't yet been implemented |
| Verifies that a specified condition is false |
| Verifies that a specified object is an instance of a specified type |
| Verifies that a specified object isn't an instance of a specified type |
| Verifies that a specified object isn't null |
| Verifies that a specified object is null |
| Verifies that a specified condition is true |
You should be able to run your tests with the click of a mouse or by entering a single command. To run these tests, right-click the test project in Solution Explorer and choose Debug
Based on the results in Figure 8-5, it looks like the unit tests aren't complete enough to adequately test the code. In this case, both unit tests failed because SayHello() and CreateMessage() throw System.ArgumentException if the parameter "name" is empty. In this case, you have to tell the unit test framework that you're expecting these methods to throw exceptions if the parameter "name" is null or empty. To do so, you add an ExpectedException attribute to each of the test methods:
[ExpectedException(typeof(System.ArgumentException))]
public void SayHelloTest()...Adding an ExpectedException tells the unit test framework that you're expecting your methods to throw an exception of type System.ArgumentException or System.ArgumentNullException because it's derived from System.ArgumentException. Now, when you pass string.Empty or null to these methods, the unit tests won't fail because you're expecting these methods to throw exceptions when these values are provided for the parameter "name". So to adequately test these methods, you need to provide an empty string, a null string, and a valid string to each of these methods. You also want to test the negative case where the expected value isn't equal to the actual value. Listing 8-3 shows the complete unit test for CreateMessage.
Example 8-3.
/// <summary>
///A test for CreateMessage
///</summary>
[TestMethod()]
[DeploymentItem("WindowsFormsApplication1.exe")]
[ExpectedException(typeof(System.ArgumentException))]
public void CreateMessageTest()
{
Form1_Accessor target = new Form1_Accessor();
string name = "Andrew";
string expected = "Hello Andrew!";
string actual;
actual = target.CreateMessage(name);
Assert.AreEqual(expected, actual);
name = string.Empty;
expected = string.Empty;
actual = target.CreateMessage(name);
Assert.AreEqual(expected, actual);
name = null;
expected = null;
actual = target.CreateMessage(name);Assert.AreEqual(expected, actual);
name = "Andrew";
expected = "Hello Joe!";
Assert.AreNotEqual(expected, actual);
}The first test sets the parameter "name" to "Andrew". In this test, you expect the resulting message to be "Hello Andrew!" as indicated by the Assert.AreEqual(). The second test passes an empty string as the "name" parameter. Because you provided the ExpectedException attribute for the System.ArgumentException, this tests two passes. The third test passes null as the "name" parameter. This time, too, the results are what you expect because CreateMessage() throws a System.ArgumentNullException. The final test tests the negative case where "Andrew" is again passed to CreateMessage(), and you test to ensure that the result isn't equal to something other than "Hello Andrew!". In this case, "Hello Joe!" doesn't match, and the Assert.AreNotEqual is true. Now, in Figure 8-6, Visual Studio shows that your tests have succeeded.
Within the past few years, unit testing has taken on a life of its own. You're not cool if you're not doing a unit test. One of the hardest choices when creating unit tests is deciding what to test. When you examine code that you want to test, consider the following issues:
Public methods: Public methods are those features of your code that are exposed to the outside world. Pretty much everyone agrees you should unit test your public methods. After all, a public method is sitting there saying, "Call me."
Private and protected methods: Private and protected methods in your code do the work of your public methods. As a result, some people believe that testing private and protected methods is wasteful. They believe that unit testing public methods implicitly tests private and protected methods. Plus, testing private and protected methods is harder. By their nature, private and protected methods are, well, private and protected. Unlike public methods, they aren't saying, "Call me." Just the opposite is true, so you need to go out of your way to test them.
Interactions with other code: Unless you're writing all your application's features in a single function, chances are high your code needs to interact with other code. The very nature of unit testing, however, is that you want to test only a single unit of code. How can you do that if you have to interact with other code?
Public methods are easily tested because, by definition, you can call a public method from anywhere in your code.
People take all kinds of approaches to solve the problems of testing private methods or code that interacts with other code. One popular approach is to use software that specializes in unit testing, usually called unit testing frameworks. (See the section "Automating Tests with Testing Frameworks," later in this chapter.)
Two approaches to testing code that interacts with other code are stubs and mocks.
By definition, a unit test is supposed to test only one unit of code. No unit of code lives in isolation, however. Most developers work around this pesky paradox by using code stubs, which act as placeholders for units of code that the unit being tested needs to interact with.
The code stub placeholder is better than interacting with the actual units of code because each unit of code introduces a new set of variables. Instead, you can program the code stub to return a consistent set of values against which the unit can interact. Returning a consistent set of values creates predictability, which makes it easier to create tests that you can repeat.
In reality, developers who use stubs usually go ahead and test the real units of code that interact with the unit they're testing — that is, as long as it's simple to do so. For example, say that you create a Customer object with a public method called GetAddress.GetAddress returns an Address object. The Customer and Address objects are each units of code that have their own set of unit tests. Many developers will create a test that allows their Customer object to interact with the Address object even though the test extends beyond the unit they want to test.
When the developer encounters a set of code that is too complicated or messy to interact with a simple unit test, the developer usually creates a stub. The stub returns a set of canned values that the developer expects to get from the code.
For example, say that your Customer object has a method called UpdateAddress.UpdateAddress accepts a new address for a customer and passes that data off to an Address object. The Address object calls a service that validates the address against a valid ZIP code database. You don't want to have to deal with all that when you're testing your Customer object.
Instead of using the Address object, you create a stub that returns two possible values. If you pass a valid address, it returns the value ValidAddress. If you pass an invalid address, it returns the value InvalidAddress. So, how does the stub know what's a valid address and what's an invalid address? You specify in the stub that 123 Main Street, for example, is a valid address. Everything else is invalid. Figure 8-7 shows an example of a stub.
Other developers believe that creating a bunch of objects and stubs is too much work. These developers use a special library to create mock objects. With mock objects — mocks — you tell the object which methods you expect it to run, how many times the methods will run, and what values to return.
For example, say that you use a mock Address object to interact with your Customer address. For your UpdateAddress test, you tell the mock Address object to call the Address object's Save method. Your test tells the mock object that you're going to send a value of 123 Main Street to the Save method and that you expect it to return a value of true. Your test also tells the mock object that you're going to send the value 123 Nowhere Boulevard and that you expect the Save method to return the value false. Figure 8-8 shows an example of a mock object.
Deciding which method to use can be a little difficult at first glance, but several key differences exist between using mocks and stubs, based on what you want to test for. Here are some of them:
Mocks
You tell your mock objects which methods to call, how many times they'll be called by your test, and what return values you expect.
You create mocks for all objects with which your code interacts.
Mocks require a library of code to create the mock objects.
Stubs usually return only values.
You create stubs only when necessary.
People who use stubs tend to test clusters of interacting objects together. For example, they might test all the code related to customers and orders together because those units of code interact.
People who use mocks, on the other hand, are more likely to take a top-down approach. They might start testing at the user interface level first and then use mock objects to mimic the behavior of business rules or data access code. They are likely to test first and code second. In other words, they use testing as a way to discover what code they still need to write to make their system work.
Unit tests are often grouped so that they can be executed by a testing framework. A framework isn't necessary, but using a framework allows you to do these things:
Formalize unit testing.
Create consistency among unit tests.
Make automating testing easier.
Execute groups of tests at one time.
Frameworks are a necessity for most systems. Because unit tests are so small in nature, even a simple system has lots of tests. In most cases, using a framework to administer the tests makes sense. Unit testing frameworks exist for nearly every programming language under the sun. Popular unit testing frameworks for the .NET Framework include
NUnit: By far, NUnit is the most-popular testing framework around. NUnit is an open source (free) framework. However, a drawback to NUnit is that it isn't integrated into Visual Studio. Still, many folks use NUnit to manage their unit tests. You can visit the NUnit Web site at
www.nunit.org.TestDriven.NET: Integrates testing frameworks, such as NUnit, into the Visual Studio development environment. TestDriven.NET gives you the ability to access testing commands from a shortcut menu in the code editor. You can visit the TestDrive.NET Web site at
www.testdriven.net.