Writing test cases is not a task that is easily automated, because the test cases have to mirror the functionality of the software being developed. However, at several steps in the process code stubs can be generated by a tool. To illustrate this, start with a fairly straightforward snippet of code to learn to write test cases that fully exercise the code. Setting the scene is a Subscription class with a public property called CurrentStatus, which returns the status of the current subscription as an enumeration value:

As you can see from the code snippet, four code paths need to be tested for the Current Status property. To test this property you create a separate SubscriptionTest test class in a new test project, into which you add a test method that contains the code necessary to instantiate a Subscription object, set the PaidUpTo property, and check that the CurrentStatus property contains the correct result. Then you keep adding test methods until all of the code paths through the CurrentStatus property have been executed and tested.

Fortunately, Visual Studio automates the process of creating a new test project, creating the appropriate SubscriptionTest class and writing the code to create the Subscription object. All you have to do is complete the test method. It also provides a runtime engine that is used to run the test case, monitor its progress, and report on any outcome from the test. Therefore, all you have to do is write the code to test the property in question. In fact, Visual Studio generates a code stub that executes the property being tested. However, it does not generate code to ensure that the Subscription object is in the correct initial state; this you must do yourself.

You can create empty test cases from the Test menu by selecting the New Test item. This prompts you to select the type of test to create, after which a blank test is created in which you need to manually write the appropriate test cases. However, you can also create a new unit test that contains much of the stub code by selecting the Create Unit Tests menu item from the right-click context menu of the main code window. For example, right-clicking within the CurrentStatus property and selecting this menu item brings up the Create Unit Tests dialog displayed in Figure 11-1. This dialog shows all the members of all the classes within the current solution and enables you to select the items for which you want to generate a test stub.

Figure 11-1. Figure 11-1

If you have a unit test project already in your solution you can generate your new test class into it by selecting it from the Output Project drop-down list; otherwise, keep the default selection and Visual Studio will create a new test project for you. Unlike alternative unit test frameworks such as NUnit, which allow test classes to reside in the same project as the source code, the testing framework within Visual Studio requires that all test cases reside in a separate test project. When test cases are created from the dialog shown in Figure 11-1, they are named according to the name of the member and the name of the class to which they belong.

With the CurrentStatus property checked as in Figure 11-1, clicking the OK button generates the following code (some comments and commented-out code have been removed from this code):

The test case generated for the CurrentStatus property appears in the final method of this code snippet. (The top half of this class is discussed later in this chapter.) As you can see, the test case was created with a name that reflects the property it is testing (in this case CurrentStatusTest) in a class that reflects the class in which the property appears (in this case SubscriptionTest). One of the difficulties with test cases is that they can quickly become unmanageable. This simple naming convention ensures that test cases can easily be found and identified.

If you look at the test case in more detail, you can see that the generated code stub contains the code required to initialize everything for the test. A Subscription object is created, and a test variable called actual is assigned the CurrentStatus property of that object. All that is missing is the code to actually test that this value is correct. Before going any further, run this test case to see what happens by opening the Test View window, shown in Figure 11-2, from the Test Windows menu.

Figure 11-2. Figure 11-2

Selecting the CurrentStatusTest item and clicking the Run Selection button, the first on the left, invokes the test. This also opens the Test Results window, which initially shows the test as being either Pending or In Progress.

Once the test has completed, the Test Results window will look like the one shown in Figure 11-3.

Figure 11-3. Figure 11-3

You can see from Figure 11-3 that the test case has returned an inconclusive result. Essentially, this indicates either that a test is not complete or that the results should not be relied upon, because changes may have been made that would make this test invalid.

You can get more information on the result of any particular test result by double-clicking it. Figure 11-4 shows the result of double-clicking the inconclusive result for the example. The results show basic information about the test, the result, and other useful environmental information such as the computer name, test execution duration, and start and end times.

Figure 11-4. Figure 11-4

When test cases are generated by Visual Studio, they are all initially marked as inconclusive by means of the Assert.Inconclusive statement. In addition, depending on the test stub that was created, there may be additional TODO statements that will prompt you to complete the test case.

Returning to the code snippet generated for the CurrentStatusTest method, you can see both an Assert.Inconclusive statement and a TODO item. To complete this test case, remove the TODO comment and replace the Assert.Inconclusive statement with Assert.AreEqual, as shown in the following code:

Each test shown in the Test Results window has a checkbox next to it allowing it to be selected. When test results are selected, clicking the Run Tests button in the Test Results window causes only those selected tests to be run. By default, after a test run any tests that did not pass are selected. After you fix the code that caused these tests to fail, click the Run Tests button to re-run these test cases and produce a successful result, as shown in Figure 11-5.

Figure 11-5. Figure 11-5

When you first created the unit test at the start of this chapter you may have noticed that, in addition to the new test project, two items were added under a new solution folder called Solution Items. These are Chapter11.vsmdi and Local.testsettings.

The .vsmdi file is a metadata file that contains information about the tests within the solution. When you double-click this file in Visual Studio, it opens the Test List Editor, which is discussed at the end of this chapter.

The .testsettings file is a Test Run Configuration file. This is an XML file that stores settings that control how a set of tests, called a test run, is executed. You can create and save multiple run configurations that represent different scenarios, and then make a specific run configuration active using the Test

When you double-click to open the Local.testsettings file, it launches a special-purpose editor. Within this editor you can configure a test run to copy required support files to a deployment directory, or link to custom startup and cleanup scripts. The editor also includes a Test Timeouts section, shown in Figure 11-6, which enables you to define a timeout after which a test will be aborted or marked as failed. This is useful if a global performance limit has been specified for your application (for example, if all screens must return within five seconds).

Figure 11-6. Figure 11-6

Most of these settings can be overridden on a per-method basis by means of test attributes, which are discussed in the next section.

So far, this chapter has examined the structure of the test environment and how test cases are nested within test classes in a test project. What remains is to look at the body of the test case and review how test cases either pass or fail. (When a test case is generated, you saw that an Assert.Inconclusive statement is added to the end of the test to indicate that it is incomplete.)

The idea behind unit testing is that you start with the system, component, or object in a known state, and then run a method, modify a property, or trigger an event. The testing phase comes at the end, when you need to validate that the system, component, or object is in the correct state. Alternatively, you may need to validate that the correct output was returned from a method or property. You do this by attempting to assert a particular condition. If this condition is not true, the testing system reports this result and ends the test case. A condition is asserted, not surprisingly, via the Assert class. There is also a StringAssert class and a CollectionAssert class, which provide additional assertions for dealing with String objects and collections of objects, respectively.

Despite Visual Studio generating the stub code for test cases you are to write, typically you have to write a lot of setup code whenever you run a test case. Where an application uses a database, that database should be returned to its initial state after each test to ensure that the test cases are completely repeatable. This is also true for applications that modify other resources such as the file system. Visual Studio provides support for writing methods that can be used to initialize and clean up around test cases. (Again, attributes are used to mark the appropriate methods that should be used to initialize and clean up the test cases.)

The attributes for initializing and cleaning up around test cases are broken down into three levels: those that apply to individual tests, those that apply to an entire test class, and those that apply to an entire test project.

When you are writing test cases, the testing engine can assist you in a number of ways, including by managing sets of data so you can run a test case with a range of data, and by enabling you to output additional information for the test case to aid in debugging. This functionality is available through the TestContext object that is generated within a test class and passed into the AssemblyInitialize and ClassInitialize methods.

Writing Test Output

Writing unit tests is all about automating the process of testing an application. Because of this, these test cases can be executed as part of a build process, perhaps even on a remote computer. This means that the normal output windows, such as the console, are not a suitable place for outputting test-related information. Clearly, you also don't want test-related information interspersed throughout the debugging or trace information being generated by the application. For this reason, there is a separate channel for writing test-related information so it can be viewed alongside the test results.

The TestContext object exposes a WriteLine method that takes a String and a series of String.Format arguments that can be used to output information to the results for a particular test. For example, adding the following line to the CurrentStatusDataTest method generates additional information with the test results:

VB

TestContext.WriteLine("No exceptions thrown for test id {0}", _
    CInt(Me.TestContext.DataRow.Item(0)))

C#

TestContext.WriteLine("No exceptions thrown for test id {0}",
                this.TestContext.DataRow[0]);

Note

Although you should use the TestContext.WriteLine method to capture details about your test executions, the Visual Studio test tools will collect anything written to the standard error and standard output streams and add that data to the Test Results window.

After the test run is completed, the Test Results window is displayed, listing all the test cases that were executed in the test run along with their results. The Test Results Details window, shown in Figure 11-8, displays any additional information that was outputted by the test case. You can view this window by double-clicking the test case in the Test Results window.

In Figure 11-8, you can see in the Additional Information section the output from the WriteLine method you added to the test method. Although you added only one line to the test method, the WriteLine method was executed for each row in the database table. The Data Driven Test Results section of Figure 11-8 provides more information about each of the test passes, with a row for each row in the table. Your results may differ from those shown in Figure 11-8, depending on the code you have in your Subscription class.

Figure 11-8. Figure 11-8

Up until now, you have seen how to write and execute unit tests. This section goes on to examine how you can add custom properties to a test case, and how you can use the same framework to test private methods and properties.

Custom Properties

The testing framework provides a number of test attributes that you can apply to a method to record additional information about a test case. This information can be edited via the Properties window and updates the appropriate attributes on the test method. At times you might want to drive your test methods by specifying your own properties, which can also be set using the Properties window. To do this, add TestProperty attributes to the test method. For example, the following code adds two attributes to the test method to enable you to specify an arbitrary date and an expected status. This might be convenient for ad hoc testing using the Test View and Properties window:

VB

<TestMethod()>
<TestProperty("SpecialDate", "1/1/2008")>
<TestProperty("SpecialStatus", "Suspended")>
Public Sub SpecialCurrentStatusTest()
    Dim target As New Subscription

    target.PaidUpTo = CType(Me.TestContext.Properties.Item("SpecialDate"), _
        Date)

    Dim val As Subscription.Status = _
        [Enum].Parse(GetType(Subscription.Status), _
        CStr(Me.TestContext.Properties.Item("SpecialStatus")))

    Assert.AreEqual(val, target.CurrentStatus, _

"Correct status not set for Paidup date {0}", target.PaidUpTo)
End Sub

C#

[TestMethod]
[TestProperty("SpecialDate", "1/1/2008")]
[TestProperty("SpecialStatus", "Suspended")]
public void SpecialCurrentStatusTest()
{
    var target = new Subscription();

    target.PaidUpTo = this.TestContext.Properties["SpecialDate"] as DateTime?;

    var val = Enum.Parse(typeof(Subscription.Status),
        this.TestContext.Properties["SpecialStatus"] as string);

    Assert.AreEqual(val, target.CurrentStatus,
        "Correct status not set for Paidup date {0}", target.PaidUpTo);

}

By using the Test View to navigate to this test case and accessing the Properties window, you can see that this code generates two additional properties, SpecialDate and SpecialStatus, as shown in Figure 11-9.

You can use the Properties window to adjust the SpecialDate and SpecialStatus values. Unfortunately, the limitation here is that there is no way to specify the data type for the values. As a result, the property grid displays and enables edits as if they were String data types.

Figure 11-9. Figure 11-9

Note

In the previous version of Visual Studio the TestContext.Properties dictionary was not automatically filled in and you had to do this by hand in your TestInitialize method. In Visual Studio 2010 this is all handled for you.

If you are using the new Code Contracts feature described in Chapter 13, then you might want to write tests that verify the behavior of your contracts. The simplest way to do this is to open the Code Contracts project properties page and uncheck the Assert on Contract Failure checkbox. When you do this the Code Contracts API will raise exceptions instead of causing Assertion failures. You can check for these exceptions with an ExpectedException attribute if you know the type of exception to expect. By default, the Code Contracts tools generate the exceptions that will be thrown and their type cannot be known at runtime. Many of the contract methods have an overload which accepts an exception type as a generic parameter.

Here is a simple class which performs a mathematical operation on positive integers and a unit test to check the case where a negative number is passed in.

Although this method of testing Code Contracts works, it is not really recommended as it may cover up errors in the code. A better option is to hook into the Code Contracts system and override its default behavior from within the test project itself. You do this by registering for the ContractFailed event on the static Contract class inside of an AssemblyInitialize method. Inside of the event handler you tell the Code Contracts API that you have handled the contract failure and that you would like to throw an appropriate exception.

Visual Studio provides both the Test View window and the Test List Editor to display a list of all of the tests in a solution. The Test View window, which was shown earlier in the chapter in Figure 11-2, simply displays the unit tests in a flat list. However, if you have hundreds, or even thousands, of unit tests in your solution, trying to manage them with a flat list will quickly become unwieldy.

The Test List Editor enables you to group and organize related tests into test lists. Because test lists can contain both tests and other test lists, you can further organize your tests by creating a logical, hierarchical structure. All the tests in a test list can then be executed together from within Visual Studio, or via a command-line test utility.

You can open the Test List Editor from the Test Windows menu, or you can double-click the Visual Studio Test Metadata (.vsmdi) file for the solution. Figure 11-10 shows the Test List Editor for a solution with a number of tests organized into a hierarchical structure of related tests.

Figure 11-10. Figure 11-10

On the left in the Test List Editor window is a hierarchical tree of test lists available for the current solution. At the bottom of the tree are two project lists, one showing all the test cases (All Loaded Tests) and one showing those test cases that haven't been put in a list (Tests Not in a List). Under the Lists of Tests node are all the test lists created for the project.

To create a new test list, click Test

After creating a test list, you can run the whole list by checking the box next to the list in the Test Manager. The Run button executes all lists that are checked. Alternatively, you can run the list with the debugger attached using the Debug Checked Tests menu item.

This chapter described how you can use unit testing to ensure the correct functionality of your code. The unit-testing framework within Visual Studio is quite comprehensive, enabling you to both document and manage test cases.

You can fully exercise the testing framework using an appropriate data source to minimize the repetitive code you have to write. You can also extend the framework to test all the inner workings of your application.

Visual Studio Premium and Ultimate contain even more functionality for testing, including the ability to track and report on code coverage, and support for load and web application testing. Chapter 56 provides more detail on these advanced testing capabilities.