20.1. Your First Test Case

Writing test cases is not a task that is easily automated, as the test cases have to mirror the functionality of the software being developed. However, at several steps in the process code stubs can be generated by a tool. To illustrate this, start with a fairly straightforward snippet of code to learn to write test cases that fully exercise the code. Setting the scene is a Subscription class with a private property called CurrentStatus, which returns the status of the current subscription as an enumeration value:

Public Class Subscription
    Public Enum Status
        Temporary
        Financial
        Unfinancial
        Suspended

End Enum

    Private _PaidUpTo As Nullable(Of Date)

    Public Property PaidUpTo() As Nullable(Of Date)
        Get
            Return _PaidUpTo
        End Get
        Set(ByVal value As Nullable(Of Date))
            _PaidUpTo = value

        End Set
    End Property

    Public ReadOnly Property CurrentStatus() As Status
        Get
            If Not Me.PaidUpTo.HasValue Then Return Status.Temporary
            If Me.PaidUpTo.Value > Now Then
                Return Status.Financial
            Else
                If Me.PaidUpTo >= Now.AddMonths(-3) Then
                    Return Status.Unfinancial
                Else
                    Return Status.Suspended
                End If
            End If
        End Get
    End Property
End Class

As you can see from the code snippet, four code paths need to be tested for the CurrentStatus property. If you were to perform the unit testing manually, you would have to create a separate SubscriptionTest class, either in the same project or in a new project, into which you would manually write code to instantiate a Subscription object, set initial values, and test the property. The last part would have to be repeated for each of the code paths through this property.

Fortunately, Visual Studio automates the process of creating a new test project, creating the appropriate SubscriptionTest class and writing the code to create the Subscription object. All you have to do is complete the test method. It also provides a runtime engine that is used to run the test case, monitor its progress, and report on any outcome from the test. Therefore, all you have to do is write the code to test the property in question. In fact, Visual Studio generates a code stub that executes the property being tested. However, it does not generate code to ensure that the Subscription object is in the correct initial state; this you must do yourself.

You can create empty test cases from the Test menu by selecting the New Test item. This prompts you to select the type of test to create, after which a blank test is created in which you need to manually write the appropriate test cases. However, you can also create a new unit test that contains much of the stub code by selecting the Create Unit Tests menu item from the right-click context menu of the main code window. For example, right-clicking within the CurrentStatus property and selecting this menu item brings up the Create Unit Tests dialog displayed in Figure 20-1. This dialog shows all the members of all the classes within the current solution and enables you to select the items for which you want to generate a test stub.

Figure 20.1. Figure 20-1

If this is the first time you have created a unit test, you will be prompted to create a new test project in the solution. Unlike alternative unit test frameworks such as NUnit, which allow test classes to reside in the same project as the source code, the testing framework within Visual Studio requires that all test cases reside in a separate test project. When test cases are created from the dialog shown in Figure 20-1, they are named according to the name of the member and the name of the class to which they belong. For example, the following code is generated when the "OK" button is selected (some comments and commented-out code have been removed from this listing):

Public Class SubscriptionTest

    Private testContextInstance As TestContext

    Public Property TestContext() As TestContext
        Get
            Return testContextInstance
        End Get
        Set(ByVal value As TestContext)
            testContextInstance = Value
        End Set
    End Property

    <TestMethod()> _
    Public Sub CurrentStatusTest()
        Dim target As Subscription = New Subscription
                 'TODO: Initialize to an appropriate value

Dim actual As Subscription.Status
        actual = target.CurrentStatus
        Assert.Inconclusive("Verify the correctness of this test method.")
    End Sub
End Class

The test case generated for the CurrentStatus property appears in the final method of this code snippet. (The top half of this class is discussed later in this chapter.) As you can see, the test case was created with a name that reflects the property it is testing (in this case CurrentStatusTest) in a class that reflects the class in which the property appears (in this case SubscriptionTest). One of the difficulties with test cases is that they can quickly become unmanageable. This simple naming convention ensures that test cases can easily be found and identified.

If you look at the test case in more detail, you can see that the generated code stub contains the code required to initialize everything for the test. A Subscription object is created, and a test variable called actual is assigned the CurrentStatus property of that object. All that is missing is the code to actually test that this value is correct. Before going any further, run this test case to see what happens by opening the Test View window, shown in Figure 20-2, from the test Windows menu.

Figure 20.2. Figure 20-2

Selecting the CurrentStatusTest item and clicking the Run Selection button, the first on the left, invokes the test. This also opens the Test Results window, which initially shows the test as being either Pending or In Progress. Once the test has completed, the Test Results window will look like the one shown in Figure 20-3.

Figure 20.3. Figure 20-3

You can see from Figure 20-3 that the test case has returned an inconclusive result. Essentially, this indicates either that a test is not complete or that the results should not be relied upon, as changes may have been made that would make this test invalid. When test cases are generated by Visual Studio, they are all initially marked as inconclusive by means of the Assert.Inconclusive statement. In addition, depending on the test stub that was created, there may be additional TODO statements that will prompt you to complete the test case.

Returning to the code snippet generated for the CurrentStatusTest method, you can see both an Assert.Inconclusive statement and a TODO item. To complete this test case, remove the TODO comment and replace the Assert.Inconclusive statement with Assert.AreEqual, as shown in the following code:

<TestMethod()> _
    Public Sub CurrentStatusTest()
        Dim target As Subscription = New Subscription
        Dim actual As Subscription.Status
        actual = target.CurrentStatus
        Assert.AreEqual(Subscription.Status.Temporary, actual, _
                        "Subscription.CurrentStatus was not set correctly.")
    End Sub

Rerunning this test case will now produce a successful result, as shown in Figure 20-4.

Figure 20.4. Figure 20-4

By removing the "inconclusive" warning from the test case, you are indicating that it is complete. Don't just leave it at this, because you have actually tested only one path through the code. Instead, add further test cases that fully exercise all code paths.

When you first created the unit test at the start of this chapter you may have noticed that, in addition to the new test project, two items were added under a new solution folder called Solution Items. These are a file with a .vsmdi extension and a LocalTestRun.testrunconfig file.

The .vsmdi file is a metadata file that contains information about the tests within the solution. When you double-click this file in Visual Studio it opens the Test List Editor, which is discussed at the end of this chapter.

LocalTestRun.testrunconfig is a Test Run Configuration file. This is an XML file that stores settings that control how a set of tests, called a test run, is executed. You can create and save multiple run configurations that represent different scenarios, and then make a specific run configuration active using the Test Select Active Test Run Configuration menu item. This will define which of the test run configurations should be used when tests are run.

When you double-click to open the LocalTestRun.testrunconfig file, it will launch a special-purpose editor. Within this editor you can configure a test run to copy required support files to a deployment directory, or link to custom startup and cleanup scripts. The editor also includes a Test Timeouts section, shown in Figure 20-5, which enables you to define a timeout after which a test will be aborted or marked as failed. This is useful if a global performance limit has been specified for your application (for example, if all screens must return within five seconds).

Figure 20.5. Figure 20-5

Most of these settings can be overridden on a per-method basis by means of test attributes, which are discussed in the next section.

20.1.1. Test Attributes

Before going any further with this scenario, take a step back and consider how testing is carried out within Visual Studio. As mentioned earlier, all test cases have to exist within test classes that themselves reside in a test project. But what really distinguishes a method, class, or project as containing test cases? Starting with the test project, if you look at the underlying XML project file, you will see that there is virtually no difference between a test project file and a normal class library project file. In fact, the only difference appears to be the project type: When this project is built it simply outputs a standard .NET class library assembly. The key difference is that Visual Studio recognizes this as a test project and automatically analyzes it for any test cases in order to populate the various test windows.

Classes and methods used in the testing process are marked with an appropriate attribute. The attributes are used by the testing engine to enumerate all the test cases within a particular assembly.

20.1.1.1. TestClass

All test cases must reside within a test class that is appropriately marked with the TestClass attribute. Although it may appear that there is no reason for this attribute other than to align test cases with the class and member that they are testing, you will later see some benefits associated with grouping test cases using a test class. In the case of testing the Subscription class, a test class called SubscriptionTest was created and marked with the TestClass attribute. Because Visual Studio uses attributes, the name of this class is irrelevant, although a suitable naming convention makes it easier to manage a large number of test cases.

20.1.1.2. TestMethod

Individual test cases are marked with the TestMethod attribute, which is used by Visual Studio to enumerate the list of tests that can be executed. The CurrentStatusTest method in the SubscriptionTest class is marked with the TestMethod attribute. Again, the actual name of this method is irrelevant, as Visual Studio only uses the attributes. However, the method name is used in the various test windows when the test cases are listed, so it is useful for test methods to have meaningful names.

20.1.2. Test Attributes

As you have seen, the unit-testing subsystem within Visual Studio uses attributes to identify test cases. A number of additional properties can be set to provide further information about a test case. This information is then accessible either via the Properties window associated with a test case or within the other test windows. This section goes through the descriptive attributes that can be applied to a test method.

20.1.2.1. Description

Because test cases are listed by test method name, a number of tests may have similar names, or names that are not descriptive enough to indicate what functionality they test. The description attribute, which takes a String as its sole argument, can be applied to a test method to provide additional information about a test case.

20.1.2.2. Owner

The Owner attribute, which also takes a String argument, is useful for indicating who owns, wrote, or is currently working on a particular test case.

20.1.2.3. Priority

The Priority attribute, which takes an Integer argument, can be applied to a test case to indicate the relative importance of a test case. While the testing framework does not use this attribute, it is useful for prioritizing test cases when you are determining the order in which failing, or incomplete, test cases are resolved.

20.1.2.4. Work Items

The WorkItem attribute can be used to link a test case to one or more work items in a work-item-tracking system such as Team Foundation Server. If you apply one or more WorkItem attributes to a test case, you can review the test case when making changes to existing functionality. You can read more about Team Foundation Server in Chapter 58.

20.1.2.5. Timeout

A test case can fail for any number of reasons. A performance test, for example, might require a particular functionality to complete within a particular time frame. Instead of the tester having to write complex multi-threading tests that stop the test case once a particular timeout has been reached, you can apply the Timeout attribute to a test case, as shown in the following shaded code. This ensures that the test case fails when that timeout has been reached.

<TestMethod()> _
    <Description("Tests the functionality of the CurrentStatus method")> _
    <Owner("David Gardner")> _
    <Priority(3)> _
    <Timeout(10000)> _
    Public Sub CurrentStatusTest()
        Dim target As Subscription = New Subscription
        Dim actual As Subscription.Status
        actual = target.CurrentStatus
        Assert.AreEqual(Subscription.Status.Temporary, actual, _
                        "Subscription.CurrentStatus was not set correctly.")
    End Sub

This snippet augments the original CurrentStatusTest method with these attributes to illustrate their usage. In addition to providing additional information about what the test case does and who wrote it, this code assigns the test case a priority of 3. Lastly, the code indicates that this test case should fail if it takes more than 10 seconds (10,000 milliseconds) to execute.