5.5. Tools for UI Testing

It's important to keep in mind that when you're testing the UI, you are testing human activity. Because of this, you should let the computer perform the tasks it is good at, and you should perform the types of tasks you are good at. When developers and testers first learn about the tools that can be used to test the UI, they often think that everything can now be automated, and this is very far from reality. Chapter 7 talks about manual testing in depth, but for now it's important that we understand that manual testing is still a very important testing discipline in the testing cycle.

Frederick Brooks taught software developers in the 1987 paper entitled "No Silver Bullet — Essence and Accidents of Software Engineering," that there is no silver bullet in software development, meaning there is not one tool that will solve the entire job you are looking to accomplish. UI testing should not be thought of as the end-all-be-all of testing. When performed properly UI testing is a very strong testing tool.

After you realize the importance of automated UI testing, the next most important decision to make is which tool you will be using to create your tests. There are three schools of thought on how to test the UI.

5.5.1. Impersonating the Browser

This is a set of headless scripts that sends the same type of requests to impersonate the browser, but measures the response times without looking at the content that was sent back. This type of UI testing is great to find bottlenecks inside web applications. The downside of this type of testing is that you are not examining the response and verifying test cases. This type of testing can only be used for volume, stress, and load testing scenarios.

The Web Capacity Analysis Tool (WCAT) is a lightweight HTTP load generation tool that is designed to simulate thousands of concurrent users, making requests to a site. WCAT uses simple scripts to impersonate the browser and fully exercise a website:

NEW TRANSACTION
    classId = 1
    NEW REQUEST HTTP
        Verb = "GET"
        URL = "http://www.google.com"

The preceding example is a WCAT test script at its simplest. Notice in this script there are no verifications to ensure that the page loaded successfully or contained a certain element; the script just navigates to the Google homepage.

5.5.2. Parsing the HTML Ourselves

This concept first starts with an impersonation of browser by opening a web request to the application, then parsing the HTML returned in the response looking for the criteria that will make the test succeed. This method of UI testing is the middle ground between browser impersonation and driving a web browser. This UI testing method is great when you have a set of tests that impersonate the browser, but you need to start moving into a method of testing to start validating responses:

using NUnit.Framework;
using System.Net;
using System.IO;

namespace ParseHttp
{
    [TestFixture]
    public class ParseHttp
    {
        [Test]
        public void Should_Navigate_To_Google_And_Return_Results_For_AspNet()
        {
            string urlToTest = "http://www.google.com/search?hl=en&rlz=1G1GGLQ_
ENUS295&q=asp.net&btnG=Search";
            string result = GetPage(urlToTest);
            Assert.IsTrue(result.IndexOf("The Official") >0, "Asp.net Page ");
        }

        private string GetPage(string urlToTest)
        {
            // create a web client object to load the page from local server

WebClient client = new WebClient();
            string result = string.Empty;

            // dump the contents of the page we navigated to into a stream
            using (StreamReader reader = new
               StreamReader(client.OpenRead(urlToTest)))
            {
                result = reader.ReadToEnd();
            }

            return result;
        }
    }
}

5.5.3. Driving the Browser

In this method of UI testing (shown in the next listing), the test will open a physical browser, perform a behavior, and then the test will verify the HTML based on the test criteria you have entered. This type of automated UI testing is the closest to manual testing you will get. Because the test needs to open a browser, perform a test, and report the results back this type of test is very fragile. These types of tests are often complex, and if your plan is to only stress test your application this type of testing would be overkill:

using NUnit.Framework;
using WatiN.Core;
using WatiN.Core.Interfaces;

namespace SimpleWatinTest
{
    [TestFixture]
    public class SimpleWatinTests
    {
        [Test]
        public void Should_Click_Search_On_Google_And_Return_Results_For_AspNet()
        {
            using (IBrowser ie =
               BrowserFactory.Create(BrowserType.InternetExplorer))
            {
                ie.GoTo("http://www.google.com");
                ie.TextField(Find.ByName("q")).Value = "asp.net";
                ie.Button(Find.ByName("btnG")).Click();
                Assert.IsTrue(ie.ContainsText
                   ("The Official Microsoft ASP.NET Site"));
            }
        }
    }
}

The previous listing contains test code that drives the browser. In this example the WatiN library is used to drive the browser. In this test, a physical instance of Internet Explorer will open on the computer, navigate to the page we are testing, and then ensure that certain text exists on the page that was rendered.

With the different types of automated UI testing methods laid out, you may be wondering which method to use. You may realize you have the need to use all three methods. The decision of which UI testing tool to use depends on which method you have decided fits best with the application you need to test. Hopefully you will see a need to use a tool that will enable you to create tests that fit into each of these testing methods.

To get the feel about how the tools work, we will create a test using multiple tools that go out to www.google.com, searches for the term "ASP.NET," and returns a set of results.

5.5.3.1. WatiN

WatiN is an open source library inspired from the Watir library developed for Ruby. WatiN is an abbreviation for "Web Application Testing in .NET" and the project is currently maintained at http://watin.sourceforge.net/. WatiN is known for its very easy to read syntax that drives the browser. WatiN supports tests that drive both Internet Explorer and Firefox. The WatiN API has a very rich API that includes very easy to use functions that map HTML to WatiN to allow for easy searching of the DOM.

To drive Internet Explorer, WatiN uses COM automation that is built into Internet Explorer. One drawback of the WatiN framework is that HTTP status codes are not exposed, which is currently a limitation of the COM automation in Internet Explorer.

Version 2.0 of WatiN has added support to Firefox. For WatiN to drive the Firefox browser, the JSHH plug-in must be installed. JSSH is a plug-in that exposes a telnet server that allows automation commands to be pushed to Firefox.

In the next listing, an Internet Explorer object is used to open a connection to www.google.com and then ensure that the string Official Microsoft ASP.NET Site appears on the page:

using NUnit.Framework;
using WatiN.Core;
using WatiN.Core.Interfaces;

namespace SimpleWatinTest
{
    [TestFixture]
    public class SimpleWatinTests
    {
        [Test]
        public void Should_Click_Search_On_Google_And_Return_Results_For_AspNet()
        {
            using (IBrowser ie =
               BrowserFactory.Create(BrowserType.InternetExplorer))
            {
                ie.GoTo("http://www.google.com");
                ie.TextField(Find.ByName("q")).Value = "asp.net";
                ie.Button(Find.ByName("btnG")).Click();
                Assert.IsTrue(ie.ContainsText
                   ("The Official Microsoft ASP.NET Site"));
            }
        }
    }
}

Below is a table of how HTML elements can be obtained using the WatiN framework.WatiN HTML Mapping:

5.5.3.2. Screen Capturing

Because UI tests are fragile, it's often difficult to figure out exactly why a test is failing. You have learned that tests should be descriptive, and that developers/testers should be able to drill into the behavior that is failing with ease, but often it's not that easy when testing the UI.

Many UI testing frameworks include methods that can take a screen capture to indicate the state of the system during the test. The next listing is an example using WatiN to capture the screen after the browser has navigated to the page, but before checking that the text exists on the page.

The next example only takes one screen capture, but depending on the test, it's acceptable to capture multi-images during the test. It's important to adopt a naming standard that makes it easy to find the screen capture images when needed. The name of the screen capture is the exact name of the test. If multiple captures were taken, then the state would be appended at the end of the capture name:

string capturePath = m_testCapturePath + fileName;

using (IE ie = new IE())
{
   ie.GoTo("http://www.google.com");
   ie.CaptureWebPageToFile(string.Format("{0}_After_Browser_Launch.jpg",
      capturePath));

   ie.TextField(Find.ByName("q")).Value = "asp.net";
   ie.CaptureWebPageToFile(string.Format("{0}_After_Text_Is_Entered
.jpg",capturePath));

   ie.Button(Find.ByName("btnG")).Click();
   ie.CaptureWebPageToFile(string.Format("{0}_Before_Results_Are_Parsed.jpg",
      capturePath));

Assert.IsTrue(ie.ContainsText("The Official Microsoft ASP.NET Site"));
}

Although the WatiN screen capturing functionality is very useful, it can also be problematic at times. The release of Internet Explorer 8 has caused issues with this functionality of some older versions of the WatiN framework. The images rendered will be solid black. If you encounter these issues, please refer to the WatiN documentation (http://watin.sourceforge.net/documentation.html) for more detailed information:

using System;
using System.Diagnostics;

using System.IO;
using NUnit.Framework;
using WatiN.Core;

namespace WatinTestWithScreenCapture
{
    [TestFixture]
    public class WatinTestWithScreenCapture
    {
        private string m_testCapturePath = string.Empty;

        [SetUp]
        public void setup()
        {
            DirectoryInfo appBasePath = new DirectoryInfo(AppDomain.CurrentDomain
               .BaseDirectory);
            m_testCapturePath = string.Format("{0}/Captures/", appBasePath.Parent
               .Parent.FullName);
        }

        [Test]
        public void Should_Click_Search_On_Google_And_Return_Results_For_AspNet()
        {
            StackFrame stackFrame = new StackFrame();
            string fileName = string.Format("{0}.jpg", stackFrame.GetMethod().Name);
            string capturePath = m_testCapturePath + fileName;

            using (IE ie = new IE())
            {
                ie.GoTo("http://www.google.com");
                ie.TextField(Find.ByName("q")).Value = "asp.net";
                ie.Button(Find.ByName("btnG")).Click();
                ie.CaptureWebPageToFile(capturePath);

                Assert.IsTrue(ie.ContainsText("The Official Microsoft ASP.NET
                   Site"));
            }
        }
    }
}

It's important to note that screen capturing is an expensive resource and should not be abused. Even though it's acceptable to have multiple captures in the same test, please use this functionality with care. Remember that the longer a test takes to run, the less frequently it will be run.

5.5.3.3. Visual Studio Web Tests

Visual Studio web tests first appeared in Visual Studio 2001 Enterprise Edition. In recent years, the Microsoft Application Center Test application was discontinued, and web tests have found a home in the Visual Studio Team System Test. Visual Studio web tests are another type of test that drives the web browser. The power of Visual Studio web tests comes from the test recorder that is provided. The code that is generated from the recorder is much more difficult to read than other tools, but with a little practice a developer would not have much trouble writing the web tests manually:

using System;
using System.Collections.Generic;

using System.Text;
using Microsoft.VisualStudio.TestTools.WebTesting;
using Microsoft.VisualStudio.TestTools.WebTesting.Rules;

namespace SimpleVSWebTest
{
    public class SimpleVSWebTest: WebTest
    {
        public SimpleVSWebTest()
        {
            this.PreAuthenticate = true;
        }

        public override IEnumerator<WebTestRequest> GetRequestEnumerator()
        {

            WebTestRequest request1 = new WebTestRequest("http://www.google.com/");
            WebTestRequest request2 = new WebTestRequest(
               "http://www.google.com/search");
            request2.QueryStringParameters.Add("hl", this.Context["@$HIDDEN1.hl"]
               .ToString(), false, false);
            request2.QueryStringParameters.Add("q", "asp.net", false, false);
            request2.QueryStringParameters.Add("aq", "f", false, false);
            request2.QueryStringParameters.Add("oq", "", false, false);

            if ((this.Context.ValidationLevel >= Microsoft.VisualStudio.TestTools
                .WebTesting.ValidationLevel.High))
            {
                ValidationRuleFindText validationRule2 = new
                   ValidationRuleFindText();
                validationRule2.FindText = "The Official Microsoft";
                validationRule2.IgnoreCase = false;
                validationRule2.UseRegularExpression = false;
                validationRule2.PassIfTextFound = true;
                request2.ValidateResponse += new EventHandler<ValidationEventArgs>(
                   validationRule2.Validate);
            }

            yield return request2;
            request2 = null;
        }
    }
}