5.6. Automating the UI Using Record and Playback

So far in this chapter, you've learned about creating UI tests by hand, meaning that each behavior is coded by hand. This next section will discuss the Record and Playback model, where a tool is used to record the steps and let an application play back performing test cases on each page being played back.

5.6.1. Recorded Tests

Tests that are recorded are the most brittle and many developers don't like to work with them because they are so fragile. Many scripts follow the same pattern: navigate to a page, poke some buttons, and then look at the results. UI tests can be hard to work with because of a lack of clarity. You start with a small script and you keep adding to it. Code in the middle of the script may depend on code that happens in the beginning, and there is seldom an indication for why a keystroke is happening. Most test recorders don't follow the 'Don't repeat yourself' (DRY) principle and duplicate logic from test to test instead. When working with an application that has no UI tests created, using a test recorder can save a great deal of time. It can be very overwhelming to try to create handwritten UI tests for an application that has only 10 web pages, let alone hundreds. We have discussed the negatives of recording tools, but some tools available create scripts that are very easy to use and do not include unneeded code. Well-written test recorders are also great to help a developer new to the UI testing tool learn how to use the API.

5.6.2. WatiN Recorder

The WatiN test recorder (http://watintestrecord.sourceforge.net/) is an automatic web test recorder for the WatiN UI testing framework. The WatiN recorder is not maintained by the same set of individuals who created the WatiN framework and therefore is not included with it. It is, however, an open source tool. As the user navigates through a website, the WatiN recorder will record the session and then convert it to WatiN framework calls that can be saved into your unit tests. The WatiN recorder creates very clean code, making it a great tool for beginners to learn how the WatiN framework works.

In Figure 5-2 the WatiN test recorder was used to navigate to Google; enter ASP.NET in the text box, then press the Search button.

The WatiN test recorder produced the following code:

IE ie = new IE("about:blank");
ie.GoTo("http://www.google.com");
ie.TextField(Find.ByName("q")).TypeText("asp.net");
ie.Button(Find.ByName("btnG")).Click();

StringAssert.Contains("The Official Microsoft ASP.NET Site",
ie.Link(Find.ByUrl("http://www.asp.net/")).Text, @"innerText does not match");

The code may not be exactly the way you would have performed the task if you were writing the code yourself by hand, but nonetheless the test recorder produced working WatiN code that could be entered into a unit test.

The WatiN recorder contains a great deal of features allowing you to perform validation on page elements contained in the DOM. The following code was generated by right-clicking the Official Microsoft ASP.NET link, and using the options contained in Figure 5-3 to create an assert to ensure that the phrase Official Microsoft ASP.NET was contained in a link on the results page:

StringAssert.Contains("The Official Microsoft ASP.NET Site",
   ie.Link(Find.ByUrl("http://www.asp.net/")).Text,
   @"innerText does not match");

When using a test recorder, you might make a mistake and need to delete an action or you might want to add an action without having to do it through the browser. The WatiN recorder takes this into account and provides an interface for deleting the WatiN commands that were generated, or even for adding events to help aid in driving the browser.

Right-clicking the browser page while using the WatiN test recorder will present a list of options to assist with your test creation. These options will help you create a powerful WatiN script to drive the browser.

Figure 5-4 shows the options available when right clicking the browser window in the WatiN Recorder.

• Visual Studio Web Testing. The roots of Visual Studio web tests started in 1999, when Microsoft introduced a tool called the Web Application Stress tool, also known as Homer. Homer recorded a browser session that was saved to a Visual Basic 6 script using COM Automation to drive Internet Explorer. Although it's not the same code base, and does not remotely generate code similar to Homer, the concept of recording the browser session is the same.

The ability to create Visual Studio web tests has bounced around between different versions of Visual Studio, but currently makes its home in Visual Studio Team System Test Edition (2005, 2008, and 2010). Visual Studio Team System Test (VSTS) allows creation of unit tests — test types such as web tests, load tests, manual tests, and database tests are introduced in this version of Visual Studio. Web tests are very closely linked to load tests which will be covered in depth in Chapter 8.

In this section you'll be introduced to the web testing tool included in VSTS, and cover many of the concepts that are supported. It is our intention that you focus more on the concepts and not how VSTS implements the concept. It's important to understand that the concept exists and relate the implementation to tools in the future. Having a firm knowledge of the concepts will allow you to transfer the knowledge to different tools. New web tests can be added to test projects from the Test Menu, and then by selecting "New Test." Figure 5-5 shows the Visual Studio Team System Team Edition Add test screen.

When you select a web test, an instance of Internet Explorer will appear with a toolbar that contains the options for recording the web session. The VSTS web test browser tools are very simplistic, allowing for pausing of the current test, stopping the test entirely, inserting a comment between requests and deletion of requests. Although simplistic, this tool is very powerful and can be of great use when having to create web tests for a system where testing was an afterthought. Having the ability to pause the testing session and delete responses are there in case you make a mistake. With early test recorders, if you made a mistake you need to start over. Figure 5-6 shows the web test browser tools docked on the right side of the browser.

5.6.2.1. Navigation Requests

After the web test has been created and the browser appears with the tools available, as shown in Figure 5-7, your navigation requests will be recorded. When you are finished with the testing session, you can either close the browser or click the Stop button available on the web test toolbar.

After the session is stopped, the results from the test will appear in Visual Studio as a new web test. Figure 5-7 shows the requests that were generated when navigating to www.google.com and searching for "ASP.NET."

5.6.2.2. The Web Test Editor

After the requests to a site have been recorded, they appear in the Visual Studio web test editor. Figure 5-8 the web test editor is where your tests are added and maintained. Figure 5-9 is a contexts menu containing options specific to the test such as allowing for tests to be run, credentials to be set, and the parameterization of the web server URL.

5.6.2.3. Validation Rules

A web test Validation Rule will perform a validation on a request to ensure that the request is valid. Think of a Validation Rule as a type of assert statement that are special to VSTS.

Validation Rules are used to determine whether the response in the test is valid; generally Validation Rules check for the existence of an HTML control or that an HTML control is set to a certain value. By default, web tests will also determine if a step in the test failed based on the HTTP status code it returned. Any return code between 400 and 599 is considered a failure. Web tests are great at finding 404 Page Not Found errors. If you forget to include an image or a script file in the application you are testing, then the web test will fail because of the 404 error.

Figure 5-10 shows the six different types of Validation VSTS supports for you to ensure your website is performing as expected:

• Form Field. This type of Validation Rule will verify the existence and value of a form value on the request.

• Find Text. The Find Text Validation Rule will parse the request and search for the text specified in the Validation Rule. The Find Text Validation Rule will take plain text or a Regular Expression. The Pass If Text Found property allows the test to pass if text is not found. For example, creating a FindText rule with the {FindText="Error" and PassIfTextFound="false"} and set for the properties will ensure that the word error is not found on the page.

• Maximum Request Time. This Validation Rule will ensure that a response does not exceed a maximum value in milliseconds set on the rule. If the maximum time is exceeded then this Validation Rule fails.

• Required Attribute Value. This type of Validation Rule verifies the existence of a specified HTML tag that contains an attribute that is set on the rule.

• Required Tag. This Validation Rule simply tests the occurrence of a specified HTML tag. The minimum number of occurrences property on this Validation Rule can be tested to ensure that a HTML tag appears a certain number of times in the request.

• Response URL. The Response URL Validation Rule verifies that the response URL is the same as the recorded response.

5.6.2.4. Inserting Comments

Comments are a great way of keeping track of what is happening during the test. The VSTS web test editor does not list the requests in a very descriptive way. Having the ability to stick comments between the requests will greatly increase the readability of the web test in the VSTS web test editor. We have talked about the importance of only testing one behavior per test. Sometimes when testing an application that was not designed with testing in mind, we are forced to test more than one behavior. It's important no matter which tool you are using, to keep track of exactly what is happening during the test. As illustrated in Figure 5-11 and Figure 5-12, there is nothing complex about comments, the interface provided only allows you to create text about the current request you are working with.

5.6.2.5. Properties of a Request

Each request in the web test contains a set of properties that provide close control of how the request acts. Figure 5-13 shows the properties that are available for a web request.

• Cache Control. Cache control should be set to true if you would like the web test to simulate the standard browser caching behavior. For example, resources on the website you are testing such as shared images will only be requested once when the cache control property is set to true. The default value of this property is false, but it's important for tests to imitate a user session. Setting the cache control property to false can be useful for load testing, but for functional tests it should be set to true.

• Encoding. This property sets the text encoding format to be used for the requests. The default is set to UTF-8.

• Expected HTTP Status Code. This property will allow the web test to validate against the HTTP status code that is returned, for example if you need to test specific HTTP error codes, such as a 404 page not found error.

• Follow Redirects. This property allows the web test to follow HTTP redirects (300–307 HTTP status codes). The default option is true, and in most cases you should leave this setting to true. There may be cases where your site is performing complex redirects and you need to validate the redirect response; you can set this option to false.

• Parse Dependent Requests. This property determines whether the internal requests that belong to the response page should be executed and parsed or just ignored. Downloading of images is an example of a dependent request, and if this property is set to false then images would not be downloaded. As stated before, VSTS web tests should imitate a browser as closely as possible, so this property should be set to true.

• Record Results. The record results property is a property that is used only for load tests. If set to true, the response timings will be included in the load test results.

• Response Time Goal. Another property that is only used for load testing, the response time goal is a goal set in seconds that the tester is expecting the website to respond in.

• Think Time. A think time is an amount of time to pause on the page before processing the next request.

• Timeout. The timeout property of requests is the maximum time in seconds to wait for the response before causing a timeout and marking the test as a failure.

• Method. Transport method of the request, either GET or PostMethod

• Version. The version property sets the HTTP version to be used in the request. This option can be used to simulate a user hitting your site using an older version of the HTTP protocol. For most tests, this test should remain at the default of 1.1.

• URL. This is the URL of the request.

Two properties that are important to single out in this section are the timeout and think time properties. A think time is an amount of time to pause on the page before processing the next request. Think times are an important concept to keep in mind when creating load tests. If you decide to add a think time at this level, ensure that the request timeout is not exceeded.

5.6.2.6. Context Parameters

How tests are run vary greatly from organization to organization. There are dramatic differences from only running manual tests, or the tests only run on one developer's machine, to having a machine dedicated for testing and builds. Ideally, organizations will have a continuous integration process in place, but how do you manage settings that are specific for a test to run? We have talked about how unit tests should not require any external dependencies, but functional tests are a bit different, often requiring settings that are specific to the environment where the testing is taking place.

A common dependency for web testing is the web server, which we have introduced you to (the Cassini pattern) to help avoid having to set this type of external dependency. In some organizations the web server dependency will be required. VSTS provides a concept called Context Parameters to help manage this requirement. Think of Context Parameters as variables that can be easily changed depending on which environment the test is being run in. A common example would be the web server URL. When you create the test the WebServer1 Context Parameter may be http://localhost, but when the test is run on a different machine it may be http://CompanyTestServer. Figure 5-14 shows the interface that allows you to change the parameterized values of a web test. In this case, the test is being pointed to www.google.com.

5.6.2.7. Running a Web Test

After you have created your VSTS web tests, click the Run Test button (the icon with the green arrow pointing to the right) on the Web Test editor toolbar, and your test will run. A new tab will appear inside Visual Studio that contains the results of your test. Figure 5-15 is the final product of the simple example of navigating to Google and searching for ASP.NET.

Because testing procedures differ greatly from organization to organization, VSTS provides an interface so web tests can be run from the command line. Figure 5-16 shows the call to mstext.exe, specifying the web test name to accomplish this task.

5.6.2.8. VSTS Final Thoughts

The ability to record a browser session, set a few Validation Rules, and then play the session back is a very powerful tool to have in your testing toolbox. Previously in this chapter we discussed the downfalls of automation tools, but VSTS does a good job of keeping tests organized and easy to change in the future. If a web test needs to change, the VSTS web test editor toolbar includes options to delete responses, change Validation Rules, and even insert new browser recording sessions. Behind the screens, the automated scripts are generated as C# code, and if you choose, you can generate the code for each script and make modifications to the test script code.

We've only scratched the surface features that are included in VSTS web tests, but have provided an overview of web testing concepts you can apply to other web testing tools.

5.6.3. Selenium Suite

The Selenium Project is a suite of open source tools used for automated testing of web UIs on multiple platforms. Selenium has the ability to drive Internet Explorer, Firefox, Opera, and Safari. Selenium uses JavaScript and Iframes to embed the test automation engine into the browser. Because browsers handle JavaScript differently, the automation engine is tweaked to support browsers on Windows, Mac, and Linux. The Selenium Project was originally started by Jason Huggins of Thoughtworks in 2004 while he was working on an internal time and expense application.

The Selenium suite has became a very popular option for web UI testing because of the support for multiple browsers and support for creating tests in many languages such as C#, Java, Python, and Ruby. The three parts of the Selenium Project we are going to discuss are the Selenium IDE, Selenium Remote Control (RC), and Selenium Grid.

In 2004 when Selenium was first created, there were only a few record/playback tools on the market. Major tools such as Quick Test Pro and Test Director, were developed by the company Mercury, who were purchased by Hewlett-Packard in 2006. The tools from Mercury were expensive, complicated to learn, and the scripts were hard to maintain.

As an astute chemist may have already noticed, the Selenium element is the antidote to mercury poisoning; the name of the Selenium software suite may have been chosen for this reason.

5.6.3.1. Selenium IDE

The Selenium IDE is a Firefox plug in that it can be used for creating and running Selenium tests to drive the web browser. The Selenium IDE has support for recording a browser session, setting break points/stepping over test code, and the playback of tests. Figure 5-17 shows a simple test that involves navigating to Google and typing in ASP.NET.

Looking at the image in Figure 5-17, you may be thinking, "Would I use the Selenium IDE to run a large test suite with hundreds of tests?" The answer is no — that's where the other tools in the Selenium suite come into play. When you have a number of tests, more than likely you will run the tests in an xUnit testing framework, such as nUnit with references to the Selenium Client Libraries. The Selenium IDE is great for recording the session and saving the output into an xUnit test. Figure 5-18 shows the languages that the Selenium IDE will export the tests into.

5.6.3.2. Selenium RC

The Selenium Remote Control (RC) contains the Selenium server which acts as a proxy for the browser and will run your Selenium tests. The Selenium tools included in Selenium RC will automatically launch and kill browsers as the tests require. The following code creates a connection to the Selenium server (which in this example is running on the local host) and will navigate to www.google.com:

ISelenium browser = new DefaultSelenium("localhost", 4444,
   "*firefox", "http://www.google.com/");

The Selenium RC makes it possible to write automated tests for a web application in just about any programming language. Having this ability allows Selenium to be integrated into existing test frameworks easily.

It's important to note that the Selenium RC comes in two parts, the server portion and the client libraries that allow you to create Selenium tests in your language of choice.

5.6.3.2.1. Selenium Server

The server portion of Selenium RC is written in Java and requires version 1.5.0 or higher of the Java Runtime Environment. The server accepts commands for the browser via HTTP and will launch browsers and follow the HTTP commands passed to the server:

Usage: java -jar selenium-server.jar [-interactive] [options]

Option	Description
port <nnnn>	Port number the Selenium server should use (default 4444)
multiWindow	Puts the server into a mode where the test website executes in a separate window, and Selenium supports frames.
forcedBrowserMode <browser>	Sets the browser mode (e.g. "*firefox") for all sessions, no matter what is passed.
browserSessionReuse	Stops the initialization and creation of the browser between tests.
htmlSuite <browser> <startURL> <suiteFile> <resultFile>	Runs a single HTML Selenese (Selenium Core) suite and then exits immediately using the specified browser (e.g. "*firefox") on the specified URL (e.g. "http://www.google.com"). You need to specify the absolute path to the HTML test suite as well as the path to the HTML results file you'll generate.

5.6.3.2.2. Selenium Client Libraries

To make writing tests easier, the Selenium Project currently provides client drivers for Python, Ruby, .NET, Perl, Java, and PHP. The Java driver can also be used with JavaScript (via the Rhino engine).

[Test]
public void Should_Click_Search_On_Google_And_Return_Results_For_AspNet()
{
   ISelenium browser = new DefaultSelenium("localhost", 4444, "*firefox",
      "http://www.google.com/");

   browser.Start();
   browser.Open("/");
   browser.Type("q", "asp.net");
   browser.Click("btnG");
   browser.WaitForPageToLoad("30000");

   Assert.IsTrue(browser.IsTextPresent("The Official Microsoft ASP.NET Site"));
   browser.Close();
}

The Selenium core uses a language named Selenese to drive the browser. The Selenium RC client library is a wrapper to this language, and will translate commands created in your favorite language into Selenese to be transported via http. Using the Selenium RC client libraries, makes it so the end user of the APIs does not need to learn the specific syntax to Selense, but it is useful to learn about how Selenese and its commands work. The commands come in three types:

• Actions. These are commands that manipulate the state of the application such as click this link.

• Accessors. These look at the state of the application and store the results.

• Assertions. These verify that the state of the application conforms to what is expected. Selenium has three types of modes for assertions:

  • Assert. When assert fails the test is aborted.

  • Verify. When verify fails the test will continue to execute, but the failure will be logged.

  • Wait For. Wait For assertions wait for a condition to become true. These types of assertions are very useful for testing AJAX.

5.6.3.2.3. Element Locators

When creating automated UI web tests, a good portion of your time will be spent using the testing framework to locate tags in the returned HTML to verify whether the test has passed. The Selenium RC client library has a concept of Element Locators that tells Selenium which HTML element a command refers to. Understanding how to specify Selenium Element Locators will help you build tests that are reliable and are easy to maintain if the application changes. Currently Selenium supports the following strategies for locating elements:

• Identifier. Selects the element with the specified id attribute. If no match is found, selects the first element whose name attribute is id.

• Id. Selects the element with the specified id attribute.

  selenium.click("id=btnClickMe");

  
  

• Name. Selects the element with the specified name attribute.

  selenium.click("name=myButton");

  
  

• Dom. Selects the element by evaluating the specified string. Using this type of locator allows JavaScript to be entered to traverse the DOM.

  selenium.click("dom=document.images[2]" );
  selenium.click("dom= function GetMyLink() { return document.links[4]; };
  GetMyLink();");

  
  

• XPath. XPath is a language for addressing parts of an XML document, designed to be used by both XSLT and XPointer:

  selenium.click("xpath=//img[@alt='Alt text of my image']");

  
  

• Link. Selects the anchor tag element which contains the following text:

  selenium.click("link=My Link");

  
  

• CSS. Selects the element using CSS selectors:

  selenium.click("css=div#MyButton");

  
  

Below is a table defining some of the methods you can use with Selenium together with the selectors above in order to identify different parts of the web page

Command	Description
captureScreenshot("File name")	Captures a PNG screenshot to the specified file.
Check("Locator")	Checks a toggle-button (checkbox/radio).
click("Locator")	Clicks on a link, button, checkbox, or radio button.
close()	Simulates the user clicking the "close" button in the title bar of a pop-up window or tab.
doubleClick("Locator")	Double-clicks a link, button, checkbox, or radio button.
getAlert()	Retrieves the message of a JavaScript alert generated during the previous action, or fails if there were no alerts.
getAllButtons()	Returns the IDs of all buttons on the page.
getAllFields()	Returns the IDs of all input fields on the page.
getAllLinks()	Returns the IDs of all links on the page.
getBodyText()	Gets the entire text of the page.
getTable("Table Cell Address")	Gets the text from a cell of a table.
getText("Locator")	Gets the text of an element.
getTitle()	Gets the title of the current page.
goBack()	Simulates the user clicking the "back" button on their browser.
isAlertPresent()	Has an alert occurred?
isChecked("Locator")	Gets whether a toggle-button (checkbox/radio) is checked.
isTextPresent("Pattern")	Determines whether some option in a drop-down menu is selected.
isVisible("Locator")	Verifies that the specified text pattern appears somewhere on the rendered page shown to the user.
open("URL")	Determines if the specified element is visible.
openWindow("URL","WindowID")	Opens a URL in the test frame.
refresh()	Opens a pop-up window (if a window with that ID isn't already open).
Start()	Launches the browser with a new Selenium session.
Stop()	Ends the test session, killing the browser.
submit("Form Locator")	Submits the specified form.
waitForCondition("JS","Timeout")	Runs the specified JavaScript snippet repeatedly until it evaluates to "true".
waitForFrameToLoad("Frame Address","Timeout")	Waits for a new frame to load.
waitForPageToLoad("Timeout")	Waits for a new page to load.
waitForPopUp("WindowID","Timeout")	Waits for a pop-up window to appear and load up.

5.6.3.3. Selenium Grid

Testing a Web UI with automation is a slow process. Automated UI tests are much slower than unit tests, and as you build a large test suite the time it takes to execute all the tests could range from a few hours to a few days. We have learned that the longer it takes to run a suite of tests, the less frequently they will be run. Selenium Grid is a tool that allows you to run Selenium tests in parallel, cutting down the time required for the entire test suite to run. Selenium Grid runs on top of the Selenium RC, and can be configured to run your tests on multiple machines.

Figure 5-19 is a representation of a traditional Selenium test. A test script in a framework such as nUnit makes a call to Selenium RC, which in turn spawns a new web browser with the application under test and then the test steps are executed. Selenium RC is slow at driving the browser and in most situations will be the bottleneck of the test suite. There is also a limitation of the number of concurrent tests that can be run on the same Selenium RC instance before the Selenium RC instance becomes unstable. The Selenium team currently recommends no more than six tests with a single instance of Selenium RC. These limitations can cause tests suites to not run as effectively as they could.

Selenium Grid builds upon the traditional Selenium test setup by adding a component called the Selenium Hub. The Selenium Hub exposes an external interface that is the same as the Selenium Remote Control, meaning an existing Selenium test suite that is using the Selenium RC can be switched to the Selenium Grid without any code changes. The Selenium Hub will allocate instances of Selenium RC for each incoming test. Figure 5-20 is a representation of using the Selenium Grid.

Another great feature of Selenium Grid is that tests can be configured to run against a specific environment. Selenium Grid Environments can be configured in the grid_configuration.yml file found in the root directory of the Selenium Grid binary file. To take advantage of specific environments in the Selenium Grid, an existing test that created an instance of Firefox could be modified to only test with Firefox on Windows. Figure 5-21 is a visualization of how a Selenium Grid environment may look.

An example of an existing Selenium test:

new DefaultSelenium("localhost", 4444, **'*firefox'**, 'http://www.Google.com'),

An example of the same tests configured to use an instance of Selenium Grid that only runs Firefox on Windows:

new DefaultSelenium("localhost", 4444, **'Firefox on Windows'**,
   'http://www.Google.com);

Figure 5-22 shows what the entire Selenium suite looks like working together.