Running UI Tests on the Build Server

If we want to run tests against the user interface we typically need to have an interactive session running that can open application windows as part of the test. If we choose to run UI tests on the build server this means we need to be running the build service interactively. This is generally not recommended because it affects the way the service is run. But if we decide to set up the build service for interactive testing, we do so from the Build Service properties in the TFS Administration Console (see Figure 27-3). A good practice if you want to run interactive tests on the build server is to set up a dedicated build service for this and use build agent tags to direct builds with UI test to that particular environment.

Figure 27-4 shows the TFS Build Service running in interactive mode. As the display text says, the service will run until someone exits the service. If someone logs off the interactive session or restarts the machine, it will unfortunately result in the build service not running.

Running UI tests on a Test Server

If we want to run user interface tests on the test server we have the option to configure the Visual Studio Test Agent to run as an interactive process (see Figure 27-6). We can set a number of settings for the interactive process:

• Specify the user that will run the interactive process: For details on the requirements for the test agent account see http://msdn.microsoft.com/en-us/library/hh546459(v=vs.110).aspx.
• Have the test agent log on as the interactive user automatically: This is a good option if we want the labs to run unmanaged but it may have implications on corporate policies because the workstation will run unlocked.
• Disable screen savers to run: Most UI test will fail if the desktop session is locked, so by default we should disable screen savers.

Create a BDT Build Definition

To create a build-deploy-test build definition we follow the procedure for a default build with the following exceptions:

• Workspace: May not be necessary as the BDT workflow will only deploy and test. If the BDT workflow has dependencies that the BDT build uses (for instance deployment scripts), then we may need to setup the workflow as well.
• Build Defaults: No drop folder is needed as the BDT workflow normally will not produce any output.
• Process: Select the LabDefaultTemplate.11.xml as the build process template.

Figure 27-11 shows our BDT workflow being setup.

A special requirement to run Visual Studio Lab Management BVT workflows that queue a compile build as part of the workflow is that there needs to be at least two agents available. If only we have one build agent then the deployment build will be blocked by itself waiting for the build workflow to begin. Figure 27-12 shows a Build Controller with two build agents.

The only required argument in the LabDefaultTemplate is that we open the configuration wizard and configure the build as described in the following section.

Selecting the Target Environment

The first step in the Lab Workflow wizard is to select the lab environment. We need to select an existing Lab Management environment and we can optionally configure the workflow to revert the environment to a specific state before the deployment is done given that the environment is using SCVMM (see Figure 27-13). See Chapter 23 for details on how to work with Visual Studio Lab Management to create test and lab environments.

Selecting Application Build to Deploy

Next we select the build to deploy. We have the following options for picking a build that the workflow should use:

• Use a Team Foundation build: Here we can either queue a new build as part of the BDT workflow or we can pick an existing build.
• Use a build from a specified location: We can also choose to deploy a build from a selected location; this is useful if other tools are used to produce builds to deploy.

Figure 27-14 shows the build to deploy configuration dialog.

Defining the Deployment Steps

With target environment and build to deploy selected, we can now add the deployment steps (see Figure 27-15). The Lab Workflow is very generic; it allows us to run commands or scripts on each of the machines in the environment. We can address the machines either by role or by name—whatever is most suitable.

The scripts are run locally on the target machine so it needs to be available as specified on the target machine. If necessary, we can also specify a working directory.

When we add deployment commands we may need to pass in arguments from the build process. Currently we can use the following properties:

• $(BuildLocation): This is the location of the build. If you specified to use the build from a shared location, then this variable represents that path. For the other options, this is the full path for the build based on the configuration that you selected to build and the build drop location in the build definition. If you build your application as part of your workflow, you can use this to access the latest files that were created by that build.
• $(InternalComputerName_ <VM Name>): This is used to obtain the computer name for a virtual machine that is part of a virtual environment. You might know the virtual machine name, but not the computer name. If you have a deployment script to set up a web server that requires the computer name, you can pass this as an argument to the script. For example, if the virtual machine name for the web server was VM1 and the computer name was MyWebServer, you would type $(InternalComputerName_VM1) as the argument for your script and this would pass the value MyWebServer to your script.
• $(ComputerName_ <VM Name>): This is the fully qualified domain name of the virtual machine. This can be used to access the computer even from outside the virtual environment. You might want to pass this as an argument to set up a web server. For example, if the virtual machine name for the Web server was VM1, you would type $(ComputerName_VM1) as the argument for your script to pass the fully qualified domain name of the virtual machine.

In the example in Figure 27-16 we locate the deployment script on the drop folder of the build. We also pass in the build location (drop folder) as an argument to the deployment script so the script can find the application to deploy.

The following simple deployment script is sufficient to deploy the built application to the app server.

REM ---
REM --- Map arguments passed from build
REM ---
SET DropPath = %1
SET DeploymentPath = c:ExpenseReporting
 
REM ---
REM --- Deploy application
REM ---
RD %DeploymentPath% /S /Q
XCOPY %DropPath%\_PublishedWebsitesMvcMusicStore %DeploymentPath% /S /Y /I

Adding Tests

Finally we add the tests we want to run as part of the BVT workflow (see Figure 27-16). The tests to run need to be defined in a test plan in Microsoft Test Manager and we can then filter out tests to run by test suite and configuration, for instance to specify that for this BVT workflow we will only run Windows 8 tests. Chapter 21 contains information on how to create and manage a test plan in MTM.

Lab Activities

Just as for normal builds there is a set of built-in lab activities to support the LabDefaultTemplate, which we can take advantage of when we want to customize or extend a lab workflow.

Table 27-1 shows some of the built-in lab workflow activities, for a complete list see http://msdn.microsoft.com/en-us/library/ff934562(v=vs.110).aspx.

Lab Activities in the TFS Build Extensions

In Chapter 26 we introduced the TFS Build Extensions, a community driven project for TFS build extensibility (http://tfsbuildextensions.codeplex.com). The project contains several build activities for lab management workflows as well; some of them are listed in Table 27-2. Chapter 26 describes what is needed to integrate the TFS Build Extension activities into a custom build workflow.

In the next section, we will look at how we can use the TFS Build Extension.

Adding Support to Start and Shut Down the Environment

As an example we will look at how to create a customized lab workflow to implement the logic for starting the environment (if not running), running the deployment as usual, and then shutting it down (if we started it). We will use activities from the TFS Build Extension project as building blocks.

We customize the lab workflow by making a copy of the LabDefaultTemplate11.1.xaml workflow file.

Custom Lab Workflow Parameters

When we created the BDT workflow earlier in this chapter we saw how there are a few predefined macros we can use to pass information to the deployment scripts. But if we would like to create a custom workflow argument and be able to pass that into the deployment script, the default template will not allow us because the known macros are validated when we add the deployment command.

We can solve this problem in a couple of ways

• Replace the Lab Workflow Wizard dialog. This gives us the power to do whatever we want to configure the process but it is quite some work to implement the code in the wizard.
• Extend the lab workflow with custom logic to handle a custom argument.

Because the second option seems to require less work let us take a look at what this could look like.

The customized deployment template extends the default lab template by allowing us to provide custom build process parameters using $[] instead of $() to reference the arguments in the Lab Workflow wizard. The custom deployment arguments are replaced before the execution of the deployment command. Figure 27-20 shows how we can use the custom parameter in the deployment script arguments.

The custom deployment parameters are added to the build definition as name-value pairs as shown in Figure 27-21.

To implement this customization, we can add the workflow logic before the Lab Agent scope, where the deployment scripts are run (see Figure 27-22).

In the Initialize Custom Deployment Arguments section each custom argument is split on the “=” sign. The argument name is then replaced with its value in the command string defined in the lab process wizard (see Figure 27-23).