The /testcontainer option requires the filename as parameter which contains information about tests that must be run. The /testcontainer file is an assembly that contains all the tests under the project, and each of the projects under a solution has its own container for the tests within the projects.

For example, the next screenshot shows the list of tests within the container unittestproject1.dll. MSTest executes all the tests within the container and shows the result as well. The summary of the Test Result is as shown in the next screenshot:

First, the MSTest will load all the tests within the project, then start executing them one by one. The result of each Test Run is shown but the detailed Test Run information is stored in the test trace file. The trace file can be loaded in Visual Studio to get the details of the Test Result.

The /testmetadata option is used for running tests in multiple Test Projects under a solution. This is based on the metadata file, which is an XML file that has the list of all the tests created under the solution.

The /testcontainer option is specific to a Test Project, whereas /testmetadata is for multiple test containers with the flexibility of choosing tests from each container.

There are instances where running all the tests within a test container is not required. To specify only the required tests, use the /test option with the /testmetadata option or the /testcontainer option. For example, the following command runs only the CodedUITest1 test from the list of all tests:

The /test option can be used along with /testmetadata or /testcontainer, but not both. There are different usages for the /test option:

And if the command contains the option /test:UnitTestProject1, then all of the preceding three tests will run as the name contains the UnitTestProject1 string in it. Even though we specify only the name to the /test option, the result will display the fully qualified name of the tests run in the results window.

The /unique option will make sure that only one test which matches the given name, is run. In the preceding examples, there are different tests with the string UnitTestProject1 in its fully qualified name. Running the following command executes all the preceding tests:

mstest /testcontainer:c:SatheeshSharedAppsEmployeeMaintenanceUnitTestProject1indebugunittestproject1.dll /test:Unittestproject1

But if the /unique option is specified along with the preceding command, the MSTest utility will return the message saying that more than one test was found with the same name. It means that the test will be successful only if the test name is unique.

The following command will execute successfully as there is only one test with the name GetTotalItemPriceTest.

The /noisolation option runs the tests within the MStest.exe process. This choice improves the Test Run speed, but increases risk to the MSTest.exe process.

Usually, the tests are run in a separate process that is allocated with separate memory from the system. By launching the MSTest.exe process with the /noisolation option, we avoid having a separate process created for the test.

The /testsettings option is used to specify the Test Run to use a specific test settings file. If the settings file is not specified, MSTest uses the default settings file. The following example forces the test to use the TestSettings1 settings file:

In all the command executions, the MSTest utility stores the Test Results to a trace file. By default, the trace file name is assigned by MSTest using the login user ID, the machine name, and the current date and time. This can be customized to store the Test Results in a custom trace file using the /resultsfile option. For example, the next screenshot shows the custom trace file named as customtestresults.trx:

The preceding screenshot shows the Test Results stored at the c:Satheesh location in the results file, customtestresult.trx.

The /noresults option informs the MSTest application not to store the Test Results to the TRX file. This option increases the performance of the test execution.

The /nologo option is to inform the MSTest tool not to display the copyright information that is usually shown at the beginning of the Test Run.

The /detail option is used for collecting the property values from each Test Run result. Each Test Result provides information about the test such as error messages, start time, end time, test name, description, test type, and many more. The /detail option is useful to get the property values after the Test Run. For example, the following screenshot shows the start and end time of the Test Run, and also the type of the Test Run:

The /detail option can be specified multiple times to get multiple property values after the Test Run.

The /publish option should be followed by the uniform resource identifier (URI) of the TFS, if the TFS is not registered in the client. If it is registered, just use the name of the server to which the Test Result has to be published, as shown in the following command:

/publish:[server name]

Refer to the following examples:

The /publishbuild option is used for publishing the builds. The parameter value is the unique name that identifies the build from the list of scheduled builds.

Publishing the Test Rresults to TFS requires /flavor as mandatory. Flavor is a string value that is used in combination with the platform name, and should match with the completed build that can be identified by the /publishbuild option. The MSTest command will run the test, and then set the flavor and platform properties, before publishing the Test Run results to the TFS:

/flavour:[flavour string value]

For example:

This is a mandatory string value used in combination with the /flavor option which should match the build option.

/platform:[string value]

For example:

MSTest stores all the Test Results in the default trace files with the extension .trx. Using the /publishresultsfile option, the Test Results file can be published to TFS using the output/trace option. The name of the file is the input to this option. If the value is not specified, MSTest will publish the current Test Run trace file to TFS.

/publishresultsfile:[file name string]

For example, to publish the current Test Run trace file, use the /publishresultsfile option.

To publish the Test Result, one can use a combination of different options we saw in previous sections, along with the option /publishresultsfile.

The Test Results from the results file are published to the build output of the solution. The steps involved in publishing are to create the test, create a build definition, build the solution, execute the test, and then publish the result to the build output.

Step 1 – create/use existing Test Project

The following screenshot contains the solution EmployeeMaintenance. The solution contains a Test Project WebAndLoadTestProject1 with a web test WebTest2. The following screenshot shows the Test Project named WebAndLoadTestProject1:

Step 2 – running the test

On running the web test, by default the Test Result is stored in the trace file <file name>.trx.

Step 3 – creating a build

The /build service in Team Foundation Server has to be configured with a controller and agents. Each build controller manages a set of build agents. Unfortunately, the steps and the details behind creating the build types will not be covered in this book as it would be too long to discuss it. The following screenshot shows the /build service configured with controller and agents:

To create the build definition using the Team Explorer, navigate to the Build definitions in Builds folder, under Team Project. Select new build definition, and then configure the options by choosing the projects in TFS and the local folder. In one of the steps, you can see the following screenshot for selecting the project and setting the configuration information for the build process:

There are different configuration sections such as Required, Basic, and Advanced, from where the project can be selected to include as part of this build definition setting such as build file formats, Agents Settings, work item creation on build failure, and other configurations.

Step 5 – publishing the result

So far, the test is run and the result is saved in the trace file, and also we have built the project using the build definition. The Test Run results should be published to the build. There are multiple options used for publishing the Test Results using the MSTest command line tool. The following command in the next screenshot publishes the Test Result to the specified build:

The command line options used in the preceding screenshot shows the Test Result trace file, TFS Team Project, and build against which the Test Result should be published. The command line also has the platform and the flavor values matching the build configurations.

After publishing the Test Results, if you open the build file, the test information along with the build summary is shown in the build summary. The information also contains a link to the trace file.

A few automated tests were created in previous chapters such as automated unit tests, but it was all through Visual Studio. There wasn't any test case for the unit test, and running the test case was also from Visual Studio IDE. This section explains how to import the tests to a Test Plan and create the test cases automatically while importing through the command line.

The Test Plans are created using the Test Manager to group the Test Suites and test cases. The following screenshot shows a few Test Plans created for the Team Project TeamProject1:

The EmployeeMaintenance solution contains the unit Test Project UnitTestProject1 with a few methods out of which there are methods such as CalculateTotalPriceTest() and CalculateTotalPricewithTaxTest() with their category defined as TotalPrice. So far there are no test cases defined in any of the Test Plans in the Test Manger for these tests. Refer to the following screenshot:

For any tests created using Visual Studio, the TCM utility can be used to import it to the Test Plan in Test Manager as test cases. The following command imports all tests with the category defined as TotalPrice from the UnitTestProject1 assembly into the Team Project TeamProject1. The category is defined to the tests to group it from all other available tests within the assembly. Refer to the following screenshot:

The command execution result shows the summary of the import, along with the names of the tests matching the command parameters.

Connect to the TeamProject1 using Test Manager and open any of the Test Plans within the project. On the Contents tab under the Plan option in Testing Center, click on Add from the toolbar in Test Suite section on the right. This will open up a new window to search for any available test cases to add to the Test Suite. By default, the Test Plan is the Test Suite, if no other Test Suite is created for the plan. In the new window, just click on the Run option to perform the default search with default parameters. You may notice that the search result shows two test cases in the name of the test methods which were imported from the Test Project. The test cases are named after the test method itself. Select either or both of the test cases and add them to the Test Suite.

After adding the test case to the Test Suite and Test Plan, open the test case using the Open toolbar option. There won't be any step except the name of the test case and few other details. Include the details of the test steps to the test case, if required.

Now the test is available, and the test case is added to the Test Suite.

The tests cases associated with the tests can be run using the TCM command line utility without using the IDE. Whenever a test is run using the TCM, it requires additional information such as the environment and roles within the environment.

Running the test case using TCM requires Test Points or the Test Suite, and the configuration information. TCM requires the IDs of the Test Plan, Test Suite, and configuration. The TCM command line can be used to retrieve all these details.

To list all configurations from the Team Project, the TCM command is like the following result:

The following is the command and output for listing all the Test Plans within the Team Project:

To list all the Test Suites within the Plan, use the following TCM command with the options as shown in the next screenshot along with the Plan ID, collection, and the Team Project name. Use the Plan ID from the previous command output:

Use the Config ID, Plan ID, and the Suite ID collected by using the TCM utility from the collection and the Team Project to run the test. This will create a run as shown in the following screenshot:

The Test Run is created and the result can be viewed in Test Manager for analysis. Open the Test Manager and select the option Test under Testing Center. Select Analyze Test Runs from the menu bar. The Analyze Test Runs window shows the Test Runs for the Test Plan. The following screenshot shows a detailed view of the Test Run. The test is still in progress but you can see the test cases and the other details provided at the command: