56.4. Generic Tests

Not every kind of test is covered in Team System. This is why Microsoft included the concept of Generic Tests so that you can easily use custom tests, but still be able to use the rest of the features like Test Results, Test List, Assign Work Items, and Publish Test Results.

To configure a Generic Test you need to specify an existing program and optionally specify its command-line arguments, additional files to deploy, and environment variables. The external application can communicate the test result back to Team System in two ways. One is with the Error Level, where a value of 0 indicates success and anything else is considered a failure. The other is to return an XML file that conforms to the SummaryResult.xsd schema located in Visual Studio's installation path. In MSDN you can find information about this schema and how to report detailed errors using XML.