Lesson 12.01: Running Unit Tests with Pipelines

It is important to validate the code of your application with unit tests to ensure the code is working as expected. Unit tests are implemented to test the code that is written by the developers using the test-driven development approaches (TDD) and behavioral driven development (BDD).

Build pipelines can be used to execute unit tests written with many types of unit testing tools. The general practice is to build the code and then execute the unit tests and package the code as deployable binaries. To execute the test with build pipelines, you can find several tasks available. See Figure 12-1.

Using a Visual Studio Test task, you can execute multiple types of tests developed with test frameworks such as MSTest, xUnit, NUnit, etc. The task allows you to distribute your test across multiple agents to run them efficiently, while using a multiple agents configuration in the agent job.

There are other tasks such as the dotnet command-line task, allowing you to run a dotnet test to execute the tests developed with .NET Core. The command-line tasks can be used to execute tasks such as machine learning code unit test runs. See Figure 12-2.

When tests are run from the VS Test task or dotnet test, the results of the test run get published to the pipeline automatically. However, if you are executing tests using command lines, such as Python tests, the output of the test result may need to be published to the pipeline so that the results are shown in the test tab of the executed build. To publish tests results, you can use the Publish Test Results task and specify the result output file and the format of test results of the test execution so that it will be published to the pipeline. See Figure 12-3.

In this lesson, we discussed how we can utilize tasks available for pipelines to set up unit test execution with build pipelines.

Lesson 12.02: Running Functional Tests with Pipelines

Integration and functional tests are used to test deployed applications. They might be implemented as API-level testing or User Interface testing using different test frameworks. Selenium is one of the main test frameworks widely used with .Net and other languages. There are other test frameworks such as Cypress, which is also facilitating scenario-based functional UI test implementation.

The implemented automated functional testing to test an application should be executed after a deployment to a given target to make sure the target application is working as expected. Automation and running the automated functional tests with deployment pipelines enable teams to run most of their tests, for each release, so that higher quality in delivered applications can be achieved.

A functional test based on Selenium requires you to run the agent in interactive mode to execute the testing. We talked about setting up agents in Chapter 3 of this book. However, Cypress framework UI tests can be executed with non-interactive agents as well.

To execute Selenium-based tests, you can utilize an agent setup in a virtual or physical machine as an interactive agent. Such an agent should be prepared for test execution with the Visual Studio Test Platform Installer task. See Figure 12-4.

Then you can use the VS Test task to execute the tests written with Selenium using C#. If you have developed the tests in Java, you can use Maven tasks to execute the tests in the release pipelines.

The test execution may require you to use virtual machines as test clients, especially when you need to run an interactive agent. You may need to set up multiple test client machines to efficiently distribute and run your functional tests. However, keeping these virtual machines continuously running in a cloud platform such as Azure would be costly. To minimize the costs, you can set up to an on-demand start for the machines, just before test execution; and once the test execution is done, shut down the test clients so the charges on the cloud platform will be minimized. See Figure 12-5.

We discussed the usage of release pipelines to execute functional tests in this lesson.

Summary

In this chapter, we discussed the usage of unit tests and functional test execution to ensure quality of the delivered applications.

We talked about several important facts in regard to Azure pipelines in this book. We have gone through why we need to implement continuous integration and delivery with the introductory chapter. Then we explored how the agent pools and agents and even the deployment setup. Next, we discussed classic build pipelines in detail in the next couple of chapters. The YAML pipelines were also discussed to enable you to understand the usage of pipelines as code. In addition to that, we discussed release pipelines usage and implementation and usage of test execution in release and build pipelines. Overall, the book has given you essential insights into implementing and using Azure pipelines, to enable you to deploy software applications with a rapid cadence while not compromising the quality of the applications delivered.