Flow of a Build on a Feature Branch

Before we discuss the steps of a private build or a CI build, let’s look at it from a high level. When you start work on a user story or software change, regardless of branching strategy, you will create a branch. Remember in Chapter 4, you learned that even those using trunk-based development still use short-lived branches. Figure 6-1 shows the flow of build activities that happen when you are working on a feature branch.

When you change code, you will run your private build at every stopping point. This keeps you safe. You will learn right away if you accidentally broke something. Because you at working in Git, a decentralized version control system, you’ll make many, short commits. This enables you to undo changes very easily. Based on your judgment, you’ll run the private build locally. In our application, it is a PowerShell script and is described in more detail later in this chapter. When you decide to push changes to your team’s Git server, the CI build will detect those changes and run the integration build process on the team’s build server. Upon success, the build will archive the built artifacts, most likely in Azure Artifacts, a NuGet repository. Then an automated deployment script will trigger and deploy those built artifacts to an environment dedicated to the continuous integration process. The best name for this environment is the “TDD environment.” The purpose of this environment is to validate that (1) the new version of the software can be deployed and (2) the new version of the software still passes all its acceptance tests. This does require that you have full-system acceptance tests in your code base. If you don’t, they are easy to start developing. After the acceptance tests succeed and you determine your changes are complete, you, as the developer, will create a pull request so that your team knows that you believe the work on your branch is complete and that the code is ready to be inspected for inclusion in the master branch.

Flow of a Build on the Master Branch

Once a pull request has been approved, your branch is automatically merged into master. This is true whether you are using GitHub or Azure Repos. The CI build, which is monitoring for changes, will initiate. Upon success, the build artifacts will be stored in Azure Artifacts as NuGet packages. Then the build will be deployed to the TDD environment for validation of deployability and for the running of the automated full-system acceptance tests. Once these acceptance tests complete successfully, the build is considered a valid release candidate. That is, it is a numbered candidate for potential release and can be validated further in manual testing environments (or even additional automated testing environments) and deployed along the pipeline toward production. Figure 6-2 shows the life cycle of a master branch build.

The deployable package for a software build can be as simple as a zip file, but in .NET, the NuGet package is the standard, and these are meant to be archived in Azure Artifacts.

Steps of a Build

Before we walk through how to configure a build on your own workstation and in Azure Pipelines, let’s review the steps a private build and a CI build must have.

Your implementation of a private build and a CI build can vary from the examples shown in this book but take care to include each of the preceding steps in a fashion that is suitable for your application. Now that you know the structure of the builds, let’s cover how to configure and run them in a .NET environment.

Enabling Continuous Delivery’s Commit Stage

Before you configure Azure Pipelines, you must have your private build. Attempting to create a CI build without this foundation is a recipe for lost time and later rework. In the source code that accompanies this book, you will find a PowerShell build script named “./build.ps1”. The full listing for this file is at the end of this chapter. Feel free to use it as a build script for your own .NET Core applications. It contains all the necessary steps narrated earlier and will serve as a good jump start for your CI build. This build scripts contains steps to restore, compile, create a local database, and run tests. The first time you clone the repository, you’ll see quite a bit of NuGet restore activity that you won’t see on subsequent builds because these packages are cached. Figure 6-4 shows the dotnet.exe restore output that you’ll only see the first time after clicking click_to_build.bat.

Click_to_build.bat is a simple helper file that makes running a private build easy and convenient by adding a “& pause” so that the command window remains open when invoked by the keyboard or mouse from Windows Explorer. In the normal course of development, you’ll run the private build repeatedly to make sure that every change you’ve made is a solid, stable step forward. You’ll be using a local SQL Server instance, and the build script will destroy and recreate your local database every time you run the script. Unit tests will run against your code. Component-level integration tests will ensure that the database schema and ORM configuration work in unison to persist and hydrate objects in your domain model. Figure 6-5 shows the full build script executive with “quiet” verbosity level enabled.

This is a simple private build script, but it scales with you no matter how much code you add to the solution and how many tests you add to these test suites. In fact, this build script doesn’t have to change even as you add table after table to your SQL Server database. This build script pattern has been tested thoroughly over the last 13 years across multiple teams, hundreds of clients, and a build server journey from CruiseControl.NET to Jenkins to Bamboo to TeamCity to VSTS to Azure Pipelines. Although parts and bits might change a little, use this build script to model your own. The structure is proven.

Now that you have your foundational build script, you’re ready to create your Azure Pipeline CI build. As an overview, Figure 6-6 shows the steps you use, including pushing your release candidate packages to Azure Artifacts.

Many of the defaults are suitable for CI builds and don’t have to be customized. Let’s go through the parts that are important. First, you’ll choose your agent pool. I’ve chosen hosted agent for Visual Studio 2019. For the purposes of illustration, I’m using the build designer rather than the YAML option. All the builds and release definitions in Azure Pipelines are being converted to YAML. At the time of this writing, the YAML tooling, editor, and marketplace integration were not yet deployed. Because of this, the designer provides the full editing experience. Expect the YAML experience to enhanced quickly. When it is fully complete, you’ll be able to save your CI build configuration as a YAML file in your Git repository right next to your application. You will want to do this because any logic not versioned with your code could break your pipeline since it is inherently not compatible with branching given that only one version of the build configuration exists.

Even though there is only one explicit parameter in the preceding command, all the build variables are available to any script as environment variables. Figure 6-7 shows the variables that are configured for this build.

We are using NUnit as our automated testing framework for this application. Notice that we hard-code very little in formulating our commands. This is to make our build script more maintainable. It can also be standardized someone across our teams and other applications given that the variances occur in the properties at the top of the file. Pay special attention to the arguments –no-restore and –no-build. By default, any call to dotnet.exe will recompile your code and perform a NuGet restore. You do not want to do this, as it is precious time wasted and creates new assemblies just before they are tested.

After the build script finishes, we can run our static analysis tools and then push the application with its various components to Azure Artifacts as ∗.nupkg files, which are essentially ∗.zip files with some specific differences.

Additionally, as you enable continuous integration, you’re asked what branches should be watched. The default is the master branch, but you’ll want to change that to any branch. As you create a branch to develop a backlog item or user story, you’ll want commits on that branch to initiate the pipeline as well. A successful build, deployment, and the full battery of automated tests will give you the confidence that it’s time to put in your pull request. This setting is tricky and not obvious. As you click in the “Branch specification,” you’ll type an asterisk (∗) and hit the Enter key. Figure 6-8 shows what you should see.

Once your CI build is up and running, add the Build History widget shown in Figure 6-9 to your project dashboard.

Notice that the build time is over 4 minutes. This is a simple application, but your build time is already up to 4 minutes and 38 seconds. Yet, your private build runs in about 1 minute locally. This is because of the hosted build agent architecture. As soon as you have your build stable, you’ll want to start tuning it. One of the first performance optimizations you can make is to attach your own build agent so that you can control the processing power as well as the levels of caching you’d like your build environment to use. Although hosted build agents will certainly improve over time, you must use private build agents in order to achieve the short cycle time necessary to move quickly. And the 3 minutes overhead you incur at the time of this writing for hosted agents is not what you want for short cycle times across your team.

At the time of this writing, internal Microsoft teams use private build agents in order to achieve the performance and control necessary for complex projects. Use the hosted agents to stabilize new build configurations. Then measure and tune them to decide if you need to provision your own private agents.