8 Complex deployments using GitHub Actions
- Deploying the Toma Toe (chapter 7) case using GitHub Actions
- Connecting to Azure from GitHub Actions
- Handling secrets when using GitHub Actions
- Monitoring workflows while they are running
Git is currently one of the most popular source control systems around, and the GitHub platform (https://github.com), originally built around Git, is one of the most popular platforms for storing (code) projects. Throughout the years, GitHub has grown to a more mature application lifecycle management platform, including an integrated automation system called GitHub Actions.
GitHub Actions is the equivalent of an Azure DevOps pipeline that allows you to hook into events in your source control to implement CI/CD solutions in GitHub. Although both GitHub Actions and Azure Pipelines have a lot in common, there are some differences between the two. This chapter will touch upon those differences and will use the same Toma Toe use case as chapter 7. The same deployment process will be created, but this time using GitHub Actions.
GitHub accounts are free, and you can create one right here: https://github.com/signup. With this free account you can create an unlimited number of Git repositories, but there are some limitations. For example, the number of automation minutes is limited, as is the amount of storage.
8.1 Forking a repository
Forking is a way to copy an existing repository into a new one, while keeping references to each other. For this chapter, we created a GitHub repository that you can fork, to get you up and running quickly. Once you have logged in to GitHub, navigate to https://github.com/AzureIaCBook/github-actions-start and click the Fork button at the top right of the screen (see figure 8.1). Once it has completed, you’ll have a copy of our repository that you can start with.
The way GitHub Actions is structured is slightly different from the way Azure DevOps is structured. Before you build the complete GitHub Actions workflow, let’s look at the terminology used in GitHub Actions.
8.2 Getting to know GitHub Actions
An automation procedure in GitHub is called a workflow. Workflows are part of a repository and contain instructions for what the workflow must do. A workflow is triggered by an event and typically stops when an error occurs, when the workflow completes, or when the workflow is cancelled. You can look back to chapter 7 to compare the Azure DevOps pipelines described there to the GitHub Actions workflows here.
Figure 8.2 shows how GitHub Actions are structured. An event triggers a workflow containing jobs, resulting in runners executing actions. In the remainder of this section, these terms are explained in more detail.
8.2.1 Workflow events
Workflows are triggered by an event. Like in Azure DevOps, an event can be an activity in the repository, such as pushing a commit or creating a pull request. It is also possible to trigger a workflow with a time schedule. You can even use a GitHub webhook to trigger a workflow.
8.2.2 Runners
GitHub Actions uses runners to execute sections of your workflow. Runners can be compared to agents in Azure DevOps. You can use runners hosted online by GitHub or host runners yourself. When your workflow starts, GitHub actions will reserve an idle runner and execute a job. Once the job completes (or fails), the runner will be disposed of. If your workflow has additional jobs defined, GitHub Actions will find a new idle runner, and start the new job on that new runner (as illustrated in figure 8.2). This process continues until your workflow completes or fails.
8.2.3 Jobs
GitHub has no implementation of something similar to a stage in Azure DevOps. Instead, a job is the largest possible entity in GitHub Actions that allows you to structure your workflow. Jobs run in parallel by default, but you can create dependencies between jobs to make them run sequentially.
8.2.4 Steps
A step can be referred to as an individual task. Tasks organized under the same job are executed on the same runner. This means that when steps are organized under the same job, they can share data with each other. You will learn how to share data between different jobs in section 8.3.1.
8.2.5 Actions
Actions are single commands typically structured in steps. There are tons of actions that can be used. You can find actions created by the GitHub community in GitHub Marketplace (https://github.com/marketplace?type=actions). If your desired action is not available, you can create an action yourself. There is a large GitHub community creating custom actions that you can also use.
We’ve touched upon the anatomy of a GitHub workflow, so now it’s time to write a workflow to deploy the Toma Toe infrastructure. As you’ll recall from the previous chapter, the Toma Toe infrastructure deploys a web application in three different Azure regions with an Azure Traffic Manager to route the traffic to the nearest location, based on the geographical location of the website visitor.
8.3 Building a GitHub Actions workflow
Like in Azure DevOps, a GitHub workflow is nothing more than a definition written in a YAML file, stored in your code repository. For GitHub, workflows are stored in a specific folder (.github/workflows). This means that you can create a new workflow by simply adding a YAML file to that specific folder.
As an alternative, you can create a workflow from the GitHub website. Once you’re logged in, navigate to your repository and click the Actions menu item at the top of the screen. GitHub has some predefined workflows that you can choose from that may fit the needs of your code project.
There is no predefined workflow for the purpose of this chapter: transpiling a Bicep file and deploying the product (an ARM template) to the Azure cloud, so we will create a workflow file from scratch.
You can start by creating a new file called .github/workflows/toma-toe.yml and adding the following content:
The first line names your workflow, and the second line hooks into the push event on your repository. This means that your workflow will now start as soon as one or more commits are pushed to your repository.
You can commit and push these changes to your repository and then view your repository online. When you now navigate to the Actions tab of your repository, you’ll see that it contains one action. The workflow will fail, however, because it doesn’t contain a job. Let’s address that right away.
8.3.1 Adding a job to a GitHub Actions workflow
Much like the Azure DevOps pipeline you built in chapter 7, this workflow will face the challenge of not being able to share data between jobs. This is because each job runs on a different runner. Luckily, GitHub Actions allows you to publish artifacts just like Azure DevOps pipelines can. Doing so allows you to store data from one job and use it in a subsequent job.
The first job in your workflow transpiles the two Bicep files in the repository to an ARM template and publishes these ARM template JSON files as an artifact, as shown in the following listing (.github/ workflows/toma-toe.yml).
Listing 8.1 Adding the transpile job
name: Toma Toe Workflow on: [push] jobs: publish-bicep: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Compile infrastructure uses: Azure/[email protected] with: ❶ azcliversion: 2.23.0 inlineScript: az bicep build --file ./deployment/main.bicep - name: Compile infrastructure uses: Azure/[email protected] with: ❶ azcliversion: 2.23.0 inlineScript: az bicep build --file ./deployment/trafficmgr.bicep - name: Publish Artifact uses: actions/upload-artifact@v2 with: name: bicep-templates path: deployment/*.json
❶ The “with” can be compared to “inputs” in Azure DevOps. It allows you to pass in parameters.
The preceding code shows the jobs declaration in the workflow. This indicates that from here on, the workflow contains jobs definitions. Jobs are defined by an identifier followed by a colon. In the preceding example, publish-bicep is the identifier of a new job. The runs-on allows you to indicate what type of operating system the runner must have and what software the runner needs to have installed. Then, in the steps property, four steps are defined. The first step checks out the branch the workflow runs on, to get all its contents. Then the az bicep build command is used to transpile the main.bicep file, followed by a similar command to transpile the trafficmgr.bicep file. Finally, the produced JSON files are published as an artifact.
When you commit and publish your changes, the workflow will immediately start again, and this time it will succeed and publish the JSON files as an artifact. If you look at the summary of your workflow, you can find these artifacts and even download them if you want to. Now let’s move on to the deployment sections of GitHub Actions.
8.4 The deployment phase in GitHub Actions
In the previous section, the Bicep templates were transpiled into ARM templates and published as an artifact. In this section, we’re going to fetch this artifact in later stages and deploy them to Azure.
Unlike Azure DevOps, GitHub does not have a mechanism like service connections, but a connection to Azure is mandatory to deploy infrastructure in an Azure environment. Also, although there are third-party tasks that can help you to deploy your template, a straightforward way is to use Azure CLI commands. In this section, you’ll learn how to overcome the problem of connecting to Azure and deploy your template from GitHub Actions.
8.4.1 Connecting to Azure from your GitHub workflow
To deploy your infrastructure, you’ll use the Azure CLI to allow a connection between the GitHub Actions runner and the Azure Cloud environment. Instead of using a service connection like in Azure DevOps, you can use a couple of Azure CLI commands to connect to Azure and select the correct Azure subscription. To do this, you need to generate connection information that you’ll store as a secret in your GitHub repository. Note that the secrets are stored as a property of your GitHub repository and are not part of your source code. These secrets can then be used to connect to Azure and select the correct subscription.
To perform these steps, you need to install Azure CLI if it isn’t installed already. You can find download and installation instructions here: https://docs.microsoft.com/cli/azure/install-azure-cli. Also, you must have access to an Azure subscription and have enough privileges (have the Owner role or the User Access Administrator role assigned) to create a service principal. This service principal will be used to connect to the Azure environment from your GitHub workflow.
8.4.2 Generating a service principal using the Azure CLI
To generate a service principal, open a command prompt (or the Windows Terminal) and type the following:
az login az account set –-subscription {subscription-id} az ad sp create-for-rbac --name {app-name} --role contributor –-scopes ➥ /subscriptions/{subscription-id}
The first command opens a browser that allows you to log in to Azure. The second command allows you to select the correct Azure subscription—you’ll need to replace the placeholder with your subscription ID. The last command creates a new service principal in Azure Active Directory with the name you entered in {app-name}, and it attaches the contributor role to your service principal at the subscription level.
After the command is executed, it will output a JSON object that looks like the following:
{ "appId": "guid", "displayName": "Name", "name": "http:/ /Name", "password": "secret password", "tenant": "guid" }
This JSON object must be stored as a secret in your GitHub repository. On the GitHub website, navigate to your repository and choose the Settings tab. Then, in the menu, choose Secrets, and click the New Repository Secret button to create a new secret. Give your secret a name, and paste the JSON object in the Value field of the secret. Click the Add Secret button to store the secret in your repository.
The GitHub workflow created in this chapter uses two secrets, AZURE_TEST and AZURE_PROD. One connects to a test subscription and the other connects to a production subscription. If you want to test this workflow but you only have one subscription available, you can create the two secrets and paste the same JSON value in for both.
Figure 8.3 shows the settings page of a GitHub repository. On the left you’ll see the menu where Secrets is selected. In the top-right corner you’ll see the button for adding secrets to your repository. At the bottom, you’ll see that two secrets have been added to this repository, AZURE_TEST and AZURE_PROD, both keeping connection secrets to Azure environments.
Now that the connection information is securely stored in your GitHub repository, let’s adjust the GitHub workflow created earlier in this chapter so it deploys the templates.
8.5 Deploying ARM templates from GitHub Actions
The deployment of the ARM template will be executed twice, one to a test and one to a production environment. GitHub Actions allows you to write workflow definitions that you can reuse in other workflows. This deployment task is a very good subject for reuse, because it is fairly large and requires only two input parameters. Separating the deployment part of this workflow into a different workflow will make the deployment task reusable and the main workflow easier to read and maintain.
As mentioned previously, there are two input parameters: one is for the environment the deployment targets (test or production), and the other is a secret, being the connection information for the correct Azure subscription. Let’s create a new workflow file and define these parameters, as shown in the following listing (.github/ workflows/deployment.yml).
Listing 8.2 Adding a reusable workflow
name: Deployment workflow on: workflow_call: inputs: ❶ environment: required: true type: string secrets: ❷ azure-login: required: true
❶ Normal input parameters are defined in the inputs section.
❷ Input parameters containing secret data are defined in the secrets section.
The preceding listing shows the new workflow file. You can see that the two input parameters you require for your workflow are separated into two different sections. One is for normal input parameters, while the secrets containing sensitive data have their own section.
Let’s continue working on this deployment workflow file and add the following steps to deploy the Bicep files to Azure:
-
Download the artifact. This will make the main.json, the trafficmgr.json, and the parameter files available.
According to the Toma Toe case introduced in chapter 7, the same template must be deployed three times in different Azure regions. The fourth deployment will provision the Traffic Manager on top of the App Services deployed in the preceding three deployments, as shown in the following listing (.github/workflows/deployment.yml continued).
Listing 8.3 Adding the deployment to test
jobs: infrastructure-incremental-test: runs-on: ubuntu-latest steps: - name: Download Artifact ❶ uses: actions/download-artifact@v2 with: name: bicep-templates path: ./infrastructure - name: Azure Login ❷ uses: azure/login@v1 with: creds: ${{ secrets.azure-login }} - name: Deploy Europe ❸ uses: Azure/[email protected] with: azcliversion: 2.23.0 inlineScript: ➥ az deployment sub create --name europe --location westeurope ➥ --template-file ./infrastructure/main.json –-parameters ➥ ./infrastructure/${{ inputs.environment }}.parameters.json ➥ --parameters locationAbbreviation=we - name: Deploy United States uses: Azure/[email protected] with: azcliversion: 2.23.0 inlineScript: ➥ az deployment sub create --name america --location eastus ➥ --template-file ./infrastructure/main.json –-parameters ➥ ./infrastructure/${{ inputs.environment }}.parameters.json ➥ --parameters locationAbbreviation=us - name: Deploy Asia uses: Azure/[email protected] with: azcliversion: 2.23.0 inlineScript: ➥ az deployment sub create --name asia --location eastasia ➥ --template-file ./infrastructure/main.json --parameters ➥ ./infrastructure/${{ inputs.environment }}.parameters.json ➥ --parameters locationAbbreviation=asi - name: Traffic Manager ❹ uses: Azure/[email protected] with: azcliversion: 2.23.0 inlineScript: az deployment sub create --location westeurope ➥ --template-file ./infrastructure/trafficmgr.json ➥ --parameters ./infrastructure/${{ inputs.environment }}.parameters.json
❶ Downloading the workflow artifact
❷ Logging in to Azure and selecting the correct subscription
❸ First deployment of the app service in the West Europe region
❹ Deployment of the Traffic Manager
The preceding code defines a deployment to Azure. Note that the deployments to the Western Europe region, East US region, and Asia region are similar except for their parameters. Also, because all the deployments are using the same subscription, a name parameter is added to prevent creating a deployment with the same name in a different region. Not adding the name results in three deployments with the same name in different regions and will result in an error. You can see the use of the secret input parameter in the Azure Login action. The environment input parameter is used in every deployment step to determine which parameter file to pass in.
Now that the deployment steps are isolated in a separate workflow, you can call this workflow from within the toma-toe workflow, completing the GitHub Actions workflow for the Toma Toe case.
8.5.1 Completing the deployment
The deployment workflow in the previous section must be called twice: once to create a test environment, and once to create a production environment. To call a reusable workflow, you must know the owner, GitHub repository, path, and filename of the workflow. Also, you need to specify a version of the workflow to use. You can use the ID of a commit or a tag to identify the version of a workflow. All this information is combined into one line that specifies the workflow to call:
To complete the workflow for this chapter, you need to add two jobs for a test and production deployment. The complete template is shown in the following listing.
Listing 8.4 The complete GitHub Actions workflow
name: Toma Toe Workflow on: [push] jobs: publish-bicep: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Compile infrastructure uses: Azure/[email protected] with: azcliversion: 2.23.0 inlineScript: az bicep build --file ./deployment/main.bicep - name: Compile infrastructure uses: Azure/[email protected] with: azcliversion: 2.23.0 inlineScript: az bicep build --file ./deployment/trafficmgr.bicep - name: Publish Artifact uses: actions/upload-artifact@v2 with: name: bicep-templates path: deployment/*.json infrastructure-incremental-test: ❶ needs: publish-bicep ❷ uses: {owner}/{repository}/.github/workflows/deployment.yml@version ❸ with: environment: test secrets: azure-login: ${{ secrets.AZURE_TEST }} infrastructure-incremental-prod: ❹ needs: infrastructure-incremental-test uses: {owner}/{repository}/.github/workflows/deployment.yml@version with: environment: prod secrets: azure-login: ${{ secrets.AZURE_PROD }}
❶ Calling the deployment workflow with parameters for the test environment
❷ The “needs” property allows you to configure a dependency. This job depends on the publish-bicep job.
❸ The path to this workflow must be adjusted for your environment.
❹ Calling the deployment workflow with parameters for the production environment
The preceding code is the complete definition of the GitHub Actions workflow that deploys the Toma Toe infrastructure to Azure. Note that the reference to the external workflow will not work as is; it must be adjusted so it contains your GitHub account name, repository name, path, and version number (or commit ID). With the needs property, jobs are configured to depend on each other. This means that the job that deploys to test will only execute when the publish-bicep job has completed successfully.
When you commit and push this workflow and navigate to your GitHub repository, you can view the workflow in the Actions tab. Figure 8.4 shows a visual representation of the last run of the workflow in Azure. The status indicates that the entire workflow ran successfully. In the middle, you’ll see the three jobs: build, deploy to test, and deploy to production. At the bottom, you’ll see the artifacts published by the workflow—in this case, there’s only one. You can download the artifact to review its content if you need to.
Summary
-
GitHub Actions is the equivalent of Azure Pipelines. It allows you to hook into events in your source control to implement CI/CD solutions in GitHub.
-
A GitHub workflow is a definition written in YAML and stored in a specific folder in your code repository.
-
GitHub Actions allows you to publish artifacts so you can store data from one job and use that data in a subsequent job.
-
Connecting to Azure from your GitHub workflow requires generating connection information and saving it as a secret in the GitHub repository. In this chapter, these secrets are used to connect to Azure and select the correct subscription.
-
GitHub Actions workflows allow for reusability. To call a reusable workflow, you must know the owner, GitHub repository, path, and filename of the workflow.