This chapter covers

• Designing a CI/CD pipeline as code on GCP
• Two-stage deployments for separating static and dynamic infrastructure
• Iterating over complex types with for_each expressions and dynamic blocks
• Implicit vs. explicit providers
• Creating custom resources with local-exec provisioners

CI/CD stands for continuous integration (CI) / continuous deployment (CD). It refers to the DevOps practice of enforcing automation in every step of software delivery. Teams that practice a culture of CI/CD are proven to be more agile and able to deploy code changes more quickly than teams who do not practice a culture of CI/CD. There is also the ancillary benefit of improving software quality, as faster code delivery tends to result in smaller, less risky deployments.

A CI/CD pipeline is a process that describes how code gets from version control systems through to end users. Each stage of a CI/CD pipeline performs a discreet task such as building, unit testing, and publishing application source code (see figure 7.1).

 

Figure 7.1 A CI/CD pipeline has multiple stages that automate the flow of software delivery.

In this chapter, we deploy a CI/CD pipeline as code. In other words, everything that makes up the pipeline will be deployed and managed with Terraform. We’ll use Google Cloud Platform (GCP) as our cloud of choice. GCP is the third largest of the four major clouds (AWS, Azure, GCP, and AliCloud), but it has seen by far the most growth in recent years. There’s a lot to like about Google Cloud, from its clean UI to its project-based system, to its managed Kubernetes offerings. But there are some awkward things about it as well, and we see a few examples in this chapter.

We start by covering the last few syntax and expression elements that we haven’t introduced previously. Specifically, we introduce for-each expressions, dynamic blocks, and resource provisioners. Although we saw dynamic and functional programming back in chapter 3, these new constructs enable writing much more powerful, expressive, and dynamic code than ever before.

Resource provisioners are especially interesting because they are essentially backdoors to the Terraform runtime. Provisioners can execute arbitrary code on either a local or remote machine, which has many obvious security implications, but we will wait until chapter 13 to cover this. You can use provisioners for many tricks. An example we’ll see in this chapter is creating custom resources with local-exec provisioners by attaching them to a null_resource.

Once our CI/CD pipeline is provisioned, we’ll test it by pushing some application code through it and watching as it deploys as a Docker container.

Note Docker containers are lightweight, standalone, executable packages of software that include everything needed to run an application: code, runtime, system tools, and settings.

7.1 A tale of two deployments

We’ve previously deployed applications with Terraform as part of the infrastructure provisioning process. This is convenient because the application can be deployed as part of terraform apply, but the process is much slower than it might be otherwise. Applications change frequently—far more frequently than the underlying infrastructure they are deployed onto. If you want to speed up the delivery of applications, the best way to do so is with a CI/CD pipeline.

As much as I love Terraform, it’s not well suited for managing things that change frequently, such as application source code. Generating an execution plan in Terraform is downright sluggish, especially if many resources need to be refreshed. This isn’t to say that you couldn’t use Terraform as part of a CI/CD pipeline (this is the subject of chapter 12, after all), but if your goal is to deploy applications, you shouldn’t be afraid to separate dynamic infrastructure from static infrastructure.

By dynamic infrastructure, I am referring to things that change a lot. By the same token, static infrastructure refers to things that only change a little. Why make a distinction? Well, managing static infrastructure—resources like virtual machines, load balancers, and so forth—is what Terraform is good at. Terraform is not so great at deploying applications, although there are plenty of examples of people doing exactly that, and we’ll see an example in chapter 8. By deploying your static infrastructure with Terraform, you form the foundation on which to deploy everything else.

TIP You could also use Terraform to deploy dynamic infrastructure. For example, you could have a Kubernetes cluster deployed with Terraform and then use a different Terraform workspace to deploy Helm charts onto it.

Figures 7.2 and 7.3 show a comparison between what we’ve been doing (an all-in-one deployment) and a two-stage deployment.

Figure 7.2 Redeploying an entire stack each time you want to make a change is slow.

Figure 7.3 By separating your project into what changes a lot vs. what changes a little, you can deploy application code changes more quickly.

7.2 CI/CD for Docker containers on GCP

Docker containers are an excellent way to package your code and ensure that it has all the resources and libraries required to run while still being portable across multiple environments. Because of the enormous popularity of containers, many tools and established architecture patterns exist for setting up a CI/CD pipeline. We’ll take advantage of some managed GCP services to deploy a complete CI/CD pipeline for building, unit testing, and deploying Docker containers.

7.2.1 Designing the pipeline

Knative is an abstraction layer over Kubernetes that enables running and managing serverless workloads with ease. It forms the backbone for a GCP service called Cloud Run that automatically scales, load-balances, and resolves DNS for containers. The purpose of using Cloud Run is to simplify this scenario, as it would be a bit more complex to deploy a Kubernetes cluster.

Note Cloud Run supports bringing your own compute by enabling Anthos on a Google Kubernetes Engine (GKE) cluster.

As mentioned earlier, CI/CD pipelines for containers generally involve stages for building, unit testing, publishing, and deploying application code. Preferably you would have multiple environments (e.g., dev, staging, prod), but for this scenario, we have only a single environment (prod). We will focus more on CI than CD.

In addition to Cloud Run, we’ll use the following managed GCP services to construct the pipeline:

• Cloud Source Repositories—A version-controlled Git source repository

• Cloud Build—A CI tool for testing, building, publishing, and deploying code

• Container Registry—For storing the built container images

• Cloud Run —For running serverless containers on a managed Kubernetes cluster

The pipeline we’ll build is shown in figure 7.4.

Figure 7.4 CI/CD pipeline for GCP. Commits to Cloud Source Repositories triggers a build in Cloud Build, which then publishes an image to the Container Registry and, finally, kicks off a new deployment to Cloud Run.

7.2.2 Detailed engineering

This project doesn’t have much in the way of code, but the code it does have is tricky. Three main components make up the code for the CI/CD pipeline:

• Enabling APIs—GCP requires that you explicitly enable the APIs that you wish to use.

• CI/CD pipeline—Provisions and wires up the stages for the CI/CD pipeline.

• Cloud Run service—Runs the serverless containers on GCP.

Figure 7.5 shows a dependency diagram of the resources we’ll provision.

Figure 7.5 There are four sets of components: one for enabling APIs, one for configuring Cloud Build, one for configuring IAM access, and one for configuring the Cloud Run service.

7.3 Initial workspace setup

If you do not already have credentials for GCP, you will need to acquire them. Refer to appendix C for a tutorial on this process.

7.3.1 Organizing the directory structure

This project has two parts: the part deployed with Terraform and the part not deployed with Terraform. Easy, right? Well, how do you organize code that’s related to a central project but different enough that it should still be kept separate? Monorepos, of course! This is a subject of much debate (see http://mng.bz/6Nwe), but for this situation, I think it makes sense.

We organize the project into a monorepo by creating a single project directory with two subdirectories: one for all things Terraform related (i.e., static infrastructure) and another for the application code (i.e., dynamic infrastructure). Do this now by creating a project folder, such as gcp-pipelines, with two subfolders, infrastructure and application. When you’re done with that, switch into the infrastructure folder, which will be the primary working directory. In the infrastructure folder, create a variables.tf file with the following content.

Listing 7.1 variables.tf

variable "project_id" {
  description = "The GCP project id"
  type        = string
}
 
variable "region" {
  default     = "us-central1"
  description = "GCP region"
  type        = string
}
 
variable "namespace" {
  description = "The project namespace to use for unique resource naming"
  type        = string
} 

Next, create a terraform.tfvars file. You can keep region and namespace the same if you like, but project_id should be changed to the ID of your GCP project.

Listing 7.2 terraform.tfvars

project_id ="<your_project_id>"         ❶
namespace  = "team-rocket"
region     = "us-central1" 

❶ Your GCP project id goes here.

Notice that var.namespace is team-rocket. Imagine, if you will, that this isn’t just any old pipeline but is going to be used by a group of millennial developers to deploy their hip new Pokémon-themed app. This reflects the fact that the code is reusable and, if you are an expert in CI/CD, you will always be asked to do work for other people.

Finally, we need to declare the Google provider. Create a providers.tf file with the following contents.

Listing 7.3 providers.tf

provider "google" {
  project = var.project_id
  region  = var.region
}

7.4 Dynamic configurations and provisioners

Google is highly opinionated and strict when it comes to matters of Identity and Access Management (IAM). For example, in a new project, you have to enable the services’ APIs before you can use them. I am not a fan of this approach and find it inconvenient at best and aggravating at worst. Regardless, there is a Terraform resource that can automate enabling APIs called google_project_service. This resource must be created before downstream resources. The code for enabling APIs is shown in listing 7.4.

Note There are two syntax features you haven’t seen before: for_each and local-exec. We’ll get to these in the next section.

Listing 7.4 main.tf

locals {
  services = [                                               ❶
    "sourcerepo.googleapis.com",
    "cloudbuild.googleapis.com",
    "run.googleapis.com",
    "iam.googleapis.com",
  ]
}
 
resource "google_project_service" "enabled_service" {
  for_each = toset(local.services)
  project  = var.project_id
  service  = each.key
 
  provisioner "local-exec" {                                 ❷
    command = "sleep 60"
  }
 
  provisioner "local-exec" {                                 ❸
    when    = destroy
    command = "sleep 15"
  }
}

❶ List of service APIs to enable

❷ Creation-time provisioner

❸ Destruction-time provisioner

7.4.1 for_each vs. count

The for_each meta argument accepts as input either a map or a set of strings and outputs an instance for each entry in the data structure. Although analogous to loop constructs in other programming languages, for_each does not guarantee sequential iteration (because sets and maps are inherently unordered collections). for_each is most similar to the meta argument count but has several distinct advantages:

• Intuitive -for_each is a much more natural concept, compared to iterating by index.

• Less verbose—syntactically, for_each is shorter and more pleasing to the eye.

• Ease of use—Instead of storing instances in an array, instances are stored in a map. This makes referencing individual resources easier. Also, if an element in the middle is added or removed, it won’t affect references to elements that come after it, as it does with count.

for_each is the recommended approach to create dynamic configurations. Unless you have a specific reason to access something by index (such as our round-robin approach to creating Mad Lib files in chapter 3), I recommend using for_each. The syntax of for_each is shown in figure 7.6.

Figure 7.6 Syntax of the for_each meta argument and its associated each object

In resource blocks where for_each is set, an additional each object is made available for use by expressions. The each object is a reference to the current entry in the iterator and has two accessors:

• each.key—The map key or set item corresponding to the entry.

• each.value—The map value corresponding to this entry (for sets, this is the same as each.key).

I personally found each confusing when I first read about it—after all, what do keys and values have to do with sets? What helped me was imagining that Terraform first transforms the set into a list of objects and then iterates over that list (see figure 7.7).

 

Figure 7.7 The input set is transformed into a list of each objects. This new iterator is used by for_each.

When for_each is set, the resource address points to a map of resource instances rather than a single instance (or list of instances, as would be the case with count). To refer to a specific instance member, simply append the iterator map key after the normal resource address: <TYPE>.<NAME>.[<KEY>]. For example, if we wanted to reference the resource instance corresponding to sourcerepo.googleapis.com, we could do so with the following expression:

google_project_service.enabled_service["sourcerepo.googleapis.com"]

7.4.2 Executing scripts with provisioners

Resource provisioners allow you to execute scripts on local or remote machines as part of resource creation or destruction. They are used for various tasks, such as bootstrapping, copying files, hacking into the mainframe, etc. You can attach a resource provisioner to any resource, but most of the time it doesn’t make sense to do so, which is why provisioners are most commonly seen on null resources. Null resources are basically resources that don’t do anything, so having a provisioner on one is as close as you can get to having a standalone provisioner.

Note Because resource provisioners call external scripts, there is an implicit dependency on the OS interpreter.

Provisioners allow you to dynamically extend functionality on resources by hooking into resource lifecycle events. There are two kinds of resource provisioners:

• Creation-time provisioners

• Destruction-time provisioners

Most people who use provisioners exclusively use creation-time provisioners: for example, to run a script or kick off some miscellaneous automation task. The following example is unusual because it uses both:

resource "google_project_service" "enabled_service" {
  for_each = toset(local.services)
  project  = var.project_id
  service = each.key
 
  provisioner "local-exec" {
    command = "sleep 60"                 ❶
  }
 
  provisioner "local-exec" {
    when    = destroy
    command = "sleep 15"
  }
}

❶ The “when” attribute defaults to “apply” if not set.

This creation-time provisioner invokes the command sleep 60 to wait for 60 seconds after Create() has completed but before the resource is marked as “created” by Terraform (see figure 7.8). Likewise, the destruction-time provisioner waits for 15 seconds before Delete() is called (see figure 7.9). Both of these pauses (determined experimentally through trial and error) are essential to avoid potential race conditions when enabling/disabling service APIs (see http://mng.bz/oGmZ).

Figure 7.8 The local-exec provisioner is called after the Create() function hook has exited but before the resource is marked as “created” by Terraform.

Figure 7.9 The local-exec provisioner is called before Delete().

7.4.3 Null resource with a local-exec provisioner

If both a creation-time and a destruction-time provisioner are attached to the same null_resource, you can cobble together a sort of custom Terraform resource. Null resources don’t do anything on their own. Therefore, if you have a null resource with a creation-time provisioner that calls a create script and a destruction time provisioner that calls a cleanup script, it wouldn’t behave all that differently from a conventional Terraform resource.

The following example code creates a custom resource that prints “Hello World!” on resource creation and “Goodbye cruel world!” on resource deletion. I’ve spiced it up a bit by using cowsay, a CLI tool that prints a picture of an ASCII cow saying the message:

resource "null_resource" "cowsay" {
  provisioner "local-exec" {                          ❶
    command = "cowsay Hello World!"
  }
 
  provisioner "local-exec" {                          ❷
    when    = destroy
    command = "cowsay -d Goodbye cruel world!"
  }
}

❶ Creation-time provisioner

❷ Destruction-time provisioner

On terraform apply, Terraform will run the creation-time provisioner:

$ terraform apply -auto-approve
null_resource.cowsay: Creating...
null_resource.cowsay: Provisioning with 'local-exec'...
null_resource.cowsay (local-exec): Executing: ["/bin/sh" "-c" "cowsay Hello 
world!"]
null_resource.cowsay (local-exec):  ______________
null_resource.cowsay (local-exec): < Hello World! >
null_resource.cowsay (local-exec):  --------------
null_resource.cowsay (local-exec):            ^__^
null_resource.cowsay (local-exec):            (oo)\_______
null_resource.cowsay (local-exec):             (__)       )/
null_resource.cowsay (local-exec):                 ||----w |
null_resource.cowsay (local-exec):                 ||     ||
null_resource.cowsay: Creation complete after 0s [id=1729885674162625250]
 
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Likewise, on terraform destroy, Terraform runs the destruction-time provisioner:

$ terraform destroy -auto-approve
null_resource.cowsay: Refreshing state... [id=1729885674162625250]
null_resource.cowsay: Destroying... [id=1729885674162625250]
null_resource.cowsay: Provisioning with 'local-exec'...
null_resource.cowsay (local-exec): Executing: ["/bin/sh" "-c" "cowsay -d 
Goodbye cruel world!"]
null_resource.cowsay (local-exec):  ______________________
null_resource.cowsay (local-exec): < Goodbye cruel world! >
null_resource.cowsay (local-exec):  ----------------------
null_resource.cowsay (local-exec):            ^__^
null_resource.cowsay (local-exec):            (xx)\_______
null_resource.cowsay (local-exec):             (__)       )/
null_resource.cowsay (local-exec):              U  ||----w |
null_resource.cowsay (local-exec):                 ||     ||
null_resource.cowsay: Destruction complete after 0s
 
Destroy complete! Resources: 1 destroyed.

TIP f you are interested in creating custom resources without writing your own provider, I recommend taking a look at the Shell provider (http://mng.bz/n2v5), which is covered in appendix D.

7.4.4 Dealing with repeating configuration blocks

Returning to the main scenario, we need to configure the resources that make up the CI/CD pipeline (see figure 7.10). To start, add the code from listing 7.5 to main.tf. This will provision a version-controlled source repository, which is the first stage of our CI/CD pipeline.

Figure 7.10 CI/CD pipeline: stage 1 of 3

 

Listing 7.5 main.tf

resource "google_sourcerepo_repository" "repo" {
  depends_on = [
    google_project_service.enabled_service["sourcerepo.googleapis.com"]
  ]
 
  name = "${var.namespace}-repo"
} 

Figure 7.11 CI/CD pipeline: stage 2 of 3

Next, we need to set up a Cloud Build to trigger a run from a commit to the source repository (see figure 7.11). Since there are several steps in the build process, one way to do this would be to declare a series of repeating configuration blocks, as shown here:

resource "google_cloudbuild_trigger" "trigger" {
  depends_on = [
    google_project_service.enabled_service["cloudbuild.googleapis.com"]
  ]
 
  trigger_template {
    branch_name = "master"
    repo_name   = google_sourcerepo_repository.repo.name
  }
 
  build {
    step {                                              ❶
      name = "gcr.io/cloud-builders/go"
      args = ["test"]
      env  = ["PROJECT_ROOT=${var.namespace}"]
    }
 
    step {                                              ❶
      name = "gcr.io/cloud-builders/docker"
      args = ["build", "-t", local.image, "."]
    }
 
    step {                                              ❶
      name = "gcr.io/cloud-builders/docker"
      args = ["push", local.image]
    }
 
    step {                                              ❶
      name = "gcr.io/cloud-builders/gcloud"
["run", "deploy", google_cloud_run_service.service.name, 
"--region", var.region, "--platform", "managed", 
"-q"]
    }
  }
}

❶ Repeating configuration blocks for the steps in the build process

As you can see, this works, but it’s not exactly flexible or elegant. Having the build steps declared statically doesn’t help if you didn’t know what those steps were at deployment time. Also, this approach is not configurable. To solve this annoying problem, HashiCorp introduced a new expression called dynamic blocks.

7.4.5 Dynamic blocks: Rare boys

Dynamic blocks are the rarest of all Terraform expressions, and many people don’t even know they exist. They were designed to solve the niche problem of how to create nested configuration blocks dynamically in Terraform. Dynamic blocks can only be used within other blocks and only when the use of repeatable configuration blocks is supported (surprisingly, not that common). Nevertheless, dynamic blocks are situationally useful, such as when creating rules in a security group or steps in a Cloud Build trigger.

Dynamic nested blocks act much like for expressions but produce nested configuration blocks instead of complex types. They iterate over complex types (such as maps and lists) and generate configuration blocks for each element. The syntax for a dynamic nested block is illustrated in figure 7.12.

Figure 7.12 Syntax for a dynamic nested block

Warning Use dynamic blocks sparingly, because they make your code more difficult to understand.

Typically, dynamic nested blocks are combined with local values or input variables (because otherwise, your code would be statically defined, and you wouldn’t need to use a dynamic block). In our case, it doesn’t matter since we are basically hard-coding the build steps anyway, but it is good practice. I like to declare such local values that serve only as helpers right above where they are used. You could also put them at the top of the file or in a separate locals.tf file, but in my opinion, doing so makes things more confusing. Append the contents of the following listing to main.tf to provision the Cloud Build trigger and the steps it will employ.

Listing 7.6 main.tf

locals {                                                                   ❶
  image = "gcr.io/${var.project_id}/${var.namespace}"
  steps = [
    {
      name = "gcr.io/cloud-builders/go"
      args = ["test"]
      env  = ["PROJECT_ROOT=${var.namespace}"]
    },
    {
      name = "gcr.io/cloud-builders/docker"
      args = ["build", "-t", local.image, "."]
    },
    {
      name = "gcr.io/cloud-builders/docker"
      args = ["push", local.image]
    },
    {
      name = "gcr.io/cloud-builders/gcloud"
      args = ["run", "deploy", google_cloud_run_service.service.name, 
"--image", local.image, "--region", var.region, "--platform", "managed", 
"-q"]
    }
 
  ]
}
 
resource "google_cloudbuild_trigger" "trigger" {
  depends_on = [
    google_project_service.enabled_service["cloudbuild.googleapis.com"]
  ]
 
  trigger_template {
    branch_name = "master"
    repo_name   = google_sourcerepo_repository.repo.name
  }
 
  build {
    dynamic "step" {
      for_each = local.steps
      content {
        name = step.value.name
        args = step.value.args
        env  = lookup(step.value, "env", null)                             ❷
      }
    }
  }
} 

❶ Declaring local values right before using them helps with readability.

❷ Not all steps have “env” set. Lookup() returns null if step.value["env"] is not set.

Before we move on to the next section, let’s add some IAM-related configuration to main.tf. This will enable Cloud Build to deploy services onto Cloud Run. For that, we need to give Cloud Build the run.admin and iam.serviceAccountUser roles.

Listing 7.7 main.tf

data "google_project" "project" {}
 
resource "google_project_iam_member" "cloudbuild_roles" {
  depends_on = [google_cloudbuild_trigger.trigger]
  for_each   = toset(["roles/run.admin", 
      "roles/iam.serviceAccountUser"])                               ❶
  project    = var.project_id
  role       = each.key
  member     = "serviceAccount:${data.google_project.project.number}
   ➥ @cloudbuild.gserviceaccount.com"
} 

❶ Grants the Cloud Build service account these two roles

7.5 Configuring a serverless container

Now we need to configure the Cloud Run service for running our serverless container after it has been deployed with Cloud Build (see figure 7.13). This process has two steps: we need to declare and configure the Cloud Run service, and we need to explicitly enable unauthenticated user access because the default is Deny All.

Figure 7.13 CI/CD pipeline: stage 3 of 3

The code for configuring the Cloud Run service is shown in listing 7.8. It’s not complicated. The only surprising thing is that we are pointing the container image to a GCP published “Hello” demo image instead of our own. The reason is that our image doesn’t yet exist in the Container Registry, so Terraform would throw an error if we tried to apply. Since image is a required argument, we have to set it to something, but it doesn’t really matter what it is because the first execution of Cloud Build will override it.

Listing 7.8 main.tf

resource "google_cloud_run_service" "service" {
  depends_on = [
    google_project_service.enabled_service["run.googleapis.com"]
  ]
  name     = var.namespace
  location = var.region
 
  template {
    spec {
      containers {
        image = "us-docker.pkg.dev/cloudrun/container/hello"       ❶
      }
    }
  }
} 

❶ The Cloud Run service initially uses a demo image that’s already in the Container Registry.

To expose the web application to the internet, we need to enable unauthenticated user access. We can do that with an IAM policy that grants all users the run.invoker role to the provisioned Cloud Run service. Add the following code to the bottom of main.tf.

Listing 7.9 main.tf

data "google_iam_policy" "admin" {
  binding {
    role = "roles/run.invoker"
    members = [
      "allUsers",
    ]
  }
}
 
resource "google_cloud_run_service_iam_policy" "policy" {
  location    = var.region
  project     = var.project_id
  service     = google_cloud_run_service.service.name
  policy_data = data.google_iam_policy.admin.policy_data
}

We are almost done. We just need to address a couple of minor things before finishing: the output values and the provider versions. Create outputs.tf and versions.tf; we will need both of them later. The outputs.tf file will output the URLs from the source repository and Cloud Run service.

Listing 7.10 outputs.tf

output "urls" {
  value = {
    repo = google_sourcerepo_repository.repo.url
    app  = google_cloud_run_service.service.status[0].url
  }
}

Finally, versions.tf locks in the GCP provider version.

Listing 7.11 versions.tf

terraform {
  required_version = ">= 0.15"
  required_providers {
      google = {
          source = "hashicorp/google"
          version = "~> 3.56"
      }
  }
}

7.6 Deploying static infrastructure

Remember that there are two parts to this project: the static (aka Terraform) part and the dynamic (or non-Terraform) part. What we have been working on so far only amounts to the static part, which is responsible for laying down the underlying infrastructure that the dynamic infrastructure will run on. We will talk about how to deploy dynamic infrastructure in the next section. For now, we will deploy the static infrastructure. The complete source code of main.tf is shown next.

Listing 7.12 Complete main.tf

locals {
  services = [
    "sourcerepo.googleapis.com",
    "cloudbuild.googleapis.com",
    "run.googleapis.com",
    "iam.googleapis.com",
  ]
}
 
resource "google_project_service" "enabled_service" {
  for_each = toset(local.services)
  project  = var.project_id
  service  = each.key
 
  provisioner "local-exec" {
    command = "sleep 60"
  }
 
  provisioner "local-exec" { 
    when    = destroy
    command = "sleep 15"
  }
}
 
resource "google_sourcerepo_repository" "repo" {
  depends_on = [
    google_project_service.enabled_service["sourcerepo.googleapis.com"]
  ]
 
  name = "${var.namespace}-repo"
}
 
locals { 
  image = "gcr.io/${var.project_id}/${var.namespace}"
  steps = [
    {
      name = "gcr.io/cloud-builders/go"
      args = ["test"]
      env  = ["PROJECT_ROOT=${var.namespace}"]
    },
    {
      name = "gcr.io/cloud-builders/docker"
      args = ["build", "-t", local.image, "."]
    },
    {
      name = "gcr.io/cloud-builders/docker"
      args = ["push", local.image]
    },
    {
      name = "gcr.io/cloud-builders/gcloud"
      args = ["run", "deploy", google_cloud_run_service.service.name, 
"--image", local.image, "--region", var.region, "--platform", "managed", 
"-q"]
    }
 
  ]
}
 
resource "google_cloudbuild_trigger" "trigger" {
  depends_on = [
    google_project_service.enabled_service["cloudbuild.googleapis.com"]
  ]
 
  trigger_template {
    branch_name = "master"
    repo_name   = google_sourcerepo_repository.repo.name
  }
 
  build {
    dynamic "step" {
      for_each = local.steps
      content {
        name = step.value.name
        args = step.value.args
        env  = lookup(step.value, "env", null) 
      }
    }
  }
}
 
data "google_project" "project" {}
 
resource "google_project_iam_member" "cloudbuild_roles" {
  depends_on = [google_cloudbuild_trigger.trigger]
  for_each   = toset(["roles/run.admin", "roles/iam.serviceAccountUser"])
  project    = var.project_id
  role       = each.key
  member     = "serviceAccount:${data.google_project.project.number}
  ➥ @cloudbuild.gserviceaccount.com"
}
 
resource "google_cloud_run_service" "service" {
  depends_on = [
    google_project_service.enabled_service["run.googleapis.com"]
  ]
  name     = var.namespace
  location = var.region
 
  template {
    spec {
      containers {
        image = "us-docker.pkg.dev/cloudrun/container/hello"
      }
    }
  }
}
 
data "google_iam_policy" "admin" {
  binding {
    role = "roles/run.invoker"
    members = [
      "allUsers",
    ]
  }
}
 
resource "google_cloud_run_service_iam_policy" "policy" {
  location    = var.region
  project     = var.project_id
  service     = google_cloud_run_service.service.name
  policy_data = data.google_iam_policy.admin.policy_data
}

When you’re ready, initialize and deploy the infrastructure to GCP:

$ terraform init && terraform apply -auto-approve
...
google_project_iam_member.cloudbuild_roles["roles/iam.serviceAccountUser"]: 
Creation complete after 10s [id=tic-
pipelines/roles/iam.serviceAccountUser/serviceaccount:[email protected]
ld.gserviceaccount.com]
 
Apply complete! Resources: 10 added, 0 changed, 0 destroyed.
 
Outputs:
 
urls = {

  "repo" = "https://source.developers.google.com/p/tia-chapter7/r/team-
rocket-repo"
}
 

Figure 7.14 The demo Cloud Run service is initially running.

At this point, your Cloud Run service is available at the urls.app address, although it is only serving the demo container (see figure 7.14).

7.7 CI/CD of a Docker container

In this section, we deploy a Docker container to Cloud Run through the CI/CD pipeline. The Docker container we’ll create is a simple HTTP server that listens on port 8080 and serves a single endpoint. The application code we deploy runs on top of existing static infrastructure (see figure 7.15).

Figure 7.15 Dynamic infrastructure is deployed on top of the static infrastructure.

From section 7.3.1, you should have two folders: application and infrastructure. All the code until now has been in the infrastructure folder. To get started with the application code, switch over to the application folder:

$ cd ../application

In this directory, create a main.go file that will be the entry point for the server.

Listing 7.13 main.go

package main
 
import (
    "fmt"
    "log"
    "net/http"
)
 
func IndexServer(w http.ResponseWriter, r *http.Request) {
    fmt.Fprint(w, "Automate all the things!")                    ❶
}
 
func main() {
    handler := http.HandlerFunc(IndexServer)
    log.Fatal(http.ListenAndServe(":8080", handler))
} 

❶ Starts the server on port 8080 and serves the string “Automate all the things!”

Next, write a basic unit test and save it as main_test.go.

Listing 7.14 main_test.go

package main
 
import (
    "net/http"
    "net/http/httptest"
    "testing"
)
 
func TestGETIndex(t *testing.T) {
    t.Run("returns index", func(t *testing.T) {
        request, _ := http.NewRequest(http.MethodGet, "/", nil)
        response := httptest.NewRecorder()
 
        IndexServer(response, request)
 
        got := response.Body.String()
        want := "Automate all the things!"
 
        if got != want {
            t.Errorf("got '%s', want '%s'", got, want)
        }
    })
}

Now create a Dockerfile for packaging the application. The following listing shows the code for a basic multistage Dockerfile that will work for our purposes.

Listing 7.15 Dockerfile

FROM golang:1.15 as builder
WORKDIR /go/src/github.com/team-rocket
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -v -o app
 
FROM alpine
RUN apk update && apk add --no-cache ca-certificates
COPY --from=builder /go/src/github.com/team-rocket/app /app
CMD ["/app"]

7.7.1 Kicking off the CI/CD pipeline

At this point, we can upload our application code to the source repository, which will kick off the CI/CD pipeline and deploy to Cloud Run. The following commands make this happen. You’ll need to substitute in the repo URL from the earlier Terraform output.

Listing 7.16 Git commands

git init && git add -A && git commit -m "initial push"
git config --global credential.https://source.developers.google.com.helper 
gcloud.sh
git remote add google <urls.repo>                  ❶
gcloud auth login && git push --all google

❶ Insert your source repo URL here.

After you’ve pushed your code, you can view the build status in the Cloud Build console. Figure 7.16 shows an example of what an in-progress build might look like.

Figure 7.16 Cloud Build triggers a build when you commit to the master branch. This will build, test, publish, and finally deploy the code to Cloud Run.

When the build completes, you can navigate to the application URL in the browser (from the app output attribute). You should see a spartan website with the words “Automate all the things!” in plain text (see figure 7.17). This means you have succesfullly deployed an app through the pipeline and completed the scenario.

Figure 7.17 Example deployed website

Warning Don’t forget to clean up your static infrastructure with terraform destroy. Alternatively, you can manually delete the GCP project from the console.

7.8 Fireside chat

We started by talking about two-stage deployments, where you separate your static infrastructure from your dynamic infrastructure. Static infrastructure doesn’t change a lot, which is why it’s a good candidate to be provisioned with Terraform. On the other hand, dynamic infrastructure changes far more frequently and typically consists of things like configuration settings and application source code. By making a clear division between static and dynamic infrastructure, you can experience faster, more reliable deployments.

Even though the Terraform code we deployed was for static infrastructure, it was the most expressive code we have seen so far. We introduced for_each expressions, dynamic blocks, and even resource provisioners. We only looked at the local-exec provisioner, but there are actually three kinds of resource provisioners: see Table 7.1 for a comparison between the different provisioner types.

Warning Backdoors to Terraform (i.e., resource provisioners) are inherently dangerous and should be avoided. Use them only as a last resort.

Table 7.1 Reference of resource provisioners in Terraform

Name	Description	Example
file	Copies files or directories from the machine executing Terraform to the newly created resource.	provisioner "file" { source = "conf/myapp.conf" destination = "/etc/myapp.conf" }
local-exec	Invokes an arbitrary process on the machine running Terraform (not on the resource).	provisioner "local-exec" { command = "echo hello" }
remote-exec	Invokes a script on a remote resource after it is created. This can be used to run configuration management tools, bootstrap scripts, etc.	provisioner "remote-exec" { inline = [ "puppet apply", ] }

Summary

• We designed and deployed a CI/CD pipeline as code on GCP. There are five stages to this pipeline: source, test, build, release, and deploy.

• There are two methods for deploying with Terraform: everything all-in-one and separating static from dynamic infrastructure.

• for_each can provision resources dynamically, like count, but uses a map instead of a list. Dynamic blocks are similar, except they allow you to generate repeating configuration blocks.

• Providers can be either implicit or explicit. Explicit providers are typically used for multi-region deployments or, in the case of GCP, for using the beta version of the provider.

• Resource provisioners can be either creation-time or destruction-time. If you have both of them on a null resource, this can be a way to create bootleg custom resources. You can also create custom resources with the Shell provider.