- Developing an S3 remote backend module
- Comparing flat vs. nested module structures
- Publishing modules via GitHub and the Terraform Registry
- Switching between workspaces
- Examining Terraform Cloud and Terraform Enterprise
Software development is a team sport. At some point, you’ll want to collaborate on Terraform projects with friends and coworkers. Sharing configuration code is easy—any version-controlled source (VCS) repository will do. Sharing state is where it gets difficult. Until now, our state has always been saved to a local backend, which is fine for development purposes and individual contributors but doesn’t accommodate shared access. Suppose Sally from site reliability engineering (SRE) wants to make some configuration changes and redeploy. Unless she has access to the existing state file, there is no way to reconcile with what’s already in production. Checking in the state file to a VCS repository is not recommended because of the potential to expose sensitive information and also because doing so doesn’t prevent race conditions.
A race condition is an undesirable event that occurs when two entities attempt to access or modify shared resources in a given system. In Terraform, race conditions occur when two people are trying to access the same state file at the same time, such as when one is performing a terraform apply and another is performing terraform destroy. If this happens, your state file can become out of sync with what’s actually deployed, resulting in what is known as a corrupted state. Using a remote backend end with a state lock prevents this from happening.
In this chapter, we develop an S3 remote backend module and publish it on the Terraform Registry. Next, we deploy the backend and store some state in it. We also talk about workspaces and how they can be used to deploy multiple environments. Finally, we introduce HashiCorp’s proprietary products for teams and organizations: Terraform Cloud and Terraform Enterprise.
6.1 Standard and enhanced backends
A backend in Terraform determines how state is loaded and how CLI operations like terraform plan and terraform apply behave. We’ve actually been using a local backend this whole time, because that’s Terraform’s default behavior. Backends can do the following tasks:
Some backends can completely overhaul the way Terraform works, but most are not much different from a local backend. The main responsibility of any backend is to determine how state files are stored and accessed. For remote backends, this generally means some kind of encryption at rest and state file versioning. You should refer to the documentation for the specific backend you want to use, to learn what is supported and what isn’t (www.terraform.io/docs/backends/types).
Besides standard remote backends, there are also enhanced backends. Enhanced backends are a relatively new feature and allow you to do more sophisticated things like run CLI operations on a remote machine and stream the results back to your local terminal. They also allow you to read variables and environment variables stored remotely, so there’s no need for a variables definition file (terraform.tfvars). Although enhanced backends are great, they currently only work for Terraform Cloud and Terraform Enterprise. Don’t worry, though: most people who use Terraform—even at scale—will be perfectly content with any of the standard backends.
The most popular standard backend is the S3 remote backend for AWS (probably because most people use AWS). In the next few sections, I show you how to build and deploy an S3 backend module, as well as the workflow for utilizing it. Figure 6.1 shows a basic diagram of how the S3 backend works.
Figure 6.1 How the S3 backend works. State files are encrypted at rest using KMS. Access is controlled by a least-privileged IAM policy, and everything is synchronized with DynamoDB.
6.2 Developing an S3 backend module
Our goal is to develop a module that can eventually be used to deploy a production-ready S3 backend. If your primary cloud is Azure or Google Cloud Platform (GCP), then the code here will not be immediately relevant, but the idea is the same. Since standard backends are more similar than they are dissimilar, you can apply what you learn here to develop a custom solution for whichever backend you prefer.
This project was designed from the exacting requirements laid out in the official documentation (www.terraform.io/docs/backends/types/s3.html), which does an excellent job of explaining what you need to do but not how to do it. We are told the parts we need but not how to assemble them. Since you’re probably going to want to deploy an S3 backend anyway, we’ll save you the trouble by working on it together. Also, we’ll publish this on the Terraform Registry so it can be shared with others.
6.2.1 Architecture
I always start by considering the overall inputs and outputs from a black-box perspective. There are three input variables for configuring various settings, which we’ll talk more about soon, and one output value that has all the information required for workspaces to initialize themselves against the S3 backend. This is depicted in figure 6.2.
Figure 6.2 There are three inputs and one output for the S3 backend module. The output value config has all the information required for a workspace to initialize itself against the S3 backend.
Considering what’s inside the box, four distinct components are required to deploy an S3 backend:
-
S3 bucket and Key Management Service (KMS) key—For state storage and encryption at rest.
-
Identity and Access Management (IAM) least-privileged role—So other AWS accounts can assume a role to this account and perform deployments against the S3 backend.
-
Miscellaneous housekeeping resources—We’ll talk more about these later.
Figure 6.3 helps visualize the relationship from a Terraform dependency perspective. As you can see, there are four independent “islands” of resources. No dependency relationship exists among these resources because they don’t depend on each other. These islands, or components, would be excellent candidates for modulization, as discussed in chapter 4, but we won’t do that here as it would be overkill. Instead, I’ll introduce a different design pattern for organizing code that’s perfectly valid for this situation. Although popular, it doesn’t have a colloquial name, so I’ll simply refer to it as a flat module.
Figure 6.3 Detailed architecture diagram showing the four distinct components that make up this module
6.2.2 Flat modules
Flat modules (as opposed to nested modules) organize your codebase as lots of little .tf files within a single monolithic module. Each file in the module contains all the code for deploying an individual component, which would otherwise be broken out into its own module. The primary advantage of flat modules over nested modules is a reduced need for boilerplate, as you don’t have to plumb any of the modules together. For example, instead of creating a module for deploying IAM resources, the code could be put into a file named iam.tf. This is illustrated in figure 6.4.
Figure 6.4 A flat module structure applied to the S3 backend module. All IAM resources go in iam.tf, and everything else goes in main.tf.
For this particular scenario, it makes a lot of sense to do it this way: the code for deploying the IAM is inconveniently long to be included in main.tf but not quite long enough to warrant being a separate module.
TIP There’s no fixed rule about how long the code in a single configuration file should be, but I try not to include more than a few hundred lines. This is an entirely personal preference.
Warning Think carefully before deciding to use a flat module for code organization. This pattern tends to result in a high degree of coupling between components, which can make your code more difficult to read and understand.
6.2.3 Writing the code
Let’s move on to writing the code. Start by creating six files: variables.tf, main.tf, iam.tf, outputs.tf, versions.tf, and README.md. Listing 6.1 shows the code for variables.tf.
Note I have published this as a module in the Terraform Registry, if you want to use that and skip ahead: https://registry.terraform.io/modules/terraform-in-action/s3backend/aws/latest.
variable "namespace" {
description = "The project namespace to use for unique resource naming"
default = "s3backend"
type = string
}
variable "principal_arns" {
description = "A list of principal arns allowed to assume the IAM role"
default = null
type = list(string)
}
variable "force_destroy_state" {
description = "Force destroy the s3 bucket containing state files?"
default = true
type = bool
}
The complete code for provisioning the S3 bucket, KMS key, and DynamoDB table is shown in the next listing. I put all this in main.tf because these are the module’s most important resources and because this is the first file most people will look at when reading through your project. The key to flat module design is naming things well and putting them where people exepect to find them.
data "aws_region" "current" {}
resource "random_string" "rand" {
length = 24
special = false
upper = false
}
locals {
namespace = substr(join("-", [var.namespace, random_string.rand.result]),
0, 24)
}
resource "aws_resourcegroups_group" "resourcegroups_group" { ❶
name = "${local.namespace}-group"
resource_query {
query = <<-JSON
{
"ResourceTypeFilters": [
"AWS::AllSupported"
],
"TagFilters": [
{
"Key": "ResourceGroup",
"Values": ["${local.namespace}"]
}
]
}
JSON
}
}
resource "aws_kms_key" "kms_key" {
tags = {
ResourceGroup = local.namespace
}
}
resource "aws_s3_bucket" "s3_bucket" { ❷
bucket = "${local.namespace}-state-bucket"
force_destroy = var.force_destroy_state
versioning {
enabled = true
}
server_side_encryption_configuration {
rule {
apply_server_side_encryption_by_default {
sse_algorithm = "aws:kms"
kms_master_key_id = aws_kms_key.kms_key.arn
}
}
}
tags = {
ResourceGroup = local.namespace
}
}
resource "aws_s3_bucket_public_access_block" "s3_bucket" {
bucket = aws_s3_bucket.s3_bucket.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}
resource "aws_dynamodb_table" "dynamodb_table" {
name = "${local.namespace}-state-lock"
hash_key = "LockID"
billing_mode = "PAY_PER_REQUEST" ❸
attribute {
name = "LockID"
type = "S"
}
tags = {
ResourceGroup = local.namespace
}
}
❶ Puts resources into a group based on tag
❸ Makes the database serverless instead of provisioned
The next listing is the code for iam.tf. This particular code creates a least-privileged IAM role that another AWS account can assume to deploy against the S3 backend. To clarify, all of the state files will be stored in an S3 bucket created by the S3 backend, so at a minimum, we expect deployment users to need permissions to put objects in S3. Additionally, they will need permissions to get/delete records from the DynamoDB table that manages locking.
Note Having multiple AWS accounts assume a least-priviliged IAM role prevents users from unauthorized access. Some state files store sensitive information in plain text that shouldn’t be read by just anyone.
data "aws_caller_identity" "current" {}
locals {
principal_arns = var.principal_arns != null ? var.principal_arns :
[data.aws_caller_identity.current.arn] ❶
}
resource "aws_iam_role" "iam_role" {
name = "${local.namespace}-tf-assume-role"
assume_role_policy = <<-EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"AWS": ${jsonencode(local.principal_arns)}
},
"Effect": "Allow"
}
]
}
EOF
tags = {
ResourceGroup = local.namespace
}
}
data "aws_iam_policy_document" "policy_doc" { ❷
statement {
actions = [
"s3:ListBucket",
]
resources = [
aws_s3_bucket.s3_bucket.arn
]
}
statement {
actions = ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"]
resources = [
"${aws_s3_bucket.s3_bucket.arn}/*",
]
}
statement {
actions = [
"dynamodb:GetItem",
"dynamodb:PutItem",
"dynamodb:DeleteItem"
]
resources = [aws_dynamodb_table.dynamodb_table.arn]
}
}
resource "aws_iam_policy" "iam_policy" {
name = "${local.namespace}-tf-policy"
path = "/"
policy = data.aws_iam_policy_document.policy_doc.json
}
resource "aws_iam_role_policy_attachment" "policy_attach" {
role = aws_iam_role.iam_role.name
policy_arn = aws_iam_policy.iam_policy.arn
}
❶ If no principal ARNs are specified, uses the current account
❷ Least-privileged policy to attach to the role
A workspace needs four pieces of information to initialize and deploy against an S3 backend:
Since this is not a root module, the outputs need to be bubbled up to be visible after a terraform apply (we’ll do this later). The outputs are shown next.
output "config" {
value = {
bucket = aws_s3_bucket.s3_bucket.bucket
region = data.aws_region.current.name
role_arn = aws_iam_role.iam_role.arn
dynamodb_table = aws_dynamodb_table.dynamodb_table.name
}
}
Note We don’t need a providers.tf because this is a module. The root module will implicitly pass all providers during initialization.
Even though we don’t declare providers, it’s still a good idea to version lock modules.
terraform {
required_version = ">= 0.15"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.28"
}
random = {
source = "hashicorp/random"
version = "~> 3.0"
}
}
}
Next, we need to create README.md. Believe it or not, having a README.md file is a requirement for registering a module with the Terraform Registry. You have to hand it to HashiCorp for laying down the law about these sorts of things. Let’s make a dirt-simple README.md to comply with this requirement (see listing 6.6).
TIp Terraform-docs (https://github.com/segmentio/terraform-docs) is a neat open source tool that automatically generates documentation from your configuration code. I highly recommend it.
# S3 Backend Module
This module will deploy an S3 remote backend for Terraform ❶
❶ You’ll probably want to write more documentation, such as what the inputs and outputs are and how to use them.
Finally, since we’ll be uploading this to a GitHub repo, you’ll want to create a .gitignore file. A pretty typical one for Terraform modules is shown next.
.DS_Store .vscode *.tfstate *.tfstate.* terraform **/.terraform/* crash.log
6.3 Sharing modules
Great—now we have a module. But how do we share it with friends and coworkers? Although I personally think the Terraform Registry is the best option, there are a number of possible avenues for sharing modules (see figure 6.5). The most common approach is to use GitHub repos, but I’ve also found S3 buckets to be a good option. In this section, I show you how to publish and source a module two ways: from GitHub and from the Terraform Registry.
Figure 6.5 Modules can be sourced from multiple possible avenues, including local paths, GitHub repos, and the Terraform Registry.
Note You’ll need to upload your code to GitHub even if you wish to use the Terraform Registry because the Terraform Registry sources from public GitHub repos.
6.3.1 GitHub
Sourcing modules from GitHub is easy. Just create a repo with a name in the form terraform-<PROVIDER>-<NAME>, and put your configuration code there (see figure 6.6). There’s no fixed rule about what PROVIDER and NAME should be, but I typically think of PROVIDER as the cloud I am deploying to and NAME as a helpful descriptor of the project. Therefore, the module we are deploying will be named terraform-aws-s3backend.
Figure 6.6 Example GitHub repo for the terraform-aws-s3backend module
A sample configuration for sourcing a module from a GitHub repo is as follows:
module "s3backend" {
source ="github.com/terraform-in-action/terraform-aws-s3backend"
}
TIP You can use a generic Git address to version-control GitHub modules by specifying a branch or tag name. Generic Git URLs are prefixed with the address git::.
6.3.2 Terraform Registry
The Terraform Registry is free and easy to use; all you need is a GitHub account to get started (https://registry.terraform.io). After you sign in, it takes just a few clicks in the UI to register a module so that other people can start using it. Because the Terraform Registry always reads from public GitHub repos, publishing your module in the registry makes your module available to everyone. One of the perks of Terraform Enterprise is that it lets you have your own private Terraform Registry, which is useful for sharing private modules in large organizations.
Note You can also implement the module registry protocol (www.terraform .io/docs/internals/module-registry-protocol.html) if you wish to create your own private module registry.
Implementing the Terraform Registry is not complicated in the least; I think of it as little more than a glorified key-value store that maps source keys to GitHub tags. Its main benefit is that it enforces certain naming conventions and standards based on established best practices for publishing modules. (HashiCorp’s best practices for modules can be found at www.terraform.io/docs/modules). It also makes it easy to version-control and search for other people’s modules by name or provider. Here’s a list of the official rules (www.terraform.io/docs/registry/modules/publish.html):
-
Have a README.md file (preferably with some example usage code).
-
Follow the standard module structure (i.e. have main.tf, variables.tf, and outputs.tf files).
Figure 6.7 Navigate to the Terraform Registry home page.
I highly encourage you to try this yourself. In the following figures, you can see how easy it is to do. First, create a release in GitHub using semantic versioning. Next, sign in to the Terraform Registry UI and click the Publish button (figure 6.7). Select the GitHub repo you wish to publish (figure 6.8), and wait for it to be published (figure 6.9).
Figure 6.8 Choose a GitHub repo to register as a module.
Figure 6.9 Published module in the Terraform Registry
6.4 Everyone gets an S3 backend
Since S3 backends are cheap, especially when using a serverless DynamoDB table like we are, there’s no reason not to have lots of them. Deploying one backend per team is a reasonable way to go about partitioning things because you don’t want all your state files in one bucket, but you still want to give people enough autonomy to do their job.
Note If you are highly disciplined about least-privileged IAM roles, it’s fine to have a single backend. That’s how Terraform Cloud and Terraform Enterprise work, after all.
Suppose we need to deploy an S3 backend for a motley crew of individuals calling themselves Team Rocket. After we deploy an S3 backend for them, we’ll need to verify that we can initialize against it. As part of this process, we’ll also cover workspaces and how they can be used to deploy configuration code to multiple environments.
6.4.1 Deploying the S3 backend
We need a root module wrapper for deploying the S3 backend module. If you published the module on GitHub or the Terraform Registry, you can set the source to point to your module; otherwise, you can use the one I’ve already published. Create a new Terraform project with a file containing the following code.
provider "aws" {
region = "us-west-2"
}
module "s3backend" {
source = "terraform-in-action/s3backend/aws" ❶
namespace = "team-rocket"
}
output "s3backend_config" {
value = module.s3backend.config ❷
}
❶ You can either update the source to point to your module in the registry or use mine.
❷ Config required to connect to the backend
TIP You can use the for-each meta-argument to deploy multiple copies of the s3backend module. We talk about how to use for-each on modules in chapter 9.
Start by running terraform init followed by terraform apply:
$ terraform init && terraform apply ... # random_string.rand will be created + resource "random_string" "rand" { + id = (known after apply) + length = 24 + lower = true + min_lower = 0 + min_numeric = 0 + min_special = 0 + min_upper = 0 + number = true + result = (known after apply) + special = false + upper = false } Plan: 9 to add, 0 to change, 0 to destroy. Changes to Outputs: + config = { + bucket = (known after apply) + dynamodb_table = (known after apply) + region = "us-west-2" + role_arn = (known after apply) } Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value:
When you’re ready, confirm and wait for the resources to be provisioned:
... s3backend.aws_iam_policy.iam_policy: Creation complete after 1s [id=arn:aws:iam::215974853022:policy/tf-policy] module.s3backend.aws_iam_role_policy_attachment.policy_attach: Creating... module. s3backend.aws_iam_role_policy_attachment.policy_attach: Creation complete after 1s [id=tf-assume-role-20190722062228664100000001] Apply complete! Resources: 9 added, 0 changed, 0 destroyed. Outputs: config = { "bucket" = "team-rocket-1qh28hgo0g1c-state-bucket" "dynamodb_table" = "team-rocket-1qh28hgo0g1c-state-lock" "region" = "us-west-2" "role_arn" = "arn:aws:iam::215974853022:role/team-rocket-1qh28hgo0g1c-tf- assume-role" }
Save the s3backend_config output value, as we’ll need it in the next step.
6.4.2 Storing state in the S3 backend
Now we’re ready for the interesting part: initializing against the S3 backend and verifying that it works. Create a new Terraform project with a test.tf file, and configure the backend using the output from the previous section (see the next listing). We have to create a unique key for the project, which is basically just a prefix to the object stored in S3. This can be anything, so let’s call it jesse/james.
terraform {
backend "s3" { ❶
bucket = "team-rocket-1qh28hgo0g1c-state-bucket" ❷
key = "jesse/james" ❷
region = "us-west-2" ❷
encrypt = true ❷
role_arn = "arn:aws:iam::215974853022:role/team-rocket- ❷
1qh28hgo0g1c-tf-assume-role" ❷
dynamodb_table = "team-rocket-1qh28hgo0g1c-state-lock" ❷
}
required_version = ">= 0.15"
required_providers {
null = {
source = "hashicorp/null"
version = "~> 3.0"
}
}
}
❶ Backends are configured within Terraform settings.
❷ Replace with the values from the previous output.
Note You need AWS credentials to assume the role specified by the backend role_arn attribute. By design, it looks for environment variables: AWS _ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, or the default profile stored in your AWS credentials file (the same behavior as the AWS provider). There are also options to override the defaults (www.terraform.io/docs/back ends/types/s3.html#configuration-variables).
Next, we need a resource with which to test the S3 backend. This can be any resource, but I like to use a special resource offered by the null provider called null _resource. You can do lots of cool hacks with null_resource and local-exec provisioners (which I’ll delve into in the next chapter), but for now, all you need to know is that the following code provisions a dummy resource that prints “gotta catch em all” to the terminal during a terraform apply.
Note null_resource does not create any “real” infrastructure, making it good for testing purposes.
terraform {
backend "s3" {
bucket = "team-rocket-1qh28hgo0g1c-state-bucket"
key = "jesse/james"
region = "us-west-2"
encrypt = true
role_arn = "arn:aws:iam::215974853022:role/team-rocket-
1qh28hgo0g1c-tf-assume-role"
dynamodb_table = "team-rocket-1qh28hgo0g1c-state-lock"
}
required_version = ">= 0.15"
required_providers {
null = {
source = "hashicorp/null"
version = "~> 3.0"
}
}
}
resource "null_resource" "motto" {
triggers = {
always = timestamp()
}
provisioner "local-exec" {
command = "echo gotta catch em all" ❶
}
}
❶ This is where the magic happens.
Run terraform init. The CLI output is a little different than what we’ve seen before, because now it’s connecting to the S3 backend as part of the initialization process:
$ terraform init Initializing the backend... Successfully configured the backend "s3"! Terraform will automatically use this backend unless the backend configuration changes. ...
When Terraform has finished initializing, run terraform apply -auto-approve:
$ terraform apply -auto-approve ull_resource.motto: Creating... null_resource.motto: Provisioning with 'local-exec'... null_resource.motto (local-exec): Executing: ["/bin/sh" "-c" "echo gotta catch em all"] null_resource.motto (local-exec): gotta catch em all ❶ null_resource.motto: Creation complete after 0s [id=1806217872068888379] Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
❶ Prints “gotta catch em all” to stdout
As you can see, the null_resource outputs the catchphrase “gotta catch em all” to the terminal. Also, your state file is now safely stored in the S3 bucket created earlier, under the key jesse/james (see figure 6.10).
Figure 6.10 The state file is safely stored in the S3 bucket with the key jesse/james.
You can download the state file to view its contents or manually upload a new version, although there is no reason to do this under normal circumstances. It’s much easier to manipulate the state file with one of the terraform state commands. For example:
$ terraform state list
null_resource.motto
6.5 Reusing configuration code with workspaces
Workspaces allow you to have more than one state file for the same configuration code. This means you can deploy to multiple environments without resorting to copying and pasting your configuration code into different folders. Each workspace can use its own variable definitions file to parameterize the environment (see figure 6.11).
Figure 6.11 Workspaces let you use the same configuration code, parameterized by different variable definitions files, to deploy to multiple environments.
You have already been using workspaces, even if you haven’t realized it. Whenever you perform terraform init, Terraform creates and switches to a workspace named default. You can prove this by running the command terraform workspace list, which lists all workspaces and puts an asterisk next to the one you are currently on:
$ terraform workspace list
* default
To create and switch to a new workspace other than the default, use the command terraform workspace select <workspace>.
Why is this useful, and why do you care? You could have saved your state files under different names, such as dev.tfstate and prod.tfstate, and pointed to them with a command like terraform apply -state=<path>. Technically, workspaces are the same as renaming state files. You use workspaces because remote state backends support workspaces and not the -state argument. This makes sense when you remember that remote state backends do not store state locally (so there is no state file to point to). I recommend using workspaces even when using a local backend, if only to get in the habit of using them.
6.5.1 Deploying multiple environments
Our null resource deployment is a cute way to test that we can initialize and deploy against the remote stack backend, but it’s impractical for describing how to use workspaces effectively. In this section, we try something more real-world-esque: using workspaces to deploy two separate environments, dev and prod. Each environment will be parameterized by its own variable definitions file to allow us to customize the environment—for example, to deploy to different AWS regions or accounts.
Create a new folder with a main.tf file, as shown in the following listing (replace bucket, profile, role_arn, and dynamodb_table as before).
terraform {
backend "s3" {
bucket = "<bucket>"
key = "team1/my-cool-project"
region = "<region>" ❶
encrypt = true
role_arn = "<role_arn>"
dynamodb_table = "<dynamodb_table>"
}
required_version = ">= 0.15"
}
variable "region" {
description = "AWS Region"
type = string
}
provider "aws" {
region = var.region
}
data "aws_ami" "ubuntu" {
most_recent = true
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-*"]
}
owners = ["099720109477"]
}
resource "aws_instance" "instance" {
ami = data.aws_ami.ubuntu.id
instance_type = "t2.micro"
tags = {
Name = terraform.workspace ❷
}
}
❶ This region is where your remote state backend lives and may be different than the region you are deploying to. Since it is evaluated during initialization, it cannot be configured via a variable.
❷ A special variable, like “path”, containing only one attribute: “workspace”
In the current directory, create a folder called environments; and in this directory, create two files: dev.tfvars and prod.tfvars. The contents of these files will set the AWS region to which the EC2 instance will be deployed. An example of the variables definition file for dev.tfvars is shown next.
region = "us-west-2"
Next, initialize the workspace as usual:
$ terraform init ... Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary.
Instead of staying on the default workspace, I suggest immediately switching to a more appropriately named workspace. Most people name workspaces after a GitHub feature branch or deployment environment (such as dev, int, prod, and so on). Let’s switch to a workspace called dev to deploy the dev environment:
$ terraform workspace new dev Created and switched to workspace "dev"! You're now on a new, empty workspace. Workspaces isolate their state, so if you run "terraform plan" Terraform will not see any existing state for this configuration.
Deploy the configuration code for the dev environment with the dev variables:
$ terraform apply -var-file=./environments/dev.tfvars -auto-approve data.aws_ami.ubuntu: Refreshing state... aws_instance.instance: Creating... aws_instance.instance: Still creating... [10s elapsed] aws_instance.instance: Still creating... [20s elapsed] aws_instance.instance: Still creating... [30s elapsed] aws_instance.instance: Creation complete after 38s [id=i-0b7e117464ae7eaa3] Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
The state file has now been created in the S3 bucket under the key env:/dev/team1/my-cool-project. Switch to a new prod workspace to deploy the production environment:
$ terraform workspace new prod Created and switched to workspace "prod"! You're now on a new, empty workspace. Workspaces isolate their state, so if you run "terraform plan" Terraform will not see any existing state for this configuration.
As we are in the new workspace, the state file is now empty, which we can verify by running a terraform state list command and noting that it returns nothing:
$ terraform state list
Deploying to the prod environment is similar to dev, except now we use prod.tfvars instead of dev.tfvars. I suggest specifying a different region for prod.tfvars, as shown in the following listing.
region = "us-east-1"
Deploy to the prod workspace with the prod.tfvars variables definition file:
$ terraform apply -var-file=./environments/prod.tfvars -auto-approve data.aws_ami.ubuntu: Refreshing state... aws_instance.instance: Creating... aws_instance.instance: Still creating... [10s elapsed] aws_instance.instance: Still creating... [20s elapsed] aws_instance.instance: Still creating... [30s elapsed] aws_instance.instance: Creation complete after 38s [id=i-042808b20164b509d] Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Note Since we are still using the same configuration code, you do not need to run terraform init again.
Now, in S3, we have two state files: one for dev and one for prod (see figure 6.12). You can also inspect the two EC2 instances that were created, named with their workspace names (dev and prod). The states are also stored separately in S3 (see figure 6.13).
Figure 6.12 There are now two state files under :env corresponding to the dev and prod workspaces.
Figure 6.13 Workspaces manage their own state files and their own resources. Here you can see two EC2 instances: one deployed from the dev workspace and one deployed from the prod workspace.
Note I deployed both instances to the same region, rather than different regions, so they would appear in the same screenshot.
6.5.2 Cleaning up
To clean up, we need to delete the EC2 instances from each environment. Then we can delete the S3 backend.
Note You could also delete the EC2 instances through the console.
First, delete the prod deployment:
$ terraform destroy -var-file=environments/prodtfvars -auto-approve data.aws_ami.ubuntu: Refreshing state... aws_instance.instance: Refreshing state... [id=i-054e08ebe7f50b9ce] aws_instance.instance: Destroying... [id=i-054e08ebe7f50b9ce] aws_instance.instance: Still destroying... [id=i-054e08ebe7f50b9ce, 10s elapsed] aws_instance.instance: Still destroying... [id=i-054e08ebe7f50b9ce, 20s elapsed] aws_instance.instance: Still destroying... [id=i-054e08ebe7f50b9ce, 30s elapsed] aws_instance.instance: Destruction complete after 32s Destroy complete! Resources: 1 destroyed. Releasing state lock. This may take a few moments...
Next, switch into the dev workspace and destroy that:
$ terraform workspace select dev Switched to workspace "dev". $ terraform destroy -var-file=environments/dev.tfvars -auto-approve data.aws_ami.ubuntu: Refreshing state... aws_instance.instance: Refreshing state... [id=i-042808b20164b509d] aws_instance.instance: Destroying... [id=i-042808b20164b509d] aws_instance.instance: Still destroying... [id=i-042808b20164b509d, 10s elapsed] 042808b20164b509d, 20s elapsed] aws_instance.instance: Still destroying... [id=i-042808b20164b509d, 30s elapsed] aws_instance.instance: Destruction complete after 30s Destroy complete! Resources: 1 destroyed. Releasing state lock. This may take a few moments...
Finally, switch back into the directory from which you deployed the S3 backend, and run terraform destroy:
$ terraform destroy -auto-approve ... module.s3backend.aws_kms_key.kms_key: Still destroying... [id=16c6c452-2e74-41d4-ae57-067f3b4b8acd, 10s elapsed] module.s3backend.aws_kms_key.kms_key: Still destroying... [id=16c6c452-2e74-41d4-ae57-067f3b4b8acd, 20s elapsed] module.s3backend.aws_kms_key.kms_key: Destruction complete after 24s Destroy complete! Resources: 8 destroyed.
6.6 Introducing Terraform Cloud
Terraform Cloud is the software as a service (SaaS) version of Terraform Enterprise. It has three pricing tiers ranging from free to business (see figure 6.14). The free tier does a lot for you by giving you a free remote state store and enabling VCS/API-driven workflows. Team management, single sign-on (SSO), and Sentinel “policy as code” are some of the bonus features you get when you pay for the higher-tiered offerings. And if you are wondering, the business tier for Terraform Cloud is exactly the same as Terraform Enterprise, except Terraform Enterprise can be run on a private datacenter, whereas Terraform Cloud cannot.
Figure 6.14 The differences between Terraform open source, Terraform Cloud, and Terraform Enterprise
The remote state backend you get from Terraform Cloud does all the same things as an S3 remote backend: it stores state, locks and versions state files, encrypts state files at rest, and allows for fine-grained access control policies. But it also has a nice UI and enables VCS/API-driven workflow.
If you would like to learn more about Terraform Cloud or want to get started, I recommend reading the HashiCorp Learn tutorials on the subject (https://learn .hashicorp.com/collections/terraform/cloud-get-started).
6.7 Fireside chat
We’ve covered a lot of new information in this chapter. We started by talking about what a remote backend is, why it’s important, and how it can be used for collaboration purposes. Then we developed a module for deploying an S3 backend using a flat module design and published it on the Terraform Registry.
After we deployed the S3 backend, we looked at a few examples of how we can use it. The simplest was to deploy a null_resource, which didn’t really do anything but verified that the backend was operational. Next, we saw how we can deploy to multiple environments using workspaces. Essentially, you have different variables on your workspace, which configure providers and other environment settings, while your configuration code stays the same. It’s also worth mentioning that Terraform Cloud has its own unique take on workspaces, which are heavily inspired by the CLI implementation but are not exactly the same thing.
NOte Testing is an important part of collaboration and is something we did not get a chance to talk about in this chapter. However, we explore this topic in chapter 10.
Summary
-
An S3 backend is used for remotely storing state files. It’s made up of four components: a DynamoDB table, an S3 bucket and a KMS key, a least-priviliged IAM role, and housekeeping resources.
-
Flat modules organize code by using a lot of little .tf files rather than having nested modules. The pro is that they use less boilerplate, but the con is that it may be harder to reason about the code.
-
Modules can be shared through various means including S3 buckets, GitHub repos, and the Terraform Registry. You can also implement your own private module registry if you’re feeling adventurous.
-
Workspaces allow you to deploy to multiple environments. The configuration code stays the same; the only things that change are the variables and the state file.
-
Terraform Cloud is the SaaS version of Terraform Enterprise. Terraform Cloud has lower-priced options with fewer features, if price is a concern for you. But it even gives you a remote state store and allows you to perform VCS driven workflows.