6
Deploying a Traditional Three-Tier Architecture
We have covered the fundamentals of Terraform for Google Cloud using simple examples so far in this book. We will apply those fundamentals in the next three chapters to build complete architectures. In this chapter, we will build a traditional three-tier architecture with virtual machines for the application layer and Cloud SQL for the database layer. In the next chapter, we will create a completely serverless architecture using Cloud Run and Redis. In Chapter 8, Deploying GKE Using Public Modules, we will then provision a Google Kubernetes Engine cluster using public repositories.
In this chapter, we will cover the following topics:
- Laying the foundation
- Provisioning a complete database using Cloud SQL
- Deploying a MIG and global load balancer
Technical requirements
We recommend creating a fresh new Google Cloud project for this sample code. As before, we need to create the bucket for the remote Terraform state file outside of Terraform using the web console or a gcloud command. The code in the GitHub repository – https://github.com/PacktPublishing/Terraform-for-Google-Cloud-Essential-Guide/tree/main/chap06 – will work for a new project. Just remember to run terraform destroy afterward, as the cloud resources will incur costs.
If you use a service account for Terraform, you need to set the appropriate IAM permission, including Project IAM Admin and Secret Manager Admin permissions.
Overview
Some may not consider a traditional three-tier architecture consisting of a load balancer, a managed instance group (MIG) of virtual machines as the application layer, and Cloud SQL for the database tier a cloud-native architecture. Yet, it is still a pervasive architecture and a good starting point to apply the concepts we have learned about so far. Figure 6.1 shows the diagram of the architecture that we will provision in this chapter:
Figure 6.1 – Three-tier architecture
Following Google Cloud best practices, we use a custom VPC with minimal subnets and firewall rules. As we are catering to HTTP traffic, we are utilizing a global load balancer, which acts as the entry point and direct traffic to virtual machines that host our application code.
We deploy the virtual machines (compute engine) in a MIG to allow scaling. For the database tier, we create a Cloud SQL instance and use Secret Manager to store the connection information, which the application code can access to connect to the database.
As described in the previous chapter, we use a layered approach to provision the resources. Each layer, or subdirectory, has its own state file, and we use the terraform_remote_state data source to expose the state information of the different layers. In the first layer, we enable the required APIs and configure the custom VPC, along with the appropriate firewall rules. Next, we provision the database instance, the database, and the database user. Finally, we provision the MIG and the load balancer.
Note
Please note that we made some simplifications to reduce the cost and complexity of the sample code. These include provisioning the database in a single zone with a public IP address and using HTTP instead of HTTPS to connect to our application.
Laying the foundation
Note
The code for this section is under the chap06/foundation directory in the GitHub repository of this book. Please note that we divided the code into small files for readability purposes.
In the first layer, we set the foundation for this architecture. First, we enable the appropriate APIs using the google_project_service resource. For this project, we require the following APIs:
- Cloud Resource Manager
- Compute Engine
- Identity and Access Management
- Secret Manager API
The google_project_service resource has an optional argument, disable_on_destroy, which is set to true by default. It is usually better to set it to false, which mimics the behavior when we enable APIs using the web console.
As we are creating multiple google_project_service resources, this is an ideal opportunity to use the for_each meta-argument. for_each requires a set – we use the toset() function which converts a list to a set. This function is useful in conjunction with the for_each meta-argument, as toset() removes any duplicates and discards the ordering.
chap06/foundation/project-services.tf
resource "google_project_service" "this" {
for_each = toset(var.services)
service = "${each.key}.googleapis.com"
disable_on_destroy = false
}In this layer, we also provision the VPC, including the firewall rules. As we only use one region, we can set up a simple VPC with only one subnet. Following the best security protocols, we provision stringent firewall rules. The global load balancer handles all inbound traffic, so we do not need to provision any firewall rule to allow http inbound traffic. However, the load balancer requires a firewall rule to allow ingress for the health checks (for more details, refer to – https://cloud.google.com/load-balancing/docs/health-checks#firewall_rules).
Now, this firewall rule allows ingress from specific IP ranges (35.191.0.0/16 and 130.211.0.0/22). We could hardcode those IP ranges as in the documentation. However, using Terraform, we can use a data source to retrieve those IP ranges dynamically. The google_netblock_ip_ranges (https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/netblock_ip_ranges) data source retrieves the unique IP ranges used by Google Cloud, and we can use it to retrieve the IP ranges for the health checkers.
During development, we want to provide ssh access to the virtual machines. Still, we want to restrict access to authorized users only, and we use Google Cloud Identity-Aware Proxy (IAP) to do so. If you are not aware of IAP, be sure to check out the documentation at https://cloud.google.com/iap. IAP is part of the Google Cloud security framework and enables virtual machines to access without the need for a VPN or bastion host.
To allow access to virtual machines using IAP, we need to set up a firewall rule to allow ssh access from a specific set of source IP ranges. We can use the google_netblock_ip_ranges data source to retrieve those specific IP ranges used in the firewall rule:
chap06/foundation/data.tf
data "google_netblock_ip_ranges" "iap_forwarders" {
range_type = "iap-forwarders"
}
data "google_netblock_ip_ranges" "health_checkers" {
range_type = "health-checkers"
}Thus, we use two google_netblock_ip_ranges data sources to retrieve the two IP ranges and then use them to configure the firewall rules, shown as follows:
chap06/foundation/firewall.tf
resource "google_compute_firewall" "allow_iap" {
name = "${local.network_name}-allow-iap"
network = local.network_name
allow {
protocol = "tcp"
ports = ["22"]
}
source_ranges = data.google_netblock_ip_ranges.iap_forwarders.cidr_blocks_ipv4
target_tags = ["allow-iap"]
}
resource "google_compute_firewall" "allow_health_check" {
name = "${local.network_name}-allow-health-check"
network = local.network_name
allow {
protocol = "tcp"
ports = ["80"]
}
source_ranges = data.google_netblock_ip_ranges.health_checkers.cidr_blocks_ipv4
target_tags = ["allow-health-check"]
}Following the security principle of least privilege, the virtual machines in the instance group use a dedicated service account that only has the required permission to complete its tasks. In this case, we require the cloudsql.client and secretmanager.secretAccessor permissions, so we need to create a service account and assign the appropriate roles. Terraform provides two resources to assign IAM roles, google_project_iam_binding and google_project_iam_member (read about them in detail here – https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/google_project_iam).
The main difference between the two resources is that the former is authoritative, whereas the latter is non-authoritative. That is, google_project_iam_binding overwrites any other IAM permission, whereas google_project_iam_member is additive, which preserves any existing IAM permission. In general, it is better to use google_project_iam_member or the use IAM module from Google (read more about the best practices here – https://cloud.google.com/docs/terraform/best-practices-for-terraform#iam):
chap06/foundation/sa.tf
resource "google_service_account" "this" {
account_id = var.sa_name
display_name = "${var.sa_name} Service Account"
}
resource "google_project_iam_member" "this" {
project = var.project_id
count = length(var.roles)
role = "roles/${var.roles[count.index]}"
member = "serviceAccount:${google_service_account.this.email}"
}Now that we have enabled all the required APIs, set up the VPC, including the firewall rules, and provisioned the service account, we can proceed to the second layer, creating the database and the database users.
If you have created a new project for this exercise, we recommend manually removing the default VPC network. Note that we only allow ssh access via IAP – as in, the specific IAP ports, and not via 0.0.0.0/0 as in the default VPC. This is a recommended security practice in Google Cloud.
Provisioning the database
Note
The code for this section is in the chap06/database directory in the GitHub repository of this book.
In the previous chapter, we provisioned a Cloud SQL instance to demonstrate the use of a remote state. We will now provision a complete database along with a user and password. Following good security practices, we use Google Cloud Secret Manager to securely store the passwords to retrieve them in the application layer.
Note
While Google Cloud Secret Manager stores the secret in an encrypted fashion, the secret remains in plaintext in the Terraform state file, so take extra care to protect the state file from unwarranted access.
So, first, we generate the root and user password using the random_password Terraform resource. Next, we store the generated passwords in the secret manager. We need to use two resources each – google_secret_manager_secret to provision the secret and google_secret_manager_secret_version to store the actual password:
chap06/database/secrets.tf
resource "random_password" "root" {
length = 12
special = false
}
resource "google_secret_manager_secret" "root_pw" {
secret_id = "db-root-pw"
replication {
automatic = true
}
}
resource "google_secret_manager_secret_version" "root_pw_version" {
secret = google_secret_manager_secret.root_pw.id
secret_data = random_password.root.result
}
resource "random_password" "user" {
length = 8
special = false
}
resource "google_secret_manager_secret" "user_pw" {
secret_id = "db-user-pw"
replication {
automatic = true
}
}
resource "google_secret_manager_secret_version" "user_pw_version" {
secret = google_secret_manager_secret.user_pw.id
secret_data = random_password.user.result
}Once we have created and stored the passwords, we can create the database instance, database, and database user. We used the remote state in the previous chapter to pass the connection name to the application layer. However, we can also store the connection name in the secret manager and pass it as a secret to the application code:
chap06/database/main.tf
resource "random_string" "this" {
length = 4
upper = false
special = false
}
resource "google_sql_database_instance" "this" {
name = "${var.db_settings.instance_name}-${random_string.this.result}"
database_version = var.db_settings.database_version
region = var.region
root_password = random_password.root.result
settings {
tier = var.db_settings.database_tier
}
deletion_protection = false
}
resource "google_sql_database" "this" {
name = var.db_settings.db_name
instance = google_sql_database_instance.this.name
}
resource "google_sql_user" "sql" {
name = var.db_settings.user_name
instance = google_sql_database_instance.this.name
password = random_password.user.result
}
resource "google_secret_manager_secret" "connection_name" {
secret_id = "connection-name"
replication {
automatic = true
}
}
resource "google_secret_manager_secret_version" "connection_name" {
secret = google_secret_manager_secret.connection_name.id
secret_data = google_sql_database_instance.this.connection_name
}Please note that to reduce the Terraform code’s complexity, we configure the Cloud SQL instance with a public IP address and access it via the SQL proxy (https://cloud.google.com/sql/docs/mysql/sql-proxy ) in our sample code. Please see the documentation at https://cloud.google.com/sql/docs/mysql/connect-overview for the various options to connect to Cloud SQL.
Now that we have completed the database provisioning, we can proceed to the last layer of provisioning the load balancer and the MIG.
Provisioning a MIG and global load balancer
Note
The code for this section is under the chap06/main directory in the GitHub repository of this book.
To create a MIG, we first create an instance template, and then the MIG, which uses the instance template. The instance group is analogous to creating a virtual machine. Please note that we specify the service account we created and set the scopes to cloud-platform. That follows Google Cloud best practices: https://cloud.google.com/compute/docs/access/service-accounts#scopes_best_practice.
Now, here, we can put the create_before_destroy life cycle rule to good use. Let’s say we want to change the instance template by, for example, changing the startup script. Usually, Terraform destroys the google_compute_instance_template resource before applying the change and creating a new resource. However, Google Cloud won’t let us destroy google_compute_instance_template because the MIG still uses it. Hence, we use the create_before_destroy life cycle meta-argument to instruct Terraform to create a new template, apply that to the MIG, and only then destroy the old one. However, there is one more thing we need to do. If we declare a fixed name for the instance template, the create_before_destroy life cycle would report a naming conflict, as it would try to create a second instance template with the same name. Hence, instead of specifying a name for google_compute_instance_template, we use the name_prefix attribute shown as follows:
chap06/main/mig.tf
resource "google_compute_instance_template" "this" {
name_prefix = var.mig.instance_template_name_prefix
region = var.region
machine_type = var.mig.machine_type
disk {
source_image = var.mig.source_image
}
network_interface {
subnetwork = data.terraform_remote_state.foundation.outputs.subnetwork_self_links["iowa"]
access_config {
// Ephemeral public IP
}
}
metadata_startup_script = file("startup.sh")
tags = [
"allow-iap",
"allow-health-check"
]
service_account {
email = data.terraform_remote_state.foundation.outputs.service_account_email
scopes = ["cloud-platform"]
}
lifecycle {
create_before_destroy = true
}
}As we want our instance group to be high availability (HA), we create a regional MIG. In this example, we want to apply updates automatically, so we choose proactive updates (https://cloud.google.com/compute/docs/instance-groups/rolling-out-updates-to-managed-instance-groups#type). Thus, as soon as a change is detected in the instance template, Google Cloud updates the virtual machines:
chap06/main/mig.tf
resource "google_compute_region_instance_group_manager" "this" {
name = var.mig.mig_name
region = var.region
base_instance_name = var.mig.mig_base_instance_name
target_size = var.mig.target_size
version {
instance_template = google_compute_instance_template.this.id
}
named_port {
name = "http"
port = 80
}
update_policy {
type = "PROACTIVE"
minimal_action = "REPLACE"
max_surge_fixed = 3
}
}Defining the global load balancer is a four-step process analogous to creating it using the web console. First, we need to create a frontend configuration using the google_compute_global_forwarding_rule resource. Next, we need to define a backend service (google_compute_backend_service), along with a health check (google_compute_health_check). Then, lastly, we create the path rules using the google_compute_url_map resource:
chap06/main/lb.tf
resource "google_compute_global_forwarding_rule" "this" {
name = var.load_balancer.forward_rule_name
ip_protocol = "TCP"
load_balancing_scheme = "EXTERNAL"
port_range = "80"
target = google_compute_target_http_proxy.this.self_link
}
resource "google_compute_health_check" "this" {
name = "http-health-check"
http_health_check {
port = 80
}
}
resource "google_compute_backend_service" "this" {
name = var.load_balancer.backend_service_name
health_checks = [google_compute_health_check.this.self_link]
load_balancing_scheme = "EXTERNAL"
backend {
balancing_mode = "UTILIZATION"
group = google_compute_region_instance_group_manager.this.instance_group
}
}
resource "google_compute_url_map" "this" {
name = var.load_balancer.url_map_name
default_service = google_compute_backend_service.this.self_link
}
resource "google_compute_target_http_proxy" "this" {
name = var.load_balancer.target_proxy_name
url_map = google_compute_url_map.this.self_link
}Now, we can run Terraform and then access the resulting website using the IP address shown in the output.
Note
Please note that the global load balancer takes a few minutes to become fully active.
We included some sample code to demonstrate the use of accessing the secret manager during startup time, and the use of an SQL proxy (https://cloud.google.com/sql/docs/mysql/sql-proxy) to use the access database. Thus, we can ssh into the instance from the web console and cut and paste the code shown into the command line of your ssh session to connect to the database. We can retrieve the password using the web console or the following command:
$ gcloud secrets versions access latest --secret="db-user-pw"
We see that we can successfully connect to the database without any hardcoded connection information. Please note that this is for demonstration purposes only.
Now, let’s make a change in the startup.sh file by changing the version number in the HTML part from V1.0 to V2.0, and run Terraform again. Terraform replaces google_compute_instance_template but creates the new instance template first before destroying the existing one due to the life cycle rule. If we go into the web console, we can observe the changes. The number of virtual machines temporarily increases to six instances.
Summary
In this chapter, we applied the concepts learned in the previous chapters. We used a layered approach to deploy a traditional three-tier architecture. In the first layer, we applied the for_each meta-attribute to efficiently enable multiple project services using only four lines of code. We also used a data source to retrieve two special IP address ranges from Google Cloud.
In the second layer, we created a complete database. We used Terraform to generate passwords and store them in Google Cloud Secret Manager so the application layer could retrieve them. In the third layer, we used the life cycle meta-attribute to deploy new instance templates. We also showed how we can store sensitive information in the secret manager that the application code can retrieve.
Now that we have used Terraform to provision a traditional architecture, we will deploy a modern, highly scalable cloud architecture using only serverless cloud resources in the next chapter.