- Developing a Terraform provider from scratch
- Implementing CRUD operations for managed resources
- Writing acceptance tests for the provider schema and resource files
- Deploying a serverless API to listen to requests from the provider
- Building and installing third-party providers
Extending Terraform by writing your own provider is one of the most satisfying things you can do with the technology. It demonstrates high-level proficiency and grants you the power to bend Terraform to your will. Nevertheless, even the simplest provider requires a considerable investment of time and effort. When might it be worth writing your own Terraform provider?
Two excellent reasons to write a provider are
Almost all Terraform providers wrap remote APIs because this is what they were designed to do. Recall from chapter 2 that Terraform Core is essentially a glorified state-management engine. Without Terraform providers, Terraform would not know how to provision cloud-based infrastructure. By creating a custom provider, you enable Terraform to manage more and new kinds of resources.
Exposing utility functions to Terraform is another reason to create a custom provider, although considerably less common. Utility functions include anything not supported by one of the built-in functions, such as zipping files (Archive provider), reading/writing files (Local provider), or creating random passwords (Random provider). Because creating your own provider has a lot of associated overhead, many people choose to implement utility functions with a local-exec provisioner or the Shell provider rather than writing a one-off provider.
In this chapter, we develop a Petstore provider by wrapping a remote Petstore API. Pets are data objects representing animal friends, with attributes such as name, species, and age. Our Petstore provider allows us to manage pets as code by exposing a petstore_pet resource that can create, read, update, and delete pets. Figure 11.1 depicts a pet resource deployed by the Petstore provider, as seen through the UI.
Figure 11.1 A pet resource provisioned with the Petstore provider, as seen through the UI
11.1 Blueprints for a Terraform provider
Although we’ve been using providers since chapter 1, we haven’t explained in much detail how they work. In this section, we go through the different parts of a provider and the surrounding ecosystem. By the end of this section, you will have a big-picture understanding of what we’ll implement in the next few sections.
11.1.1 Terraform provider basics
The primary purpose of any Terraform provider is to expose resources to Terraform and initialize shared configuration objects. Resources, as you already know, come in two flavors: managed and unmanaged. Managed resources are regular resources that implement create, read, update, delete (CRUD) methods for lifecycle management. Unmanaged resources, also known as data sources or read-only resources, are less complex and implement only the Read part of CRUD.
Shared configuration objects are exactly as the name suggests: configuration objects that are shared between resource entities, usually for optimization or authentication purposes. These can be things like client and database connections, mutexes (concurrency locks), and temporary access keys. Terraform always initializes these shared configuration objects before performing any CRUD actions.
NOTE If a provider fails or hangs during initialization, it is almost always due to a shared configuration object having invalid or expired credentials.
There are two prerequisites for creating your own provider that wraps a remote API:
-
Existing remote API—Since Terraform makes calls against a remote API, there must be an existing remote API to make calls to. This can be your own API or someone else’s.
-
Golang client SDK for the API—Providers are written in golang, so you should have a golang client SDK for your API in place before proceeding. This will save you from having to make ugly, raw HTTP requests against the API.
Tip Always have separate repositories for the client SDK and the provider! Providers are sufficiently complicated, and there’s no need to make it harder on yourself by combining SDK code with provider code.
Using a Terraform provider with a golang client SDK to talk to a remote API is shown in figure 11.2.
Figure 11.2 Terraform Core communicates with providers over RPC, which then uses a client SDK written in golang to make HTTP requests against a remote API.
11.1.2 Petstore provider architecture
In this chapter, we develop a custom Terraform Petstore provider from scratch. This provider is relatively simple, with minimal schema configuration, and it exports only a single resource, but it allows all the best practices and can be used as a template for developing new providers.
-
main.go—The entry point for the provider, which is mostly uninteresting boilerplate
-
petstore/provider.go—Contains the provider definition, resource mapping, and initialization of shared configuration objects
-
petstore/provider_test.go—A file for basic acceptance tests of the provider
-
petstore/resource_ps_pet.go—The pet resource that defines CRUD operations for managing a pet resource
-
petstore/resource_ps_pet_test.go—More basic acceptance tests, this time for the pet resource
The complete file structure is as follows:
$ tree . ├── main.go └── petstore ├── provider.go ├── provider_test.go ├── resource_ps_pet.go └── resource_ps_pet_test.go
Note Normally, provider authors create matching read-only resources (aka data sources) to complement managed resources. We will not do that here to save space, but you can find an example in appendix E.
As discussed previously, we need a remote API to make calls against a golang client SDK to wrap the API. The API will be handled by a serverless Petstore app deployed on AWS, adapted from one we deployed in chapter 4. We’ll use an SDK that I prepared in advance (https://github.com/terraform-in-action/go-petstore) because creating an SDK is largely tedious and uninteresting work, no matter what people say.
11.2 Writing the Petstore provider
In this section, we write all the functional code that goes in the Petstore provider. We’ll start by setting up the Go project’s entry point before configuring the provider schema and finally defining our pet resource. By the end of this section, we will have a complete provider—minus acceptance tests, which come in the next section.
11.2.1 Setting up the Go project
I will assume you have some familiarity with Go—but if you don’t, that’s ok. Golang is easy to understand, especially if you have previous experience with a scripting language like JavaScript or a C-based language like Java. The first thing you need to do when getting started with Go is create a new project under your GOPATH. The GOPATH environment variable specifics the location of your Go workspace, which is where all Go code is typically kept. If no GOPATH is set, it is assumed to be $HOME/go on Unix systems and %USERPROFILE%go on Windows. Under GOPATH are two subdirectories: src and bin. Create a new Go project by making an empty directory under src with a corresponding package directory for the Petstore provider. For example,
$ mkdir $GOPATH/src/github.com/terraform-in-action/terraform-provider-petstore
Note The package directory is based on a GitHub username. You may want to replace it with your own username.
Next, create a main.go file in this directory containing the following code.
package main ❶ import ( "github.com/hashicorp/terraform-plugin-sdk/v2/plugin" "github.com/terraform-in-action/terraform-provider-petstore/petstore" ❷ ) func main() { plugin.Serve(&plugin.ServeOpts{ ProviderFunc: petstore.Provider}) ❸ }
❶ Declares that the file is part of the main package
❷ Import local and external packages
The main.go file is the primary entry point for the plugin when Terraform invokes it. The first line, package main, declares that this file is part of the main package, which is the root Go package for any given project. There are two declared imports: one from the terraform-plugin-sdk and one locally referenced import.
After that comes the main function, func main() {...}, which is the first thing called when executing the binary. All this does is serve up the Petstore provider, which is a plugin implementing the terraform.ResourceProvider interface, as defined by the Terraform plugin SDK.
11.2.2 Configuring the provider schema
The provider schema defines the attributes for the provider configuration, exports resources, and initializes any shared configuration objects. All this takes place during the terraform init step when the provider is first installed.
We start by defining a Provider() function, which will return a terraform .ResourceProvider interface. The ResourceProvider interface has several mandatory fields; I always like to start with Schema. Not to be confused with the overall provider schema, Schema is a parameter that outlines the allowed provider configuration attributes in Terraform. Ultimately, this will let us declare our provider in HCL:
provider "petstore" {
address = var.address
}
I begin with Schema because the design of the provider configuration often influences the design of any resources or data sources implemented by the provider. Usually, what is passed into the provider configuration is for setting up shared configuration objects. Things like access keys, addresses, and other shared secrets are appropriate, whereas resource-specific data is not. Our provider configuration is easy enough, as there’s only a single attribute called address (of type string), which configures the endpoint of the Petstore server. Note that the Petstore API is unauthenticated; hence there is no need for shared secrets.
Warning You should always implement authentication for any production API and never bake secrets into the provider source code.
One more thing to mention about address is that we may wish to optionally set it with an environment variable rather than a Terraform variable so the provider can be run in automation. We can do this with the help of a prebuilt function from the plugin SDK called schema.EnvDefaultFunc. This function makes it possible to set a default environment variable if the attribute is not directly set in the provider configuration.
Tip It is a good idea to make critical configuration attributes, such as access keys and addresses, optionally configurable as environment variables for ease of use in automation.
Go ahead and create a petstore directory, and in it, create a provider.go file with the following code.
package petstore
import (
"net/url"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
sdk "github.com/terraform-in-action/go-petstore"
)
func Provider() *schema.Provider {
return &schema.Provider{
Schema: map[string]*schema.Schema{
"address": &schema.Schema{
Type: schema.TypeString,
Optional: true,
DefaultFunc: schema.EnvDefaultFunc("PETSTORE_ADDRESS", nil), ❶
},
},
}
}
❶ Allows attribute to be optionally set from an environment variable
Note The petstore directory can also be referred to as a golang package.
Any provider’s schema can be printed with the terraform providers schema command. An example of printing the Petstore provider’s schema is shown here:
{
"format_version": "0.1",
"provider_schemas": {
"registry.terraform.io/terraform-in-action/petstore": {
"provider": {
"version": 0,
"block": {
"attributes": {
"address": {
"type": "string",
"description_kind": "plain",
"optional": true
}
},
"description_kind": "plain"
}
},
"resource_schemas": {
"petstore_pet": {
"version": 0,
"block": {
"attributes": {
"age": {
"type": "number",
"description_kind": "plain",
"required": true
},
"id": {
"type": "string",
"description_kind": "plain",
"optional": true,
"computed": true
},
"name": {
"type": "string",
"description_kind": "plain",
"optional": true
},
"species": {
"type": "string",
"description_kind": "plain",
"required": true
}
},
"description_kind": "plain"
}
}
}
}
}
}
Now that we have the basic provider schema, we must register all resources that the provider exports to Terraform in a map structure. The map keys will be the names of the resources in Terraform, and the map value will be a pointer to schema.Resource objects. This map will have only a single resource, petstore_pet, which manages the lifecycle of a pet entity. We have not created it yet, but let’s preemptively add a function called resourcePSPet() that we define in the next section. Edit provider.go to add this resource map.
package petstore
import (
"net/url"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
sdk "github.com/terraform-in-action/go-petstore"
)
func Provider() *schema.Provider {
return &schema.Provider{
Schema: map[string]*schema.Schema{
"address": &schema.Schema{
Type: schema.TypeString,
Optional: true,
DefaultFunc: schema.EnvDefaultFunc("PETSTORE_ADDRESS", nil),
},
},
ResourcesMap: map[string]*schema.Resource{
"petstore_pet": resourcePSPet(),
},
}
}
Finally, we need to initialize shared configuration objects. For our purposes, this is the client that the SDK uses to make API requests against the Petstore server. The logic for doing this is encapsulated in the ConfigureFunc field of the provider schema. The output of this function is a shared configuration object that will be made available to all resources. The complete code for provider.go is shown next.
package petstore
import (
"net/url"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
sdk "github.com/terraform-in-action/go-petstore"
)
func Provider() *schema.Provider {
return &schema.Provider{
Schema: map[string]*schema.Schema{
"address": &schema.Schema{
Type: schema.TypeString,
Optional: true,
DefaultFunc: schema.EnvDefaultFunc("PETSTORE_ADDRESS", nil),
},
},
ResourcesMap: map[string]*schema.Resource{
"petstore_pet": resourcePSPet(),
},
ConfigureFunc: providerConfigure,
}
}
func providerConfigure(d *schema.ResourceData) (interface{}, error) {
hostname, _ := d.Get("address").(string)
address, _ := url.Parse(hostname)
cfg := &sdk.Config{
Address: address.String(),
}
return sdk.NewClient(cfg)
}
11.3 Creating a pet resource
The function resourcePSPet() returns a schema.Resource interface. Our pet resource is an implementation of this interface. As you might have guessed, four of the fields on this interface have to do with function hooks invoked during CRUD lifecycle management:
-
Create—A pointer to a function that’s invoked when a create lifecycle event is triggered. Create lifecycle events are triggered when new resources are created, such as during an initialapplyand during force-new updates. -
Read—A pointer to a function that’s invoked when a read lifecycle event is triggered. Read events are triggered during the generation of an execution plan to determine whether configuration drift has occurred. Additionally, theRead()function is typically called as a side effect ofCreate()andUpdate(). -
Update—A pointer to a function that’s invoked when an update lifecycle event is triggered. It handles in-place (aka non-destructive) updates. This field may be omitted if all attributes in the resource schema are marked asForceNew. -
Delete—A pointer to a function that’s invoked when a delete lifecycle event is triggered. Delete lifecycle events are triggered duringterraformdestroy; when a resource is removed from configuration (or marked as tainted), followed by aterraformapply; and when an attribute marked asForceNewhas been changed.
It’s important to know when each of the four CRUD functions will be invoked so you can predict and handle any errors. During an initial apply with no previous state, Terraform calls Create(), which has the side effect of calling Read(). During terraform plan, Read() is called by itself. During an in-place update, Read() is called first, like during the plan, and then Update() is called, which has the side effect of calling Read() again. Force-new updates call Read(), then Delete(), then Create(), and finally Read() again. Destroy operations always call Read() and then Delete(). Figure 11.3 is a reference diagram.
Figure 11.3 Different methods are invoked based on the command as well as the current state and configuration. Some methods (Create() and Update()) have the side effect of calling other methods (Read()).
Besides CRUD methods, the resource schema has another required field called Schema. Like the provider schema, this is a map of attributes that a resource defines. The type of each attribute must be specified, as well as whether the attribute is required, optional, or ForceNew. Our pet resource has three attributes : name, species, and age. name is an optional attribute because not all pets have names. species will be marked as required and ForceNew (because making a change to a pet’s species is kind of a big deal). age is an integer type that’s required but not marked as ForceNew, because it’s highly likely the pet will have a birthday in the future, meaning we have to update its age.
Let’s now define the function for the pet resource in a separate file called resource_ps_pet.go.
Listing 11.5 resource_ps_pet.go
package petstore
import (
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
sdk "github.com/terraform-in-action/go-petstore"
)
func resourcePSPet() *schema.Resource {
return &schema.Resource{
Create: resourcePSPetCreate,
Read: resourcePSPetRead,
Update: resourcePSPetUpdate,
Delete: resourcePSPetDelete,
Importer: &schema.ResourceImporter{
State: schema.ImportStatePassthrough,
},
Schema: map[string]*schema.Schema{
"name": {
Type: schema.TypeString,
Optional: true, ❶
Default: "",
},
"species": {
Type: schema.TypeString,
ForceNew: true, ❷
Required: true,
},
"age": {
Type: schema.TypeInt,
Required: true, ❸
},
},
}
}
❶ Not all pets have a name, so this is optional.
❷ All pets are part of a species.
❸ Pets have an age attribute that can be updated in-place.
Next, we will define the Create(), Read(), Update, and Delete() methods.
11.3.1 Defining Create()
Create() is a function responsible for provisioning a new resource based on user-supplied input and setting the resource’s unique ID. The ID is important because without it, the resource won’t be marked as created by Terraform, and it also will not be persisted to Terraform state. The implementation of Create() usually means performing a POST request against the remote API, waiting for a response, handling any retry logic, and invoking a Read() operation afterward.
Tip Although you could write the logic for performing a raw HTTP POST request inline in the Create() function, I do not recommend doing so. That’s what the client SDK is for.
Because we already have a Petstore client SDK (which encapsulates much of the tedious logic of interacting with the API), the Create() method becomes incredibly simple.
Listing 11.6 resource_ps_pet.go
package petstore
import (
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
sdk "github.com/terraform-in-action/go-petstore"
)
func resourcePSPet() *schema.Resource {
return &schema.Resource{
Create: resourcePSPetCreate,
Read: resourcePSPetRead,
Update: resourcePSPetUpdate,
Delete: resourcePSPetDelete,
Importer: &schema.ResourceImporter{
State: schema.ImportStatePassthrough,
},
Schema: map[string]*schema.Schema{
"name": {
Type: schema.TypeString,
Optional: true,
Default: "",
},
"species": {
Type: schema.TypeString,
ForceNew: true,
Required: true,
},
"age": {
Type: schema.TypeInt,
Required: true,
},
},
}
}
func resourcePSPetCreate(d *schema.ResourceData, meta interface{}) error {
conn := meta.(*sdk.Client) ❶
options := sdk.PetCreateOptions{
Name: d.Get("name").(string),
Species: d.Get("species").(string),
Age: d.Get("age").(int),
}
pet, err := conn.Pets.Create(options)
if err != nil {
return err
}
d.SetId(pet.ID) ❷
return resourcePSPetRead(d, meta) ❸
}
❶ Meta comes from the output of the provider configuration.
❷ The resource ID is set using a unique parameter from the response object.
❸ Best practice is to call Read() after Create().
11.3.2 Defining Read()
Read() is a non-destructive operation that retrieves the actual state of a resource from a remote API. It’s called whenever a refresh occurs and as a side effect of both Update() and Create(). Generally, Read() uses a unique resource ID to perform a lookup against the API, although it could also use a combination of other attributes to uniquely identify a resource. Regardless of how the lookup is done, the response from the API is considered authoritative. If the actual state doesn’t match the desired state, as described in the current configuration/state file, an update will be triggered during the subsequent apply.
Warning Read() should always return the same resource from the API. If it does not, you will end up with orphaned resources. Orphaned resources are resources that were originally created by Terraform but that have been lost track of and are now unmanaged.
Add the code from the following listing to the bottom of the resource_ps_pet.go file to implement Read(). This code uses the Petstore SDK to look up the pet resource based on ID, throw an error if one has occurred, and set the attributes based on the response from the API.
Listing 11.7 .resource_ps_pet.go
...
func resourcePSPetCreate(d *schema.ResourceData, meta interface{}) error {
conn := meta.(*sdk.Client)
options := sdk.PetCreateOptions{
Name: d.Get("name").(string),
Species: d.Get("species").(string),
Age: d.Get("age").(int),
}
pet, err := conn.Pets.Create(options)
if err != nil {
return err
}
d.SetId(pet.ID)
return resourcePSPetRead(d, meta)
}
func resourcePSPetRead(d *schema.ResourceData, meta interface{}) error {
conn := meta.(*sdk.Client)
pet, err := conn.Pets.Read(d.Id())
if err != nil {
return err
}
d.Set("name", pet.Name) ❶
d.Set("species", pet.Species) ❶
d.Set("age", pet.Age) ❶
return nil
}
❶ Setting resource attributes based on the remote or actual state
11.3.3 Defining Update()
Although Terraform is often touted as an immutable infrastructure as code technology (and I describe it as such in chapter 1), strictly speaking, it isn’t one. Almost all resources that Terraform manages are mutable to some degree. As a reminder, immutable infrastructure is the concept of never performing updates in place. If an update occurs, it takes place by tearing down the old infrastructure (such as a server) and replacing it with new infrastructure preconfigured to the desired state. By contrast, with mutable infrastructure, existing resources are allowed to persist through in-place updates or patches instead of resources being deleted and re-created. Only if every attribute on a resource is marked ForceNew (and almost no resource is this way) could the resource be described as immutable.
The purpose of Update() is to perform non-destructive, in-place updates on existing infrastructure. It’s a tricky method to implement, and it may be tempting to skip the need for it by marking all attributes as ForceNew, but I wouldn’t recommend doing this. Force-new updates are inconvenient from a user perspective because changes take longer to propagate. This is an example where a good user experience matters more than ease of development or strict adherence to infrastructure immutability.
The sole responsibility of Update() is to do whatever it takes to transform the actual state of a resource into the desired state. Typically, this means performing a PATCH request followed by a GET; but since we have a client SDK, we’ll use that instead of making raw HTTP requests. Add the following code to the bottom of resource_ps_pet.go to define and implement Update().
Listing 11.8 resource_ps_pet.go
...
func resourcePSPetRead(d *schema.ResourceData, meta interface{}) error {
conn := meta.(*sdk.Client)
pet, err := conn.Pets.Read(d.Id())
if err != nil {
return err
}
d.Set("name", pet.Name)
d.Set("species", pet.Species)
d.Set("age", pet.Age)
return nil
}
func resourcePSPetUpdate(d *schema.ResourceData, meta interface{}) error {
conn := meta.(*sdk.Client)
options := sdk.PetUpdateOptions{}
if d.HasChange("name") { ❶
options.Name = d.Get("name").(string)
}
if d.HasChange("age") { ❶
options.Age = d.Get("age").(int)
}
conn.Pets.Update(d.Id(), options) ❷
return resourcePSPetRead(d, meta) ❸
}
❶ Checks each non-ForceNew attribute to see if it has changed
❸ Like Create(), Update() needs to call Read() as a side effect.
11.3.4 Defining Delete()
The last lifecycle method to implement is Delete(). This method is responsible for making an API request to delete an existing resource and set its resource ID to nil (which marks the resource as destroyed and removes it from the state file). I always find Delete() the easiest method to implement, but it’s still important not to make any mistakes. If Delete() fails to delete (such as if the API experienced an internal error due to poor implementation), you will be left with orphaned resources.
Note You can call Read() after Delete() if you wish to ensure that a resource has actually been destroyed, but usually this is not done. Delete() is presumed to succeed if the response from the server says it has succeeded. Server errors should be handled by the server or SDK.
The code for Delete() is shown in the following listing.
Listing 11.9 resource_ps_pet.go
...
func resourcePSPetUpdate(d *schema.ResourceData, meta interface{}) error {
conn := meta.(*sdk.Client)
options := sdk.PetUpdateOptions{}
if d.HasChange("name") {
options.Name = d.Get("name").(string)
}
if d.HasChange("age") {
options.Age = d.Get("age").(int)
}
conn.Pets.Update(d.Id(), options)
return resourcePSPetRead(d, meta)
}
func resourcePSPetDelete(d *schema.ResourceData, meta interface{}) error {
conn := meta.(*sdk.Client)
err := conn.Pets.Delete(d.Id())
if err != nil {
return err
}
return nil
}
For your reference, the complete code for resource_ps_pet.go is presented next.
Listing 11.10 resource_ps_pet.go
package petstore
import (
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
sdk "github.com/terraform-in-action/go-petstore"
)
func resourcePSPet() *schema.Resource {
return &schema.Resource{
Create: resourcePSPetCreate,
Read: resourcePSPetRead,
Update: resourcePSPetUpdate,
Delete: resourcePSPetDelete,
Importer: &schema.ResourceImporter{
State: schema.ImportStatePassthrough,
},
Schema: map[string]*schema.Schema{
"name": {
Type: schema.TypeString,
Optional: true,
Default: "",
},
"species": {
Type: schema.TypeString,
ForceNew: true,
Required: true,
},
"age": {
Type: schema.TypeInt,
Required: true,
},
},
}
}
func resourcePSPetCreate(d *schema.ResourceData, meta interface{}) error {
conn := meta.(*sdk.Client)
options := sdk.PetCreateOptions{
Name: d.Get("name").(string),
Species: d.Get("species").(string),
Age: d.Get("age").(int),
}
pet, err := conn.Pets.Create(options)
if err != nil {
return err
}
d.SetId(pet.ID)
return resourcePSPetRead(d, meta)
}
func resourcePSPetRead(d *schema.ResourceData, meta interface{}) error {
conn := meta.(*sdk.Client)
pet, err := conn.Pets.Read(d.Id())
if err != nil {
return err
}
d.Set("name", pet.Name)
d.Set("species", pet.Species)
d.Set("age", pet.Age)
return nil
}
func resourcePSPetUpdate(d *schema.ResourceData, meta interface{}) error {
conn := meta.(*sdk.Client)
options := sdk.PetUpdateOptions{}
if d.HasChange("name") {
options.Name = d.Get("name").(string)
}
if d.HasChange("age") {
options.Age = d.Get("age").(int)
}
conn.Pets.Update(d.Id(), options)
return resourcePSPetRead(d, meta)
}
func resourcePSPetDelete(d *schema.ResourceData, meta interface{}) error {
conn := meta.(*sdk.Client)
err := conn.Pets.Delete(d.Id())
if err != nil {
return err
}
return nil
}
11.4 Writing acceptance tests
A provider isn’t complete until it’s been thoroughly tested. Tests are important because they give you the confidence to know that your code is working and (relatively) bug-free. Writing good tests can be tough, but it’s worth the effort. In this section, we write two test files: one for the provider schema and one for the pet resource.
Note Expect to include tests for any contribution you make to an open source provider.
11.4.1 Testing the provider schema
The primary purpose of testing the provider schema is to ensure that the provider
Note Sometimes people also test the individual attributes of the provider, along with various ways to configure the provider.
Create a provider_test.go file containing the following code.
Listing 11.11 provider_test.go
package petstore
import (
"context"
"testing"
"github.com/hashicorp/terraform-plugin-sdk/v2/diag"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
"github.com/hashicorp/terraform-plugin-sdk/v2/terraform"
)
var testAccProviders map[string]*schema.Provider
var testAccProvider *schema.Provider
func init() { ❶
testAccProvider = Provider()
testAccProviders = map[string]*schema.Provider{
"petstore": testAccProvider,
}
}
func TestProvider(t *testing.T) { ❷
if err := Provider().InternalValidate(); err != nil {
t.Fatalf("err: %s", err)
}
}
func TestProvider_impl(t *testing.T) { ❸
var _ *schema.Provider = Provider()
}
func testAccPreCheck(t *testing.T) { ❹
if os.Getenv("PETSTORE_ADDRESS") == "" {
t.Fatal("PETSTORE_ADDRESS must be set for acceptance tests")
}
if diags := Provider().Configure(context.Background(),
&terraform.ResourceConfig{}); diags.HasError() {
for _, d := range diags {
if d.Severity == diag.Error {
t.Fatalf("err: %s", d.Summary)
}
}
}
}
❶ Initializes global variables
❷ Tests that the provider schema is valid
❸ Tests that the provider can be initialized
❹ Tests that the PETSTORE_ADDRESS environment variable is set
11.4.2 Testing the pet resource
Writing a test for a Terraform resource is more difficult than writing tests for the provider schema because it requires utilizing a custom testing framework developed by HashiCorp. Don’t worry: you don’t have to know much about this testing framework to get through this scenario. However, the framework is worth looking into because it allows you to do cool stuff like run test sequences against resources with various configurations and run pre-processor and post-processor functions. It was tailor-made for testing Terraform resources and is certainly easier than rolling your own framework.
Although you can do a lot with resource testing, at a bare minimum you need the following:
-
A basic create/destroy test with validation that attributes get set in the state file
-
A function that tests the HCL configuration with all input attributes set
The test code for the pet resource is shown in the next listing. Copy it into a resource_ps_pet_test.go file under the petstore directory.
Listing 11.12 resource_ps_pet_test.go
package petstore
import (
"fmt"
"testing"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource"
"github.com/hashicorp/terraform-plugin-sdk/v2/terraform"
sdk "github.com/terraform-in-action/go-petstore"
)
func TestAccPSPet_basic(t *testing.T) { ❶
resourceName := "petstore_pet.pet"
resource.Test(t, resource.TestCase{
PreCheck: func() { testAccPreCheck(t) }, ❷
Providers: testAccProviders, ❸
CheckDestroy: testAccCheckPSPetDestroy, ❹
Steps: []resource.TestStep{ ❺
{
Config: testAccPSPetConfig_basic(),
Check: resource.ComposeTestCheckFunc(
resource.TestCheckResourceAttr(resourceName, "name",
"Princess"),
resource.TestCheckResourceAttr(resourceName, "species",
"cat"),
resource.TestCheckResourceAttr(resourceName, "age", "3"),
),
},
},
})
}
func testAccCheckPSPetDestroy(s *terraform.State) error { ❻
conn := testAccProvider.Meta().(*sdk.Client)
for _, rs := range s.RootModule().Resources {
if rs.Type != "petstore_pet" {
continue
}
if rs.Primary.ID == "" {
return fmt.Errorf("No instance ID is set")
}
_, err := conn.Pets.Read(rs.Primary.ID)
if err != sdk.ErrResourceNotFound {
return fmt.Errorf("Pet %s still exists", rs.Primary.ID)
}
}
return nil
}
func testAccPSPetConfig_basic() string {
return fmt.Sprintf(`
resource "petstore_pet" "pet" { ❼
name = "Princess"
species = "cat"
age = 3
}
`)
}
❶ Basic acceptance test for a Terraform resource
❷ PreCheck ensures that PETSTORE_ADDRESS has been set.
❸ Uses the global provider initialized with init()
❹ Ensures that the resource gets destroyed
❺ Simple test that creates a resource using a sample configuration and checks that the set attributes are as expected
❻ Destroy function implementation
❼ Function that returns a string containing resource configuration
11.5 Build, test, deploy
The code for the provider is now complete, but we still have a few tasks to do. First, we need an actual Petstore API to test against, then we need to test and build the provider binary, and finally we need to run end-to-end tests with real configuration code.
11.5.1 Deploying the Petstore API
For your convenience, I have packaged the API into a module that can easily be deployed with a few lines of Terraform code. This module deploys a serverless backend with an API gateway, a lambda function, and a Relational Database Service (RDS) database. It parallels the architecture of the serverless app deployed in chapter 5, except it’s on AWS rather than Azure. Basically, I took the web app deployed in chapter 4 and modified it to run on serverless technologies.
Following is the code for the Petstore module. Create a new, separate Terraform workspace with this file.
terraform {
required_version = ">= 0.15"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.28"
}
random = {
source = "hashicorp/random"
version = "~> 3.0"
}
}
}
provider "aws" {
region = "us-west-2"
}
module "petstore" {
source = "terraform-in-action/petstore/aws"
}
output "address" {
value = module.petstore.address
}
Deploy as usual by performing terraform init followed by terraform apply:
$ terraform init ... Terraform has been successfully initialized! $ terraform apply ... Plan: 24 to add, 0 to change, 0 to destroy. Changes to Outputs: + address = (known after apply) Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value:
After you confirm the apply, deploying the serverless application should take about 5-10 minutes. At the end, you will get the address for your deployed API:
Apply complete! Resources: 1 added, 0 changed, 0 destroyed. Outputs: address = https://tcln1rvts1.execute-api.us-west-2.amazonaws.com/v1
If you navigate to this address in the browser, it will redirect you to a simple web UI. Notice that the UI is empty to start with because there are no pets in the database yet (see figure 11.4).
Figure 11.4 Initially there are no pets in the database, so the web UI doesn’t show anything.
11.5.2 Testing and building the provider
Create a new Go module with go mod init, and then download dependencies with go mod get:
$ go mod init go: creating new go.mod: module github.com/terraform-in-action/terraform- provider-petstore $ go mod get go: finding module for package github.com/hashicorp/terraform-plugin- sdk/v2/plugin go: finding module for package github.com/hashicorp/terraform-plugin- sdk/v2/helper/schema go: finding module for package github.com/terraform-in-action/go-petstore go: found github.com/hashicorp/terraform-plugin-sdk/v2/plugin in github.com/hashicorp/terraform-plugin-sdk/v2 v2.4.0 go: found github.com/terraform-in-action/go-petstore in github.com/terraform-in-action/go-petstore v0.1.1
Now set TF_ACC to 1 to enable running acceptance tests:
$ export TF_ACC=1
Note TF_ACC is an environment variable required by design to prevent developers from incurring unintended charges when running tests (see http://mng .bz/ZY2P).
If we were to run the acceptance tests now, we would get an error because the PETSTORE_ADDRESS environment variable has not been set. This is due to the PreCheck function in TestAccPSPet_basic():
$ go test -v ./petstore
=== RUN TestProvider
--- PASS: TestProvider (0.00s)
=== RUN TestProvider_impl
--- PASS: TestProvider_impl (0.00s)
=== RUN TestAccPSPet_basic
provider_test.go:35: PETSTORE_ADDRESS must be set for acceptance tests
--- FAIL: TestAccPSPet_basic (0.00s)
FAIL
FAIL github.com/terraform-in-action/terraform-provider-petstore/petstore
0.354s
FAIL
To proceed, we must set PETSTORE_ADDRESS to the address of our deployed Petstore API. We need to do this because otherwise, Terraform will not know where to send requests:
$ export PETSTORE_ADDRESS=<your Petstore address>
Now the acceptance tests pass:
$ go test -v ./petstore
=== RUN TestProvider
--- PASS: TestProvider (0.00s)
=== RUN TestProvider_impl
--- PASS: TestProvider_impl (0.00s)
=== RUN TestAccPSPet_basic
--- PASS: TestAccPSPet_basic (2.89s)
PASS
ok github.com/terraform-in-action/terraform-provider-petstore/petstore
3.082s
Since the tests pass, the provider is ready to be built. You can do that with go build:
$ go build
The binary will appear in your working directory:
$ ls -o
total 56976
-rw-r--r-- 1 swinkler 216 Jan 20 19:56 go.mod
-rw-r--r-- 1 swinkler 45873 Jan 20 19:56 go.sum
-rw-r--r-- 1 swinkler 337 Jan 20 21:20 main.go
drwxr-xr-x 6 swinkler 192 Jan 20 21:21 petstore
-rwxr-xr-x 1 swinkler 29108564 Jan 20 22:26 terraform-provider-petstore
Tip Most provider authors use a Makefile and CI triggers to automate the steps of building, testing, and distributing the provider. I recommend looking at some simpler providers, like terraform-provider-null and terraform -provider-tfe, for inspiration.
11.5.3 Installing the provider
There are a few different ways to install custom providers, as described on HashiCorp’s website (http://mng.bz/RKAK). For development providers, the easiest method is to edit your Terraform CLI configuration file (.terraformrc) to point to a directory containing your developer provider plugins. Let’s do that now.
The CLI configuration is a single file named terraform.rc on Windows and .terraformrc on Linux or Mac. It applies per-user settings for CLI behaviors across all Terraform working directories. Add the following code to override where Terraform looks to install the Petstore plugin.
provider_installation {
dev_overrides {
"terraform-in-action/petstore" =
"PATH/TO/DIRECTORY/WITH/PETSTORE/BINARY" ❶
}
direct {} ❷
}
❶ Overrides the location of the Petstore plugin
❷ Allows all other providers to be downloaded from the registry as usual
11.5.4 Pets as code
Now we are ready to manage pets as code. Create a new Terraform workspace with a main.tf file.
terraform {
required_providers {
petstore = {
source = "terraform-in-action/petstore"
version = "~> 1.0"
}
}
}
provider "petstore" {
address = "https://tcln1rvts1.execute-api.us-west-2.amazonaws.com/v1" ❶
}
resource "petstore_pet" "pet" {
name = "snowball"
species = "cat"
age = 20
}
❶ Your provider address goes here.
Initializing Terraform installs the Petstore provider plugin from the directory specified in .terraformrc:
$ terraform init Initializing the backend... Initializing provider plugins... - Reusing previous version of terraform-in-action/petstore from the dependency lock file - Installing terraform-in-action/petstore v1.0.0... - Installed terraform-in-action/petstore v1.0.0 (self-signed, key ID 37082CDD8344B056) Partner and community providers are signed by their developers. If you'd like to know more about provider signing, you can read about it here: https://www.terraform.io/docs/plugins/signing.html Warning: Provider development overrides are in effect ❶ The following provider development overrides are set in the CLI configuration: - terraform-in-action/petstore in /Users/swinkler/go/src/github.com/terraform-in-action/terraform-provider- petstore The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with published releases. Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary.
❶ Developer override for the provider plugin
Now that Terraform has detected the provider version and installed it successfully, run an apply in the workspace:
$ terraform apply Warning: Provider development overrides are in effect The following provider development overrides are set in the CLI configuration: - terraform-in-action/petstore in /Users/swinkler/go/src/github.com/terraform-in-action/terraform-provider- petstore The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with published releases. An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # petstore_pet.pet will be created + resource "petstore_pet" "pet" { + age = 7 + id = (known after apply) + name = "snowball" + species = "cat" } Plan: 1 to add, 0 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value:
As you can see, Terraform recognizes our provider as valid and plans to create a new pet resource! Confirm the apply to proceed:
petstore_pet.pet: Creating...
petstore_pet.pet: Still creating... [10s elapsed]
petstore_pet.pet: Creation complete after 11s [id=1308d843-337f-4fc4-8eb6-
3e522553d217]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Note It can take up to 30 seconds for the initial API request to succeed due to the serverless nature of the API. Once the lambda function has warmed up, the request time will be much faster. If you are still not getting a response after 30 seconds, it could be an error with the API request/response—turning on trace-level logs with TF_LOG=TRACE may help identify the problem.
The resource now exists as a record in the Petstore database. You can view it by navigating to the UI again and verifying that a new resource exists (see figure 11.5).
Figure 11.5 The provisioned pet resource, as seen in the UI
Note Another way to verify that the resource exists is to query the raw API: for example, using a GET against https:/./tcln1rvts1.execute-api.us-west-2.amazonaws.com/v1/api/pets.
The resource has been recorded in the state file, which we can view with terraform state show:
$ terraform state show petstore_pet.pet
# petstore_pet.pet:
resource "petstore_pet" "pet" {
age = 7
id = "1308d843-337f-4fc4-8eb6-3e522553d217"
name = "snowball"
species = "cat"
}
If we make changes to the configuration code, such as incrementing age from 7 to 8, we get the following message during the next apply:
$ terraform apply petstore_pet.pet: Refreshing state... [id=1308d843-337f-4fc4-8eb6-3e522553d217] An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: ~ update in-place Terraform will perform the following actions: # petstore_pet.pet will be updated in-place ~ resource "petstore_pet" "pet" { ~ age = 7 -> 8 id = "1308d843-337f-4fc4-8eb6-3e522553d217" name = "snowball" # (1 unchanged attribute hidden) } Plan: 0 to add, 1 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value:
After updating, clean up by deleting the resource from the API with terraform destroy:
$ terraform destroy -auto-approve petstore_pet.pet: Destroying... [id=1308d843-337f-4fc4-8eb6-3e522553d217] petstore_pet.pet: Destruction complete after 0s Destroy complete! Resources: 1 destroyed.
This concludes the scenario. Don’t forget to tear down the Petstore API with terraform destroy!
11.6 Fireside chat
In this chapter, we developed a custom Terraform Petstore provider (http://mng.bz/2zX0). The Petstore provider invokes a remote API with a client SDK written in go-lang. Instead of directly calling the API to provision resources, customers can now use Terraform to manage their pets as code.
Custom providers work best with micro APIs and self-service platforms. If you are a service owner, you probably already make your service available to customers through a RESTful API. Unfortunately, most customers do not want to go through the trouble of learning how to authenticate against an API and provision resources. This can lower the adoption rate of even a great self-service platform. By writing a Terraform provider for your API, you make it easy for people to start using your API with little or no knowledge of the API or underlying protocols and procedures.
Before ending the chapter, I want to cover some commonly asked questions about developing Terraform providers:
-
How do I create a data source? See appendix E on this topic. It was omitted here for length reasons.
-
How do I publish providers? There are a few steps to publishing a provider. First, you need to register the provider at
registry.terraform.io. You also need to create markdown documentation that will appear on the website, create a GitHub release using semantic versioning, and publish using CI/CD, typically via a GitHub action calling a GoReleaser script. Refer to the official documentation for more information (http://mng.bz/1Azj) or review the Petstore provider source code on GitHub for an example implementation (http://mng.bz/2zX0). -
How do I implement a private provider registry? Although most providers are distributed using the public provider registry, you can create your own private provider registry by implementing the provider registry protocol (http://mng.bz/JvyV). This could make sense for in-house providers that you do not want to make available to the general public.
-
How do I handle errors and implement retry logic and timeouts? The Petstore provider doesn’t handle edge cases as well as it could. Although this logic could be self-contained within the client SDK, I recommend keeping the client SDK as streamlined as possible and making error-handling the provider’s responsibility. You can see examples in the HashiCorp documentation (http://mng.bz/w0BP) or by reviewing source code from existing providers such as the AWS and Azure providers.
Summary
-
Terraform providers make it easy for people to use APIs without knowing how they work. In this spirit, you should always design providers to be as user friendly as possible.
-
Providers expose resources and data sources to Terraform. These are implemented as functions referenced by the provider schema.
-
Managed resources implement CRUD operations: create, read, update, and delete. These methods are invoked when the relevant lifecycle event is triggered.
-
Acceptance testing means writing tests for the provider schema and any resources exposed by the provider. Acceptance testing hardens code and is crucial for production readiness.
-
A provider is built like any other golang program. You should set up a CI/CD pipeline to automate building, testing, publishing, and distributing the provider.