Chapter 5. Building Infrastructure Stacks as Code
In Chapter 3, I said that an infrastructure platform is a pool of infrastructure resources that you can provision and change on demand using an API. In Chapter 4, I explained the value of using code to define the infrastructure for your system. This chapter puts these together by describing patterns for implementing code to manage infrastructure.
The concept that I use to talk about doing this is the infrastructure stack.
What is an infrastructure stack?
An Infrastructure Stack is a collection of infrastructure resources that you define, provision, and update as a unit.
You write source code to define the elements of a stack, which are resources and services that your infrastructure platform provides. For example, your stack may include a virtual machine (“Compute Resources”), disk volume (“Storage Resources”), and a subnet (“Network Resources”).
You run a stack management tool, which reads your stack source code and uses the cloud platform’s API to assemble the elements defined in the code to provision an instance of your stack.
Figure 5-1. An infrastructure stack is a collection of infrastructure elements managed as a group.
Examples of stack management tools include:
Some server configuration tools (which I’ll talk about much more in [Link to Come]) have extensions to work with infrastructure stacks. Examples of these are Ansible Cloud Modules, Chef Provisioning (now end-of-lifed1), Puppet Cloud Management, and Salt Cloud.
“Stack” as a term
Most stack management tools don’t call themselves stack management tools. Each tool has its own terminology to describe the unit of infrastructure that it manages. In this book, I’m describing patterns and practices that should be relevant for any of these tools.
I’ve chosen to use the word “stack.”
Various people have told me there is a far better term for this concept than “stack.” Each of these people had a completely different word in mind. As of this writing, there is no agreement in the industry as to what to call this thing. So until there is, I’ll continue to use the word “stack.”
Stack code
Each stack is defined by source code that declares what infrastructure elements it should include. Terraform code (.tf files) and CloudFormation templates are both examples of infrastructure stack code. A stack project contains the source code that defines the infrastructure for a stack.
Example 5-1 shows the folder structure for a stack source code project. The language and tool for this example are fictitious. I use pseudo-code examples throughout this chapter.
Example 5-1. Project folder structure of a stack project using a fictitious tool
stack-project/├── src/│ ├── dns.infra│ ├── load_balancers.infra│ ├── networking.infra│ └── webserver.infra└── test/
Stack instance
You can use a single stack project to provision more than one stack instance. When you run the stack tool for the project, it uses the platform API to ensure the stack instance exists, and to make it match the project code. If the stack instance doesn’t exist, the tool creates it. If the stack instance exists but doesn’t exactly match the code, then the tool modifies the instance to make it match.
I often describe this process as “applying” the code to an instance.
If you change the code and rerun the tool, it changes the stack instance to match your changes. If you run the tool one more time without making any changes to the code, then it should leave the stack instance as it was.
Configuring servers in a stack
Infrastructure codebases for systems that aren’t fully container-based or serverless application architecture tend to include much code to provision and configure servers. Even container-based systems need to build host servers to run containers. The first mainstream infrastructure as code tools, like CFEngine, Puppet, and Chef, were used to configure servers.
You should decouple code that builds servers from code that builds stacks. Doing this makes the code easier to understand, simplifies changes by decoupling them, and supports reusing and testing server code.
Stack code typically specifies what servers to create, and passes information about the environment they will run in, by calling a server configuration tool. Example 5-2 is an example of a stack definition that calls the fictitious servermaker tool to configure a server.
Example 5-2. Example of a stack definition calling a server configuration tool
virtual_machine:name:appserver-burgerbarn-${environment}source_image:foodspin-base-appservermemory:4GBprovision:tool:servermakerparameters:maker_server:maker.foodspin.iorole:appserverenvironment:${environment}
This stack defines an application server instance, created from a server image called foodspin-appserver, with 4 GB of RAM. The definition includes a clause to trigger a provisioning process that runs servermaker. The code also passes several parameters for the servermaker tool to use. These parameters include the address of a configuration server (maker_server), which hosts configuration files, and a role, appserver, which servermaker uses to decide which configurations to apply to this particular server. It also passes the name of the environment, which the configurations can use to customize the server.
[Link to Come] describes patterns for managing servers as code.
Patterns and antipatterns for structuring stacks
One challenge with infrastructure design is deciding how to size and structure stacks. You could create a single stack code project to manage your entire system. But this becomes unwieldy as your system grows. In this section, I’ll describe patterns and antipatterns for structure infrastructure stacks.
The following patterns all describe ways of grouping the pieces of a system into one or more stacks. You can view them as a continuum:
-
A monolithic stack puts an entire system into one stack,
-
An application group stack groups multiple, related pieces of a system into stacks,
-
A service stack puts all of the infrastructure for a single application into a single stack,
-
A microstack breaks the infrastructure for a given application or service into multiple stacks.
Antipattern: Monolithic Stack
A Monolithic Stack is an infrastructure stack that includes too many elements, making it difficult to maintain.
Figure 5-2. A Monolithic Stack is an infrastructure stack that includes too many elements, making it difficult to maintain.
What distinguishes a monolithic stack from other patterns is that the number or relationship of infrastructure elements within the stack is difficult to manage well.
Also Known As
Spaghetti stack, big ball of mud
Motivation
People build monolithic stacks because the simplest way to add a new element to a system is to add it to the existing project. Each new stack adds more moving parts, which may need to be orchestrated, integrated, and tested. A single stack is simpler to manage.
Applicability
A monolithic stack may be appropriate when your system is small and simple. It’s not appropriate when your system grows, taking longer to provision and update.
Consequences
Changing a large stack is riskier than changing a smaller stack. More things that can go wrong-it has a larger blast radius. The impact of a failed change may be broader since there are more services and applications within the stack. Larger stacks are also slower to provision and change, which makes them harder to manage.
As a result of the speed and risk of changing a monolithic stack, people tend to make changes less frequently and take longer to do it. This added friction can lead to higher levels of technical debt.
Blast Radius
The term blast radius3 describes the potential damage a given change could make to a system. It’s usually based on the elements of the system you’re changing, what other elements depend on them, and what elements are shared.
Implementation
You build a monolithic stack by creating an infrastructure stack project and then continuously adding code, rather than splitting it into multiple stacks.
Related patterns
The opposite of a monolithic stack is a micro stack (“Pattern: Micro Stack”), which aims to keep stacks small so that they are easier to maintain and improve. A monolithic stack may be an application group stack “Pattern: Application Group Stack” that has grown out of control.
Pattern: Application Group Stack
An Application Group Stack includes the infrastructure for multiple related applications or services. The infrastructure for all of these applications is provisioned and modified as a group.
Figure 5-3. An Application Group Stack hosts multiple processes in a single instance of the stack.
For example, an online shopping company’s product application group may include separate services for browsing products, searching for products, and managing a shopping basket. An application group stack could provide the infrastructure that runs all three of these services.
Also Known As
Combined stack, service group stack, multi-application stack.
Motivation
Defining the infrastructure for multiple related services together can make it easier to manage the application as a single unit.
Applicability
This pattern can work well when a single team owns the infrastructure and deployment of all of the pieces of the application. An application group stack can align the boundaries of the stack to the team’s responsibilities.
Multi-service stacks are sometimes useful as an incremental step from a monolithic stack to service stacks.
Consequences
Grouping the infrastructure for multiple applications together also combines the time, risk, and pace of changes. The team needs to manage the risk to the entire stack for every change, even if only one part is changing. This pattern is inefficient if some parts of the stack change more frequently than others.
The time to provision, change, and test a stack is based on the entire stack. So again, if it’s common to change only one part of a stack at a time, having it grouped adds unnecessary overhead and risk.
Implementation
To create an application group stack, you define an infrastructure project that builds all of the infrastructure for a set of services. You can provision and destroy all of the pieces of the application with a single command.
Related patterns
This pattern risks growing into a Monolithic Stack (“Antipattern: Monolithic Stack”). In the other direction, breaking each service in an application group stack into a separate stack creates a service stack (“Pattern: Service Stack”).
Pattern: Service Stack
A Service Stack manages the infrastructure for each deployable application component in a separate infrastructure stack.
Figure 5-4. A Service Stack manages the infrastructure for each deployable application component in a separate infrastructure stack.
Also Known As
Stack per app, single service stack.
Motivation
Service stacks align the boundaries of infrastructure to the software that runs on it. This alignment limits the blast radius for a change to one service, which simplifies the process for scheduling changes. Service teams can own the infrastructure that relates to their software.
Consequences
If you have multiple applications, each with an infrastructure stack, there could be an unnecessary duplication of code. For example, each stack may include code that specifies how to provision an application server. Duplication can encourage inconsistency, such as using different operating system versions, or different network configurations. You can mitigate this by using modules to share code (as in Chapter 6).
Implementation
Each application or service has a separate infrastructure code project. When creating a new application, a team might copy code from another application’s infrastructure. Or the team could use a reference project, with boilerplate code for creating new stacks.
In some cases, each stack may be complete, not sharing any infrastructure with other application stacks. In other cases, teams may create stacks with infrastructure that supports multiple application stacks. You can learn more about different patterns for this in [Link to Come].
Related Patterns
The service stack pattern falls between an application group stack (“Pattern: Application Group Stack”), which has multiple applications in a single stack, and a micro stack (“Pattern: Micro Stack”), which breaks the infrastructure for a single application across multiple stacks.
Pattern: Micro Stack
The Micro Stack pattern divides the infrastructure for a single service across multiple stacks.
Figure 5-5. Micro stacks divide the infrastructure for a single service across multiple stacks.
For example, you may have a separate stack project each for the networking, servers, and database.
Motivation
Different parts of a service’s infrastructure may change at different rates. Or they may have different characteristics which make them easier to manage separately. For instance, some methods for managing server instances involve frequently destroying and rebuilding them6. However, some services use persistent data in a database or disk volume. Managing the servers and data in separate stacks means they can have different lifecycles, with the server stack being rebuilt much more often than the data stack.
Consequences
Although smaller stacks are themselves simpler, handling the dependencies between them is more complicated.
Implementation
Adding a new microstack involves creating a new stack project. You need to draw boundaries in the right places between stacks to keep them appropriately sized and easy to manage. The related patterns include solutions to this. You may also need to integrate different stacks, which I describe in [Link to Come].
Related Patterns
Micro stacks are the opposite end of the spectrum from a monolithic stack (“Antipattern: Monolithic Stack”), where a single stack contains all the infrastructure for a system.
Conclusion
Infrastructure stacks are fundamental building blocks for automated infrastructure. The patterns in this chapter are a starting point for thinking about organizing infrastructure into stacks. Given that a stack is a unit of change, the main principle for deciding where to draw boundaries between stacks in your system is making it easy and safe to make changes. [Link to Come] explores this topic in more detail.
In the next chapter, Chapter 6, I describe patterns and antipatterns for sharing code across stacks using modules.
1 https://github.com/chef-boneyard/chef-provisioning
2 The architect Christopher Alexander originated the idea of design patterns in A Pattern Language: Towns, Buildings, Construction (Alexander, Jacobson, Silverstein, Ishikawa, 1977, Oxford University Press). Kent Beck, Ward Cunningham, and others adapted the concept to Software design patterns. Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides, known as the “Gang of Four” (or “GoF”), cataloged common patterns and antipatterns in Design Patterns: Elements of Reusable Object-Oriented Software (1994, Addison Wesley).
3 I don’t know who, if anyone, can be said to have coined the term “blast radius” in the context of software system risks, but Charity Majors popularized it, probably starting with her post Terraform, VPC, and why you want a tfstate file per env
4 See Microservices by James Lewis.
5 See The Art of Building Autonomous Teams by John Ferguson Smart, among many other references
6 Examples of these are the phoenix server pattern ([Link to Come]) and immutable servers ([Link to Come])