1 Why Kubernetes exists

This chapter covers

• Why Kubernetes exists
• Commonly used Kubernetes terms
• Specific use cases for Kubernetes
• High-level Kubernetes features
• When not to run Kubernetes

Kubernetes is an open source platform for hosting containers and defining application-centric APIs for managing cloud semantics around how these containers are provisioned with storage, networking, security, and other resources. Kubernetes enables continuous reconciliation of the entire state space of your application deployments, including how they are accessed from the outside world.

Why implement Kubernetes in your environment as opposed to manually provisioning these sorts of resources using a DevOps-related infrastructure tool? The answer lies in the way we define DevOps to be increasingly integrated into the overall application life cycle over time. DevOps has evolved increasingly to include processes, engineers, and tools that support a more automated administration of applications in a data center. One of the keys to doing this successfully is reproducibility of infrastructure: a change made to fix an incident on one component that’s not replicated perfectly across all other identical components means one or more components differ.

In this book, we will take a deep dive into the best practices for using Kubernetes with DevOps, so components are replicated as needed and your system fails less often. We will also explore the under-the-hood processes to better understand Kubernetes and get the most efficient system possible.

1.1 Reviewing a few key terms before we get started

In 2021, Kubernetes was one of the most commonly deployed cloud technologies. Because of this, we don’t always fully define new terms before referencing them. In case you’re new to Kubernetes or are unsure of a few terms, we provide some key definitions that you can refer back to throughout the first few chapters of this book as you ramp up on this new universe. We will redefine these concepts with more granularity and in greater context as we dig into them later in this book:

• CNI and CSI—The container networking and storage interfaces, respectively, that allow for pluggable networking and storage for Pods (containers) that run in Kubernetes.

• Container—A Docker or OCI image that typically runs an application.

• Control plane—The brains of a Kubernetes cluster, where scheduling of containers and managing all Kubernetes objects takes place (sometimes referred to as Masters).

• DaemonSet—Like a deployment, but it runs on every node of a cluster.

• Deployment—A collection of Pods that is managed by Kubernetes.

• kubectl—The command-line tool for talking to the Kubernetes control plane.

• kubelet—The Kubernetes agent that runs on your cluster nodes. It does what the control plane needs it to do.

• Node—A machine that runs a kubelet process.

• OCI—The common image format for building executable, self-contained applications. Also referred to as Docker images.

• Pod—The Kubernetes object that encapsulates a running container.

1.2 The infrastructure drift problem and Kubernetes

Managing infrastructure is a reproducible way of managing the “drift” of that infrastructure’s configuration as hardware, compliance, and other data-center requirements change over time. This applies to both the definition of applications as well as to the management of the hosts these apps run on. IT engineers are all too familiar with common toil such as

• Updating the Java version on a fleet of servers

• Making sure certain applications don’t run in specific places

• Replacing or scaling old or broken hardware and migrating applications from it

• Manually managing load-balancing routes

• Forgetting to document new infrastructure changes when lacking a common enforced configuration language

As we manage and update servers in a data center, or in the cloud, the odds that their original definitions “drift away” from the intended IT architecture increases. Applications might be running in the wrong places, with the wrong resource allotment, or with access to the wrong storage modules.

Kubernetes gives us a way to centrally manage the entire state space of all applications with one handy tool: kubectl (https://kubernetes.io/docs/tasks/tools/), a command-line client that makes REST API calls to the Kubernetes API server. We can also use Kubernetes API clients to do these tasks programmatically. It’s quite easy to install kubectl and to test it on a kind cluster, which we’ll do early on in this book.

Previous approaches to managing this complex application state space include technologies such as Puppet, Chef, Mesos, Ansible, and SaltStack. Kubernetes borrows from these different approaches by taking the state management capabilities of tools such as Puppet, while borrowing concepts from some of the application and scheduling primitives provided by software such as Mesos.

Ansible, SaltStack, and Terraform typically have played a major role in infrastructure configuration (paving OS-specific requirements such as firewalls or binary installations). Kubernetes manages this concept as well, but it uses privileged containers on a Linux environment (these are known as HostProcess Pods on Windows v1.22). For example, a privileged container in a Linux system can manage iptables rules for routing traffic to applications, and in fact, this is exactly what the Kubernetes Service proxy (known as the kube-proxy) does.

Google, Microsoft, Amazon, VMware, and many companies have adopted containerization as a core and enabling strategy for their customers to run fleets of hundreds or thousands of applications on different cloud and bare metal environments. Containers are, thus, a fundamental primitive for both running apps and managing application infrastructure (such as providing containers with IP addresses) that run the services these apps depend on (such as the provisioning of bespoke storage and firewall requirements), and, most importantly, run the applications themselves.

Kubernetes is (at the time of this writing) essentially undisputed as the modern standard for orchestrating and running containers in any cloud, server, or data center environment.

1.3 Containers and images

Apps have dependencies that must be fulfilled by the host on which they run. Developers in the pre-container era accomplished this task in an ad hoc manner (for example, a Java app would require a running JVM along with firewall rules to talk to a database).

At its core, Docker can be thought of as a way to run containers, where a container is a running OCI image (https://github.com/opencontainers/image-spec). The OCI specification is a standard way to define an image that can be executed by a program such as Docker, and it ultimately is a tarball with various layers. Each of the tarballs inside an image contains such things as Linux binaries and application files. Thus, when you run a container, the container runtime (such as Docker, containerd, or CRI-O) takes the image, unpacks it, and starts a process on the host system that runs the image contents.

Containers add a layer of isolation that obviates the need for managing libraries on a server or preloading infrastructure with other accidental application dependencies (figure 1.1). For instance, if you have two Ruby applications that require different versions of the same library, you can use two containers. Each Ruby application is isolated inside a running container and has the specific version of the library that it requires.

There is a phase that is well known: “Well, it runs on my machine.” When installing software, it can often run in one environment or machine but not in another. Using images simplifies running the same software on different servers. We’ll talk more about images and containers in chapter 3.

Combine using images with Kubernetes, allowing for running immutable servers, and you have a level of simplicity that is world-class. As containers are quickly becoming an industry standard for the deployment of software applications, a few data points are worth mentioning:

• Surveying over 88,000 developers, Docker and Kubernetes ranked third among the most loved development technologies of 2020. This just behind Linux and Docker (http://mng .bz/nY12).

• Datadog recently found that Docker encompasses 50% or more of the average developer’s workflow. Likewise, company-wide adoption is over 25% of all businesses (https:// www.datadoghq.com/docker-adoption/).

The bottom line is that we need automation for containers, and this is where Kubernetes fits in. Kubernetes dominates the space much like the Oracle database and the vSphere virtualization platform did during their heydays. Years later, Oracle databases and vSphere installations still exist; we predict the same longevity for Kubernetes.

We’ll begin this book with a basic understanding of Kubernetes features. Its purpose is to take you beyond the basic principles to the lower-level core. Let’s dive in and look at an extremely over-simplified Kubernetes (also referred to as “K8s”) workflow that demonstrates some of the higher-order tenants of building and running microservices.

1.4 Core foundation of Kubernetes

At its core, we define everything in Kubernetes as plain text files, defined via YAML or JSON, and it runs your OCI images for you in a declarative way. We can use this same approach (YAML or JSON text files) to configure networking rules, role-based authentication and authorization (RBAC), and so on. By learning one syntax and how it is structured, any Kubernetes system can be configured, managed, and optimized.

Let’s look at a quick sample of how one might run Kubernetes for a simple app. Don’t worry; we’ll have plenty of real-world examples to walk you through the entire life cycle of an application later in the book. Consider this just a visual guide to our hand-waving we’ve done thus far. To start with a concrete example of a microservice, the following code snippet generates a Dockerfile that builds an image capable of running MySQL:

FROM alpine:3.15.4
RUN apk add --no-cache mysql
ENTRYPOINT ["/usr/bin/mysqld"]

One would typically build this image (using docker build) and push it (using something like docker push) to an OCI registry (a place where such an image can be stored and retrieved by a container at run time). You can find a common open source registry to host on your own at https://github.com/goharbor/harbor. Another such registry that is also commonly used for millions of applications worldwide resides at https://hub.docker.com/. For this example, let’s say we pushed this image, and now we are running it, somewhere. We might also want to build a container to talk to this service (maybe we have a custom Python app that serves as a MySQL client). We might define its Docker image like so:

FROM python:3.7
WORKDIR /myapp
COPY src/requirements.txt ./
RUN pip install -r requirements.txt
COPY src /myapp
CMD [ "python", "mysql-custom-client.py" ]

Now, if we wanted to run our client and the MySQL server as containers in a Kubernetes environment, we could easily do so by creating two Pods. Each one of these Pods might run one of the respective containers like so:

apiVersion: v1
kind: Pod
metadata:
  name: core-k8s
  spec:
  containers:
    - name: my-mysql-server
      image: myregistry.com/mysql-server:v1.0
---
apiVersion: v1
kind: Pod
metadata:
  name: core-k8s-mysql
  spec:
  containers:
    - name: my-sqlclient
      image: myregistry.com/mysql-custom-client:v1.0
      command: ['tail','-f','/dev/null']

We would, typically, store the previous YAML snippet in a text file (for example, my-app.yaml) and execute it using the Kubernetes client tool (for example, kubectl create -f my-app.yaml). This tool connects to the Kubernetes API server and transfers the YAML definition to be stored. Kubernetes then automatically takes the definitions of the two Pods that we have on the API server and makes sure they are up and running somewhere.

This doesn’t happen instantly: it requires the nodes in the cluster to respond to events that are constantly occurring and updates that state in their Node objects via the kubelet communicating to the API server. It also requires that the OCI images are present and accessible to the nodes in our Kubernetes cluster. Things can go wrong at any time, so we refer to Kubernetes as an eventually consistent system, wherein reconciliation of the desired state over time is a key design philosophy. This consistency model (compared with a guaranteed consistency model) ensures that we can continually request changes to the overall state space of all applications in our cluster and lets the underlying Kubernetes platform figure out the logistics of how these apps are set in motion over time.

This scales into real-world scenarios quite naturally. For example, if you tell Kubernetes, “I want five applications spread across three zones in a cloud,” this can be accomplished entirely by defining a few lines of YAML utilizing Kubernetes’ scheduling primitives. Of course, you need to make sure that those three zones actually exist and that your scheduler is aware of them, but even if you haven’t done this, Kubernetes will at least schedule some of the workloads on the zones that are available.

In short, Kubernetes allows you to define the desired state of all the apps in your cluster, how they are networked, where they run, what storage they use, and so on, while delegating the underlying implementation of these details to Kubernetes itself. Thus, you’ll rarely find the need to do a one-off Ansible or a Puppet update in a production Kubernetes scenario (unless you are reinstalling Kubernetes itself, and even then, there are tools such as the Cluster API that allow you to use Kubernetes to manage Kubernetes (now we’re getting in way over our heads).

1.4.1 All infrastructure rules in Kubernetes are managed as plain YAML

Kubernetes automates all of the aspects of the technology stack using the Kubernetes API, which can be entirely managed as YAML and JSON resources. This includes traditional IT infrastructure rules (which still apply in some manner or other to microservices) such as

• Server configuration of ports or IP routes

• Persistent storage availability for applications

• Hosting of software on specific or arbitrary servers

• Security provisioning, such as RBAC or networking rules for applications to access one another

• DNS configuration on a per-application and global basis

All of these components are defined within configuration files that are representations of objects within the Kubernetes API. Kubernetes uses these building blocks and containers by applying changes, monitoring those changes, and addressing momentary failures or disruptions until achieving the desired end state. When “things go bump in the night,” Kubernetes will handle a lot of scenarios automatically, and we do not have to fix the problems ourselves. Properly configuring more elaborate systems with automation permits the DevOps team to focus on solving complex problems, to plan for the future, and to find the best-in-class solutions for the business. Next, let’s review the features that Kubernetes provides and how they support the use of Pods.

1.5 Kubernetes features

Container orchestration platforms allow developers to automate the process of running instances, provisioning hosts, linking containers to optimize orchestration procedures, and extending application life cycles. It’s time to dive into the core features within a container orchestration platform because, essentially, containers need Pods and Pods need Kubernetes to

• Expose a cloud-neutral API for all functionality within the API server

• Integrate with all major cloud and hypervisor platforms within the Kubernetes controller manager (also referred to as KCM)

• Provide a fault-tolerant framework for storing and defining the state of all Services, applications, and data center configurations or other Kubernetes-supported infrastructures

• Manage deployments while minimizing user-facing downtime, whether to an individual host, Service, or application

• Automate scaling for hosts and hosted applications with rolling update awareness

• Create internal and external integrations (known as ClusterIP, NodePort, or LoadBalancer Service types) with load balancing

• Provide the ability to schedule applications to run on specific virtualized hardware, based on its metadata, via node labeling and the Kubernetes scheduler

• Deliver a highly available platform via DaemonSets and other technology infrastructures that prioritizes containers that run on all nodes in the cluster

• Allow for service discovery via a domain name service (DNS), implemented previously by KubeDNS and, most recently, by CoreDNS, which integrates with the API server

• Run batch processes (known as Jobs) that use storage and containers in the same way persistent applications run

• Include API extensions and construct native API-driven programs using custom resource definitions, without building any port mappings or plumbing

• Enable inspection of failed cluster-wide processes including remote execution into any container at any time via kubectl exec and kubectl describe

• Allow the mounting of local and/or remote storage to a container and manage declarative storage volumes for containers with the StorageClass API and PersistentVolumes

Figure 1.2 is a simple diagram of a Kubernetes cluster. What Kubernetes does is by no means trivial. It standardizes the life cycle management of multiple applications running in or on the same cluster. The foundation of Kubernetes is a cluster, consisting of nodes. The complexity of Kubernetes is, admittedly, one of the complaints that engineers have about Kubernetes. The community is working on making it easier, but Kubernetes is solving a complex problem that is hard to solve to begin with.

If you don’t need high availability, scalability, and orchestration, then maybe you don’t need Kubernetes. Let’s now consider a typical failure scenario in a cluster:

1. A node stops responding to the control plane.

2. The control plane reschedules the Pods running on the unresponsive node to another node or nodes.

3. When a user makes an API call into the API server via kubectl, the API server responds with the correct information about the unresponsive node and the new location of the Pods.

4. All clients that communicate to the Pod’s Service are rerouted to its new location.

5. Storage volumes attached to Pods on the failing node are moved to the new Pod location so that its old data is still readable.

The purpose of this book is to give you deeper insight into how this all really works under the hood and how the underlying Linux primitives complement the high-level Kubernetes components to accomplish these tasks. Kubernetes relies heavily on hundreds of technologies in the Linux stack, which are often hard to learn and lack deep documentation. It is our hope that by reading this book, you’ll understand a lot of the subtleties of Kubernetes, which are often overlooked in the tutorials first used by engineers to get up and running with containers.

It is natural to run Kubernetes on top of immutable operating systems. You have a base OS that only updates when you update the entire OS (and thus is immutable), and you install your Nodes/Kubernetes using that OS. There are many advantages to running an immutable OS that we will not cover here. You can run Kubernetes in the cloud, on bare metal servers, or even on a Raspberry Pi. In fact, the U.S. Department of Defense is currently researching how to run Kubernetes on some of its fighter jets. IBM even supports running clusters on its next generation mainframes, PowerPCs.

As the cloud native ecosystem around Kubernetes continues to mature, it will continue to permit organizations to identify best practices, proactively make changes to prevent issues, and maintain environment consistency to avoid drift, where some machines behave slightly differently from others because patches were missed, not applied, or improperly applied.

1.6 Kubernetes components and architecture

Now, let’s take a moment to look at the Kubernetes architecture at a high level (fig-ure 1.3). In short, it consists of your hardware and the portion of your hardware that runs the Kubernetes control plane as well as the Kubernetes worker nodes:

• Hardware infrastructure—Includes computers, network infrastructure, storage infrastructure, and a container registry.

• Kubernetes worker nodes—The base unit of compute in a Kubernetes cluster.

• Kubernetes control plane—The mothership of Kubernetes. This covers the API server, scheduler, controller manager, and other controllers.

1.6.1 The Kubernetes API

If there’s one important thing to take away from this chapter that will enable you to go forth on a deep journey through this book, it’s that administering microservices and other containerized software applications on a Kubernetes platform is just a matter of declaring Kubernetes API objects. For the most part, everything else is done for you.

This book will dive deeply into the API server and its datastore, etcd. Almost anything that you can ask kubectl to do results in reading, or writing, to a defined and versioned object in the API server. (The exceptions to this are things like using kubectl to grab logs for a running Pod, wherein this connection is proxied through to a node.) The kube-apiserver (Kubernetes API server) allows for CRUD (create, read, update, and delete) operations on all of the objects and provides a RESTful (REpresentational State Transfer) interface. Some kubectl commands like describe are a composite view of multiple objects. In general, all Kubernetes API objects have

• A named API version (for instance, v1 or rbac.authorization.k8s.io/v1)

• A kind (for example, kind: Deployment)

• A metadata section

We can thank Brian Grant, one of the original Kubernetes founders, for the API versioning scheme that has proven to be robust over time. It may seem complicated, and, frankly, a bit of a pain at times, but it allows us to do things such as upgrades and contracts defining API changes. API changes and migration are often nontrivial, and Kubernetes provides a well-defined contract for API changes. Take a look at the API versioning documents on the Kubernetes website (http://mng.bz/voP4), and you can read through the contracts for Alpha, Beta, and GA API versions.

Throughout the chapters in this book, we will focus on Kubernetes but keep returning to the basic theme: virtually everything in Kubernetes exists to support the Pod. In this book, we’ll look at several API elements in detail including

• Runtime Pods and deployments

• API implementation details

• Ingress Services and load balancing

• PersistentVolumes and PersistentVolumeClaims storage

• NetworkPolicies and network security

There are around 70 different API types that you can play with, create, edit, and delete in a standard Kubernetes cluster. You can view these by running kubectl api-resources. The output should look something like this:

$ kubectl api-resources | head
NAME                    SHORTNAMES  NAMESPACED  KIND
bindings                            true        Binding
componentstatuses       cs          false       ComponentStatus
configmaps              cm          true        ConfigMap
endpoints               ep          true        Endpoints
events                  ev          true        Event
limitranges             limits      true        LimitRange
namespaces              ns          false       Namespace
nodes                   no          false       Node
persistentvolumeclaims  pvc         true        PersistentVolumeClaim

We can see that each of the API resources for Kubernetes itself has

• A short name

• A full name

• An indication of whether it is bounded to a namespace

In Kubernetes, Namespaces allow certain objects to exist inside of a specific . . . well . . . Namespace. This gives developers a simple form of hierarchical grouping. For example, if you have an application that runs 10 different microservices, you commonly might create all of these Pods, Services, and PersistentVolumeClaims (also referred to as PVCs) inside the same Namespace. That way, when it’s time for you to delete the app, you can just delete the Namespace. In chapter 15, we’ll look at higher-level ways to analyze the life cycle of applications, which are more advanced than this simplistic approach. But for many cases, the namespace is the most obvious and intuitive solution for separating all the Kubernetes API objects associated with an app.

1.6.2 Example one: An online retailer

Imagine a major online retailer that needs to be able to quickly scale with demand seasonally, such as around the holidays. Scaling and predicting how to scale has been one of their biggest challenges—maybe the biggest. Kubernetes solves a multitude of problems that come with running a highly available, scalable distributed system. Imagine the possibilities of having the ability to scale, distribute, and make highly available systems at your fingertips. Not only is it a better way to run a business, but it is also the most efficient and effective platform for managing systems. When combining Kubernetes and cloud providers, you can run on someone else’s servers when you need extra resources instead of buying and maintaining extra hardware just in case.

1.6.3 Example two: An online giving solution

For a second real-world example of this transition that is worth mentioning, let’s consider an online donation website that enables contributions to a broad range of charities per a user’s choice. Let’s say this particular example started out as a WordPress site, but eventually, business transactions lead to a full-blown dependency on JVM frameworks (like Grails) with a customized UX, middle tier, and database layer. The requirements for this business tsunami included machine learning, ad serving, messaging, Python, Lua, NGINX, PHP, MySQL, Cassandra, Redis, Elastic, ActiveMQ, Spark, lions, tigers, and bears . . . and stop already.

The initial infrastructure was a hand-built cloud virtual machine (VM), using Puppet to set everything up. As the company grew, they designed for scale, but this included more and more VMs that only hosted one or two applications. Then they decided to move to Kubernetes. The VM count was reduced from around 30 to 5 and scaled more easily. They completely eliminated Puppet and the server setup, and thus the need to manually manage machine infrastructure by hand, thanks to their transition to heavy use of Kubernetes.

The transition to Kubernetes for this company resolved the entire class of VM administration problems, the burden of DNS for complex service publishing, and much more. Additionally, the recovery times in cases of catastrophic failures were much more predictable to manage from an infrastructure standpoint. When you experience the benefits of moving to a standardized API-driven methodology that works well and has the power to make massive changes quickly, you begin to appreciate the declarative nature of Kubernetes and its cloud-native approach to container orchestration.

1.7 When not to use Kubernetes

Admittedly, there are always use cases where Kubernetes might not be a good fit. Some of these include

• High-performance computing (HPC)—Using containers adds a layer of complexity and, with the new layer, a performance hit. The latency created by using a container is getting much smaller, but if your application is influenced by nano- or microseconds, using Kubernetes might not be the best option.

• Legacy—Some applications have hardware, software, and latency requirements that make it difficult to simply containerize. For example, you may have applications that you purchased from a software company that does not officially support running in a container or running their application within a Kubernetes cluster.

• Migration—Implementations of legacy systems may be so rigid that migrating them to Kubernetes offers little advantage other than “we are built on Kuber-netes.” But some of the most significant gains come after migrating, when monolithic applications are parsed up into logical components, which can then scale independently of each other.

The important thing here is this: learn and master the basics. Kubernetes solves many of the problems presented in this chapter in a stable, cost-sensitive manner.

Summary

• Kubernetes makes your life easier!

• The Kubernetes platform can run on any type of infrastructure.

• Kubernetes builds an ecosystem of components that work together. Combining the components empowers companies to prevent, recover, and scale in real time when urgent changes are required.

• Everything you do in Kubernetes can be done with one simple tool: kubectl.

• Kubernetes creates a cluster from one or more computers, and that cluster provides a platform to deploy and host containers. It offers container orchestration, storage management, and distributed networking.

• Kubernetes was born from previous configuration-driven, container-driven approaches.

• The Pod is the basic building block of Kubernetes. It supports the myriad of features that Kubernetes allows: scaling, failover, DNS lookup, and RBAC security rules.

• Kubernetes applications are entirely managed by simply making API calls to the Kubernetes API server.