The last piece we'll touch on in our HA chapter is custom resources. These are an extension of the Kubernetes API, and are compliment with the admission controllers we discussed previously. There are several methods for adding custom resources to your cluster, and we'll discuss those here.

As a refresher, keep in mind that a non-custom resource in Kubernetes is an endpoint in the Kubernetes API that stores a collection of similar API objects. You can use custom resources to enhance a particular Kubernetes installation. We'll see examples of this with Istio in later chapters, which uses CRDs to put prerequisites into place. Custom resources can be modified, changed, and removed with kubectl.

When you pair custom resources with controllers, you have the ability to create a declarative API, which allows you to set the state for your gathered resources outside of the cluster's own life cycle. We touched on an example of the custom controller and custom resource pattern earlier in this book with the operator pattern. You have a couple of options when deciding whether or not to create a custom resource with Kubernetes. The documentation recommends the following decision table when choosing:

A key point in deciding to write a custom resource is to ensure that your API is declarative. If it's declarative, it's a good fit for a custom resource. You can write custom resources in two ways, with custom resource definitions or through API aggregation. API aggregation requires programming, and we won't be getting into that topic for this chapter, but you can read more about it here: https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/.