The first two chapters of this book have described how the Kubernetes API is designed, and how to access it using HTTP requests. Specifically, you have seen that resources managed by the API are organized into Group-Version-Resources, and that objects exchanged between the client and the API Server are defined as Kinds by the Kubernetes API. The chapter also shows that this data can be encoded in JSON, YAML, or Protobuf during the exchange, depending on the HTTP headers the client has set.
In the next chapters, you will see how to access this API using the Go language. The two important Go libraries needed to work with the Kubernetes API are the apimachinery and the api.
The API Machinery is a generic library that takes care of serializing data between Go structures and objects written in the JSON (or YAML or Protobuf) format. This makes it possible for developers of clients, but also API Servers, to write data using Go structures and transparently use these resources in JSON (or YAML or Protobuf) during the HTTP exchanges.
The API Machinery Library is generic in the sense that it does not include any Kubernetes API resource definitions. It makes the Kubernetes API extendable and makes the API Machinery usable for any other API that would use the same mechanisms—that is, Kinds and Group-Version-Resources.
The API Library, for its part, is a collection of Go structures that are needed to work in Go with the resources defined by the Kubernetes API.
API Library Sources and Import
The sources of the API Library can be accessed from https://github.com/kubernetes/api. If you want to contribute to this library, note that the sources are not managed from this repository, but from the central one, https://github.com/kubernetes/kubernetes, in the staging/src/k8s.io/api directory, and the sources are synchronized from the kubernetes repository to the api repository.
Content of a Package
Let’s examine the files included in a package—for example, the k8s.io/api/apps/v1 package.
types.go
Then, continue in the same way for every structure used as a type in a previous structure.
You can see that every Kind embeds two structures: metav1.TypeMeta and metav1.ObjectMeta. It is mandatory for them to be recognized by the API Machinery. The TypeMeta structure contains information about the GVK of the Kind, and the ObjectMeta contains metadata for the Kind, like its name.
register.go
This file defines the group and version related to this package and the list of Kinds in this group and version. The public variable, SchemeGroupVersion, can be used when you need to specify the group and version of a resource from this group-version.
It also declares a function, AddToScheme, that can be used to add the group, version, and Kinds to a Scheme. The Scheme is an abstraction used in the API Machinery to create a mapping between Go structures and Group-Version-Kinds. This will be discussed further in Chapter 5, The API Machinery.
doc.go
This file and the following ones contain advanced information that you will not need to comprehend to start writing your first Kubernetes resources in Go, but they will help you understand how to declare new resources with Custom Resource Definitions in the next chapters.
The first instruction is used by the deepcopy-gen generator to generate the zz_generated.deepcopy.go file. The second instruction is used by the go-to-protobuf generator to generate these files: generated.pb.go and generated.proto. The third instruction is used by the genswaggertypedocs generator to generate the types_swagger_doc_generated.go file.
generated.pb.go and generated.proto
These files are generated by the go-to-protobuf generator. They are used by the API Machinery when serializing the data to and from the Protobuf format.
types_swagger_doc_generated.go
This file is generated by the genswaggertypedocs generator. It is used during the generation of the complete swagger definition of the Kubernetes API.
zz_generated.deepcopy.go
This file is generated by the deepcopy-gen generator. It contains the generated definition of the DeepCopyObject method for each type defined in the package. This method is necessary for the structures to implement the runtime.Object interface, which is defined in the API Machinery Library, and the API Machinery expects that all Kind structures will implement this runtime.Object interface.
Specific Content in core/v1
The core/v1 package defines, in addition to defining the structures for the Core resources, utility methods for specific types that can be useful when you incorporate these types into your code.
ObjectReference
SetGroupVersionKind(gvk schema.GroupVersionKind) – this method will set the fields APIVersion and Kind based on the values of the GroupVersionKind value passed as a parameter.
GroupVersionKind() schema.GroupVersionKind – this method will return a GroupVersionKind value based on the fields APIVersion and Kind of the ObjectReference.
GetObjectKind() schema.ObjectKind – this method will cast the ObjectReference object as an ObjectKind. The two previous methods implement this ObjectKind interface. Because the DeepCopyObject method on ObjectReference also is defined, the ObjectReference will respect the runtime.Object interface.
ResourceList
The ResourceList type is defined as a map, the keys of which are ResourceName, and values are Quantity. This is used in various Kubernetes resources to define the limits and requests of resources.
Cpu() *resource.Quantity – returns the quantity for the CPU key of the map, in decimal format (1, 250 m, etc.)
Memory() *resource.Quantity – returns the quantity for the Memory key of the map, in binary format (512 Ki, 64 Mi, etc.)
Storage() *resource.Quantity – returns the quantity for the Storage key of the map, in binary format (512 Mi, 1 Gi, etc.)
Pods() *resource.Quantity – returns the quantity for the Pods key of the map, in decimal format (1, 10, etc.)
StorageEphemeral() *resource.Quantity – returns the quantity for the StorageEphemeral key in the map, in binary format (512 Mi, 1 Gi, etc.)
For each of these methods, if the key is not defined in the map, a Quantity with a Zero value will be returned.
Name(name ResourceName, defaultFormat resource.Format) *resource.Quantity – this returns the quantity for the Name key, in defaultFormat format.
The defined enumeration values for the ResourceName type are ResourceCPU, ResourceMemory, ResourceStorage, ResourceEphemeralStorage, and ResourcePods.
Taint
MatchTaint(taintToMatch *Taint) bool – this method will return true if the two taints have the same key and effect values.
ToString() string – this method will return a string representation of the Taint in this format: <key>=<value>:<effect>, <key>=<value>:, <key>:<effect>, or <key>.
Toleration
MatchToleration(tolerationToMatch *Toleration) bool – this method returns true if the two tolerations have the same values for Key, Operator, Value, and Effect.
- ToleratesTaint(taint *Taint) bool – this method returns true if the toleration tolerates the Taint. The rules for a toleration to tolerate a taint are as follows:
Effect: For an empty Toleration effect, all Taint effects will match; otherwise, Toleration and Taint effects must match.
Operator: If TolerationOperator is Exists, all Taint values will match; otherwise, TolerationOperator is Equal and Toleration and Taint values must match.
Key: For an empty Toleration Key, TolerationOperator must be Exists, and all Taint keys (with any value) will match; otherwise, Toleration and Taint keys must match.
Well-Known Labels
The control-plane adds labels on nodes; well-known keys that are used and their usage can be found in the file well_known_labels.go of the core/v1 package. The following are the most well-known.
Writing Kubernetes Resources in Go
You will need to write Kubernetes resources in Go to create or update resources into the cluster, either using an HTTP request or, more generally, using the client-go library. The client-go library is discussed in a future chapter; but for now, let’s focus on writing the resources in Go.
To create or update a resource, you will need to create the structure for the Kind associated with the resource. For example, to create a deployment, you will need to create a Deployment kind; and for this, initiate a Deployment structure, which is defined in the apps/v1 package of the API Library.
Importing the Package
Before you can work with the structure, you need to import the package that defines this structure. As was seen at the beginning of the chapter, the pattern for the package name is k8s.io/api/<group>/<version>. The last part of the path is a version number, but you should not confuse it with a version number of a Go module.
The difference is that when you are importing a specific version of a Go module (e.g., k8s.io/klog/v2), you will use the part before the version as a prefix to access symbols of the package, without defining any aliases. The reason is that in the library, the v2 directory does not exist but represents a branch name, and the files going into the package start with the line package klog, not package v2.
On the contrary, when working with the Kubernetes API Library, the version number is the real name of a package in it, and the files into this package really start with package v1.
If you do not define an alias for the import, you would have to use the version name to use symbols from this package. But the version number alone is not meaningful when reading the code, and if you include several packages from the same file, you would end up with several v1 package names, which is not possible.
All structures related to Kinds first embed two generic structures: TypeMeta and ObjectMeta. Both are declared in the /pkg/apis/meta/v1 package of the API machinery library.
The TypeMeta Fields
You generally will not have to set values for these fields yourself because the API Machinery infers these values from the type of the structure by maintaining a Scheme—that is, a mapping between Group-Version-Kinds and Go structures. Note that the APIVersion value is another way to write the Group-Version as a single field that contains <group>/<version> (or only v1 for the legacy core group).
The ObjectMeta Fields
The package /pkg/apis/meta/v1 of the API Machinery Library defines Getters and Setters for fields of this structure. As the ObjectMeta is embedded in the resource structures, you can use these methods in the resources objects themselves.
Name
The most important information of this structure is the name of the resource. You can either use the Name field to specify the exact name of the resource or use the GenerateName field to request that the Kubernetes API selects a unique name for you; it is built by adding a suffix to the GenerateName value to make it unique.
Namespace
If you specify a namespace in the Pod structure different from project1, you will get an error: “the namespace of the provided object does not match the namespace sent on the request.” For these reasons, when creating a resource, it is not necessary to set the Namespace field.
UID, ResourceVersion, and Generation
The UID is a unique identifier across past, present and future resources in a cluster. It is set by the control plane and is never updated during the lifetime of a resource. It must be used to reference a resource, rather than its kind, name, and namespace, which could describe various resources across time.
The ResourceVersion is an opaque value representing the version of the resource. The ResourceVersion changes every time a resource is updated.
This ResourceVersion is used for Optimistic concurrency control: if you get a specific version of a resource from the Kubernetes API, modify it then send it back to the API to update it; the API will check that the ResourceVersion of the resource you are sending back is the last one. If another process modified the resource in the meantime, the ResourceVersion will have been modified, and your request will be rejected; in this case, it is your responsibility to read the new version and update it again. This is different from a Pessimist concurrency control where you would need to acquire a lock before reading the resource and release it after updating it.
The Generation is a sequence number that can be used by the resource’s controller to indicate the version of the desired state (the Spec). It will be updated only when the Spec part of the resource is updated, not the other parts (labels, annotations, status). The controller generally uses an ObservedGeneration field in the Status part to indicate which generation was processed last and is reflected in the status.
Labels and Annotations
Labels and annotations are defined in Go as a map in which the keys and values are strings.
Even if labels and annotations have very different usage, they can be populated the same way. We will discuss in the present section how to populate the labels field, but this is also applicable to annotations.
The function ConvertSelectorToLabelsMap transforms a selector string into a Set.
The function Conflicts checks that two Sets do not have conflicting labels. Conflicting labels are labels that have the same key but different values.
The function Merge will merge two Sets into a single Set. If there are conflicting labels between the two sets, the label in the second set will be used in the resulting Set,
The function Equals checks that the two Sets have the same keys and values.
The method Has indicates whether a Set contains a key.
The method Get returns the value for a given key in the Set, or an empty string if the key is not defined in the Set.
GetLabels() map[string]string
SetLabels(labels map[string]string)
GetAnnotations() map[string]string
SetAnnotations(annotations map[string]string)
OwnerReferences
An OwnerReference is set on a Kubernetes resource when you want to indicate that this resource is owned by another one, and you want this resource to be collected by the garbage collector when the owner does not exist anymore.
This is used widely when developing controllers and operators. A controller or operator creates some resources to implement the specifications described by another resource, and it places an OwnerReference into the created resources, pointing to the resource giving the specifications.
For example, the Deployment controller creates ReplicaSet resources based on specifications found in a Deployment resource. When you delete the Deployment, the associated ReplicaSet resources are deleted by the garbage collector without any intervention from the controller.
To know the UID of the object to reference, you need to get the object from the Kubernetes API, using a get (or list) request.
Setting APIVersion and Kind
The APIVersion contains the same information as the Group and Version. You can get the information from the SchemeGroupVersion variable of type schema.GroupVersion, which is defined in the package of the API library related to the resource (here k8s.io/api/core/v1 for the Pod resource). You can then add the Kind to create a schema.GroupVersionKind.
For the first solution, you can use the method SetGroupVersionKind on the referenced object to set the APIVersion and Kind from the GroupVersionKind. For the second solution, use the ToAPIVersionAndKind method on the GroupVersionKind to get the corresponding APIVersion and Kind values before moving them to the OwnerReference.
Chapter 5 describes the API Machinery Library and all the types and methods related to Group, Version, and Kinds. The OwnerReference structure also contains two optional boolean fields: Controller and BlockOwnerDeletion.
Setting Controller
The Controller field indicates whether the referenced object is the managing Controller (or Operator). A controller, or operator, must set this value to true on the owned resource to indicate that it is managing this owned resource.
The Kubernetes API will refuse to add two OwnerReferences with the Controller set to true on the same resource. This way, it is not possible that a resource is managed by two different controllers.
Note that this is different from being owned by various resources. A resource can have different owners; and in this case, the resource will be deleted when all its owners have been deleted, independent of which owner is the Controller, if any.
This uniqueness of controllers is useful for those that can “adopt” resources. For example, a ReplicaSet can adopt an existing pod that matches the selectors of the ReplicaSet, but only if the Pod is not already controlled by another ReplicaSet or another controller.
Setting BlockOwnerDeletion
The OwnerReference is useful for a controller or other process to not have to take care about deletion of owned resources: when the owner is deleted, the owned will be deleted by the Kubernetes Garbage Collector.
Orphan: To indicate to the Kubernetes API to orphan the owned resources, so they will not be deleted by the garbage collector.
Background: To indicate to the Kubernetes API to return from the DELETE operation immediately after the owner resource is deleted, and not wait for owned resources to be deleted by the garbage collector.
Foreground: To indicate to the Kubernetes API to return from the DELETE operation after the owner and the owned resources with BlockOwnerDeletion set to true are deleted. The Kubernetes API will not wait for other owned resources to be deleted.
So, if you are writing a controller or another process that needs to wait for all the owned resources to be deleted, the process will need to set the BlockOwnerDeletion field to true on all the owned resources and to use the Foreground propagation policy when deleting the owner resource.
Spec and Status
After the common Type and Metadata, resource definitions are generally composed of two parts: the Spec and the Status.
Note that it is not true for all resources. For example, the core ConfigMap and Secret resources, to name a couple, do not contain Spec and Status parts. More generally, resources containing configuration data, which are not managed by any controller, do not contain these fields.
The Spec is the part that the user will define, which indicates the desired state by the user. The Spec will be read by the Controller managing this resource, which will create, update, or delete resources on the cluster according to the Spec, and retrieve the status of its operations into the Status part of the resource. This process used by a controller to read the Spec, apply to the cluster and retrieve the Status is called the Reconcile Loop.
Comparison with Writing YAML Manifests
The manifest starts with apiVersion and kind.
The metadata field contains all the metadata for the resource.
The Spec and Status fields (or others) follow.
The type of the structure determines the apiVersion and Kind; there is no need to specify them.
The metadata can be defined either by embedding the metav1.ObjectMeta structure, or by using metadata setters on the resource.
The Spec and Status fields (or others) follow, using their own Go structures or other types.
A Complete Example
➊ Build a Pod object using a Go structure, as has been shown earlier in this chapter
➋ Serialize the Pod object in JSON using a serializer (see Chapter 5 for more about this)
➌ Build an HTTP POST request with the body that contains the Pod to create, serialized in JSON
➍ Call the Kubernetes API with the request built
➎ Get the body from the response
➏ Depending on the response status code:
➐ Deserialize the response body as a Pod Go structure
➑ Display the created Pod object as JSON for information;
otherwise:
➒ Deserialize the response body as a Status Go structure
➓ Display the Status object as JSON for information:
Conclusion
In this chapter, you have discovered a first library to work with Kubernetes in Go—the API Library. It is essentially a collection of Go structures to declare Kubernetes resources. The chapter also has explored the definition of the metadata fields common to all resources, defined in the API Machinery Library.
At the end of the chapter, there is an example of a program for building a Pod definition using the API Library and then creating this Pod in the cluster by calling the API Server using an HTTP request.
The next chapters explores other fundamental libraries—the API Machinery and the Client-go—with which you will no longer need to build HTTP requests.