Global Tags

Global tags are written into a package’s doc.go. A typical pkg/apis/group/version/doc.go file looks like this:

// +k8s:deepcopy-gen=package

// Package v1 is the v1alpha1 version of the API.
// +groupName=cnat.programming-kubernetes.info
package v1alpha1

The first line of this file tells deepcopy-gen to create deep-copy methods by default for every type in that package. If you have types where deep copy is not necessary, not desired, or even not possible, you can opt out for them with the local tag // +k8s:deepcopy-gen=false. If you do not enable package-wide deep copy, you have to opt in to deep copy for each desired type via // +k8s:deepcopy-gen=true.

The second tag, // +groupName=example.com, defines the fully qualified API group name. This tag is necessary if the Go parent package name does not match the group name.

The file shown here actually comes from the cnat client-go example pkg/apis/cnat/v1alpha1/doc.go file (see “Following sample-controller”). There, cnat is the parent package, but cnat.programming-kubernetes.info is the group name.

With the // +groupName tag, the client generator (see “Typed client created via client-gen”) will generate a client using the correct HTTP path /apis/foo.project.example.com. Besides +groupName there is also +groupGoName, which defines a custom Go identifier (for variable and type names) to be used instead of the parent package name. For example, the generators will use the uppercase parent package name for identifies by default, which in our example is Cnat. A better identifier would be CNAt for “Cloud Native At.” With // +groupGoName=CNAt we could use that instead of Cnat (though we don’t do that in this example—we’ve stayed with Cnat), and the client-gen result would look like the following:

type Interface interface {
    Discovery() discovery.DiscoveryInterface
    CNatV1() atv1alpha1.CNatV1alpha1Interface
}

Local Tags

Local tags are written either directly above an API type or in the second comment block above it. Here are the main types in the types.go file of the cnat example:

// AtSpec defines the desired state of At
type AtSpec struct {
    // Schedule is the desired time the command is supposed to be executed.
    // Note: the format used here is UTC time https://www.utctime.net
    Schedule string `json:"schedule,omitempty"`
    // Command is the desired command (executed in a Bash shell) to be executed.
    Command string `json:"command,omitempty"`
    // Important: Run "make" to regenerate code after modifying this file
}

// AtStatus defines the observed state of At
type AtStatus struct {
    // Phase represents the state of the schedule: until the command is executed
    // it is PENDING, afterwards it is DONE.
    Phase string `json:"phase,omitempty"`
    // Important: Run "make" to regenerate code after modifying this file
}

// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// At runs a command at a given schedule.
type At struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`

    Spec   AtSpec   `json:"spec,omitempty"`
    Status AtStatus `json:"status,omitempty"`
}

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// AtList contains a list of At
type AtList struct {
    metav1.TypeMeta `json:",inline"`
    metav1.ListMeta `json:"metadata,omitempty"`
    Items           []At `json:"items"`
}

In the following sections we’ll walk through the tags of this example.

deepcopy-gen Tags

Deep-copy method generation is usually enabled for all types by default via the global // +k8s:deepcopy-gen=package tag (see “Global Tags”)—that is, with possible opt-out. However, in the preceding example file (and actually the whole package), all API types need deep-copy methods. Hence, we don’t have to opt out locally.

If we had a helper struct in the API types package (this is usually discouraged to keep API packages clean), we would have to disable deep-copy generation. For example:

// +k8s:deepcopy-gen=false
//
// Helper is a helper struct, not an API type.
type Helper struct {
    ...
}

runtime.Object and DeepCopyObject

There is a special deep-copy tag that needs more explanation:

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

In “Kubernetes Objects in Go” we saw that runtime.Objects have to implement the DeepCopyObject() runtime.Object method. The reason is that generic code within Kubernetes has to be able to create deep copies of objects. This method allows that.

The DeepCopyObject() method does nothing other than calling the generated DeepCopy method. The signature of the latter varies from type to type (DeepCopy() *T depends on T). The signature of the former is always DeepCopyObject() runtime.Object:

func (in *T) DeepCopyObject() runtime.Object {
    if c := in.DeepCopy(); c != nil {
        return c
    } else {
        return nil
    }
}

Put the local tag // +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object above your top-level API types to generate this method with deepcopy-gen. This tells deepcopy-gen to create such a method for runtime.Object, called DeepCopyObject().

It happens that other interfaces need a way to be deep-copied. This is usually the case if, for example, API types have a field of interface type Foo:

type SomeAPIType struct {
  Foo Foo `json:"foo"`
}

As we have seen, API types must be deep-copyable, and hence the field Foo must be deep-copied too. How could you do that in a generic way (without type-casts) without adding DeepCopyFoo() Foo to the Foo interface?

type Foo interface {
    ...
    DeepCopyFoo() Foo
}

In that case the same tag can be used:

// +k8s:deepcopy-gen:interfaces=<package>.Foo
type FooImplementation struct {
    ...
}

There are a few examples beyond runtime.Object in the Kubernetes source where this tag is actually used:

// +k8s:deepcopy-gen:interfaces=.../pkg/registry/rbac/reconciliation.RuleOwner
// +k8s:deepcopy-gen:interfaces=.../pkg/registry/rbac/reconciliation.RoleBinding

client-gen Tags

Finally, there are a number of tags to control client-gen, one of which we saw in the earlier example for At and AtList:

// +genclient

It tells client-gen to create a client for this type (this is always opt-in). Note that you don’t have to and indeed must not put it above the List type of the API objects.

In our cnat example, we use the /status subresource and update the status of the CRs with the UpdateStatus method of the client (see “Status subresource”). There are instances of CRs without a status or without a spec-status split. In those cases, the following tag avoids the generation of that UpdateStatus() method:

// +genclient:noStatus

The client generator has to choose the right HTTP path, either with or without a namespace. For cluster-wide resources, you have to use the tag:

// +genclient:nonNamespaced

The default is to generate a namespaced client. Again, this has to match the scope setting in the CRD manifest. For special-purpose clients, you might also want to control in detail which HTTP methods are offered. You can do this by using a couple of tags, for example:

// +genclient:noVerbs
// +genclient:onlyVerbs=create,delete
// +genclient:skipVerbs=get,list,create,update,patch,delete,watch
// +genclient:method=Create,verb=create,
// result=k8s.io/apimachinery/pkg/apis/meta/v1.Status

The first three should be pretty self-explanatory, but the last one warrants some explanation.

The type this tag is written above will be create-only and will not return the API type itself, but a metav1.Status. For CRs this does not make much sense, but for user-provided API servers written in Go (see Chapter 8) those resources can exist, and they do in practice.

One common case for the // +genclient:method= tag is the addition of a method to scale a resource. In “Scale subresource” we describe how the /scale subresource can be enabled for CRs. The following tags create the corresponding client methods:

// +genclient:method=GetScale,verb=get,subresource=scale,
//    result=k8s.io/api/autoscaling/v1.Scale
// +genclient:method=UpdateScale,verb=update,subresource=scale,
//    input=k8s.io/api/autoscaling/v1.Scale,result=k8s.io/api/autoscaling/v1.Scale

The first tag creates the getter GetScale. The second creates the setter UpdateScale.

informer-gen and lister-gen

Both informer-gen and lister-gen process the // +genclient tag of client-gen. There is nothing else to configure. Each type that opted in to client generation gets informers and listers automatically that match the client (if the whole suite of generators is called via the k8s.io/code-generator/generate-group.sh script).

The documentation of the Kubernetes generators has a lot of room for improvement and will certainly be refined slowly over time. For more information about the different generators, it is often helpful to look at examples inside Kubernetes itself—for example, k8s.io/api and OpenShift API types. Both repositories have many advanced use cases.

Moreover, don’t hesitate to look into the generators themselves. deepcopy-gen has some documentation available inside its main.go file. client-gen has some documentation available in the Kubernetes contributor documentation. informer-gen and lister-gen currently don’t have further documentation, but generate-groups.sh shows how each is invoked.