Chapter 10. Security
Running applications in Kubernetes comes with a shared responsibility between developers and ops folks to ensure that attack vectors are minimized, least-privileges principles are followed, and access to resources is clearly defined. In this chapter, we will present recipes that you can, and should, use to make sure your cluster and apps run securely. The recipes in this chapter cover:
-
The role and usage of service accounts
-
Role-Based Access Control (RBAC)
-
Defining a pod’s security context
10.1 Providing a Unique Identity for an Application
Solution
Create a service account and use it in a pod specification.
To begin, create a new service account called myappsa and have a closer look at it:
$ kubectl create serviceaccount myappsa
serviceaccount "myappsa" created
$ kubectl describe sa myappsa
Name: myappsa
Namespace: default
Labels: <none>
Annotations: <none>
Image pull secrets: <none>
Mountable secrets: myappsa-token-rr6jc
Tokens: myappsa-token-rr6jc
$ kubectl describe secret myappsa-token-rr6jc
Name: myappsa-token-rr6jc
Namespace: default
Labels: <none>
Annotations: kubernetes.io/service-account.name=myappsa
kubernetes.io/service-account.uid=0baa3df5-c474-11e7-8f08...
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1066 bytes
namespace: 7 bytes
token: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9 ...
You can use this service account in a pod, like so:
kind:PodapiVersion:v1metadata:name:myappspec:serviceAccountName:myappsacontainers:-name:mainimage:centos:7command:-"bin/bash"-"-c"-"sleep10000"
You can then verify whether the service account myappsa has been properly used by your pod by running:
$ kubectl exec myapp -c main
cat /var/run/secrets/kubernetes.io/serviceaccount/token
eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9 ...
Indeed, the myappsa service account token has been mounted in the expected place in the pod and can be used going forward.
While a service account on its own is not super useful, it forms the basis for fine-grained access control; see Recipe 10.2 for more on this.
Discussion
Being able to identify an entity is the prerequisite for authentication and authorization. From the API server’s point of view, there are two sorts of entities: human users and applications. While user identity (management) is outside of the scope of Kubernetes, there is a first-class resource representing the identity of an app: the service account.
Technically, the authentication of an app is captured by the token available in a file at the location /var/run/secrets/kubernetes.io/serviceaccount/token, which is mounted automatically through a secret. The service accounts are namespaced resources and are represented as follows:
system:serviceaccount:$NAMESPACE:$SERVICEACCOUNT
Listing the service accounts in a certain namespace gives you something like the following:
$ kubectl get sa NAME SECRETS AGE default 1 90d myappsa 1 19m prometheus 1 89d
Notice the service account called default here. This is created automatically; if you don’t set the service account for a pod explicitly, as was done in the Solution, it will be assigned the default service account in its namespace.
10.2 Listing and Viewing Access Control Information
Solution
The following solution assumes you’re using Role-Based Access Control as the authorization method.
To check if a certain action on a resource is allowed for a specific user, use kubectl auth can-i. For example, you can execute this command to check if the service account system:serviceaccount:sec:myappsa is allowed to list pods in the namespace sec:
$ kubectl auth can-i list pods --as=system:serviceaccount:sec:myappsa -n=sec yes
Note
If you want to try out this recipe in Minikube, you’ll need to add --extra-config=apiserver.Authorization.Mode=RBAC when executing the binary.
To list the roles available in a namespace, do this:
$ kubectl get roles -n=kube-system NAME AGE extension-apiserver-authentication-reader 1d system::leader-locking-kube-controller-manager 1d system::leader-locking-kube-scheduler 1d system:controller:bootstrap-signer 1d system:controller:cloud-provider 1d system:controller:token-cleaner 1d $ kubectl get clusterroles -n=kube-system NAME AGE admin 1d cluster-admin 1d edit 1d system:auth-delegator 1d system:basic-user 1d system:controller:attachdetach-controller 1d system:controller:certificate-controller 1d system:controller:cronjob-controller 1d system:controller:daemon-set-controller 1d system:controller:deployment-controller 1d system:controller:disruption-controller 1d system:controller:endpoint-controller 1d system:controller:generic-garbage-collector 1d system:controller:horizontal-pod-autoscaler 1d system:controller:job-controller 1d system:controller:namespace-controller 1d system:controller:node-controller 1d system:controller:persistent-volume-binder 1d system:controller:pod-garbage-collector 1d system:controller:replicaset-controller 1d system:controller:replication-controller 1d system:controller:resourcequota-controller 1d system:controller:route-controller 1d system:controller:service-account-controller 1d system:controller:service-controller 1d system:controller:statefulset-controller 1d system:controller:ttl-controller 1d system:discovery 1d system:heapster 1d system:kube-aggregator 1d system:kube-controller-manager 1d system:kube-dns 1d system:kube-scheduler 1d system:node 1d system:node-bootstrapper 1d system:node-problem-detector 1d system:node-proxier 1d system:persistent-volume-provisioner 1d view 1d
The output shows the predefined roles, which you can use directly for users and service accounts.
To further explore a certain role and understand what actions are allowed, use:
$ kubectl describe clusterroles/view -n=kube-system Name: view Labels: kubernetes.io/bootstrapping=rbac-defaults Annotations: rbac.authorization.kubernetes.io/autoupdate=true PolicyRule: Resources Non-Resource URLs ... ... --------- ----------------- --- --- bindings [] ... ... configmaps [] ... ... cronjobs.batch [] ... ... daemonsets.extensions [] ... ... deployments.apps [] ... ... deployments.extensions [] ... ... deployments.apps/scale [] ... ... deployments.extensions/scale [] ... ... endpoints [] ... ... events [] ... ... horizontalpodautoscalers.autoscaling [] ... ... ingresses.extensions [] ... ... jobs.batch [] ... ... limitranges [] ... ... namespaces [] ... ... namespaces/status [] ... ... persistentvolumeclaims [] ... ... pods [] ... ... pods/log [] ... ... pods/status [] ... ... replicasets.extensions [] ... ... replicasets.extensions/scale [] ... ... replicationcontrollers [] ... ... replicationcontrollers/scale [] ... ... replicationcontrollers.extensions/scale [] ... ... replicationcontrollers/status [] ... ... resourcequotas [] ... ... resourcequotas/status [] ... ... scheduledjobs.batch [] ... ... serviceaccounts [] ... ... services [] ... ... statefulsets.apps [] ... ...
In addition to the default roles defined in the kube-system namespace, you can define your own; see Recipe 10.3.
Tip
When RBAC is enabled, in many environments (including Minikube and GKE) you might see a Forbidden (403) status code and an error message as shown below when you try to access the Kubernetes dashboard:
User “system:serviceaccount:kube-system:default” cannot list pods in the namespace “sec”. (get pods)
To access the dashboard, you’ll need to give the kube-system:default service account the necessary rights:
$ kubectl create clusterrolebinding admin4kubesystem --clusterrole=cluster-admin --serviceaccount=kube-system:default
Note that this command gives the service account a lot of rights and might not be advisable in a production environment.
Discussion
As you can see in Figure 10-1, there are a couple of moving parts when dealing with RBAC authorization:
-
An entity—that is, a group, user, or service account
-
A resource, such as a pod, service, or secret
-
A role, which defines rules for actions on a resource
-
A role binding, which applies a role to an entity
Figure 10-1. The RBAC concept
The actions on a resource that a role uses in its rules are the so-called verbs:
-
get,list,watch -
create -
update/patch -
delete
Concerning the roles, we differentiate between two types:
-
Cluster-wide: cluster roles and their respective cluster role bindings
-
Namespace-wide: roles and role bindings
In Recipe 10.3, we will further discuss how you can create your own rules and apply them to users and resources.
See Also
-
Kubernetes Authorization Overview
10.3 Controlling Access to Resources
Solution
Let’s assume you want to restrict an app to only be able to view pods—that is, list pods and get details about pods.
You’d start off with a pod definition in a YAML manifest, pod-with-sa.yaml, using a dedicated service account, myappsa (see Recipe 10.1):
kind:PodapiVersion:v1metadata:name:myappnamespace:secspec:serviceAccountName:myappsacontainers:-name:mainimage:centos:7command:-"bin/bash"-"-c"-"sleep10000"
Next, you’d define a role—let’s call it podreader in the manifest pod-reader.yaml—that defines the allowed actions on resources:
kind:RoleapiVersion:rbac.authorization.k8s.io/v1beta1metadata:name:podreadernamespace:secrules:-apiGroups:[""]resources:["pods"]verbs:["get","list"]
Last but not least you need to apply the role podreader to the service account myappsa, using a role binding in pod-reader-binding.yaml:
kind:RoleBindingapiVersion:rbac.authorization.k8s.io/v1beta1metadata:name:podreaderbindingnamespace:secroleRef:apiGroup:rbac.authorization.k8s.iokind:Rolename:podreadersubjects:-kind:ServiceAccountname:myappsanamespace:sec
When creating the respective resources, you can use the YAML manifests directly (assuming the service account has already been created):
$ kubectl create -f pod-reader.yaml $ kubectl create -f pod-reader-binding.yaml $ kubectl create -f pod-with-sa.yaml
Rather than creating manifests for the role and the role binding, you can use the following commands:
$ kubectl create role podreader
--verb=get --verb=list
--resource=pods -n=sec
$ kubectl create rolebinding podreaderbinding
--role=sec:podreader
--serviceaccount=sec:myappsa
--namespace=sec -n=sec
Note that this is a case of namespaced access control setup, since you’re using roles and role bindings. For cluster-wide access control, you’d use the corresponding create clusterrole and create clusterrolebinding commands.
Tip
Sometimes it’s not obvious if you should use a role or a cluster role and/or role binding, so here are a few rules of thumb you might find useful:
-
If you want to restrict access to a namespaced resource (like a service or pod) in a certain namespace, use a role and a role binding (as we did in this recipe).
-
If you want to reuse a role in a couple of namespaces, use a cluster role with a role binding.
-
If you want to restrict access to cluster-wide resources such as nodes or to namespaced resources across all namespaces, use a cluster role with a cluster role binding.
See Also
-
Antoine Cotten’s blog post “Kubernetes v1.7 Security in Practice”
10.4 Securing Pods
Solution
To enforce policies on the pod level in Kubernetes, use the securityContext field in a pod specification.
Let’s assume you want an app running as a nonroot user. For this, you’d use the security context on the container level as shown in the following manifest, securedpod.yaml:
kind:PodapiVersion:v1metadata:name:secpodspec:containers:-name:shellimage:centos:7command:-"bin/bash"-"-c"-"sleep10000"securityContext:runAsUser:5000
Now create the pod and check the user under which the container runs:
$ kubectl create -f securedpod.yaml pod "secpod" created $ kubectl exec secpod ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 5000 1 0.0 0.0 4328 672 ? Ss 12:39 0:00 sleep 10000 5000 8 0.0 0.1 47460 3108 ? Rs 12:40 0:00 ps aux
As expected, it’s running as the user with ID 5000. Note that you can also use the securityContext field on the pod level rather than on specific containers.
A more powerful method to enforce policies on the pod level is to use pod security policies (PSP). These are cluster-wide resources that allows you to define a range of policies, including some similar to what you’ve seen here but also restrictions around storage and networking. For a walk-through on how to use PSPs, see “Secure a Kubernetes Cluster with Pod Security Policies” in the Bitnami docs for Kubernetes.