Chapter 8: Securing Kubernetes Pods
Even though a pod is the most fine-grained unit that serves as a placeholder to run microservices, securing Kubernetes pods is a vast topic as it should cover the entire DevOps flow: build, deployment, and runtime.
In this chapter, we choose to narrow our focus to the build and runtime stages. To secure Kubernetes pods in the build stage, we will talk about how to harden a container image and configure the security attributes of pods (or pod templates) to reduce the attack surface. Although some of the security attributes of workloads, such as AppArmor and SELinux labels, take effect in the runtime stage, security control has already been defined for the workload. To clarify matters further, we're trying to secure Kubernetes workloads by configuring the runtime effect security attributes in the build stage. To secure Kubernetes pods in the runtime stage, we will introduce a PodSecurityPolicy with examples along with the facilitating tool, kube-psp-advisor.
Later chapters will go into more detail regarding runtime security and response. Also note that exploitation of the application may lead to pods getting compromised. However, we don't intend to cover application in this chapter.
In this chapter, we will cover the following topics:
- Hardening container images
- Configuring the security attributes of pods
- The power of PodSecurityPolicy
Hardening container images
Container image hardening means to follow security best practices or baselines to configure a container image in order to reduce the attack surface. Image scanning tools only focus on finding publicly disclosed issues in applications bundled inside the image. But, following the best practices along with secure configuration while building the image ensures that the application has a minimal attack surface.
Before we start talking about the secure configuration baseline, let's look at what a container image is, as well as a Dockerfile, and how it is used to build an image.
Container images and Dockerfiles
A container image is a file that bundles the microservice binary, its dependencies, and configurations of the microservice, and so on. A container is a running instance of an image. Nowadays, application developers not only write code to build microservices; they also need to build the Dockerfile to containerize the microservice. To help build a container image, Docker offers a standardized approach, known as a Dockerfile. A Dockerfile contains a series of instructions, such as copy files, configure environment variables, configure open ports, and container entry points, which can be understood by the Docker daemon to construct the image file. Then, the image file will be pushed to the image registry from where the image is then deployed in Kubernetes clusters. Each Dockerfile instruction will create a file layer in the image.
Before we look at an example of a Dockerfile, let's understand some basic Dockerfile instructions:
- FROM: Initialize a new build stage from the base image or parent image. Both mean the foundation or the file layer on which you're bundling your own image.
- RUN: Execute commands and commit the results on top of the previous file layer.
- ENV: Set environment variables for the running containers.
- CMD: Specify the default commands that the containers will run.
- COPY/ADD: Both commands copy files or directories from the local (or remote) URL to the filesystem of the image.
- EXPOSE: Specify the port that the microservice will be listening on during container runtime.
- ENTRYPOINT: Similar to CMD, the only difference is that ENTRYPOINT makes a container that will run as an executable.
- WORKDIR: Sets the working directory for the instructions that follow.
- USER: Sets the user and group ID for any CMD/ENTRYPOINT of containers.
Now, let's take a look at an example of a Dockerfile:
FROM ubuntu
# install dependencies
RUN apt-get install -y software-properties-common python
RUN add-apt-repository ppa:chris-lea/node.js
RUN echo "deb http://us.archive.ubuntu.com/ubuntu/ precise universe" >> /etc/apt/sources.list
RUN apt-get update
RUN apt-get install -y nodejs
# make directory
RUN mkdir /var/www
# copy app.js
ADD app.js /var/www/app.js
# set the default command to run
CMD ["/usr/bin/node", "/var/www/app.js"]
From the preceding Dockerfile, we can tell that the image was built on top of ubuntu. Then, it ran a bunch of apt-get commands to install the dependencies, and created a directory called /var/www. Next, copy the app.js file from the current directory to /var/www/app.js in the filesystem of the image. Finally, configure the default command to run this Node.js application. I believe you will see how straightforward and powerful Dockerfile is when it comes to helping you build an image.
The next question is any security concern, as it looks like you're able to build any kind of image. Next, let's talk about CIS Docker benchmarks.
CIS Docker benchmarks
Center for Internet Security (CIS) put together a guideline regarding Docker container administration and management. Now, let's take a look at the security recommendations from CIS Docker benchmarks regarding container images:
- Create a user for a container image to run a microservice: It is good practice to run a container as non-root. Although user namespace mapping is available, it is not enabled by default. Running as root means that if an attacker were to successfully escape from the container, they would gain root access to the host. Use the USER instruction to create a user in the Dockerfile.
- Use trusted base images to build your own image: Images downloaded from public repositories cannot be fully trusted. It is well known that images from public repositories may contain malware or crypto miners. Hence, it is recommended that you build your image from scratch or use minimal trusted images, such as Alpine. Also, perform the image scan after your image has been built. Image scanning will be covered in the next chapter.
- Do not install unnecessary packages in your image: Installing unnecessary packages will increase the attack surface. It is recommended that you keep your image slim. Occasionally, you will probably need to install some tools during the process of building an image. Do remember to remove them at the end of the Dockerfile.
- Scan and rebuild an image in order to apply security patches: It is highly likely that new vulnerabilities will be discovered in your base image or in the packages you install in your image. It is good practice to scan your image frequently. Once you identify any vulnerabilities, try to patch the security fixes by rebuilding the image. Image scanning is a critical mechanism for identifying vulnerabilities at the build stage. We will cover image scanning in more detail in the next chapter.
- Enable content trust for Docker: Content trust uses digital signatures to ensure data integrity between the client and the Docker registry. It ensures the provenance of the container image. However, it is not enabled by default. You can turn it on by setting the environment variable, DOCKER_CONTENT_TRUST, to 1.
- Add a HEALTHCHECK instruction to the container image: A HEALTHCHECK instruction defines a command to ask Docker Engine to check the health status of the container periodically. Based on the health status check result, Docker Engine then exits the non-healthy container and initiates a new one.
- Ensure that updates are not cached in Dockerfile: Depending on the base image you choose, you may need to update the package repository before installing new packages. However, if you specify RUN apt-get update (Debian) in a single line in the Dockerfile, Docker Engine will cache this file layer, so, when you build your image again, it will still use the old package repository information that is cached. This will prevent you from using the latest packages in your image. Therefore, either use update along with install in a single Dockerfile instruction or use the --no-cache flag in the Docker build command.
- Remove setuid and setgid permission from files in the image: setuid and setgid permissions can be used for privilege escalation as files with such permissions are allowed to be executed with owners' privileges instead of launchers' privileges. You should carefully review the files with setuid and setgid permissions and remove those files that don't require such permissions.
- Use COPY instead of ADD in the Dockerfile: The COPY instruction can only copy files from the local machine to the filesystem of the image, while the ADD instruction can not only copy files from the local machine but also retrieve files from the remote URL to the filesystem of the image. Using ADD may introduce the risk of adding malicious files from the internet to the image.
- Do not store secrets in the Dockerfile: There are many tools that are able to extract image file layers. If there are any secrets stored in the image, secrets are no longer secrets. Storing secrets in the Dockerfile renders containers potentially exploitable. A common mistake is to use the ENV instruction to store secrets in environment variables.
- Install verified packages only: This is similar to using the trusted base image only. Observe caution as regards the packages you are going to install within your image. Make sure they are from trusted package repositories.
If you follow the security recommendations from the preceding CIS Docker benchmarks, you will be successful in hardening your container image. This is the first step in securing pods in the build stage. Now, let's look at the security attributes we need to pay attention to in order to secure a pod.
Configuring the security attributes of pods
As we mentioned in the previous chapter, application developers should be aware of what privileges a microservice must have in order to perform tasks. Ideally, application developers and security engineers work together to harden the microservice at the pod and container level by configuring the security context provided by Kubernetes.
We classify the major security attributes into four categories:
- Setting host namespaces for pods
- Security context at the container level
- Security context at the pod level
- AppArmor profile
By employing such a means of classification, you will find them easy to manage.
Setting host-level namespaces for pods
The following attributes in the pod specification are used to configure the use of host namespaces:
- hostPID: By default, this is false. Setting it to true allows the pod to have visibility on all the processes in the worker node.
- hostNetwork: By default, this is false. Setting it to true allows the pod to have visibility on all the network stacks in the worker node.
- hostIPC: By default, this is false. Setting it to true allows the pod to have visibility on all the IPC resources in the worker node.
The following is an example of how to configure the use of host namespaces at the pod level in an ubuntu-1 pod YAML file:
apiVersion: v1
kind: Pod
metadata:
name: ubuntu-1
labels:
app: util
spec:
containers:
- name: ubuntu
image: ubuntu
imagePullPolicy: Always
hostPID: true
hostNetwork: true
hostIPC: true
The preceding workload YAML configured the ubuntu-1 pod to use a host-level PID namespace, network namespace, and IPC namespace. Keep in mind that you shouldn't set these attributes to true unless necessary. Setting these attributes to true also disarms the security boundaries of other workloads in the same worker node, as has already been mentioned in Chapter 5, Configuring Kubernetes Security Boundaries.
Security context for containers
Multiple containers can be grouped together inside the same pod. Each container can have its own security context, which defines privileges and access controls. The design of a security context at a container level provides a more fine-grained security control for Kubernetes workloads. For example, you may have three containers running inside the same pod and one of them has to run in privileged mode, while the others run in non-privileged mode. This can be done by configuring a security context for individual containers.
The following are the principal attributes of a security context for containers:
- privileged: By default, this is false. Setting it to true essentially makes the processes inside the container equivalent to the root user on the worker node.
- capabilities: There is a default set of capabilities granted to the container by the container runtime. The default capabilities granted are as follows: CAP_SETPCAP, CAP_MKNOD, CAP_AUDIT_WRITE, CAP_CHOWN, CAP_NET_RAW, CAP_DAC_OVERRIDE, CAP_FOWNER, CAP_FSETID, CAP_KILL, CAP_SETGID, CAP_SETUID, CAP_NET_BIND_SERVICE, CAP_SYS_CHROOT, and CAP_SETFCAP.
You may add extra capabilities or drop some of the defaults by configuring this attribute. Capabilities such as CAP_SYS_ADMIN and CAP_NETWORK_ADMIN should be added with caution. For the default capabilities, you should also drop those that are unnecessary.
- allowPrivilegeEscalation: By default, this is true. Setting it directly controls the no_new_privs flag, which will be set to the processes in the container. Basically, this attribute controls whether the process can gain more privileges than its parent process. Note that if the container runs in privileged mode, or has the CAP_SYS_ADMN capability added, this attribute will be set to true automatically. It is good practice to set it to false.
- readOnlyRootFilesystem: By default, this is false. Setting it to true makes the root filesystem of the container read-only, which means that the library files, configuration files, and so on are read-only and cannot be tampered with. It is a good security practice to set it to true.
- runAsNonRoot: By default, this is false. Setting it to true enables validation that the processes in the container cannot run as a root user (UID=0). Validation is done by kubelet. With runAsNonRoot set to true, kubelet will prevent the container from starting if run as a root user. It is a good security practice to set it to true. This attribute is also available in PodSecurityContext, which takes effect at pod level. If this attribute is set in both SecurityContext and PodSecurityContext, the value specified at the container level takes precedence.
- runAsUser: This is designed to specify to the UID to run the entrypoint process of the container image. The default setting is the user specified in the image's metadata (for example, the USER instruction in the Dockerfile). This attribute is also available in PodSecurityContext, which takes effect at the pod level. If this attribute is set in both SecurityContext and PodSecurityContext, the value specified at the container level takes precedence.
- runAsGroup: Similar to runAsUser, this is designed to specify the Group ID or GID to run the entrypoint process of the container. This attribute is also available in PodSecurityContext, which takes effect at the pod level. If this attribute is set in both SecurityContext and PodSecurityContext, the value specified at the container level takes precedence.
- seLinuxOptions: This is designed to specify the SELinux context to the container. By default, the container runtime will assign a random SELinux context to the container if not specified. This attribute is also available in PodSecurityContex, which takes effect at the pod level. If this attribute is set in both SecurityContext and PodSecurityContext, the value specified at the container level takes precedence.
Since you now understand what these security attributes are, you may come up with your own hardening strategy aligned with your business requirements. In general, the security best practices are as follows:
- Do not run in privileged mode unless necessary.
- Do not add extra capabilities unless necessary.
- Drop unused default capabilities.
- Run containers as a non-root user.
- Enable a runAsNonRoot check.
- Set the container root filesystem as read-only.
Now, let's take a look at an example of configuring SecurityContext for containers:
apiVersion: v1
kind: Pod
metadata:
name: nginx-pod
labels:
app: web
spec:
hostNetwork: false
hostIPC: false
hostPID: false
containers:
- name: nginx
image: kaizheh/nginx
securityContext:
privileged: false
capabilities:
add:
- NETWORK_ADMIN
readOnlyRootFilesystem: true
runAsUser: 100
runAsGroup: 1000
The nginx container inside nginx-pod runs as a user with a UID of 100 and a GID of 1000. In addition to this, the nginx container gains extra NETWORK_ADMIN capability and the root filesystem is set to read-only. The YAML file here only shows an example of how to configure the security context. Note that adding NETWORK_ADMIN is not recommended for containers running in production environments.
Security context for pods
A security context is used at the pod level, which means that security attributes will be applied to all the containers inside the pod.
The following is a list of the principal security attributes at the pod level:
- fsGroup: This is a special supplemental group applied to all containers. The effectiveness of this attribute depends on the volume type. Essentially, it allows kubelet to set the ownership of the mounted volume to the pod with the supplemental GID.
- sysctls: sysctls is used to configure kernel parameters at runtime. In such a context, sysctls and kernel parameters are used interchangeably. These sysctls commands are namespaced kernel parameters that apply to the pod. The following sysctls commands are known to be namespaced: kernel.shm*, kernel.msg*, kernel.sem, and kernel.mqueue.*. Unsafe sysctls are disabled by default and should not be enabled in production environments.
- runAsUser: This is designed to specify the UID to run the entrypoint process of the container image. The default setting is the user specified in the image's metadata (for example, the USER instruction in the Dockerfile). This attribute is also available in SecurityContext, which takes effect at the container level. If this attribute is set in both SecurityContext and PodSecurityContext, the value specified at the container level takes precedence.
- runAsGroup: Similar to runAsUser, this is designed to specify the GID to run the entrypoint process of the container. This attribute is also available in SecurityContext, which takes effect at the container level. If this attribute is set in both SecurityContext and PodSecurityContext, the value specified at the container level takes precedence.
- runAsNonRoot: Set to false by default, setting it to true enables validation that the processes in the container cannot run as a root user (UID=0). Validation is done by kubelet. By setting it to true, kubelet will prevent the container from starting if run as a root user. It is a good security practice to set it to true. This attribute is also available in SecurityContext, which takes effect at the container level. If this attribute is set in both SecurityContext and PodSecurityContext, the value specified at the container level takes precedence.
- seLinuxOptions: This is designed to specify the SELinux context to the container. By default, the container runtime will assign a random SELinux context to the container if not specified. This attribute is also available in SecurityContext, which takes effect at the container level. If this attribute is set in both SecurityContext and PodSecurityContext, the value specified at the container level takes precedence.
Notice that the attributes runAsUser, runAsGroup, runAsNonRoot, and seLinuxOptions are available both in SecurityContext at the container level and PodSecurityContext at the pod level. This gives users both the flexibility and extreme importance of security control. fsGroup and sysctls are not as commonly used as the others, so only use them when you have to.
AppArmor profiles
An AppArmor profile usually defines what Linux capabilities the process owns, what network resources and files can be accessed by the container, and so on. In order to use an AppArmor profile to protect pods or containers, you will need to update the annotation of the pod. Let's look at an example, assuming you have an AppArmor profile to block any file write activities:
#include <tunables/global>
profile k8s-apparmor-example-deny-write flags=(attach_disconnected) {
#include <abstractions/base>
file,
# Deny all file writes.
deny /** w,
}
Note that AppArmor is not a Kubernetes object, like a pod, deployment, and so on. It can't be operated through kubectl. You will have to SSH to each node and load the AppArmor profile into the kernel so that the pod may be able to use it.
The following is the command for loading the AppArmor profile:
cat /etc/apparmor.d/profile.name | sudo apparmor_parser -a
Then, put the profile into enforce mode:
sudo aa-enforce /etc/apparmor.d/profile.name
Once the AppArmor profile is loaded, you can update the annotation of the pod to use the AppArmor profile to protect your container. Here is an example of applying an AppArmor profile to containers:
apiVersion: v1
kind: Pod
metadata:
name: hello-apparmor
annotations:
# Tell Kubernetes to apply the AppArmor profile
# "k8s-apparmor-example-deny-write".
container.apparmor.security.beta.kubernetes.io/hello:
localhost/k8s-apparmor-example-deny-write
spec:
containers:
- name: hello
image: busybox
command: [ "sh", "-c", "echo 'Hello AppArmor!' && sleep 1h" ]
The container inside hello-apparmor does nothing but sleep after echoing the Hello AppArmor! message. When it is running, if you launch a shell from a container and write to any file, it will be blocked by AppArmor. Even though writing a robust AppArmor profile is not easy, you can still create some basic restrictions, such as denying writing to certain directories, denying accepting raw packets, and making certain files read-only. Also, test the profile first before applying it to the production cluster. Open source tools such as bane can help create AppArmor profiles for containers.
We do not intend to dive into the seccomp profile in this book since writing a seccomp profile for a microservice is not easy. Even an application developer doesn't have knowledge of what system calls are legitimate for the microservice they developed. Although you can turn the audit mode on to avoid breaking the microservice's functionality, building a robust seccomp profile is still a long way off. Another reason is that this feature is still in the alpha stage up to version 1.17. According to Kubernetes' official documentation, being alpha means it is disabled by default, perhaps buggy, and only recommended to run in a short-lived testing cluster. When there are any new updates on seccomp, we may come back to introduce seccomp in more detail at a later date.
We've covered how to secure Kubernetes pods in the build time. Next, let's look at how we can secure Kubernetes pods during runtime.
The power of PodSecurityPolicy
A Kubernetes PodSecurityPolicy is a cluster-level resource that controls security-sensitive aspects of the pod specification through which the access privileges of a Kubernetes pod are limited. As a DevOps engineer, you may want to use a PodSecurityPolicy to restrict most of the workloads run in limited access privileges, while only allowing a few workloads to be run with extra privileges.
In this section, we will first take a closer look at a PodSecurityPolicy, and then we will introduce an open source tool, kube-psp-advisor, which can help build an adaptive PodSecurityPolicy for the running Kubernetes cluster.
Understanding PodSecurityPolicy
You can think of a PodSecurityPolicy as a policy to evaluate the security attributes defined in the pod's specification. Only those pods whose security attributes meet the requirements of PodSecurityPolicy will be admitted to the cluster. For example, PodSecurityPolicy can be used to block the launch of most privileged pods, while only allowing those necessary or limited pods access to the host filesystem.
The following are the principal security attributes that are controlled by PodSecurityPolicy:
- privileged: Determines whether a pod can run in privileged mode.
- hostPID: Determines whether a pod can use a host PID namespace.
- hostNetwork: Determines whether a pod can use a host network namespace.
- hostIPC: Determines whether a pod can use a host IPC namespace. The default setting is true.
- allowedCapabilities: Specifies a list of capabilities that could be added to containers. The default setting is empty.
- defaultAddCapabilities: Specifies a list of capabilities that will be added to containers by default. The default setting is empty.
- requiredDropCapabilities: Specifies a list of capabilities that will be dropped from containers. Note that a capability cannot be specified in both the allowedCapabilities and requiredDropCapabilities fields. The default setting is empty.
- readOnlyRootFilesystem: When set to true, the PodSecurityPolicy will force containers to run with a read-only root filesystem. If the attribute is set to false explicitly in the security context of the container, the pod will be denied from running. The default setting is false.
- runAsUser: Specifies the allowable user IDs that may be set in the security context of pods and containers. The default setting allows all.
- runAsGroup: Specifies the allowable group IDs that may be set in the security context of pods and containers. The default setting allows all.
- allowPrivilegeEscalation: Determines whether a pod can submit a request to allow privilege escalation. The default setting is true.
- allowedHostPaths: Specifies a list of host paths that could be mounted by the pod. The default setting allows all.
- volumes: Specifies a list of volume types that can be mounted by the pod. For example, secret, configmap, and hostpath are the valid volume types. The default setting allows all.
- seLinux: Specifies the allowable seLinux labels that may be set in the security context of pods and containers. The default setting allows all.
- allowedUnsafeSysctl: Allows unsafe sysctls to run. The default setting allows none.
Now, let's take a look at an example of a PodSecurityPolicy:
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: example
spec:
allowedCapabilities:
- NET_ADMIN
- IPC_LOCK
allowedHostPaths:
- pathPrefix: /dev
- pathPrefix: /run
- pathPrefix: /
fsGroup:
rule: RunAsAny
hostNetwork: true
privileged: true
runAsUser:
rule: RunAsAny
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
volumes:
- hostPath
- secret
This PodSecurityPolicy allows the NET_ADMIN and IPC_LOCK capabilities, mounts /, /dev, and /run from the host and Kubernetes' secret volumes. It doesn't enforce any filesystem group ID or supplemental groups and it also allows the container to run as any user, access the host network namespace, and run as a privileged container. No SELinux policy is enforced in the policy.
To enable this Pod Security Policy, you can run the following command:
$ kubectl apply -f example-psp.yaml
Now, let's verify that the Pod Security Policy has been created successfully:
$ kubectl get psp
The output will appear as follows:
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
example true NET_ADMIN, IPC_LOCK RunAsAny RunAsAny RunAsAny RunAsAny false hostPath,secret
After you have created the Pod Security Policy, there is one more step required in order to enforce it. You will have to grant the privilege of using the PodSecurityPolicy object to the users, groups, or service accounts. By doing so, the pod security policies are entitled to evaluate the workloads based on the associated service account. Here is an example of how to enforce a PodSecurityPolicy. First, you will need to create a cluster role that uses the PodSecurityPolicy:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: use-example-psp
rules:
- apiGroups: ['policy']
resources: ['podsecuritypolicies']
verbs: ['use']
resourceNames:
- example
Then, create a RoleBinding or ClusterRoleBinding object to associate the preceding ClusterRole object created with the service accounts, users, or groups:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: use-example-psp-binding
roleRef:
kind: ClusterRole
name: use-example-psp
apiGroup: rbac.authorization.k8s.io
subjects:
# Authorize specific service accounts:
- kind: ServiceAccount
name: test-sa
namespace: psp-test
The preceding use-example-pspbinding.yaml file created a RoleBinding object to associate the use-example-psp cluster role with the test-sa service account in the psp-test namespace. With all of these set up, any workloads in the psp-test namespace whose service account is test-sa will run through the PodSecurityPolicy example's evaluation. And only those that meet the requirements will be admitted to the cluster.
From the preceding example, think of there being different types of workloads running in your Kubernetes cluster, and each of them may require different privileges to access different types of resources. It would be a challenge to create and manage pod security policies for different workloads. Now, let's take a look at kube-psp-advisor and see how it can help create pod security policies for you.
Kubernetes PodSecurityPolicy Advisor
Kubernetes PodSecurityPolicy Advisor (also known as kube-psp-advisor) is an open source tool from Sysdig. It scans the security attributes of running workloads in the cluster and then, on this basis, recommends pod security policies for your cluster or workloads.
First, let's install kube-psp-advisor as a kubectl plugin. If you haven't installed krew, a kubectl plugin management tool, please follow the instructions (https://github.com/kubernetes-sigs/krew#installation) in order to install it. Then, install kube-psp-advisor with krew as follows:
$ kubectl krew install advise-psp
Then, you should be able to run the following command to verify the installation:
$ kubectl advise-psp
A way to generate K8s PodSecurityPolicy objects from a live K8s environment or individual K8s objects containing pod specifications
Usage:
kube-psp-advisor [command]
Available Commands:
convert Generate a PodSecurityPolicy from a single K8s Yaml file
help Help about any command
inspect Inspect a live K8s Environment to generate a PodSecurityPolicy
Flags:
-h, --help help for kube-psp-advisor
--level string Log level (default "info")
To generate pod security policies for workloads in a namespace, you can run the following command:
$ kubectl advise-psp inspect --grant --namespace psp-test
The preceding command generates pod security policies for workloads running inside the psp-test namespace. If the workload uses a default service account, no PodSecurityPolicy will be generated for it. This is because the default service account will be assigned to the workload that does not have a dedicated service account associated with it. And you certainly don't want to have a default service account that is able to use a PodSecurityPolicy for privileged workloads.
Here is an example of output generated by kube-psp-advisor for workloads in the psp-test namespace, including Role, RoleBinding, and PodSecurityPolicy in a single YAML file with multiple pod security policies. Let's take a look at one of the recommended PodSecurityPolicy:
# Pod security policies will be created for service account 'sa-1' in namespace 'psp-test' with following workloads:
# Kind: ReplicaSet, Name: busy-rs, Image: busybox
# Kind: Pod, Name: busy-pod, Image: busybox
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
creationTimestamp: null
name: psp-for-psp-test-sa-1
spec:
allowedCapabilities:
- SYS_ADMIN
allowedHostPaths:
- pathPrefix: /usr/bin
readOnly: true
fsGroup:
rule: RunAsAny
hostIPC: true
hostNetwork: true
hostPID: true
runAsUser:
rule: RunAsAny
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
volumes:
- configMap
- secret
- hostPath
Following is the Role generated by kube-psp-advisor:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: use-psp-by-psp-test:sa-1
namespace: psp-test
rules:
- apiGroups:
- policy
resourceNames:
- psp-for-psp-test-sa-1
resources:
- podsecuritypolicies
verbs:
- use
---
Following is the RoleBinding generated by kube-psp-advisor:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
creationTimestamp: null
name: use-psp-by-psp-test:sa-1-binding
namespace: psp-test
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: use-psp-by-psp-test:sa-1
subjects:
- kind: ServiceAccount
name: sa-1
namespace: psp-test
---
The preceding section is the recommended PodSecurityPolicy, psp-for-psp-test-sa-1, for the busy-rs and busy-pod workloads, since these two workloads share the same service account, sa-1. Hence, Role and RoleBinding are created to use the Pod Security Policy, psp-for-psp-test-sa-1, respectively. The PodSecurityPolicy is generated based on the aggregation of the security attributes of workloads using the sa-1 service account:
---
# Pod security policies will NOT be created for service account 'default' in namespace 'psp-test' with following workdloads:
# Kind: ReplicationController, Name: busy-rc, Image: busybox
---
The preceding section mentions that the busy-rc workload uses a default service account, so there is no Pod Security Policy created for it. This is a reminder that if you want to generate pod security policies for workloads, don't use the default service account.
Building a Kubernetes PodSecurityPolicy is not straightforward, although it would be ideal if a single restricted PodSecurityPolicy was to apply to the entire cluster and all workloads complied with it. DevOps engineers need to be creative in order to build restricted pod security policies while not breaking workloads' functionalities. kube-psp-advisor makes the implementation of Kubernetes pod security policies simple, adapts to your application requirements and, specifically, is fine-grained for each one to allow only the privilege of least access.
Summary
In this chapter, we covered how to harden a container image with CIS Docker benchmarks, and then we gave a detailed introduction to the security attributes of Kubernetes workloads. Next, we looked at the PodSecurityPolicy in detail and introduced the kube-psp-advisor open source tool, which facilitates the establishment of pod security policies.
Securing Kubernetes workloads is not a one-shot thing. Security controls need to be applied from the build, deployment, and runtime stages. It starts with hardening container images, and then configuring security attributes of Kubernetes workloads in a secure way. This happens at the build stage. It is also important to build adaptive pod security policies for different Kubernetes workloads. The goal is to restrict most of the workloads to run with limited privileges, while allowing only a few workloads to run with extra privileges, and without breaking workload availability. This happens at the runtime stage. kube-psp-advisor is able to help build adaptive pod security policies.
In the next chapter, we will talk about image scanning. It is critical in helping to secure Kubernetes workloads in the DevOps workflow.
Questions
- What does HEALTHCHECK do in a Dockerfile?
- Why use COPY instead of ADD in a Dockerfile?
- If your application doesn't listen on any port, which default capabilities can be dropped?
- What does the runAsNonRoot attribute control?
- When you create a PodSecurityPolicy object, what else do you need to do in order to enforce that Pod Security Policy on workloads?
Further reading
You can refer to the following links for more information on the topics covered in this chapter:
- To learn more about kube-psp-advisor, please visit the following link: https://github.com/sysdiglabs/kube-psp-advisor
- To learn more about AppArmor, please visit the following link: https://gitlab.com/apparmor/apparmor/-/wikis/Documentation
- To learn more about bane, please visit the following link: https://github.com/genuinetools/bane