Chapter 4: Applying the Principle of Least Privilege in Kubernetes

The principle of least privilege states that each component of an ecosystem should have minimal access to data and resources for it to function. In a multitenant environment, multiple resources can be accessed by different users or objects. The principle of least privilege ensures that damage to the cluster is minimal if users or objects misbehave in such environments.

In this chapter, we will first introduce the principle of least privilege. Given the complexity of Kubernetes, we will first look into the Kubernetes subjects, and then the privileges available for the subjects. Then, we will talk about the privileges of Kubernetes objects and possible ways to restrict them. The goal of this chapter is to help you understand a few critical concepts, such as the principle of least privilege and Role-Based Access Control (RBAC). In this chapter, we will talk about different Kubernetes objects, such as namespaces, service accounts, Roles, and RoleBindings, and Kubernetes security features, such as the security context, the PodSecurityPolicy, and the NetworkPolicy, which can be leveraged to implement the principle of least privilege for your Kubernetes cluster.

In this chapter, we will cover the following topics:

  • The principle of least privilege
  • Least privilege of Kubernetes subjects
  • Least privilege of Kubernetes workloads

The principle of least privilege

Privilege is the authority to perform an action such as accessing a resource or processing some data. The principle of least privilege is the idea that any subject, user, program, process, and so on should only have the minimum required privileges to perform its function. For example, Alice, a regular Linux user, is able to create a file under her own home directory. In other words, Alice at least has the privilege or permission to create a file under her home directory. However, Alice may not be able to create a file under another user's directory because she doesn't have the privilege or permission to do so. If none of Alice's daily tasks actually exercises the privilege to create a file in the home directory, but she does have the privilege to do so, then the administrator for the machine is not complying with the principle of least privilege. In this section, we will first introduce the concept of the authorization model from which the concept of least privilege derived, and then, we will talk about the benefits of implementing the principle of least privilege.

Authorization model

When we talk about least privilege, most of the time we talk in the context of authorization, and in different environments, there will be different authorization models. For example, an Access Control List (ACL) is widely used in Linux and network firewalls, while RBAC is used in database systems. It is also up to the administrator of the environment to define authorization policies to ensure least privilege based on authorization models available in the system. The following list defines some popular authorization models:

  • ACL: An ACL defines a list of permissions associated with objects. It specifies which subjects are granted access to objects, as well as what operations are allowed on given objects. For example, the -rw file permission is read-write-only by the file owner.
  • RBAC: The authorization decision is based on a subject's roles, which contain a group of permissions or privileges. For example, in Linux, a user is added to different groups (such as staff) to grant access to some folders instead of individually being granted access to folders on the filesystem.
  • Attribute-Based Access Control (ABAC): The authorization decision is based on a subject's attributes, such as labels or properties. An attribute-based rule checks user attributes such as user.id="12345", user.project="project", and user.status="active" to decide whether a user is able to perform a task.

Kubernetes supports both ABAC and RBAC. Though ABAC is powerful and flexible, the implementation in Kubernetes makes it difficult to manage and understand. Thus, it is recommended to enable RBAC instead of ABAC in Kubernetes. Besides RBAC, Kubernetes also provides multiple ways to restrict resource access. Before we look into RBAC and ABAC in Kubernetes in the next sections, let's discuss the benefits of ensuring least privilege.

Rewards of the principle of least privilege

Though it might take quite some time to understand what the minimum privileges for subjects are in order to perform their functions, the rewards are also significant if the principle of least privilege has been implemented in your environment:

  • Better security: Inside threats, malware propagation, lateral movement, and so on can be mitigated with the implementation of the principle of least privilege. The leak by Edward Snowden happened because of a lack of least privilege.
  • Better stability: Given the subjects are properly granted with necessary privileges only, subjects' activities become more predictable. In return, system stability is bolstered.
  • Improved audit readiness: Given the subjects are properly granted with necessary privileges only, the audit scope will be reduced dramatically. Additionally, many common regulations call for the implementation of the principle of least privilege as a compliance requirement.

Now that you have seen the benefits for implementing the principle of least privilege, I want to introduce the challenge as well: the openness and configurability of Kubernetes makes implementing the principle of least privilege cumbersome. Let's look at how to apply the principle of least privilege to Kubernetes subjects.

Least privilege of Kubernetes subjects

Kubernetes service accounts, users, and groups communicate with kube-apiserver to manage Kubernetes objects. With RBAC enabled, different users or service accounts may have different privileges to operate Kubernetes objects. For example, users in the system:master group have the cluster-admin role granted, meaning they can manage the entire Kubernetes cluster, while users in the system:kube-proxy group can only access the resources required by the kube-proxy component. First, let's briefly talk about what RBAC is.

Introduction to RBAC

As discussed earlier, RBAC is a model of regulating access to resources based on roles granted to users or groups. From version 1.6 onward, RBAC is enabled by default in Kubernetes. Before version 1.6, RBAC could be enabled by running the Application Programming Interface (API) server with the --authorization-mode=RBAC flag. RBAC eases the dynamic configuration of permission policies using the API server.

The core elements of RBAC include the following:

  1. Subject: Service accounts, users, or groups requesting access to the Kubernetes API.
  2. Resources: Kubernetes objects that need to be accessed by the subject.
  3. Verbs: Different types of access the subject needs on a resource—for example, create, update, list, delete.

Kubernetes RBAC defines the subjects and the type of access they have to different resources in the Kubernetes ecosystem.

Service accounts, users, and groups

Kubernetes supports three types of subject, as follows:

  • Regular users: These users are created by cluster administrators. They do not have a corresponding object in the Kubernetes ecosystem. Cluster administrators usually create users by using the Lightweight Directory Access Protocol (LDAP), Active Directory (AD), or private keys.
  • Service accounts: Pods authenticate to the kube-apiserver object using a service account. Service accounts are created using API calls. They are restricted to namespaces and have associated credentials stored as secrets. By default, pods authenticate as a default service account.
  • Anonymous users: Any API request that is not associated with a regular or a service account is associated with an anonymous user.

Cluster administrators can create new service accounts to be associated with pods by running the following command:

$ kubectl create serviceaccount new_account

A new_account service account will be created in the default namespace. To ensure least privilege, cluster administrators should associate every Kubernetes resource with a service account with least privilege to operate.

Role

A role is a collection of permissions—for example, a role in namespace A can allow users to create pods in namespace A and list secrets in namespace A. In Kubernetes, there are no deny permissions. Thus, a role is an addition of a set of permissions.

A role is restricted to a namespace. On the other hand, a ClusterRole works at the cluster level. Users can create a ClusterRole that spans across the complete cluster. A ClusterRole can be used to mediate access to resources that span across a cluster, such as nodes, health checks, and namespaced objects, such as pods across multiple namespaces. Here is a simple example of a role definition:

kind: Role

apiVersion: rbac.authorization.k8s.io/v1

metadata:

  namespace: default

  name: role-1

rules:

- apiGroups: [""]

  resources: ["pods"]

  verbs: ["get"]

This simple rule allows the get operation to over-resource pods in the default namespace. This role can be created using kubectl by executing the following command:

$ kubectl apply -f role.yaml

A user can only create or modify a role if either one of the following is true:

  • The user has all permissions contained in the role in the same scope (namespaced or cluster-wide).
  • The user is associated with an escalated role in the given scope.

This prevents users from performing privilege escalation attacks by modifying user roles and permissions.

RoleBinding

A RoleBinding object is used to associate a role with subjects. Similar to ClusterRole, ClusterRoleBinding can grant a set of permissions to subjects across namespaces. Let's see a couple of examples:

  1. Create a RoleBinding object to associate a custom-clusterole cluster role to the demo-sa service account in the default namespace, like this:

    kubectl create rolebinding new-rolebinding-sa

         --clusterrole=custom-clusterrole

         --serviceaccount=default:demo-sa

  2. Create a RoleBinding object to associate a custom-clusterrole cluster role to the group-1 group, like this:

    kubectl create rolebinding new-rolebinding-group

         --clusterrole=custom-clusterrole

         --group=group-1

         --namespace=namespace-1

The RoleBinding object links roles to subjects and makes roles reusable and easy to manage.

Kubernetes namespaces

A namespace is a common concept in computer science that provides a logical grouping for related resources. Namespaces are used to avoid name collisions; resources within the same namespace should have unique names, but resources across namespaces can share names. In the Linux ecosystem, namespaces allow the isolation of system resources.

In Kubernetes, namespaces allow a single cluster to be shared between teams and projects logically. With Kubernetes namespaces, the following applies:

  • They allow different applications, teams, and users to work in the same cluster.
  • They allow cluster administrators to use namespace resource quotas for the applications.
  • They use RBAC policies to control access to specific resources within the namespaces. RoleBinding helps cluster administrators' control permissions granted to users within the namespace.
  • They allow network segmentation with the network policy defined in the namespace. By default, all pods can communicate with each other across different namespaces.

By default, Kubernetes has three different namespaces. Run the following command to view them:

$ kubectl get namespace

NAME          STATUS    AGE

default       Active    1d

kube-system   Active    1d

kube-public   Active    1d

The three namespaces are described as follows:

  • default: A namespace for resources that are not part of any other namespace.
  • kube-system: A namespace for objects created by Kubernetes such as kube-apiserver, kube-scheduler, controller-manager, and coredns.
  • kube-public: Resources within this namespace are accessible to all. By default, nothing will be created in this namespace.

Let's take a look at how to create a namespace.

Creating a namespace

A new namespace in Kubernetes can be created by using the following command:

$ kubectl create namespace test

Once a new namespace is created, objects can be assigned to a namespace by using the namespace property, as follows:

$ kubectl apply --namespace=test -f pod.yaml

Objects within the namespace can similarly be accessed by using the namespace property, as follows:

$ kubectl get pods --namespace=test

In Kubernetes, not all objects are namespaced. Lower-level objects such as Nodes and persistentVolumes span across namespaces.

Wrapping up least privilege for Kubernetes subjects

By now, you should be familiar with the concepts of ClusterRole/Role, ClusterRoleBinding/RoleBinding, service accounts, and namespaces. In order to implement least privilege for Kubernetes subjects, you may ask yourself the following questions before you create a Role or RoleBinding object in Kubernetes:

  • Does the subject need privileges for a namespace or across namespaces?

    This is important because once the subject has cluster-level privileges it may be able to exercise the privileges across all namespaces.

  • Should the privileges be granted to a user, group, or service account?

    When you grant a role to a group, it means all the users in the group will automatically get the privileges from the newly granted role. Be sure you understand the impact before you grant a role to a group. Next, a user in Kubernetes is for humans, while a service account is for microservices in pods. Be sure you know what the Kubernetes user's responsibility is and assign privileges accordingly. Also, note that some microservices do not need any privilege at all as they don't interact with kube-apiserver or any Kubernetes objects directly.

  • What are the resources that the subjects need to access?

    When creating a role, if you don't specify the resource name or do set * in the resourceNames field, it means access is granted to all the resources of the resource type. If you know which resource name the subject is going to access, do specify the resource name when creating a role.

Kubernetes subjects interact with Kubernetes objects with the granted privileges. Understanding the actual tasks your Kubernetes subjects perform will help you grant privileges properly.

Least privilege for Kubernetes workloads

Usually, there will be a service account (default) associated with a Kubernetes workload. Thus, processes inside a pod can communicate with kube-apiserver using the service account token. DevOps should carefully grant necessary privileges to the service account for the purpose of least privilege. We've already covered this in the previous section.

Besides accessing kube-apiserver to operate Kubernetes objects, processes in a pod can also access resources on the worker nodes and other pods/microservices in the clusters (covered in Chapter 2, Kubernetes Networking). In this section, we will talk about the possible least privilege implementation of access to system resources, network resources, and application resources.

Least privilege for accessing system resources

Recall that a microservice running inside a container or pod is nothing but a process on a worker node isolated in its own namespace. A pod or container may access different types of resources on the worker node based on the configuration. This is controlled by the security context, which can be configured both at the pod level and the container level. Configuring the pod/container security context should be on the developers' task list (with the help of security design and review), while pod security policies—the other way to limit pod/container access to system resources at the cluster level—should be on DevOps's to-do list. Let's look into the concepts of security context, PodSecurityPolicy, and resource limit control.

Security context

A security context offers a way to define privileges and access control settings for pods and containers with regard to accessing system resources. In Kubernetes, the security context at the pod level is different from that at the container level, though there are some overlapping attributes that can be configured at both levels. In general, the security context provides the following features that allow you to apply the principle of least privilege for containers and pods:

  • Discretionary Access Control (DAC): This is to configure which user ID (UID) or group ID (GID) to bind to the process in the container, whether the container's root filesystem is read-only, and so on. It is highly recommended not to run your microservice as a root user (UID = 0) in containers. The security implication is that if there is an exploit and a container escapes to the host, the attacker gains the root user privileges on the host immediately.
  • Security Enhanced Linux (SELinux): This is to configure the SELinux security context, which defines the level label, role label, type label, and user label for pods or containers. With the SELinux labels assigned, pods and containers may be restricted in terms of being able to access resources, especially volumes on the node.
  • Privileged mode: This is to configure whether a container is running in privileged mode. The power of the process running inside the privileged container is basically the same as a root user on a node.
  • Linux capabilities: This is to configure Linux capabilities for containers. Different Linux capabilities allow the process inside the container to perform different activities or access different resources on the node. For example, CAP_AUDIT_WRITE allows the process to write to the kernel auditing log, while CAP_SYS_ADMIN allows the process to perform a range of administrative operations.
  • AppArmor: This is to configure the AppArmor profile for pods or containers. An AppArmor profile usually defines which Linux capabilities the process owns, which network resources and files can be accessed by the container, and so on.
  • Secure Computing Mode (seccomp): This is to configure the seccomp profile for pods or containers. A seccomp profile usually defines a whitelist of system calls that are allowed to execute and/or a blacklist of system calls that will be blocked to execute inside the pod or container.
  • AllowPrivilegeEscalation: This is to configure whether a process can gain more privileges than its parent process. Note that AllowPrivilegeEscalation is always true when the container is either running as privileged or has a CAP_SYS_ADMIN capability.

We will talk more about security context in Chapter 8, Securing Pods.

PodSecurityPolicy

The PodSecurityPolicy is a Kubernetes cluster-level resource that controls the attributes of pod specification relevant to security. It defines a set of rules. When pods are to be created in the Kubernetes cluster, the pods need to comply with the rules defined in the PodSecurityPolicy or they will fail to start. The PodSecurityPolicy controls or applies the following attributes:

  • Allows a privileged container to be run
  • Allows host-level namespaces to be used
  • Allows host ports to be used
  • Allows different types of volumes to be used
  • Allows the host's filesystem to be accessed
  • Requires a read-only root filesystem to be run for containers
  • Restricts user IDs and group IDs for containers
  • Restricts containers' privilege escalation
  • Restricts containers' Linux capabilities
  • Requires an SELinux security context to be used
  • Applies seccomp and AppArmor profiles to pods
  • Restricts sysctls that a pod can run
  • Allows a proc mount type to be used
  • Restricts an FSGroup to volumes

We will cover more about PodSecurityPolicy in Chapter 8, Securing Kubernetes Pods. A PodSecurityPolicy control is basically implemented as an admission controller. You can also create your own admission controller to apply your own authorization policy for your workload. Open Policy Agent (OPA) is another good candidate to implement your own least privilege policy for a workload. We will look at OPA more in Chapter 7, Authentication, Authorization, and Admission Control.

Now, let's look at the resource limit control mechanism in Kubernetes as you may not want your microservices to saturate all the resources, such as the Central Processing Unit (CPU) and memory, in the system.

Resource limit control

By default, a single container can use as much memory and CPU resources as a node has. A container with a crypto-mining binary running may easily consume the CPU resources on the node shared by other pods. It's always a good practice to set resource requests and limits for workload. The resource request impacts which node the pods will be assigned to by the scheduler, while the resource limit sets the condition under which the container will be terminated. It's always safe to assign more resource requests and limits to your workload to avoid eviction or termination. However, do keep in mind that if you set the resource request or limit too high, you've caused a resource waste on your cluster, and the resources allocated to your workload may not be fully utilized. We will cover this topic more in Chapter 10, Real-Time Monitoring and Resource Management of a Kubernetes Cluster.

Wrapping up least privilege for accessing system resources

When pods or containers run in privileged mode, unlike the non-privileged pods or containers, they have the same privileges as admin users on the node. If your workload runs in privileged mode, why is this the case? When a pod is able to assess host-level namespaces, the pod can access resources such as the network stack, process, and Interprocess Communication (IPC) at the host level. But do you really need to grant host-level namespace access or set privileged mode to your pods or containers? Also, if you know which Linux capabilities are required for your processes in the container, you'd better drop those unnecessary ones. And how much memory and CPU is sufficient for your workload to be fully functional? Please do think through these questions for the purpose of implementing the principle of least privilege for your Kubernetes workload. Properly set resource requests and limits, use security context for your workload, and enforce a PodSecurityPolicy for your cluster. All of this will help ensure the least privilege for your workload to access system resources.

Least privilege for accessing network resources

By default, any two pods inside the same Kubernetes cluster can communicate with other, and a pod may be able to communicate with the internet if there is no proxy rule or firewall rule configured outside the Kubernetes cluster. The openness of Kubernetes blurs the security boundary of microservices, and we mustn't overlook network resources such as API endpoints provided by other microservices that a container or pod can access.

Suppose one of your workloads (pod X) in namespace X only needs to access another microservice A in namespace NS1; meanwhile, there is microservice B in namespace NS2. Both microservice A and microservice B expose their Representational State Transfer (RESTful) endpoints. By default, your workload can access both microservice A and B assuming there is neither authentication nor authorization at the microservice level, and also no network policies enforced in namespaces NS1 and NS2. Take a look at the following diagram, which illustrates this:

Figure 4.1 – Network access without network policy

Figure 4.1 – Network access without network policy

In the preceding diagram, Pod X is able to access both microservices, though they reside in different namespaces. Note also that Pod X only requires access to Microservice A in namespace NS1. So, is there anything we can do to restrict Pod X's access to Microservice A only for the purpose of least privilege? Yes: a Kubernetes network policy can help. We will cover network policies in more detail Chapter 5, Configuring Kubernetes Security Boundaries. In general, a Kubernetes network policy defines rules of how a group of pods are allowed to communicate with each other and other network endpoints. You can define both ingress rules and egress rules for your workload.

Note

Ingress rules: Rules to define which sources are allowed to communicate with the pods under the protection of the network policy.

Egress rules: Rules to define which destinations are allowed to communicate with the pods under the protection of the network policy.

In the following example, to implement the principle of least privilege in Pod X, you will need to define a network policy in Namespace X with an egress rule specifying that only Microservice A is allowed:

Figure 4.2 – Network policy blocks access to microservice B

Figure 4.2 – Network policy blocks access to microservice B

In the preceding diagram, the network policy in Namespace X blocks any request from Pod X to Microservice B, and Pod X can still access Microservice A, as expected. Defining an egress rule in your network policy will help ensure least privilege for your workload to access network resources. Last but not least, we still need to bring your attention to the application resource level from a least-privilege standpoint.

Least privilege for accessing application resources

Though this topic falls into the category of application security, it is worth bringing up here. If there are applications that your workload accesses that support multiple users with different levels of privileges, it's better to examine whether the privileges granted to the user on your workload's behalf are necessary or not. For example, a user who is responsible for auditing does not need any write privileges. Application developers should keep this in mind when designing the application. This helps to ensure the least privilege for your workload to access application resources.

Summary

In this chapter, we went through the concept of least privilege. Then, we discussed the security control mechanism in Kubernetes that helps in implementing the principle of least privilege in two areas: Kubernetes subjects and Kubernetes workloads. It is worth emphasizing the importance of implementing the principle of the principle of least privilege holistically. If least privilege is missed in any area, this will potentially leave an attack surface wide open.

Kubernetes offers built-in security controls to implement the principle of least privilege. Note that it is a process from development to deployment: application developers should work with security architects to design the minimum privileges for the service accounts associated with the application, as well as the minimum capabilities and proper resource allocation. During deployment, DevOps should consider using a PodSecurityPolicy and a network policy to enforce least privileges across the entire cluster.

In the next chapter, we will look at the security of Kubernetes from a different angle: understanding the security boundaries of different types of resources and how to fortify them.

Questions

  1. What is a Role object in Kubernetes?
  2. What is a RoleBinding object in Kubernetes?
  3. What is the difference between RoleBinding and ClusterRoleBinding objects?
  4. By default, a pod can't access host-level namespaces. Name a few settings that allow pods to access host-level namespaces.
  5. If you want to restrict pod access to external network resources (for example, the internal network or the internet), what you can do?

Further reading

You may have noticed that some of the security control mechanisms we talked about in this chapter have been around for a long time: SELinux Multi-Category Security/Multi-Level Security (MCS/MLS), AppArmor, seccomp, Linux capabilities, and so on. There are already many books or articles introducing these technologies. I would encourage you to take a look at the following materials for a better understanding of how to use them to achieve the least privilege goal in Kubernetes:

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
52.54.111.228