Security Context and Policies

A security context defines privilege and access control settings on either the pod or container level. The supported settings are as follows:

Implement discretionary access control

Set permissions to access operating system objects, such as files, based on user or group ID as well as running as an (un)privileged process.

Capabilities

Rather than giving someone root access, you can use capabilities to split the (unrestricted) root access into a set of separate permissions such as CHOWN or NET_RAW.

Apply profiles

Configure seccomp by filtering system calls for processes or configure AppArmor to restrict the capabilities of processes.

Implementing mandatory access control

Through configuring SELinux, by assigning security labels to operating system objects.

Using a security context is straightforward: use the securityContext field on either the pod level or on the level of a particular container. For example, imagine you want to define the following:

• All containers in the pod must run under user 1001, through the runAsUser setting.

• In the webserver container, prevent setuid binaries from changing the effective user ID as well as prevent files from enabling extra capabilities by setting allowPrivilegeEscalation to false.

This translates into the following pod specification we store under secconpod.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: securepod
spec:
  securityContext:
    runAsUser: 1001
  containers:
  - name: webserver
    image: quay.io/mhausenblas/pingsvc:2
    securityContext:
      allowPrivilegeEscalation: false
  - name: shell
    image: centos:7
    command:
      - "bin/bash"
      - "-c"
      - "sleep 10000"

Now you can launch the pod and check the user that the respective container is running under:

$ kubectl apply -f secconpod.yaml
pod "securepod" created

$ kubectl exec -it securepod --container=webserver -- id
uid=1001 gid=0(root) groups=0(root)

$ kubectl exec -it securepod --container=shell -- id
uid=1001 gid=0(root) groups=0(root)

So that works great; we can make sure that, for example, a container in a pod doesn’t run as root. But how can you make sure, as a cluster or namespace admin, that your developers use appropriate security contexts?

Enter pod security policies. A pod security policy is a cluster-wide resource that allows you to enforce the usage of security contexts. The API server automatically enforces those policies for you, using admission controllers, as described in “Admission Control”. If a pod’s specification doesn’t meet the requirements of the pod security policy, it won’t be run (although note that for pod security policies to take effect, the PodSecurityPolicy admission plugin must be enabled, and permission must be granted to use that policy for the appropriate users).

PodSecurityPolicy allows us to define securityContext context settings, along with other security-related settings such as the seccomp and AppArmor profiles.

To enable the default Docker seccomp profile, include the following annotations set in your pod security policies:

annotations:
  seccomp.security.alpha.kubernetes.io/allowedProfileNames:
    'docker/default'
  seccomp.security.alpha.kubernetes.io/defaultProfileName:
    'docker/default'

Let’s see how we can enforce the “must run as nonroot user” scenario, along with Docker’s default seccomp and AppArmor profiles, using the following policy:

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: nonroot
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames:
      'docker/default'
    apparmor.security.beta.kubernetes.io/allowedProfileNames:
      'runtime/default'
    seccomp.security.alpha.kubernetes.io/defaultProfileName:
      'docker/default'
    apparmor.security.beta.kubernetes.io/defaultProfileName:
      'runtime/default'
spec:
  privileged: false
  allowPrivilegeEscalation: false
  runAsUser:
    rule: MustRunAsNonRoot
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: MustRunAs
    ranges:
      - min: 1000
        max: 1500
  fsGroup:
    rule: MustRunAs
    ranges:
      - min: 1000
        max: 1500

Network Policies

Limiting the traffic that can flow between pods adds a good layer of security:

• Even if an external attacker is able to reach the cluster network, network policy can stop that attacker from sending traffic that reaches application code running inside pods.

• If a container somehow becomes compromised, an attacker will typically try to explore the network to move laterally to other containers or hosts. By restricting the addresses, ports, and pods that can be contacted, the attacker’s ability to reach other parts of the deployment is curtailed.

Applications running in Kubernetes can potentially communicate with outside clients (north-south traffic) as well as with other applications running within the Kubernetes cluster (east-west traffic).

By default, all kinds of ingress (incoming) and egress (outgoing) traffic are allowed, but you can control how pods are allowed to communicate by using a Kubernetes feature called network policies. From version 1.7 onward, this feature is considered stable and hence ready for use in production.

Different Kubernetes distributions support network policies to different degrees. Popular network policy providers include Calico, Weave Net, OpenShift SDN, and Cilium. The latter is a BPF-based (Berkeley Packet Filter) implementation with a promising future.

Example Network Policy

Let’s look at an example network policy preventing all ingress traffic to and egress traffic from all pods in the namespace lockeddown:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: nonetworkio
  namespace: lockeddown
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

A network policy applies to the set of pods that match the podSelector defined in the spec. Typically, a label selector is used to match a subset of pods; an empty podSelector as in the preceding example matches all pods.

If a pod is not matched by any network policies, all traffic is allowed to and from that pod.

The Kubernetes documentation includes example network policies that you might use to limit traffic to and from all pods by default.

Effective Network Policies

For network policies to be at their most effective, we want to ensure that traffic can flow only where it is needed, and nowhere else. To achieve this, you will typically start with a DenyAll default policy that matches all pods with an empty podSelector, just as in the preceding lockeddown example. Then take a structured approach to adding network policies that allows traffic between application pods as necessary.

Suppose we have an application called my-app that stores data in a Postgres database. The following example defines a policy that allows traffic from my-app to my-postgres on the default port for Postgres:

  apiVersion: networking.k8s.io/v1
  kind: NetworkPolicy
  metadata:
    name: allow-myapp-mypostgres
    namespace: lockeddown
  spec:
    podSelector:
      matchLabels:
        app: my-postgres
    ingress:
      - from:
        - podSelector:
            matchLabels:
              app: my-app
        ports:
          - protocol: TCP
            port: 5432

We then likely want to allow traffic from the internet to access my-app, which we can achieve with another network policy like this:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: allow-external
  namespace: lockeddown
spec:
  podSelector:
    matchLabels:
      app: my-app
  ingress:
  - from: []

The combination of these three network policies allows the application traffic to flow as desired from external users to the application pods, and from the application pods to the database, but traffic is not permitted anywhere else in this lockeddown namespace.

Ahmet Alp Balkan has put together a set of useful network policy recipes as well as a good post on the topic of “Securing Kubernetes Cluster Networking”. Another helpful backgrounder is “Kubernetes Security Context, Security Policy, and Network Policy” by Mateo Burillo.

Going forward, service meshes will also play a role in this area. See “Service Meshes” for more on this topic.