In this recipe, we will enable CloudWatch agents to collect cluster metrics from our EKS Kubernetes cluster:

1. Create a namespace called amazon-cloudwatch on your cluster using the following command:

$ cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Namespace
metadata:
  name: amazon-cloudwatch
  labels:
    name: amazon-cloudwatch
EOF

2. Create a Service account for the CloudWatch agent in the namespace amazon-cloudwatch you have created in step 1. The following command will also create cloudwatch-agent-role ClusterRole and ClusterRoleBinding:

$ kubectl apply -f cloudwatch/cwagent-serviceaccount.yaml

3. Get the name of your EKS cluster using the eksctl command or from the Amazon Container Services dashboard. Here, we will use eksctl to get the cluster name. In our example, the cluster name is adorable-rainbow-1571556654:

$ eksctl get cluster
NAME                        REGION
adorable-rainbow-1571556654 us-west-2

4. Create a ConfigMap for the CloudWatch agent. Before you run the following command, replace "cluster_name": "adorable-rainbow-1571556654" with the name of your cluster from step 3:

$ cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: cwagentconfig
  namespace: amazon-cloudwatch
data:
  cwagentconfig.json: |
    {
      "logs": {
        "metrics_collected": {
          "kubernetes": {
            "cluster_name": "{{cluster_name}}",
            "metrics_collection_interval": 60
          }
        },
        "force_flush_interval": 5
      }
    }
EOF

5. Deploy the CloudWatch agent as a DaemonSet. The preceding command will use StatsD, a network daemon that listens for statistics, such as counters and timers, sent over UDP or TCP and sends aggregates to CloudWatch, and also pluggable backend services if they're available:

$ kubectl apply -f cloudwatch/cwagent.yaml

6. Verify that CloudWatch agent pods are created by running the following command. Since agents run as DaemonSets, you should be able to see one pod per worker node listed. In our example, we have two worker nodes and two agent pods running:

$ kubectl get pods -n amazon-cloudwatch
NAME                   READY STATUS  RESTARTS AGE
cloudwatch-agent-dtpxt 1/1   Running 0        67s
cloudwatch-agent-j7frt 1/1   Running 0        67s

When complete, the CloudWatch agent will start sending performance log events to the CloudWatch Container Insights service.