In this recipe, we will manually remove pods in our deployment to show how Kubernetes replaces them. Later, we will learn how to automate this using a user-defined health check. Now, let's test Kubernetes' self-healing for destroyed pods:

1. Create a deployment or StatefulSet with two or more replicas. As an example, we will use the MinIO application we used in the previous chapter, in the Configuring and managing S3 object storage using MinIO recipe. This example has four replicas:

$ cd src/chapter7/autoheal/minio
$ kubectl apply -f minio.yaml

2. List the MinIO pods that were deployed as part of the StatefulSet. You will see four pods:

$ kubectl get pods |grep minio
minio-0 1/1 Running 0 4m38ms
minio-1 1/1 Running 0 4m25s
minio-2 1/1 Running 0 4m12s
minio-3 1/1 Running 0 3m48s

3. Delete a pod to test Kubernetes' auto-healing functionality and immediately list the pods again. You will see that the terminated pod will be quickly rescheduled and deployed:

$ kubectl delete pod minio-0
pod "minio-0" deleted
$ kubectl get pods |grep miniominio-0
minio-0 0/1 ContainerCreating 0 2s
minio-1 1/1 Running           0 8m9s
minio-2 1/1 Running           0 7m56s
minio-3 1/1 Running           0 7m32s

With this, you have tested Kubernetes' self-healing after manually destroying a pod in operation. Now, we will learn how to add a health status check to pods to let Kubernetes automatically kill non-responsive pods so that they're restarted.