So, we have talked about handling and fixing failures. They say the more you sweat in peace, the less you bleed in war. So, it is a good idea to keep yourself prepared for any expected or unexpected failures.

We need to make sure we have a recovery process in place. A simple example is that when we are moving to a newer version of a service, it will make sense to keep the older version live for some time so that if something goes wrong, we can quickly move back to an older version.

Netflix, which relies heavily on Microservices for their system to work properly, came up with the approach of Chaos Monkey. The idea is simple, yet powerful: randomly bring down instances of services in place and see how your system behaves. Does it completely shut down or is it able to handle the situation? Is the service be able to heal itself? Is the system be able to take the required actions to bring it into a healthy state?

This way, we will can determine our weaknesses and take the required actions in time.