When you move from a static environment to a dynamically scaled cloud-based environment, you need to pay close attention to the way you store, capture, and analyze log files generated by the OS and your application. As EC2 instances are instantiated and deleted by the autoscaling group, dynamically storing the log files locally is not recommended. Hence, there is a need for a centralized logging service to which all the applications and OSs log their data to; this makes it very convenient to search, view, analyze, and take action regarding the logs in real time from a centralized console.

You have the option to either rollout your own centralized logging infrastructure using the open source ELK stack (Elasticsearch, Logstash, and Kibana), or subscribe to one of the many third-party logging service providers. Since this a book is about AWS, we will work with CloudWatch as the logging and monitoring service.