When you configure an Elasticsearch domain, an instance is provisioned for you behind the scenes to run the open source software that comprises Elasticsearch and Kibana. The CloudWatch Logs agent that runs on the instances that you want to monitor watches log files according to the configuration that you specify and sends the logs to CloudWatch in batches. CloudWatch, in turn, passes those log entries on to the Elasticsearch domain. Once they have been ingested by Elasticsearch, you can query and search them with Kibana.

One important thing to keep in mind with this logging solution is that application logs on your instances often log sensitive data, so be sure to safeguard all the aspects of this solution in the same way that you would safeguard customer databases. Many infamous data breaches are the result of web application developers logging things such as usernames and passwords to log files, and then those log files are passed to an unprotected system. Don't be one of those administrators who simply assume that log files are innocuous and don't deserve rigorous data protection controls!

In the preceding recipe, you had the option of configuring an open domain that allows access to the public. While this made completing the recipe easy, when it comes to production applications, lock your domain down to a VPC and apply the lessons you have learned about security to limit access to a small subset of your users.