Creating a data lake using a CloudFormation template

The following are the steps to create a data lake using a CloudFormation template:

  1. Follow the instructions on the GitHub page for this example. Copy the link to the S3 location for the template file. Change the region (on the Management Console to point to your region, if required). Click on the Next button:
  1. Specify a Stack name as DataLakeTestStack, which is shown as follows:
  1. Specify the Administrator Configuration details, including name, email address, and access IP for the ElasticSearch cluster (for test purposes this can be 0.0.0.0 for now). You can choose the Send Anonymous Usage Data parameter as Yes. Send anonymous usage data to AWS. This data is used by AWS to better understand how customers use this solution and related services and products:
  1. Specify tags for the various resources in your stack. For now, we leave them blank:
  1. Leave the IAM Role as CloudFormationRole as shown:
  1. Review all the details, and check the acknowledgment and click on the Create button:
  1. At this stage, you should see the CREATE_IN_PROGRESS for your stack (and the nested stacks within) on the CloudFormation console:
  1. Select a stack (DataLakeTestStack) from the list to see more detailed information. Click on the Overview tab:
  1. Click on the Events tab to see the statuses as the creation events execute:
  1. The Resources tab displays the AWS resources and their current status:
  1. If you want to view the CloudFormation template and then click on the Template tab:
  1.  After 20-25 minutes, you should see the CREATE_COMPLETE message against your stack (and the nested stacks):
  1. Go to the DynamoDB console to verify that the tables were created successfully:
  1. You can view the details of a specific DynamoDB table by selecting it from the list:
  1. Similarly, you can go to the Amazon Elasticsearch Service dashboard to verify the creation of the Elasticsearch domain:
  1. Click on the domain name to verify the details of the Elasticsearch cluster:
  1. Click on the Cluster health tab to ensure that the Status is Green:
  1. At this stage, you have a fully functional data lake provisioned using a CloudFormation template. You should have received an email with the user ID and temporary password in the Administrator's mailbox. Use the credentials to sign in to your data lake and explore it. As it is expensive to keep this stack running, select the Delete Stack option from the Actions menu:
  1. Here, we get a warning because we tried to delete a nested stack. As we don't want to be in an unstable state with respect to the root stack, we cancel out of this screen:
  1. Select the root stack (DataLakeTestStack) and choose Delete Stack option from the Actions menu. Click on the Yes, Delete button:
  1. You should see the Status for the stacks change to DELETE_IN_PROGRESS as shown:
  1. After the stack is deleted, select the Deleted filter to see the details of stacks you deleted:
  1. You will see the root stack and the nested stack listed with the Status as DELETE_COMPLETE:
  1. You can quickly verify that the DynamoDB tables, Elasticsearch Domains, and other resources have been deleted by going to their respective consoles. The following screen shows that the DynamoDB tables no longer exist:
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.15.6.226