Step 1: Key pair and access key configuration

We assume that you have EC2 accounts already created. Well! The first requirement is to create EC2 key pairs and AWS access keys. The EC2 key pair is the private key that you need when you will make a secure connection through SSH to your EC2 server or instances. For making the key, you have to go through AWS console at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#having-ec2-create-your-key-pair. Please refer to the following figure that shows the key-pair creation page for an EC2 account:

Figure 17: AWS key-pair generation window

Name it aws_key_pair.pem once you have downloaded it and save it on your local machine. Then ensure the permission by executing the following command (you should store this file in a secure location for security purpose, say /usr/local/key):

$ sudo chmod 400 /usr/local/key/aws_key_pair.pem

Now what you need are the AWS access keys and the credentials of your account. These are needed if you want to submit your Spark job to computing nodes from your local machine using the spark-ec2 script. To generate and download the keys, login to your AWS IAM services at http://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html#Using_CreateAccessKey.

Upon the completion of download (that is, /usr/local/key), you need to set two environment variables in your local machine. Just execute following commands:

$ echo "export AWS_ACCESS_KEY_ID=<access_key_id>" >> ~/.bashrc 
$ echo " export AWS_SECRET_ACCESS_KEY=<secret_access_key_id>" >> ~/.bashrc
$ source ~/.bashrc
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.133.155.216