We'll continue using the specifications from the vfarcic/k8s-specs repository, so the first thing we'll do is to go inside the directory where we cloned it, and pull the latest version.
1 cd k8s-specs
2 3 git pull
I will assume that you already have an AWS account. If that's not the case, please head over to Amazon Web Services (https://aws.amazon.com/) and sign-up.
The first thing we should do is get the AWS credentials.
Please open Amazon EC2 Console (https://console.aws.amazon.com/ec2/v2/home), click on your name from the top-right menu and select My Security Credentials. You will see the screen with different types of credentials.
Expand the Access Keys (Access Key ID and Secret Access Key) section and click the Create New Access Key button. Expand the Show Access Key section to see the keys.
You will not be able to view the keys later on, so this is the only chance you'll have to Download Key File.
We'll put the keys as environment variables that will be used by the AWS Command Line Interface (AWS CLI).
Please replace [...] with your keys before executing the commands that follow.
1 export AWS_ACCESS_KEY_ID=[...]
2 3 export AWS_SECRET_ACCESS_KEY=[...]
We'll need to install AWS CLI (https://aws.amazon.com/cli/) and gather info about your account.
If you haven't already, please open the Installing the AWS Command Line Interface (http://docs.aws.amazon.com/cli/latest/userguide/installing.html) page, and follow the installation method best suited for your OS.
I found the most convenient way to get AWS CLI installed on Windows is to use Chocolatey (https://chocolatey.org/). Download and install Chocolatey, then run choco install awscli from an Administrator Command Prompt. Later on in the chapter, Chocolatey will be used to install jq.
Once you're done, we'll confirm that the installation was successful by outputting the version.
You might need to reopen your GitBash terminal for the changes to the environment variable PATH to take effect.
1 aws --version
The output (from my laptop) is as follows.
aws-cli/1.11.15 Python/2.7.10 Darwin/16.0.0 botocore/1.4.72
Amazon EC2 is hosted in multiple locations worldwide. These locations are composed of regions and availability zones. Each region is a separate geographic area composed of multiple isolated locations known as availability zones. Amazon EC2 provides you the ability to place resources, such as instances, and data in multiple locations.
Next, we'll define the environment variable AWS_DEFAULT_REGION that will tell AWS CLI which region we'd like to use by default.
1 export AWS_DEFAULT_REGION=us-east-2
For now, please note that you can change the value of the variable to any other region, as long as it has at least three availability zones. We'll discuss the reasons for using us-east-2 region and the need for multiple availability zones soon.
Next, we'll create a few Identity and Access Management (IAM) resources. Even though we could create a cluster with the user you used to register to AWS, it is a good practice to create a separate account that contains only the privileges we'll need for the exercises that follow.
First, we'll create an IAM group called kops.
1 aws iam create-group 2 --group-name kops
The output is as follows.
{
"Group": {
"Path": "/",
"CreateDate": "2018-02-21T12:58:47.853Z",
"GroupId": "AGPAIF2Y6HJF7YFYQBQK2",
"Arn": "arn:aws:iam::036548781187:group/kops",
"GroupName": "kops"
}
}
We don't care much for any of the information from the output except that it does not contain an error message thus confirming that the group was created successfully.
Next, we'll assign a few policies to the group thus providing the future users of the group with sufficient permissions to create the objects we'll need.
Since our cluster will consist of EC2 (https://aws.amazon.com/ec2/) instances, the group will need to have the permissions to create and manage them. We'll need a place to store the state of the cluster so we'll need access to S3 (https://aws.amazon.com/s3/). Furthermore, we need to add VPCs (https://aws.amazon.com/vpc/) to the mix so that our cluster is isolated from prying eyes. Finally, we'll need to be able to create additional IAMs.
In AWS, user permissions are granted by creating policies. We'll need AmazonEC2FullAccess, AmazonS3FullAccess, AmazonVPCFullAccess, and IAMFullAccess.
The commands that attach the required policies to the kops group are as follows.
1 aws iam attach-group-policy 2 --policy-arn arn:aws:iam::aws:policy/AmazonEC2FullAccess 3 --group-name kops
4 5 aws iam attach-group-policy 6 --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess 7 --group-name kops
8
9 aws iam attach-group-policy 10 --policy-arn arn:aws:iam::aws:policy/AmazonVPCFullAccess 11 --group-name kops
12 13 aws iam attach-group-policy 14 --policy-arn arn:aws:iam::aws:policy/IAMFullAccess 15 --group-name kops
Now that we have a group with the sufficient permissions, we should create a user as well.
1 aws iam create-user 2 --user-name kops
The output is as follows.
{
"User": {
"UserName": "kops",
"Path": "/",
"CreateDate": "2018-02-21T12:59:28.836Z",
"UserId": "AIDAJ22UOS7JVYQIAVMWA",
"Arn": "arn:aws:iam::036548781187:user/kops"
}
}
Just as when we created the group, the contents of the output are not important, except as a confirmation that the command was executed successfully.
The user we created does not yet belong to the kops group. We'll fix that next.
1 aws iam add-user-to-group 2 --user-name kops 3 --group-name kops
Finally, we'll need access keys for the newly created user. Without them, we would not be able to act on its behalf.
1 aws iam create-access-key 2 --user-name kops >kops-creds
We created access keys and stored the output in the kops-creds file. Let's take a quick look at its content.
1 cat kops-creds
The output is as follows.
{
"AccessKey": {
"UserName": "kops",
"Status": "Active",
"CreateDate": "2018-02-21T13:00:24.733Z",
"SecretAccessKey": "...",
"AccessKeyId": "..."
}
}
Please note that I removed the values of the keys. I do not yet trust you enough with the keys of my AWS account.
We need the SecretAccessKey and AccessKeyId entries. So, the next step is to parse the content of the kops-creds file and store those two values as the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
In the spirit of full automation, we'll use jq (https://stedolan.github.io/jq/) to parse the contents of the kops-creds file. Please download and install the distribution suited for your OS.
Using Chocolatey, install jq from an Administrator Command Prompt via choco install jq.
1 export AWS_ACCESS_KEY_ID=$( 2 cat kops-creds | jq -r 3 '.AccessKey.AccessKeyId')
4 5 export AWS_SECRET_ACCESS_KEY=$( 6 cat kops-creds | jq -r 7 '.AccessKey.SecretAccessKey')
We used cat to output contents of the file and combined it with jq to filter the input so that only the field we need is retrieved.
From now on, all the AWS CLI commands will not be executed by the administrative user you used to register to AWS, but as kops.
Next, we should decide which availability zones we'll use. So, let's take a look at what's available in the us-east-2 region.
1 aws ec2 describe-availability-zones 2 --region $AWS_DEFAULT_REGION
The output is as follows.
{
"AvailabilityZones": [
{
"State": "available",
"RegionName": "us-east-2",
"Messages": [],
"ZoneName": "us-east-2a"
},
{
"State": "available",
"RegionName": "us-east-2",
"Messages": [],
"ZoneName": "us-east-2b"
},
{
"State": "available",
"RegionName": "us-east-2",
"Messages": [],
"ZoneName": "us-east-2c"
}
]
}
As we can see, the region has three availability zones. We'll store them in an environment variable.
Please use tr ' ' ', ' instead of tr ' ' ',' in the command that follows.
1 export ZONES=$(aws ec2 2 describe-availability-zones 3 --region $AWS_DEFAULT_REGION 4 | jq -r 5 '.AvailabilityZones[].ZoneName' 6 | tr ' ' ',' | tr -d ' ')
7 8 ZONES=${ZONES%?}
9 10 echo $ZONES
Just as with the access keys, we used jq to limit the results only to the zone names, and we combined that with tr that replaced new lines with commas. The second command removes the trailing comma.
The output of the last command that echoed the values of the environment variable is as follows.
us-east-2a,us-east-2b,us-east-2c
We'll discuss the reasons behind the usage of three availability zones later on. For now, just remember that they are stored in the environment variable ZONES.
The last preparation step is to create SSH keys required for the setup. Since we might create some other artifacts during the process, we'll create a directory dedicated to the creation of the cluster.
1 mkdir -p cluster
2 3 cd cluster
SSH keys can be created through the aws ec2 command create-key-pair.
1 aws ec2 create-key-pair 2 --key-name devops23 3 | jq -r '.KeyMaterial' 4 >devops23.pem
We created a new key pair, filtered the output so that only the KeyMaterial is returned, and stored it in the devops.pem file.
For security reasons, we should change the permissions of the devops23.pem file so that only the current user can read it.
1 chmod 400 devops23.pem
Finally, we'll need only the public segment of the newly generated SSH key, so we'll use ssh-keygen to extract it.
1 ssh-keygen -y -f devops23.pem 2 >devops23.pub
All those steps might look a bit daunting if this is your first contact with AWS. Nevertheless, they are pretty standard. No matter what you do in AWS, you'd need to perform, more or less, the same actions. Not all of them are mandatory, but they are good practice. Having a dedicated (non-admin) user and a group with only required policies is always a good idea. Access keys are necessary for any aws command. Without SSH keys, no one can enter into a server.
The good news is that we're finished with the prerequisites, and we can turn our attention towards creating a Kubernetes cluster.