Azure Resource Manager
The Azure Resource Manager (ARM) can be used in an infrastructure as code approach to resource provisioning and management in Azure. ARM enables us to define the desired state of our infrastructure using a JSON-based templating language. All the intermediate steps—provisioning, redundancy, dependency management, and fault tolerance—are handled seamlessly by Azure. ARM templates are completely self-contained and can easily be deployed to and from Azure. The Azure Resource Manager must carry out the tasks of creating or modifying resources, as defined in an ARM template. A resource is just a virtual machine, virtual network, storage account, load balancer, or similar. Azure Resource Manager also enables us to organize resources into groups, and organize resources through tagging.
Benefits:
• Deploy, manage, and monitor all resources in a group.
• Repeatable deployments.
• Declarative templates for defining deployments.
• Define resource dependencies and ARM will optimize deployments.
• Apply role-based access control to resources in the group.
• Logically organize resource by tags.
ARM is flexible and enables us to use either an imperative or declarative approach. That means we can define our infrastructure in a template and then send that to ARM to handle the provisioning, or we can create a script to call APIs for provisioning as we might have done in the past. Using the declarative approach, templates can be created and managed in source control, where changes to the infrastructure can be versioned and tracked. We can make changes to the template and pass it to ARM to apply those changes to an existing resource group, or create a completely new one. We can rest easy knowing that we can quickly and reliably recreate our infrastructure.
ARM Templating Language
The ARM templating language is a JSON-based document for defining an Azure infrastructure. A template contains configuration details and metadata about each resource, including the type of resource and where it should be deployed. In addition to schema and content version, a template contains four top level elements.
Top level elements:
• Parameters provide input for when a deployment is executed and can be used to customize resource deployment. An example of this is the number of machines we want to deploy in the cluster.
• Variables are used to simplify template language expressions. One use for variables might be a string we commonly use through the template, or formatting of a parameter to be used throughput the template.
• Resources are the types of services to be deployed or updated in the resource group. Examples of resource include virtual machines, virtual networks, and storage accounts.
• Outputs are values that are returned after a deployment. These can be used to provide DNS names or IP addresses created as part of the deployment.
The following JSON document shows the basic structure of a template. From this simple document we only need to define a resource in the ‘resources’ section.
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/
deploymentTemplate.json#",
"contentVersion": "",
"parameters": { },
"variables": { },
"resources": [ ],
"outputs": { }
}
Let’s have a closer look at each section. Before we do that we need to first cover template functions and expressions.
Functions
Expressions and functions extend the JSON available in the template. This enables us to create values that are not strictly literal. Expressions are enclosed in square brackets “[],” and they are evaluated when the template is deployed. Template functions can be used for things like referencing parameters, variables, and value manipulation such as type conversions, addition, math calculations, and string concatenation.
As in JavaScript, function calls are formatted as functionName(arg1, arg2, arg3)
"variables": {
"location": "[resourceGroup().location]",
"masterDNSPrefix":"[concat(parameters('dnsNamePrefix'),'mgmt')]",
"agentCount": "[parameters('agentCount')]"
}
Parameters
The parameters section is used to define values that can be set when deploying the resources. We can then use the parameter values throughout the template to set values for the deployed resources.
The JSON example following defines some parameters starting with the “agentCount,” followed by “masterCount.” For each parameter we must define a type, and we can optionally define default values and some other constraints, as well as some metadata that can contain additional information for other consuming the template. As we see in the following, we have set a default value to the “agentCount” parameter and we have declared the minimum value for this parameter of 1 and a maximum value of 40.
"parameters": {
"agentCount": {
"type": "int",
"defaultValue": 1,
"metadata": {
"description": "The number of Mesos agents for the cluster."
},
"minValue":1,
"maxValue":40
},
"masterCount": {
"type": "int",
"defaultValue": 1,
"allowedValues": [1, 3, 5],
"metadata": {
"description": "The number of Mesos masters for the cluster."
}
}
Variables
In the variables section we can construct values that can be used to simplify template language expressions. These variables are commonly based on values from the parameters as we saw with “agentCount.”
The values can be transformed using ARM template language expressions like the “concat” function. We can see this with the “masterDNSPrefix” variable in the following example. The value for “masterDNSPrefix” will contain a string value of whatever was passed in the parameter “dnsNamePrefix” with “mgmt” concatenated at the end of it. Now we can simply reference this variable in the template instead of having to perform this concatenation. This improves the readability and maintenance of the template.
"variables": {
"masterDNSPrefix":"[concat(parameters('dnsNamePrefix'),'mgmt')]",
"agentDNSNamePrefix":"[concat(parameters('dnsNamePrefix'),'agents')]"
"agentCount": "[parameters('agentCount')]",
"masterCount": "[parameters('masterCount')]",
"agentVMSize": "[parameters('agentVMSize')]",
"sshRSAPublicKey": "[parameters('sshRSAPublicKey')]",
"adminUsername": "azureuser"
}
Resources
In the resources section we define the resources that are deployed and updated. This section is the bulk of most templates and contains a list of resources. The following template snippet defines an Azure Container Service (ACS) and some parameters. The “apiVersion,” “type,” and “name” elements are required elements for every resource. The apiVersion must match a supported version for the “type” of resource defined, and the name defines a unique name for the resource that can be used to reference it. There are some additional documented elements not shown, like the “dependsOn” element used to define resource dependencies that need to be provisioned first. The “properties” element contains all the resource properties. For example, the “orchestratorProfile” is a property of the container service as we have defined it here.
"resources": [
{
"apiVersion": "2015-11-01-preview",
"type": "Microsoft.ContainerService/containerServices",
"location": "[resourceGroup().location]",
"name":"[concat('containerservice-',resourceGroup().name)]",
"properties": {
"orchestratorProfile": {
"orchestratorType": "Swarm"
}
}
}
]
Outputs
In the optional outputs section we can specify values that are returned from deployment. For example, we could return the Fully Qualified Domain Name (FQDN) for the master and agent nodes created in the deployment. This could then be used by a user or a script to connect to these endpoints using this domain name.
"outputs": {
"masterFQDN": {
"type": "string",
"value":
"[reference(concat('Microsoft.ContainerService/containerServices/',
'containerservice-', resourceGroup().name)).masterProfile.fqdn]"
},
"agentFQDN": {
"type": "string",
"value": "[reference(concat('Microsoft.ContainerService/
containerServices/', 'containerservice-', resourceGroup().name)).
agentPoolProfiles[0].fqdn]"
}
}
Using an ARM template, we can repeatedly deploy our infrastructure through the entire lifecycle and have confidence that the resources are deployed in a consistent manner. We define our infrastructure in a template that is managed in source control, where we can track changes to the infrastructure. When the template is changed we can apply the new template to existing deployments and the Azure Resource Manager will determine what needs to change and make the necessary changes based on the current state of the template and the existing state of the deployment.
Linked Templates
As templates grow, we can compose templates from other templates through linking. This is a great technique used in managing templates, especially as they grow in size and complexity. We could create a template responsible for deploying Consul, for example, and then reference that template in our Docker Swarm template.
Parameters for each environment can be maintained in source control or some other configuration store. Like environment configuration for an application, we would generally store this in the operations source control repository or a special database. Parameters would generally be things like the names, locations, and number of instances of the various resources. We might want to deploy a cluster into a staging or test environment that has a fewer number of nodes or smaller nodes. Instead of making the number of nodes in the cluster part of the template, we can make it a parameter. Then, when we are deploying a template we can pass the parameter value containing the number of nodes in the deployment suitable for the environment. The deployments of our cluster are consistent and the same across environments, with some variation that can be controlled with parameters and tested, like the number of nodes in this case.
Inside the Box
Once the Azure Resource Manager (ARM) has provisioned the virtual machines and other services outside the box, there is often a need for some provisioning inside the box. Often we will need to install some software, like our cluster management software. Once the virtual machine is ready, we can have ARM invoke a script and pass some context in to perform the provisioning inside the box. Some ARM extensions are available and can be defined in the template, with the context passed, such as the information needed to join the cluster. A couple of extensions that are commonly used when provisioning machines are the script extension and the Docker extension. The script extension enables us to define a set of scripts to download to the machine and a command to execute. The Docker extension will ensure the Docker daemon is installed on the machines, and use a JSON document matching the Docker Compose format to start Docker containers on the instance after they have been provisioned.
These extensions are commonly used to bootstrap the provisioned machines with cluster management software and join the cluster. For example, we can use the script extension to install the Apache Mesos agent on each node defined in the template. We could use the Docker extension to install the Docker Swarm daemon and register nodes with the cluster manager.
In addition to using extensions to bootstrap our machine, we can embed a cloud-config into the template as custom data. Cloud-config files are scripts designed to be run by the cloud-init process, which is used for initial configuration on the first boot of a server. We can install our cluster management software through this mechanism as well. The cloud-config works like a configuration file for common tasks and enables you to define scripts for more complex functionality. More information on cloud-init is available in the documentation at https://cloudinit.readthedocs.org/en/latest/. Not all virtual machine images in Azure support this, or have cloud-init enabled.