Cluster Management
In Service Fabric as well as in a containerized microservices architecture, there is a notion of a cluster, which consists of a number of nodes. In the context of this chapter, a node relates to a physical or virtual machine instance running Service Fabric.
In Service Fabric, a cluster is defined through the cluster manifest described in an Azure Resource Model (ARM) JSON document. The cluster manifest specifies infrastructure metadata such as number of nodes, node names and types, network topology, cluster security, and so on. One big difference compared to a Docker Swarm or Mesos cluster is that Service Fabric does not require us to set up a separate service for managing cluster availability with tools like Zookeeper or Consul. These are normally needed for leader election among several machines, which then manage the cluster. Service Fabric is a single stack designed specifically to work together vs. a coordinated set of open-source projects that are used together. As a result, we only need to specify the nodes needed to run our microservices in the ARM template, and the Service Fabric cluster bootstraps itself from there. To ensure the highest-availability clustering, Service Fabric uses the quorum model and requires us to use at least five nodes for a minimal cluster when setting up a cluster in Azure.
What is Quorum?
Quorum is a term used in high-availability clustering and means that there needs to be a minimum number of nodes online to be able to communicate with each other. Of those, in most cases a majority need to agree on a piece of information for it to be considered “the truth.” In the case of Service Fabric Quorum has to be established between the replicas of a stateful service.
At the time of writing, Azure Service Fabric was still in preview mode and was using Azure virtual machines for its nodes.
Virtual Machine Scale Sets
For general availability, Service Fabric will use virtual machine scale sets to take advantage of its integrated dynamic scale features. This blog post at https://azure.microsoft.com/en-us/blog/azure-vm-scale-sets-public-preview/ has great information for virtual machine scale sets.
To make sure that not all virtual machines can go down at the same time—for example, for planned maintenance—Service Fabric Service uses Azure availability sets. Every virtual machine in an availability set is put into an upgrade domain and is spread across fault domains by the underlying Azure platform.
Fault domain: A fault domain is a physical unit of failure consisting of hardware components such as host machines, switches, and so on, that share a single point of failure. A server rack is a good example of a fault domain. By default, Azure spreads an availability set over two fault domains. Service Fabric, however, is using a minimum of three fault domains to offer the highest possible degree of availability.
Upgrade domain: An update domain is a logical unit of a set of nodes that can be updated together. In Azure, an availability set contains twenty update domains.
Figure 8.2 shows a simplified model of how Service Fabric sets up a five-node cluster across upgrade and fault domains.
For clusters with more than five nodes, Azure continues to put the nodes into unique fault domains, upgrade domain combinations using more than just three fault domains. Azure will follow this pattern for any number of nodes. Figure 8.3 shows an example distribution of VMs across fault and upgrade domains in a seven-node cluster.
Now we know what a Service Fabric cluster is and how it works at a high level; we just need to know how to set one up. Service Fabric clusters in Azure can be set up like any other Azure Resource Manager-based environment. We can use the Azure portal, CLI, or PowerShell to provision a Service Fabric cluster. Figure 8.4 shows how to create a Service Fabric cluster through the Azure portal.
