Consul started in 2014 when Kubernetes was also entering the market. It is a first-class citizen for configuring and discovering services, especially when the infrastructure (Compute, Storage, and Network) is dynamic, which is a combination of Kubernetes clusters and VMs in multiple data centers.
The following table will give you a clear picture of the traditional and dynamic infrastructures:
| Traditional infrastructure | Dynamic infrastructure | |
| What is it? | In a traditional infrastructure, there is static connectivity in an insecure flat network protected by firewall rules. |
In a dynamic infrastructure, ephemeral workloads with dynamic IP addresses can run on any machine in a zero-trust network. |
| How does it handle network traffic? | Traffic is routed through a hardware or software load balancer across multiple applications (horizontal scalability), which is why it is sometimes known as North-South traffic. | The load balancer sits just before the Ingress gateway, and then traffic is distributed dynamically to different service endpoints. This is why it is sometimes known as East-West traffic. |
| Examples | Multiple federated Kubernetes clusters can be viewed as serving North-South traffic. | A single Kubernetes cluster can be viewed as serving East-West traffic. |
Consul can be viewed as serving both North-South (WAN Gossip Protocol) and East-West (LAN Gossip Protocol) traffic:
- North-South traffic: The traffic travels between outside and inside of a k8s cluster; that is, the traffic goes through the Ingress controller.
- East-West traffic: The traffic travels between the services inside a k8s cluster.
One of the benefits of Consul is that it can run in heterogeneous environments such as Kubernetes and VMs or directly on a bare-metal machine. It provides functionality for service catalogs, configuration, TLS certificates, authorization, and so on.
Consul is a single Go binary that runs as an agent on each node in a cluster, and it manages/monitors all services on that node. Consul agents (clients) hold service registration and health check data.
The Consul cluster is made up of members, collections of Consul agents and Consul servers. A typical production environment comprises an odd number of servers (3 and 5 maximum) to ensure that the majority of the quorum is met in the event of failures.
Consul is a distributed system where agent nodes communicate with server nodes:
- Consul servers are responsible for maintaining the state of the cluster.
- Consul Client (agent) is responsible for performing a health check of a node and the services running on that node.
Consul provides the following features:
- Multi-data center deployment: One of the main features of Consul is its support for multiple data centers using the gossip protocol to register members leaving and joining the cluster and to check the health status of members and services.
- Service discovery: When applications are broken down into microservices, they are no longer available through a memory call to a public function. However, these microservices can reside on any machine in a data center, and the call is done through the network. The IP address can change any time the pod is rescheduled. Service registration is automatic in the Kubernetes environment. In a VM or bare-metal environment, applications can register to a centralized Consul service discovery, which is maintained as a key-value store.
- Configuration: Monolithic applications have a centralized configuration, but when microservices are built there is a need for a centralized configuration that provides a consistent view of all the services. Consul, through its key-value store, provides a central place in which configurations can be stored as a name/value pair, which can be pushed down dynamically to the microservices.
- Key-value store: This is a hierarchical key-value store for configuration data.
- Network segmentation: For microservices, Consul provides network segmentation to allow services to communicate securely in a flat zero-trust network.
A Consul service mesh provides a very good integration of traditional and dynamic infrastructures by way of service discovery, secure communication, network segmentation, and a multi-data center approach.
Next, we will go through the Consul architecture to understand the core components that will help us use Consul from an implementation standpoint.