5. Service Orchestration and Connectivity
It’s likely our microservices-based application is composed of more than one service, and multiple instances of each service. So how do we deploy and manage multiple services, multiple instances of each service, and the connectivity between the services? With a microservices architecture, it’s even more important that we automate everything. Orchestration tools are used to deploy and manage multiple sets of containers in production, and service discovery tools are used to help route traffic to the appropriate services.
The services used to compose our application could be deployed into a growing number of cloud-hosted compute offerings available today. Using some combination of Infrastructure as a Service (IaaS) or Platform as a Service (PaaS), the microservices can be deployed and run in their own dedicated machines or a set of shared machines. In this chapter we will discuss using a cluster of machines to host our Dockerized microservices.
Before jumping into the details of orchestration, scheduling, cluster management, and service discovery, let’s start with a conceptual overview of a typical cluster used to host a microservice-based application. Figure 5.1 shows a conceptual diagram of a typical environment. In addition to the machines hosting our services, we have management and orchestration systems used to deploy and manage the services that compose our application.
Conceptually the architecture is fairly simple, but the complexities are often in the details.
• Host Nodes are the machines our services are deployed to and run on. This is our pool of resources doing all the heavy lifting and delivering the application functionality. Clustering and scheduling technologies will often refer to these as agents, minions, or nodes.
• Management Services are responsible for selecting and deploying the services to the host nodes, and often provide the API or UI. Many technologies refer to these as “masters.” Technologies like Mesos with Marathon, Docker Swarm, and Kubernetes are commonly used.
• Cluster State Store is used by the management services to elect a leader and store cluster state, like the nodes and information about each node. Technologies like Zookeeper, Consul, and etcd are often used.
• A Service Discovery store is used by our services to announce that they are ready to receive requests and the endpoint they can be reached on. Consumers use this information to route requests to the service endpoints. Zookeeper, Consul, and etcd are popular options for storing service discovery information.
• The Application Gateway is used for routing and load balancing inbound traffic into the cluster. In addition to routing and load balancing, this service can also perform tasks like SSL offload, request aggregation, and authentication. NGINX and HAProxy are popular technologies.
• Artifact Store/Image Registry is not shown in the diagram, the artifact store is used to manage deployment of artifacts like the various files used when scheduling services in the cluster. Docker images are artifacts that are commonly stored in a public hosted image registry such as Docker Hub. Other hosted options are available and there are some very good reasons to host these services yourself, including security, performance, and availability.
Some of these services are often deployed to the same machines. For example, the management services can actually be running on the same machines hosting the cluster state store. Sometimes the cluster state store and service discovery registry share a common deployment of a KV store. The application gateway can be running on the same cluster host nodes.
In this chapter we will learn more about these concepts and the technologies used in creating an environment to deploy and run our microservices-based applications.