When operating distributed clusters of servers, we need to figure out a good approach to meet the need to communicate with the servers and the services they offer. Imagine a cluster of hundreds, thousands, or even tens of thousands of servers running on an environment such the spot service, across several regions and availability zones. Now, imagine having to determine the following at any given time:

• The network address or DNS name of each server
• The current load on the server
• Which server is available to take a request and process it

This would be a daunting task with a stable environment, let alone a spot environment where the servers can pop in and out of existence on the EC2 compute layer.

So, instead of trying to figure out which server I should send my request to, we can simply let the servers themselves decide when there are enough resources to process a request. In this kind of setup, we need to post the request on a message broker layer where the message can be queued up for processing and stored until the worker comes along, reads the message, and processes it.

In essence, this type of message queue service decouples the relationship of the producer of the request and the worker that consumes and processes the request. Before implementing a message queue service, we would have needed to keep track of all the servers and account for their load before we would be able to select the most appropriate server to process our request. After introducing the message queue service, the only configuration on both layers is the endpoint address of the message queue service and the protocol it operates on.