Fortunately, the Spring Cloud ecosystem comes to the rescue and tries to address the issue when a server-side load balancer becomes a hot-point in a system. Instead of providing a workaround for the external load balancer, the Spring team decided to follow Netflix's best practices of building distributed systems. One of the ways of achieving scalability and location transparency is through client-side load balancing.

The idea of client-side load balancing is straightforward and means that a service communicates through a sophisticated client that is aware of the available instances of a target service so that it may balance the load among them easily:

In the preceding diagram, the numbered points mean the following:

1. This point depicts a few services that communicate with Service A.
2. These are the client-side load balancers. As we can see now, the load balancer is a part of each service. Consequently, all coordination should be done before the actual HTTP call occurs.
3. The actual group of Service A instances is depicted here.

In this example, all callers execute a call to the different replicas of Service A. Even though the technique provides independence from a dedicated external load balancer (thereby providing better scalability), it has its limitations. First of all, the mentioned load-balancing technique is client-side balancing. Consequently, each caller is responsible for balancing all requests locally by choosing an instance of the target service. This helps to avoid a single point of failure, thereby providing better scalability. On the other hand, information about available services should somehow be accessible to the rest of the services in the system.

To become familiar with the modern techniques used for service discovery, we are considering one of the popular libraries in the Java and Spring ecosystem—Ribbon. The Ribbon library is an implementation of the client-side load balancer pattern created by Netflix. Ribbon offers two common techniques to provide access to the list of available services. The simplest way of providing such information is through a static preconfigured list of services' addresses. This technique is depicted in the following diagram:

In the aforementioned diagram, the numbered points mean the following:

1. This point depicts a few services that communicate with Service A.
2. These are client-side load balancers. Each service has access to the particular group of service instances.
3. A representation of the internal list of pre-configured Service A instances. Here, each caller service independently measures the load on each target Service A instance and applies a balancing process concerning that. The bold service name in the list also refers to the current execution target instance of Service A.
4. The actual group of Service A instances is depicted here.

In the preceding diagram, the shapes with different borders outline areas of knowledge of different callers. Unfortunately, the client-side load balancing technique has its gaps. First of all, there is no coordination between client-side balancers, so it is possible that all callers may decide to call the same instance and overload it, as depicted in this diagram:

Each section of the numbered diagram is explained as follows:

1. This refers to the caller services with their list of pre-configured Service A instances. Local load measurements are different across services. Consequently, in this example, the case is shown in which all services call the same target instance of Service A. This may cause an unexpected spike in the load.
2. This is one of the Service A instances. Here, the actual load on the instance is different than what each caller service assumed based on the local measurements.

Moreover, such a simple way of managing the load with a static list of service instances is far from reactive system requirements, mainly from the elastic load management standpoint.

As a solution, Ribbon is capable of integrating with a service registry such as Eureka, so the registry service is continuously updating the list of available service replicas. Consider the following diagram:

In the preceding diagram, the numbered points mean the following:

1. This refers to caller services with the list of available Service A instances. Here, to keep the list of alive instances in Service A up to date, the client-side balancer refreshes that list periodically and grabs the latest information from the registry (2).
2. This is the representation of the registry service. As we can see here, the registry service keeps its own list of discovered services and their statuses.
3. The dotted lines represent the heartbeat of the services or health status check requests.

As may be noticed from the preceding diagram, the problem of coordination for client-side load balancers remains. In this case, the registry is responsible for keeping a list of healthy service instances and continuously updating them at runtime. Here, both the client-side balancer and the registry service may hold the information about a load of target service instances, and the client-side balancer may periodically synchronize internal load statistics with the load collected by a registry service. Furthermore, all interested parties can access that information and execute their load balancing based on that information. This way of managing the list of available services is much broader than the previous one, making it possible to update the list of available destinations dynamically.

First of all, that technique works well for a small cluster of services. However, the discovery of the dynamic services using a shared registry is far from ideal. As with server-side load balancing, a classic registry service such as Eureka becomes a single point of failure because there is a huge effort required to keep the information of the system's state updated and accurate. For instance, when the state of the cluster rapidly changes, the information about registered services may become outdated. To track a service's health status, service instances usually send heartbeat messages periodically. Alternatively, a registry may execute a health check request periodically. In both cases, very frequent status updates may consume an unreasonably high percentage of cluster resources. Therefore, the duration between health checks usually lasts from a couple of seconds to a few minutes (the default duration for Eureka is 30 seconds). Consequently, a registry may propose a service instance that was healthy during the last health check but has already been destroyed. Hence, the more dynamic a cluster is, the harder it is to track services' statuses accurately using a centralized registry.

In addition, all balancing is still taking place on the client side. This leads to the same problem of uncoordinated load balancing, which means that there is a chance that the actual load on the service can be unbalanced. Moreover, accurate and honest request coordination provided by the client-side load balancers based on service metrics in a distributed system is another challenge, which is probably even harder than the previous one. Hence, we have to find a better solution to build a reactive system.