Load balancing offers traffic management for transactions at Layer 4 of the OSI model, which is the network protocol layer (TCP/UDP). Load balancing at L4 delivers traffic with limited network information. It does this with an algorithm (that is, round-robin), which calculates the best server based on the low number of connections and fast server response times.

The debate of L4/L5 versus L5/L7 is irrelevant for us if the L4 layer cannot provide load balancing for gRPC or HTTP/2 protocol, which uses a long-lived session with multiple requests.

HTTP/1.1 protocol load balancing works well at the connection level (L4) since one connection can have only one active request.

L7 load balancing works at the highest level of the OSI model. L7 bases its routing decisions on various characteristics of the HTTP/HTTPS header, the content of the message, the URL type, and the information in cookies.

In gRPC/HTTP/2, a connection can have multiple active requests (request multiplexing).  L4 connection-level load balancing will route traffic from this one long-lived connection to just one microservice, even if we have "x" number of replicas running. This can be seen in the following diagram:

The preceding diagram shows an L4 load balancer. For gRPC, all the requests end up at one backend service, even though other replicas are available. Linkerd addresses this problem at the proxy level by resorting to request-level routing for HTTP/2 and gRPC by using L7 load balancing:

In the preceding diagram, you can see that a sidecar proxy opens connections to all of the replicas of a backend service through the L7 load balancing request for gRPC/HTTP/2.

As an add-on to load balancing, sidecar will regularly check the health of each service instance that's deployed within a platform. The sidecar proxy classifies a service instance as unhealthy or healthy based on its health checks. If a service health check has multiple failures and it surpasses the defined threshold, it will be removed from the load balancer. In parallel, when a health check runs again on that service instance and it passes the specified threshold, it will be added back into the load balancer. For example, if a service instance is a shopping application and the page unexpectedly responds with an HTTP 5xx error, the load balancer will immediately remove this service from the load balancer until the error is corrected by the operator or other sources, such as a DB.

The following are the load balancing features that can be applied through destination rules:

• Round-robin
• Random
• Weighted
• Least requests

The following is an example of load balancing three subsets. As we can see, a single destination rule is used to define multiple policies. There's a simple random load balancer for the v1 and v3 subsets and a round-robin local balancer for v2:

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: my-destination-rule
spec:
  host: my-svc
  trafficPolicy:
    loadBalancer:
      simple: RANDOM
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
    trafficPolicy:
      loadBalancer:
        simple: ROUND_ROBIN
  - name: v3
    labels:
      version: v3

The reviews virtual service has two rules:

• All incoming requests with the Foo header that have a bar value go to the reviews service's v2 subset.
• All other requests go to the v1 subset.

This can be seen in the following code:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
  - reviews
  http:
  - match:
    - headers:
        Foo:
          exact: bar
    route:
    - destination:
        host: reviews
        subset: v2
  - route:
    - destination:
        host: reviews
        subset: v1

One very important point to note about load balancing is that it occurs at the mesh level, without requiring the use of an external proxy load balancer.