Chapter 6. Hands-on Cluster Management, Failover, and Load Balancing

In Chapter 5, we had a quick introduction to Linux containers, and cluster management. Let’s jump into using these things to solve issues with running microservices at scale. For reference, we’ll be using the microservice projects we developed in Chapters 2, 3, and 4 (Spring Boot, MicroProfile, and Apache Camel, respectively). The following steps can be accomplished with any of those three Java frameworks.

Getting Started

To deploy our microservices, we will assume that a Docker image exists. Each microservice described here already has a Docker image available at the Docker Hub registry, ready to be consumed. However, if you want to craft your own Docker image, this chapter will cover the steps to make it available inside your Kubernetes/OpenShift cluster.

Each microservice uses the same base Docker image provided by the Fabric8 team. The image fabric8/java-alpine-openjdk8-jdk uses OpenJDK 8.0 installed on Alpine Linux distribution, which makes the image as small as 74 MB.

This image also provides nice features like adjusting the JVM arguments -Xmx and -Xms, and makes it really simple to run fat JARs.

An example Dockerfile to build a Java fat jar image would be as simple as:

FROM fabric8/java-alpine-openjdk8-jdk
ENV JAVA_APP_JAR <your-fat-jar-name>
ENV AB_OFF true
ADD target/<your-fat-jar-name> /deployments/

The environment variable JAVA_APP_JAR specifies the name of the JAR file that should be called by the java -jar command. The environment variable AB_OFF disables the agent bond that enables jolokia and JMX exporter.

Packaging Our Microservices as Docker Images

Navigate to the hello-springboot directory, and create a Dockerfile there with the following contents:

FROM fabric8/java-alpine-openjdk8-jdk
ENV JAVA_APP_JAR hello-springboot-1.0.jar
ENV AB_OFF true
ADD target/hello-springboot-1.0.jar /deployments/

You can then run the following command to build the Docker image:

$ docker build -t rhdevelopers/hello-springboot:1.0 .
Sending build context to Docker daemon  111.3MB
Step 1/4 : FROM fabric8/java-alpine-openjdk8-jdk
 ---> 4353c2196b11
Step 2/4 : ENV JAVA_APP_JAR demo-thorntail.jar
 ---> Running in 777ae0de6868
 ---> 220aece2f437
Removing intermediate container 777ae0de6868
Step 3/4 : ENV AB_OFF true
 ---> Running in 02b3780c59a3
 ---> fc1bc50fc932
Removing intermediate container 02b3780c59a3
Step 4/4 : ADD target/demo-thorntail.jar /deployments/
 ---> 3a4e91b41727
Removing intermediate container b291546e0725
Successfully built 3a4e91b41727

The switch -t specifies the tag name, in the following format: <organization-name>/<image-name>:<version>.

If you want to try your newly built image, you can create a container using the command:

$ docker run -it --rm  
-p 8080:8080 rhdevelopers/hello-springboot:1.0

The -it switch instructs Docker to create an interactive process and assign a terminal to it. The --rm switch instructs docker to delete this container when we stop it. The -p 8080:8080 instructs Docker to assign port 8080 from the container to port 8080 in the Docker daemon host.

Once your container starts, you can open a new terminal and access it using the curl command:

$ curl $(minishift ip):8080/api/hello
Guten Tag aus 172.17.0.6

These are the basics steps to deploy your application in a Docker container and try it. However, we will not focus on pure Docker containers. Instead, in the next chapter, we will see how to use Kubernetes to run these microservices at scale. Before moving on, don’t forget to stop the running hello-springboot Docker container by pressing Ctrl-C.

Deploying to Kubernetes

There are several ways that we could deploy our microservices/containers inside a Kubernetes/OpenShift cluster. However, for didatic purposes, we will use YAML files that express very well what behavior we expect from the cluster.

In the source code for this report, for each microservice example there is a folder called kubernetes containing two files: deployment.yml and service.yml. The deployment file will create a Deployment object with one replica.

The Deployment also provides at least two environment variables. The one called JAVA_OPTIONS specifies the JVM arguments, like -Xms and -Xmx. The one called GREETING_BACKENDSERVICEHOST replaces the values we defined in our first two microservices to find the BACKEND service, as you’ll see in “Service discovery”.

Here is the deployment.yml file used for the hello-springboot microservice.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: hello-springboot
  labels:
    app: hello-springboot
    book: microservices4javadev
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello-springboot
      version: v1
  template:
    metadata:
      labels:
        app: hello-springboot
        book: microservices4javadev
        version: v1
    spec:
      containers:
      - env:
        - name: JAVA_OPTIONS
          value: -Xmx256m -Djava.net.preferIPv4Stack=true
        - name: GREETING_BACKENDSERVICEHOST
          value: backend
        image: rhdevelopers/hello-springboot:1.0
        imagePullPolicy: IfNotPresent
        livenessProbe:
          httpGet: # make an HTTP request
            port: 8080 # port to use
            path: /actuator/health # endpoint to hit
            scheme: HTTP # or HTTPS
          initialDelaySeconds: 20
          periodSeconds: 5
          timeoutSeconds: 1
        name: hello-springboot
        ports:
        - containerPort: 8080
          name: http
          protocol: TCP
        readinessProbe:
          httpGet: # make an HTTP request
            port: 8080 # port to use
            path: /actuator/health # endpoint to hit
            scheme: HTTP # or HTTPS
          initialDelaySeconds: 10
          periodSeconds: 5
          timeoutSeconds: 1

Since the deployment.yml and service.yml files are stored together with the source code for this report, you can deploy the microservices by pointing directly to those files using the following commands. First, deploy the backend microservice:

$ oc create -f http://raw.githubuserconoc create -f http://
  raw.githubusercontent.com/redhat-developer/
  microservices-book/master/backend/kubernetes/deployment.yml

$ oc create -f http://raw.githubuserconoc create -f http://
  raw.githubusercontent.com/redhat-developer/
  microservices-book/master/backend/kubernetes/service.yml

Then deploy the hello-springboot microservice:

$ oc create -f http://raw.githubusercontent.com/
  redhat-developer/microservices-book/master/
  hello-springboot/kubernetes/deployment.yml

$ oc create -f http://raw.githubusercontent.com/
  redhat-developer/microservices-book/
  master/hello-springboot/kubernetes/service.yml

and the hello-microprofile microservice:

$ oc create -f http://raw.githubusercontent.com/
  redhat-developer/microservices-book/
  master/hello-microprofile/kubernetes/deployment.yml

$ oc create -f http://raw.githubusercontent.com/
  redhat-developer/microservices-book/
  master/hello-microprofile/kubernetes/service.yml

Finally, deploy the api-gateway microservice:

$ oc create -f http://raw.githubusercontent.com/
  redhat-developer/
  microservices-book/master/api-gateway/kubernetes/deployment.yml

$ oc create -f http://raw.githubusercontent.com/
  redhat-developer/
  microservices-book/master/api-gateway/kubernetes/service.yml

The deployment files will create four pods (one replica for each microservice). The service files will make each of these replicas visible to each other. You can check the pods that have been created through the command:

$ oc get pods
NAME                                 READY     STATUS
api-gateway-5985d46fd5-4nsfs         1/1       Running
backend-659d8c4cb9-5hv2r             1/1       Running
hello-microprofile-844c6c758-mmx4h   1/1       Running
hello-springboot-5bf5c4c7fd-k5mf4    1/1       Running

What advantages does Kubernetes bring as a cluster manager? Let’s start by exploring the first of many. Let’s kill a pod and see what happens:

$ oc delete pod hello-springboot-5bf5c4c7fd-k5mf4
pod "hello-springboot-5bf5c4c7fd-k5mf4" deleted

Now let’s list our pods again:

$ oc get pods
NAME                                 READY     STATUS
api-gateway-5985d46fd5-4nsfs         1/1       Running
backend-659d8c4cb9-5hv2r             1/1       Running
hello-microprofile-844c6c758-mmx4h   1/1       Running
hello-springboot-5bf5c4c7fd-28mpk    1/1       Running

Wow! There are still four pods! Another pod was created after we deleted the previous one. Kubernetes can start/stop/auto-restart your microservices for you. Can you imagine what a headache it would be to manually determine whether your services are started/stopped at any kind of scale? Let’s continue exploring some of the other valuable cluster management features Kubernetes brings to the table for managing microservices.

External access

Now that all our microservices have been deployed inside the cluster, we need to provide external access. Since we have an API Gateway defined, only this microservice needs to be exposed; it will be the single point of access to invoke the hello-microprofile and hello-springboot microservices. For a refresher on our microservices architetcure, take a look at Figure 6-1.

Figure 6-1 Calling another service
Figure 6-1. Calling another service

To enable external access using OpenShift, we need to run the oc expose service command, followed by the name of the service that we want to expose (in this case, api-gateway):

$ oc expose service api-gateway
route.route.openshift.io/api-gateway exposed

Now you can try the curl command to test all microservices.

$ curl http://api-gateway-tutorial.$(minishift ip).nip.io
  /api/gateway
["Hello from cluster Backend at host: 172.17.0.7",
  "Hello Spring Boot from cluster Backend at host: 172.17.0.7"]

The output should show that you reached both microservices through the API gateway and both of them accessed the backend microservice, which means that everything is working as expected.

Scaling

One of the advantages of deploying in a microservices architecture is independent scalability. We should be able to replicate the number of services in our cluster easily without having to worry about port conflicts, JVM or dependency mismatches, or what else is running on the same machine. With Kubernetes, these types of scaling concerns can be accomplished with the Deployment/ReplicationController. Let’s see what deployments exist in our deployment:

 $ oc get deployments
NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE
api-gateway          1         1         1            1
backend              1         1         1            1
hello-microprofile   1         1         1            1
hello-springboot     1         1         1            1

All the deployment.yml files that we used have a replicas value of 1. This means we want to have one pod/instance of our microservice running at all times. If a pod dies (or gets deleted), then Kubernetes is charged with reconciling the desired state for us, which is replicas=1. If the cluster is not in the desired state, Kubernetes will take action to make sure the desired configuration is satisfied. What happens if we want to change the desired number of replicas and scale up our service?

$ oc scale deployment hello-springboot --replicas=3
deployment.extensions/hello-springboot scaled

Now if we list the pods, we should see three pods running our hello-springboot application:

$ oc get pods
NAME                                 READY     STATUS
api-gateway-76649cffc-dgr84          1/1       Running
backend-659d8c4cb9-5hv2r             1/1       Running
hello-microprofile-844c6c758-mmx4h   1/1       Running
hello-springboot-5bf5c4c7fd-j77lj    1/1       Running
hello-springboot-5bf5c4c7fd-ltv5p    1/1       Running
hello-springboot-5bf5c4c7fd-w4z7c    1/1       Running

If any of those pods dies or gets deleted, Kubernetes will do what it needs to do to make sure the replica count for this service is 3. Notice also that we didn’t have to change ports on these services or do any unnatural port remapping. Each of the services is listening on port 8080 and does not collide with the others.

Kubernetes also has the ability to do autoscaling by watching metrics like CPU, memory usage, or user-defined triggers and scaling the number of replicas up or down to suit. Autoscaling is outside the scope of this report but is a very valuable piece of the cluster management puzzle.

Service discovery

In Kubernetes, a Service is a simple abstraction that provides a level of indirection between a group of pods and an application using the service represented by that group of pods. We’ve seen how pods are managed by Kubernetes and can come and go. We’ve also seen how Kubernetes can easily scale up the number of instances of a particular service. In our example, we deployed our backend service from the previous chapters to play the role of service provider. How does our hello-springboot service communicate with that service?

Let’s take a look at what Kubernetes services exist:

$ oc get services
NAME                 TYPE        CLUSTER-IP       PORT(S)
api-gateway          ClusterIP   172.30.227.148   8080/TCP
backend              ClusterIP   172.30.169.193   8080/TCP
hello-microprofile   ClusterIP   172.30.31.211    8080/TCP
hello-springboot     ClusterIP   172.30.200.142   8080/TCP

The CLUSTER-IP is assigned when a Service object is created and never goes away. It’s a single, fixed IP address that is available to any applications running within the Kubernetes cluster and can be used to talk to backend pods. Take a look at the service.yml file for backend:

apiVersion: v1
kind: Service
metadata:
  name: backend
  labels:
    app: backend
    book: microservices4javadev
spec:
  ports:
  - name: http
    port: 8080
  selector:
    app: backend

The pods are “selected” with the selector field. As we saw in “Labels”, pods in Kubernetes can be “labeled” with whatever metadata we want to apply (like “version” or “component” or “team”), and those labels can subsequently be used in the selector for a Service. In this example, we’re selecting all the pods with the label app=backend. This means any pods that have that label can be reached just by using the cluster IP. There’s no need for complicated distributed registries (e.g., ZooKeeper, Consul, or Eureka) or anything like that; it’s all built right into Kubernetes. Cluster-level DNS is also built into Kubernetes. Using DNS in general for microservice discovery can be very challenging, if not downright painful. In Kubernetes, the cluster DNS points to the cluster IP, and since the cluster IP is a fixed IP and doesn’t go away, there are no issues with DNS caching and other gremlins that can pop up with traditional DNS.

When we deployed our hello-microprofile and hello-springboot microservices, the deployment.yml files each declared an environment variable called GREETING_BACKENDSERVICEHOST with the value backend. That value matches the name of the backend service that we saw in the output of the command oc get services. This name was declared in the backend service’s service.yml file.

This environment value replaces the value that we declared in their application.properites file for our hello-springboot microservice (greeting.backendServiceHost=localhost; see “Calling Another Service”). In our hello-microprofile microservice, the value was declared using the annotation @ConfigProperty(name = "greeting.backendServiceHost", defaultValue = "localhost") (see Example 3-5).

Because we declared the name of the service and not the IP address, all it takes to find it is a little bit of DNS and the power of Kubernetes service discovery. One big thing to notice about this approach is that we did not specify any extra client libraries or set up any registries or anything. We happen to be using Java in this case, but using Kubernetes cluster DNS provides a technology-agnostic way of doing basic service discovery!

Fault Tolerance

Complex distributed systems like microservices architectures must be built with an important premise in mind: things will fail. We can spend a lot of energy trying to prevent failures, but even then we won’t be able to predict every case of where and how dependencies in a microservices environment can fail. A corollary to our premise that things will fail is therefore we must design our services for failure. Another way of saying that is that we need to figure out how to survive in an environment where there are failures.

Cluster Self-Healing

If a service begins to misbehave, how will we know about it? Ideally, you might think, our cluster management solution could detect and alert us about failures and let human intervention take over. This is the approach we typically take in traditional environments. But when running microservices at scale, where we have lots of services that are supposed to be identical, do we really want to have to stop and troubleshoot every possible thing that can go wrong with a service? Long-running services may experience unhealthy states. An easier approach is to design our microservices such that they can be terminated at any moment, especially when they appear to be behaving incorrectly.

Kubernetes has a couple of health probes we can use out of the box to allow the cluster to administer and self-heal itself. The first is a readiness probe, which allows Kubernetes to determine whether or not a pod should be considered in any service discovery or load-balancing algorithms. For example, some Java apps may take a few seconds to bootstrap the containerized process, even though the pod is technically up and running. If we start sending traffic to a pod in this state, users may experience failures or inconsistent states. With readiness probes, we can let Kubernetes query an HTTP endpoint (for example) and only consider the pod ready if it gets an HTTP 200 or some other response. If Kubernetes determines a pod does not become ready within a specified period of time, the pod will be killed and restarted.

Another health probe we can use is a liveness probe. This is similar to the readiness probe; however, it’s applicable after a pod has been determined to be “ready” and is eligible to receive traffic. Over the course of the life of a pod or service, if the liveness probe (which could also be a simple HTTP endpoint) starts to indicate an unhealthy state (e.g., HTTP 500 errors), Kubernetes can automatically kill the pod and restart it.

In the deployment.yml file shown earlier we declared both of these, with livenessProbe and readinessProbe:

livenessProbe:
  httpGet: # make an HTTP request
      port: 8080 # port to use
      path: /actuator/health # endpoint to hit
      scheme: HTTP # or HTTPS
...
readinessProbe:
  httpGet: # make an HTTP request
      port: 8080 # port to use
      path: /actuator/health # endpoint to hit
      scheme: HTTP # or HTTPS

This means the “readiness” of a hello-springboot pod will be determined by periodically polling the /actuator/health endpoint of the pod (the endpoint was added when we added the actuator to our Spring Boot microservice earlier). If the pod is ready, it returns:

{"status":"UP"}

The same thing can be done with MicroPorfile/Thorntail and Apache Camel.

Circuit Breaker

As a service provider, your responsibility is to your consumers to provide the functionality you’ve promised. Following promise theory, a service provider may depend on other services or downstream systems but cannot and should not impose requirements upon them. A service provider is wholly responsible for its promise to consumers. Because distributed systems can and do fail, however, there will be times when service promises can’t be met or can be only partly met. In our previous examples, we showed our “hello” microservices reaching out to a backend service to form a greeting at the /api/greeting endpoint. What happens if the backend service is not available? How do we hold up our end of the promise?

We need to be able to deal with these kinds of distributed systems faults. A service may not be available; a network may be experiencing intermittent connectivity; the backend service may be experiencing enough load to slow it down and introduce latency; a bug in the backend service may be causing application-level exceptions. If we don’t deal with these situations explicitly; we run the risk of degrading our own service; holding up threads, database locks, and resources, and contributing to rolling, cascading failures that can take an entire distributed network down. The following subsections present two different approaches to help us account for these failures (one for each technology, Spring Boot and MicroProfile).

Circuit breaker with MicroProfile

Let’s start with MicroProfile, by adding a microprofile-fault-tolerance dependency to our Maven pom.xml:

<dependency>
    <groupId>io.thorntail</groupId>
    <artifactId>microprofile-fault-tolerance</artifactId>
</dependency>

Now we can annotate our greeting() method with the annotations @CircuitBreaker and @Timeout. But if a backend dependency becomes latent or unavailable and MicroProfile intervenes with a circuit breaker, how does our service keep its promise? The answer to this may be very domain-specific. Consider our earlier example of a personalized book recommendation service. If this service isn’t available or is too slow to respond, the backend could default to sending a book list that’s not personalized. Maybe we’d send back a book list that’s generic for users in a particular region, or just a generic “list of the day.” To do this, we can use MicroProfile built-in fallback() method. In Example 6-1, we add a fallback() method to return a generic response if the backend service is not available. We also add the @Fallback annotation to our GreeterRestController class specifying the method to call if the greeting invocation fails, along with the @Timeout annotation to avoid the microservice waiting more than one second for a reply.

Example 6-1. src/main/java/com/example/hellomicroprofile/rest/GreeterRestController.java
@RestController
@RequestMapping("/api")
@ConfigurationProperties(prefix="greeting")
public class GreeterRestController {

    // Not showing variables

    @GET
    @Produces("text/plain")
    @Path("greeting")
    @CircuitBreaker
    @Timeout
    @Fallback(fallbackMethod = "fallback")
    public String greeting(){
      // greeting implementation
    }

    public String fallback(){
        return saying + " at host "  +
            System.getenv("HOSTNAME") + " - (fallback)";
    }

}

Circuit breaker with Spring Boot

For Spring Boot, we will use a library from the NetflixOSS stack named Hystrix. This library is integrated with Spring through a Spring library called Spring Cloud.

Hystrix is a fault-tolerant Java library that allows microservices to hold up their end of a promise by:

  • Providing protection against dependencies that are unavailable

  • Monitoring and providing timeouts to guard against unexpected dependency latency

  • Load shedding and self-healing

  • Degrading gracefully

  • Monitoring failure states in real time

  • Injecting business logic and other stateful handling of faults

First, let’s add the spring-cloud-starter-netflix-hystrix dependency to our Maven pom.xml:

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
  <version>2.0.2.RELEASE</version>
</dependency>

In Example 6-2, we add the annotation @HystricCommand and the fallback() method to to the GreeterRestController class.

Example 6-2. src/main/java/com/example/GreeterRestController.java
@RestController
@RequestMapping("/api")
@ConfigurationProperties(prefix="greeting")
public class GreeterRestController {

    // Not showing variables and setters

    @RequestMapping(value = "/greeting",
    method = RequestMethod.GET, produces = "text/plain")
    @HystrixCommand(fallbackMethod = "fallback")
    public String greeting(){

        String backendServiceUrl = String.format(
            "http://%s:%d/api/backend?greeting={greeting}",
            backendServiceHost, backendServicePort);

        System.out.println("Sending to: " + backendServiceUrl);

        BackendDTO response = template.getForObject(
        backendServiceUrl, BackendDTO.class, saying);

        return response.getGreeting() + " at host: " +
        response.getIp();
    }

    public String fallback(){
        return saying + " at host "  +
                System.getenv("HOSTNAME") + " - (fallback)";
    }

}

We also need to add one last annotation, @EnableCircuitBreaker as shown in Example 6-3; that’s necessary to tell Spring Cloud that the hello-springboot application uses circuit breakers and to enable their monitoring, opening, and closing (behavior supplied, in our case, by Hystrix).

Example 6-3. src/main/java/com/redhat/examples/hellospringboot/HelloSpringbootApplication
@EnableCircuitBreaker
@SpringBootApplication
public class HelloSpringbootApplication {

	public static void main(String[] args) {
		SpringApplication
                .run(HelloSpringbootApplication.class, args);
	}

}

Testing the circuit breaker

We know that both microservices (hello-microprofile and hello-springboot) depend on backend. What should be the expected behavior if we stop the backend application after we’ve made the latest modifications?

To test this behavior, we need to recreate the Docker image and restart the containers so the new image will be loaded.

If you want to build your own source code, navigate to the directory where your microservices are, create the Dockerfile as instructed in “Packaging Our Microservices as Docker Images”, and run the following commands to rebuild the Docker images:

$ cd <PROJECT_ROOT>/hello-springboot
$ mvn clean package
$ docker build -t rhdevelopers/hello-springboot:1.0 .

$ cd <PROJECT_ROOT>/hello-microprofile
$ mvn clean package
$ docker build -t rhdevelopers/hello-microprofile:1.0 .

Now that the images have been rebuilt, we can delete the previous running containers—Kubernetes will restart them using the new Docker images:

$ oc delete pod -l app=hello-springboot
$ oc delete pod -l app=hello-microprofile

We also need to scale the backend service to 0 replicas:

$ oc scale deployment backend --replicas=0

Now, we wait for all pods but the backend to be Running and try the api-gateway service to get the response from both microservices:

$ curl http://api-gateway-tutorial.$(minishift ip).nip.io
  /api/gateway
["Hello at host hello-
  microprofile-57c9f8f9f4-24c2l- (fallback)",
"Hello Spring Boot at host hello-
 springboot-f797878bd-24hxm- (fallback)"]

The response now shows that the fallback() methods in both microservicies were invoked because no communication with the backend service was possible.

The idea behind this contrived example is ubiquitous. However, the decision of whether to fall back, gracefully degrade, or break a promise is very domain-specific. For example, if you’re trying to transfer money in a banking application and a backend service is down, you may wish to reject the transfer, or you may wish to make only a certain part of the transfer available while the backend gets reconciled. Either way, there is no one-size-fits-all fallback method. In general, the ideal fallback solution is dependent on what kind of customer experience gets exposed and how best to gracefully degrade considering the domain.

Load Balancing

In a highly scaled distributed system, we need a way to discover and load balance against services in the cluster. As we’ve seen in previous examples, our microservices must be able to handle failures; therefore, we have to be able to load balance against services that exist, services that may be joining or leaving the cluster, or services that exist in an autoscaling group. Rudimentary approaches to load balancing, like round-robin DNS, are not adequate. We may also need sticky sessions, autoscaling, or more complex load-balancing algorithms. Let’s take a look at a few different ways of doing load balancing in a microservices environment.

Kubernetes Load Balancing

The great thing about Kubernetes is that it provides a lot of distributed systems features out of the box; no need to add any extra components (server side) or libraries (client side). Kubernetes Services provided a means to discover microservices, and they also provide server-side load balancing. If you recall, a Kubernetes Service is an abstraction over a group of pods that can be specified with label selectors. For all the pods that can be selected with the specified selector, Kubernetes will load balance any requests across them. The default Kubernetes load-balancing algorithm is round robin, but it can be configured for other algorithms, such as session affinity. Note that clients don’t have to do anything to add a pod to the Service; just adding a label to your pod will enable it for selection and make it available. Clients reach the Kubernetes Service by using the cluster IP or cluster DNS service provided out of the box by Kubernetes. Also recall the cluster DNS service is not like traditional DNS and does not fall prey to the DNS caching TTL problems typically encountered with using DNS for discovery/load balancing. Furthermore, there are no hardware load balancers to configure or maintain; it’s all just built in.

To demonstrate load balancing, let’s scale up the backend services in our cluster and scale our hello-springboot service back to one replica to reduce the resource usage:

$ oc scale deployment hello-springboot --replicas=1
deployment.extensions/hello-springboot scaled

$ oc scale deployment backend --replicas=3
deployment.extensions "backend" scaled

Now if we check our pods, we should see three backend pods:

$ oc get pods
NAME                                  READY     STATUS
api-gateway-9786d4977-2jnfm           1/1       Running
backend-859bbd5cc-ck68q               1/1       Running
backend-859bbd5cc-j46gj               1/1       Running
backend-859bbd5cc-mdvcs               1/1       Running
hello-microprofile-57c9f8f9f4-24c2l   1/1       Running
hello-springboot-f797878bd-24hxm      1/1       Running

If we describe the Kubernetes services backend, we should see the the selector used to select the pods that will be eligible for taking requests. The Service will load balance to the pods listed in the Endpoints field:

$ oc describe service backend
Name:              backend
Namespace:         microservices4java
Labels:            app=backend
Annotations:       <none>
Selector:          app=backend
Type:              ClusterIP
IP:                172.30.169.193
Port:              http  8080/TCP
TargetPort:        8080/TCP
Endpoints:         172.17.0.11:8080,172.17.0.13:8080,
                   172.17.0.14:8080
Session Affinity:  None
Events:            <none>

We can see here that the backend service will select all pods with the label app=backend. Let’s take a moment to see what labels are on one of the backend pods:

$ oc describe pod/backend-859bbd5cc-ck68q | grep Labels
Labels:                     app=backend

The backend pods have a label that matches what the service is looking for, so any communication with the service will be load-balanced over these matching pods.

Let’s make a few calls to our api-gateway service. We should see the responses contain different IP addresses for the backend service:

$ curl http://api-gateway-tutorial.$(minishift ip).nip.io
  /api/gateway
 ["Hello from cluster Backend at host: 172.17.0.11",
 "Hello Spring Boot from cluster Backend at host: 172.17.0.14"]

$ curl http://api-gateway-tutorial.$(minishift ip).nip.io
  /api/gateway
 ["Hello from cluster Backend at host: 172.17.0.13",
"Hello Spring Boot from cluster Backend at host: 172.17.0.14"]

We used curl here, but you can use your favorite HTTP/REST tool, including your web browser. Just refresh your browser a few times; you should see that the backend that gets called is different each time, indicating that the Kubernetes Service is load balancing over the respective pods as expected.

When you’re done experimenting, don’t forget to reduce the number of replicas in your cluster to reduce the resource consumption:

$ oc scale deployment backend --replicas=1
deployment.extensions/backend scaled

Where to Look Next

In this chapter, you learned a little about the pains of deploying and managing microservices at scale and how Linux containers can help. We can leverage true immutable delivery to reduce configuration drift, and we can use Linux containers to enable service isolation, rapid delivery, and portability. We can leverage scalable container management systems like Kubernetes features that are built in, such as service discovery, failover, health-checking, and much more. You don’t need complicated port swizzling or complex service discovery systems when deploying on Kubernetes because these are problems that have been solved within the infrastructure itself. To learn more, please review the following resources:

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.218.224.226