In Chapter 1, we discussed microservice architecture. Businesses are increasingly moving away from the monolith architecture to take advantage of the microservice architecture. But a microservices system is a distributed system. To use it efficiently, we need to adopt additional infrastructure components. These additional components prescribe a new set of guidelines that must be followed with each microservice.
Microservice architecture advocates granule services for evolving business needs. Depending on changing business needs, development teams can create or combine services. Moreover, in the production environment, each of the services is deployed and scaled independently. Cloud-based autoscaling often replicates instances based on service load. Thus, the architecture is in constant evolution, and there is no end-state list of microservices.
A dynamic ecosystem requires a catalog of the latest running microservices. This is also known as the service registry. In a nutshell, the registry is a database of services with details of their instances and the corresponding locations. To work efficiently, services must be registered on startup and removed on shutdown. There are many ways to achieve this, but the process of service self-registration is the recommended mechanism.
Once the services are registered with the registry, a client needs a lookup for the same service. This client-side process is known as service discovery (see Figure 6-1). The client first queries the service registry to find the available instances of a service. After getting the list of active service instances, the client can send a request to the required service.
Figure 6-1
Service registration and service discovery
The service registry is often a key-value store of information. Many times, you must register additional information about the service. This information can be related to the client’s type in a multitenant system, or the view provided, like web or mobile, or any other information. Every service is responsible for adding this data to the store. In this chapter, we use the service self-registration mechanism to integrate Traefik with the microservices.
Microservice architecture recommends service collaboration. This means that one service can invoke other services to get the required data for a user request. But the mechanism has its own set of issues. When one service synchronously invokes another service, there is always the possibility that the other service is unavailable or is exhibiting such high latency that it is essentially unusable.
Precious resources such as threads might be consumed in the caller while waiting for the other service to respond. This might lead to resource exhaustion, which would make the calling service unable to handle other requests.
The failure of one service can potentially cascade to other services throughout the application. The problem can be fixed by adapting circuit breakers (see Figure 6-2) in the application design. When the number of consecutive failures crosses a threshold, the circuit breaker trips, and for the duration of a timeout period, all attempts to invoke the remote service fail immediately.
Figure 6-2
Circuit breaker pattern
After the timeout expires, the circuit breaker allows a limited number of test requests to pass through. If those requests succeed, the circuit breaker resumes normal operation. Otherwise, if there is a failure, the timeout period begins again. In this chapter, we integrate circuit breakers while invoking different microservices from Traefik.
API gateways are an essential part of any microservice-based architecture. Cross-cutting concerns such as authentication, load balancing, dependency resolution, data transformations, and dynamic request dispatching can be handled conveniently and generically. Microservices can then focus on their specific tasks without code duplication. This results in easier and faster development of each microservice.
Implement an API gateway that is the single entry point for all clients. The API gateway handles requests in one of two ways. Some requests are proxied/routed to the appropriate service. It handles other requests by fanning out to multiple services. In previous chapters, we configured Traefik for the requirements for single services. In this chapter, we configure Traefik as a microservices gateway.
Pet-Clinic Application
In this chapter, we need a microservices-based application. The application must have at least two or more microservices integrated with Traefik. We work with a PetClinic application. PetClinic is a Java-based application which was packaged with the Spring framework for learning purpose. The Spring community maintains the application. It explains Spring framework-based technologies in detail. Thus, the application is a good test-bed for enterprise technologies.
The PetClinic application is designed for the needs of a veterinary clinic. The application enables its users to view and manage veterinarians, customers, and their pets. The application supports the following use cases.
View a list of veterinarians and their specialties
View information about a pet owner
Update the information about a pet owner
Add a new pet owner to the system
View information about a pet
Update the information about a pet
Add a new pet to the system
View information about a pet’s visitation history
Add information about a visit to the pet’s visitation history
Spring PetClinic microservices are built around small independent services (a few hundred lines of code), running in their own JVM and communicating over HTTP via a REST API. These microservices are all written in Java. Each of the three customer, vet, and visit business microservices is an application in the Spring Boot sense. To work in a distributed environment, these microservices rely on a set of tools offered by Spring Cloud: centralized configuration management, automated discovery of other microservices, and load balancing (see Figure 6-3). The application UI is developed in Angular and deployed in Nginx. Traefik will perform request-routing. We build and deploy an application in this chapter. Some important aspects are covered, but complete application technical details are beyond the scope of the book.
The config-server also removes the requirement of repackaging the application in the event of configuration changes. Since all the latest configuration is available on the listed REST endpoints, we only need a service restart. Services can also be configured for hot-reload by using @RefreshScope annotation or the EnvironmentChangeEvent event listener.
Spring follows the convention of loading application configuration from application.properties. But as discussed previously, when the spring-cloud-config server is used, application.properties is no longer part of the application. Instead, a spring-cloud-context is configured to load the configuration properties from the external sources. It can also be configured for decrypting properties in the external configuration files. Spring Cloud application initiates a bootstrap context that loads its configuration from the bootstrap.yml file. The bootstrap.yml file is minimalist. It contains the name of the microservice and the URL of the configuration server. The following is an example from the vets-service microservice.
spring:
cloud:
config:
uri: http://localhost:8888
application:
name: vets-service
We specified the spring-config-server location at localhost:8888 in the configuration. The vets-service microservice requires the server to be up on the said location. It queries the server to determine the configuration values and then complete the server startup. It is important to note that the config server’s location can be injected using environment variables like SPRING_CLOUD_CONFIG. But it can’t be discovered using the service registry.
Consul Service Registry
Previously we discussed the evolving nature of microservice architecture. When services are deployed in the cloud, you can hardly anticipate the number of instances of the same microservices (depending on the load) or where they are deployed (and on which IP and port they are accessible). Thus, there is a need for a service registry. In the PetClinic application, we used the Consul service registry. At startup, each microservices registers itself with the service registry. Post-registration, each service periodically provides a heartbeat to the registry. The book does not aim to cover details around the Consul service registry. Please refer to the documentation for more information.
Now let’s download consul from www.consul.io/. Post-download, extract the zipped file and start the service.
We can load the Consul UI at http://localhost:8500/ui/dc1/services. (see Figure 6-4)
Figure 6-4
Consul services
Deploy Pet-Clinic
Now we can run the PetClinic microservices in any order. The application consists of the following three microservices.
Vets service
Visits service
Customer service
All the services are based on Spring Boot. They are packaged as executable JAR files. Executing the service-specific jar starts the service with an embedded servlet engine. Since the service fetches are configuration from the spring-config server, so let’s make sure that we are the correct location of the config server in bootstrap.yml.
Next, make sure that the config-server points to the correct git location. In the example, https://github.com/rahul0208/spring-petclinic-microservices-config is the application configuration source. We recommended that you clone this configuration and update it per the environment.
The preceding URL is configured in bootstrap.yml for config-server.
The details of the configuration are beyond the scope of this book. We recommend that you update the URI with its own clone. Now, we need to build the services using the packages maven wrapper.
In the current setup, we deployed all services in the same box. Thus, localhost address is used in the configuration. You are free to deploy the service on any host, by making an appropriate update in their git config. As a first step, you need to start the config server with the following command.
2020-07-26 21:56:26.401 INFO 7442 --- [ main] o.s.s.p.config.ConfigServerApplication : No active profile set, falling back to default profiles: default
The next step is to run each microservice. But first let’s make sure that we have the correct address of the Consul service registry in application.yml.
spring:
cloud:
consul:
host: localhost
port: 8500
Let’s now start the vets-service with the following command.
2020-07-21 15:34:11.693 INFO [vets-service,,,] 26509 --- [ main] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at : http://localhost:8888
2020-07-21 15:34:15.525 INFO [vets-service,,,] 26509 --- [ main] c.c.c.ConfigServicePropertySourceLocator : Located environment: name=vets-service, profiles=[default], label=null, version=062fb94b71dc6b99e6518fe7088a0bff3a9431d1, state=null
Similarly, we need to start the customer-service and visits services. Each service registers itself in the Consul service registry. You can validate the service details in the Consul dashboard (see Figure 6-5). Additionally, each of these services may report a failure for the Zipkin-based request tracing. Request tracing offers various benefits. In Chapter 5, we covered the integration of these tools with Traefik. We do not cover these integrations in this chapter.
Figure 6-5
Dashboard UI
Pet-Clinic UI
The application UI is built using 1.7 version on AngularJS. These HTML pages are deployed as static resources of a Spring web Application. Alternatively, we can pack and deploy them in a server like Apache Tomcat or HTTPD. The UI is also packaged as an executable JAR. Let’s now start the UI with the following command.
target $ java -jar spring-petclinic-ui-2.3.1.jar
2020-07-27 22:15:21.996 INFO [petclinic-ui,,,] 17732 --- [ restartedMain] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at : http://localhost:8888
2020-07-27 22:15:22.870 INFO [petclinic-ui,,,] 17732 --- [ restartedMain] c.c.c.ConfigServicePropertySourceLocator : Located environment: name=petclinic-ui, profiles=[default], label=null, version=8adeb754f96df6e7308344e7bb2ceddcca09b93f, state=null
The UI is deployed on 9000 port. We can access it as http://localhost:9000/#!/welcome. (see Figure 6-6)
Figure 6-6
PetClinic UI
The UI is also updating the Consul service registry. This way, the registry is a comprehensive catalog of the running services.
Configure Gateway
In the preceding setup, we deployed all microservices of the application. Now we need to configure Traefik to render the UI and route calls to each service. We can write this configuration in a file as done in previous chapters. It configures Traefik, but the approach does not work with the dynamic nature of the microservice architecture. We would need to keep updating the configuration as a new service gets added to the ecosystem. Also, it is difficult to keep updating the IP address of all instances of a service.
Alternatively, Traefik can be used with Consul key-value store as a configuration provider. All Traefik configuration is added hierarchically as key and values under a configured root node. The default root node is named “traefik”. Table 6-1 highlights some of the Traefik keys.
Traefik documentation provides the updated list of keys to configure it. As a first step, let’s add the configuration to Consul. You can create it using Consul GUI. Alternatively, you can import keys from a JSON file. The values for each key are encoded in Base64 format.
Once the configuration is imported, we should see all key-values in the Consul store. Now we need to update the Traefik static configuration to use Consul provider.
entryPoints :
web :
address : ":80"
providers:
consul:
endpoints:
- "127.0.0.1:8500"
rootKey : "traefik"
api :
insecure : true
dashboard : true
We specified the Consul provider instead of FileProvider in the configuration. We also specified the location and the root key. There are additional options to configure authentication and TLS information. Let’s start the Traefik server and look up the dashboard.
The dashboard shows the configuration from Consul key-value store. Traefik has created four new routes, one for each deployed service. If the services are not running, then routes added by the configuration are in error, as shown in Figure 6-7. If you click a route, you see an error message about the corresponding service’s missing details.
Figure 6-7
Traefik configuration from Consul
Service Details
Traefik would need the server details for each service. In a microservice architecture, service registration is the process of adding all details to the registry. In our opinion, self-registration is the simplest possible mechanism that supports every possible scenario. We extend the self-registration to add Traefik specific details for the required Consul keys. The ServiceRegistry class accomplishes this responsibility.
@Configuration
public class ServiceRegistry implements ApplicationListener<ServletWebServerInitializedEvent> {
final String serviceKey = "/traefik/http/services/{0}/loadBalancer/servers/";
final String serverKey = "/traefik/http/services/{0}/loadBalancer/servers/{1}/";
final String urlKey = "/traefik/http/services/{0}/loadBalancer/servers/{1}/url";
Determines the port on which the application is started
Adds the host and port information to the Consul KV store by using ConsulClient
Adds the values to the Traefik keys /traefik/http/services/{0}/loadBalancer/servers/{1}/url
Removes the keys at service shutdown
The ServiceRegistry class is part of every service. If you start all services and reload the dashboard, you see that all the errors are fixed. (see Figure 6-8)
Figure 6-8
PetClinic configuration
Note
Traefik continues to watch values in the KVS store. It automatically reloads the configuration as it is updated in Consul.
Now let’s load the PetClinic application on http://localhost/. The application performs as expected. We can load and save data across the three different microservices. (see Figure 6-9)
Figure 6-9
PetClinic
Circuit Breaker
We said that microservices often collaborate to deliver a complete user function. A service invokes other services to get relevant data. But in a distributed system like microservices, the remote service call can hang for a while before failing. An unresponsive service call blocks the resources from the calling service. If there are many of these calls, the system may run out of critical resources leading to cascading failures across multiple systems.
A circuit breaker is often applied to address this problem with fail-fast behavior. The circuit breaker tracks remote calls. In an unhealthy response, the circuit breaker returns immediately without sending the call to the destination service. This book does not cover the pattern in detail.
Traefik provides middleware that can configure a circuit breaker. Since the circuit breaker is configured as part of the middleware chain, the circuit breaker only alters the behavior after its execution. It is important to note that even though the circuit breaker is declared once in the configuration, but it configures as individual instances for each route. Traefik can detect service error rates in the following metrics.
Latency: Traefik can measure service quantile latency time. The circuit breaker can be triggered if the measured time is more than a configured value (e.g., LatencyAtQuantileMS(50.0) > 100). The argument specifies the quantile.
Response code: Traefik can measure service response status codes (e.g., ResponseCodeRatio(500, 600, 0, 600) > 0.25). The four arguments here are HTTP status codes.
Error status code From
Error status code To
Application Status code From
Application Status code To
Each of these metric values can be checked by using the following operators.
Greater than (>)
Greater or equal than (>=)
Lesser than (<)
Lesser or equal than (<=)
Equal (==) and
Not Equal (!=) operators
You can also combine two or more metrics using AND (&&) and OR (||) operators. When Traefik determines that a circuit breaker has been triggered, it does not forward the call to the destination service; instead, it returns a 503 response. (see Figure 6-10)
Figure 6-10
Traefik circuit breaker
Let’s now add the following response status circuit breaker to over services.
We added circuit-breaker middleware for the customer-service route to the configuration. The circuit breaker is triggered when customer service returns 500 error code. Traefik also shows the circuit breaker middleware configuration on the dashboard. (see Figure 6-11)
Figure 6-11
Route with circuit breaker
Retries
In a dynamic ecosystem, service instances can be in a starting state. There can be intermittent network connectivity errors. These transient errors are generally self-correcting. If you retry the service call, chances are it will succeed. Retry is another mechanism that makes an application fault tolerant.
The retry pattern states that you can retry a failed request. It is important to identify which failures may work with this approach. If the application reports an invalid data error, then the chances are high that it does not work on retry. Additionally, a failed request propagated throughout the system creates unnecessary bottlenecks. On the other hand, if a request has failed due to the connection or a response timeout, chances are high that it will succeed if retried.
Traefik supports the retry pattern by using retry middleware, which reissues request a specified number of times to a service if there are timeout errors. The middleware stops reissue as soon as there is a response from the service. The middleware done validates if the response received is erroneous. We can add retry middleware configuration and enable it for routes in the following manner.
We configured the retry middleware for four retries. The middleware is applied to the vets-service route in the configuration. It is difficult to test such a configuration. The configuration is applicable when the service is slow to respond. The error is only replicated in-case of connection timeouts. These are actual network errors. The retry mechanism does not kick in for request timeouts. In these cases, the request is processing by the service, but the processing can be very slow. Such requests, if retried, can cause unintended issues, like dual debit in account debit requests.
Retries can only work properly if there are configured timeouts in Traefik. This is a global level configuration. Traefik provides serversTransport.forwardingTimeouts static configuration attributes that can control the timeouts to the servers.
idleConnTimeout: Specifies the maximum amount of time an idle connection remain idle before closing
responseHeaderTimeout: Specifies the amount of time to wait for a server’s response headers
dialTimeout: Specifies the time spent establishing a connection
serversTransport:
forwardingTimeouts:
responseHeaderTimeout: 1s
dialTimeout: 1s
idleConnTimeout: 1s
We configured one-second timeouts for service response and idle connection in the configuration. Let’s restart Traefik and validate the retry middleware configuration on the dashboard.
Figure 6-12
Route with retry
The configuration shows that three retries are performed for a failed request.
Throttling
In a microservice architecture, different services collaborate to deliver user value. This involves applying enterprise integration techniques to applications to address various issues. One of the common issues in application integration is about controlling the consumption of resources. Some of the resources are expensive to create, so their access must be moderated with service level agreements. Throttling is the method that controls misbehaving or rouge services by sending more requests than the service level agreement. It is essential to apply this to a critical-section service as the complete ecosystem stall if the critical-section service fails. This can help improve application capacity planning.
Throttling is often implemented by rejecting overflowing requests. Traefik supports throttling by using rate-limit middleware. It can measure average calls within a defined period from a particular source. The middleware sends HTTP status 429 (too many calls) to the source service if it invokes calls more than the configured limit. The middleware provides the following three attributes to configure the API rate.
Average: Counts the number of requests in the configured period
Period: Specifies the time (the rate is defined as the average calls/period)
Burst: Specifies how to handle the maximum request within a short time
There are options to control the source identification. We can enable the rate-limit middleware for the vets service.
Traefik updates the configuration and shows it in the dashboard, as shown in Figure 6-13.
We defined the rate as 1 request/30 seconds. If we try to make a couple of requests for /api/vets/vets, you see the following response.
Figure 6-13
Throttle dashboard
Request URL: http://localhost/api/vet/vets
Request Method: GET
Status Code: 429 Too Many Requests
Remote Address: [::1]:80
Referrer Policy: no-referrer-when-downgrade
Middleware Chain
Traefik provides a chain middleware that can be used to simplify configurations applied across different services. The chain middleware can be used to group middleware in order. The complete group can be applied to a route, removing the need to apply each middleware separately. The complete chain is specified by providing a comma-separated list to the chain middleware attribute. In this scenario, we can configure the chain to consist of the circuit breaker and rate limit middleware.
Traefik updates the configuration and shows it in the dashboard, as shown in Figure 6-14.
Figure 6-14
Chained middleware
Canary Deployments
Traefik supports Canary deployments using weighted round-robin. In the previous section, we added the ServiceRegistry class to add instance details. These instances are used in a round-robin manner. In Chapter 3, we discussed the weighted round-robin, where weights were added to server instances. Traefik divides received requests in the ratio of the provide weights. As we start new instances, you see new services in the Consul service registry (see Figure 6-15). Weight can be added from the UI for each instance.
Figure 6-15
Multiple instances
Redirecting a subset of requests to a new service is one of the fundamental foundations for canary deployments. The mechanism can be automated by using a Consul client. But the complete end-to-end flow needs additional components which can provision and deploy the newly released version.
Note
This chapter integrated Traefik with the Consul KV store. Traefik also provides integration with Zookeeper and etcd. Configuration with others is similar to Consul-based integration, but some features do not work as expected.
Summary
In this chapter, we deployed a microservices-based solution and configured Traefik as a gateway to it. We used the Consul service registry for the microservices. Traefik is enabled to read the configuration from the Consul KV store. Traefik can detect updates in the KV store and perform hot reloads. This keeps the configuration updated in a dynamic ecosystem like microservices.
Services can register/deregister them as they start/shutdown. These updates are picked by Traefik, which provides middleware that can be configured in the microservice architecture. This chapter looked at the circuit breaker, retry, rate limit, and chain middleware. It also looked at the weighted round-robin-based deployment, which can split the requests for canary deployments. In the next chapter, we deploy the microservices solution on an orchestration engine and configure Traefik.
