Installing the plugin

In this case, we are making use of the CloudBees Docker Build and Publish plugin, which you can install either using the graphical interface (from the main page, choose Manage Jenkins and then Manage Plugins), or using the command line while logged into the Jenkins server:

/usr/local/bin/install-plugins.sh docker-build-publish

You need to restart Jenkins after this to activate the plugin.

Creating the DockerHub credentials

Unless you are publishing your Docker container images to a private repository that doesn’t require authentication, you need to provide Jenkins with some kind of authentication mechanism when pushing each new container image. In fact, the CloudBees Docker Build and Publish plugin uses Docker Hub by default, so you need a Docker Hub account to proceed. Here are basic steps to do this in Jenkins:

1. Go to Credentials and then Add Credentials, as shown in Figure 10-2.

   

   Figure 10-2. Creating new credentials in Jenkins

2. Select “Username with password” from the Kind drop-down menu.

3. Leave the scope as Global or restrict it to something more specifically aligned to your own security policies.

4. Give your credentials a meaningful name in the ID field, like DockerHub, and optionally a description.

5. Enter the username used to publish images.

6. Enter the password and then click OK. Figure 10-3 shows the selection of these options.

   

   Figure 10-3. Adding new Docker Hub credential to Jenkins

Building and publishing

Finally, you can now create a step within your job definition to build the Docker container image and publish it into Docker Hub (or any other registry):

1. When adding the new step, select the type Docker Build and Publish to open the window shown in Figure 10-4.

2. Indicate the name that you want to give to the published image; this is equivalent to using the -t option in the command line when building the Docker image.

3. Indicate the credentials to use when pushing the image to Docker Hub.

   

   Figure 10-4. Creating a new build step to build and publish a Docker image

4. If the necessary Dockerfile isn’t in the root folder of your repository, indicate the folder where it is to be found, as shown in Figure 10-5.

   

   Figure 10-5. Indicating the folder where the Dockerfile resides

With this, your build pipeline can now create Docker images for your applications in an automated manner, and you are ready to deploy them to your platform of choice. 

The plugin example: Kubernetes

As explained in Chapter 4, Kubernetes is an orchestrator platform for deploying immutable services encapsulated in containers. In Chapter 8, we already indicated how Docker and Kubernetes could be used to allow developers to work locally, mimicking a production environment. Now we will indicate how this deployment can be automated to deploy to the production Kubernetes cluster by using a plugin in Jenkins. A fully functional example is available at the Extended Java Shop repository—more precisely, at the jenkins-kubernetes folder. You can follow the instructions at the README.md file in that folder to run the example locally. (Incidentally, this example will also make use of a locally running minikube instance, but the process will be no different from a real-life Kubernetes cluster.)

/usr/local/bin/install-plugins.sh kubernetes-cd

---
apiVersion: v1
kind: Service
metadata:
  name: featureflags
  labels:
    app: featureflags
spec:
  type: NodePort
  selector:
    app: featureflags
  ports:
  - protocol: TCP
    port: 8040
    name: http

---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  name: featureflags
  labels:
    app: featureflags
spec:
  replicas: 1
  selector:
    matchLabels:
      app: featureflags
  template:
    metadata:
      labels:
        app: featureflags
    spec:
      containers:
      - name: featureflags
        image: quiram/featureflags
        ports:
        - containerPort: 8040
        livenessProbe:
          httpGet:
            path: /health
            port: 8040
          initialDelaySeconds: 30
          timeoutSeconds: 1

Creating the deployment job

Finally, your Jenkins server is ready to configure the deployment job. For this, you can create a new Freestyle element as indicated in “Jenkins”, configure the repository location, and add a build step of type Deploy to Kubernetes, as shown in Figure 10-7.

Figure 10-7. Adding a build step to deploy to Kubernetes

The configuration of the Deploy to Kubernetes step will need references to only two of the elements previously constructed: the Kubeconfig, for which you can select the Kubeconfig Credentials that was created previously (“kubernetes” in Figure 10-8), and the path to the service definition within the repository. You can now save this job, and you will have an automated way to deploy your services to Kubernetes.

Figure 10-8. Configuring the build step to deploy to Kubernetes

You can then repeat this step for each of the services, or you can tweak the existing one to take a parameter that indicates the service to deploy; this is the course that was chosen in the example available at the jenkins-kubernetes folder.

The CLI example: Amazon ECS

One of the advantages of using plugins in your build pipeline of choice is that they are usually nicer and more user-friendly to operate with. On the other side, one of the disadvantages is the lack of portability: if you ever want to change to a different automated build platform, you’ll probably have to start from scratch. This is one of the reasons that you may choose to leave plugins aside and use command-line tools instead.

This is the option that has been chosen in the second of our examples: deploying to Amazon Elastic Container Service (ECS). Amazon ECS is in many ways similar to Kubernetes, in the sense that Amazon provides a managed cluster of computers and orchestrates the deployment of containers across them; this way, you just need to provide Amazon ECS with the Docker image information, and the platform will do the rest. The main difference is that, in Kubernetes the nodes in the cluster can either be physical or virtual machines, but in Amazon ECS, the computers must be Amazon EC2 instances, keeping everything within the Amazon ecosystem. This means that adding or removing EC2 instances to an Amazon ECS cluster is a streamlined operation, although it adds the risk of vendor lock-in.

The setup and management of an Amazon ECS cluster is beyond the scope of this book (just as setting up and managing a Kubernetes cluster also is), so in this section we will assume the Amazon ECS cluster is already available and we will just focus on how to deploy services to it. The fully functional example available at the Extended Java Shop repository (more precisely, at the jenkins-aws-ecs folder) does include scripts to create and configure a minimal Amazon ECS cluster; readers can follow the instructions at the README.md to run the example locally and check the relevant scripts to know more.

aws ecs register-task-definition 
    --family ${FAMILY}  # The family of the task, i.e. the name
    --cli-input-json file://%PATH_TO_JSON_FILE%  # File with the definition
    --region ${REGION} # Region for the task definition, default if omitted

{
  "family": "shopfront",
  "containerDefinitions": [
    {
      "image": "quiram/shopfront",
      "name": "shopfront",
      "cpu": 10,
      "memory": 300,
      "essential": true,
      "portMappings": [
        {
          "containerPort": 8010,
          "hostPort": 8010
        }
      ],
      "healthCheck": {
        "command": [ "CMD-SHELL", "curl -f http://localhost:8010/health || exit 1" ],
        "interval": 10,
        "timeout": 2,
        "retries": 3,
        "startPeriod": 30
      }
    }
  ]
}

aws ecs create-service 
    --service-name ${SERVICE_NAME}  # Name for the service
    --desired-count 1  # Desired number of tasks when this service is running
    --task-definition ${FAMILY}  # Family of the task definition to use
    --cluster ${CLUSTER_NAME}  # Cluster where the service is to be created
    --region ${REGION} # The region where the cluster is, default if omitted

aws ecs update-service 
    --service ${SERVICE_NAME}  # Name of the service to update
    --task-definition ${FAMILY}:${VERSION}  # Task definition to use
    --cluster ${CLUSTER_NAME}  # Cluster where the service currently runs
    --region ${REGION} # The region where the cluster is, default if omitted

Providing health-check endpoints

Health-check endpoints have become so ubiquitous that multiple tools and frameworks can automatically attach them to your services, so you don’t even need to create one yourself. This is the path that has been taken in the Extended Java Shop sample application, with two available variants.

Most of the web services in the Extended Java Shop are based in Spring Boot, which automatically adds a /health endpoint to any service without further action. This can be noticed in the log as the application starts up, and can be verified by simply contacting /health in the desired service after it has been deployed. For more details, see Example 10-3, an extract (edited and broken into multiple lines for readability) from the log of the Stock Manager service, where you can see that /health and /health.json have been automatically registered.

2018-05-02 11:49:37.487  INFO 56166 --- [main] o.s.b.a.e.mvc.EndpointHandlerMapping:
	Mapped "{[/health || /health.json],methods=[GET],
	produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}"
	onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.
	HealthMvcEndpoint.invoke(
	javax.servlet.http.HttpServletRequest,java.security.Principal

A different example is shown in the Product Catalogue service, which is a web service based in Dropwizard, as opposed to Spring Boot. Setting up a health check with Dropwizard requires a couple of steps, but it also comes with greater flexibility. The first step is to create a class that overrides HealthCheck, implementing the check() method; this is done in the BasicHealthCheck class, which is shown in Example 10-4. (In this case, the check simply returns the version number of the running application.)

public class BasicHealthCheck extends HealthCheck {
    private final String version;

    public BasicHealthCheck(String version) {
        this.version = version;
    }

    @Override
    protected Result check() throws Exception {
        return Result.healthy("Ok with version: " + version);
    }
}

Once the health check has been created, you need to register it in your application. This is shown in the ProductServiceApplication class, with the specific line required for registration copied next for reference:

final BasicHealthCheck healthCheck = new BasicHealthCheck(config.getVersion());
environment.healthChecks().register("healthCheck", healthCheck);

The advantage of this approach is that multiple health checks can be created and registered, all of them consulted when calling /healthcheck.

Needless to say, one doesn’t need to be constrained to whatever framework is in use, and can choose to build health-check endpoints manually. At the end of the day, the health check is just an ordinary endpoint, which can be created just like any other endpoint in the application.

Consulting health-check endpoints

Creating health-check endpoints in your services is one side of the coin, and configuring the orchestrating platform to use them is the other one. This, again, is pretty straightforward with most orchestrating platforms.

Let’s check our two examples: Kubernetes and Amazon ECS. With Kubernetes, the health check is configured at the very service definition. In fact, keen readers might have noticed the following section in Example 10-1:

livenessProbe:
  httpGet:
    path: /health
    port: 8040
  initialDelaySeconds: 30
  timeoutSeconds: 1

The parameters indicated here, together with others that have been omitted and for which default values are being used, tell Kubernetes the strategy to follow when checking the health of our service instances. Let’s check these values in detail:

initialDelaySeconds

It is understood that services need time to become fully operational after they’ve been deployed. This value represents how long Kubernetes waits before it starts checking the health of a service instance.

timeoutSeconds

The maximum time we expect the health check to take.

periodSeconds

The time between two consecutive health checks (the default value is 10).

failureThreshold

The number of consecutive times a health check has to fail for Kubernetes to give up on the service instance and restart it (the default value is 3).

Configuring Amazon ECS to use health checks is similar. Let’s look at the following extract from the task definition for the Stock Manager service (located at /jenkins-aws-ecs/task-definitions/stockmanager-task.json):

"healthCheck": {
  "command": [ "CMD-SHELL", "curl -f http://localhost:8030/health || exit 1" ],
  "interval": 10,
  "timeout": 2,
  "retries": 3,
  "startPeriod": 30
}

Amazon ECS leverages the HEALTHCHECK command in Docker to verify whether the service instance is healthy (which is why the command parameter follows that particular syntax), but other than that, the parameters are analogous:

interval

The time between two consecutive health checks

timeout

The maximum time a health check is expected to take

retries

The number of consecutive times a health check has to file for ECS to consider the task unhealthy and restart it

startPeriod

The wait time after the task has been deployed before it starts performing health checks

What these examples show is that, regardless of the technology in place, creating and configuring health checks is an easy and powerful task. Every orchestrating platform will offer the ability to perform health checks in one form or another and follow similar parameters, which means that you should always be able to count on them.

Single target deployment

This is the simplest of the strategies and the one that requires the fewest resources. In this case, you assume the service has only one running instance, and whenever you need to update it, you will have to take it down and then deploy the new one. This implies that there will be a gap in the service, but no additional resources will be needed. The values that define this strategy therefore are as follows:

• desired: 1

• minimum: 0%

• maximum: 0%

This is represented in Figure 10-9 with the following steps:

• Beginning: One instance of the previous version exists.

• Step 1: The one instance has been substituted by another of the new version; the service is effectively unavailable while this new instances starts up.

• End: The new instance is now up and running, available for requests.

Figure 10-9. Single target deployment: the old instance is killed and replaced by a new one

All-at-once deployment

This strategy is similar to the single target deployment, with the only difference that, instead of having a single instance, you may have any fixed number of them. When a deployment is needed, all the current instances are taken down, and once they are down, all the new ones are brought up. As in the previous case, there is no additional need for resources during the upgrade, but there will also be a gap in the service. The parameters for this case are as follows:

• desired: n

• minimum: 0%

• maximum: 0%

This is represented in Figure 10-10 with the following steps:

• Beginning: Five instances of the previous version exist.

• Step 1: All five instances are taken down at the same time and substituted with five instances of the new version of the service; the service is effectively unavailable while the new instances start up.

• End: The new instances are now up and running.

Figure 10-10. All-at-once deployment: all old instances are killed at the same time, and new ones are brought in as the old ones are gone

Minimum in-service deployment

The two previous instances have one major inconvenience: they both imply a service gap. You can improve this by tweaking your strategy and ensuring that there is always a minimum number of healthy instances. This way, instead of bringing down all the old instances at once, you bring down only a number of them and create new instances when they are gone. Once the new instances are up and running, you can remove another batch of old instances, substituting them with new ones. You can repeat this process until all old instances have been replaced by new ones.

This process prevents a gap in the service without the need of additional resources, but it does mean that the minimum in-service instances must take an additional hit of traffic to compensate for the fact that there are fewer of them; make sure not to set this limit too low, or the remaining instances may not be able to cope with the load.

For the case represented in Figure 10-11, the parameters are as follows:

• desired: 5

• minimum: 40% (or 2, if expressed as absolute number)

• maximum: 0%

The process can be described as follows:

• Beginning: Five instances of the previous version exists.

• Step 1: Since at least two instances have to be always operational, only three are taken down and substituted by new instances; deployment remains in the step while the new instances start up.

• Step 2: Once the new instances are operational, the remaining two old versions can be taken down and substituted with new versions.

• End: All the new instances are now operational.

Figure 10-11. Minimum in-service deployment: at least two instances, either new ones or old ones, have to be operational at any given point

Rolling deployment

A rolling deployment can be seen as a different form of minimum in-service deployment: the focus isn’t placed on the minimum number of healthy instances, but the maximum number of absent instances. The most typical case of a rolling deployment sets this maximum at one, meaning only one instance can be in the process of being updated at any given time. This means that one instance will be brought down, and then a new one brought up; and only when the new one is operational will we continue the process with the next one. In some cases, a rolling deployment may set higher limits, allowing for two, three, or more instances to be in transition at any given time.

As a variant of the minimum in-service deployment, the rolling deployment presents pretty much the same characteristics: it prevents service gaps without the need for additional resources. The main advantage with regards to minimum in-service is that, by limiting the number of absent instances, you limit the extra strain that the remaining instances may have to endure; the main disadvantage is that deployments will take longer and, depending on the number of instances, the startup time, and the rate of redeployment, the platform could end up with a batch of queued redeployments.

For the case of a rolling deployment, the defining parameters are as follows:

• desired: 5

• minimum: 80% (or 4, if expressed as an absolute number)

• maximum: 0%

Note that the rolling deployment is equivalent to a minimum in-service deployment where minimum = desired - 1.

This process is shown step-by-step in Figure 10-12:

• Beginning: Five instances of the previous version exist.

• Step 1: One instance is taken down and substituted with a new one; deployment remains in the step while the new instance starts up.

• Step 2: After the new instance created in step 1 is operational, another old instance is taken down and substituted with a new one; deployment remains in the step while the new instance starts up.

• Steps 3, 4, and 5: The same process is repeated for the remaining old instances.

• End: All the new instances are now operational.

Figure 10-12. Rolling deployment, where the maximum number of absent instances is one.

Blue/green deployment

Blue/green deployment is one of the most popular strategies in the realm of microservices deployments, and perhaps for this reason is one whose meaning is not entirely set. The available literature seems to refer to two slightly different strategies under the same name, each solving a slightly different problem.

The first version of the blue/green deployment aims at fixing one of the disadvantages of both minimum in-service and rolling deployments: during the upgrade, there will be fewer healthy instances to share the total load, potentially causing a strain on them. To address this, blue/green deployments create new instances first, and only when these are available does it starts to take down the old ones. The number of healthy instances never goes below the desired total, but it does mean that we will need extra resources during the upgrade. Considering the standard parameters, a blue/green deployment described this way would have the following:

• desired: n

• minimum: 100%

• maximum: m% (where 0 < m ≤ 100)

Setting m to a higher value will exacerbate the peak use of resources, but will also shorten the length of the deployment.

The second version of the blue/green deployment goes beyond this, though, and cannot simply be obtained by a combination of the desired/minimum/maximum parameters. There is another disadvantage in the previous strategies: during the deployment, there will be a mix of old and new versions of the application in production. This might be OK in many scenarios, particularly if one is being careful with the release of functionality (see “Releasing Functionality”). However, when this mix is to be avoided, the second version of blue/green deployment adds a twist: no new instances will be available to the users until all of them are ready, and at that moment all old instances will be made immediately unavailable.

The way to achieve this is by manipulating the routing of requests, in addition to the orchestration of services; this is displayed graphically in Figure 10-13 via the following steps:

• Beginning: A number of instances of the previous version exists (two in this example). The load balancer/router, represented in the graph by a cylinder, is configured to send incoming requests to the old versions of the service.

• Step 1: New instances are created, aside from the old ones. These instances are not visible to the public, and the load balancer/router still sends all incoming traffic to the old instances. Deployment remains in this step while the new instances start up.

• Step 2: The new instances have finished starting up and are now operational, but no traffic is being directed to them yet.

• Step 3: The load balancer/router is reconfigured so all incoming traffic is directed to the new versions of the service. This switch is meant to be almost instantaneous. No new requests are sent to the old versions, although existing ones are allowed to finish.

• End: After the old instances are no longer useful, they are taken down.

As can be deduced, the second version of the blue/green deployment provides the best user experience, but at the cost of added complexity and high resource utilization.

Figure 10-13. A blue/green deployment: new instances are not visible by the user until all of them are ready; at that moment, the router changes to point to the new instances, making the old ones inaccessible to users

Canary deployment

Canary deployment is another case that cannot be achieved by tweaking the combination of desired/minimum/maximum parameters. The idea of the canary deployment is to try out a new version of the service without fully committing to it. This way, instead of completely replacing the old version of an application with the new one, you simply add an instance to the mix with the new version. The load balancer placed on top of the service instances can divert some traffic to the canary instance, and by inspecting logs, metrics, etc., you can understand how this behaves. If the result is satisfactory, you can then perform a full deployment of the new version. A canary deployment is performed in two steps, as shown in Figure 10-14:

• Beginning: Multiple instances of the previous version exist (four in this example).

• Step 1: An instance of the new version is created, without removing any of the old ones.

• End: The new instance is now up and running and can serve requests together with the old ones.

Canary deployments involve a number of challenges when it comes to orchestration. For instance, when the platform is checking the health status of the different instances, it needs to treat the canary one differently from the rest: if the canary instance is unhealthy, it needs to replace it with another canary instance, but if any of the other instances is unhealthy, then it needs to be replaced with a noncanary version. Also, canary instances sometimes need to run for relatively long periods to properly understand the effects of the change under study, during which time you are probably making new versions of the normal instance, triggering redeployments that need to update the normal instances while leaving the canary one intact.

Fortunately, the use case for canary deployments is rather slim. If you just want to expose a new functionality to a subset of users, you can do this by using feature flags (see “Feature Flags”). The added value of a canary deployment is testing out deeper changes that cannot be hidden via feature flags, like a change in the logging or metrics framework, an alteration of the garbage-collection parameters, or the trial of a newer version of the JVM.

Figure 10-14. Canary deployment, where a new instance is simply added to the group of current-version instances without replacing any

Which deployment type should I choose?

As has been shown, the different deployment strategies provide solutions to different problems at the cost of additional resources and/or complexity, meaning each of them will be better suited for different scenarios. Teams will have to analyze what their needs are and the investments they are willing to make. Table 10-1 provides a quick look at the different things that may need to be taken into account to aide in this decision.

A generic strategy

Regardless of your deployment strategy, a deployment will be composed of a combination of three types of action, each potentially executed more than once and not necessarily in this order:

• Deploying new instances

• Bringing down old instances

• Rerouting

For instance, the “Rolling deployment” strategy will imply a succession of the following:

1. Update routing to exclude one of the existing instances.

2. Bring down that instance.

3. Deploy the new instance.

4. Update routing to include the new instance.

5. Repeat for all other existing instances.

Similarly, the “Blue/green deployment” strategy will imply the following:

1. Deploy all the new instances.

2. Update routing to point to the new instances.

3. Bring down all the old instances.

By breaking down the deployment logic into these three types of action, you can focus on each of them separately, knowing that you just need to combine them in different ways to achieve the different deployment strategies. Let’s now cover the three actions in detail.

Managing database deployments

Changing a database needs to be seen under the same light as changing an application: like an automated and tested process within a CI/CD strategy. This implies that things like a schema modification or a data migration cannot be done manually or in isolation, but rather must be done as part of a CVS-registered change that triggers a build in our CI/CD pipeline. Moreover, you need to recognize that, as the needs of the business evolve, so does the optimum data schema. Therefore, you need to be comfortable with the idea of refactoring databases, just as you refactor application code.

Multiple tools can help with this, and DBDeploy, probably the first tool designed to ease database changes as part of a CI/CD pipeline, is worth highlighting. Although DBDeploy still does its job, it’s fair to say that development has been somewhat abandoned (the last commit in the DBDeploy repository as of the time of writing this book dates to 2011), so new adopters should probably look at more recent alternatives like Flyway or Liquibase. In either case, all these tools work in a similar way:

• Changes to the database, either structure or data, are performed via migration scripts. These scripts are standard SQL files.

• Each migration script must have a unique name and sequence number.

• The migration tool keeps a table for itself in which it keeps track of which migration scripts have been run in the database and which ones haven’t.

• Whenever the migration tool is invoked, it will scan the totality of available migration scripts, compare it with the ones that have already been run against a particular database, identify the ones that haven’t been run, and then run those.

This process allows you to keep track of changes across multiple environments, so you can try out a database schema change in test without affecting production. It also gives you a history of changes, and lets you jump to any particular version of the database at any point in time: you just need a blank database and then run the migration scripts up to your desired point.

Undoing or amending changes is a bit tricky, though. It may be tempting to think that, if you identify a bug after testing a particular migration script in the test environment, you can just amend that particular migration script and rerun it. The truth is that the migration tools don’t usually understand changes in a migration script, they consider them only as either “already run” or “not yet run”; if a particular script has already been run against a database (for instance, staging), then an amendment to the script won’t trigger a rerun: the tool considers it has already processed it, and therefore skips it. In these situations, the best option is to wipe out the database every time so that all the scripts are always run (definitely not a good idea for production), or simply add further migration scripts that undo or amend the previous ones.

Separating database and application deployments

One of the requirements for a continuous delivery process is minimizing the impact of potential failures so as to allow for continuous changes. This is why you should always try to avoid multi-application deployments: each application should be deployable independently of others. The same is true for database and application deployments.

Whenever you prepare a migration script, you should make it so it can be deployed without breaking any compatibility with any running applications, for the following reasons:

• If multiple applications use your database, there is no guarantee that all of them will be deployed at the same time (an individual deployment may fail at any time); if this were to happen, the applications that haven’t been upgraded to the next version will fail to communicate with the database from now on.

• Even if only one application uses your database, you could have a mix of old and new instances of your application, depending on your deployment strategy (e.g., rolling deployment); the new instances will work while the old ones won’t, defeating the purpose of a deployment with zero downtime.

• Finally, even in the extreme case where only one application uses your database and you don’t use a deployment strategy that mixes old and new instances (e.g., all-at-once deployment), there is no guarantee that database and application will be deployed at exactly the same time, providing a window of failure.

Regardless of your case, you need to acknowledge that your database is an independent component that has its own deployment cycle, and respect the relationship with connecting applications. For changes that can potentially break compatibility, take a look at “Multiple-Phase Upgrades”.

A different matter is whether the migration scripts need to be on their own repository. This is a debating point, and different individuals may lean toward different solutions. In general, if a database is used by multiple applications, and each of these applications has its own repository, then it is advisable that the migration scripts have their own space, too. If, on the other hand, the database is used by only a single application, or it is used by a set of applications that are all managed together in the same repository (a monorepo), then it is OK to include the migration scripts in the same repository.

Communicating via stored procedures: turning the database into just another service

Stored procedures have somehow grown out of fashion over the last few years. Many developers today tend to associate them with bureaucratic organizations in which everything that is database-related is managed by DBAs, and developers are allowed to access data only via a set of rigidly designed stored procedures. This has the unfortunate effect of blaming the messenger: if appropriately managed, stored procedures can be a great ally to provide a faster development environment, and can even let you look at your databases in a new, imaginative way.

The main reason a microservices architecture works is that it provides the right balance between exposure and encapsulation. Each service will hide its internal workings to others, and will allow communication only via a number of known endpoints. In the same way, you can look at your database as just another microservice where the stored procedures are the known endpoints, and where communication is performed via SQL as opposed to HTTP. Knowing that connecting applications access the database only via stored procedures, and assuming that you keep the behavior of the stored procedures constant, you can perform as many refactorings to the internals of the database as you want without affecting any applications.

This, of course, requires that your stored procedures are appropriately tested, just as you test the endpoints of your services. Testing databases is out of the scope of this book, but from a Java point of view, the best tools to consider are DbUnit and Unitils.

Avoid versioning

The first approach to manage versioning in an API is simply to avoid it: if your team needs to make a particular change to the API, make it in a way that keeps backward compatibility. In practice, this means keeping the endpoint URL as it is, and changing the structure of the returned object only to add new fields. Existing clients can continue to use the API, oblivious to the change, while clients who need the new feature have it there available for them.

An example of this approach can be found in the Extended Java Shop—more precisely, in the Feature Flags service. The story goes as follows. The Feature Flags service was initially designed as a throttle flag with three parameters: the flag ID, the flag name, and the rate of requests that should be granted access to the feature (the “portion in”). At some point, the business realized there was a downside to the current way feature flags were managed: since each request was independent from the others, it could happen that the same user was granted access to the feature in one request and rejected in another one, potentially providing an inconsistent experience. Depending on the feature, this might be an acceptable situation, or it may not. To address this, a new functionality was to be added to the Feature Flags service: flags should include a “sticky” parameter, which indicated whether the behavior across requests would be the same for a given user or whether it was allowed to change.

The implementation of the sticky parameter has been done in a backward-compatible way to avoid the need for a new version: a new field has been added to the response. Existing consumers of feature flags, like the shopfront service, can simply ignore this new field until they are ready and/or willing to make use of it.

A similar approach can be used when you need to modify an existing field: instead of modifying it, you can choose to add a new one with the new meaning. This, of course, can work for only so long, and eventually your API will be littered with a mixed bag of old and new fields. If you get to this point, or if you need to make changes that cannot be implemented by simply adding new fields, then you need to create a new version of the API as indicated next.

Version the endpoint

A simple and effective way to create a new version for a backward-incompatible change in the API is to include the version number as part of the endpoint itself. This way, if the current version of the resources is located under /resource, the new version can be located at /v2/resource. This is a common approach, used in well-known services like AWS, and it’s one that is simple to implement and communicate. You can easily switch from one version to the other by quickly editing the URL. You can give someone a link to easily try out the new version. The pragmatism under this approach is its main advantage.

The Extended Java Shop includes a case of versioning through endpoints in the Product Catalogue service. Let’s say that, in our example, the business has decided to have two different prices for the products: the single price, when the item is purchased in small amounts, and the bulk price, with an implicit discount for large purchases. The Product Catalogue would then also indicate, for each product, the number of units that need to be purchased at the same time to qualify for the bulk price. Developers decide that it would be too messy to implement this new feature simply by adding new fields to the response, and decide to create a new version of the API. Version 1 of the Product Catalogue returns a Product object like this:

GET /stocks/1

200 OK
{
  "id": "1",
  "name": "Widget",
  "description": "Premium ACME Widgets",
  "price": 1.20
}

Version 2 returns a modified object with extra information:

GET /v2/stocks/1

200 OK
{
  "id": "1",
  "name": "Widget",
  "description": "Premium ACME Widgets",
  "price": {
    "single": {
      "value": 1.20
    },
    "bulkPrice": {
      "unit": {
        "value": 1.00
      },
      "min": 5
    }
  }
}

You can see how this has been implemented by looking at the two versions of the ProductResource class in the Product Catalogue service.

This is a backward-incompatible change that has been implemented using the version-endpoint pattern. This way, requests to /products would return the first version, while requests to /v2/products would return the second version. Clients using the first version of the API can continue to operate as normal, but those who want to use the new features can make the necessary arrangements to use the second version. Note that, in our sample application, the Shopfront service is still using version 1 of the API.

Version the content

Detractors of the version-in-endpoint approach usually point out how it breaks the semantics of the RESTful principles: in a pure RESTful API, an endpoint is meant to represent a resource, not a version of a resource, so version numbers should not be included in the URL. Instead of versioning the endpoint, you can version the content by means of the Content-Type header.

Let’s assume your service provides a response in JSON format. The value of the Content-Type header in this situation will most typically be application/json. You can, however, provide a versioned content type using the pattern application/vnd.<resource-name>.<version>+json, where <resource-name> is the name of the resource that this type refers to, and <version> is the version of the resource format. Clients can then indicate the version that they want to be provided by using the Accept header. This way, the same endpoint can serve different versions of the same resource.

An example of this can be found in the Stock Manager service of the Extended Java Shop. In this case, we can say that the business realized that some of the pieces on sale were particularly heavy, and they wanted to impose a limit on the number of them that a customer could buy in a single purchase to ease the packaging and delivery. (Whether it is credible that any business would willingly limit the number of products that they sell is something that we will not debate here—just roll with it.) For this, the development team decided that stocks should include both the total number of available units, and the number of units that could be purchased at the same time. Once again, the developers decided that it would be better to do a new version of the API than try to add the changes to the existing one in a backward-compatible manner, and decided to implement this change by using the version-in-content approach. This way, the old API still worked in the following way:

Accept: application/json
GET /stocks/1

200 OK
Content-Type: application/json
{
  "productId": "1",
  "sku": "12345678",
  "amountAvailable": 5
}

The new API is available by changing the header:

Accept: application/vnd.stock.v2+json
GET /stocks/1

200 OK
Content-Type: application/vnd.stock.v2+json
{
    "productId": "1",
    "sku": "12345678",
    "amountAvailable": {
        "total": 5,
        "perPurchase": 2
    }
}

Details of this implementation can be found in the StockResource class in the Stock Manager service. Note that, in our example, the Shopfront service is still using version 1 of the API.

Advanced change management

On the far end of the spectrum lies the argument that the mere fact that a RESTful API needs to be versioned is an antipattern, since it doesn’t strictly follow the rules of hypermedia communication. The URIs of our resources should be immutable, and the provided content should use a language that is parsable upon a definite set of rules; this way, changes in the content can just be reinterpreted by the consumer, and no coordination between provider and consumer is needed.

Although strictly true, most people find this approach harder than it’s worth, and revert back to one of the approaches outlined previously. For this reason, we have decided not to cover it in this book, although readers who want to investigate further are encouraged to check Roy Fielding’s work.

Deprecating old APIs

Keeping every historical version of an API would be a maintenance nightmare. That’s why, even though you want to make it easier for consumers to adopt new APIs at their own pace, you also want to make sure they do move on.

You can track the number of people using each version of your APIs (if at all) by keeping usage metrics of each of your versioned interfaces (see Chapter 13 for information about metrics). Once you are confident that nobody is using the old versions, either because you know or control all the potential consumers, or because you can see in the metrics reports that there is no usage, you can confidently delete the old versions.

Sometimes you won’t feel in a position to remove the old endpoints straightaway, either because you know there is some usage but you can’t track the owner, or because yours is a public API and you can’t assert with confidence that nobody is using the old API anymore. If this is the case, you might be able to nudge the slow movers by keeping the old interface but removing the implementation: requests to the old API can be replied to with an HTTP redirect instruction:

GET /v1/resource

301 MOVED PERMANENTLY
Location: /v2/resource
Content-Type: text/plain
This version of the API is no longer supported, please use /v2/resource

Chances are, the consumer will still be broken by this, since the request will be redirected to the new version, for which the consumer is probably not ready. However, at least they will be notified of what they need to do to fix the situation.