Immutable Delivery

Immutable delivery concepts help us reason about these problems. With immutable delivery, we try to reduce the number of moving pieces into prebaked images as part of the build process. For example, imagine in your build process you could output a fully baked image with the operating system, the intended version of the JVM, any sidecar applications, and all configuration? You could then deploy this in one environment, test it, and migrate it along a delivery pipeline toward production without worrying about “whether the environment or application is configured consistently.” If you needed to make a change to your application, you rerun this pipeline, which produces a new immutable image of your application, and then do a rolling upgrade to deliver it. If it doesn’t work, you can rollback by deploying the previous image. No more worrying about configuration or environment drift or whether things were properly restored on a rollback.

This sounds great, but how do we do this? Executable JARs is one facet to get us part of the way there, but still falls short. The JVM is an implementation detail of our microservice, so how do we bundle the JVM? JVMs are written in native code and have native OS-level dependencies that we’ll need to also package. We will also need configuration, environment variables, permissions, file directories, and other things that must be packaged. All of these details cannot be captured within a single executable JAR. Other binary formats like virtual machine (VM) images can properly encapsulate these details. However, for each microservice that may have different packaging requirements (JVM? NodeJS? Golang? properties files? private keys?), we could easily see an explosion of VM images and combinations of language runtimes. If you automate this with infrastructure as code and have access to infrastructure as a service with properly exposed APIs, you can certainly accomplish this. In fact, building up VMs as part of an automated delivery pipeline is exactly what Netflix did to achieve this level of immutable delivery. But VMs become hard to manage, patch, and change. Each VM virtualizes an entire machine with required device drivers, operating systems, and management tooling.

What other lightweight packaging and image formats can we explore?

Docker, Docker, Docker

Docker came along a few years ago with an elegant solution to immutable delivery. Docker allows us to package our applications with all of the dependencies it needs (OS, JVM, other application dependencies, etc.) in a lightweight, layered, image format. Additionally, Docker uses these images to run instances which run our applications inside Linux containers with isolated CPU, memory, network, and disk usage. In a way, these containers are a form of application virtualization or process virtualization. They allow a process to execute thinking it’s the only thing running (e.g., list processes with ps and you see only your application’s process there), and that it has full access to the CPUs, memory, disk, network, and other resources, when in reality, it doesn’t. It can only use resources it’s allocated. For example, I can start a Docker container with a slice of CPU, a segment of memory, and limits on how much network IO can be used. From outside the Linux container, on the host, the application just looks like another process. No virtualization of device drivers, operating systems, or network stacks, and special hypervisors. It’s just a process. This fact also means we can get even more applications running on a single set of hardware for higher density without the overhead of additional operating systems and other pieces of a VM which would be required to achieve similar isolation qualities.

What’s happening under the covers is nothing revolutionary either. Features called cgroups, namespaces, and chroot, which have been built into the Linux kernel (and have for some time), are used to create the appearance of this application virtualization. Linux containers have been around for over 10 years, and process virtualization existed in Solaris and FreeBSD even before that. Traditionally, using these underlying Linux primitives, or even higher-level abstractions like lxc, have been complicated at best. Docker came along and simplified the API and user experience around Linux containers. Docker brought a single client CLI that can easily spin up these Linux containers based on the Docker image format, which has now been opened to the larger community in the Open Container Initiative (OCI). This ease of use and image format is changing the way we package and deliver software.

Once you have an image, spinning up many of these Linux containers becomes trivial. The layers are built as deltas between a base image (e.g., RHEL, Debian, or some other Linux operating system) and the application files. Distributing new applications just distributes the new layers on top of existing base layers. This makes distributing images much easier than shuttling around bloated cloud machine images. Also, if a vulnerability (e.g., shell shock, heartbleed, etc.) is found in the base image, the base images can be rebuilt without having to try patch each and every VM. This makes it easy to run a container anywhere: they can then be moved from a developer’s desktop to dev, QA, or production in a portable way without having to manually hope that all of the correct dependencies are in the right place (does this application use JVM 1.6, 1.7, 1.8?). If we need to redeploy with a change (new app) or fix a base image, doing so just changes the layers in the image that require changes.

When we have standard APIs and open formats, we can build tooling that doesn’t have to know or care what’s running in the container. How do we start an application? How do we stop? How do we do health checking? How do we aggregate logs, metrics, and insight? We can build or leverage tooling that does these things in a technology-agnostic way. Powerful clustering mechanics like service discovery, load balancing, fault tolerance, and configuration also can be pushed to lower layers in the application stack so that application developers don’t have to try and hand cobble this together and complicate their application code.

Kubernetes

Google is known for running Linux containers at scale. In fact, “everything” running at Google runs in Linux containers and is managed by their Borg cluster management platform. Former Google engineer Joe Beda said the company starts over two billion containers per week. Google even had a hand in creating the underlying Linux technology that makes containers possible. In 2006 they started working on “process containers,” which eventually became cgroups and was merged into the Linux kernel code base and released in 2008. With its breadth and background of operating containers at scale, it’s not a surprise Google has had such an influence on platforms built around containers. For example, some popular container management projects that preceded Kubernetes were influenced by Google:

• The original Cloud Foundry creators (Derek Collison and Vadim Spivak) worked at Google and spent several years using Google’s Borg cluster management solution.

• Apache Mesos was created for a PhD thesis, and its creator (Ben Hindman) interned at Google and had many conversations with Google engineers around container clustering, scheduling, and management.

• Kubernetes, an open source container cluster management platform and community, was originally created by the same engineers who built Borg at Google.

Back in 2013 when Docker rocked the technology industry, Google decided it was time to open source their next-generation successor to Borg, which they named Kubernetes. Today, Kubernetes is a large, open, and rapidly growing community with contributions from Google, Red Hat, CoreOS, and many others (including lots of independent individuals!). Kubernetes brings a lot of functionality for running clusters of microservices inside Linux containers at scale. Google has packaged over a decade of experience into Kubernetes, so being able to leverage this knowledge and functionality for our own microservices deployments is game changing. The web-scale companies have been doing this for years, and a lot of them (Netflix, Amazon, etc.) had to hand build a lot of the primitives that Kubernetes now has baked in. Kubernetes has a handful of simple primitives that you should understand before we dig into examples. In this chapter, we’ll introduce you to these concepts; and in the following chapter, we’ll make use of them for managing a cluster of microservices.

Getting Started with Kubernetes

Docker and Kubernetes are both Linux-native technologies; therefore, they must run in a Linux host operating system. We assume most Java developers work with either a Windows or Mac developer machine, so in order for us to take advantage of the great features Docker and Kubernetes bring, we’ll need to use a guest Linux VM on our host operating system. You could download Docker machine and toolbox for your environment but then you’d need to go about manually installing Kubernetes (which can be a little tricky). You could use the upstream Kubernetes vagrant images, but like any fast-moving, open source project, those can change swiftly and be unstable at times. Additionally, to take full advantage of Docker’s portability, it’s best to use at least the same kernel of Linux between environments but optimally the same Linux distribution and version. What other options do we have?

Microservices and Linux Containers

To get started developing microservices with Docker and Kubernetes, we’re going to leverage a set of developer tools called the Red Hat Container Development Kit (CDK). The CDK is free and is a small, self-contained VM that runs on a developer’s machine that contains Docker, Kubernetes, and a web console (actually, it’s Red Hat OpenShift, which is basically an enterprise-ready version of Kubernetes with other developer self-service and application lifecycle management features; but for this book we’ll just be using the Kubernetes APIs).

$ vagrant up

$ eval "$(vagrant service-manager env docker)"

$ vagrant service-manager env docker
export DOCKER_HOST=tcp://192.168.121.195:2376
export DOCKER_CERT_PATH=/home/john/cdk/components/rhel/rhel-ose/
.vagrant/machines/default/libvirt/.docker
export DOCKER_TLS_VERIFY=1
export DOCKER_MACHINE_NAME=081d3cd

$ oc login 10.1.2.2:8443
The server uses a certificate signed by an unknown authority.
You can bypass the certificate check, but any data you send to
the server could be intercepted by others.
Use insecure connections? (y/n): y

Authentication required for https://10.1.2.2:8443 (openshift)
Username: admin
Password:
Login successful.

You have access to the following projects and can switch between
them with 'oc project <projectname>':

  * default

Using project "default".
Welcome! See 'oc help' to get started.

$ oc new-project microservice-book
Now using project "microservice-book" on server
"https://10.1.2.2:8443".

$ docker ps
$ docker images

Where to Look Next

In this chapter, we learned a little about the pains of deploying and managing microservices at scale and how Linux containers can help. We can leverage immutable delivery to reduce configuration drift, enable repeatable deployments, and help us reason about our applications regardless of whether they’re running. We can use Linux containers to enable service isolation, rapid delivery, and portability. We can leverage scalable container management systems like Kubernetes and take advantage of a lot of distributed-system features like service discovery, failover, health-checking (and more!) that are built in. You don’t need complicated port swizzling or complex service discovery systems when deploying on Kubernetes because these are problems that have been solved within the infrastructure itself. To learn more, please review the following links:

• Kubernetes Reference Documentation

• “An Introduction to Immutable Infrastructure” by Josha Stella

• “The Decline of Java Application Servers when Using Docker Containers” by James Strachan

• Docker docs

• OpenShift Enterprise 3.1 Documentation

• Kubernetes

• Kubernetes Reference Documentation: Pods

• Kubernetes Reference Documentation: Labels and Selectors

• Kubernetes Reference Documentation: Replication Controller

• Kubernetes Reference Documentation: Services