preface

I first tried building applications with microservices around 2013. That was the year Docker was initially released, but back then, I hadn’t heard about it. At that time, we built an application with each microservice running on a separate virtual machine. As you might expect, that was a really expensive way to run microservices.

Because of the high running costs, we then opted to create fewer rather than more microservices, pushing more and more functionality into the existing microservices to the point where we couldn’t really call these microservices. It was still a distributed application of course, just not micro-sized in the way we had hoped.

I already knew at that stage that microservices were a powerful idea, if only they were cheaper. I put microservices back on the shelf, but made a note that I should look at those again later.

Over the years, I watched from the sideline as the tools and technology around microservices developed, powered by the rise (and rise) of open source coding. And I looked on as the cost of cloud computing continued to drop, spurred on by competition between vendors. Over time, it was clear that building and running a distributed application with micro-sized components was becoming more cost effective.

After what seemed like a lifetime, in early 2018, I officially returned to the world of microservices. I had two opportunities for which I believed microservices were the right fit. Both were startups. The first was a contract job to bootstrap a new microservices application for a promising young company. The second was building a microservices application for my own startup.

To be successful, I knew that I needed new tools. I needed an effective way to package microservices. I needed a computing platform on which I could deploy microservices. Crucially, I needed to be able to automate deployments.

By then, Docker had already gained a big foothold in our industry, so I knew it was a safe bet as a way to package microservices. I also liked the look of Kubernetes as a computing platform for microservices, but early on, I was extremely uncertain about it. Kubernetes, however, promised a future of freedom from the tyranny of cloud vendor lockin-that was very appealing.

At this point, I’d read quite a few books on microservices. These were all interesting, providing good value on a theoretical level. I do enjoy reading the theory, but these books lacked the practical examples that would have helped me smash through my own learning curve. Even as an experienced developer, I was struggling to know where to start! I knew from past experience that bad technical decisions made at the beginning of a project would haunt me to the end.

Learning Kubernetes was especially difficult. From the outside, it seemed incredibly difficult to penetrate. But I had a job to do, and I needed a way to deliver software. So I pushed on. The going was tough, and I almost gave up on Kubernetes a few times.

The situation changed when I discovered Terraform. This was the missing piece of the puzzle for me. It’s what made Kubernetes understandable and usable to the point where I could do nothing else but commit to using it.

Terraform is the tool that allowed me to describe the structure of my application. Terraform could then live in my continuous delivery (CD) pipeline and automatically keep my application up to date! I began writing infrastructure as code, and it felt like I had moved to the big league.

I forced my way through the learning curve, bolstered by my long-time experience of evaluating technology and learning quickly on the job, with a splash of trial and error mixed in for good measure. My efforts delivered software that is performant, flexible, reliable, scalable, extensible, and still running to this day. Through this time, my desire to write this book sparked and grew to the point where I had to take action.

A new mission formed-I wanted to make microservices more accessible. I felt compelled to write this book; it’s the book I wanted but didn’t have. I knew I could help people, and the best way to do that was with a practical book, this book. A book that shows you, step by step, that microservices don’t have to be difficult or complex; it all depends on your approach and the perspective you take. You now have in your hands the fruits of that labor. I learned the hard way so that you don’t have to.

acknowledgments

In Bootstrapping Microservices, I share my years of hard-won experience with you. Such experience wouldn’t be possible without being surrounded by people who supported and encouraged me.

There are many who helped me get to where I am today. I wouldn’t be a developer without my parents, Garry and Jan, who bought me my first PC. My partner in life, Antonella, who has tirelessly supported me through two books now. My partner in business, Majella, who listens to all my rants about technology, and still pushes me forward. Thank you all!

Of course, thank you to Manning for the opportunity and especially to Helen Stergius, who once again edited my book. Hopefully, I made your job easier this time, now that I’m a more experienced author. Thanks as well to the entire team at Manning for their efforts.

A big thank you to the technical proof reader, Alain Couniot, and all the reviewers who have played such a huge part in taking this book to the next level: Angelo Simone Scotto, Anupam Sengupta, Barnaby Norman, Björn Neuhaus, Bonnie Malec, Chris Kolosiwsky, Chris Viner, Dan Sheikh, Dhruvesh Patel, Donald McLamb, Eric Platon, Ernesto Bossi Carranza, Giampiero Granatella, John Guthrie, Julien Pohie, Marcin Sęk, Michele Adduci, Miguel Montalvo, Rich Ward, Rinor Maloku, Ruben Vandeginste, and Weyert de Boer.

Of course, Jeanne Boyarsky, the technical editor, deserves a special thank you. She did a wonderful job “nitpicking” (as she put it), and the book is so much better for her involvement.

Finally, I’d like to thank the development community. Your feedback and encouragement made this book a joy to write. I wrote this book for you!

about this book

Building applications with microservices-building distributed applications-can be a complicated process and can be difficult to learn. If you are plunged into a modern complex application, it can be difficult to see the trees from the forest. There’s so much more to consider than simply coding. And this is not an easy journey to take on your own.

To use microservices, you must understand how to build a distributed application. But by itself, that’s not enough. You also have to learn the deep and complex tools that are necessary to develop, test, and deploy such an application. How do we assemble a robust toolkit for development? Where do we start?

Along the way are many more questions. How do we package and deploy a microservice? How do we configure our development environment for local testing? How do we get our microservices communicating with each other, and how do we manage the data? Most importantly, how do we deploy our microservices to production? Then, once in production, how do we manage, monitor, and fix problems with potentially hundreds of microservices?

This book, Bootstrapping Microservices, answers these questions and more! It’s your guide to building an application with microservices using the latest tools. We’ll start from nothing and go all the way to a working microservices application running in production.

You won’t find much theory in this book. Bootstrapping Microservices is practical and project-based. Together, we’ll work through numerous examples of microservices, eventually getting to production, and covering everything you need to know to be a confident microservices developer.

Each example in this book comes with working code that is available on GitHub. You can try it out for yourself and make your own experimental changes.

Who should read this book

This book is aimed at anyone who wants to learn more about the practical aspects of working with microservices; those who need a clear guide on how to assemble their toolkit and take their application all the way to production. This book doesn’t teach coding, so basic coding skills are advisable.

Note If you have some basic or entry-level experience with modern programming languages like C#, Java, Python, or JavaScript, you should be able to follow along with this book.

The code examples are as simple as they can be, but this book isn’t about the code. It’s more about teaching you how to assemble the toolkit you need for building a microservices application.

If you don’t have coding experience, but you are a fast learner, you can learn basic JavaScript (through another book, tutorials, videos, and so forth) while you read Bootstrapping Microservices. Like I said, the code examples are as simple as these can be, so you stand a good chance of being able to read the code and get the gist of it without much coding experience. Our coding adventure starts in chapter 2, where you learn how to build a simple microservice using JavaScript and Node.js.

How this book is organized: A roadmap

In the 11 chapters of this book, we go from building a single microservice all the way to running multiple microservices in a production-ready Kubernetes cluster.

• Chapter 1 is an introduction to microservices and explains why we want to use these.

• Chapter 2 works through building a simple microservice using Node.js and JavaScript. We learn how to use live reload for a more streamlined development process.

• Chapter 3 introduces Docker for packaging and publishing our microservice to get it ready for deployment.

• Chapter 4 scales up to multiple microservices and introduces Docker Compose for simulating our microservices application on your development workstation during development. We then cover data management for microservices, including having a database and external file storage.

• Chapter 5 upgrades our development environment for whole application live reload. We then cover communications among microservices, including HTTP for direct messaging and RabbitMQ for indirect messaging.

• Chapter 6 introduces Terraform and Kubernetes. We use Terraform to create a private container register and a Kubernetes cluster on Microsoft Azure.

• Chapter 7 uses Terraform to deploy microservices to our Kubernetes cluster. We deploy a database, a RabbitMQ server, and finally, a microservice. We also look at how to create a continuous delivery (CD) pipeline that automates the deployment of our application to production.

• Chapter 8 shows how we can apply multiple levels of automated testing to microservices.

• Chapter 9 is an overview of the example application and a review of the skills you learned thus far in the context of deploying the example application for yourself.

• Chapter 10 explores the ways that we can build reliable and fault-tolerant microservices and then monitor those to maintain a healthy application.

• Chapter 11 wraps up by showing practical ways that your microservices application can be scaled to support your growing business and organized to manage your growing development team. It also touches on security, refactoring a monolith, and how to build with microservices on a budget.

About the code

This book contains many examples of source code both in numbered listings and in line with normal text. In both cases, source code is formatted in a fixed-width font like this to separate it from ordinary text.

In many cases, the original source code has been reformatted; we’ve added line breaks and reworked indentation to accommodate the available page space in the book. In rare cases, even this was not enough, and listings include line-continuation markers (➥). Additionally, comments in the source code have often been removed from the listings when the code is described in the text. Code annotations accompany many of the listings, highlighting important concepts.

The code for the examples in this book is available for download from the Manning website at https://www.manning.com/books/bootstrapping-microservices-with-docker-kubernetes-and-terraform and from GitHub at https://github.com/bootstrapping-microservices.

You can download a zip file to accompany each chapter (chapters 2-9), or you can use Git to clone the Git code repository for each chapter. Each example is designed to be as simple as possible, self-contained, and easy to run. As you progress through the book, you will run the code in different ways.

We start by running code for a single microservice directly under Node.js (chapter 2), then under Docker (chapter 3). We then run multiple microservices under Docker Compose (chapters 4 and 5).

Next, we run code under Terraform, first locally (chapter 6) and then within our continuous delivery pipeline (chapter 7). Also, in chapters 6 and 7, we’ll run our microservices on a Kubernetes cluster in the cloud. In chapter 8, we come back to Node.js to run automated tests under Jest and Cypress. And, finally, in chapter 9, we revise the skills learned thus far (from chapters 2-8). The example code for chapter 9 is a simple, yet complete, microservices application that you can get running for yourself both in development and production.

Throughout the code examples, I aim to follow standard conventions and best practices. I ask that you provide feedback and report any issues through GitHub.

liveBook discussion forum

Purchase of Bootstrapping Microservices with Docker, Kubernetes, and Terraform includes free access to a private web forum run by Manning Publications where you can make comments about the book, ask technical questions, and receive help from the author and from other users. To access the forum, go to https://livebook.manning.com/#!/book/bootstrapping-microservices-with-docker-kubernetes-and-terraform/discussion. You can also learn more about Manning’s forums and the rules of conduct at https://livebook.manning.com/#!/discussion.

Manning’s commitment to our readers is to provide a venue where a meaningful dialogue between individual readers and between readers and the author can take place. It is not a commitment to any specific amount of participation on the part of the author, whose contribution to the forum remains voluntary (and unpaid). We suggest you try asking the author some challenging questions lest his interest stray! The forum and the archives of previous discussions will be accessible from the publisher’s website as long as the book is in print.

about the author

Ashley Davis is a software craftsman, entrepreneur, and author with over 20 years of experience in software development, from coding to managing teams, then to founding companies. He has worked for a range of companies, from the tiniest startups to the largest internationals. Along the way, he has contributed back to the community through his writing and open source coding.

Ashley is the CTO of Sortal, a product that automatically sorts digital assets through the magic of machine learning. He is the creator of Data-Forge Notebook, a notebook-style desktop application for exploratory coding and data visualization using JavaScript and TypeScript. Ashley is also a keen algorithmic trader, actively trading and developing quantitative trading software.

For updates on his book, his open source coding, and more, follow Ashley on Twitter @ashleydavis75, follow him on Facebook at The Data Wrangler, or visit his blog at http://www.the-data-wrangler.com.

For more on Ashley’s background, see his personal web page (http://www.codecapers .com.au) or his Linkedin profile (https://www.linkedin.com/in/ashleydavis75).

about the cover illustration

The figure on the cover of Bootstrapping Microservices with Docker, Kubernetes, and Terraform is captioned “Catalan” or a man from Catalonia, in northeast Spain. The illustration is taken from a collection of dress costumes from various countries by Jacques Grasset de Saint-Sauveur (1757-1810), titled Costumes civils actuels de tous les peoples connus, published in France in 1788. Each illustration is finely drawn and colored by hand. The rich variety of Grasset de Saint-Sauveur’s collection reminds us vividly of how culturally apart the world’s towns and regions were just 200 years ago. Isolated from each other, people spoke different dialects and languages. In the streets or in the countryside, it was easy to identify where they lived and what their trade or station in life was just by their dress.

The way we dress has changed since then and the diversity by region, so rich at the time, has faded away. It is now hard to tell apart the inhabitants of different continents, let alone different towns, regions, or countries. Perhaps we have traded cultural diversity for a more varied personal life-certainly for a more varied and fast-paced technological life.

At a time when it is hard to tell one computer book from another, Manning celebrates the inventiveness and initiative of the computer business with book covers based on the rich diversity of regional life of two centuries ago, brought back to life by Grasset de Saint-Sauveur’s pictures.