This chapter covers

• The learning approach of this book
• The what and why of microservices
• The benefits and drawbacks of using microservices
• What’s wrong with the monolith?
• The basics of microservices design
• A quick overview of the application we build

As software continues to become larger and more complicated, we need better ways of managing and mitigating its complexity. As it grows alongside our business, we need better ways of dividing it up so that multiple teams can participate in the construction effort.

As our demanding customer base grows, we must also be able to expand our software. At the same time, our applications should be fault-tolerant and able to scale quickly to meet peak demand. How do we then meet the demands of modern business while evolving and developing our application?

Microservices are an architectural pattern that plays a pivotal role in contemporary software development. A distributed application composed of microservices solves these problems and more, but typically it is more difficult, more complex, and more time-consuming to architect than a traditional monolithic application. If these terms are new-microservices, distributed application, and monolithic application-they will be explained soon.

Conventional wisdom says that microservices are too difficult. We are told to start “monolith-first” and later restructure to microservices when necessary to scale. But I argue that this attitude doesn’t make the job of building an application any easier! Your application is always going to tend toward complexity and, eventually, you will need to scale it. When you do decide you need to change, you now have the extremely difficult job of safely converting your monolith to microservices when staff and customers already depend on it.

Now is also the perfect time to be building microservices. The confluence of various factors-accessible and cheap cloud infrastructure, ever improving tools, and increasing opportunities for automation-is driving an industry-wide movement toward smaller and smaller services, aka microservices. Applications become more complex over time, but microservices offer us better ways to manage such complexity. There is no better time than now to go “microservices-first.”

In this book, I will show you that a microservices-first approach is no longer as daunting as it once was. I believe the balance is firmly tipping toward microservices. The remaining problem is that learning microservices is difficult. The learning curve is steep and holds back many developers in their quest to build microservices. Together, we will break the learning curve. We will say “Boo” to the monolith, and we’ll build from the ground up a simple but complete video-streaming application using microservices.

1.1 This book is practical

Why are you reading this book? You are reading this because you want or need to build a microservices application, which is an important skill set for modern developers, but it’s a difficult skill set to obtain, and you need some guidance. You may have read other books on microservices and been left wondering where do I begin? I understand your torment.

Microservices are tough to learn. Not only do you have to learn deep and complicated tools, you must also learn to build a distributed application. This requires new design patterns, protocols, and methods of communication. That’s a lot to learn in anyone’s book.

In this book, we cut through the seemingly impenetrable learning curve of building microservices applications. The learning curve you must endure can seem insurmountable when tackled by yourself, but, rather than that, we’ll undergo this development adventure together. We’ll start as simple as possible and, piece-by-piece, we’ll build up to deploying our application to production.

This book is about busting through the learning curve and bootstrapping a working application that will last indefinitely, an application that we can continuously update and build on to satisfy the ongoing and changing needs of our customers and users. Figure 1.1 illustrates this idea of cutting through the learning curve. While our example application is small and simple, from the start, we will build-in pathways to scalability that will later allow it to be expanded out to a truly massive distributed application.

Figure 1.1 Cutting through the learning curve. In this book, we’ll learn only the bare minimum, just enough to bootstrap our application.

How is this book different from all the other books on microservices? Other books are notably theoretical. That’s a good approach for an experienced developer or architect looking to broaden their knowledge, but acquiring practical skills that way is challenging and doesn’t help you navigate the minefield of bootstrapping a new application. The technical choices you make at project inception can haunt you for a long time.

This book is different; this book is not theoretical. We will take a practical approach to learning. There is a small amount of theory interspersed throughout, and we will actually build a substantial microservices application. We will start from nothing and work through bringing our application into existence and getting it into production. We’ll build and test the application on our development workstation (or personal computer), and ultimately, we’ll deploy it to the cloud.

Together we’ll get our microservices application off the ground without having to learn the deepest details of any of the tools or technologies. An example of this book’s learning model is illustrated in figure 1.2.

Figure 1.2 The learning model for this book. We will skim the surface of these deep and complicated technologies to only use what is necessary to bootstrap our application.

This book is about building a microservices application, starting with nothing. Some people have already asked why I didn’t write this book to show how to convert a monolith to a microservices application? This is something that many people would like to learn.

I wrote the book in this way because it’s much easier to learn how to write an application from scratch than it is to learn how to refactor an existing application. I also believe these skills are useful because, in time, more and more applications will be written microservices-first.

In any case, refactoring an existing application is much more complicated than building a fresh application. It’s a process with many complex variables and depends heavily on the particulars of the legacy codebase. I make the presumption that it will be easier for you to figure out your own monolith conversion strategy once you know (indeed, once you have experienced) how to create a greenfield (new) microservices application.

I can assure you that when you can build an application microservices-first, you will be much better equipped to clearly see a route from your existing monolith to microservices. That journey from monolith to microservices will no doubt still be demanding, so stay tuned. In chapter 11, we will discuss more on this topic.

Throughout this book, you will learn concrete and practical techniques for getting a microservices application off the ground. Of course, there are many diverse ways to go about this and many different tools you can use. I am teaching you one single recipe and one set of tools (albeit a popular toolset). You will, no doubt, find many ways to improve on this recipe and enhance it for your own situation. Other experienced developers will, of course, already have their own recipes for doing this. What I’m trying to say is that this is my way, and it is just one of many ways that will work; however, I can attest that I have tried every technique in this book in production and found these to work well. So without further ado, let us commence our journey of learning and discovery.

1.2 What will I learn?

Throughout the book, we will progress from easy to more difficult. We’ll start with the simplest task-creating a single microservice. Over 11 chapters, we’ll build up to a more complex application and infrastructure, but we’ll do it in incremental steps so that you never get lost. After reading this book and practicing the skills taught, you can expect to be able to

• Create individual microservices

• Package and publish microservices using Docker

• Develop a microservices application on your development workstation using Docker Compose

• Test your code, microservices, and application using Jest and Cypress

• Integrate third-party servers into your application (like MongoDB and RabbitMQ, as examples)

• Communicate between microservices using HTTP and RabbitMQ messages

• Store the data and files your microservices need to operate

• Create production infrastructure with Kubernetes using Terraform

• Deploy your microservices to production using Terraform

• Create a continuous delivery pipeline to automatically deploy your application as you update the code

1.3 What do I need to know?

You might be wondering, what you need to know going into this book. I have made an effort to write this book with as few assumptions as possible about what you already know. We are going on a journey that takes you from absolute basics all the way through to some rather complicated concepts. I think there’s something here for everyone, no matter how much experience you might have already as a developer.

It’s best coming into this book if you have some entry-level understanding of computer programming. I don’t think you’ll need much, so long as you can read code and get the gist of what it’s doing. But don’t worry; I’ll explain as much as possible about anything important that is happening in the code.

If you have a background in programming, you’ll have no problem following along with the examples in this book. If you are learning programming while reading this book, you could find it to be quite a bit more challenging, but not impossible, and you might have to put in some extra work.

This book uses Node.js for examples of microservices, but starting out, you don’t need to know JavaScript or Node.js. You’ll pick up enough along the way to follow along. This book also uses Microsoft Azure for examples of production deployment. Again, starting out, you don’t need to know anything about Azure either.

Rest assured that this book isn’t about Node.js or Azure; it’s about building microservices applications using modern tooling like Docker, Kubernetes, and Terraform. Most of the skills you will take away from this book are transferable to other languages and other cloud providers. Because I had to pick a programming language and cloud vendor that I could use to demonstrate the techniques in this book, I chose Node.js and Azure. That’s mostly what I use in production these days.

If Node.js and Azure aren’t your thing, with some extra research and experimentation on your part, you’ll be able to figure out how to replace Node.js and JavaScript with your favorite programming language and replace Azure with your preferred cloud vendor. In fact, the main reason I use Docker, Kubernetes, and Terraform in the first place is precisely because these tools offer freedom-freedom of choice for programming language and freedom from cloud vendor lock-in.

1.4 Managing complexity

A microservice application, like any application, will become more complex over time. But it doesn’t need to start that way! This book takes the approach that we can begin from a simple starting point and that each iteration of development can also be just as simple. In addition, each microservice is small and simple. As you read this book, you’ll find that it isn’t as difficult as you might think to build applications with microservices (despite what some people say).

Microservices give us a way to manage complexity at a granular level, and it’s the level we work at almost every day-the level of a single microservice. At that level, microservices are not complex. In fact, to earn the name microservice, they have to be small and simple. A single microservice is intended to be manageable by a single developer or a small team!

It is true, though, that through continued development and evolution, a complex system will emerge. There’s no denying that a microservices application will become complex. But that doesn’t happen immediately; it takes time. Along the way, we’ll use microservices to manage the growing complexity of your application so that it doesn’t become a burden.

A microservices application is a form of complex adaptive system, where complexity emerges naturally from the interactions of its constituent parts. Even though the system as a whole can become far too complex for any mere mortal to understand, each of its components remains small, manageable, and easy to understand. Don’t worry though; the example application we build in this book isn’t that complicated.

Development with this microservices attitude (with help from our tools and automation) allows us to build extremely large and scalable applications without being overwhelmed by the complexity. And, after reading this book, you’ll be able to zoom in and look at any part of the most complex microservices application and find its components to be straightforward and understandable.

1.5 What is a microservice?

Before we can understand a microservices application, we must first understand what it means to be a microservice.

Definition A microservice is a tiny and independent software process that runs on its own deployment schedule and can be updated independently.

Let’s break that definition down. A microservice is a small, independent software process that has its own separate deployment frequency. That is to say that it must be possible to update each microservice independently from other microservices.

A microservice can be owned and operated either by a single developer or a team of developers. A developer or team might also manage multiple other microservices. Each developer/team has the responsibility for the microservice(s) they own. In the modern world of programming, this often includes development, testing, deployment, and operations. We might find, however, that when we work for a small company or a startup (as I do), or when we are learning (as we are in this book), we must manage multiple microservices or, indeed, even an entire microservices application on our own.

An individual microservice might be exposed to the outside world so our customers can interact with it or it might be purely an internal service and not externally accessible. It typically has access to a database, file storage, or some other method of state persistence. Figure 1.3 illustrates these internal and external relationships.

Figure 1.3 A single microservice can have connections to the outside world or other services, and it also can have a database and/or attached file storage.

By itself, a single microservice doesn’t do much. A well-designed system, however, can be decomposed into such simple services. The services must collaborate with each other to provide the features and functionality of the greater application. This brings us to the topic of the microservices application.

1.6 What is a microservices application?

A microservices application is traditionally known as a distributed application, a system composed of tiny components that live in separate processes and communicate via the network. Each service or component resides on a logically distinct (virtual) computer and sometimes even on a physically separate computer.

Definition A microservices application is a distributed program composed of many tiny services that collaborate to achieve the features and functionality of the overall project.

Typically, a microservices application has one or more services that are externally exposed to allow users to interact with the system. Figure 1.4 shows two such services acting as gateways for web-based and mobile phone users. You can also see in figure 1.4 that many services are working together within the cluster. It is called a cluster because it is a group of computers that are represented to us (the developers) as a single cohesive slab of computing power to be directed as we will. Somewhere close by we also have a database server. In figure 1.4, it is shown to be outside the cluster, but it could just as easily be hosted inside the cluster. We’ll talk more about this in chapter 4.

Figure 1.4 A microservices application is composed of multiple, small independent services running in a cluster.

The cluster is hosted on a cluster orchestration platform, and we use Kubernetes for this purpose. Orchestration is the automated management of our services. This is what Kubernetes does for us-it helps us to deploy and manage our services.

The cluster itself, our database and other virtual infrastructure, are all hosted on our chosen cloud vendor. We will learn how to deploy this infrastructure on Microsoft Azure, but with some work on your own, you can change the examples in this book to deploy to Amazon Web Services (AWS) or Google Cloud Platform (GCP).

A microservices application can take many forms, is very flexible, and can be arranged to suit many situations. Any particular application might have a familiar overall structure, but the services it contains will do different jobs, depending on the needs of our customers and the domain of our business.

1.7 What’s wrong with the monolith?

What is a monolith and what is so wrong with it that we’d like to use microservices instead? Although distributed computing has been around for decades, applications were often built in the monolithic form. This is the way that the majority of software was developed before the cloud revolution and microservices. Figure 1.5 shows what the services in a simple video-streaming application might look like and compares the differences between a monolithic version of the application and a microservices version.

Definition A monolith is an entire application that runs in a single process.

Figure 1.5 Monolith vs. microservices. You can see that building with microservices offers many advantages over the traditional monolithic application.

It is much easier to build a monolith than a microservices application. You need fewer technical and architectural skills. It’s a great starting point when building a new application, say for an early-stage product, and you want to test the validity of the business model before you commit to the higher technical investment required by a microservices application.

A monolith is a great option for creating a throw-away prototype. It also might be all that you need for an application that has a small scope or an application that stabilizes quickly and does not need to evolve or grow over its lifetime. If your application will always be this small, it makes sense for it to be a monolith.

Deciding whether to go monolith-first or microservices-first is a balancing act that has traditionally been won by the monolith. However, in this book, I’ll show you, given the improvements in modern tooling and with cheap and convenient cloud infrastructure, that it’s important that you at least consider building microservices-first.

Most products generally need to grow and be evolved, and as your monolith grows bigger and has more useful features, it becomes more difficult to justify throwing away the throw-away prototype. So down the road, you could find yourself stuck with the monolith at a time when what you really need is the flexibility, security, and scalability of a microservices application.

Monoliths come with a host of potential problems. These start out small, and we always have the best of intentions of keeping the code clean and well organized. A good team of developers can keep a monolith elegant and well organized for many years. But as time passes, the vision can be lost or sometimes there wasn’t a strong vision in the first place. All the code runs in the same process, so there are no barriers and nothing to stop us writing a huge mess of spaghetti code that will be near impossible to pick apart later.

Staff turnover also has a big effect. As developers leave the team they take crucial knowledge with them, and they are replaced by new developers who will have to develop their own mental model of the application, which could easily be at odds with the original vision. Time passes, code changes hands many times, and these negative forces conspire to devolve the codebase into what is called a big ball of mud. This name denotes the messy state of the application when there is no longer a discernible architecture.

Updating the code for a monolith is a risky affair. It’s all or nothing. When you push a code change that breaks the monolith, the entire application ceases operation, your customers are left high and dry, and your company bleeds money. We might only want to change a single line of code, but still, we must deploy the entire monolith and risk breaking it. This risk stokes deployment fear. Fear slows the pace of development.

In addition, as the structure of the monolith degenerates, our risk of breaking it in unanticipated ways increases. Testing becomes harder and breeds yet more deployment fear. Have I convinced you that you should try microservices? Wait, there’s more!

Due to the sheer size of an established monolith, testing is problematic, and because of its extremely low level of granularity, it is difficult to scale. Eventually, the monolith expands to consume the physical limits of the machine it runs on. As the aging monolith consumes more and more physical resources, it becomes more expensive to run. I have witnessed this! To be fair, this kind of eventuality might be a long way off for any monolith, but even after just a few years of growth, the monolith leads to a place that you would prefer not to be.

Despite the eventual difficulties with the monolith, it remains the simplest way to bootstrap a new application. Shouldn’t we always start with a monolith and later restructure when we need to scale? My answer: it depends.

Many applications will always be small. There are plenty of small monoliths in the wild that do their job well and don’t need to be scaled or evolved. Because these are not expanding, they do not suffer the problems of growth. If you believe your application will remain small and simple and doesn’t need to evolve, you should definitely build it as a monolith.

There are many applications, however, that we can easily predict will benefit from a microservices-first approach. These are the kinds of applications we know will continually be evolved over many years. Other applications that can benefit are those that need to be flexible, scalable, or have security constraints from the start. Building these types of applications is much easier if you start with microservices because converting an existing monolith is difficult and risky.

By all means, if you need to validate your business idea first, do so by initially building a monolith. However, even in this case, I would argue that with the right tooling, prototyping with microservices isn’t much more difficult than prototyping with a monolith. After all, what is a monolith if not a single large service?

You might even consider using the techniques in this book to bootstrap your monolith as a single service within a Kubernetes cluster. Now you have the best of both worlds! When the time comes to decompose to microservices, you are already in the best possible position to do so and, at your leisure, you can start chipping microservices off the monolith. And with the ease of automated deployment that modern tooling offers, it is easy to tear down and recreate your application or create replica environments for development and testing. If you want or need to create a monolith first, you can still benefit from the techniques and technologies presented in this book.

If you do start with a monolith, for your own sanity and as early as possible, either throw it away and replace it or incrementally restructure it into microservices. We’ll talk more about breaking up existing monoliths in chapter 11.

1.8 Why are microservices popular now?

Why does it seem that right now microservices are exploding in popularity? Is this just a passing fad?

No, it is not a passing fad. Distributed computing has been around for a long time and has always had many advantages over monolithic applications. Traditionally though, it has been more complex and more costly to build applications in this way. Developers only reached for these more powerful application architectures for the most demanding problems: those where the value of the solution would outweigh the cost of the implementation.

In recent times, however, with the advent of cloud technology, virtualization, and the creation of automated tools for managing our virtual infrastructure, it has become much less expensive to build such distributed systems. As it became cheaper to replace monolithic applications with distributed applications, we naturally considered the ways this could improve the structure of our applications. In doing so, the components of our distributed systems have shrunk to the tiniest possible size so that now we call them microservices.

That’s why microservices are popular now. Not only are they generally a worthwhile way to build complex modern applications, but they are also increasingly cost-effective. Distributed computing has become more accessible than ever before, so naturally more developers are using it. Right now, it appears to be nearing critical mass, and so it’s reaching the mainstream.

But why are microservices so good? How do they improve the structure of our application? This question leads to the benefits of microservices.

1.9 Benefits of microservices

Building distributed applications brings many advantages. Each service can potentially have its own dedicated CPU, memory, and other resources. Typically though, we share physical infrastructure between many services and that’s what makes microservices cost-effective. But we are also able to separate these out when necessary so that the services with the heaviest workloads can be allocated dedicated resources. We can say that each small service is independently scalable, and this gives us a fine-grained ability to tune the performance of our application. In this section, we look at these benefits:

• Allows for fine-grained control-Microservices allow us to build an application with fine-grained control over scalability

• Minimizes deployment risk-Microservices help us minimize deployment risk while maximizing the pace of development

• Lets you choose your own tech stack-Microservices allow us to choose the right stack for the task at hand so that we aren’t constrained to a single tech stack

Having a distributed application offers us the potential for better reliability and reduced deployment risk. When we update a particular service we can do so without the risk of breaking the entire application. Of course, we might still risk breaking a part of the application, but that is better and easier to recover from than bringing down the entire application. When problems occur, it’s easier to rollback just a small part of the system rather than the whole. Reduced deployment risk has the knock-on effect of promoting frequent deployments, and this is essential to agility and sustaining a fast pace of development.

These benefits are nothing new. After all, we have built distributed applications for a long time, but such systems have become cheaper to build and the tools have improved. It is easier than ever before to build applications this way and to reap the rewards. As costs decreased and deployment convenience increased, our services tended towards the micro-level, and this brought its own complement of benefits.

Smaller services are quicker to boot than larger services. This helps make our system easier to scale because we can quickly replicate any service that becomes overloaded. Smaller services are also easier to test and troubleshoot. Even though testing an overall system can still be difficult, we can more easily prove that each individual part of it is working as expected.

Building applications with many small and independently upgradeable parts means we can have an application that is more amenable to being extended, evolved, and rearranged over its lifetime. The fact that we have enforced process boundaries between our components means that we will never be tempted to write spaghetti code. And, indeed, if we do write terrible code (we all have bad days, right?), the impact of bad code is controlled and isolated because every microservice (to earn the name) should be small enough that it can be thrown away and rewritten within a matter of weeks, if not days. In this sense, we are designing our code for disposability. We are designing it to be replaced over time. The ongoing and iterative replacement of our application is not only made possible, but it is actively encouraged, and this is what we need for our application architecture to survive the continuously evolving needs of the modern business.

Another benefit that really excites developers using microservices is that we are no longer constrained to a single technology stack for our application. Each service in our application can potentially contain any tech stack. For larger companies, this means that different teams can choose their own tech stack; they can choose it based on their experience or based on the stack that is best for the job at hand. Various tech stacks can co-exist within our cluster and work together using shared protocols and communication mechanisms.

Being able to change between tech stacks is important for the long-term health of the application. As the tech landscape evolves, as it always does, older tech stacks fall out of favor and must eventually be replaced by new ones. Microservices create a structure that can be progressively converted to newer tech stacks. As developers, we no longer need languish on out-of-date technologies.

1.10 Drawbacks of microservices

This chapter would not be complete without addressing the two main problems that people have with microservices:

• Microservices are more difficult

• People often fear complexity

The first problem is the steep learning curve. Learning how to build microservices requires you to learn not just a complicated arrangement of technologies, but also the principles and techniques for building distributed applications. Although learning how to build microservices is difficult, this book will help you shortcut the learning curve.

Note I can understand if you feel daunted by what’s in front of you. But recently, huge progress has been made in the development of tooling for building distributed applications. Our tools are now more sophisticated, easier to use, and most importantly, more automatable than ever before.

These days, a single experienced developer is now capable of bootstrapping a microservices application on their own without the support of a team. I know this because I have done this multiple times for startups. Still, it surprises me how much can be achieved on one’s own. We’ll talk more about how startups, small teams, and solo developers can work with microservices quickly and effectively in chapter 11.

To be fair, the tools are still complicated. Ordinarily, it would take months or longer to conquer the learning curve on your own-mastering any of these tools takes significant time! But this book takes a different approach. Together we will only learn the bare minimum necessary to bootstrap our application and get it running in production. Together we will produce a simple but working microservices application. Along the way, we’ll also learn the basics of structuring distributed applications.

As I mentioned, there are actually two problems facing microservices developers. The second is that building a microservices application, indeed any distributed application, is going to be more complicated than building the equivalent monolith. It is hard to argue with this. The first thing I would say is that yes, building a monolith is simpler in the beginning and in many cases it is the right decision. If your application is one of those that must later be converted or restructured to microservices however, then you should consider the eventual cost of unraveling your big ball of mud.

Don’t be frightened by complexity; it happens whether you like it or not. Fortunately, microservices offer us tangible ways of managing complexity.

If you think this through, you might concede that building microservices, at least in certain situations, is actually less complicated than building a monolith. If this discussion hasn’t convinced you, consider this: any significant application is going to become complex. If not at the start, it will grow more complex over time. You can’t hide from complexity in modern software development, it always catches up with you, eventually. Instead, let’s take control of this situation and meet the complexity head-on. What we want are better tools to help manage complexity. Microservices as an architectural pattern is one such tool.

Think of microservices as a way to bring the pain forward, to a place where it’s more economical to deal with. What do we get in return for this pain? Microservices offer us tangible ways to manage complexity in our application. They provide hard boundaries that prevent us from writing spaghetti code. Microservices allow us to more easily rewire our application, scale it, and upgrade it over time. Microservices also force us to apply better design. We can’t prevent complexity, but we can manage it, and modern tooling for distributed applications is already here to help us.

1.11 Modern tooling for microservices

This book is all about the tooling. Together, we will learn the basics of a number of dif-ferent tools. To start with, we must be able to create a microservice. We’ll use JavaScript and Node.js to do this, and the next chapter will teach you the basics of that.

We are using Node.js because that’s my weapon of choice. However, as far as microservices are concerned, the technology stack within the service is not particularly important. We could just as easily build our microservices with Python, Ruby, Java, Go, or virtually any other language. We’ll encounter numerous tools along our journey, but these are the most important ones:

• Docker —To package and deploy our services

• Docker Compose —To test our microservices application on our development workstation

• Kubernetes —To host our application in the cloud

• Terraform —To build our cloud infrastructure, our Kubernetes cluster, and deploy our application

The technological landscape is always changing and so are the tools. So why should we learn any particular toolset when the tools are constantly outdated and replaced? Well, it’s because we will always need good tools to work effectively. And with better tools, we can do a better job, or maybe we just get to do the same job but more effectively. Either way, this helps us to be more productive.

I selected the tools for this book because these make the job of building microservices applications significantly easier and quicker. All technologies change in time, but I don’t think these particular tools are going anywhere soon. They are popular, are currently the best we have, and all fill useful positions in one’s toolkit.

Of course, these tools will eventually be replaced, but hopefully, in the meantime, we’ll have extracted significant value from these and built many good applications. And when the tools do change, they will certainly be replaced by better tools that lift the bar of abstraction even higher, making our jobs easier and less frustrating.

Docker is the one tool out of all the tools that is more or less ubiquitous. It seems to have almost come from nowhere and has taken over our industry. Kubernetes on the other hand is not quite as ubiquitous as Docker, although it does have a strong future because it allows us to transcend the boundaries of cloud vendors. This is good news if you ever felt trapped with your particular cloud provider. We can run our Kubernetes-based application on pretty much any cloud platform, and we have freedom of movement when needed.

Terraform is a relative newcomer, but I think it’s a game-changer. It’s a declarative configuration language that allows us to script the creation of cloud resources and the deployment of our services. The important thing about Terraform is that it’s one language that can work with potentially any cloud vendor. No matter which cloud vendor you choose, now or in the future, chances are that Terraform will support it, and you won’t have to learn something new.

Think about this for a moment: Terraform means we can easily code the creation of cloud infrastructure. This is something! In the past, we would laboriously and physically piece together infrastructure, but now we are able to create it with code. This concept is called infrastructure as code and it is a key enabler for continuous delivery, something important to modern software development that we’ll look at in chapter 7.

1.12 Designing a microservices application

This isn’t a book about theory, but I do have to touch on some of the software design aspects before we get into the practical stuff. I promise this is just some foundational principles, and there are plenty of other books to help you get a better grounding in this space.

At the outset, I’d like to say that designing a microservices application isn’t particularly different from designing any software. You can read any good book on software design and apply those same principles and techniques to microservices. There aren’t many hard and fast rules that I follow, but I feel these few are especially important:

• Don’t over design or try and future proof your architecture. Start with a simple design for your application.

• Apply continuous refactoring during development to keep it as simple as it can be.

• Let a good design emerge naturally.

I feel that the last rule is especially encouraged by microservices. You can’t conclusively preplan a big microservices application. The architecture has to emerge during development and over the lifetime of the application.

I’m not saying that you shouldn’t do any planning. You definitely should be planning at every stage of development. What I am saying is that you should be planning for your plan to change! You should be able to respond quickly to changing circumstances, and that’s another thing that’s well supported by microservices. Rules aside, let’s briefly discuss three principles that seem particularly relevant to microservices:

• Single responsibility principle

• Loose coupling

• High cohesion

Generally, we’d like to have each microservice be as small and simple as possible. One individual service should cover only a single conceptual area of the business. That is to say that each service should have a single, well-defined area of responsibility. This is normally known as the single responsibility principle.

Microservices should be loosely coupled and have high cohesion. Loosely coupled means that the connections between services are minimal and that they don’t share information unless necessary. When we reduce the connections and dependencies between microservices, we make it easier to upgrade individual services without having problems propagate through the application. Loose coupling helps us pull apart and rewire our application into new configurations. This makes our application more flexible and responsive to the changing needs of the business.

The code contained within a microservice should be highly cohesive. This means that all the code in a microservice belongs together and contributes to solve the problem that is the service’s area of responsibility. If a microservice solves more than one problem or has a larger area of responsibility, then this is an indication that it is not highly cohesive.

A design paradigm that works well for microservices is called domain driven design (DDD). Using DDD is a great way to understand the domain of a business and to model the business as software. The technique comes from the book, Domain Driven Design, by Eric Evans (2003). I have used it multiple times myself and find that it maps well to designing distributed applications. Specifically, the concept of the bounded context fits well to the boundary of a microservice as illustrated in figure 1.6.

Figure 1.6 Bounded contexts from domain driven design (DDD) equate to the boundaries of microservices.

This figure shows how the boundaries of concepts in our video-streaming domain might fit into microservices. Concepts such as User, Like, and Video live within our microservices, and some concepts (like Video) create the relationships between microservices. For example, in figure 1.6, the idea of a video is almost the same (but there can be differences) between the recommendations and the video-storage microservices.

There is a coding principle that seems like it might be under attack by microservices. Many developers live by the motto don’t repeat yourself (DRY). But in the world of microservices, we are developing a higher tolerance for duplicated code than what was previously considered acceptable.

The hard process boundaries in a microservices application certainly make it more difficult to share code, and the practice of DDD seems to encourage duplicating concepts, if not replicating code. Also, when microservices are owned by separate teams, we then encounter all the usual barriers to sharing code that already exists between teams.

Be assured, there are good ways to share code between microservices, and we aren’t simply going to throw out DRY. We’d still like to share code between microservices when it makes sense to do so.

1.13 An example application

By the end of this book, we’ll have built a simple but complete microservices application. In this section, we’ll develop an idea of what the final product looks like.

The example product we will build is a video-streaming application. Every good product deserves a name, so after much brainstorming and throwing around various ideas, I’ve landed on the name FlixTube, the future king of the video-streaming world. Gotta start somewhere right?

Why choose video streaming as the example? Simply because it’s a fun example and is surprisingly easy to create (at least in a simple form). It’s also a well-known use case for microservices, being the approach successfully taken to the extreme by Netflix. (Reports vary, but we know they run 100s if not 1,000s of microservices.)

We’ll use the FlixTube example application to demonstrate the process of constructing a microservices application. It will only have a small number of microservices, but we will build-in the pathways we need for future scalability, including adding more virtual machines to the cluster, replicating services for scale and redundancy, and extracting services to separate code repositories so these can have separate deployment schedules and be managed by separate teams.

Our application will have a browser-based front end so our users can view a list of videos. From there they can select a video and it will begin playing. During development, we’ll boot our application using Docker Compose, which we’ll cover in chapters 4 and 5. We’ll build and publish Docker images for our microservices in chapter 3. In chapters 6 and 7, we’ll deploy our application to production. In chapter 8, we’ll swing back to development for some automated testing.

Our application will contain services for video streaming, storage, and upload, plus a gateway for the customer-facing front end. We’ll work up to deploying the full application in chapter 9, which figure 1.7 illustrates. In chapters 10 and 11, we’ll look at all the ways this architecture can help us scale in the future as our application grows. Are you ready to start building with microservices?

Figure 1.7 Our example application running in production on Kubernetes.

Summary

• We take a practical rather than a theoretical approach to learning how to build a microservices application.

• Microservices are small and independent processes that each do one thing well.

• A microservices application is composed of numerous small processes working together to create the application’s features.

• A monolith is an application composed of a single massive service.

• Although building a microservices application is more complicated than building a monolith, it’s not as difficult as you might think.

• Applications built from microservices are more flexible, scalable, reliable, and fault-tolerant than monolithic applications.

• The union of the modern tools like Docker, Kubernetes, and Terraform make building a microservices application much easier than previously possible.

• Domain driven design (DDD) is an effective way to design a microservices application.

• Bounded contexts from DDD map well to the boundaries of microservices.

• We previewed the example application that we’ll build in this book.