Micro-services are discrete components working together to provide functionality and business logic for a larger application, usually communicating over a network protocol (such as HTTP/2 or some other binary transport) and distributed across many physical machines. Each component is isolated from the others, and they take in well-defined inputs and yield well-defined outputs. Multiple instances of the same service can run across many servers and traffic can be load balanced between them. If designed correctly, it is possible for an individual instance to fail without bringing down the whole system and for new instances to be spun up during runtime to help handle load spikes.
Go kit (refer to https://gokit.io) is a distributed programming toolkit for the building of applications with a micro-service architecture founded by Peter Bourgon (@peterbourgon on Twitter) and now maintained by a slice of Gophers in the open. It aims to solve many of the foundational (and sometimes boring) aspects of building such systems as well as encouraging good design patterns, allowing you to focus on the business logic that makes up your product or service.
Go kit doesn't try to solve every problem from scratch; rather, it integrates with many popular related services to solve SOA (service-oriented architecture) problems, such as service discovery, metrics, monitoring, logging, load balancing, circuit breaking, and many other important aspects of correctly running micro-services at scale. As we build our service by hand using Go kit, you will notice that we will write a lot of boilerplate or scaffold code in order to get things working.
For smaller products and services with a small team of developers, you may well decide it is easier to just expose a simple JSON endpoint, but Go kit really shines for larger teams, building substantial systems with many different services, each being run tens or hundreds of times within the architecture. Having consistent logging, instrumentation, distributed tracing, and each item being similar to the next means running and maintaining such a system becomes significantly easier.
"Go kit is ultimately about encouraging good design practice within a service: SOLID design, or domain-driven-design, or the hexagonal architecture, etc. It's not dogmatically any of those, but tries to make good design/software engineering tractable.” —Peter Bourgon
In this chapter, we are going to build some micro-services that address various security challenges (in a project called vault) –upon which we would be able to build further functionality. The business logic will be kept very simple, allowing us to focus on learning the principles around building micro-service systems.
Note
There are some alternatives to Go kit as a technology choice; most of them have a similar approach but with different priorities, syntax, and patterns. Ensure that you look around at other options before embarking on a project, but the principles you learn in this chapter will apply across the board.
Specifically, in this chapter, you will learn:
- How to hand code a micro-service using Go kit
- What gRPC is and how to use it to build servers and clients
- How to use Google's protocol buffers and associated tools to describe services and communicate in a highly efficient binary format
- How endpoints in Go kit allow us to write a single service implementation and have it exposed via multiple transport protocols
- How Go kits-included subpackages help us solve lots of common problems
- How Middleware lets us wrap endpoints to adapt their behavior without touching the implementation itself
- How to describe method calls as requests and response messages
- How to rate limit our services to protect from surges in traffic
- A few other idiomatic Go tips and tricks
Some lines of code in this chapter stretch over many lines; they are written with the overflowing content right-aligned on the next line, as shown in this example:
func veryLongFunctionWithLotsOfArguments(one string, two int, three
http.Handler, four string) (bool, error) {
log.Println("first line of the function")
}
The first three lines in the preceding snippet should be written as one line. Don't worry; the Go compiler will be kind enough to point out if you get this wrong.
There are many options when it comes to how our services will communicate with each other and how clients will communicate with the services, and Go kit doesn't care (rather, it doesn't mind–it cares enough to provide implementations of many popular mechanisms). In fact, we are able to add multiple options for our users and let them decide which one they want to use. We will add support the familiar JSON over HTTP, but we are also going to introduce a new technology choice for APIs.
gRPC, short for Google's Remote Procedure Call, is an open source mechanism used to call code that is running remotely over a network. It uses HTTP/2 for transport and protocol buffers to represent the data that makes up services and messages.
An RPC service differs from RESTful web services because rather than making changes to data using well-defined HTTP standards, as you do with REST (POST to create something, PUT to update something, DELETE to delete something, and so on), you are triggering a remote function or method instead, passing in expected arguments and getting back one or more pieces of data in response.
To highlight the difference, imagine that we are creating a new user. In a RESTful world, we could make a request like this:
POST /users
{
"name": "Mat",
"twitter": "@matryer"
}
And we might get a response like this:
201 Created
{
"id": 1,
"name": "Mat",
"twitter": "@matryer"
}
RESTful calls represent queries or changes to the state of resources. In an RPC world, we would use generated code instead in order to make binary serialized procedure calls that feel much more like normal methods or functions in Go.
The only other key difference between a RESTful service and a gPRC service is that rather than JSON or XML, gPRC speaks a special format called protocol buffers.