The biggest fallacy about monoliths is you can have only one.
All organizations want to deliver business value as fast as possible. At the same time, they must ensure their software consistently works as expected. Increasing the speed of development risks an increase in bugs and decrease in reliability. The larger a software solution becomes, the greater this risk.
To mitigate these risks, organizations are required to reduce velocity in software delivery by coordinating through meetings. These meetings seek to optimize delivery while addressing any risks along the way. The larger the software solution, the more meetings that are required to mitigate associated risks. Yet, every meeting slows down the delivery process. Therefore, the balance between speed and delivering quality software is important.
Decomposing APIs into microservices (Figure 10.1) is one option for teams to address this need for balance. This chapter explores the topic of microservices, including benefits, challenges, and alternatives to microservices.
Microservices are small, independently deployed components that deliver one or a small number of bounded digital capabilities. Each service offers one of the many digital capabilities required, ensuring that each service has a limited scope. When combined, microservices deliver a highly complex solution using smaller building blocks than the traditional service-oriented approach. This is illustrated in Figure 10.2.
Microservice adoption has typically been used to decompose highly complex systems into independently deployed components rather than containing the complexity within a single codebase. The cognitive load required to understand a single microservice is lowered compared to that of understanding a single codebase. Testing becomes more approachable as well as automated test suites become more focused on the single component. This is shown in Figure 10.3.
The idea of microservices have been around for more than a decade but have only recently gained widespread use by the late majority. In the early days of microservices, teams had to weigh the effort required to establish and maintain the infrastructure necessary for a microservices architecture. Over time, many of these factors were addressed through cloud native infrastructure, the growth of the DevOps culture, better delivery pipeline automation, and the use of containerization for producing self-contained deployment packages.
A Warning About the Term Microservices
It is important to recognize that there are a variety of definitions and scope assigned to microservices. Some organizations or individuals may define microservices as individual entities that offer a web API, resulting in many network calls between services unnecessarily. There are other definitions that exist as well. Use caution when the organization makes a broad declaration that they are moving to microservices.
First, be sure to understand what is meant by the term. Be specific in the definition and intent. Next, seek a reference architecture and one or more reference applications to demonstrate the desired target state. Ask questions as necessary to ensure a shared understanding of the purpose and outcomes desired when shifting to microservices. Otherwise, everyone will assign their own definition of a microservice, and chaos will reign across the organization.
Finally, recognize that organizations may be using the term microservices in a specific way, while others may simply use the term to indicate that teams should “think smaller” than large, siloed systems that exist today. Do not assume understanding without following these recommended steps to align on a mutual definition and goals when shifting to microservices.
With many of these factors addressed today, organizations are now taking a microservice-based approach by default. However, it is important to understand both the benefits and the challenges of architectural decisions around microservices. This includes both technical and non-technical factors that can have a positive or negative impact on the people behind the services.
The cost of coordinating many teams working within the same codebase is extremely high. Meeting after meeting is required to ensure that developers don’t introduce bugs and that merge conflicts are avoided. For large organizations, the introduction of additional middle managers is required to coordinate the coordination.
The single greatest benefit of microservices is to reduce team coordination. A team operating independently to maintain one or a few microservices can coordinate within their team with limited coordination points outside their team.
Based on Metcalfe’s Law, smaller teams result in fewer communication paths. The benefit is that it takes fewer meetings to communicate intent and resolve issues across the organization. The result is a team with more time to design, code, test, and deliver their services.
Coordination across teams is not eliminated with microservices, however. Integration must be coordinated to ensure that all the microservices fit the solution needs. Timelines must still be coordinated between product managers, business, and service teams. Therefore, the number of smaller team meetings may increase, while the number of attendees and the scope of discussion is greatly reduced for each meeting. Teams are given more independence and meetings are more efficient as coordination efforts are limited to the scope of the team’s deliverables.
To achieve the goal of reduced team coordination, several factors are required:
■ Self-service, automated infrastructure resources that ensure rapid onboarding of new services. This is commonly associated with a DevOps culture of automation tooling combined with continuous delivery processes
■ Team ownership of services throughout the software development lifecycle, including enhancements and support services. This results in a culture of “you own it, you manage it” rather than siloed delivery to operations teams but may also include software reliability engineers and other roles to augment the team
■ Removal of centralized data ownership, allowing each service to own and manage the data associated with their services
Without incorporating these important factors, any shift to microservices will be met with challenges, including bloated microservices, slower velocity of delivery, and even project failure. This is discussed further in Chapter 6 of the book Strategic Monoliths and Microservices in the section titled “Open-Host Service.”
The benefits of moving to microservices have less to do with technology choices and more to do with the impact they have on the organization. This includes how microservices positively or negatively impact day-to-day development and operations.
While API products and microservices each offer network-based APIs, the differences between them are vast:
■ API products target stability and evolvability while microservices enable experimentation. Consumers of an API expect the contract to never break, unless migrating to a new version of an API. Microservices are designed for experimentation and constant change. As such, microservices may be split, combined, or removed at any time
■ API products offer a set of digital capabilities for integration into solutions. Microservices decompose a solution into distributed components. They are not an external contract with developer beyond the immediate boundary. If this becomes a requirement, the service must be transitioned to an API product with a stable interface
Just because the codebase is small doesn't make it a microservice. A microservice is an internal component and shouldn't be shared directly with external consumers. API products may be shared within a specific team, across teams, across the organization, and/or with partners/public developers.
The most important factor when considering microservices is the complexity of the solution. Complexity cannot be fully removed from a software solution. However, it may be distributed across the solution. Microservices allow the complexity to be spread across components, making each individual component easier to build and manage. However, by separating the problem into distributed components, other complexity is introduced.
Each team and organization must consider both the complexity of a solution and the complexities that microservices introduce to determine if a shift to microservices will help or hinder the organization’s ability to deliver solutions with both speed and safety. While a single microservice may offer lower complexity, the infrastructure and automation requirements to deliver, monitor, and protect the service at runtime increases.
If the solution has a low factor of complexity, then microservices are often unnecessary and may even be detrimental to solution success. If the complexity of the solution is unknown, weight the factors below and then consider starting with a minimal solution that balances these factors, migrating to microservices when and if the complexity increases.
Microservices require a self-service, fully automated infrastructure. Teams must be able to design a microservice, build it, and deploy code without any manual processes or approvals. Organizations that have not fully automated their provisioning and deployment pipeline will encounter considerable friction. Without full automation support, new code will be added to existing microservices to avoid manual processes, resulting in a few, very large, siloed services.
Microservices must have their own release cycle. Some organizations opt to use their existing release processes, such as a two-week sprint and release, rather than allowing microservices to be released when they are ready. This coordinated deployment of all microservices at once results in a large release process, rather than independent teams that may deploy their microservices as needed.
Each microservice should be owned, monitored, and managed by a single team. Teams should own only one or a few microservices to focus their efforts. They must own the service from definition to design and delivery. They must support the service, much like a product that seeks to incorporate improvements as feedback is received from other teams.
Smaller organizations find it challenging to assign single team ownership, instead sharing the ownership of all services across a small number of developers. Developers spend more time moving between codebases and dealing with the challenges of distributed computing rather than delivering solutions to market.
Microservices require proper organizational support and structure. Organizational structure and culture may be at odds with the ownership and independence of microservice teams. Reporting structures may be optimized for larger delivery teams. Challenges may arise trying to coordinate service integrations across teams that span managers. Organizations that prefer centralized oversight may encounter difficulties shifting control to individual teams.
These organizational challenges may create an unhealthy tension that makes it difficult to move to microservices while achieving the speed and safety often promised with microservices. Keep the org structure in mind before shifting to microservices by ensuring that buy-in exists from the executive team and managers that oversee service teams.
Don’t discount the organizational and cultural impact of adopting microservices. The shift from product or project-based ownership to the ownership of one or a few microservices within a bounded area will have an impact on reporting structures and team alignment. Count the cost before proceeding. Otherwise, the organization may be trading code complexity for organizational complexity.
Microservices must own their own data. This can be a challenging item, as rarely do teams think beyond the source code when it comes to shifting to microservices. When services do not own their data, the coordination cost of underlying schema changes can ripple across multiple microservices that share the data. This can require large, coordinated release efforts to bring every service in line with a breaking schema change within a shared data source.
Microservices require considerable data management and governance. Since microservices own their own data, investment must be made to ensure that proper data management policies exist for reporting and analytics. Today this is typically handled through ETL-based processes that migrate data into an OLAP-based data store for optimized queries and decision support.
Shifting to microservices requires shifting to data streaming rather than ETL processes to bring together data from multiple services for the purposes of reporting. More emphasis needs to be placed on managing glossaries that create a strong ontology and taxonomy to unify distributed data models. Organizations with centralized data model governance and large shared databases must use caution when migrating to a microservice architecture. Finally, don’t underestimate the effort required to separate a monolithic data store into a data store per service.
The journey toward microservices requires a deep understanding of distributed systems. Those not as familiar with the concepts of distributed tracing, observability, eventual consistency, fault tolerance, and failover will encounter a more difficult time with microservices. The eight fallacies of distributed computing, written in 1994 and still applicable today, must be understood by every developer.
Additionally, many find that architectural oversight is required to initially decompose and subsequently integrate services into solutions. Teams unable to have architectural support may suffer from lack of architectural consideration in the design of their microservices, resulting in poor boundaries and overlapping team responsibilities that produces increased cross-team coordination. The Align Phase of the ADDR Process seeks to address this concern early.
Finally, layered architectures are common within a monolithic codebase but are frowned upon with microservices. If microservices are layered incorrectly, a change to a single microservice may ripple to other services and require additional coordination efforts to synchronize the changes. Microservices that apply a layered approach must ensure that it will limit the impact of a service change. Revisit the layered principle of REST to see how layers may be used to add independence between components.
With more microservices comes greater complexity when calls between services are required. Synchronous microservices require call chaining across a network and are therefore susceptible to network failure.
Resilience must be built into each microservice to ensure retries and failover occurs in the event of a temporary network outage. The concept of a service mesh, discussed further in Chapter 15, was introduced to address these cross-cutting concerns but introduce further deployment and operational complexity that may be unnecessary for simple solutions.
Another side effect of synchronous call chaining is that failures beyond the first call require previous service calls to rollback transactions. During the height of SOA, transaction managers were used to create distributed transactions, usually through the use of 2-phase commit (2PC) transactions. This isn’t an option with a highly distributed microservice architecture.
Instead, distributed transactions are often implemented using the saga pattern. A transactional context is applied within each service call, with compensating transactions used to apply the opposite operation when a rollback is required. State machines are required for each resource involved. Event sourcing is often used alongside the saga pattern to ensure that all operations are atomic transactions backed by a ledger for auditing and troubleshooting purposes.
Refactoring code is more challenging with microservices as IDEs and other refactoring tools can only refactor within a single codebase. Refactoring code across multiple microservice codebases becomes more error prone.
When microservices use the same programming language, the tendency is to utilize a shared codebase for common code. Sharing code between services can create coordination coupling, requiring more meetings to ensure a change to code shared across microservices doesn’t negatively impact others. When sharing code between services, all changes must be optional to avoid forcing other teams to be in lockstep.
Do You Really Need Microservices?
After weighing the challenges of microservices and the underlying operational complexity, it may be determined that an API boundary doesn't need to be decomposed into microservices. Instead of microservices, perhaps all that is needed is one or more monolithic APIs that are designed to be modular, known as modular monoliths.
Modular monoliths apply loose coupling and high cohesion within a single codebase to avoid the complexity of distributed computing. Over time, the monolith may be decomposed into microservices if the solution becomes too complex for a single codebase. However, only apply this approach once all paths to refactoring and re-organizing the modules of a single codebase have been exhausted.
Remember that organizations aren’t limited to a single monolith. Multiple, modular monoliths may be sufficient for the needs of the team. Each monolith offers one or a few APIs that support the operations within the bounded contexts contained within the monolith.
Microservices may be designed to be synchronous or asynchronous. Synchronous microservices apply a more traditional request/response model, typically via HTTP using REST constraints, RPC, or a query-based API.
While synchronous, request/response-based APIs are more familiar for developers, it can create fragile integrations. Services that orchestrate API calls between services may fail mid-stream due to a problem with a single service, requiring a reversal of previously successful API calls. Services that call other services, termed call chaining, may also fail mid-stream but are unable to reverse the previous API calls themselves. Figure 10.4 illustrates this concern as the service client only called Service A, which results in more service calls that can fail due to a downstream error.
Alternatively, an asynchronous access pattern may be used for microservice integration. In this style, messages are submitted to a message queue or topic hosted on a message broker or streaming server. One or more microservices listen for messages, process them in turn, and then emit messages containing business events as the result.
Asynchronous microservices offer several advantages. The greatest advantage is that new microservices can be brought online to replace older ones, without the knowledge of the consumer. The new microservice subscribes the same topic or queue and begins processing messages.
Additionally, consumers have the flexibility to use one or more of the following interaction patterns, as needed: fire-and-forget, fire-and-listen for events, or fire-and-follow-up using the provided response URL.
Finally, asynchronous error handling and recovery is built-in to message brokers and streaming solutions. Avoiding the need for synchronous call chaining error recovery greatly simplifies the infrastructure requirements, reducing or eliminating the need for a service mesh.
Of course, asynchronous integration is a more complex interaction than a standard request/response approach. Developers must learn to integrate with asynchronous services and handle failures by checking for error response messages and process unprocessed messages using dead letter queues (DLQs).
A microservice-based architecture is not limited to a single style or approach. There are three common styles of applying microservices. Each one offers a slight variation on how microservices may be used to reduce coordination between teams. Some have chosen to apply one or more of these styles in combination to support the needs and culture of the organization.
In this style, each service communicates with other services directly using a synchronous or asynchronous model. This approach is the most common style found during the early days of microservices. Those using a synchronous model encounter challenges such as service communication failure and call chain fragility. The introduction of a service mesh helps to overcome these challenges, as does the shift to a more asynchronous model that is message driven. Figure 10.6 depicts this more traditional microservice architecture style.
This style starts with the design of an API that is further decomposed into microservices as appropriate. The API becomes the stable orchestration layer across one or more microservices, offering a more stable contract externally while supporting experimentation and splitting of microservices internally. This is a style chosen by organizations that have struggled with some of the challenges of the direct service communication model. Many of the organizations that were early adopters of microservices are moving to this model. This is shown in Figure 10.7.
A cell-based architecture blends the previous two styles to bring a more modular approach to microservices. Each cell offers one or more digital capabilities, offered through a synchronous or asynchronous API. The API is externalized via a gateway, hiding the internal details of service decomposition through encapsulation. Cells are combined to create larger solutions. Because of the modular composability of this style, it is often found in large organizations as it offers better management for their evolving systems.
Uber recently shifted from the integration of many small services to this cell-based architecture model. They discovered that complexity increases far outweighed the value that microservices provided. Uber engineering refers to this approach as Domain-Oriented Microservice Architecture (DOMA) and have written a nice article that summarizes the approach. It resembles many of the elements of a cell-based architecture by reducing the complexity of a large-scale microservice architecture while maintaining the flexibility and benefits that it provides.
Organizations on the path to microservices often struggle with finding the right size for microservices. Teams will often ask, "What is the maximum allowable size for a microservice?" A better question would be, "What is the right size for this microservice based on what is needed today?"
Microservices aren’t frozen in time. Instead, they grow and become more complex. Over time, it a microservice may need to be split. At other times, two microservices may become co-dependent and benefit from being combined into a single service. Therefore, the size of a microservice will change over time.
It is also important to note that services tend to grow over time, requiring that the boundaries of a microservice be re-evaluated often. This can only be done efficiently if service ownership resides with a single team. Services shared across teams require further coordination meetings.
Right sizing microservices requires a continuous process of design and re-evaluation:
1. First, identifying where transactional boundaries exist to find candidate service boundaries. This will help to reduce the chances of spreading transactions across services
2. Design two or a few course-grained microservices based on the identified boundaries. This will ensure your microservice operations retain integrity within a transactional boundary and avoid the challenges of rolling back transactions across multiple microservice calls over the network
3. Keep splitting services as they grow, being guided by the needs of transactional boundaries while keeping team coordination costs low
It is best to focus less on the size of the microservice and more on the purpose of the service. Microservices should seek to make future change easier, even if that means the service is more course-grained at the start.
If the team has determined that decomposing the API into two or more microservices would be beneficial, then there are a few additional steps needed when starting the delivery phase. This includes extending previously created API sequence diagrams with more detail, identifying candidate services, and capturing the service design details.
The first step in decomposing APIs is to identify candidate microservices. Start by expanding the web sequence diagrams, created during the API modeling and design phases, to include external systems and data stores. This helps to identify natural boundaries between services. Figure 10.9 expands the Shopping API with the inclusion of an external search engine that will support basic and advanced query support.
Since the search engine integration is read-only within the search books operation of the Shopping API, this is a good candidate for decomposing into a separate service. The team that will own this candidate microservice will be responsible for ensuring the search engine indexes are both performant and deliver the search capabilities required by customers. Figure 10.10 show the boundary for the candidate microservice that will support book searches.
Next, revise the sequence diagram to show the introduction of the candidate microservice. Determine if the integration should use a synchronous API, such as REST, or if an asynchronous service would be better. An updated sequence diagram for the Shopping API is shown in Figure 10.11.
Review the updates and determine if the candidate microservice is doing too much and should be further decomposed. Or, perhaps the service is doing too little, introducing too many network calls and therefore should be combined into a slightly larger service.
Finally, capture the design details of the candidate microservice. The use of the Microservice Design Canvas (MDC) is recommended as it helps to focus on the commands, queries, and events that the service will support. If the details of the service cannot fit into a single page MDC, it may be responsible for too much. In this case, revisit the design to see if it should be further decomposed or if it is right-sized for supporting the needs of the API. Figure 10.12 shows an example MDC for the Book Search Service.
At this point, the MDC provides sufficient details to proceed with building and integrating the service with one or more APIs. However, there are some additional design considerations to address before proceeding.
Note that not all APIs will benefit from service decomposition. Anytime there is a new microservice involved, there is an opportunity for increased network latency that could negatively impact API clients.
This is of particular importance when service call chaining occurs as a result of a synchronous service calling another, which may call another, and so on. The total time for a client to receive a response is the sum of the time required to execute each service call sequentially. For highly efficient service implementations that are < 10 milliseconds each, that may not be too much of a concern. For services that integrate with legacy systems that may suffer from degraded performance during peak usage, this could result in several seconds of wait time for an end user. Finally, for some microservice ecosystems, it may not be possible to know how many services are involved or predict the total time required for execution.
When possible, keep a transaction within a single service boundary. Transaction boundaries that span multiple service calls require additional design considerations. If a service call fails, any previous service calls must be rolled back. Since each service manages its own transactional boundary, a compensating transaction may be required to reverse the transaction. This is referred to as the Saga pattern. Whenever possible, seek to decompose microservices such that transaction integrity is maintained.
Additionally, consider whether a dedicate team will own the microservice. If so, does the introduction of the candidate microservice reduce or increase cross-team coordination. Not all decisions about service decomposition are about reducing code size.
Finally, avoid splitting services based on the CRUD lifecycle, creating one service per operation, e.g., Create Project Service, Update Project Service, Read Project Service, List Projects Service, Delete Project Service. This pattern creates more coordination requirements between each service team. A change to the resource representation for a project requires coordinating with each of the teams that own the service. The exception is when complexity dictates the need to split one part of a CRUD lifecycle due to increased complexity. For example, the complexity of payment processing integration may merit shifting this behavior to a separate microservice.
While there are many benefits to moving to microservices, the transition shouldn't be taken lightly. After some time and reflection, some organizations have chosen to simplify their microservice journey, others have decided to abandon their journey in favor of thinking smaller but without microservices, and the rest continue move forward with microservices.
First, verify that a microservices-based approach is being applied to the correct context. Some microservice initiatives are dictated from the executive team without proper context. It usually starts with an executive that mandates microservices so that teams can increase the velocity of delivery. However, context isn’t provided to inform teams to avoid microservice complexity when the solution is simple, e.g., an application that offers CRUD-based forms to manage a dataset. The result is wasted time and effort to decompose a simple solution into microservices that introduce unnecessary complexity around runtime management, troubleshooting complexity, and distributed transaction management.
Next, be sure that the organization’s reporting structure and culture are ready to shift alongside the move to microservices. Some organizations are not prepared for teams to own services long-term. Instead, they treat microservices as projects that are delivered but never owned long-term. The team that built the service have moved onto other projects and higher priority initiatives. Teams that could benefit from a minor change to an existing service are required to build their own service as a result.
Finally, find ways to build smaller. Modularize code within a single codebase. Design clear APIs for consumers to use. Decompose APIs into microservices only when high complexity makes it necessary.
Microservices are independently deployable units of code that are combined to create distributed systems. The benefits of moving to microservices have less to do with technology choices and more to do with the positive or negative impact they have on the organization. This includes reducing the coordination costs of multiple teams operating on the same codebase.
Be wary of technology trends that do not inject more benefit than the complexity they require. Microservices have offered benefits to some organizations, but not without their challenges. Organizations must count the cost of moving to microservices to determine if the complexity of designing, building, and operating microservices outweighs the complexity of a single, monolithic codebase.
Alternatives, such as modular monoliths and cell-based architectures, support many of the goals of microservices but with varying support for reduced coordination and local decision optimization. When in doubt, apply the “you ain’t gonna need it” (YAGNI) principle of agile software by starting with a modular monolith API and decomposing it into microservices when the need arises.