Goals and Scope

Our mission is to help overcome the complexity of designing and evolving APIs through proven, reusable solution elements:

While much has been said and written about HTTP, Web APIs, and integration architectures in general (including service-oriented ones), the design of individual API endpoints and message exchanges has received less attention so far:

• How many API operations should be exposed remotely? Which data should be exchanged in request and response messages?

• How is loose coupling of API operations and client-provider interactions ensured?

• What are suitable message representations: flat or hierarchically nested ones? How is agreement reached on the meaning of the representation elements so that these elements are processed correctly and efficiently?

• Should API providers be responsible for processing data provided by their clients, possibly changing the provider-side state and connecting to backend systems? Or should they merely provide shared data stores to their clients?

• How are changes to APIs introduced in a controlled way that balances extensibility and compatibility?

The patterns in this book help answer these questions by sketching proven solutions to specific design problems recurring in certain requirements contexts. Focusing on remote APIs (rather than program-internal ones), they aim at improving the developer experience on both the client side and the provider side.

Usage Scenarios

Our objective is to make API design and usage a pleasant experience. To that end, three main use cases for our book and its patterns are as follows:

1. Facilitate API design discussions and workshops by establishing a common vocabulary, pointing out required design decisions, and sharing available options and related trade-offs. Empowered by this knowledge, API providers are enabled to expose APIs of quality and style that meet their clients’ needs, both short term and long term.

2. Simplify API design reviews and speed up objective API comparisons so that APIs can be quality assured—and evolved in a backward-compatible and extensible way.

3. Enhance API documentation with platform-neutral design information so that API client developers can grasp the capabilities and constraints of provided APIs with ease. The patterns are designed to be embeddable into API contracts and observable in existing designs.

We provide a fictitious case study and two real-world pattern adoption stories to demonstrate and jumpstart this pattern usage.

We do not expect readers to know any particular modeling approach, design technique, or architectural style already. However, such concepts—for instance, the Align-Define-Design-Refine (ADDR) process, domain-driven design (DDD), and responsibility-driven design (RDD)—have their roles to play. They are reviewed briefly in Appendix A.

Existing Design Heuristics (and Knowledge Gaps)

You can find many excellent books that provide deep advice on specific API technologies and concepts. For instance, the RESTful Web Services Cookbook [Allamaraju 2010] explains how to build HTTP resource APIs—for example, which HTTP method such as POST or PUT to pick. Other books explain how asynchronous messaging works in terms of routing, transformation, and guaranteed delivery [Hohpe 2003]. Strategic DDD [Evans 2003; Vernon 2013] can get you started with API endpoint and service identification. Service-oriented architecture, cloud computing, and microservice infrastructure patterns have been published. Structuring data storages (relational, NoSQL) is also documented comprehensively, and an entire pattern language for distributed systems design is available as well [Buschmann 2007]. Finally, Release It! extensively covers design for operations and deployment to production [Nygard 2018a].

The API design process, including goal-driven endpoint identification and operation design, is also covered well in existing books. For instance, Principles of Web API Design: Delivering Value with APIs and Microservices [Higginbotham 2021] suggests four process phases with seven steps. The Design of Web APIs [Lauret 2019] proposes an API goal canvas, and Design and Build Great Web APIs: Robust, Reliable, and Resilient [Amundsen 2020] works with API stories.

Despite these invaluable sources of design advice, the remote API design space still is not covered sufficiently. Specifically, what about the structures of the request and response messages going back and forth between API client and provider? Enterprise Integration Patterns [Hohpe 2003] features three patterns representing message types (event, command, and document message) but does not provide further details on their inner workings. However, “data on the outside,” exchanged between systems, differs from “data on the inside” that is processed program-internally [Helland 2005]. There are significant differences between the two types of data in terms of their mutability, lifetime, accuracy, consistency, and protection needs. For instance, increasing a local stock-item counter internal to an inventory system probably requires somewhat less architecture design than product pricing and shipment information that is exchanged between manufacturers and logistics companies jointly managing a supply chain via remote APIs and messaging channels.

Message representation design—data on the outside [Helland 2005] or the “Published Language” pattern [Evans 2003] of an API—is the main focus area of this book. It closes the knowledge gaps regarding API endpoint, operation, and message design.

Microservice API Patterns

Our pattern language, called Microservice API Patterns (MAP), provides comprehensive views on API design and evolution from the perspective of the messages exchanged when APIs are exposed and consumed. These messages and their pay-loads are structured as representation elements. The representation elements differ in their structure and meaning because API endpoints and their operations have different architectural responsibilities. The message structures strongly influence the design time and runtime qualities of an API and its underlying implementations; for instance, few large messages cause network and endpoint workloads (such as CPU consumption and network bandwidth usage) that differ from those caused by many small messages. Finally, successful APIs evolve over time; the changes over time have to be managed.

We chose the metaphor and acronym MAP because maps provide orientation and guidance, just as pattern languages do; they educate their readers on the options available in an abstract solution space. APIs themselves also have a mapping nature, as they route incoming requests to the underlying service implementations.

We admit that “Microservice API Patterns” might come across as click-bait. In case microservices are no longer fashionable shortly after this book is published, we reserve the right to rename the language and repurpose the acronym. For instance, “Message API Patterns” outlines the scope of the language well too. In the book, we refer to MAP as “the pattern language” or “our patterns” most of the time.

Scope of the Patterns in This Book

This book is the final outcome of a volunteer project focused on the design and evolution of Web APIs and other remote APIs addressing endpoint and message responsibility, structure, and quality concerns as well as service API evolution. The project started in the fall of 2016. The resulting pattern language, presented in this book, helps answer the following questions:

• What is the architectural role played by each API endpoint? How do the endpoint roles and the responsibilities of operations impact service size and granularity?

• What is an adequate number of representation elements in request and response messages? How are these elements structured? How can they be grouped and annotated with supplemental information?

• How can an API provider achieve a certain level of API quality while at the same time using its resources in a cost-effective way? How can quality tradeoffs be communicated and accounted for?

• How can API professionals deal with life-cycle management concerns such as support periods and versioning? How can they promote backward compatibility and communicate unavoidable breaking changes?

We collected our patterns by studying numerous Web APIs and API-related specifications and reflecting on our own professional experience (before writing any pattern). We observed many occurrences of the patterns—known uses—both in public Web APIs and in application development and software integration projects in industry. Intermediate versions of many of our patterns went through the shepherding and writers’ workshop processes at EuroPLoP¹ from 2017 to 2020; they were later published in the respective conference proceedings.²

1. https://europlop.net/content/conference.

2. We decided not to include big collections of known uses in the book; such information is available online and in the EuroPLoP conference proceedings from 2016 to 2020. In some of the supplemental resources, you can find extra implementation hints as well.

Entry Points, Reading Order, and Content Organization

When maneuvering a complex design space to solve wicked problems [Wikipedia 2022a] (and API design certainly qualifies as sometimes wicked), it is often hard to see the forest for the trees. It is neither possible nor desirable to serialize or standardize the problem-solving activities. Therefore, our pattern language has multiple entry points. Each book part can serve as a starting point, and Appendix A suggests even more.

The book has three parts: Part 1, “Foundations and Narratives,” Part 2, “The Patterns,” and Part 3, “Our Patterns in Action (Now and Then).” Figure P.1 shows these parts with their chapters and logical dependencies.

Part 1 introduces the domain of API design conceptually, starting with Chapter 1, “Application Programming Interface (API) Fundamentals.” Lakeside Mutual, our case study and primary source of examples, appears for the first time with its business context, requirements, existing systems, and initial API design in Chapter 2, “Lakeside Mutual Case Study.” We provide decision models that show how the patterns in our language relate to each other in Chapter 3, “API Decision Narratives.” Chapter 3 also provides pattern selection criteria and shows how the featured decisions were made in the Lakeside Mutual case. These decision models may serve as navigation aids when reading the book and when applying the patterns in practice.

Part 2 is the pattern reference; it starts with Chapter 4, “Pattern Language Introduction,” followed by five chapters full of patterns: Chapter 5, “Define Endpoint Types and Operations,” Chapter 6, “Design Request and Response Message Representations,” Chapter 7, “Refine Message Design for Quality,” Chapter 8, “Evolve APIs,” and Chapter 9, “Document and Communicate API Contracts.” Figure P.2 illustrates these chapters and possible reading paths in this part; for instance, you can learn about basic structure patterns such as Atomic Parameter and Parameter Tree in Chapter 4 and then move on to element stereotypes such as Id Element and Metadata Element found in Chapter 6.

Each pattern description can be seen as a small, specialized article on its own, usually a few pages long. These discussions are structured identically: First, we introduce when and why to apply the pattern. Then we explain how the pattern works and give at least one concrete example. Next, we discuss the consequences of applying the pattern and direct readers to other patterns that become eligible once a particular one has been applied. The names of our patterns are set in small caps (example: Processing Resource). This pattern template, introduced in detail in Chapter 4, was derived from the EuroPLoP conference template [Harrison 2003]. We refactored it slightly to take review comments and advice into account (thank you Gregor and Peter!). It puts particular emphasis on quality attributes and their conflicts, as our patterns deal with architecturally significant requirements; consequently, trade-offs are required when making API design and evolution decisions.

Part 3 features the application of the patterns in two real-world projects in rather different domains, e-government and offer/order management in the construction industry. It also reflects, draws some conclusions, and gives an outlook.

Appendix A, “Endpoint Identification and Pattern Selection Guides,” provides a problem-oriented cheat sheet as another option to get started. It also discusses how our patterns relate to RDD, DDD, and ADDR. Appendix B, “Implementation of the Lakeside Mutual Case,” shares more API design artifacts from the book’s case study. Appendix C, “Microservice Domain-Specific Language (MDSL),” provides a working knowledge of MDSL, a language for microservices contracts with built-in pattern support via decorators such as <<Pagination>>. MDSL provides bindings and generator support for OpenAPI, gRPC protocol buffers, GraphQL, and other interface description and service programming languages.

You will see some (but not much) Java and quite a bit of JSON and HTTP (for instance, in the form of curl commands and responses to them) as you find your way through the book. Very little, if any, gRPC, GraphQL, and SOAP/WSDL might also come your way; if so, it is designed to be simple enough to be understandable without expertise in any of these technologies. Some of our examples are described in MDSL (if you are wondering why we created yet another interface description language: OpenAPI in its YAML or JSON renderings simply does not fit on a single book page when going beyond HelloWorld–ish examples!).

Supplemental information is available through the Web site companion to this book:

https://api-patterns.org

We hope you find the results of our efforts useful so that our patterns have a chance to find their way into the body of knowledge of the global community of integration architects and API developers. We will be glad to hear about your feedback and constructive criticism.

Olaf, Mirko, Daniel, Uwe, Cesare
June 30, 2022