11. Improving the Developer Experience

Every useful API that delivers value will typically have multiple consumers. This is a natural asymmetry, which will only increase over time.

Mark O’Neill

When teams think about delivering an API, the primary focus is on the code that must be built. This includes considerations such as the target programming language, frameworks that aid in building the API, CI/CD pipelines, and other factors. While all of these decisions are important, they focus on the API provider only. They do not directly empower the tens, hundreds, or thousands of future API consumers that will use the API.

As an API provider, it is important to keep the API consumers first in everything that is designed and delivered. This includes creating mock APIs to help early adopters provide feedback early on the API design (Figure 11.1). It also requires consideration on whether to offer helper libraries and command line interfaces to reduce the integration time by developers across all skill levels consuming the API. This chapter addresses these concerns in an effort to multiply the impact across the many current and future API consumers that will integrate the API.

Creating a Mock API Implementation

API design is a mixture of patterns and subjective design decisions. What makes sense during the API design phase may not make sense to developers once they integrate the API. API mocking is the creation of a simulated version of an API design. Generating a mock version of an API design helps to verify that the API design will meet the needs of target developers.

Mock implementations are quick to deliver as they lack production-ready code. They also bypass backend database servers and legacy systems. Mock APIs implement the API design while returning static responses or responses based upon synthesized data sets.

With mock implementations, developers are able to integrate portions of an API before the implementation has begun. This allows API teams to see if the API design is missing critical functionality. It also helps to identify important data elements that may be missing from the API design.

API design includes making compromises. Once developers start to integrate with an API, they will provide feedback on how it should be changed. If the API design is frozen, this change must wait until a new version of the API is released. Integrating with a mock implementation identifies these problem areas early, when the cost of change is much lower.

An added benefit is that mock implementations help accelerate the delivery process. Rather than waiting until the entire API has been coded, mock implementations may be used to produce API integration code for front end development. They may also be used to drive automated test creation. Over time, the mock integration is replaced with the actual API until the mock is no longer needed and removed completely. The interface remains constant, but the implementation is replaced over time. Meanwhile, teams are able to proceed in parallel.

There are three primary types of API mock implementations: static mocking, prototyping, and README-based mocking. Each may be used independently or in combination to explore an API design prior to delivery. Mocks may also be used for standing up a local or cloud-based learning environment that is separate from production.

Static API Mocking

One of the easiest ways to explore an API design before writing code is to write a static version of some or all the expected API requests and responses. These static mocks capture API interactions through JSON or XML-based files that may be shared with developers and API design reviewers. These static mocks offer examples to view and make improvements upon prior to coding.

The mock response below demonstrates a book resource instance for the Shopping API example using the JSON:API specification:

{
  "data": {
    "type": "books",
    "id": "12345",
        "attributes": {
        "isbn": "978-0321834577",
        "title": "Implementing Domain-Driven Design",
        "description": "With Implementing Domain-Driven Design, Vaughn has
made an important contribution not only to the literature of the Domain-
Driven Design community, but also to the literature of the broader enterprise
application architecture field."
    },
    "relationships": {
      "authors": {
          "data": [
              {"id": "765", "type": "authors"}
          ]
        }
    },
    "included": [
            {
               "type": "authors",
               "id": "765",
               "fullName": "Vaughn Vernon",
               "links": {
                   "self": { "href": "/authors/765" },
                   "authoredBooks": { "href": "/books?authorId=765" }
               }
            }
  }
}

Static mocks may be provided using a web server, such as Apache or nginx, to allow front-end developers to integrate the mock API responses into the user interface. They will then be able to provide feedback early and often as they start to parse and integrate the static mocks into their code.

It is important to note that static mocks lack any implementation, so mock integration will be limited to GET-based operations only. However, creating a static mock of an API operation that retrieves a resource representation is quite useful, easy to build, and provides opportunities for plenty of feedback.

API Prototype Mocking

Constructing a throwaway prototype provides greater validation of an API design than a static mock. Unlike a static mock, which is often limited to GET-based operations, an API prototype is able to support all types of operations, including those that create or modify resource state.

However, API prototypes take more effort to produce manually. Typically, teams select a preferred programming language and framework that is optimized for rapid delivery. Ruby, Python, PHP, and Node.js are popular choices due to their fast development and abundant libraries for producing APIs and synthesized data sets.

Note

Teams may wish to select a language and framework that isn’t supported for production by the organization. Doing so will ensure that prototypes intended to be throwaway don’t suddenly become production code.

The use of an API mocking tool, often based upon an API description format such as the OpenAPI Specification (OAS), allows teams to skip most or all development efforts. These tools will produce simple mock implementations that store data temporarily for common CRUD-based operations. Some tools generate code for the mock implementation, while others create the mock API on the fly.

It is recommended to keep API prototypes simple at first. Expand the prototype as needed to deep-dive into any contentious areas that need further exploration or areas that can encourage parallel development.

README-Based Mocking

README-based mocking provides an alternative prototyping style without the need to write code. A README file is created to demonstrate how to use an API to accomplish one or more desired outcomes. README-based mocks help to validate the API design before implementation starts by sharing the intent of API usage to produce desired outcomes.

Most README-based mocks use Markdown, enabling the combination of text and code examples to be easily produced and rendered in a browser. Tools such as GitHub or GitLab have built-in Markdown support, although static site generation tools such as Jekyll or Hugo may also be used.

Below is a README-based mock that demonstrates how to retrieve book details, then add it to a cart using the JSON:API media format:

1. Retrieve Book Details

GET /books/12345 HTTP/1.1
Accept: application/vnd.api+json

HTTP/1.1 200 OK
Content-Type: application/vnd.api+json
...

{
  "data": {
    "type": "books",
    "id": "12345",
        "attributes": {
        "isbn": "978-0321834577",
        "title": "Implementing Domain-Driven Design",
        "description": "With Implementing Domain-Driven Design, Vaughn has
made an important contribution not only to the literature of the Domain-
Driven Design community, but also to the literature of the broader enterprise
application architecture field."
    },
    "relationships": {
      "authors": {
          "data": [
              {"id": "765", "type": "authors"}
          ]
        }
    },
    "included": [
            {
               "type": "authors",
               "id": "765",
               "fullName": "Vaughn Vernon",
               "links": {
                   "self": { "href": "/authors/765" },
                   "authoredBooks": { "href": "/books?authorId=765" }
               }
            }
  }
}

2. Add Book to Cart

POST /carts/6789/items HTTP/1.1
Accept: application/vnd.api+json

HTTP/1.1 201 Created
Content-Type: application/vnd.api+json
...

{
  "data": {
    "type": "carts",
    "id": "6789",
        "attributes": {
           ... truncated for space ...
        }
  }
}

3. Remove a Book from a Cart

...

Using this approach gives teams time to think through the API design and how it will be used to produce outcomes – without the overhead of writing or changing code. It also increases the quality of documentation and the surrounding conversations about the design. README-driven design can be thought of as the hand-written version of an acceptance test using behavior-driven development (BDD) frameworks such as Cucumber.

Providing Helper Libraries and SDKs

Client-side helper libraries wrap all of the HTTP connection management, error detection, JSON marshaling, and other concerns for a single programming language. Some developers prefer helper libraries as it helps speed development by avoiding the need to deal with low-level HTTP concerns. They also enable code completion within popular IDEs that isn’t possible when working directly with HTTP.

A Software Developer Kit, or SDK, is a packaged solution that includes helper libraries, documentation, example code, reference applications, and other resources for developers. While SDKs may be distributed by API providers, the growth of API developer portals have replaced the need to package a complete SDK.

While there is a distinct difference between an SDK and helper library, many developers tend to use the term interchangeably. The important thing is to be clear about what is provided in the distribution to set proper expectations with the developer.

Don’t expect all developers to take advantage of helper libraries, however. Those familiar with HTTP generally prefer working with it directly rather than a helper library. This preference is centered on the inflexibility of some helper libraries that may prevent the ability to fit the exact needs of the use case due to missing features within the library.

Options for Offering Helper Libraries

There are three options for offering helper libraries:

■ Provider supported: Provider supported helper libraries are built and maintained by the API provider. They own them, manage them, and keep them in sync as API operations are added or enhanced through manual coding or code generation.

■ Community contributed: Instead of the vendor offering the helper library, the community contributes the SDK. This may be the case for all programming languages or just for currently unsupported programming languages. Vendors may choose to allow the community-contributed helper libraries to thrive on their own, work with the authors to make them better, or eventually offer to take over maintenance. Be aware that community-contributed SDKs have a tendency to lose interest or available maintainers over time and may become abandoned. Communication with community supporters is critical, as many developers may assume that they are vendor-backed and complain if they are no longer maintained.

■ Consumer generated: With the growth of API definition formats such as Swagger, RAML, Blueprint, and others, it is becoming easier for API consumers to generate their own client library from any of these formats. This gives the consumer the most flexibility, as they may opt to create a lightweight wrapper around the HTTP layer, or perhaps generate a robust library with objects/structures that mimic API resources.

API teams must determine how they plan to provide helper libraries, which programming languages they plan to support, and how community or consumer generated helper libraries may impact their developer support program.

Versioning Helper Libraries

Helper libraries will have their own version numbering scheme, which may confuse developers. This is common when helper libraries make breaking changes to how they surface the API as objects.

For example, version 1 of a helper library may return a hash of name/value pairs containing resource properties, eventually choosing to abandon this approach in favor of returning objects. While the API may still remain as v1, the helper library may be on version 2.1.5 for Ruby while the Python module may be on version 1.8.5.

Including SDK language and version number in the User-Agent header for all requests can help. However, the most important factor will be to ensure that everything is logged on the client side and server side.

Support emails will become more confusing when trying to determine the language, helper library version, and API version being used. Add community contributed helper libraries into the process and more confusion will occur. This confusion can exist for even the most experienced developer.

The addition of a request identifier or correlation identifier is a common solution to this problem. These identifiers help to correlate client requests with server-side logs as developers correspond with API support team. Application performance management (APM) tools may be useful for diagnosing issues as well.

Helper Library Documentation and Testing

Developers integrating an API will not want to move between API documentation and an undocumented helper library while trying to figure out how to code up their idea. To overcome this poor developer experience, thorough helper library documentation will be required for every programming language. Additionally, example code within the developer portal should include examples for each supported programming language.

For each release, API teams will need to factor in sufficient time to keep helper library documentation updates across all supported programming languages. Automated tests for each helper library will also be maintained to ensure that libraries are in sync with the latest API operation enhancements as they are released.

Offering CLIs for APIs

While most APIs target developers that will integrate it into a larger application, it is important not to overlook command line interfaces (CLIs) as another developer use case. It is not uncommon to encounter CLIs that wrap an API, much like helper libraries offer programming language specific wrappers around a web-based API.

Unlike helper libraries, CLIs offer a human friendly method of interacting with remote systems without requiring coding skills. The CLI is both an API consumer and an automation tool. They may used for many purposes, including:

■ Automation engineers that need a quick, one-off scripting option

■ Extracting data locally for proof of concepts (POCs)

■ Infrastructure automation and tooling, e.g., Kubernetes, Heroku, Amazon Web Services (AWS), Google Cloud (gcloud)

Offering a CLI tool expands the reach of an API beyond full time developers to automation engineers that are better equipped to write shell scripts rather than applications to integrate with APIs. CLI tools may offer human-friendly output, in addition to JSON, CSV, or other output formats that support better automation and tool chaining.

Designing a CLI tool that wraps an API is no different than designing the API itself. It requires understanding the desired outcomes, activities, and steps required to accomplish the jobs to be done (JTBD). Then, design the CLI interface to meet these outcomes. The code block below shows how a CLI interface could be designed to support the Shopping API designed in previous chapters:

$> bookcli books search "DDD"

| Title                          | Authors        | Book ID        |
|--------------------------------|----------------|----------------|
| Implementing Domain-Driven ... | Vaughn Vernon  | 12345          |

$> bookcli cart add 40321834577

Success!

$> bookcli cart show

Cart Summary:

| Total        | Estimated Sales Tax |
|--------------|---------------------|
| $42.99 USD   | $3.44 USD           |

Cart Items:

| Title                          | Price      | Qty | Book ID        |
|--------------------------------|------------|-----|----------------|
| Implementing Domain-Driven ... | $42.99 USD | 1   | 12345          |

$> ...

To offer a great CLI experience, API teams will need to become students of human-first command line interface design. The excellent Command Line Interface Guidelines site offers in-depth details on how to design a human-first CLI based on 40 years of patterns and practices across tooling and operating systems.

Also, teams should seek to understand the pipe and filter design pattern commonly seen across *nix tools such as sed, awk, and grep to better understand how tool chaining works. Finally, carefully examining popular CLIs from Kubernetes, Heroku, and others help teams to see how to design a user-friendly CLI that wraps remote APIs.

Summary

API design doesn’t stop with the details of API operations and protocol semantics. It requires thoughtful consideration regarding how the API will be integrated by developers. While some code decisions are important for the API provide, these are internal concerns that do not have a direct impact on the many API consumers that will use the API. The more complex the API, the more tooling is required to support the design and delivery process. This includes the use of API mocks, helper libraries, and CLIs.

API teams must consider how their decisions may have a positive or negative impact on future API consumers. Avoid making decisions that provide local optimizations for a few developers, instead opting to make global optimizations for the many of current and future consumers of the API.