Customers today want to have access to enterprise data and services through a variety of digital devices and channels. To meet customer expectations, enterprises need to open their assets in an agile, flexible, secure, and scalable manner. APIs form the window into an enterprise’s data and services. They allow applications to easily communicate with each other using a lightweight protocol like HTTP. Developers use APIs to write applications that interact with the back-end system. Once an API has been created, it needs to be managed using an API management platform. An API management platform helps an organization publish APIs to internal, partner, and external developers to unlock the unique potential of their assets. It provides the core capabilities to ensure a successful API program through developer engagement, business insights, analytics, security, and protection. An API management platform helps business accelerate outreach across digital channels, drive partner adoption, monetize digital assets, and provide analytics to optimize investments in digital transformation (see Figure 2-1).
Figure 2-1. API management offerings
An API management platform enables you to create, analyze, and manage APIs in a secure and scalable environment (see Figure 2-2). An API management platform should provide the following capabilities:
Developer Enablement for APIs
Secure, Reliable and Flexible Communications
API lifecycle Management
API Auditing, Logging and Analytics
Figure 2-2. API management capabilities
API management capabilities can be delivered by any API management vendor in a public cloud as a hosted service or can be deployed on-premise in a private cloud. A hybrid approach can also be followed, with some components of the API management platform being offered as a hosted solution and others deployed on-premise for increased security and control.
An API management platform provides these capabilities as three major types of services (and as illustrated in Figure 2-3):
API gateway services allow you to create and manage APIs from existing data and services. They allow you to add security, traffic management, interface translation, orchestration, and routing capabilities into your API.
Analytics services monitor traffic from individual apps and provide business with insight and operational metrics, API and app performance, and developer engagement metrics.
Developer portals provide capabilities for developer and app registration and onboarding, API documentation, community management, and API monetization.
Figure 2-3. API management platform services
This chapter introduces you to the different capabilities required for an API management platform and shows how the different services provided by the platform help enable these capabilities. In the process, it also introduces the various concepts and technologies for API management.
Secure, Reliable, and Flexible Communication
APIs help digital apps to communicate with back-end services. Communication forms the core of APIs. Communication can use REST, SOAP, Plain Old XML (POX), or any other protocol of choice. REST is by far the most preferred communication protocol for APIs due to its inherent characteristics, which are described later in this book. An API management platform must provide a framework that allows secure, reliable, and flexible channels of communication. The API gateway within the API management platform provides the services that form the core capabilities required for API communications.
The API Gateway
An API gateway forms the heart of any API management solution that enables secure, flexible, and reliable communication between the back-end services and digital apps (see Figure 2-4). It helps to expose, secure, and manage back-end data and services as RESTful APIs. It provides a framework to create a facade in front of the back-end services. This facade intercepts the API requests to enforce security, validate data, transform messages, throttle traffic, and finally route it to the back-end service. The static response may be cached to improve the performance. The API gateway can optionally orchestrate requests between multiple back-end services and also connect to databases to service the request. All of these functionalities can be implemented in a gateway, mostly through configurations and scripting extensions.
Figure 2-4. API Gateway capabilities
The main features of an API gateway include—but are not limited to—the following.
API Security
APIs provide access to valuable and protected data and assets. Therefore, security for APIs is of utmost importance to protect the underlying assets from unauthenticated and unauthorized access. Due to the programmatic nature of APIs and their accessibility over the public cloud, they are also prone to a different kind of threat attack. The API management platform should therefore address the following aspects of API security.
Authentication: Authentication is the process of uniquely determining and validating the identity of a client. An app acts like a client making an API call. It is a piece of software that consumes an API to get access to enterprise assets, data, and services. It can run on the Internet, a computer, smartphones, tablets, or any other electronic device. Apps are usually made available by their developers through a distribution platform, such as Apple’s App Store, or Google Play, or the Windows Phone Store. Every app is identified by its name and a unique UUID known as the app key. The app key often serves as an identity for the app making a call to the API. It is normally issued and managed via the API management platform of the API provider. An app key is also known as an API key, an app ID, or a client ID. The API management platform must have the ability to issue, track, and revoke the app key. Authentication services may also require integration with identity management systems that control user access to applications and other services.
Authorization: Authorization controls the level of access that is provided to an app making an API call. It controls which API resources and methods that an app can invoke. When an app makes an API call, it normally passes an OAuth access token in the HTTP headers. This token is generated as part of the OAuth handshake and is associated with scopes that determine the APIs that can be accessed using the token. An access token can be associated with one or multiple scopes. Each access token may have an expiry duration that controls the duration for which the token is valid. If the token is expired, a new access token would be required to be generated. An app can do this automatically by presenting a refresh token. The refresh token may be exchanged to get a new access token with a renewed validity period. The use of a refresh token by an app to regenerate the access token helps to improve the overall user experience.
Identity mediation: APIs normally use OAuth protocols for implementing security. However, the back-end services may be secured using SAML or any other WS-Security headers. Hence, the API management platform must have the capability to integrate with back-end IDM platforms and do identity mediation. OAuth to SAML is a very common identity mediation requirement.
Data privacy: APIs expose data that may be sensitive; such data should be visible only to its intended recipient. Any sensitive data in transit should be encrypted. If such data gets logged anywhere, it must be masked. The API management platform must therefore possess data privacy capabilities. Data privacy can be achieved through encryption and data masking. Sensitive data should be encrypted with digital certificates in transit. The API management platform should have support for SSL/TLS. For some use cases, additional encryption of specific elements within the payload may also be required. Masking sensitive data at rest within audits and log files is yet another data privacy requirement that an API management platform should provide.
Key and certificate management: The API management platform should also provide the capability to manage keys and certificates required for data privacy.
DoS protection: APIs open valuable data and assets outside the firewalls of the enterprise. This increases the attack surface and makes them more prone to attacks. Hackers may try to bring down back-end systems by pumping unexpectedly high traffic through the APIs. Denial-of-service (DoS) attacks are very common on APIs. Hence, the API management platform should be able to detect and stop such attacks.
Threat detection: For public APIs, the likelihood of bad actors making attacks using malicious content is high. Content-based attacks can be in the form of malformed XML or JSON, malicious scripts, or SQL within the payload. Such attacks can also happen to private and enterprise APIs. The API management platform should be able to identify malformed request formats or malicious content within the payload and then protect against such attacks. Error visualization capability can also help detect any hacker attempting to find an exploitable weakness in APIs.
API Traffic Management
Depending on the nature of data and services provided by the API, traffic management offers a different business value to different classes of customers. Each customer class may be willing to pay differently for access. For example, some app developers prefer to try out APIs for free. The API provider may provision such users to make a small number of API calls in a day/week/month. Paying customers, however, want access to a higher or an unlimited number of API calls. Again, the API provider may allow customers a different level of access depending on their location or the time of the day; for example, internal enterprise users may get unlimited access to a high-performing API, whereas public Internet users may have limited access. More API calls may be allowed during off-peak hours but there is a limited number allowed during peak business hours. The API provider may have different requirements to throttle and manage the API traffic. The API platform should provide the following capabilities for traffic management.
Consumption quota: Defines the number of API calls that an app is allowed to make to the back end over a given time interval. Calls exceeding the quota limit may be throttled or halted. The quota allowed for an app depends on the business policy and monetization model of the API. A common purpose for a quota is to divide developers into categories, each of which has a different quota and thus a different relationship with the API. For example, free developers who sign up might be allowed to make a small number of calls. But paid developers (after their verification) might be allowed to make a higher number of calls.
Spike arrest: Identifies an unexpected rise in the API traffic . It helps to protect back-end systems that are not designed to handle a high load. API traffic volume exceeding the spike arrest limit may be dropped by the API management platform to protect back-end systems in the event of DoS attacks.
Usage throttling: Provides a mechanism to slow down subsequent API calls. This can help to improve the overall performance and reduce impacts during peak hours. It helps to ensure that the API infrastructure is not slowed down by high volumes of requests from a certain group of customers or apps.
Traffic prioritization: Helps the API management platform determine which class of customers should be given higher priority. API calls from high-priority customers should be processed first. Not all API management platforms support this capability. Hence, an alternative approach or design may be required to implement traffic prioritization.
Interface Translation
When an enterprise creates an API to expose its data and services, it needs to ensure that the API interface is intuitive enough for developers to easily use. APIs should be created with an API-First approach, which promotes API creation with a consumer focus. Hence, the interface for the API will most likely be different from that of the back-end services that it exposes. The API gateway should therefore be able to transform the API interface to a form that the back end can understand. To support interface translation, the API gateway should support the following:
Format translation: The back-end system might expect data in SOAP, or XML, or CSV or any other proprietary format. Such data format cannot be easily consumed by the API consumer. Hence, the API gateway should have the capability to easily transform from one format to other. Most API management platforms provide the capability to transform data from XML to JSON (and vice versa) with a one-to-one mapping of the data elements. Mapping from JSON to any other data format may be supported through customization.
Protocol translation: Most back-end systems that host services provide a SOAP interface for consumers. However, SOAP is not a protocol that is suitable for APIs to build apps for digital devices. API management platforms must be able to do a protocol transformation from SOAP to REST to provide a lightweight interface for consumers. Support for other protocol transformations—like HTTP(s) to JMS/FTP/JDBC—may be a nice to have feature in the API management platform.
Service and data mapping: An API management platform should provide a graphical representation of the different back-end service component that maps to provide an API service. It should incorporate service mapping tools that enable the discovery and description of existing service delivery assets so that they can be wired into your API design.
Caching
Caching is a mechanism to optimize performance by responding to requests with static responses stored in-memory. An API proxy can store back-end responses that do not change frequently in memory. As apps make requests on the same URI, the cached response can be used to respond instead of forwarding those requests to the back-end server. Thus caching can help to improve an API’s performance through reduced latency and network traffic.
Similarly, some static data required for request processing may also be stored in-memory. Instead of referring to the main data source each time, such data can be retrieved from the cache for processing the request. An expiry date/time can be set for the cached data or the data can be invalidated based on defined business rules. If the data is expired, new data would be retrieved from the original data source and the cache would be refreshed with the updated data .
Service Routing
APIs need to route requests from consumers to the right back-end service providing the business functionality. There may be one more backend systems providing the backend functionality. Hence, the API management platform should be able to identify and route the request to the correct instance of the back-end. The API management platform should support the following routing capabilities:
URL mapping: The path of the incoming URL may be different from that of the back-end service. A URL mapping capability allows the platform to change the path in the incoming URL to that of the back-end service. This URL mapping happens at runtime so that the requested resource is retrieved by the consumer via service dispatching.
Service dispatching: This allows the API management platform to select and invoke the right back-end service. In some cases, multiple services may have to be invoked to perform some sort of orchestration and return an aggregated response to the consumer.
Connection pooling: The API management platform should be able to maintain a pool of connections to the back-end service. Connection pooling improves overall performance. Also, it may be required for traffic management purposes to ensure that only a fixed maximum number of active connections are opened at any point in time to the back-end service.
Load balancing: Load balancing helps to distribute API traffic to the back-end services. Various load balancing algorithms may be supported. Based on the selected algorithm, the requests must be routed to the appropriate resource that is hosting the service. Load balancing capabilities also improve the overall performance of an API .
Service Orchestration
In many scenarios , the API gateway may need to invoke multiple back-end services in a particular sequence or in parallel and then send an aggregated response to the client. This is known as service orchestration. The service orchestration capability helps to create a coarse-grained service by combining the results of multiple back-end services invocation. This helps to improve overall performance of the client by reducing latency introduced due to multiple API calls. Service orchestration capability may require the API gateway to maintain states in-between the API calls . However, the API gateway should be kept as light and stateless as possible. Hence, it is recommended that the API gateway only be involved in the orchestration of read-only services that are non-transactional in nature .
API Auditing, Logging and Analytics
Businesses need to have insight into the API program to justify and make the right investments to build the right APIs. They need to understand how an APIs is used, know who is using it, and see the value generated from it. With proper insight, business can then make decisions on how to enhance the business value either by changing the API or by enriching it. An API gateway should provide the capability to measure, monitor, and report API usage analytics. Good business-friendly dashboards for API analytics measure and improve business value. A monetization report on API usage measure business value; hence, it is yet another desirable feature on an API management platform.
API Analytics
Analytics provide you with information to make future decisions about your API. When you see an increase in API traffic, you need to know whether this indicates the success of your API program or whether it is being used in a malicious way, resulting in inflated traffic. How do you determine the adoption of your API? Is there an increased interest in your APIs within the developer community? Is there an increase in the number of apps built using your APIs? How has the performance of the APIs been in terms of response time and throughput? What are the different kinds of devices being used to access the APIs? How have the APIs been adopted across the globe? As an API provider and consumer, you need to know the answer to these questions and many others. The more you know, the better you are able to determine what’s going on. You need metrics to decide which features should be added to your API program. API analytics is the answer to all queries.
The API management platform should be able provide the following capabilities required for analytics.
Activity Logging
Activity logging provides basic logging of API access, consumption, performance, and any exceptions. The platform should capture and provide information on who is using an API, what types of apps and devices the API are being called from, and which geographical region is the source of the API traffic. It should log the IP address of the clients, as well as the date and time when a request was received and the response was sent. The gateway within the API management platform should log which API and method is being invoked by the client. Various metainformation, such as URI, HTTP verb, API proxy, developer app, and other information can be logged into the gateway for every API call. The platform can process this information at a later time to provide meaningful reports for API analysis. API performance metrics and response/error codes should also be logged as part of activity logging.
User Auditing
User auditing can help the API administrator review historical information to analyze who accesses an API, when it is accessed, how it is used, and how many calls are made from the various consumers of the API.
Business Value Reports
Business value reports gauge the monetary value associated with the API program. Monetization reports of API usage provide information on the revenue generated from the API. The API gateway should be able to provide API usage monetization reports. Some APIs may be directly monetized, but many have an indirect model for monetization. Hence, additional value-based reporting should also be possible within an API management platform to measure customer engagements. Engagements can be measured by the number of unique users, the number of developers registered, the number of active developers, the number of apps built using the APIs, the number of active apps , and many other items.
Advanced Analytics
The API management platform should be able to extract and log custom variables from within the message payload for advanced analytics reporting. It should provide API administrators and product managers the capability to create pluggable and custom reports from the captured information.
Service-level Monitoring
The API management platform should provide performance statistics that track the latency within the platform and the latency for back-end calls. This helps the API administrator find the source of any performance issues reported on any API. The platform should have the capability to provide reports on errors raised during the processing of the API traffic within the platform, or ones that are received from the back end. Classifying the errors by type, frequency, and severity gives API administrators a valuable aid for troubleshooting.
Developer Enablement for APIs
An API program cannot be successful without the active involvement of a developer community. Application developers use APIs to build mobile apps or to build a custom integration between two or more applications. Hence, developers need to know which APIs are available, what their functionalities are, and how they can be used. Developers should have a playground to experience and test APIs to effectively use them in their applications. An API management platform should provide services that enable developers to build apps using the APIs. A developer portal can provide these services.
Developer Portal
A developer portalis a customized web site that allows an API provider to provide services to the developer community. It is essentially a content-management system that documents the APIs—their functionalities, interfaces, getting-started guides, terms of use, and much more. Developers can sign up through the portal and register their applications to use the APIs. The can interact with other developers in the community through blogs and threaded forums. The portal can also be used to configure and control the monetization of the APIs. Monetization gives developers self-service access to billing and reports, catalogs and plans, and monetization-specific settings.
An API management platform developer portal should include the capabilities described in the following sections.
API Catalog and Documentation
As an API provider, you need a platform to publicize and document your APIs. Developer enablement services should allow an API provider to publish a discoverable catalog of APIs. An API catalog is also sometimes referred to as an API registry. Developers should be able to search the catalog based on various metadata and tags. The catalog should document the API functionality, its interface, how-to guides, terms of use, reference documents, and so forth. Information about the API versions available should also be included in the documentation.
Developer Support
Properly designed REST APIs are normally very intuitive for developers to understand. App developers can easily start using them for app development. Still, the API provider should provide resources that developers can use to build innovative apps. Good API documentation and accelerators in the form of test and development kits can help speed up the adoption of APIs. API documentation should not only describe the API interface, but must also provide how-to guides for interacting with the APIs. The developer portal can provide embedded test consoles that developers can use to play with an API and get a feel for it. Sample code that demonstrates the use of APIs can act as a quick start guide and be very helpful to app developers. App developers often look for device-specific libraries to interact with the services exposed by the APIs, such as downloadable SDKs within the developer portal.
Developer Onboarding
To start consuming the APIs, developers must register with the API provider to get access credentials. Developers can either sign up independently or as part of a company. The signup process should be simple and easy. Developers should be able to go through a self-registration process and view the APIs available from the API provider. Developers can then select an API product and register their apps to use it. After successful registration and approval, an API key is generated along with a secret to uniquely identify the app. The API key is also referred to as an app key or a client ID. The approval process may be automatic or manual, based on the terms and conditions and the monetization model setup. In a manual approval, a member of the API management team approves the registration request. The API key is generated only after successful approval of the app. In some cases, developers may form part of a company. In such scenarios, a key management capability is important so that API consumers can add, modify, or revoke the API keys within their organization .
Community Management
App developers often like to know the views of other developers in the community. They may want to collaborate and share their API usage learnings and experiences with one another. Blogs and forums form a major part of collaboration and community management. Developers may share their experiences with API usage via blog posts; such posts may need to be moderated by the API provider before they become visible to everyone. An API provider may also create a blog to share updates and future plans with the API consumer community. Advice and best practices on API usage may also be shared on blogs and discussion forums. A developer should also be able to report any issues with an API or its usage to the API provider’s support team. The developer portal may have a link to raise support tickets. Integrated blogs and forums can help build a truly dynamic community to enhance the use of the provider’s APIs.
API Lifecycle Management
API lifecycle management provides the capability to control how an API is developed and released to consumers. Published APIs can be can be used by consumers to build apps. They can report problems or raise a request for a new API feature. An API management platform should provide the following capabilities required for API lifecycle management.
API Creation
An API acts as a facade to interact with the back-end services. The API team should be able to design the REST interface for the API and create an API proxy to interact with the back-end services. An API proxy acts as a facade to securely expose the back-end services to its consumers. Policies attached in the flow paths of the API proxy should be able to implement security, traffic management, message translation, encryption, filtering, caching, orchestration, and routing. Once the development is complete, the API team must be able to deploy and test the API through a console. An embedded console to test APIs can be very handy and can help reduce development time. The API management platform should provide tools that enable the creation of the APIs and subsequently deploy and test them on an environment before they are published for production .
API Publication
Once an API has been created, it must be published to an environment before it can be discovered and consumed. The API management platform must therefore provide tools that can be used to migrate the APIs from lower environments and deploy to production. Once it is deployed to production, the API specifications and other details should be published in the developer portal for consumers to discover and use in their apps. In case of any incorrect deployment, the platform must provide the ability to roll back to a previously deployed version of the API.
Version Management
APIs evolve over time with newer business requirements. Hence, managing multiple versions of an API to support existing consumers is an important capability that must be provided by the API management platform. Version management should also provide the ability to deprecate and retire older versions smoothly. When an API version is marked as deprecated, the existing consumers should be notified though deprecation warnings. Deprecated APIs may continue to serve traffic from existing consumers. However, new consumers should not be able to sign up to use deprecated APIs. With proper notice and period, deprecated APIs should be retired and removed from the platform so as to avoid any maintenance overheads. The API management platform should therefore provide the capability to manage the retirement of an API.
Change Notification
Changes to an API may adversely affect its consumers. Hence, consumers must be notified of any planned changes to the API. Developers using the APIs should be made aware of any changes to the API. The API management platform must therefore provide a mechanism to notify API consumers of any API upgrades or outages. Notification can be made via email, SMS, or social media. Release notifications can provide updates about new releases and features added to the API. API consumers should be notified about planned or unplanned downtimes. An API developer portal can be used to send release and availability notifications to subscribed users.
Issue Management
The API management platform should provide API consumers with the facility to log issues found in the APIs. App developers consuming APIs must be able to report any issues or shortcomings related to their APIs. They should be able to raise support tickets and seek help regarding API usage. The issues can be reported through the developer portal. The API management platform should provide the capability to integrate defects reporting and issue management capabilities in existing systems within the enterprise.