APIs are important from different points of views. The business looks at APIs from a revenue-driven angle whereas engineers see technical benefits. Arguments on both sides must be considered; both sides must understand each other’s desires. This chapter gives an overview of the arguments on both sides.
The Business Value of APIs
Businesses look at APIs to reduce costs, open their system to a broader audience, or monetize services. Although I am an engineer, I do get involved in business-related discussions when I attend meetings with enterprise customers. Those discussions are usually driven by engineers who seek advice on how to leverage my company’s product in order to satisfy business needs.
Omnipresent : Any user should be able to consume services at any point in time and from any kind of device.
Mobile first : Similar as above but with a focus on being present in app stores. Even if it is just a simple app, they want their company’s name to be found.
Integrations: Enable easy user on-boarding and third-party systems integrations.
Modernization: Understanding that times change and monolithic systems that use closed and undocumented interfaces are difficult to maintain, upgrade, replace, and support.
Automatization : This may sound like a technical requirement, but businesses are aware that a modernized infrastructure can only be managed if processes, such as testing, upgrading, and deployments are automated. Automatization is possible only if APIs are available for those tasks.
Monetization : Monetize the usage of APIs. Some businesses can provide data for which third-party developers are willing to pay.
Not a single item of above was mentioned by accident or without reasoning. Let me share the details for each one.
Omnipresent
The business requirement for “omnipresent” could be as simple as this: Users need to be able to fulfill their needs within our application, independent of time and location.
The interpretation could sound harmless and be communicated as such: We need to provide a unified user experience across all online platforms , including mobile and desktop applications. The user’s workflow should not be interrupted when switching apps or platforms.
Components connected via APIs
“Omnipresent” references applications enclosed by the dotted lines. Everything within that square should be usable “as one” app and practically everywhere available. Here is an example use case: a user is on his daily commute home from work and opens an app on his mobile phone. He checks account details and realizes that he needs to update them. To do so, he opens the same app on his tablet and expects to be able to continue his work where he left it on the phone. Once he gets home, he turns on his desktop computer, opens the same application, and finishes the work.
If you are an engineer, you may have an idea of the challenges associated with this scenario. For everyone else, I will explain what those few sentences mean. Applications running on different platforms are implemented using different programming languages, often accompanied by proprietary protocols. For example, an application implemented for Windows cannot be installed on an Android platform. If it’s about desktop applications only, existing technologies such as CORBA1 can be leveraged in some cases, even though it is heavyweight and complicated. Unfortunately, that technology is not suitable for mobile applications.
On the other hand, if it’s about mobile applications only, newer standards such as Bluetooth2 and NFC3 can help. Sadly, these two technologies are not generally available for applications installed on desktop computers. Running out of options, a completely different solution, and then applications that communicate directly with each other, must be found.
Businesses have discovered that APIs are a way out of this issue. APIs enable indirect communication between applications, no matter which platform they are running on. Figure 3-1 indicates that communication via a component named ApiProxy (more details on it in Chapter 6). The ApiProxy provides RESTful APIs and all applications can leverage them. In combination with a protocol that specifies how applications can be addressed and messages exchanged, the problem can be solved and business requirements supported.
Mobile First
“Mobile first” expresses the desire to support mobile applications with a higher priority than desktop applications. Businesses are aware that users expect availability on any device, be it a phone, a tablet, or even a watch. Another association with mobile applications refers to geographical independency. Mobile applications are installed on devices that are carried around all over the globe. Independent of the current location, users expect their applications to be available and work just fine.
Mobile-first requirements are not only driven by businesses that provide online services. They also apply to hardware providers such as in cars, cameras, hi-fi systems, lights, washing machines, TVs, and many others. The availability of mobile applications in these areas becomes more and more relevant. There are multiple reasons for that, as far as I have understood it from customers. On one hand, users want their devices to be connected with systems such as Amazon Alexa and Google Home. On the other hand, third-party vendors want to provide services based on those abilities.
Businesses are aware that providing at least a simple and well working mobile application is often a minimum requirement to attract users, and with that, customers. Businesses are also aware that APIs are the only realistic way to support these apps. APIs become the enabler to get into the mobile application market and, with that, participation in markets such as house automation.
Integration
Integration is always a big topic. Businesses want to be able to integrate with other systems. They also want other systems to integrate with theirs. These cases do not only require a business to provide APIs for its own and potential third-party usage. It is also important to be able to leverage external third-party APIs. Otherwise, a one-directional system gets created, which does not surface all of the features that would be possible in a system supporting both directions, similar to a car that does not have a reverse gear. It will only take you so far before you get stuck!
Here are typical cases that have been brought up during discussions:
Easy on-boarding of new users. In many cases, this refers to social login, which is a process that enables users to reuse an existing social platform account (such as an existing Gmail address) for creating a local account in the business’ system. This feature is purely driven through bidirectional API integrations. Businesses know that a missing “... or use your existing xyz-platform account ...” button on their web site or their mobile app may cost them potential new users. Having to create yet another username/password-based account is not an acceptable process for many users.
In other cases, easy on-boarding refers to the process of users switching apps or devices. Let’s say a user logged into the system using social login; he now has a session within the businesses system. If the same user opens a second app, his expectation is to reuse the existing session. In other words, he should automagically be logged in. A similar expectation can be observed when opening the same app on a different device. Here’s an example: my 10-year-old son used to play a game on my wife’s phone. One day, he got his own phone. When I told him we now have to find out how to transfer his gaming data to his phone, he answered, Dad, just place the phones next to each other and the data will move over.
The last case of easy on-boarding involves different platforms. It’s the same idea as above, but in this case, users want to seamlessly switch from mobile apps to web applications on their desktops. Many of you may have seen this feature with WhatsApp4 or WeChat.5 These mobile applications allow users to extend their sessions to web applications by simply scanning a QR code6 using their mobile app. Nothing else is required. With that, switching from the mobile app to the web application only takes one click!
All these scenarios are API driven!
Do it all here use cases. Businesses do not want users to leave their application due to what I call “missing completeness.” I refer to “completeness” like this:
Enable users to complete all their needs within one application!
Businesses are usually aware of their competitors. They are also aware that the quality of their own online services may be one of very few differentiators. Here is an example.
The fictional company SaschasSlowPackages is in the business of moving packages from A to B. This company provides a mobile application with just one feature, tracking packages. Other information, such as store locations, office hours, package prices, is available on a static web site only. This mobile application has almost no value and can’t count as a competitive advantage. To find information about sending packages with SaschasSlowPackages, customers must switch between the mobile app, the web site, and other random online locations.
In comparison, the fictional competitor DirksFlyingParcels, which is more or less a clone of SaschasSlowPackages, provides a feature-rich mobile application. This application has all kinds of features: tracking packages, showing store locations, calculating shipping prices in different currencies and with different options, integrating with Google Home (When will my DirksFlyingParcels package arrive in Vancouver?), chat-based communication with the parcel’s sender, and feedback systems.
Most customers will choose DirksFlyingParcels simply because the non-functional features are much better covered.
Businesses are very much aware of this. The most prominent example that I am aware of is the Chinese-based app WeChat. My Chinese colleagues tell me, that WeChat has everything they need. They can chat, they can make calls, they can pay, and they can use city services, WeChat even counts steps! There seems to be hardly any need to switch to another app.
Without knowing more details, it is pretty much a given that the system behind WeChat uses (and provides) many API-based integrations. If you look at Figure 3-1 again and concentrate on the lower part, you can see that multiple components are connected to each other. Although it is just a very small example, bigger systems are not much different from an architecture point of view. Each connection represents messages that are being exchanged via APIs.
Modernization
Modernization is a general topic, nothing much to say here. However, businesses are aware that new technologies are born every day and older technologies move into IT heaven. Discussions with customers often go like this: Sascha, we have this new product and we need to integrate it with our backend systems using APIs. How do we do that and what do we need? The technology behind “we have this new product” changes over time but “using APIs” is a pattern that has been around for a while and has increasingly become more important.
What I have observed over the last few years is a general increase of products, technologies, and tools that are very much dependent on APIs. The next section on automatization will talk about it more, but the complete build process of software products can nowadays be automated using APIs only. Just two or three years ago that wasn’t possible, at least not in general!
Due to these trends, businesses have realized that all of the new features must be API driven!
Automatization
In the past, businesses usually did not care too much about the process involved in producing software. They also did not care about processes that were not automated. Needless to say, that those times have changed. With new types of expectations, newer technologies, and faster turn-arounds of product versions, the process of creating software has gained visibility into all layers of executive levels.
Scalability : Depending on the current demand, systems need to scale up and down.
Upgradability: Upgrades into production environments with no downtime.
Testability: Whatever code it is, automation has to include tests.
Isolation: Different components should have the least possible dependencies to each other.
Configurability: Configurations for any environment
Deployability: Deployments into any environment
Version-ability : Any system needs to be versioned and rollbacks must be possible.
You may say that none of these terms are especially new in IT. No, they are not. But in the past, each one had manual steps associated with it. Here is an example: until recently I heard the term white gloves action often, referring to a few manual steps during a software installation or upgrade procedure. Unfortunately, executing these few manual steps took up to 12 hours, with no option of rolling back any of them. Luckily, those days are gone or at least they are disappearing. With good automatization coverage, the same process takes about 30 minutes. This is not only an overall shorter timeframe; this also eliminates many potential errors.
Monetization
This is a very important topic, although it does not come up too often during discussions I have. However, businesses are very interested in creating environments that can support APIs for monetization purposes as soon as their systems are ready. In the end, this comes back to the readiness of the omnipresent infrastructure. To monetize APIs, a system needs to be able to audit transactions from an initial request all the way until a response gets returned to the requesting client. Since API-based systems are not directly coupled with each other, they still have to be designed so that values, such as RequestIDs, can be tracked at each step of the way. An example of a business requirement could be this:
Any request needs to be audited and logged in a way so that invoices can be created based on API usage per client and per user.
This requirement may sound simple, but if you look at the small example in Figure 3-1, you can see that it has a big impact on all system components. All APIs have to accept the input and output values included within any request. This has to be considered right from the beginning.
Without an explicit monetization model, APIs are indirectly monetized through products that provide access to audiences that weren’t available without the APIs.
Technical Value of APIs
First of all, APIs are the foundation for realizing business requirements, especially the ones listed above. This is a very high-level statement, but it is important to be conscious about it (please close your eyes and say it to yourself, very slowly!). Once that becomes the mindset, nobody will be scared of new requirements! The answer will be We do not have an API for that, but, we can build one!
I have been in many discussions that interpreted missing APIs as broken products rather than requirements that had not been requested yet. In the right environment, new APIs can be published in a fast, reliable, scalable, well-documented, and well-tested way to support business value within a short period of time.
For some readers, the technical value of APIs may be very obvious, specifically for those who have experience with API-based systems. But for readers who tackle this topic the first time, it may be difficult to see the light at the end of the tunnel. I will try my best to nail it down.
Here is something to remember:
APIs are contracts!
Yes, they are contracts! Chapter 4 talks more about this, but it is so important that I must mention it frequently. Once the contract is written, there shouldn’t be any questions and all involved parties can start working. Just like a contract between a homeowner and construction workers, this contract should define requirements that all parties have to adhere to.
These contracts define sets of rules that dictate how a specified business value can be accessed. These contracts are expressed as human-readable and machine-readable documents. They are implemented as programming language-independent interfaces. Here is an example, which is explained in multiple steps.
- 1.
The business owns a database that contains lists of users and the products they have purchased in the past.
- 2.
The business wants to make this data available.
- 3.
The data should be made available to different audiences for different purposes.
- 1.
Enable users to retrieve the list of their purchased products.
- 2.
Enable third-party partners to retrieve anonymized lists of all purchased products.
These are two different requirements for the same dataset. This could be a challenge, but not with APIs!
API 1: The List for Users
Human-readable documentation: To retrieve the list of products, an authenticated user and an authorized application are required. The communication has to be encrypted. The used message protocol is based on HTTP and the accepted HTTP method is GET. Lists are only produced for the authenticated user. An application needs to provide an oauth access_token as a credential associated with the user and the application itself. The list is returned as a JSON message.
API 2: The List for Third Parties
Human-readable documentation: To retrieve the list of products, an authorized application is required. The communication has to be encrypted. The used message protocol is based on HTTP and the accepted HTTP method is GET. Lists are produced with no user information. An application needs to provide an oauth access_token as a credential associated with the application itself. The list is returned as a JSON message.
Judged by the documentation, the differences are very small. Judged by the responses, the differences are huge! Whereas API 1 may return a list of tens of values for just one user (see the “user” in the response body), API 2 may produce hundreds of values without user context.
From an API point of view, it comes down to differences only in the Path component of the machine-readable documentation: /list/products/users vs. /lists/products. In this example, two APIs have to be maintained to support two requirements.
However, in a real-life scenario these APIs could be merged. This is due to the fact that one requirement stated access_token associated with user and application and the other stated access_token associated with application. This means that API implementations are able to distinguish between access_token associated with users and access_token associated with applications only. This reduces the number of required APIs by 50%. And with that, maintenance, testing, documentation, and scaling tasks are also reduced by 50%.
The documentation could be changed to the following.
API: The List of Products
Human-readable documentation: To retrieve the list of products, an oauth access_token is required. The communication has to be encrypted. The used message protocol is based on HTTP and the accepted HTTP method is GET. Lists are produced without user context unless the given oauth access_token is associated with an authenticated user. The list is returned as a JSON message, either with or without user information.
- 1.
A missing or invalid access_token : The response would include HTTP status code 401 (unauthorized) and a message containing invalid_request.
- 2.
Unsupported http method: The response would include HTTP status code 405 (method not allowed).
Foremost, the API consumer can be any type of application, independent of the programming language.
Any HTTP-capable testing tool can be used. No specific programming language is required.
Components providing interfaces of different programming languages
Communications via APIs, on the other hand, have no notion of language whatsoever. Components can be replaced, and implementations can be updated. As long as the APIs stay the same, there is not even a need to share the information about updated or replaced components.
Another topic is testing!
The API is well defined, and all types of requests and responses are documented. Testing tools that are capable of processing machine readable documents can generate code stubs. These code stubs can be completed, and automated tests can be executed. Due to the nature of APIs not exposing or documenting implementation details, they can be tested using a black box8 approach. This reduces the number of required test cases and eliminates the need for manual tests. Except for sanity checks, APIs do not need a lot of personal love!
Integrations: This is the top reason for using APIs. I have not seen any other topic named as often as this one. Enterprise systems are fully stacked with all kinds of products. All of these products were acquired from different vendors that use different technologies and were not designed to work hand-in-hand with other products. Needless to say, customers still want those products to work together and exchange messages using one way or the other.
The only common feature practically all products share is the ability to consume and provide APIs. With that, integrations are possible and actively used. Of course, it is not always as straightforward as it may sound. There are cases where products are flexible enough to be modified, so that required APIs can be made available. But sometimes that is not possible. In those scenarios ApiProxys come into play. They are made to be adjusted to any API and take on the role of a mediator. They receive messages on one API, translate it to another, and forward the message to the recipient.
Either way, with or without ApiProxy, APIs enable integrations and with that business requirements can be satisfied
Upgradability/deployability : Upgrading and deploying software components are very different tasks. However, from an API point of view these two tasks are very similar. Customers usually do not specifically require APIs as such in this context, but they require command-line interfaces (CLIs). CLIs are used for integrations with build tools and to eliminate the need for manual actions. Even in this case APIs are the enabler. The exciting part is that those APIs can support command-line interfaces but also UI-driven tools. One API, multiple use cases supported!
Configurability: Configuration files are persisted and managed using version control systems such as Git.9 In conjunction with online services such as GitHub,10 these files can be managed via APIs. Having the APIs available, it can be imagined how these APIs can be used to move configurations onto a software component.
Testability: I have spoken to developers who work in a fully automated environment. They finish their implementation, including tests, and submit the code to the version control system. Once that is done, they do not even known when and how their code moves into the production environment. They will only hear back if a test has failed during the automated deployment process.
Version-ability: This does not only reference the ability of defining version numbers after a configuration has been applied to a software component. No, this also refers to the idea of activating and deactivating different components without manual intervention. A typical use is the selection of a specific deployment by version. Workflows can be supported this way, such as deploy version 1.2.00, test it and tag it, if successfully tested, but remove it and redeploy the previous version if the test fails. This is another use case where APIs are the enabler!
I hope this section helps you understand the importance and relevance of APIs from a technical point of view. I encourage you to take some time and experiment with APIs using existing tools and platforms. After a few successful tests, the whole picture will come together nicely. I can almost promise that!
Business Requirements vs. Technical Requirements
Most (verbal) fights that I have witnessed during my IT career were fought between members of business and engineering teams. Product owners and architects want a slick and easy-to-use application. These applications should also be safe to use, reliable, maintainable, modularized, future proof, scalable, modern, well architected, documented, and tested. Generally, product owners and architects are on the same page. But for some reason, product owners will sometimes say something along these lines, which emphasizes their different views:
We have a release date and we need to get something out of the door. Please come up with an MVP (minimum viable product) that I can take to my boss!
My personal reaction is this: Really? How is the proposed product not an MVP already? And why do we still talk about release dates in a CI/CD11-driven world?
We have scheduled the first public flight with passengers of SpacePlane-X75 for September. The engines won’t be fully tested by then and cannot be started if the outside temperature is below 25 degrees Celsius. However, we cannot move the date. Please come up with an MVP that I can take to my boss.
OK, I got it, here it is: preheat the engines and do not run them with more than 80% of their capacity. That should take us off the ground and three months later we will fly again to show off what we can really do!
Topic: Authentication = No redirect wanted
Business: We want to use the authorization_code flow but do not want users to be redirected to a different location.
Engineer: It is essential to redirect to the third-party server to highlight to users that they are not sharing their third-party credentials with our system.
Business: Redirects scare users and give us bad ratings in the app store.
Conclusion: An iFrame will be displayed on the current web site that contains the third-party server login page. Unfortunately, this will leave users skeptical since they cannot identify the owner of the login screen. Critical users will not consider using your application in the future.
Recommendation: Redirect users to the third-party server. Users need to see the security-icon and who’s hosting the login page in the browser URL. This addresses privacy concerns.
Topic: Authorization = No explicit consent wanted
Business: We do not want users to see a consent page.
Engineer: Third-party applications need an explicit consent by our users before sharing personal details.
Business: Most users are not aware of that anyways and the consent screen will distract them. This will give us bad ratings in the app stores.
Conclusion: Authorization flows will be modified to skip the consent screen; third-party apps will be configured as preauthorized.
Recommendation: Display the consent screen at least once! Whenever you empower users, they feel respected and in charge of their own data, leading them to trust us more.
Topic: Session = Never-ending session wanted
Business: We do not want users to log in more than once until they actively logout.
Engineer: We cannot trust any type of token-based credential with an endless lifetime if no proof-of-possession is supported.
Business: You know what we want. Find a solution.
Conclusion: Authorization servers will be configured to issue a session token that has an endless lifetime. Leaking tokens enable unauthorized entities (either applications or persons or both) to impersonate users. If users ever discover this, they will abandon this company.
Recommendation: Issue a short-lived token to consume resources and a long-lived token to retrieve a new short-lived token. Exchange the long-lived token only via backchannel communication. Implement a flow that proves that users still own the session but avoid any knowledge-based techniques (most famous one: What is your mother’s maiden name?). Instead, have users prove the ownership of devices. For anything users feel is important to them (like bank accounts) they will not mind having to prove that they are the eligible owner of almost anything.
These three example discussions come up over and over again. Usually it is the business side that wants engineers to find solutions that enable custom message flows, for example, in OAuth 2.0. When I get involved, I do not feel that I am answering technical questions. I mean, the questions I get are always based on technology. But they reflect requirements that were stated by businesses, and with that, an implementation change is not a change in technology but an enablement for the business.
In the beginning, I interpreted these types of changes/requirements as a kind of disconnect between businesses and their engineers. But over the last few years I have realized that I am wrong. Although there will always be conflicts, especially when required modifications raise privacy or security concerns, in the end, technology is not available for the sake of technology but to make businesses happen. And what I have learned from that is this:
APIs are the foundation for realizing business requirements!
Summary
It is important for any involved party to know why APIs are relevant in their specific case. Without knowing this, wrong decisions will be made, and the success of any API project will be in jeopardy. The overall business goals of reducing costs and gaining revenue will not be achieved. If this chapter was a success, it should be easier to follow arguments toward systems that are fully API-driven.