Theory and Justification
The process of inventing, designing, and writing application programming interfaces (APIs) can either be seen as an artistic or as a scientific pursuit. Depending on your point of view, the API architect is an artist trying to change worlds or an engineer building bridges between them. Most people I know would rather be artists, because artists are associated with creativity, spontaneity, and beauty. However, a purely artistic approach has one significant problem: emotions are not transferable. They are extremely subjective. Explaining them to others is fraught with complexity. Although it is possible to create work that generates an emotion, ensuring that two people respond identically is beyond the capabilities, and probably also the goals, of art. As a result, an architect writing an API, just like an artist producing a painting, risks the possibility that the work will be misunderstood. In that case, developers using it will create a feeling very different from the architect’s intentions.
This might not be all bad. However, problems arise when the API architect decides, or is forced, to develop an API in a group. Immediately, various attributes of an API are at risk. For example, one of the most important attributes of an API is consistency, which avoids unpleasantly surprising the API users. For an API to be consistent, there needs to be significant coordination between members of the designing group. Coordination is difficult to achieve when each member approaches the work as an individual artist. There needs to be a shared vision. There needs to be a vocabulary that the group can use to describe the vision. And there needs to be a methodology that can be used to create a result that fulfills the vision. As soon as this is recognized, the API architect probably begins praying for the API design process to become an engineering discipline, rather than an artistic one.
Anyone who has ever tried to organize the work of 20 artists, and compared it to the work of 20 engineers, can easily understand why.
DESIGN BY COMMITTEE
I designed and wrote most of NetBeans’ APIs from its earliest days. At the time, we strongly believed that a good API cannot be designed by committee. That is why the goal for the other developers was mainly to use the designed API, write implementations of their features against it, and potentially comment on and improve what already existed. But gradually some of the other developers started to create their own APIs too.
I monitored such attempts closely. I commented on them, occasionally telling the others to do something differently or to remove pieces that looked odd to me. I sometimes even rewrote them myself and insisted that they use my version. However, I found myself in an increasingly uncomfortable position. Although I knew how I wanted the APIs to look, I was unable to explain my vision properly. Nor could I explain why they should consider my advice useful, because my colleagues did not always want to listen and follow my advice. I could use my right to veto their work, but that did not solve the problem either. I felt that something was wrong. However, I was unable to explain what or why. We lacked a shared set of terms, which limited our communication. This situation was inevitable because the APIs had been designed by means of a purely artistic approach.
I found proof of this a year later. One of the APIs developed further and, as NetBeans is an open source project, it attracted an external developer. At first, the external contributor worked on bug fixes. Next, he started his own subproject and designed its API. The original owner of the API came to me and asked me whether I could help him find reasons why the subproject’s API was turning out badly: he did not like the newly created API and wanted to prevent it from being integrated into his project. However, he was not able to formulate what was wrong with it. At most, he managed to say that it did not adhere to his original vision. That was absolute déjà vu for me. I used to tell him that his work did not fit into the NetBeans API’s vision— or at least my own interpretation of it!
Although it took me a while, after a few years of designing APIs and working with others to design APIs that I liked, I came to realize that API design can be approached as an engineering discipline. Although it’s not obvious at first, this discipline does have a lot of objective attributes. As a consequence, it’s possible to turn the API design process into something that engineers can understand—a kind of science.
Every real discipline needs a theory behind it: a theory that defines the world that forms the subject of its domain. Such a world needs to be sharp and clearly defined. The less sharp it is, the less rigorous the science is, and the more it becomes an art. This chapter defines the world of API design. It identifies the various objects you have to deal with in API design. It analyzes situations that you have to face during the process of designing an API. Moreover, it begins to build a common vocabulary that people knowing the same theory can use to better identify the objects of the API world and their relations. Based on these fundamentals, we can later build in more complicated and less obvious conclusions about the subject of API design theory.
It’s hard to write good APIs that can be consumed by a wide audience of users, especially an international one. Everyone has their own style of understanding and their own way of viewing problems. Satisfying all of them is hard. Satisfying all of them at once is usually impossible. Moreover, if your API is targeted to an international audience, it needs to deal with various cultural differences. That’s why writing good, widely approachable APIs is hard.
Writing a book for an international audience is hard as well. Again, the matter of personal and cultural preferences influences the way people want to read. Some would like to understand the background first. Others would like to jump to examples directly to find out if the whole thing is useful to them or not. Satisfying both these camps at once seems impossible. Ultimately, as Edsger W. Dijkstra wonderfully describes in his text, “On the fact that the Atlantic Ocean has two sides,”1 some people believe that a theoretical approach is difficult and boring and that, by extension, practical examples are much more interesting. They are probably right, at least in their cultural context. On the other hand, another group of people would rather spend time building a common vocabulary and increase their understanding of the world being explored step by step. This is difficult without a thorough introduction. Without it, terms can have dual meanings, often leading to confusion. Obviously, it is hard to satisfy both camps at once. However, I will do my best to make everybody happy.
A STABLE API
One of the basic terms in the world that our discipline wants to explore is stable API. I had been using this term quite a lot when talking to various people in the NetBeans project, without expecting any problem. All along, it seemed to me to be a word with a clear and obvious meaning.
Then I listened to an explanation of the term by one of my colleagues. He said that the API is stable if and only if it will never undergo change! Although a stable API is likely to have some kind of “stability,” it will still need to change sometimes.
I face this kind of misinterpretation all the time, where a term from the basic API vocabulary has different meanings for different people. Of course, this ruins the whole conversation. If people think they understand each other, when in fact they do not, then it is better not to talk at all. The images that crystallize while talking are completely divergent, and as such are absolutely useless for communication.
That’s why I believe it’s good to spend some time defining basic terms.
To avoid confusion about terms, this part is dedicated to vocabulary building and analysis of the general aspects of API design. It builds a vocabulary of basic terms, describes the motivations of the whole API design effort, and outlines the main goals of the design process. If this isn’t your preferred style of learning and if you believe you don’t need this elementary material, feel free to jump directly to Part 2, which contains many more code samples, tips, tricks, and various hacks. Don’t be surprised if the purpose of some of them seems strange. That might be a sign that you’re missing the background material discussed in this part. Also, if you’re eager to acquire more practical advice on tools, compilers, and so on, feel free to start reading Part 3. However, again, keep the previous warning in mind: if the advice given doesn’t make sense, that might be due to a missing piece of understanding relating to the API design theory behind it.
Without further ado, let’s dive into the theory. First we’ll go over the basics, which will help us all to get up to speed. To get us in the mood for proper API design, let’s start with the most basic questions that address the why, what, and how.
________________
1. Edsger Dijkstra, “On the fact that the Atlantic Ocean has two sides” (1976), http://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD611.html.