So, how does a typical programmer spend his or her time at work, and how do we know?
There’s a long tradition of studying workers to account for their time and to look for possible productivity improvements. Starting in the early 20th century, pioneers such as Frank and Lillian Gilbreth, immortalized in the book Cheaper by the Dozen, did time and motion studies to reduce the time and effort needed for factory workers and office clerks to perform repeated tasks. In such studies, a researcher, equipped with a stopwatch, clipboard, and camera, observes a worker performing his or her normal work activities. The researcher then measures the time each body movement takes and looks for new possibilities that are quicker, more accurate, or less fatiguing. As one example, the Gilbreths observed surgeons fumbling to pick up instruments during surgeries and invented the now familiar protocol by which a surgeon asks an assisting nurse for each instrument as it is needed. This innovation both reduces the time surgeries take and avoids mistakes.
Of course, software development isn’t factory work or surgery. Time and motion studies focus on physical movement, whereas software development is mostly brain work. Nonetheless, the same study techniques can be useful.
In the early 1990s, Perry, Staudenmayer, and Votta ran a study, in the spirit of a time and motion study, with a group of programmers working on real-time switching systems [Perry et al. 1994]. Because there was no established methodology for studying programmer work practice, they tried two different methods: a diary study and an observational study.
First, the researchers asked some of the participating programmers to fill out daily diaries of their work activities, for the period of a full year. Their custom diary form had a box for each hour of the work day, in which the participant summarized what he did during that hour. In each box, he would write down what aspect of his assigned task he worked on (planning, requirements, design, coding, testing, etc.) or, if he did not work on their assigned task, the reason why (higher-priority task, blocked waiting on resources, or personal choice to work on something else). Amazingly, 13 programmers from four teams were willing to do this.
Based on this data, their biggest result was that the participants spent only 40% of their time working on their assigned tasks. The rest of the time they were either blocked or doing other work. The researchers noticed that, because of the frequent blocking, programmers would often keep two assigned tasks close at hand. That way, if they were blocked on one task, they could always switch to the other to keep from being idle.
Of course, given that the participants were recording their own time usage, it’s possible that their diaries were not very accurate. Even if every participant acted diligently about filling out the diary every hour on the hour, a one-hour time period is very coarse-grained—a lot happens in an hour! So, both to double-check the accuracy of the diary data and to get more detailed data, the researchers ran a second study where they shadowed programmers to see how they spent their time.
In their second study, Perry, Staudenmayer, and Votta directly observed five participants during their work days. They shadowed these developers for their entire work days, typically eight to ten hours long, for five work days for each participant, totaling an impressive 300 hours of data. During an observed work day, the researcher would ask to be treated like a student trying to learn the job and would prompt participants with questions such as, “What are you working on now?” when the answer wasn’t obvious just from watching [Perry et al. 1996]. These observations provided much a finer level of detail about what the programmers were doing than the diary study, down to intervals of three minutes.
From the observation data, they saw that programmers spent an average of 75 minutes of each work day in informal communication, which took the form of email, phone calls, voice mail, and face-to-face visits. This 75 minutes was in addition to their scheduled meetings for design reviews, code reviews, organizational updates, etc., which clearly also involve a lot of communication. Aside from incoming emails with broadcast announcements, the most frequent type of communication was face-to-face visits, which occurred about two to three times as often as other media. Programmers spoke with an average of seven different people throughout the day. The maximum seen in this study was 17 unique visitors in one day. Most of these interactions were short—68% under five minutes—but it was not uncommon for some observed phone calls to last a half hour and some in-person visits to last up to an hour. Clearly, this is a lot of talk!
If we naively think that programmers are productive when they are pounding out code at the keyboard, these results might seem alarming. After all, the studies claim that programmers spend the majority of their time away from their assigned tasks, a lot of that time just talking with other programmers. We might then wonder whether there was something wrong with the studies.
One concern is that the diary study relies on self-reported data. To reduce this risk, the researchers chose to overlap some participants in both the diary and observational studies. On many days, therefore, the researchers had both the participant’s self-reported diaries and the researcher’s observed data. As you might expect, programmers differed in the accuracy with which they estimated the time spent on tasks, sometimes over-reporting and sometimes under-reporting. However, the programmers often were not off by much: the average amount of over-reporting was 2.8% [Perry et al. 1996]. In many cases, the under-reporting resulted from ignoring the time lost to interruptions and blockages.
Of course, we might also be suspicious of the observed data itself. Intuitively, we would assume that the participating programmers were acting “on their best behavior” during the observations, or maybe even showing off. In fact, this phenomenon is so familiar in experimental psychology that it has a name: the Hawthorne Effect.
In the late 1920s, researchers at the Hawthorne Works electronics factory in Chicago ran experiments to see whether improving the lighting on the factory floor would make the workers more productive. They tried many different lighting levels over the course of many weeks. The funny thing was that, no matter how minute a lighting change they tried, the workers were always more productive! But, as soon as the study was over, productivity would slump back to its normal levels. The one common factor across all these experiments was that the workers knew they were the subjects of experiments. Today, we use the term Hawthorne Effect to refer to the tendency people have to improve their work behavior when they know that researchers are observing them.
So, were the participants in Perry, Staudenmayer, and Votta’s study influenced by the Hawthorne Effect? Since the effect is unavoidable, the answer is undoubtedly “yes” to some extent. So a better question is: how much inaccuracy did the Hawthorne Effect cause in their observational data? Because the researchers knew about the effect, they designed their study to mitigate it. First, they had several participants in the diary study who were not in the observational study, which allowed them to see how programmers being observed compared with those not being observed. Second, they made sure that several of the observed work days were not scheduled in advance. Because the researchers just showed up at random, the programmers could not arrange their work days to be as “interesting” as possible for the observer. In the end, these random days were not particularly different from the scheduled days. Finally, in both studies, all of the data was kept completely anonymous, which reduced the motivation for the programmers to make themselves “look good” for their coworkers, managers, or anyone else.