The original study that found huge variations in individual programming productivity was conducted in the late 1960s by Sackman, Erikson, and Grant [Sackman et al. 1968]. They studied professional programmers with an average of 7 years’ experience and found that the ratio of initial coding time between the best and worst programmers was about 20 to 1; the ratio of debugging times over 25 to 1; of program size 5 to 1; and of program execution speed about 10 to 1. They found no relationship between a programmer’s amount of experience and code quality or productivity.
Detailed examination of Sackman, Erickson, and Grant’s findings shows some flaws in their methodology (including combining results from programmers working in low-level programming languages with those working in high-level programming languages). However, even after accounting for the flaws, their data still shows more than a 10-fold difference between the best programmers and the worst.
In the years since the original study, the general finding that “There are order-of-magnitude differences among programmers” has been confirmed by many other studies of professional programmers [Curtis 1981], [Mills 1983], [DeMarco and Lister 1985], [Curtis et al. 1986], [Card 1987], [Boehm and Papaccio 1988], [Valett and McGarry 1989], [Boehm et al. 2000].
There is also lots of anecdotal support for the large variation between programmers. When I was working at the Boeing Company in the mid-1980s, one project with about 80 programmers was at risk of missing a critical deadline. The project was critical to Boeing, and so they moved most of the 80 people off that project and brought in one guy who finished all the coding and delivered the software on time. I didn’t work on that project, and I didn’t know the guy, but I heard the story from someone I trusted, and it struck me as credible.
This degree of variation isn’t unique to software. A study by Norm Augustine found that in a variety of professions—writing, football, invention, police work, and other occupations—the top 20% of the people produced about 50% of the output, whether the output is touchdowns, patents, solved cases, or software [Augustine 1979]. When you think about it, this just makes sense. We’ve all known people who are exceptional students, exceptional athletes, exceptional artists, exceptional parents. These differences are just part of the human experience, and why would we expect software development to be any different?
Augustine’s study observed that, since some people make no tangible contribution whatsoever (quarterbacks who make no touchdowns, inventors who own no patents, detectives who don’t close cases, and so on), the data probably understates the actual variation in productivity.
This appears to be true in software. In several of the published studies on software productivity, about 10% of the subjects in the experiments weren’t able to complete the experimental assignment. The studies’ write-ups say, “Therefore those experimental subjects’ results were excluded from our data set.” But in real life, if someone “doesn’t complete the assignment,” you can’t just “exclude their results from the data set.” You have to wait for them to finish, assign someone else to do their work, and so on. The interesting (and frightening) implication of this is that something like 10% of the people working in the software field might actually be contributing negative productivity to their projects.
Again, this lines up well with real-world experience. I think many of us can think of specific people we’ve worked with who fit that description.
Some people have objected to the “10x” label, making an argument like, “We had a super programmer on our team once. He was so obnoxious he alienated the whole team, and overall productivity was actually better without him.”
In general, any practical definition of a real 10x programmer has to include the effect the programmer has on the rest of the team. I have known super programmers who were obnoxious. More often, a supposed super programmer with an obnoxious personality was really an average programmer, or worse than average, using an obnoxious personality to hide a lack of performance. The true super programmers I’ve worked with have generally also been cooperative team players, although of course there are always exceptions.