1. Why Programmers Seem Unmanageable
Programming has been a Serious Profession for well over 60 years. The number of people now employed as programmers is in the millions in the United States and in the many millions throughout the world. And those numbers do not include the legions of students and hobbyists who program computers seriously but are not employed or paid to do so.
In spite of the history and the magnitude of the numbers, software engineering has a reputation for being unmanageable. There are several reasons.
First, programming as a serious profession is different from related engineering professions, such as electrical or civil engineering. Since 19681 attempts have been made to apply the term software engineering to the art of programming. But writing a new program from scratch is much more akin to writing a novel than to the established practices of civil or electrical engineering. New programs frequently begin with the equivalent of a blank sheet of paper, whereas engineering projects are typically assembled from libraries of components and rigorous codes of acceptability. We shall refer to software engineering as programming in this book because it continues to be much more a craft than a rigidly defined engineering discipline.
1. It was in 1968 that the term software engineering was coined to describe “the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software.” See “Software’s Chronic Crisis,” Scientific American, September 1994.
Second, anyone can be a programmer. You do not need a formal education to be a programmer, and there are no required certification standards or tests.2 All you need is a job as a programmer.3
2. The Association for Computing Machinery (ACM) had a professional certification program in the early 1980s, which was discontinued. The ACM examined the possibility of professional certification of software engineers in the late 1990s but decided that such certification was inappropriate for the professional industrial practice of software engineering. A similar review at Stanford reached the same conclusion: “[Licensing] will not be helpful at all and [will] actually cause more harm than good. The ‘body of knowledge’ necessary for licensure is too large for any test, since software engineering is so broad and still changing due its immaturity. Not only will it be ineffective, but since it will create a false sense of security, it has the potential to bring about more harm than ineffectiveness. We thus stand with the ACM against licensing of software engineers.” See https://cs.stanford.edu/people/eroberts/cs181/projects/2010-11/Licensure/indexba53.html?page_id=2.
3. Many organizations, such as Microsoft, Cisco, and others, provide certification courses and tests that are widely used throughout the industry, but these apply to specific technologies or domains of expertise. They may be required for a specific job but are not required throughout the industry.
Third, and in part as a result of the first two reasons, though many steps have been taken to formalize the process of software engineering (e.g., CMMI4 Levels 1–5), these steps have had minimal impact. Much of the software that continues to be developed by the legions of programmers does not follow such formalized frameworks. And even when it does, the result has improved the process but has not transformed programming into an engineering discipline. Plus, the formalized frameworks address only the process of making software, not how to manage programmers. Following processes only minimally makes the job of managing programmers easier. Programming managers are still mostly left to their own devices to manage their staff of programmers effectively.
4. Capability Maturity Model Integration (CMMI) is a process improvement approach developed by the Software Engineering Institute (SEI) that provides organizations with essential elements of effective processes that ultimately improve their performance. See www.cmmiinstitute.com.
The fact is that while there are lots of books, articles, and Web sites about software engineering and managing the software development process, there are scant examples of how to manage programmers effectively. As any manager of a baseball team will tell you, managing the mechanics of baseball is easy compared to managing the personalities. So it also is with managing programmers.
Managing programmers has been a challenge since the early days of computers, as illustrated by this 1961 quote from Grace Hopper, who became one of the world’s first programmers during WWII:
Programmers are a very curious group . . . They arose very quickly, became a profession very rapidly, and were all too soon infected with a certain amount of resistance to change. The very programmers whom I have heard almost castigate a customer because he would not change his system of doing business are the same people who at times walk into my office and say, “But we have always done it this way.” It is for this reason that I now have a counterclockwise clock hanging in my office.5
5. Quoted in G. Pascal Zachary, Show-Stopper!: The Breakneck Race to Create Windows NT and the Next Generation at Microsoft (Free Press/Simon & Schuster, 1994).
A first step in managing programmers is to better understand them. What is it that attracts these millions of people to the “art of computer programming”? The answer is sometimes quite simple—it’s a job that pays pretty well and lets you work indoors all day. However, as many programmers can tell you, the answer is usually not quite that simple. The people who would give that simple answer often don’t last as programmers.
In fact, it takes a certain kind of person to be a programmer and a very special kind of person to be a great programmer. Understanding what it takes to be a great programmer begins with understanding what programmers do.
What Do Programmers Do?
First and perhaps foremost, programmers have fun! Fred Brooks, in one of the classics of software engineering, The Mythical Man-Month,6 wrote brilliantly of why programming is fun:
6. Frederick P. Brooks Jr., The Mythical Man-Month, Anniversary Edition (Addison-Wesley, 1995; originally published in 1975). This classic book on software development is a must-read for anyone who is managing programmers and/or software development projects.
First is the sheer joy of making things. . . .
Second is the pleasure of making things that are useful to other people. . . .
Third is the fascination of fashioning complex puzzle-like objects of interlocking moving parts and watching them work in subtle cycles. . . .
Fourth is the joy of always learning, which springs from the non-repeating nature of the task. . . .
Finally, there is the delight of working in such a tractable medium. The programmer, like the poet, works only slightly removed from pure thought-stuff.
If having fun is what most programmers do, you may begin to understand why managing programmers is so challenging. If you are being paid to have fun, why would you want to be managed? Being managed takes part of the fun out of the work!
This also explains why it is often so hard to work with programmers. Before there were computers, many programmers would have been engineers, accountants, or educators. About 50 percent of the programmers we have hired over the years have fit into this category. However, the other 50 percent are harder to categorize. Many would have been artists, musicians, writers, or other “right-brained” people and are essentially more freespirited. And more surprisingly, this right-brained group often produces the more talented programmers.7
7. Several industry leaders have openly expressed their opinion that musicians make terrific programmers. As a musician himself, Mickey heartily concurs with that opinion.
Additionally, the fact that programmers work in a medium that is “only slightly removed from pure thought-stuff” encourages a free-spirited, undisciplined approach to programming that makes managing such programmers almost impossible. There are no widely used tools that impose a rigid, disciplined, comprehensive approach to designing and implementing a program. Rather, all too often programmers are allowed to start designing or even coding a program with a blank slate. Sadly, there are still no widely used tools in 2019, a decade after we first drafted this paragraph for this book’s first edition!
It is the necessity to blend these two disparate groups of programmers into a cohesive, effective software development team that creates many of the challenges in managing. Managing engineers is relatively straightforward compared to managing a bunch of artists, musicians, and writers. Without externally imposed processes or procedures, when left to their own devices, many programmers will jump in and just start writing code, designing as they go. Managers must cultivate a software development culture built upon solid development practices, or programming projects will likely fail.
Though programmers can be divided into groups, the key to managing them successfully is to recognize that they are individuals. Programmers truly are quite different from one another, and you must endeavor to play to the strengths of each while striving to improve, or at least neutralize, each individual’s weaknesses. While this is a sound management principle in any field, it seems to be much more important when managing programmers.
Even with good software practices and development processes, how do you know progress is being made when the work product is intangible? In almost all software, the actual tangible results of a program (i.e., the printed reports, the data output, or even the user interface) are not directly proportional to the state of completion of the actual program. One of the great systems programmers Mickey worked with at Evans & Sutherland designed and wrote every line of code of a very complicated device driver over a period of many months before compiling the program even once. At the other extreme, when Ron arrived at Fujitsu, the programming team had been telling management that the product was one week from being functionally complete every week for the previous three months. In both cases, there was no meaningful measure of progress that could predict successful completion. Thank goodness for the emergence of agile’s iterations and sprints!
Even worse, a program can produce the desired results and be so poorly designed and/or executed that it cannot realistically be enhanced or maintained. These hidden, or intangible, aspects of the program may not be visible even to a seasoned programmer.
And frankly, many—perhaps most—programmers work on making changes to existing programs, the majority of which someone else wrote. Even if they themselves wrote a program, it was probably more than six months previously, so Eagleson’s Law applies: “Any code of your own that you haven’t looked at for six or more months might as well have been written by someone else.” Which is to say, the code is gnarly and nearly impossible to figure out, and, again, there’s no meaningful measure of progress that can predict successful completion.
Similarly, for too many managers (especially less technical managers, such as CEOs or CFOs), prototypes appear to be “done.” The fact that software is so intangible makes it more difficult to judge its state of internal completion, regardless of the quality of the results the program produces. Through experience, tools, and looking “under the covers,” managers must be able to gauge the actual progress of a program.
Ultimately, it is best to avoid these problems by hiring great programmers—those special programmers who bring discipline and process to the art of producing computer programs. Are these great programmers artists, engineers, or craftspeople?
While there is much talk of the art of computer programming, few programmers would consider themselves artists in the purest sense of the word.
Similarly, while software engineering is a goal that we aspire to reach, few programmers today really are software engineers in the literal sense. The IEEE8 provided a formalized certification program for software engineers from 2004 to 2014, and certification courses are still available from a variety of vendors. However, in our experience, not only are such programs ignored by most professional programmers, but these certification efforts are largely unknown outside of academic circles and a few companies. There is scant effort or desire among rank-and-file programmers to pursue the discipline of software engineering.9
8. The Institute of Electrical and Electronic Engineers (IEEE) professional society offers the Certified Software Development Professional (CSDP) certification, which is mostly based on practices outlined in the SWEBOK (Software Engineering Body of Knowledge, www.computer.org/education/bodies-of-knowledge/software-engineering). Whether such certifications should be sought is a topic that generates considerable debate, but it’s worth noting that certification is the norm in “real” engineering fields.
9. For a pragmatic look at certification, see Jack Ganssle, “More on Certification,” September 7, 2005, www.embedded.com/electronics-blogs/break-points/4025582/More-on-Certification.
What about “craftspeople”? A core group of programmers think that a better metaphor for programmers is that of a craftsperson.10 A craftsperson doesn’t start out as a master craftsperson but rather must spend years as an apprentice and years as a journeyperson and have proven skills and a portfolio to earn the title of master craftsperson. Knowledge, experience, and a proven track record provide a means of tangible certification that cannot be easily assessed in a formal certification program. We think that this metaphor, more than others, suits the type of programmer we like to call a “great programmer.”
10. Pete McBreen, Software Craftsmanship: The New Imperative (Addison-Wesley, 2001).
What is it that makes a great programmer? It is much more than just raw programming talent. In fact, raw programming talent often detracts from the skills of the great programmer. Rather, a great programmer is a master of the medium and is methodical and disciplined. That is, a great programmer has an intuitive sense for structuring code and programs, is disciplined enough to design before coding, and writes code in a minimal amount of time that is clear, concise, functional, of high quality, and produces the desired result, often with tests written to ensure the code works. In other words, a great programmer is a master craftsperson of the medium.
Programmers who are motivated primarily by schedules, management pressure, or money are usually not great programmers. Most great programmers are actually motivated by a higher calling: making a difference in the world and making programs or products that people actually use. Great programmers want and need to work on projects that make a difference in the world. They want to feel their work is meaningful, even if only in some small way. Great programmers gravitate to companies and projects that fulfill their higher calling. Great programmers care intensely about what they are doing and work beyond normal limits to deliver desired results.
A great programmer can often be an order of magnitude (i.e., ten times or greater) more effective than an average competent programmer.
But there are too few great programmers in the world for every project team to have even just one. Furthermore, most teams cannot tolerate more than one or two great programmers in their midst. We find that we rely primarily on ordinary programmers, not truly great programmers, to create most programs and projects. The ordinary programmer, while competent and distinctly professional and very capable, may look at programming as a job.
The challenge, then, is how to find and hire a capable team of programmers, how to motivate and coach a few of them to be great programmers, how to manage the rest to deliver successful results, and how to deliver continuous improvement even if most or all of your team are merely competent.
Why Is Becoming a Successful Programming Manager Hard?
Most great programmers do not campaign to be managers of other programmers. Most ultimately recognize the need to have software managers but feel most comfortable allowing others to do the actual management. They are usually not truly happy managing people or projects.
Managing programmers is hard! The often-quoted line “Managing programmers is a lot like herding cats” captures the essence of why it is so hard to become an effective and successful programming manager. Cats are free-spirited, individualistic, wily, playful, curious, independent creatures. All of these adjectives can be applied to programmers as well.
In our experience highly capable software managers are rare. And it is the rare software manager who can actually succeed at and find joy in herding the free spirits who are the majority of programmers.
Because programmers are free spirits, typical motivational techniques are not usually successful. It is critical to be able to appeal to a programmer’s sense of self and desire to change the world, in addition to providing the right amount of technical oversight and putting development practices and processes in place. It requires a special type of manager, one who both understands how programmers work and understands the work, to be able both to effectively motivate them to perform exceptionally and to deliver results on time.
Unlike many jobs where compensation is a primary motivator, the work itself and the work environment tend to be much more important motivators. Programming is a creative process that requires special handling to be done effectively. Good managers have to pay attention to these things while establishing a programming-positive culture, too.
As described throughout this book, becoming an effective and successful programming manager is possible, but we think it usually takes having been a good or great programmer to succeed as a programming manager.
Of course, that is usually part of the problem. Most programming managers were promoted to be managers because they were good or great programmers and showed some people skills—they demonstrated a capacity, and perhaps even an inclination, to direct the activity of other programmers.
Few programming managers have any formal management training,11 and what management experience they have is usually gained on the job and “under fire.” Some of these rookie managers succeed. Some fail immediately. Many fail over time.
11. In surveys conducted during hundreds of trainings, talks, and workshops that Ron and Mickey have conducted since the publication of the first edition of this book, we have found that less than 5 percent of programming managers had any formal management training before becoming a manager and less than 50 percent had received any formal management training ever.
The ones who succeed are likely to have a mentor in their organization or network who can help guide them and protect them as they earn their stripes and make their mistakes along the way. Having been managers of programmers for over 65 years, we have had the opportunity to hire and manage thousands of talented programmers. With that experience, along with insights from other experienced programming managers scattered throughout the chapters and having had considerable practice ourselves mentoring scores of managers as we moved from managing teams to managing organizations, we hope that this book provides the kind of guidance a mentor would provide—to provide a surrogate mentor for those who must struggle alone to manage programmers.
This book does not and cannot “fix” programmers. They will continue to code before they design, and they will continue to produce tangible results only when necessary. Our goal is to provide some insights, suggestions, tools and techniques, and rules of thumb that may help you “herd the cats” you have on your software projects and to successfully manage the seemingly unmanageable programmers on your team.