6
Changing Engineering Behavior

Charlie ran a software group for a major bank and asked me to join his staff meeting. Almost a year earlier, they had started a process improvement effort, and he planned to review their status. The corporate office had set a goal that every software group reach CMM^* level 2 by the end of this year [1, 2]. After reviewing each department’s status, we could see that this organization had made little progress.

* The Capability Maturity Model (CMM) is a software process assessment framework that guides management through a five-level improvement program. At level 1, the organization’s work is essentially unplanned and uncontrolled. At level 2, standard planning, configuration management, and quality controls are in place. At level 3, the organization has defined its basic processes and practices, and at level 4, it has established and is gathering process measurements. Finally, at level 5, the organization has a regular way to introduce technology improvements and to continually improve its processes. CMM and Capability Maturity Model are registered with the U.S. Patent and Trademark Office.

The process manager completed her report, and Charlie asked for my comments. When I asked whom he held responsible for reaching level 2, Charlie pointed to Beth, the process manager. I told him that this was an impossible job; the only person who can change a department’s process is the department manager. Beth could provide support and assistance, but there is no way that she or any other staff person could change the way people worked in any department but her own.

Next I asked whether Charlie and his senior managers participated in the bank’s bonus plan. Charlie said that they did. When I asked if any of their bonuses depended on the process improvement work, he said that only his did. I then suggested that he tie his managers’ bonuses to their process improvement performance. Then Beth and her staff could assess the status of each department each month and post the measures in Charlie’s conference room for their monthly management meeting.

Charlie and his managers agreed to make 25% of their bonuses contingent on achieving their process improvement goals and to track and review their progress every month. When I next visited Charlie, six months later, all the departments had achieved CMM level 2, and one was nearly at level 3. Once process improvement was personally important to the line managers, they made the needed changes very quickly.

What, not how

As is clear from Charlie’s experience, if you want to make changes, the managers must agree and help to make them happen. If they don’t, not much will get done. So obtaining management agreement is necessary, but is it sufficient? Unfortunately, the answer is no.

If your objective is “faster, better, cheaper,” that requires changing engineering behavior. Based on Charlie’s experiences, you might think that the managers could just tell their engineers to change. This will not work. Software is an intellectual activity, and to direct software engineers to work in a particular way is useless. Intellectual work cannot be ordered or directed. If the engineers do not want to work as you direct, they won’t, and there’s a good chance that you’ll never know it.

Trying to change the behavior of software engineers is like herding cats. They are very independent people; they all have their own ideas, and they won’t tell you what they think, particularly if they disagree with you. They just do what they want to do. Software engineers respond to guidance on what to do, but they resent being told how to do it. They properly see themselves as creative professionals. Over many years, they have developed a set of skills and capabilities, and they will not change these habits unless they are convinced that the changes will help them to do better work.

Most software engineers learn to write programs in high school or when they start college. Computer curricula typically concentrate on technical topics and do not address such subjects as estimating and planning, or how to measure and manage quality. Even when engineers have had planning courses, the subject is treated as something that managers do once at the beginning of the job.

Therefore, if you want your engineers to make detailed plans, to base these plans on historical performance data, and to adjust their plans as they do the work, you must provide them with the proper training. You must also convince them that planning, data gathering, and tracking are important and personally helpful. To gather data consistently, engineers must be expected to use the data. Of course, the managers must understand why detailed plans and disciplined work are important and how they will help their projects. So, to change the behavior of software engineers, you must convince both them and their managers that what you want them to do will help them and the business.

Disciplined Software Practices

After 27 years with IBM, I joined the Software Engineering Institute (SEI). After a few years, the institute director named me an SEI fellow and gave me the opportunity to pursue my interest in quality management. The question I had long pondered was “If software engineers used the best known quality methods, could they consistently produce defect-free programs?”

My personal work as an engineer had been many years earlier, and subsequently I had learned a great deal about quality and quality management. Among other things, I had directed IBM’s programming work, been IBM’s director of software quality and process, participated in assessing semiconductor quality methods, and been on the Malcolm Baldrige Quality Award board of examiners. I had also led SEI’s CMM development and helped many organizations use the CMM to improve their software capabilities. I had long believed that the typical software attitude of “build it quickly and fix it later” was wrong, so I decided to apply the quality principles I had learned to the task of writing programs.

When I started my research, I decided to call the method the Personal Software Process (PSP). My objective was to concentrate on how engineers personally write the small module-sized programs that form the building blocks for large programs. I believed that the quality of the product was determined by the quality of the process that produced the product. I further believed that a quality process must address all aspects of the work, including planning, tracking, and quality management.

The rationale for focusing on the quality of each engineer’s personal work was that if every small program in a big system is not of high quality, the system cannot be of high quality either. Large programs generally have thousands of module-sized elements, and you can get a quality program only if every single module is of high quality. Therefore, the quality of every engineer’s personal work is important. Improving average quality levels is not enough. For example, in a system with 1,000 modules, if 999 were defect-free but one was highly defective, the system would have a high average quality level. However, it would almost certainly have quality problems. Since every mistake by every single programmer contributes to quality problems, everyone on the project must do quality work.

I started my PSP research by defining a programming process and a set of measurements. Then I wrote the first program. In writing each program, I first made a plan, then I gathered data on the time spent by phase, I recorded every defect found, and I measured the sizes of the products produced. Over the next three years, I wrote 62 programs in several languages and I learned a great deal. I also found that, while PSP methods require considerable discipline, they produce high-quality programs. The PSP also improved my planning accuracy and my productivity.

Convincing Others

Now that I had reams of personal data, I began speaking, writing papers, and meeting with engineering groups. I wanted others to use the PSP so they could see that it would work for them. I now knew that the standard quality principles used in other fields could be tailored specifically for software work and that, when they were, software professionals would produce high-quality programs in less time and on more predictable schedules than ever before.

People listened politely and one group of five engineers even agreed to try the PSP. They said they would use it when they had time. I called them almost every week; they never had the time. I concluded that when engineers work on real projects, they will not try anything new or unfamiliar. They are in a hurry and are unwilling to risk the learning time and unknown problems that invariably accompany new methods.

The engineers did not argue with my data, but they obviously did not believe that the PSP would help them. I needed to prove to each engineer that the PSP would help him or her do better work. Clearly, without data on their personal work, engineers would not try the PSP, but they could not get these data without using the PSP. While I knew that the PSP worked, I was stuck.

The PSP Course

The answer to this conundrum was a course. Over the next year, I designed a PSP course, wrote a textbook, and taught the course at Carnegie Mellon University [3]. The results were better than I had hoped. Even more important, the student-engineers were convinced that the PSP worked for them. Several of them even changed their careers and are now working with me to transition the PSP into general practice.

The basic principles of the PSP course are shown in Figure 6.1. Since the purpose of the course is to convince engineers that the PSP will work for them, they must use it to write programs and gather data. To be truly convinced, however, they must write enough programs to learn the methods and gather enough data to show that their work has improved. This requires at least nine or ten exercise programs.

Before taking the PSP course, engineers must be capable programmers. Then they use the PSP to plan each program, use a defined process to write it, and gather precise data. They learn to plan, to use sound design practices, and to measure and manage quality. They also see from their own data that the disciplined PSP methods help them to do better work.

The PSP course results of 298 experienced engineers have been published by the SEI [4]. Examples are shown in Figures 6.2, 6.3, and 6.4. In these figures, point 1 at the left shows data for program 1 at the beginning of the course. Since the engineers have not yet learned the PSP, this first point represents their performance with their current software practices. Point 10 at the right shows their data when using the PSP. The difference is the improvement caused by the PSP.

Figure 6.1 The PSP Course

Figure 6.2 shows how estimating accuracy improves. Point 1 at the left shows the estimating accuracy for program 1. Since the engineers have not yet learned estimating methods and have no historical data on their work, this first estimate is a guess. Point 10 at the right shows the estimating accuracy for program 10, the final course program. Here the engineers have learned how to use their historical data to make statistically sound estimates for program size and development time. They have also learned how to estimate the likely error range of these estimates.

As is clear from Figure 6.2, the engineers’ estimating accuracy improves right up through the last program. This implies that estimating accuracy will improve even further, as long as the engineers continue using sound estimating methods and as long as they continue gathering and using personal data.

Figure 6.2 Time Estimating Error

Figure 6.3 shows the defects these 298 engineers found in compiling and testing the ten PSP programs. Software engineers typically write programs quickly and then address quality during compiling and testing. With today’s typical practices, engineers spend about half their development time compiling and testing. Even then, the resulting programs generally are defective. With the PSP, engineers concentrate on quality from the beginning, and they find and fix most of the defects before first compiling. They are surprised to see that by using the PSP, they reduce compile and test defects by five or more times and cut the time in compile and test by three or more times.

Figure 6.3 Defects Removed in Compile and Test

For most engineers, the results in Figure 6.4 are most surprising. They see from their personal data that when they use the PSP to make plans and to track their time, size, and defect data, their productivity does not drop. In fact, for many, productivity actually increases. On average, engineers are as personally productive when they use disciplined methods as they are today. As we will see later, even when the engineers’ personal productivity does not change, when they all make personal plans and measure and manage quality, their team’s performance improves dramatically.

While over 90% of the engineers who take the PSP course are convinced of its value; a few are not. Most of them are like one young woman who told me, “I didn’t believe the PSP would work, so I didn’t use it.” Rather illogically, she then claimed that her data showed that the PSP methods were not effective. Some engineers are so convinced their current methods are right that they will not try anything new. Some will ultimately see the benefits of disciplined methods, particularly if their peers use them, but a few will not. They tend to leave the organization when management insists that they use the PSP on the job.

Figure 6.4 Productivity

Coming Full Circle

After the success of the PSP course, our SEI team started training engineers in industry and monitoring PSP performance on the job [5]. Although the initial results were positive, I was disappointed to find that PSP use died out shortly after course completion. Several engineers complained that they couldn’t use the PSP because their managers objected to their making plans, gathering data, or doing quality reviews. As far as the managers were concerned, quality was for the testing department; the engineers should concentrate on coding and testing. We concluded that while PSP training was necessary, it was not sufficient. Engineers also needed a supportive working environment.

This should not have been a surprise. Disciplined performance in sports, the performing arts, medicine, or science all require the support of coaches, peers, and managers and the reinforcement of a professional working environment. The next step was to see if engineers would follow the PSP disciplines when they worked on teams where everyone was PSP trained and where management understood and supported the PSP practices. This leads to the next issue: how to build disciplined and motivated teams. That is the subject of the next chapter.

Summary and Conclusions

The following five principal points are made in this chapter:

1. For engineers to use disciplined methods, their managers must agree with the methods and support their use.

2. To actually change engineering and management behavior, the managers must be responsible for implementing the changes.

3. If software engineers are to change their behavior, they must believe that the new methods will work for them.

4. When engineers take the PSP course, they use disciplined engineering methods to write ten programs.

5. From their personal PSP data, the engineers see that by using disciplined methods, it takes them no longer to predictably write high-quality programs than it took them to write programs before.

References

1. W. S. Humphrey. Managing the Software Process. Reading, MA: Addison-Wesley, 1989.

2. Mark C. Paulk, et al. The Capability Maturity Model: Guidelines for Improving the Software Process. Reading, MA: Addison-Wesley, 1995.

3. W. S. Humphrey. A Discipline for Software Engineering. Reading, MA: Addison-Wesley, 1995.

4. Will Hayes and James W. Over. “The Personal Software Process: An Empirical Study of the Impact of PSP on Individual Engineers.” CMU/SEI-97-TR-001.

5. Pat Ferguson, Watts S. Humphrey, Soheil Khajenoori, Susan Macke, and Annette Matvya, “Introducing the Personal Software Process: Three Industry Case Studies.” IEEE Computer, vol. 30, no. 5 (May 1997), pp 24–31.