Chapter 11. Stop Promising Miracles[12]
Note
When I was a software developer I once received a phone call from my manager. "Got an hour?" he asked. My manager had received what seemed to be a trivial request from one of our customers. However, by the time I had understood the customer’s real need and written a small application to satisfy it, I had devoted 100 hours to this project. Not all project estimates are off by a factor of 100 but this is a striking example that it is indeed possible to go that far wrong.
Most software professionals must provide estimates for their work, but few of us are skillful estimators. Many of us haven’t been trained in estimation techniques. We’re too optimistic, with short memories that mask the painful overruns from previous projects. We don’t incorporate contingency buffers to accommodate unexpected events or risks that materialize. And we often overlook necessary aspects of an activity, so when we eventually confront those tasks, we either perform them—thereby exceeding our estimates—or skip them, perhaps compromising quality in the process.
Estimates generated during the project initiation stage are particularly likely to be misleading. The "Cone of Uncertainty" illustrated in Figure 11-1 cautions that estimates created very early in a project have extremely large deviations from the ultimate reality (McConnell 2006). These preliminary estimates based on limited information and many assumptions can be off by a factor of four on both the high side and the low side. Nonetheless, stakeholders do want to receive some kind of estimates early in the project, recognizing that estimates will become both more accurate and more precise as the project continues.
There are several ways to become a better estimator. The most fundamental is to record effort, duration, or size estimates along with your estimating processes and assumptions, and then record the actual results from each estimated activity. Comparing actual outcomes to the estimates helps generate more accurate estimates in the future. Estimating procedures and templates that itemize tasks help avoid the common problem of overlooking necessary work.
Another approach builds on the principle that multiple heads are better than one. Developed at the Rand Corporation in 1948, the Delphi estimation method asks a small team of experts to anonymously generate individual estimates from a problem description and reach consensus on a final set of estimates through iteration. In the early 1970s, Barry Boehm and his Rand colleagues modified this method into Wideband Delphi, which included more estimation team interaction (Boehm 1981). Mary Sakry and Neil Potter of The Process Group later created a repeatable procedure for performing Wideband Delphi estimation on software projects (Stellman and Greene 2006).
Using the Wideband Delphi method provides several advantages over obtaining just one estimate from a single individual. First, it helps build a comprehensive task list or work breakdown structure for major activities, because each participant will think of different tasks to perform. The consensus approach helps eliminate bias in estimates produced by self-proclaimed experts, inexperienced estimators, or influential individuals who have hidden agendas or divergent objectives. People are generally more committed to estimates they help produce than to those that others generate. No participant in an estimation activity knows the "right" answer; creating multiple estimates acknowledges this uncertainty. Finally, users of the Delphi approach recognize the value of iteration on any complex activity.
Wideband Delphi can be used to estimate virtually anything—the labor months needed to implement a specific subsystem, the lines of code or number of classes in an entire product, or the gallons of paint needed to redecorate your house. I used Wideband Delphi once with a process improvement group to estimate the effort it would take a particular organization to achieve Level 2 of the Capability Maturity Model for Software (Paulk et al. 1995).
The Delphi method helps you develop a detailed work breakdown structure, which provides the foundation for bottom-up size, effort, or schedule estimation. The starting point for a Delphi session could be a specification of the problem being estimated, an initial high-level task list, or a preliminary project schedule. The outputs are a detailed project task list; a list of associated quality, process-related, and overhead tasks; estimation assumptions; and a set of task and overall project estimates, one from each participant.
Figure 11-2 illustrates the Wideband Delphi process flow. During planning, the project manager defines the problem being estimated and selects the participants. The kickoff meeting gets all estimators focused on the problem. Each participant then individually prepares his initial task lists and estimates. Estimators bring these items to the estimation meeting, during which several estimation cycles lead to a more comprehensive task list and a revised set of estimates. The moderator or project manager then consolidates the assorted estimation information offline, and the team reviews the estimation results. The session is completed when some predetermined exit criteria are satisfied. The resulting range of estimates will almost certainly be a more realistic predictor of the future than any single estimate could be. Let’s look at each of these process steps in turn.
[12] This chapter was originally published in Software Development, 2000, 8(2): 49–54. It is reprinted here, with modifications, with permission of CMP Media Inc.