CHAPTER 11

The Psychology and Politics of Estimating

Estimation errors arise from three sources: political, psychological, and technical (process-related). Internal politics plays a large role in such errors because corporate planners often intentionally make unrealistic estimations to get their projects approved. Optimism bias, planning fallacy, the rule of pi, and other cognitive biases also skew estimates. A number of simple techniques can improve estimations: avoid making wild guesses, do reality checks, collect relevant historical data, and perform independent assessments.

How Do We Make Estimates?

We now come to one of the most important topics in project decision analysis and project management: estimations. How many books have been written on the subject, how many consultants make a good living at it, how many estimation software packages have we bought (and are not using)? Unfathomable numbers. Still, we cannot always deliver accurate project estimates.

In essence, estimates are merely forecasts of the future; unfortunately, we are not very good at forecasting. It is difficult to make forecasts of natural phenomena like the weather; it is even harder to make forecasts of any processes that involve people, who come with certain knowledge, behaviors, and biases. Project management is one of these processes.

Estimating is an extremely important step in modeling and decision analysis. Without proper assessments of project duration, finish time, cost, resources, success rate, and other parameters, it is almost impossible to select the best alternative for carrying out the project.

Why do many activities—in our professional projects or in our personal lives—take much longer then we originally estimated? Let’s analyze a simple hypothetical example that shows how estimation is done in a software development project. We’ll assume that a programmer is already familiar with the scope of the task but has not yet performed any estimates. Here is a conversation between the programmer and the project manager at the launch meeting.

The project manager asks the programmer, “How long is it going to take to build this component?”

“Oh, abo-o-out . . . five days,” the programmer answers.

The project manager asks, “Are you sure?”

“Yes; if everything goes well, it should take about five days.”

A few hours later the project manager enters this information into the project schedule. As schedules go, it is a masterpiece, containing more than a hundred different tasks and a dozen milestones. The project manager prints the schedule and sticks it to a wall. She is enjoying her creation the same way artists might enjoy theirs. “We had a big rush at the end of our last project,” she thinks. “But this time everything will be great.”

Now the project starts. Actually, it does not start on time because three programmers are busy cleaning up some problems from a previous project. Then there is a change in the requirements. By the time they get around to the task discussed by the project manager and the programmer, the project is already at least three days behind schedule. The project schedule may be in tatters, but no one really cares. They all know there is more realism in a Star Trek movie than in the Gantt chart hanging on the wall.

Finally, after a delay of three days, the programmer is ready to work on his task. He recalls his original estimate. “I should be able to do it in five days, maybe even faster,” he thinks. On Monday he starts to code and realizes that he doesn’t have a clue how to start, even though originally everything looked very straightforward. Tuesday, he spends the entire day learning about the new programming tools that he must use. Wednesday starts off with a fire drill, and he spends an hour twiddling his thumbs and worrying in the parking lot. After that, he spends a few hours on the phone answering questions about a previous project. Still he manages to begin coding. Throw in donut day on Friday, which is slow at the best of times, and our programmer has already burned through his original estimate.

The following Monday, the programmer talks to the project manager.

“Everything is almost ready,” he asserts. “I had some delays because I had to learn a new tool, but I’ve solved almost all of the problems,” the programmer reports during the meeting. (“Almost,” in IT language, means about half the work.) The project manager realizes she’s hearing IT talk but prefers to ignore it; she doesn’t have a choice. After resolving a few major issues and working on some other minor issues that were discovered the previous week, the programmer finally completes the task on Friday, a week longer than the original estimate. Fortunately, the task is not on the critical path. Unfortunately, other tasks that are on the critical path have suffered the same fate.

Software projects don’t necessarily fail completely or get canceled. Often, releases are delayed or released with are euphemistically called “quality and usability issues.” In the software world, this means that it will work, but not completely as planned. If it were a car, it might not have headlights and would occasionally shift into reverse when you turn on the windshield wipers. Customers may complain, but in most cases the software will not be used extensively anyway. It becomes “shelfware.”

This is the end of our saga, which started with a poor estimate and the creation an of unrealistic project schedule and ended ignominiously. Now we’ll try to understand how these mistakes were made. The root of the problem lies in human psychology, which we will try to analyze first.

How Do We Think When We Make Estimates?

Let’s go back to the original conversation between the programmer and the project manager. How long does it take to say, “Oh, I think abo-o-out . . . five days”? Try to do it. It shouldn’t take longer than four seconds, even if you pause before the sentence. Remember, we assume the programmer already understands the scope of the task. He uses this time to make an estimate. Here is his mental process: he recalls the scope of the task. He tries to recall similar projects, mostly with comparable scope or similar development tools. He checks how relevant these projects are to the current task. That gives him an approximate answer: around four or five days. The programmer does this analysis almost instantaneously. It is not really an analysis; it is an intuitive response based on his memory input.

The programmer spends the rest of these four seconds trying to come up with an acceptable estimate. If it’s too low, he won’t be able to complete his job on time. Too high, and the project manager will be unhappy (she is not an idiot and has her own silent estimate). This is a good example of a motivational bias, where the programmer has a personal interest in expressing this judgment.

When the project manager asks, “Are you sure?” the programmer doesn’t repeat his analysis; he is just trying to determine whether his answer has satisfied the manager. Moreover, as you might have noticed, he did not have the time or information to understand any potential pitfalls, which he implicitly acknowledges with the phrase “if everything goes well.”

It is truly amazing how much thinking can be done in only four seconds. It is even more amazing that estimates done this way may be absolutely correct. This type of estimation works well when the person has participated in similar activities many times and has an excellent memory of those experiences. However, in research and development projects or almost all new projects, this is often impossible.

Impact of Politics on Estimation

Why do people generally make poor estimates? There are at least three reasons:

1. Political and organizational pressure.

2. Psychological issues.

3. Technical issues, including problems related to the implementation of the project management process.

Bent Flyvbjerg and his colleagues reviewed a significant number of large transportation projects (Flyvbjerg et al. 2002). They found that project planners often intentionally underestimate costs and overestimate benefits to get their projects approved. Flyvbjerg found that costs are underestimated in nine out of ten transportation infrastructure projects (table 11.1). He studied data for the last 70 years and found that cost underestimation has not decreased over time, even with new project management tools and techniques.

So, how do project planners underestimate? One of the ways to “cook” a forecast is to deliberately reduce the probability of risk occurrence. When Eurotunnel, a private company that owns the tunnel under the English Channel, went public in 1987, investors were told that risks of cost escalations would be relatively small, just 10%. The actual risks were significantly greater, and the real cost was two times higher than the forecast.

The British politician Benjamin Disraeli famously said, “There are three kinds of lies: lies, damned lies, and statistics.” He was noting an interesting phenomenon about probabilistic analysis. By deliberately using inaccurate probabilities for certain events, it is possible for a project manager or planner to improperly influence a decision, which may have disastrous consequences for the organization or stakeholders. In addition, it is difficult to catch these activities: they never said the event could not happen—they just knew that the chance of its occurrence was very small.

Impact of Psychology on Estimation and Rule of Pi

Organizational politics definitely plays an important role in estimation. But what about cognitive psychology?

An interesting phenomenon in the world of estimation is the rule of pi, which states that the actual duration (cost) of an activity will be about pi (3.1415 . . .) longer or bigger than the original estimate, even if the estimator was aware of this rule. Regardless of how we do our estimates, we always underestimate, even if we are aware of the tendency to underestimate. Sound strange? Well, not if you think about it. Why are we repeatedly late and running out of time and money? We try to fit too many activities into the project and hope against all hope that we will be successful. This wishful thinking is often the cause of the problem.

Table 11-1. Inaccuracy of Transportation Project Cost Estimates (Adapted from Flyvbjerg et al. 2002)

Project Type	Number of Cases	Average Cost Escalation
Rail	58	44.7%
Fixed-link (bridges and tunnels)	33	33.8%
Roads	167	20.4%
All projects	258	27.6%

You might ask why it is called the “rule of pi.” In fact, it is an arbitrary number, and the reference is a bit tongue in cheek, but it emphasizes the fact that mistakes in estimates can be very significant. Also, this rule was invented by programmers, who like to remind everybody that they know math.

The rule of pi is related to known psychological issues, such as the planning fallacy (Buehler et al. 1994) and the optimism bias (Kahneman and Lovallo 2003). How do these issues pertain to estimation? According to one explanation, people often fail to account for risks or other factors that they perceive as lying outside the specific scope of a project. They also may discount multiple and improbable high-impact risks because each one has a glancingly small probability of occurring. For example, our programmer did not take into account four potential events: answering questions regarding a previous project, learning about the new programming tools, participating in a fire drill, and enjoying donut Friday.

This problem is the result of limitations of our mental capacity, and in many cases it may be impossible to account for such events without special tools, such as data-mining and risk-management software.

The optimism bias is widely acknowledged by project management practitioners. For example, some organizations perform an adjustment for optimism bias through the performance of a risk analysis to come up with accurate estimates. It is good idea to perform this type of adjustment. However, here is an interesting paradox of the rule of pi: adjustments for the rule of pi often cannot be done. In other words, even if a project manager has adjusted duration and cost estimates for a forecast, it will still take longer, roughly 3.1415 times longer, to complete the project. This sounds almost too fantastic to be true, and most likely mathematically incorrect, for it can lead to an infinite project duration and cost (3.1415 × 3.1415 × 3.1415 . . .). However, the problem is that project managers do not adjust duration and cost significantly enough to keep these parameters within expectations, as a result of confirmation bias and other mental traps. And if they do, they are subject to another bias: student syndrome.

Student Syndrome

In his book Critical Chain (2002), Eliyahu Goldratt described “student syndrome.” It refers to a student’s way of waiting until the last minute to cram for an exam, which leads to the wasting of any contingency buffers built into the estimates of the task duration. A good example is how we saw that our programmer fully concentrated on the job only when the deadline was looming. Therefore, setting really distant deadlines may not help, because most of the work will be done just before the deadline anyway.

By the way, our own project to write this book took us much longer than we originally estimated—because of the rule of pi. How could this happen if we knew about these rules, as well as many other mental pitfalls? The answer is that knowledge of an illusion does not spare us from being a subject of the illusion. If you were a professional magician with many industry awards and citations, you would still enjoy somebody else’s magic show. Moreover, according to another bias called the bias blind spot, people tend not to see their own cognitive biases (Pronin et al. 2002).

Other Cognitive Biases in Estimating

When people perform estimates, they often apply heuristics or simplification techniques that help them to reduce the burden of processing complex information, which we discussed in general in chapter 2.

One simplification technique is the anchoring heuristic. Again, let’s return to our original example of estimation. The programmer instantly comes up with the number of five days. This number might be wrong, but further estimates will likely remain close to his original estimate. For example, the programmer might then reestimate the duration of an activity at between four and six days.

The availability heuristic can also affect our estimates. To see how, let’s do a very simple psychological exercise:

1. Take three seconds to try to recall all the projects or large activities you were involved in during the last year.

2. Now repeat step 1, but take 15 seconds.

3. Repeat steps 1 and 2, taking up to three minutes for each step, and write down the results.

You will find it difficult to remember more than a few previous activities unless you spend some time thinking about it. Even then, you will probably have a clearer memory of your most recent projects. It is also easier to recall your most successful or largest projects. If the programmer calculates the likelihood of the task’s taking five days based on the projects he remembers, and if he remembers only his successful activities, he will underestimate the duration. To illustrate this phenomenon, table 11.2 shows the set of his previous activities relevant to the current task of developing a computer program to display a bar chart.

Since at the time of estimation the programmer clearly remembers only two out of the five activities, he deems it very probable that the activity will be completed in five days. In reality, it could take much longer.

Table 11-2. Example of Previous Activities Related to the Current Task

Further Explanations of Problems with Estimation

A number of other issues will lead to forecasting errors. Among them:

• There is no established project estimation process

• Inaccurate data is used, or historical data may not be complete

• The forecasting techniques and tools are inefficient

• There is no ability to track actual project performance, which can be used to refine estimates

• The project planners are inexperienced

The PMBOK Guide points out that performing a project estimate is part of two knowledge areas:

1. Project time management, including activity duration estimation, activity resource estimation, and the schedule development processes

2. Project cost management, including cost-estimating and cost-budgeting processes.

Organizations that have established project management processes that forecast both time and cost usually produce more consistent and accurate estimates.

Where Does the Problem Lie—in Psychology or Politics?

The balance between the optimism bias and pressure of politics was the subject of an interesting debate involving the Nobel laureate Daniel Kahneman, along with Dan Lovallo and Bent Flyvbjerg, conducted in the Harvard Business Review. Kahneman and Lovallo (2003) believe that the optimism bias is to blame for inaccurate estimates. For his part, Flyvbjerg (2003, 2006) acknowledges the existence of the optimism bias and believes that in fact it plays a major role when political pressure is insignificant. However, when political and organizational pressures are significant, he believes that the problem with incorrect estimates is mostly related to deliberate deception by project planners. He argues that wrong estimates are so widespread and have continued for so many years that it is highly unlikely that human psychology plays a major role.

It is hard to make a clear distinction between cognitive biases (such as mental mistakes in processing information) and motivational biases (intentional lies) in forecasting and estimating. For example, in large capital projects throughout the construction industry, politics and organizational pressures play major roles in causing incorrect estimates. In other industries and in smaller projects, political pressure is not as significant. For example, in research and development projects, including IT projects, mental mistakes and psychological biases play a critical role in causing poor estimates. The two main reasons for this are:

1. Only James Bond, in movies such as Dr. No and Moonraker, can dress up as a member of a project team for the purpose of completely destroying the project. As far as we know, all other project managers and planners want projects to succeed. As a result, they will have some expectations about the project. As soon as people have expectations, they will demonstrate selective perception: “What I see is what I want to see.” Planners or project managers sometimes cease to review data as soon as they find something that supports their estimate. They can also discard evidence that contradicts their estimates. This is called a selective search of evidence. Similarly, there is a confirmation bias, or a tendency to assign more weight to evidence that confirms an estimate and to ignore evidence that could refute their hypothesis.

2. Planning of large projects is performed by many people, and each of them expresses different biases. Different project stakeholders have different preferences and different risk profiles. Because of this, it is hard to judge how the balance between cognitive and motivational bias can affect a particular project. It is even more difficult to extrapolate conclusions on a group of projects.

Why is understanding the balance between optimism bias and politics so important? Depending upon what the cause of the errors is in estimation, various solutions could be suggested to resolve the problem. If wrong estimations are caused by optimism bias, then the ultimate solution is a systematic analysis of projects. To avoid motivational biases, project managers should analyze similar projects and use actual data from completed projects to make estimations for new ones. In reality, project managers often use a “hybrid” approach for their estimations: using historical data and decision analysis methods.

Many Mental Errors and One Wrong Estimate

When people make estimates, they can make many mental mistakes at the same time. In the movie The Father of the Bride, George Banks (played by Steve Martin) was trying to organize his daughter’s wedding. Because it was a complex project, he and his family hired professional event planners, who can be viewed for our purposes as a project management office. If your CEO is not completely convinced of the importance of a project management office, tell the CEO to watch that movie. When George got the price tag from the event organizers, there was the typical comic moment where he had to reconcile his (affordable) expectations with the projected bill of the event planners. In fact, he was so far off that he had a nervous breakdown and ended up in jail, much to the embarrassment of himself and his family. But the real issue we want to investigate is why his estimate was so different from the realistic cost.

First, daughters do not get married very often, so George had few historical data of his own to draw upon. He had to make up his own estimation process, during which he made the following mental errors:

1. He wanted to spend as little as possible and therefore was motivated to come up with an unrealistically cheap estimate. He ignored all evidence that would have suggested the true cost of the wedding (expensive cake, large number of guests, a relative whose two airline tickets from Europe must be paid) when he did his rough estimate.

2. He used a reference point (the cost of a house) to come up with the estimate. It was completely irrelevant to a wedding and therefore was nonsensical for this particular estimate (though good for the movie). This is a classic example of anchoring.

3. Finally, he was overconfident. The wedding was a one-day event that would take place in his house, so he thought the amount of money required would be manageable.

What should George Banks have done to avoid a nervous breakdown?

1. Ask some other people how much the wedding would cost and use their number as a reference point.

2. Make a rough estimate by summing up all known expenses without ignoring any items.

3. Make necessary contingency plans by continually asking, “What else could happen?”

Figure 11-1. Risks and Uncertainties of Budgeting

Weddings will always be more expensive than we plan because: (a) the number of guests will be twice the original estimate; (b) food will cost at least three times more, even if you serve hamburgers instead of tenderloin; and (c) you will be charged for many things that you did not imagine (figure 11-1). If in the future you are planning a wedding, make accurate estimates and hire a professional project manager.

Simple Remedies

So, how can we integrate information about risks and uncertainties into our estimates?

NEVER MAKE A WILD GUESS

Estimates are possible even when based on partial information; however, we often try to make estimates with very little information, or none at all. We can call this type of estimate a “wild guess.” (If you dislike the word “wild,” just call it an “intelligent guess.”) For example, how much would it cost to develop one medication to treat all forms of cancer? There is no reliable information to support an answer. Still, we will try to answer the question, either because we don’t want to look incompetent or because we are being pushed by management or our colleagues to do so. The manager’s position is quite understandable; he or she does not want to end up with question marks on the project schedule.

Unfortunately, as soon as we deliver our estimate, everybody instantly forgets that it is a wild guess and, because of the anchoring heuristic, this estimate becomes the anchor for all future discussions. Could you imagine this newspaper headline: “Project manager of PharmaCo Inc. estimates that a universal cancer drug will cost only $5 billion”? Inevitably, people will use this number as a starting point in any future analysis or discussions.

What should we do if we are asked to make an estimate without any information? The only solution is to try to get as much relevant information as possible. If previous relevant data is not available, we can take a small task and see what happens. How long will it take and what level of resources did it require? For example, you can make a prototype or an evaluation tool. Unfortunately, management often wants to forego this strategy and asks for an estimate immediately. This is where the big problems begin.

COLLECT RELEVANT HISTORICAL DATA

Most project managers know how important it is to collect and analyze data from previous projects, but very few actually do it. If we had a full data set on relevant activities, our estimates would be far more accurate. In some industries this data is available through various software applications, forms, and methodologies. If you are lucky, you work for an organization that routinely collects and analyzes historical data as part of its project and portfolio management processes.

But what if you don’t have any such tools? The simple solution is to keep your old project schedules handy, so that you can easily access and review them when you are trying to make estimates.

Here is a simple way to analyze your information:

1. Look at previous project schedules or try to recall similar activities.

2. Write down activities and their relevance to the current one (table 11.3).

3. Use this table to assess the duration of the current activity.

Table 11-3. Analysis of Relevant Activities

Collection and analysis of relevant historical data will help to mitigate the negative effect of both motivational and cognitive biases, and will specifically help you address the availability heuristic.

PERFORM REALITY CHECKS AND BENCHMARKING

Reality checks are a simple way to improve the accuracy of your estimates. Their objective is to compare your estimates with known project results. Below is an example of cost estimation for movie production.

As you probably realize, movie production has significant uncertainties in project cost and duration:

• Movie set collapses

• Producer has a bad day

• Actress goes to rehab

• Director decides to change the plot and kill (or resurrect) the protagonist

Cost estimation with many uncertainties can be extremely challenging. The movie studio may have planned other movies of similar budget and projected revenue, thus needs to determine whether the new budget is roughly in line with previously released movies (figure 11-2).

Benchmarking is more advanced than simple reality checks. Benchmarking helps to compare business processes and their performance indicators with other similar processes. In project management, benchmarking assists you in comparing the cost and duration of various projects together with other indicators, such as scope, deliverables, net present value, and resource usage. In theory, you can benchmark everything:

Figure 11-2. Gross Revenue of Most Expansive Movies vs. Budget

• Oil well drilling

• Missile production

• Brain surgeries

• Cake baking

Moreover, everything can be measured. However, it is important to determine if a measurement provides value in view of the resources expended to gather it.

Many benchmarking exercises require detailed analysis of business and technological processes. You can use specialized software tools to perform benchmarking. Estimates for project or activity cost and duration are only a few of the results that benchmarking easily produces. Primarily, the results of benchmarking are recommendations for process improvements.

CONDUCT AN INDEPENDENT ASSESSMENT

For some projects, especially those with huge budgets, independent assessment is a firm requirement. But getting independent estimates by different members of the same team or by different teams can be a double-edged sword. On the one hand, they offer an additional look at the estimated parameters. On the other hand, psychologists know that assessments made by separate individuals may be different and difficult to reconcile. Sometimes an independent assessment is as biased as the original estimate.

For example, the cost of NASA’s space missions is often independently assessed by separate NASA divisions. Most important, after the independent assessment, an analysis is usually performed to discover the causes of the difference.

Sometimes an independent assessment may not be practical, because it might be difficult to find independent experts familiar with the particular project. Nevertheless, any discussions regarding the results of estimates by team members or independent experts have proven to be effective in helping to incorporate more information into the estimates.