CHAPTER 3

Understanding the Decision Analysis Process

The Americans will always do the right thing . . . after they’ve exhausted all alternatives.

—WINSTON CHURCHILL

We want to make rational choices, we would like a transparent decision-making process, and we want a mechanism for correcting mistakes. These goals can be accomplished through the decision analysis process: decision-framing, modeling, quantitative analysis and implementation, and finally monitoring and reviewing of decisions. Decision analysis is simple and fully adaptable to different types of project decisions.

Example of Decision Analysis Process

An example of applying decision analysis to a complex project can be seen at the National Aeronautics and Space Administration (NASA). The U.S. Congress mandated that NASA be more accountable when it evaluates its advanced technology projects. As a public agency, NASA is concerned about maximizing the value it gets from its expenditures and measures of cost reduction. But taking risks is inherent in all space exploration, so eliminating all risk would mean eliminating NASA’s very purpose. (From another perspective, what do you think was the primary objective of the Wright brothers—maintaining a healthy net present value or getting an airplane to fly?) To manage its inherent risk, many NASA divisions are today applying decision analysis methods to improve their ability to select the best courses of action (Williams-Byrd et al. 2016).

For example, NASA exploration mission directorates need to determine which capabilities and technologies will enable human exploration. NASA developed a decision analysis process that looks at multiple functional capabilities:

• Autonomous Systems and Avionics

• Communication and Navigation

• Cryogenic Fluid Management

• Environmental Control and Life Support System

• Entry, Descent, and Landing

• Extravehicular Activity (Spacewalks)

• Fire Safety

• Human Research and Crew Health & Performance

• Plus quite few others

NASA’s process involves elicitation of judgments from subject matter experts (the scientific term for “we ask experts about their opinion”), as well as mathematical modeling that takes into account experience from previous missions. Using decision analysis, NASA is trying to determine the risks associated with different missions and technologies, as well as how much would they would cost. The result of the process is a selection of technologies that will be used in future human space missions, as well as an allocation of investment funds in the development of new technologies.

When Decision Makers Go Bad

Everybody wants to make good decisions, but in reality many project decisions turn out to be wrong. For example:

• Why did a company decide to develop a product that it could not actually sell?

• Why did its project team decide to use supplies of poor quality?

• Why did its management appoint a project manager who clearly does not understand the business?

You might say that the company did not really want a product it could not sell, did not really wish to use poor-quality materials, and did not really intend to appoint an incompetent project manager. But it did make those decisions, using its own decision-making process.

The process that leads to such decisions is often cloaked. Decisions are frequently made behind closed doors, so it is difficult to know how, when, where, and why a decision that results in dire consequences was made. And once people make a decision, they tend to stick to it, even in the face of mounting evidence that it is wrong. This is a known psychological bias. People tend to be consistent, even if it means behaving irrationally; they don’t want to admit that they made a bad decision. As a result, poor decisions tend to take on a life of their own, which can exacerbate an already bad situation.

But there is a better way—decision analysis.

The Decision Analysis Manifesto

Here are three basic things you should want from a decision analysis process. We call it the Decision Analysis Manifesto:

• You want rational decisions. Quality decisions should lead to maximizing project value while minimizing expenditures. Good decisions should be based on an unbiased assessment of all possible alternatives. They should benefit the whole business, not just the interests of a specific individual or group.

• You want transparent decision-making. You want to know how decisions are made, who made them, and who should be accountable if a decision is wrong. You want to be able to participate in decision-making.

• You want a mechanism for correcting mistakes. If a problem is found with the original decision, you should be able to recognize it and take corrective actions in a timely manner.

In sum, what you want is a process. As with many other business processes, decision analysis has identified a set of procedures and tools that an organization can readily follow.

The “3C” Principle of Project Decision Analysis

Decision analysis is not a single, fixed process, but rather an adaptable framework that an organization can tailor to meet its specific needs.

There is no exact recipe for how to structure a decision analysis process. Different processes can be adopted for various companies, types of projects, and the types of decisions required. Your organization may already have, if not a fully established process, at least some components of decision analysis. For example, how do you review project risks and uncertainties? How are decisions made at product launch meetings?

But what does a “fully established decision analysis process” imply? Any established or mature decision analysis process is based on three main rules, which we call the “3C” principle: consistency, comprehensiveness, and continuity (figure 3-1).

CONSISTENCY

The decision analysis process should be standardized for similar kinds of problems and opportunities. Inconsistency in decision-making can cause projects to change directions unnecessarily, which can lead to failure. This necessitates that organizations must have the same set of rules and preferences for making decisions in all similar types of projects.

Figure 3-1. “3C” Principle of Project Decision Analysis

Suppose, for example, an oil company has numerous offices around the world evaluating exploration prospects. The company does not have enough resources to drill everywhere at the same time, so it must make choices. The evaluations on potential drilling prospects are forwarded from the various outlying offices to the corporate planning headquarters, where decisions about resource allocation are made. One of the main difficulties that the corporate planners face is that this information is submitted by different groups looking to develop their prospects in different locations, which are often in different countries. The planners don’t want to compare apples with oranges, as the saying goes, so the company tries to ensure that the methods used to generate the data regarding prospects are consistent across the organization. Otherwise it would be impossible to make a comparison of potential oil reserves and then make decisions on which prospects to develop (Rose 2001).

COMPREHENSIVENESS

Decision analysis processes should include a comprehensive assessment and analysis of the business situation. Missing or incomplete information can lead to incorrect decisions.

Let’s say that your manager approaches you and shows you a project schedule. He says, “We performed a comprehensive analysis on all of our possible alternatives and have decided to go ahead with this particular one.”

After a brief look at the schedule, you say, “Looks like it’s a very tight schedule. Did you account for any risk events? Where is the contingency time?”

“Don’t worry about contingencies,” he replies. “We covered everything. If you recall, we have done some similar projects in the past and didn’t have any problems. Plus, the decision has come down from up high, so it’s out of our hands.”

Two days into the project, a major event occurs—and the project is delayed.

Did upper management perform a comprehensive decision analysis, as claimed? No, because they relied simply on their experience of some past projects and did not include risks and uncertainty. The analysis was not comprehensive, and therefore it was flawed.

CONTINUITY

Decision analysis is a continuous process of evaluating and refining decisions during the course of a project. High-quality results can be achieved only through constant and consistent adaptive management.

Here’s an example of failing to use adaptive management:

You recently completed a project and presented it to your client. Unfortunately, the client is not very happy with it. Why? A couple of years ago, when the project was initiated, somebody made an incorrect decision that appears to have reduced the value of the project as a result of huge cost overruns. The person responsible for that decision is no longer on the team, so it will be difficult to understand the basis for it. And why were no corrective actions taken when it became obvious that the evaluation of cost was flawed?

Upon further investigation, you discover that it is not a common practice in your organization to revisit a decision once it has been made. In addition, when the problem surfaced, a great many resources had already been expended, and there was great reluctance at that point to change course. The final result is that your project failed to meet its requirements because the team did not adapt to changing circumstances.

Decision Analysis Process versus the PMBOK Guide’s Project Risk Management Process

You may question how decision analysis differs from what is described in the PMBOK Guide, chapter 11, “Project Risk Management.” Decision analysis and risk management both deal with uncertainties, but they do so from different perspectives. So, are both processes required? Could you imagine a country with two presidents—one responsible for decision-making and the other for managing risks? Just think: two White Houses, two sets of Secret Service agents, and two White House chefs. This might not be the end of the world, for we have a lot of redundant government agencies and we could probably survive with two White Houses. However, the two presidents would have a difficult time with their appointed roles, since you can’t make a decision without risk analysis, and conversely you can’t manage risks without making decisions.

The good news is that you do not need two (or more) business processes to manage related issues. You can integrate decision analysis into your overall project management process, as described in the PMBOK Guide. Decision analysis and risk management processes have a significant overlap. A good decision analysis should include risk analysis. In other words, decision analysis is a broader exercise than merely risk analysis.

In this book we do not discuss the PMBOK Guide processes in detail. (A number of very good books on risk management are listed in the Future Reading section.) We will, however, show you how the PMBOK Guide’s risk-management processes parallel the decision analysis. We will mention all of the PMBOK Guide’s risk-management processes, so you will have a complete, logical picture of how the two approaches relate to each other.

Phases of the Decision Analysis Process

The decision analysis process includes a number of phases, each one containing a number of steps. We can illustrate the process with a project you may be familiar with from your childhood, an incident involving Winnie the Pooh:

As you may recall, Pooh dropped in on Rabbit one day and ended up jammed in Rabbit’s doorway after helping himself to all of Rabbit’s honey. For Pooh, it was supposed to be a very short project: (1) visit Rabbit; (2) consume honey; and (3) go home. But Pooh, being Pooh, ate too much honey during the “consume honey” activity. This is a good example of the psychological bias of overconfidence. As a result of this event, the trivial activity (“go home”) could not be accomplished as scheduled, for Pooh was firmly wedged in the doorway. Now Pooh and his friends had a decision to make: they had to select the best alternative to solve this problem (figure 3-2).

Let’s examine how a decision analysis process (figure 3-3) would help identify the best solution for the problem facing Rabbit and Pooh.

PHASE 1: DECISION-FRAMING

Decision-framing helps decision-makers identify potential problems or opportunities; assess business situations; determine project objectives, trade-offs, and success criteria; and finally identify uncertainties. The project manager defines the scope of the decision. Depending on the situation, a project manager alone, an independent expert, or a team of experts can perform the decision-framing. The processes of risk-management planning and risk identification described in the PMBOK Guide can be accomplished along with decision-framing.

Figure 3-2. Using Decision Analysis to Resolve Complex Problems

Step 1.1. Identifying Potential Problems and Opportunities In some cases it is difficult to identify problems and opportunities, especially when they are related to a strategic decision. For example, what causes different projects within the organization to be consistently late?

In our Pooh example, the problem was clear: Pooh was stuck and was not happy about it (neither was Rabbit). Both of them need Pooh to be removed from Rabbit’s house as soon as possible.

Step 1.2. Assessing the Business Situation Before making a decision, it is important to assess the business environment and define the constraints related to the problem. Business environments can influence resource availability as well as costs. The assessment may also include an analysis of markets, competition, prices, or anything else that can be related to the problem or opportunity. During this step, it is important to list all external factors that may have an impact on the problem.

Who or what could be used to get Pooh out of his predicament? Of course, it could be Christopher Robin and Pooh’s other friends. Wise Owl also had some project management experience. In addition, Gopher had the expertise and tools to provide some engineering work.

Figure 3-3. Decision Analysis Process

Step 1.3. Determining Project Objectives, Trade-Offs, and Success Criteria Projects usually have multiple objectives and therefore multiple criteria for decision-making, which can make the analysis very complex. Decision-making criteria include project duration, cost, scope, quality, and safety, among other parameters. Project managers should find the right balance between these objectives and make trade-offs when necessary.

In Pooh’s situation the success criteria were:

• Remove Pooh from the doorway as soon as possible

• Do not harm Pooh during this process (safety concern)

• Do not damage Rabbit’s dwelling

Step 1.4. Identifying Uncertainties Understanding uncertainties is the key to the decision analysis process. In the decision-framing step, risks and uncertainties should be identified. Uncertainties can be found in a project’s cost, scope, duration, quality, safety, or environment.

In this project—removing Pooh—we primarily have uncertainties in time, as well as uncertainties in cost.

Step 1.5. Generating Alternatives In decision-framing, it is important to generate key alternatives. First, you must identify what cannot be changed—that is, what the constraints are in the particular decision analysis. Then you can determine potential alternatives.

Pooh needed to be removed, one way or another. He could not be stuck in Rabbit’s doorway forever. Therefore, project scope was a constraint. There was, however, the possibility of bringing in additional resources to accelerate the project. As a result, we have three potential project scenarios:

• External contractor Gopher digs out Pooh

• Gopher blasts Pooh out with dynamite

• Christopher Robin’s suggestion—waiting until Pooh loses enough weight and is slim enough to slip through the doorway

PHASE 2: MODELING THE SITUATION

A mathematical model can help to analyze and estimate future events. For example, to understand how a structure withstands particular loads, engineers perform structural analysis on buildings, using mathematical models. They don’t want to find out that a particular beam cannot bear a load after it has been put in place, because by then it may be too late to change it. The same situation exists with any other projects.

Step 2.1. Creating Models for Each Project Alternative Project managers constantly create mathematical or valuation models of projects. In most cases, it is simply the project schedule. A more comprehensive model of a project includes a breakdown of resources, costs, and other project variables.

Sometimes, quite elaborate models are required. For example, in the analysis of a product’s life cycle, comprehensive models will include not only product development but also marketing and sales efforts.

Each model includes input and output variables, as well as a calculation algorithm. In the case of a project schedule, input parameters are tasks with their start and finish times, costs, and resources, as well as the relationships among all the tasks. Outputs include project cost, duration, start and finish times, critical tasks, resource allocation, and other parameters.

Modeling the situation in general, and project scheduling in particular, can be a highly complex process. A model should be created for each project scenario; in most situations, however, these models are variations of a single base model.

In the Pooh removal project, a schedule for each alternative was needed.

Based on Owl’s request, Gopher estimated the duration of the excavation alternative. He did a review of the site and performed some exploratory excavation. He estimated that the work would take two or three days. He also performed a cost analysis. He based his calculation on his hourly rate and estimated project duration. He also added overtime and 10% for contingency.

Gopher estimated that using dynamite would lead to a quick removal of Pooh, but with uncertain effects on Rabbit’s doorway and Pooh’s rear end.

The slimming alternative, suggested by Christopher Robin, seemed to have the least risk and cost, but the longest duration.

Step 2.2. Quantifying the Uncertainties The uncertainties in a project, identified through the decision-framing process, should be quantified. One of the ways to quantify uncertainties is to define ranges for their parameters. For example, define low (optimistic), base (expected), and high (pessimistic) duration or cost estimates for each task.

Another way to define uncertainties is to list all the potential events that could affect the project schedule, then quantify their probabilities and impact.

The PMBOK Guide’s steps for risk management recommend using a qualitative risk-analysis process to assign probability and impact.

• Gopher estimated the uncertainty in duration of the excavation as a range (between two and three days).

• The blasting alternative had minimal uncertainties in estimating duration.

• There were several uncertainties with the last alternative (Pooh’s slimming down). Nobody knew for certain how long before Pooh’s stout frame would melt away enough to free him from the door. They were also faced with the prospect that Pooh would continue to eat (on the sly) during the course of the project. That risk, which should be given both a high impact and a high probability, could significantly increase project duration.

PHASE 3: QUANTITATIVE ANALYSIS

After the mathematical model is ready, the analysis may include a number of steps, depending on the situation. It is possible to apply simulation techniques to analyze the project. These techniques can usually give project managers enough data to make an informed decision.

Even with the most advanced analytical tools and techniques, interpretation of the results of an analysis is performed subjectively by project managers; therefore, it is subject to the types of mental traps discussed in chapter 2. For example, quantified analysis may show that the chance that a project will be completed on time equals 80%. Is this acceptable? It may be in some projects, but not in others. In addition, the results of quantitative analysis must be communicated properly to decision-makers to minimize the potential for biased decisions.

Quantitative methods such as sensitivity analysis, Monte Carlo analysis, and decision tree analysis are described in chapter 11 of the PMBOK Guide.

Step 3.1. Determining What Is Most Important A model of a project may include a considerable number of variables: large numbers of tasks, resources, risks, and other parameters. Some of these parameters may significantly affect the course of the project. For example, certain risks will cause failure of the project, whereas other risks will have no noteworthy effect. It is impossible for a project manager and a project team to concentrate their efforts on mitigating all possible risks. The team should therefore focus first on mitigating critical risks.

To determine which project parameters are the most critical, project managers can use sensitivity analysis, which is described in detail in part 4 of this book.

In the Pooh situation, the risk associated with feeding Pooh would probably have the most effect on the project duration and was therefore a critical risk. To mitigate this risk, Rabbit set up a poster, warning “Do Not Feed Bear.”

Step 3.2. Quantifying Risks Associated with the Project Uncertainties associated with input parameters were already quantified during the modeling step. Now you need to analyze how the combination of all these uncertainties could affect the project. The goal of this analysis is to create a “risk profile” of the project. You may need to know the following information:

• The chance that the project will be completed within a certain period and on budget

• The project’s success rate, or the chance that it will be completed

• The low, average, and high estimates for duration, cost, and other project parameters.

Again, a number of analytical techniques can be applied to this analysis.

Here is the result of the analysis of Pooh’s project, based on the success criteria identified during the decision-framing stage. Three alternatives had been identified:

• “Excavate Pooh.” There was a 100% chance of damaging Rabbit’s home, a significant chance of harming Pooh, and a very significant chance that the project would be completed within a few days.

• “Blast out Pooh.” There was a 100% chance of damaging Rabbit’s home, a very significant chance of harming Pooh, and a very significant chance that the project would be completed almost instantly.

• “Slim down Pooh.” There was zero chance of damaging Rabbit’s home, zero chance of harming Pooh (although he might have an extended period of relative deprivation), and large uncertainties in the project duration.

Step 3.3. Determining the Value of New Information A useful decision analysis technique is to assess the value of new information.

For example, Gopher needed to estimate the duration of the excavation project to remove Pooh from the doorway. Gopher could perform exploratory excavation, but it could be costly and time-consuming. This analytical technique helps establish the value of new information—how much money can be saved if additional information is obtained through an exploratory excavation.

Step 3.4. Deciding on a Course of Action In some cases, it is easy to select the most efficient alternatives, based on the results of risk analysis. The best alternative based on selected success criteria may be obvious and can be easily selected without further steps.

In many situations, however, the selection between the alternatives is not so easy. Sometimes, different scenarios are executed only if certain conditions exist. In many cases, decisions are made using numerous criteria, which complicates the selection of the most efficient alternative.

In the Pooh example, the decision was based on multiple criteria. The safety of Pooh was the first priority; therefore, the blasting alternative had to be rejected. The excavation alternative was also rejected because it did not provide adequate safety and could cause substantial damage to Rabbit’s house. Therefore, despite concerns about project duration, the slimming-down alternative was selected. Interestingly, in a Russian cartoon version of the same Pooh story, made in the early 1970s, Pooh’s friends and Pooh himself decided that he would not be able to withstand the deprivation imposed by the slimming alternative, so they just yanked him out, causing a great deal of structural damage to Rabbit’s house in the process. Does this say something interesting about the differences between Western and Soviet psychology? Or perhaps the producers of the cartoon were on a limited budget and decided that having the characters forcefully dislodge Pooh was cheaper to produce and therefore a better choice.

PHASE 4: IMPLEMENTATION, MONITORING, AND REVIEW

Phase 4 involves two steps.

Step 4.1. Project Implementation and Monitoring Now the decision has been made, and the selected course of action is under way. But what if, in the middle of the project, an unforeseen event occurs, causing the plan to deviate? Luckily, a valuation model of the selected project alternative is available (created in step 1.5), so the project manager can update that model, perform a new analysis, and then make a decision. It is very important to track project performance constantly and analyze all potential pitfalls and opportunities. In many cases, a new decision-framing step will be required if a new decision within the same project is required.

Tracking project performance helps you to forecast what could happen to the project, even if some activities are only partially completed. Before the project started, you had only one source of input information for the decision analysis process: historical data, which either is objectively defined based on certain records or is the result of expert judgment based on past experience. Now the project manager can also use actual project performance to make decisions. This process for continually improving decisions by learning from the outcomes of earlier ones is one reason that adaptive management is so powerful.

Pooh’s friends continually checked Pooh’s slowly shrinking girth, trying to estimate when he would be slim enough to pop out. Eventually, when their measurements indicated that the time was ripe, they managed to extract Pooh without damage to either Rabbit’s home or Pooh himself.

Step 4.2. Review of the Decision Experience You need to know whether your analysis and decisions were correct. Otherwise, you will make the same mistakes all over again.

Apparently, in this situation, the decision was correct. Some small things could have been done better, however. For example, the sign “Do Not Feed Bear” could have been installed at the beginning of the project rather than after Gopher’s offer of food to Pooh.

Big and Small Decisions

When we ask project managers why a decision analysis process has not been implemented in their organizations, they usually give some form of the following replies:

• We don’t know what the decision analysis process means

• We already have processes, such as project management, and we don’t need to introduce yet one more

• Decision analysis processes are not suitable for our organization or our types of projects

• Our projects are small, so we make decisions based on our experience and intuition

• We are planning to implement the process in the future, but we are too busy right now

• Decision analysis processes require significant resources, including training that we don’t want to invest in at the moment

Interestingly, most project managers use a number of answers from this list instead of just one. However, the third answer (“Decision analysis processes are not suitable for our organization or our types of projects”) happens to be most common. Most managers believe that decision analysis is suitable only for large projects. In their opinion, intuitive decision-making is sufficient for small projects. But what differentiates a small project from a large one? Is drilling one well that costs $2 million a small project for a large oil company? Maybe. But a $2 million software project is definitely a large project. So, it is all relative to the particular industry or organization.

We agree that small projects probably do not require as complex an analysis as large ones. Yet we strongly advise against relying purely on intuitive decision-making for a project of any size.

Table 3-1. Decision Analysis Process for Various Types of Projects

Type of Decision	Suitable Decision Analysis Process	Some Comments
Small tactical decisions during the course of projects	Try to process information logically by answering a few simple questions: • What is the problem? • What do we want to achieve? • What are the uncertainties? • What are the alternatives? • What will happen if each alternative is implemented?	You may use any components of the process described in this chapter that you find both easy to implement and useful. For example, you may start with decision-framing with some simple analysis.
Important decisions concerning small projects or tactical decisions in large projects	You may use some components of the process described in this chapter, if you find them easy to implement and useful. For example, you may start with decision-framing with some simple analysis.	This is the first step toward a formalized decision analysis process in the organization.
Strategic project decisions	Apply the decision analysis process described in this chapter for a comprehensive evaluation of alternatives.	If the complete project depends on this decision, a full decision analysis process will be useful.
Strategic enterprise-wide decisions	Use a consistent, continued, and comprehensive decision analysis process for all project decisions within a portfolio.	Enterprise-wide decisions should be made based on a comprehensive analysis of alternatives, with continued monitoring of results.

The decision analysis process shown in figure 3-3 was developed for strategic decision-making. Making strategic or individual decisions is different than making project decisions. The process should be tailored for project management. In subsequent chapters of the book we will show practical steps how to apply various steps of the decision analysis process to projects. Table 3.1 offers some advice on how you can tailor the decision analysis process to different types of decisions.

In any case, remember that the process should be as simple as possible. If you suggest a full-blown process for your organization, but your organization is not ready for it, it will kill the whole idea (and any hope we have of selling more books to your colleagues!). In addition, remember that in 99% of cases, project decision analysis is simply an exercise in rational thinking.