Information from a Decision Perspective

Determining which information matters and which does not is a constant challenge. In decision making, information connects what we can do (alternatives) with what we want (values); it helps us predict the outcomes of each alternative in terms of our values. And because the future is uncertain, we need to describe the future with possibilities—what could happen—and probabilities—our beliefs about their likelihood. For example, when we flip a balanced coin, there are two possibilities for the flip's outcome (heads or tails), and each has a probability of 50%. There really is no other way to make sense of information when dealing with future outcomes. We could create a few scenarios about what might happen in the future, and add a lot of color to make them very memorable—but those scenarios need to be converted into a full set of possibilities, each with assigned probabilities. Doing that makes them useful for sound reasoning in making decisions.

It may seem logical to insist that decision making be based solely on facts. However, we must acknowledge that data and factual information have one glaring limitation: They are about the present and past, while decisions are concerned with the future, about which there is no certainty. For it to be useful, the best historical information must be translated with judgment into the possible outcomes of our choices and their probabilities. Though humankind has tried for thousands of years to avoid uncertainty—even by reading the stars, tea leaves, and sheep entrails—there is no escape from making judgments in the face of uncertainty.

When we are driving, if the road has no obstacles or unpredictable turns, we could drive just by looking into the rearview mirror. However, few of us would trust such an approach. If we have significant uncertainty ahead, we have to look forward through the windshield and anticipate what comes next. We also can't take in every detail while we drive—we have to pay attention to what is important to our driving decisions. And we can't be sure what the drivers around us will do, so the environment is uncertain. We must appreciate what we don't know—the limits of available information and the uncertainty about the future—and incorporate these limits in our description of possibilities and probabilities.

So what type of information is needed for decision making? Quality information must be both relevant and reliable. Information is relevant when it helps us anticipate the value outcomes that may arise for each alternative under consideration. For example, when considering a new business opportunity, a decision maker needs information such as estimates of future costs and revenues to understand the value of the opportunity. Costs may consist of several components, including production, materials, and facilities costs. And revenues depend on the size of the market, its growth rate, and the share of the market captured by the new business. Laying out the structure of the decision, that is, specifying what information is needed to estimate the value that can be anticipated, is the first step in understanding relevance. This chapter introduces a useful structuring tool called the decision tree.

Even with careful structuring, a decision may require many pieces of information. But not all of these will have the same impact on the value outcomes. So, which deserve the most attention? Answering this question requires consideration of uncertainty. First, the uncertain information should be described by using a range estimate rather than a specific single number estimate. For example, we may estimate that annual maintenance cost for the warehouse facility for the new business will be between $40,000 and $90,000 per year. Then, by applying sensitivity analysis tools, we can cut through the clutter and get to what truly matters for a specific decision—the most relevant information. Systematic sensitivity analysis identifies which uncertainty ranges have the largest impact on the value outcomes, leading to the handful of factors that are the value drivers that truly matter. Efforts to improve our understanding should be concentrated on these value drivers. One of the most powerful sensitivity analysis tools, the tornado diagram, is introduced in Chapter 8, “Sound Reasoning.”

Information is relevant when it is important in the structure of the decision, particularly when it is a key value driver. Information must also be reliable, drawn from trustworthy sources in a way that captures the expert's judgments (including the uncertainty about the future) in a way that minimizes the impact of decision traps and biases. This chapter highlights approaches that can give decision makers confidence in using expert judgments.

An Extended Example: Michael's Job Choice

Michael, a mid-level manager, is facing an important job decision. The decision is a strategic one for Michael—worthy of careful consideration, including deliberate information gathering. His decision will provide a context to show how information can be structured and collected to support quality decision making.

Michael is happy with his current job, but he sees limited opportunities for greater compensation in the years ahead. A new job offer from a small startup company has come his way, and Michael has a decision to make: take the new job or stick with the one he has. Obviously, he could create and consider other alternatives, but to simplify this example, Michael will deal with only two alternatives.

Michael has thought hard about his values. (That thought process is explored further in the next chapter.) He knows that job satisfaction is most important to him, and he believes that income and work hours are the factors that will have the biggest impact on his job satisfaction. He also knows that he must gather relevant information about his two alternatives and their potential outcomes.

Structuring the Relevant Information in a Decision

Michael's information gathering began at the startup company that offered him a new job. During an initial interview with the company's CEO, Michael learned that his responsibilities would be very similar to those of his current position. More hours would be required of him initially as the company struggled to get off the ground, and maybe even more if the business took off rapidly. His starting pay would be $70,000—less than the $80,000 he now earns. But if the company survived its first year, he could be assured of a salary boost to $120,000. Of course, startup companies don't always survive. Michael knows that. If the fledgling company were to tank, he would find himself with no job. Michael needs help—a decision tool—to sort this out.

The Decision Tree: A Tool for Structuring a Decision

A decision tree specifies the sequence of decisions and uncertainties that must be considered. Figure 6.1 begins to map out Michael's situation. The square represents Michael's decision: whether to take a job with the startup company or stay put. The outcomes he might get in terms of job satisfaction show up on the right. If Michael keeps his current job, he is confident that he will stay at 45 hours a week for $80,000. This outcome is shown at the end of the bottom decision tree branch. The outcomes associated with taking the new job aren't so clear-cut.

Whether the startup will succeed is a key uncertainty. A circle is added to the top branch of the tree to represent this uncertainty. Michael has learned that if the company obtains at least $2 million in new funding by the end of the year, the startup will continue successfully. If not, the venture will fold.

Describing Michael's work-hour and income outcomes for the startup job requires another step. These factors are uncertain. To learn more about his work hours, Michael has spoken with the hiring manager at the startup firm. He learned he should expect to work 50 hours per week during the first year. If the startup is successful after that first year, work weeks might stay at 50 hours, but they would be more likely to rise to 55 hours, or even 60 or higher. This new information is represented in the tree in Figure 6.2.

Salary is another uncertainty in Michael's case. By inquiring directly, he learned that if the company is successful, his salary will jump to $120,000 in the second year, but that he shouldn't expect to see any significant raises beyond that for the next couple of years. These outcomes have been added to the decision tree after “startup succeeds.”

But what if the startup fails? Michael is confident that his current employer would take him back, but he's likely to take a pay cut. On speaking with his employer's human resources department, he learned about starting salaries for people with his experience. He concluded that he might be able to regain his current salary when he returns, but he might be forced to take a 10% cut in pay. These outcomes can be added to the decision tree after “startup fails.”

Michael now has a structure for his decision problem, but he needs more information. He recognizes the relevant uncertainties and the possible outcomes, but he hasn't specified how likely they are. What, for example, is the likelihood that the startup company will succeed or fail? He might say, “There's a pretty good chance of success,” but what does that mean? The greater specificity of probabilities is needed. Judgments about what is likely to happen can be added as probabilities on each branch of the decision tree.

Michael must somehow estimate the probability of the uncertain outcomes: of the startup company succeeding, of the hours he would work there, and of his current company hiring him back at his same salary. To answer these questions, Michael solicits the opinions of informed experts, and then forms his own judgment. He may speak with the startup's CEO or CFO about its capital-raising campaign. However, to offset any biases that those executives might have to promise good results, he may then consult with a friend at a venture capital firm to get further insight. What about being hired back by his current employer at the same salary if the startup folds? A chat with a colleague in the human resources department can help him establish a reasonable probability.

Because every case is different, there is no single answer to the question, “How can we find the information needed to estimate the probability of a future event?” However, by approaching the most informed and reliable sources and doing so in a manner that ensures that biases are addressed, it is possible to reach an informed judgment that can be expressed as a probability. This probability is a number, but just because we use a number to express our judgment, that doesn't make it any less subjective. Judgments are inherently subjective; using a number simply allows us to express those judgments clearly.

In Michael's case, the probabilities he estimates for each branch of the decision tree are shown in Figure 6.3. For example, he believes there is a 70% probability that the startup will succeed. If that happens, the probability of his working 50 hours per week is estimated at 35%.

A tree like this is an effective tool for specifying decision information. The sequence of possibilities, including the whole path of what follows what, and the probabilities are clear. Michael has identified the information that he believes is relevant to his decision, gathered it from reliable sources in a way that avoided biases, and assigned probabilities to the uncertainties. Chapter 8 will demonstrate how Michael can use this decision tree and sound reasoning to identify the alternative with the highest value for him.

What Is Reliable?

Besides being relevant, information for quality decisions must be reliable—that is, accurate and unbiased, not based on false beliefs or obtained from untrustworthy sources. Decision quality (DQ) cannot be achieved without reliable information. Furthermore, in complex cases, decision makers simply won't accept information or analysis if they can't judge its reliability. They know all too well that it is possible to find information to support almost any conclusion. Advocates of a particular idea, or people hoping to sell something, always find a way to justify their claims.

So how can we be confident that we have reliable information that provides a solid basis for deciding? Two key things are critical. First, we must leverage trustworthy experts who are credible and willing to acknowledge the uncertainty about the future. In some cases, these experts are readily available, but in other cases, it may be necessary to look beyond common sources to find them. Second, we must elicit judgments from these experts in such a way that we avoid the biases that make information useless.

Is it really possible to get reliable information from a credible expert? The answer is yes. An experienced facilitator who understands the problem of human bias can guide the discussion away from the common traps and pitfalls. This is a task that takes considerable attention and experience. It doesn't happen automatically. If Michael didn't understand how to get reliable information from his friend Victor the venture capitalist (VC), he might end up having a conversation like this:

Would a conversation like that lead to reliable information? Probably not. The venture capitalist's viewpoint may have been overly influenced by the recent failures that are readily available in his memory. That availability bias is a big problem in many cases. Also, the VC didn't ask any questions about the startup itself. Instead, he made a huge number of assumptions about Michael's potential employer, about the amount of capital it required to succeed, about its products, its markets, and its management team. And, as an expert advising his friend about a risky situation, Victor may have felt a need to be conservative, limiting the chance of a bad outcome for Michael. In other words, Victor may have had a motivational bias to give Michael a low probability of success.

As it happened, Michael did understand the importance of getting quality information from Victor. So rather than holding a casual conversation like the one above, he prepared carefully before talking to Victor. With the help of a colleague who is a decision professional, he took several steps to get quality information, including:

• When he learned about Victor's recent experience of funding failures, Michael encouraged him to draw on his full set of experiences to think broadly about the possibilities. Michael also helped avoid motivational bias by telling Victor that he would take responsibility for his own decision.
• Michael defined very specifically what he wanted to know from Victor. Then, when asking Victor for his estimate, Michael avoided throwing out any numbers that would anchor or influence Victor's response.
• Before Michael asked Victor for his probability of success, he asked Victor to list all the things that could make the startup successful, and all the things that could lead to failure. This helped Victor avoid the very common problem that we invariably think we know more than we do. Too often this problem leads to an underestimation of the uncertainty actually faced in a decision, particularly in uncertainty ranges (such as a range on how much funding might be received in a year) where the lows and highs are not extreme enough.

These ideas illustrate some of the steps in the standardized approach¹ that decision professionals routinely use to collect reliable information. Because Michael knew a decision professional who could coach him on this process, he was able to get quality information from Victor, integrate it with what he heard from the company leaders, and settle on a judgment of 70% likelihood that the startup company would succeed in getting the $2 million it needed.²

In the pursuit of reliable information, the goal is not to eliminate uncertainty, but to obtain informed and unbiased estimates of probabilities and ranges for uncertain outcomes. This can be done when the information provider is a respected expert, and when the process for obtaining the information is transparent and specifically designed to avoid biases and traps. With the proper training and feedback, experts can become well-calibrated in making probability and range estimates. That is, they can develop the ability to make probability estimates that over time prove consistent with the frequency that events actually happen. Although it is not something that humans naturally do well, we can learn to make good judgments about an uncertain future.

Things That Can Go Wrong

One of the most common information problems is people's failure to seek the information they need to reach unbiased conclusions. Instead, they look for evidence to support a particular point of view. This is often easy to do, but it will not lead to quality decisions. Decision quality demands a commitment to gather the information that best represents what will happen with a decision. In seeking information, decision makers should be on guard against:

• Biased sources
• Data containing errors
• Information sources without expertise
• Information that has been cherry picked to justify certain conclusions
• The temptation to seek only the information that supports biased opinions and assumptions
• Overconfidence—that is, believing that we know more than we do

Delaying a decision too long can also be a problem. Some people may wish to defer decisions until they've collected the last tiny scrap of information. That would be fine if all the time in the world were available, and if collecting all of the information were free. Unfortunately, additional effort and delay usually carry a monetary cost and often lead to missed opportunities. Information should be gathered only up to the point at which the time and cost of obtaining more is not worth the improvement in the decision. At that point, the information component of decision quality has reached 100%.

Judging the Quality of Information

The quality of information should be judged before a decision is made, with a goal of reaching 100%. Both relevance and reliability are important. Initial questions about the relevance of information should focus on the structure of the decision:

• “Have the possibilities and probabilities for the decision been defined?”
• “Is the structure of the decision well understood, including what follows what? Is there a decision tree that describes that structure?”

Additional probing questions should explore reliability of the information:

• “Who provided information on the key uncertainties? Are those sources trustworthy and reliable?”
• “What steps have been taken to ensure that biases haven't crept into the information?
• “Do experts disagree? If yes, have the points of disagreement been taken into account?”

Together, answers to these questions provide insight on the early quality of information.

Once sound reasoning has been applied with the available information, we will know how that information affects the value outcomes of each alternative. This evaluation will also include sensitivity analysis, which highlights the key value drivers. When the results from the evaluation are complete, more detailed questions can be asked about the information's relevance:

• “Which uncertainties are the key value drivers, leading to the greatest changes in value depending on how they turn out?”
• “How much would be gained if we could learn more about the key uncertainties?”
• “If more time and resources were available, what additional information would we seek?”
• “What outcomes for the uncertainties would make us change our mind about which alternative is best?”

An assessment of the information quality—when the decision is first structured and information is collected, and then later after sound reasoning is completed—will tell us whether to fill gaps or move on.

Getting information to 100% is critical to making a good decision. Of course, more than information is required. We need clear values and sound reasoning to make sense of the information we have, to integrate our thinking about possible outcomes and their probabilities, and to reveal the value of our alternatives. These important requirements for decision quality are explored next.

Key Points to Remember

• All decisions are future-oriented, but there are no facts from the future. Facts and data from the past and present must be translated into judgments about the future.
• Decisions about the uncertain future must be approached in terms of possibilities and probabilities. Possibilities define the potential outcomes that might happen in the future. Probabilities represent our best judgments about the likelihoods of the different outcomes.
• To avoid information overload, we should gather information that is directly related to our alternatives and the value we seek. A decision tree can guide us in that quest.
• Decision trees represent the sequence of decisions and uncertainties, showing the possible outcomes and probabilities for each.
• Decision makers need information that is both relevant and reliable to make good decisions.
• Information is relevant when it helps us anticipate the value outcomes that may arise after an alternative is chosen, and when sensitivity analysis shows it to be a key driver of value during sound reasoning.
• Information is reliable when it is trustworthy and unbiased.