CHAPTER 14

How to Make Better Decisions with Less Data

by Tanya Menon and Leigh Thompson

Maria, an executive in financial services, stared at another calendar invite in Outlook that would surely kill three hours of her day. Whenever a tough problem presented itself, her boss’s knee-jerk response was, “Collect more data!” Maria appreciated her boss’s focus on the data, but as the surveys, reports, and stats began to pile up, it was clear that the team was stuck in analysis paralysis. And despite the many meetings, task forces, brainstorming sessions, and workshops created to solve any given issue, the team tended to offer the same solutions—often ones that were recycled from previous problems.

As part of our research for our book, Stop Spending, Start Managing, we asked 83 executives how much they estimated that their companies wasted on relentless analytics on a daily basis. They reported a whopping $7,731 per day—$2,822,117 per year! Yet despite all of the data available, people often struggle to convert it into effective solutions to problems. Instead, they fall prey to what Stanford University professor James G. March and his coauthors describe as “garbage can” decision making: a process whereby actors, problems, and possible solutions swirl about in a metaphorical garbage can and people end up agreeing on whatever solution rises to the top.¹ The problem isn’t lack of data inside the garbage can; the vast amount of data means managers struggle to prioritize what’s important. As a result, they end up applying arbitrary data to new problems, reaching a subpar solution.

To curb garbage-can decision making, managers have to think more strategically about which information they need to solve a problem and how it should be applied in their decision making and actions. We recommend the “data DIET” approach, which provides four steps of intentional thought to help convert data into knowledge and wisdom.

Step 1: Define

When teams and individuals think about a problem, they likely jump right into suggesting possible solutions. Indeed, many brainstorming sessions begin this way. But while the prospect of “problem solving” sounds positive, people tend to fixate on familiar approaches rather than stepping back to understand the contours of the problem.

Instead, start with a problem-finding mindset, where you loosen the definitions around the problem and allow people to see it from different angles, thereby exposing hidden assumptions and revealing new questions before the hunt for data begins.² With your team, think of critical questions about the problem in order to fully understand its complexity: How do you understand the problem? What are its causes? What assumptions does your team have? Alternatively, write about the problem (without proposing solutions) from different perspectives—the customer, the supplier, and the competitor, for example—to see the situation in new ways.

Once you have a better view of the problem, you can move forward with a disciplined data search. Avoid decision-making delays by holding data requests accountable to if-then statements. Ask yourself a simple question: If I collect the data, then how would my decision change? If the data won’t change your decision, you don’t need to track down the additional information.

Step 2: Integrate

Once you’ve defined the problem and the data you need, you must use that information effectively. In the example above, Maria felt frustrated because as the team collected more and more pieces of the jigsaw puzzle, they weren’t investing the same amount of time to see how the pieces fit together. Their subconscious beliefs or assumptions about problems guided their behavior, causing them to follow the same tired routine time and time again: collect data, hold meetings, create strategy moving forward. But this is garbage-can decision making. To keep the pieces from coming together in an arbitrary fashion, you need to look at the data differently.

In design thinking, you break down the problem and then synthesize it, putting the pieces together again. One technique you can use with your team to integrate the different elements of the problem is the KJ diagram (named after its creator, Kawakita Jiro). The goal is to sort discrete facts and data points into causal relationships. Write the facts on notecards and then sort them into piles based on observable relationships—for example, an increase in clients after a successful initiative, a drop in sales caused by a delayed project, or any other data points that may indicate correlated items or causal relationships. In doing this, you can create a visual model of the patterns that emerge and make connections in the data.

Step 3: Explore

At this point in the process, you may have some initial ideas or solutions based on your KJ diagrams. Now’s the time to develop them. To facilitate collaborative exploration, one of our favorite exercises is what we call the “passing game”: Assign distinct ideas to each team member and give each individual five minutes to develop it by drawing or writing in silence. Then have them pass their work to a teammate, who continues drafting the idea while they take over a teammate’s creation.

Discuss the collaborative output. Teammates recognize how it feels to give up “ownership” of an idea and how it feels to both edit and be edited; they also recognize their implicit assumptions about collaboration. The new perspective forces them to confront directions that they didn’t choose or never would have considered. You can also add multiple sequential passes (like a telephone game) to demonstrate the idea’s unpredictable evolution as three or four teammates play with it. After allowing people this space for exploration, discuss the directions that are most fruitful.

Step 4: Test

The last dimension requires team members to use their powers of critical thinking to consider feasibility and correct for overreach. Design tests to see if your plan for moving forward will work. Under which types of situations will the solution fail? Select a few critical tests and run them. While people often over-collect data that supports their prior beliefs—a tendency known as confirmation bias—people under-collect disconfirming data. By running even a single test that fights this bias, you can see what you need to see, even if you don’t want to.

Instead of ignoring the data entirely or getting overloaded by it, think strategically about your data needs. This involves doing more with less—widening, deepening, integrating, extending, and testing the data you do have to convert it into knowledge and wisdom. In practicing the mental exercises we’ve described with your team, you can curb your appetite for data while getting better at using the data you have.

__________

Tanya Menon is an Associate Professor of Management and Human Resources at the Ohio State University’s Fisher College of Business. Leigh Thompson is the J. Jay Gerber Professor of Dispute Resolution and Organizations at the Kellogg School of Management and the author of Creative Conspiracy: The New Rules of Breakthrough Collaboration (Harvard Business Review Press, 2013). They are the coauthors of Stop Spending, Start Managing: Strategies to Transform Wasteful Habits (Harvard Business Review Press, 2016).

NOTES

1. Michael D. Cohen, James G. March, and Johan P. Olsen, “A Garbage Can Model of Organizational Choice,” Administrative Science Quarterly 17, no. 1 (March 1972): 1–25.

2. J. W. Getzels, “Problem-Finding and the Inventiveness of Solutions,” Journal of Creative Behavior 9, no. 1 (March 1975): 12–18.

Adapted from “How to Make Better Decisions with Less Data” on hbr.org, November 7, 2016 (product #H038UJ).