21
Evaluating Your Product Decisions
Product Management: A Game of Poker or Chess?
Look at the business- and strategy-related books on your bookshelf. You may find that most of those books around strategy illustrate chess pictures on the cover, which is a cute, if misleading, metaphor for strategy. Not in the sense that chess is a game without strategy—on the contrary, it entails a lot of strategizing. But some aspects of the game do not translate well to the profession of product management.
For starters, the rules and chess pieces allow for only certain specific and limited moves. It’s not like knights can start moving diagonally suddenly, whereas we have all seen customers and users behave in ways we didn’t expect or report problems and ask for features and solutions we didn’t have in mind. The biggest difference between chess and product management, though, is that all the pieces are on the board in a game of chess. There is no hidden information. If you think long and hard, or if you use computers, data, and AI, you can likely calculate and find all potential and possible chess moves. Therefore, computers can pretty much always beat humans and win the game.1 Product management is not like that at all. It’s much more like a game of poker.
1. 1997 Deep Blue beats Kasparov, who had run circles around the same computer the year before.
A Quick Game to Test Your Decision-Making Skills
Let’s explore decision making via a quick game of rolling the dice. Are you up for it? We will play four games; the rules are listed in Table 21.1. Look at each of the four games and decide whether you will join the game and bet some money or will bail out. There is a fixed investment and fixed return if you make the right decision, so here we go.
Table 21.1 A Quick Rolling-the-Dice Game to Test Your Decision-Making Skills
Game 1 | Game 2 |
|---|---|
Pay $10 to join and play the game. | Pay $10 to join and play the game. |
Get $20 in return if you roll a 1, 2, 3, 4, or 5. | You get $20 in return if you roll a 1. |
You get nothing if you roll a 6. | You get nothing if you roll a 2, 3, 4, 5, or 6. |
Game 3 | Game 4 |
Pay $10 to join and play the game. | Pay $10 to join and play the game. |
Get $20 in return if you roll a 1, 2, 3, 4, or 5. | You get $20 in return if you roll a 1. |
You get nothing if you roll a 6. | You get nothing if you roll a 2, 3, 4, 5, or 6. |
Are your bets placed in the four games? Then let’s inspect your results.
If you think that we made a copy-paste error in the table, we didn’t. You may have noticed that games 1 and 3 are the same, and so are games 2 and 4. It’s not a mistake. In these four games, most people seem to be willing to bet their $10 in games 1 and 3, but they are reluctant to play games 2 and 4. However, when we run this exercise in a course setting and reveal a game’s results after betting, people get the hang of it at game 4 and end up placing their bets after all. So, after playing four games, here are the results:
Game 1: won
Game 2: lost
Game 3: lost
Game 4: won
Let’s say you played all four games. You won twice and lost twice. So, in which of these games did you make a good decision? And which ones were bad decisions?
Game 1: In this game, you would have made a good decision to place your bet. Deciding to join the game led to a good outcome or result, and the decision was a good decision because the odds were strongly in your favor.
Game 2: In the second game, it would have been a bad decision to place your bet, and this decision would have led to a bad outcome or result. In this case, the odds were clearly not in your favor, yet you took the bet anyway. You could have expected a bad outcome or result to happen.
Game 3: In this game, you made a good decision to place your bet, but you had a bad outcome or result in the end. Although the odds were strongly in your favor once more, you had some bad luck this time and didn’t double your investment.
Game 4: In this game, you made a bad decision but were lucky to achieve a good outcome or result. Although the odds were not in your favor, you just happened to win this round.
This gaming exercise reveals a pattern across organizations when it comes to decision making. When talking about good or bad decisions that people made in the past, they often identify decisions as “good” ones if the results or outcomes were positive. Similarly, they identify “bad” decisions as decisions with negative results or outcomes. This is often referred to as a resulting pitfall. Of course, considering the results and outcomes is important when evaluating decisions. However, the results and outcomes achieved are not always directly related to the quality of the decision itself. As we’ve seen in the four games, sometimes we get lucky, and sometimes we don’t.
Evaluating Decisions in an Honest and Transparent Way
When you want to evaluate decisions more effectively, it helps to look at decisions from a broader perspective than the results only. As with any skill, to become better at decision making, you need regular practice and evaluation to learn from. You must inspect and learn from your decisions, regardless of the result. To evaluate decisions more effectively and to learn, here are some reflective questions that might help:
Was the decision made by the right person or the right group of people? Was the decision made alone, or after consulting others?
What information was available, transparent, and used to make the decision? What information was missing?
Could more information have been gathered before making the decision? Would that have changed the decision?
What type of decision was it? Easy? Complicated? Complex? Or chaotic?2
2. The Cynefin framework discusses these different kinds of environments, including several ways of dealing with situations, work, and decisions within those environments. We recommend you explore this model, as it may help you in dealing effectively with work and decisions in various types of environments.
Did the decision require a lot of buy-in or little buy-in from other people?
What was the process for making the decision? How was it made?
Was the decision made at the right moment? Was it made fast/slow enough?
Was it a good decision? Or were you lucky this time?