Reading the plot of the Prisoner’s Dilemma

The story goes like this. A bookmaker is robbed, and the police apprehend two suspects for the crime. The detective investigating the robbery puts the two suspects in separate rooms — so that they can’t communicate — and makes each of them a one-time offer of a deal:

• If neither participant confesses: Both get six months in the slammer.
• If one participant confesses and the other doesn’t: The participant who confesses is released, while the one who doesn’t confess gets a five-year sentence.
• If both participants confess: They both go to jail for two years.

The question is: When the suspects are presented individually and separately with the same deal, what happens?

On the face of it, you may think this is a no-brainer for the suspects. Obviously, the best outcome is that neither of them confess and they each end up serving six months. After all, this means that the total time served by the two suspects is the least. Obviously, they’d be crazy to take any other action. So neither suspect should confess, right?

If you are thinking like that (and many do before they investigate the scenario), alas, you’d be wrong. To see this, suppose the first suspect, whom we’ll call Mr. Pink, thinks like you did and decides he doesn’t want to confess. He doesn’t know what Mr. Blue (his partner in crime) has decided. Mr. Pink would like to think that Mr. Blue hasn’t already taken a deal that puts Pink in prison for five years. However, on second thought, Mr. Pink realizes that Mr. Blue has a big incentive to rat on him if Mr. Blue thinks for a moment that Mr. Pink won’t confess. And of course, if Mr. Pink does confess, Mr. Blue is better off confessing too. Similarly, locked away from Mr. Pink, Mr. Blue doesn’t know whether or not Mr. Pink has decided to rat him out too, sending Mr. Blue to prison for five years. Because neither suspect can take the chance that the other person isn’t a stool pigeon cooperating with the police to cut himself a better deal, neither can rely on the other not confessing. So both confess and get a two-year sentence each — game over.

The key to understanding this result is that both suspects (players) are considered economically rational agents (as defined in Chapter 2), and therefore each player wants to do the best for him, given what he anticipates the other player will do. Both players need to think through what is the best action to take given their best guess about what the other player will do. The reasoning goes like this: “If my partner is not going to confess, then I’m better off confessing. If my partner is going to confess, then I’m better off confessing as well. So no matter what my partner does, I should confess.” Both players reason like this and both confess. The incentive to confess or the payoffs from confessing are key to this game, and that is why they make the police’s job easy.

Solving the Prisoner’s Dilemma with a payoff matrix

Economists look at the Prisoner’s Dilemma through some of the tools of game theory. The first and perhaps clearest way to do so is with a payoff matrix — a simple chart that shows what the payoffs are to both participants given their choices in the game. In Figure 16-1, we label the two participants as Mr. Pink and Mr. Blue, and the two actions they can perform in this setting as “Confess” and “Don’t confess.”

When you read down the columns, the payoffs to Mr. Blue’s actions (or moves, in game terminology) are shown against the payoffs to Mr. Pink’s moves. For example, in the situation where Mr. Blue doesn’t confess but Mr. Pink does, the payoffs are no loss of time in jail for Mr. Pink and a loss of five hard years for Mr. Blue. Similarly, when you read along the rows, the same pair of moves gives the payoffs a loss of nothing for Mr. Pink and a loss of five years for Mr. Blue.

The payoffs are dependent on the actions of both participants. For instance, although confessing lets Mr. Pink out of serving jail time, it does so only if Mr. Blue doesn’t confess. If Mr. Blue also confesses, the advantage Mr. Pink gets from being the only one to confess is wiped out.

This sort of matrix is called the strategic form or normal form of the game, and its purpose is to clarify how the payoffs depend on the strategies chosen so that you can see clearly which strategies wouldn’t be chosen and which would be. Most importantly, it helps you to see one of the most important concepts in game theory: the Nash equilibrium.

Finding the best outcome: The Nash equilibrium

Named after the celebrated mathematician John Nash (played by Russell Crowe in the biopic A Beautiful Mind), the Nash equilibrium has a special role in game theory. It’s defined as that set of strategies where no player has an incentive to change. Or less formally, the combination of strategies where players are doing the best they can, given what other players are doing.

In the Prisoner’s Dilemma, the Nash equilibrium is a pair of strategies — in this case, one for Blue and one for Pink — such that playing that strategy makes the player as well off as he or she can be given the other player’s strategy. The outcomes for the Nash equilibrium are in the top left corner of Figure 16-2. This pair of strategies make each player as well off as he or she can be, given each other’s options.

The Nash equilibrium does not lead to the payoff where any given player is best off, nor does it lead to the best outcome for both participants collectively. Instead, it takes into account that both players want to do the best for themselves, not knowing what the other player is going to do. Chapter 17 talks more about the Nash equilibrium’s special properties.

Applying the Prisoner’s Dilemma: The problem of cartels

In economics, a cartel is a collection of organizations that act collectively to get the best deal for themselves. The most famous example is the oil and petroleum exporters’ group OPEC, which attempts to restrict the supply of oil to the market in order to get as high a price as possible for its exports. Cartels in the U.S. are illegal under the Sherman Act. The collusive agreement to raise prices is a restrictive trade practice that operates in their members’ interests and against the public good. Therefore, the fines for being a cartel or secretly operating one are extremely high. The U.S. Department of Justice has already imposed $2.7 billion in cartel fines this year, representing more than 85 per cent of the total fines handed down across the world. (OPEC, by the way, is not an illegal cartel because it is made up of countries, rather than companies, and U.S. competition law doesn’t make it illegal for countries to form cartels.)

The fact that cartels are illegal means that cartel members can’t rely on an enforceable contract to ensure that everyone acts in the collective best interests of the cartel. Even if all the members of the cartel do agree, for a period of time, to forego the best individual outcome in favor of the collective best, they always have an incentive to return to doing their individual best. If one does, the other members can’t go to court and sue the defecting member for breach of contract — because that would mean admitting being part of a criminal conspiracy. (OPEC is an interesting exception. See the nearby sidebar “Oiling the wheels: The OPEC cartel.”)

Let’s think of the cartel’s dilemma as a Prisoner’s Dilemma and look at it using the strategic form of the game. We begin by walking through the payoff matrix in Figure 16-3 as representing an agreement between two companies, (Blue PLC and Pink, Ltd.). Either company has a choice of playing Hawk or Dove (refer back to the section “Setting the Game: Mechanism Design” for a definition of these terms).

Looking at the payoffs in Figure 16-3 (which are all in millions of dollars), you can see that the collective best, which the cartel or collusive agreement wants to achieve, is in the bottom right corner, where both Blue and Pink are playing Dove. But if one plays Hawk while the other plays Dove, the Hawk (which means cutting prices and selling more) gets a higher return ($130 million in this case) by stealing market share from the other, who in this case takes a loss being the high price seller (top right corner).

Neither party can trust the other to stay in line with the agreement, and so they both default to playing Hawk, and acting in their own best interests. As a result, the Nash equilibrium is (Hawk, Hawk) in the top left corner, and the cartel falls apart.

This is one reason why cartels don’t tend to be long-lasting, irrespective of whether a regulator is able to use legal action to keep them in line. As the saying goes, there’s no honor among thieves, and the logic of the Prisoner’s Dilemma bears that out to some extent.

Of course, regulators also help cartels break down and get to the Nash equilibrium — as the police do in the earlier section “Reading the plot of the Prisoner’s Dilemma” by giving the parties an extra incentive, such as shorter prison terms, to give in evidence. In the case of competition law, the practice is to give a free pass to the first party who talks to the authorities. That way, any party that thinks it may get caught has an incentive to come forward and tell tales on its former cartel partners.

Escaping the Prisoner’s Dilemma

An alternative heading for this section could be “How to form an organized crime syndicate without really trying.” Game theory has investigated one particularly interesting example of cartel behavior: that of organized crime syndicates, when they operate a cartel that tends to work for the benefit of those under its influence, although not generally for society as a whole.

In Chicago during the reign of Al Capone, for instance, the local mafia had a monopoly on the supply of pasta-making machines, which were an essential piece of physical capital for any Italian restaurant in the city. However, the restaurants benefitted from Capone’s patronage. Sure, they had to pay him fees, but they could rely on him to take care of any competition that threatened to undercut them. Thus, the mob ended up as a way of enforcing a cartel agreement and guaranteeing higher profits for its favored establishments.

An organized crime syndicate can achieve a similar outcome by changing the payoffs for its members. When you look at the syndicate as a cartel and understand its payoffs, you can see that, in essence, it changes the payoff for confessing to a crime from whatever offer the police make to the easily understood dead. So if you’re a member of a syndicate, and you’re offered a deal that tempts you to confess, you don’t choose it — unless you feel like sleeping with the fishes.

In Figure 16-4, we summarize the strategic form of the Prisoner’s Dilemma when a crime syndicate can change the payoffs. As you can see, the payoff to any actions involving confessing are now “Dead” (which you can interpret as infinitely large and negative). Thus the Nash equilibrium is now not to confess.

This change in the Nash equilibrium provides the rationale for the formation of syndicates. If you can’t get criminals to cooperate with each other, change the payoffs so that it’s in their interests at least not to confess. Capone, Corleone, and several others took that lesson to heart.

Designing the Stag Hunt

The Stag Hunt is set somewhere around the dawn of time. Littlenose and Bigfoot set out from their village one morning to go hunting. They can choose from two possible locations: One field contains a population of hares, and the other field has stags. Any hunter on his own can capture a hare, but to capture a stag, both hunters must work together. The model assumes that they must both make their decisions with no idea of whether the other is going to make the same choice or a different one.

We give names to the two possible strategies for Littlenose and Bigfoot:

• Hawk: They strike out individually looking for hares.
• Dove: They turn up independently at the same field as the stags, hoping to coordinate their actions so that they can bring home a bigger meal for the village.

Next we assign values to the potential meals:

• Hare: As the smaller creature, the hare gets a smaller value. Call that a value of 1.
• Stag: To the larger stag, we assign a larger value of 4. Any number larger than 1 will do, but we’ll use 4 to keep things clear.

We want to compare possibilities and solve for a Nash equilibrium, so we put up the payoffs in strategic form, as in Figure 16-5. Now look at the possible strategies and outcomes to see where the Nash equilibrium lies. Can you spot it?

Sorry, that was a sly question, because in fact two Nash equilibria exist: one that maximizes payoffs (top left) and one that minimizes risk (bottom right). To see why, we think through Littlenose’s decisions:

• Littlenose plays Dove: He goes to the field where the stags reside and hopes that Bigfoot makes the same decision to play Dove. If Bigfoot doesn’t, Littlenose will have an incentive to change. But if Bigfoot does, they successfully get a stag and the higher payoff, and neither has an incentive to change.
• Littlenose plays Hawk: He goes to the hare field and whatever happens, he gets something to eat. If Bigfoot also plays Hawk, they bring home a nice hare each and are in equilibrium. If not, and Bigfoot plays Dove, it has no effect on Littlenose, although he could’ve brought home a far bigger meal if he had also played Dove and visited the stag field.

Therefore, Dove, Dove is the payoff-maximizing equilibrium (a stag). Hawk, Hawk is a risk-minimizing equilibrium (at minimum, a hare each). But neither Hawk, Dove nor Dove, Hawk can be an equilibrium, because given what the other player chose, a player could make himself better off by changing to the same strategy as the other player (hence the game being seen as a coordination game).

Economists use the Stag Hunt game in situations where both parties can gain by taking an action, but because each party also has a defensive strategy and because they don’t know what the other party will do, they play their defensive strategy to avoid losses. This case often applies in international negotiations where a treaty is the collective best, but the risk-minimizing strategy is to act in your own immediate interests.

Examining the Stag Hunt in action

To our great delight, a Stag Hunt came to light in an economics class at an American university, where a poor choice of mechanism design led to students being able to exploit the system.

The students in a microeconomics course were to be graded to a curve — that is, by their relative position in the class instead of by the raw results that they themselves would achieve on the final exam. The students were upset about this, but they knew a bit of game theory and used it to their advantage to ensure that everyone got the top mark in the class. If no one sits the exam, a zero grade is the top grade in the course and every student gets it (that is, no one must sit the final exam to get it). We immediately gave them a long-distance standing ovation for brilliant use of game theory.

The key to this problem lies in two points:

• Analyzing this situation, an economist would use the Stag Hunt model to see where the benefit of collective action is.
• The corollary of a ranking system is that if everyone is equally terrible, then everyone is equally good. Another way of looking at it is that if everyone gets last place, they all tie for first.

Suppose that the exact marking scheme gives the first-place student 90 percent, the second 89, the third 88, and so on. Call that scheme the “teacher’s offer” and denote it M:

• If all students play Hawk: They play by the test scheme and their mark at the end is M_i.

• If all students play Dove, don’t play by the test rules, and instead submit blank papers: They all receive the payoff for first place: a mark of 90.

  Hawk and Dove refer to the strategies of players in cooperating with each other and not the teacher.

• If some of the players play Hawk and some play Dove: The Hawks take the test and get the teacher’s offer based on their place in class, M. The Doves don’t take the test and get a mark that reflects their worse performance; call it W (for this to work the offer, W, must be worse than any given possible mark M).

Figure 16-6 shows the strategic form of the game. As in the traditional Stag Hunt in the preceding section, two Nash equilibria exist:

• Hawk, Hawk equilibrium: Every student goes along and takes the exam as normal and is placed on the curve.
• Dove, Dove equilibrium: Every student submits a blank paper and guarantees themselves 90 per cent.

The coordination problem is how to coordinate students’ decisions so that they all end up playing Dove. This particular class solved it by waiting outside the exam room so that they were able to monitor anyone going in to the room. If they saw someone go in, they rushed after the person eager to get started on the exam with the hope that they’d get M (rather than W) at least. In fact, no one walked into the room and all students got 90!

A better mechanism for assessing and grading performance would have been to spring the test on the students, or to tell them the grading scheme after the test, or even to set the test in different rooms so that no one could monitor whether anyone else was cooperating. That way, the students would have then been unable to solve the game’s coordination problem.

Investigating a repeated game using the extensive form

The strategic or normal form of a two-player game is a technical way of saying that the payoffs are arranged in a matrix, where player 1 is the Row player, and player 2 the Column player. The Nash equilibrium for a one-shot game can be found by elimination of worse or dominated strategies. Games that are repeated or go on over many rounds are often better understood by looking at them as if they were a set of sequential decisions. This representation of the game is called the extensive or tree form of the game.

We do this for the Prisoner’s Dilemma in Figure 16-7. The important thing to notice is that the payoffs at the end of the tree are the same as the payoffs in the strategic form of the game. We’ve done this for a one-shot game because it keeps everything simpler for now — the key point to note is that the payoffs at the end of the extensive form should be the same as the payoffs in the strategic form we used up until now.

The extensive form and the strategic form are equivalent ways of depicting a game, in the sense that the final payoffs have to be the same. The key advantage of the extensive form is that it can make the sequence of moves clearer where that’s something that matters in the game. In these one-shot games where players move at the same time, we don’t really need the extensive form all that much to tell you about the equilibrium. As games get more fiendish, however, the extensive form yields more information about the underlying structures of the game.

Mixing signals: Looking at pure and mixed strategies

In many games — especially sporting matches — a player gains by being unpredictable. To understand why a tennis player or a quarterback wants to keep their opponent guessing, game theorists make a distinction between a pure and a mixed strategy for a player:

• Pure strategy: Describes what the player will do, with probability 1 at each point in the game in the game. In the Prisoner’s Dilemma, both Hawk and Dove strategies are pure strategies.
• Mixed strategy: Describes what the player will do, with some probability at each point. Imagine that the two players from the Prisoner’s Dilemma example toss a coin in order to decide whether to play Hawk or Dove. In this case, each plays Hawk with 50 percent probability and Dove with 50 percent probability.

As a general rule, for a player to want to “mix” her moves — that is, play Dove if heads or play Hawk if tails — she has to be indifferent between playing Hawk and playing Dove. For if she weren’t indifferent, she could do better by playing Hawk (or Dove) against her opponent, in which case she shouldn’t be mixing. When playing sports, you typically want to keep your opponent guessing about what you are going to do. A mixed strategy can tell you what is the best way to keep your opponent guessing.

Game theory can help us look at the best strategy for a penalty taker in a soccer match, as mentioned at the beginning of this chapter.

Another example is what is called the madman strategy. This game makes it clear how “mixing” strategies is a way to play a repeated game. Here a player deliberately tries to confuse the other player into making a sub-optimal move by switching between moves unpredictably. In a repeated Prisoner’s Dilemma, a player using the madman strategy switches between Hawk and Dove in such a way that the other player can’t extract rhyme or reason from his moves, sometimes confessing and sometimes not confessing. The effect is to create uncertainty in the other player, who’s deprived of information about what the first player may do and therefore has less knowledge upon which to base his actions.

The madman strategy isn’t very good for getting cooperation. There is an important difference in economics between the type of game in which one party can make a gain over the other player and a more cooperative game where both parties gain from achieving cooperation. In a competitive market setting, holding an advantage over the other party in a repeated game can be desirable. But if cooperation is the desired goal, building trust is often the first stage, and you will have difficulty trusting someone who keeps changing his moves on a seeming whim.