17. Feedback Loops

If winning isn’t everything, then why do they keep score?

—VINCE LOMBARDI

You are playing a game of Monopoly with your friends, or at least you were earlier. Now, two of your friends have gone bankrupt and it’s just you and one opponent. He has amassed four monopolies and a massive pile of cash. You have no monopolies, and it’s just a matter of time before you hit one of his properties and the game ends. The game is a foregone conclusion, so your decisions have become meaningless. Your odds of coming back to win are astronomical, and besides, it would take a couple of hours of fruitless rolling. Your opponent does not want you to give up; he wants to earn the victory. It’s not a lot of fun. What happened? The game at some point went from competitive and fair to a situation with meaningless decision-making. The rules didn’t change, so what caused this?

Any game in which a player’s actions affect the game state for later actions has the possibility of entering what is known as a feedback loop. In a feedback loop, the player’s actions affect the ability for that player to get closer to a goal. In other words, successes or failures early affect the chances of successes or failures late.

Feedback loops come in two flavors: positive and negative.

Positive Feedback Loops

Positive feedback loops in games are mechanics that reinforce the success or failure of the player and make future successes/failures more probable. You will find this type of feedback loop in numerous games, especially in role-playing games. The cliché often used to explain positive feedback loops is that the rich get richer. Having money is often a prerequisite to gaining more money: You need capital to start a business, so the ones making money are the ones who already have money.

In Risk, players control armies that are trying to take over the world. They do so by battling other armies. When players win battles and control entire continents, they receive additional armies every turn—this, in turn, “feeds back” into the simple result that having more armies makes you more likely to win battles. So not only does winning a battle give you a direct advantage (the opponent has fewer units), but you also feed back that victory into gaining more units, which should ensure further victories.

Quake II also contains positive feedback loops. The relevant mechanic is that when you die, you restart with guns that are not as powerful as the ones that you can pick up on the battlefield. Therefore, the player with the first kill has an inherent advantage—he’ll have better weapons than the newly respawned player and will be more likely to kill her again, given equal skill and health.

Professional baseball in America has a somewhat muted positive feedback loop. Major League Baseball (MLB) does not have a salary cap, so teams can spend as much money as they have to acquire the best players. The best players lead to the best success on the field. Success on the field leads to more money for the team as people bid up tickets and buy merchandise. That money can then be fed into buying even better players.

The Facebook game Mafia Wars originally had a pretty severe positive feedback problem. Players could buy properties that would earn them in-game money over time. Players could dump all their money into buying Mega Casinos, which in turn gave them a lot of additional money in rent, which they could then use to buy additional Mega Casinos (Figure 17.1). The amount of money players could earn from this dynamic made any other choices in the game irrelevant. Players could log off for months and then come back with enough money to buy any item in the game.

MAFIA WARS IS A TRADEMARK OF ZYNGA INC. AND IS USED WITH ITS PERMISSION. © 2015 ZYNGA, INC. ALL RIGHTS RESERVED.

Figure 17.1 Positive feedback loops in Mafia Wars lead to exponential growth.

During the development of NCAA Football 08, its designers implemented a positive reinforcement system named DPR (dynamic player ratings). The idea was to implement the most straightforward of positive reinforcements. Player events tied to random die rolls were affected by ability ratings. A player with a 90 catching ability would catch more often than a player with 70 catching rating. As a player succeeded during the course of a single game, his ratings would be temporarily boosted, causing more successes. When a player failed, his ratings would decrease. However, this became a problem. Player ratings converged to either essential perfection, as successes bred more successes, or complete ineptitude, as failures bred more failures. That system had to be tweaked; the smallest change could spiral out of control, creating perfect passers or punters who could not hit the broad side of a barn. In the final product, the feature’s effect was tuned down to look significant to the player, but the probabilities affected were only slightly nudged. The essential nature of the positive feedback dynamic was never addressed. This problem is endemic to many sports games, as “stat boosts” seem to be the only reward structure worth pursuing.

Positive feedback is not always bad. For one, it helps direct player actions. Players want to become richer and more powerful, so giving them rewards for succeeding makes sense. It also breaks stalemates. Consider a game such as the card game War in which the player with the highest card value wins. It’s entirely random, so on average, each player should win 50 percent of the time. A series of War games will always be in stalemate. No previous game affects the current game. But if the design added a positive feedback loop that changed this dynamic, the game could progress toward an end state. Say that players get to turn over an additional card if they won the last hand and can add that new card to their total. This gives the winning player an advantage that should cause a positive feedback loop that leads to an end-game state where one player has a distinct advantage over the other and can be decided the winner. I am not saying that this will make War fun, but it can end the thing.

Always be vigilant to ensure that positive feedback loops don’t get out of hand.

Negative Feedback Loops

Negative feedback loops in games are mechanics that hinder succeeding players from further success or failing players from further failures. The classic example is Mario Kart 64.

Mario Kart 64 is a racing game in which players have weapons that they can use to speed themselves up or to hinder the progress of their opponents. In this installment of the Mario Kart series, Nintendo added the blue shell weapon. The blue shell is the ultimate negative feedback loop. It hones in and destroys the player in first place no matter where they are on the race track. There is little the first-place player can do to avoid this. The appearance of the blue shell is almost inevitable—almost every race has one. It’s always on the mind of the first-place player. Succeeding players are punished. No one wants to be too far ahead of the pack because the blue shell will knock them back. Instead, it encourages a dynamic where players want to be in second place until the final parts of the race—safe from blue shells but also close enough to take the lead before the finish.

Many sports, such as football and basketball, use negative feedback. You need the ball to score, but after a score, the opponent receives the ball. This allows the opposing team a higher probability of responding with a score of their own, keeping the game close. If the team that just scored receives the ball again, they can score over and over, destabilizing the game.

Negative feedback is dangerous because it sends mixed signals. You direct players to succeed—win the race, kill the bad guys, and so forth—but at the same time, the negative feedback mechanics do not agree with the stated goals if you punish your players for striving toward those goals. The goal of Mario Kart players is to outrace their opponents. However, the blue shell punishes those who achieve the goals. This is the type of dissonance that is dangerous when you implement negative feedback.

Some racing games suffer from this problem. Since most racing games are more interesting when you are jockeying for position with other cars, if a player does too well and leaves the pack behind, the artificial intelligence (AI) will cheat and make the opponents go supernaturally fast to catch up with the lead player, which encourages more nail-biting racing. This essentially punishes the player for succeeding, which goes against all the other mechanics in many of these games that are tailored to reward the player for racing well. Players are almost unanimously against this technique—do a search in racing game reviews for “rubber-band AI” and you’ll see universal disdain. So designers are faced with a conundrum—have no “rubber banding” and let skilled players race off into the distance (effectively racing time trials), or have the AI cheat to make the race more exciting.

Even relatively simple games can use negative feedback. In the trivia game Buzz!, the player in first place never gets to select the trivia category. The player who is selecting should naturally try to pick something that she knows the leader does not know, and thus this should serve as negative reinforcement.

Eight-ball pool is another example of a particularly elegant negative feedback loop. The player who is behind has more balls left on the table to choose from and so has a better chance of having an “easy shot” lined up. The player in the lead is also encumbered by the trailing players’ remaining balls; the trailing player does not have to deal with as many of the opponent’s obstacles.

Feedback Loops in Action

Feedback loops can come in four forms: two for creating positive feedback and two for creating negative feedback. These can be combined in a number of ways.

Positive Feedback Loop Methods

The first of two methods for creating a positive feedback loop is to reward success. The simulation results in Figure 17.2 show how this affects the game over time. In the simulation, each player starts with an identical probability of success. Each trial then adjusts the probability of future successes back on the feedback loop method. In this one, the player’s probability of success increases whenever he has a success.

Figure 17.2 A simulation wherein positive feedback is achieved by rewarding success. Early victories cause increasing leads.

In the simulation, the red player gets off to some early successes, which allow her an increased chance of having future successes. The blue player has early failures, however. This allows the red player to pull ahead and have an ever-increasing gap between herself and her opponent. Blue will have a difficult time catching up.

The second way of implementing positive feedback is to punish failure, which shows results similar to rewarding success. In this simulation, if the player has a failure, his odds of success are reduced (Figure 17.3). Since both players start from a point where they have the same chance of success, they both seem to stay close over time. However, the positive feedback adds up for the blue player, who plateaus earlier than the red player. These plateaus happen because failures eventually push the player’s probability of success so low that it becomes very unlikely that he will succeed. Because of earlier victories, that plateau happens later for the red player.

Figure 17.3 A simulation wherein positive feedback is achieved by punishing failure. Early failures lead to increasing leads for the player who does not fail.

You aren’t limited to using one method or the other. Many games both reward success and punish failure. This happens most often in zero-sum games where resources shift from player to player on a success. An example is heads-up poker. When a player wins a hand, she gains money and her opponent loses money. This means the winning player is rewarded for success and the losing player is punished for failure. This accelerates the differences between the two players (Figure 17.4).

Figure 17.4 A simulation wherein positive feedback is achieved by both rewarding success and punishing failure. Winners of early trials gain a gigantic advantage.

Negative Feedback Loop Methods

Negative feedback loops work to moderate differences between players and can similarly be achieved in two ways: by rewarding failure and punishing success. In both cases, players stay close together, changing leads back and forth. In cases where both methods are employed, as in Mario Kart 64, this creates a higher possibility of lead swapping (Figures 17.5–17.7).

Figure 17.5 A simulation wherein negative feedback is achieved by rewarding failure. Scores remain close; however, players are incentivized not to play at their best.

Figure 17.6 A simulation wherein negative feedback was achieved by punishing success. A pattern similar to rewarding failure emerges.

Figure 17.7 A simulation using both techniques to achieve negative feedback.

Mixing Feedback Methods

Many games pair positive feedback with negative feedback:

• In the card game Dominion, players must collect victory cards to win. But victory cards have no other purpose and thus slow down the player’s ability to perform actions with his cards on his turn. Thus, the player has to balance between acquiring cards that give him more power and cards that will eventually allow him to win the game.

• In the board game The Settlers of Catan, players who have the most resources can build the most victory-point yielding buildings or cards. But since trading is a large portion of the game, players tend not to trade with players who are clearly in the lead. Thus, players who are not comfortably in the lead have an advantage that the leader does not have.

• In the Civilization series, players who attempt a science victory gain access to more technologies that help them advance in the game, including amassing more technologies. Players who are further behind in technology can use less-expensive espionage on the leading civilizations, which gives them a bonus to make up for their early disadvantage.

Fixing Problems

Positive feedback loops occur naturally for a simple reason: Players direct themselves toward actions that make them stronger. In RPGs, they buy swords that let them kill bigger creatures that drop better loot that allow them to buy even better swords. Nothing is inherently wrong with that, but it can cause problems. Negative feedback loops are much harder to come by because they can be simply contradictory to stated goals and nonintuitive. Designers guide players to help them succeed, and then they sabotage them behind the scenes with negative feedback so that the result of succeeding is bad.

Designers spend more time figuring out solutions to positive feedback situations than negative ones because positive loops cause more problems:

• POOR INTEREST CURVES. Even single-player games can fall victim to poor interest curves. What if the player spends all her time trying to level up instead of exploring all the great content you have for her? What if the drops she gets from leveling up make the rest of the game too easy? How can you reward the player with one hand and take it away with the other so that she keeps progressing through the game?

• PROBLEMS OF BALANCE. Multiplayer games need to use positive feedback sparingly. If the entire game is decided by the time the first winning move enters the positive feedback loop (such as in the first kill in Quake II), then other players may wonder why they should play the rest of the game. In such cases, the loser is continually beaten by the first winner, and without the hope of winning himself, he is probably not having much fun.

• ENDGAME PROBLEMS. World of Warcraft has a problem. Players get stronger and stronger, but the game can only offer a finite amount of content. Eventually, the player has to reach the strongest sword in the game. At that point, players cannot be directed to kill bigger bad guys for the purpose of better drops that have better stats. The designers have to deal with the problem of “what to give the player who has everything.” Naturally, they have done a good job overall because they have a healthy base of players at the maximum level, but this problem exists only because of the positive feedback loops in the game itself.

An additional reason to root out positive feedback loops is that close games are more exciting for both the winners and the losers.¹ In one study, participants bet on simulated horse races and then rated them for personal entertainment value. The close races were rated as more interesting. Even for players who bet on a horse that took an early lead and sustained that lead (thus having a monetary reason to find the race compelling), the close races were more enjoyable.

¹ Reid, R. L. (1986). “The Psychology of the Near Miss.” Journal of Gambling Behavior, 2(1), 32–39.

One technique you can use to fix positive feedback so it does not get out of control is to decouple what the reward affects and what the goals require. FarmVille and other social games are all about walking up a treadmill to get the next doodad. But generally, the doodads in these games are not things that help the player succeed. Often they are cosmetic or tangential. For instance, although the harvester in FarmVille allows the player to harvest in fewer clicks, it does not make the plants grow faster. The reward is reinforcing the click-to-harvest mechanic and not the XP-gaining growth mechanic. Thus, players can lust over the harvester and harvest like silly without breaking the game. This is a fine technique, but it is not perfect because the player has to desire a reward of something other than what will give him more power to achieve. Thus, it can be applied in only some situations.

Often, if the positive reinforcement is knocking the game out of balance and the designer cannot simply remove the offending mechanic, the best solution is to pair positive reinforcements with negative ones. RPGs do this: When a player levels up, monsters also become tougher. So the player gets stronger, but her enemies do as well, likely at the same rate. The power increase is positive feedback. The enemy’s increase is negative feedback. This would be noticeable to players if put that explicitly, so what designers do is allow the player to really beat up on some lower-level thugs after powering up; this gets them feeling invincible before they are thrown against newly beefed-up baddies in the next level. This has to be carefully balanced.

In The Elder Scrolls III: Morrowind, players could choose to level up, and if they did so, stronger baddies and increased player stats would appear instantly. However, a player could stay as a level 1 player for as long as he chose. The quest items that the game gave did not care about the player’s level and got better and better as the game progressed, even if the player’s level did not increase. If a player chose to stay at level 1, it meant that the game never threw harder enemies at the player, while at the same time, it gave better and better items as the game progressed. After all, why should the player make the game harder for himself? It was certainly not what the designers intended, but many players chose that path, remaining at level 1 for the entire game.

“Fixing” positive feedback does not even have to really fix the problem—it only has to fix the appearance of a problem. The National Football League (NFL) saw the same positive feedback problem that Major League Baseball (MLB) experiences at the seasonal level and added negative reinforcements—a salary cap (a limit on how much a team can spend on personnel) and reverse draft selection (the worst team gets the first chance to choose the best player coming out of the collegiate system). This is generally seen as more fair. However, from 1967 to 2014 in MLB, which does not have a salary cap and whose draft is less important, the top four teams (the Yankees, Cardinals, Athletics, and Giants) combined won 39 percent (18 of 46) of all world championships. In the NFL during the same period, the four winningest Super Bowl teams (the Steelers, Cowboys, 49ers, and Packers) combined for 43 percent (20 of 47) of all Super Bowls. Although professional football’s system is widely considered to be more egalitarian, the concentration of top teams is not significantly different.

Summary

• Positive feedback loops reward the player for success, punish the player for failure, or both. This can cause an imbalance between players.

• Negative feedback loops reward the player for failure, punish the player for success, or both. This can be demotivating by directing players away from the behaviors that lead to the goals of the game.

• One of the issues with feedback loops is that it overweighs the importance of earlier successes or failures, thereby making later actions in the game less meaningful with regard to the goal of the game.

• Positive and negative feedback loops can be paired to mitigate some feedback loop problems.

• Additionally, a key way to avoid destabilizing positive feedback loop problems is to decouple the reward from any effects that may influence future success. Finding a decoupled reward that the player still desires can be challenging.