Every UX designer must also be, to some extent, a good psychologist. Understanding how people perceive and think about the world allows us to design user interfaces that make intuitive sense. Such expertise is important in game design as well. Tetris is very perceptual (as you try to figure out which piece is which) and very cognitive (as you decide where the pieces should be stacked). But there’s also something else going on. Through its design, Tetris pulls you in and encourages you to keep going. Although you might have planned to play just one round when you first sat down, by the time it’s over you feel like playing another, and then you end up playing a few more times, followed immediately by a few more rounds. The act of playing the game creates new drives that you didn’t have when you first sat down to play.
In UX design, we’re accustomed to supporting the drives that people bring to the experience, not determining what those drives should be. But video game designers have access to a powerful set of behavioral tools that can create new motivations for players, opening up a new angle on design that we usually don’t have access to when working in user experience. These tools can be applied to engage people deeply in the game and involve them in constructing the experience. The principles underlying these tools were formally defined by the behaviorists—psychologists who based their study of the mind on the externally observable actions of human beings and animals.
It’s very important to remember that these tools cannot generate interest in an uninteresting game, trick people into doing something they don’t want to do, or hold people captive to an experience they don’t enjoy. That’s not the point. Instead, they create advantages for design by increasing the likelihood that players will adopt particular modes of play. Applied well, these tools can give shape to the gameplay and introduce more meaningful choices to the experience.
Because games give us access to this mode of design that’s so far outside of the way we normally work, it’s worth taking a moment to review the basics of behaviorism. After a primer, I’ll present a pair of case studies examining how these elements of psychology underlie the success of two popular video games.
A rat is placed inside a small box in a laboratory. On one side of the box is a lever and a chute (Figure 9-1). When the rat accidentally brushes against the lever, a food pellet comes flying down the chute. The rat gobbles it up, finding it very tasty. Then, while trying to find a way out through the top of the box, it props a front paw against the lever and another delicious pellet appears. Ooh, pellet! A few more such happy accidents, and the rat starts to catch on. Before long, it’s pressing the lever repeatedly every time it feels hungry.
Figure 9-1. Experiments using animals in Skinner boxes revealed the effects that environmental factors can have on behavior.
Variations on this experiment, devised by behaviorist B. F. Skinner, demonstrated the basic principles of a form of learning called operant conditioning. In this type of learning, an animal or a person acts on the outside world, and the way the world responds makes it more or less likely that the behavior will be repeated in the future. Skinner showed that by manipulating the rules that control how the box’s mechanism issues rewards and punishments, he could drive animals toward selected behaviors.
Notice that all of the characteristics of games described in Chapter 2 apply to the Skinner box as well:
An objective. The animal is always working toward obtaining the food pellet.
Environmental constraints. The animal is confined to a box and doesn’t have access to food except through the feeding mechanism.
Formal constraints. The animal must take a particular action, such as pressing a lever, to get the reward. It also must not perform other forbidden actions, such as pressing the wrong lever.
Machine-based arbitration. The box’s automated mechanism handles compliance with the rules and issues the rewards and punishments.
Skinner boxes, then, fall into the same class of devices as video games, pinball, pachinko, and slot machines. This commonality matters to us as designers because the same methods that allowed Skinner to shape the behavior of animals can also be applied to affect people’s actions in games.
When you give a reward for doing something, you make it more likely that action will be repeated. Giving a reward is called reinforcing the behavior, and there are two primary ways to do it.
When the rat gets a pellet for pressing the lever, it is receiving positive reinforcement. In other words, in response to performing a desired action, it receives something that it appreciates. People have their pellets too. One of my coworkers brings a bowl of candy with her every time she calls an especially tedious meeting. People come knowing they’ll be bored to tears, but they also know they can at least look forward to some candy (these meetings do always happen to be well attended). Another example of positive reinforcement is leaving a big tip for a waiter who provides especially good service. Because tipping is a social norm, waiters know that, in general, the amount of money they make depends on the quality of the job they do.
The other way to reinforce a behavior is to alleviate people of something they don’t like. This method is called negative reinforcement. People don’t like to spend their hard-earned money. Because we have to pay money for almost everything, anything that’s available for free or at a substantial discount is extremely attractive. Coffee shops, for example, capitalize on behaviorism by giving customers loyalty cards that award them a free cup of coffee after they buy 10. Governments encourage citizens to buy homes, save for retirement, and install solar panels by offering tax breaks. These incentives may not work for everyone, but they don’t have to. By reducing their fees, coffee shops and governments increase the probability that people will do the things they want them to do.
Games are very good at offering players many different types of rewards, from points to unlockables to achievements. Chapter 10 examines these reward systems in depth.
Sometimes the Skinner box is wired to dole out electric shocks when the poor rat does something the experimenter doesn’t want it to do, such as pressing the lever when a red light is on. Punishment is reinforcement’s sinister twin; it makes it less likely that an action will be repeated.
As with reinforcement, punishment can be applied in either of two ways. Getting zapped by a Skinner box is a form of positive punishment—having something negative inflicted as a disincentive to further action. Human beings tend to develop plenty of ways to punish, from small to big. Some parents spank their children; schools issue detentions, suspensions, and expulsions; police give out tickets for speeding and reckless driving; nations levy sanctions and impose trade tariffs on one another. Negative punishment, by contrast, takes away something valued or enjoyed, such as when parents ground their children or revoke driving privileges for acting irresponsibly.
Most UX design books don’t spend much time discussing the merits of punishment as a design choice. But in game experiences, punishment is a tool at your disposal like any other, with appropriate and inappropriate uses. If a virtual-pet game is intended to teach players how to be responsible for the care of another living thing, then it’s appropriate to set them back when they fail to live up to their responsibilities. Their pet may lose health points and then require more care from the player to bring it back to its previous level of health. Selective punishments can make a game experience more enjoyable because consequences are an inherent part of gameplay.
There are some important nuances in the ways that rewards and punishments work. Understanding these concepts is important, in part because it allows you to design more successful systems of consequences, and in part because it can shine a light on an opportunity to achieve something that you might have otherwise missed.
Reinforcements and punishments are more effective when they’re given in a timely fashion so that there’s a clear relationship between the action and its consequence. If a rat gets its pellet a full minute after pressing the lever, it’s more likely to associate the reward with whatever it has moved on to doing at that point—taking a drink of water or cleaning its tail, for example. This type of confusion sometimes shows up in usability testing of conventional interfaces, as when a user is working with an unresponsive system and attributes changes on-screen to the wrong actions. Punishments that are administered too late will seem random, cruel, and beyond one’s control. This is why, as discussed in Chapter 5, maintaining a clear relationship between actions and consequences is an important principle in game design as well.
If a learned action suddenly stops producing the expected reward, you’ll eventually give up. This response is called extinction. Very young children learn to cry more often when crying consistently gets them attention from grown-ups. Parents can break this behavior by withholding attention when it’s clear that their child is just being a little dramatic. Role-playing games encourage players to continually invest in new weapons and magic spells by presenting them with new enemies that are less affected by the trusty old standbys. People stop using a website’s search engine when it starts bringing back lousy results.
One of the problems with punishment is that it discourages bad behavior only as long as the authority giving out the punishment is watching. In real life, people sometimes delight in being defiant when they feel they won’t get caught. Fourth-graders wait until the teacher’s back is turned to start passing notes and pulling hair. A teacher who leaves the classroom unsupervised altogether for 10 minutes can expect to return to mayhem. In the United States, Amazon.com doesn’t collect the state sales taxes owed on purchases, and people generally neglect to pay them on their own, knowing that there is no one to enforce the rules.[35]
Some games actually turn defiance into an element of the gameplay. In Grand Theft Auto, a crime is punishable only if a police officer is nearby to observe it (Figure 9-2). If the crime goes unseen, the player can get away with it. Creative UX designers might use a similar strategy to explore questions of morality and ethics, challenging people to consider what they would be willing to do in situations where they believe no one would be willing to stop them, and then exposing them to the consequences of those actions.
Punishments can go too far. A single, especially sharp physical or psychological assault can be enough to discourage a person from ever trying the associated action again, and it could even result in a deep-seated fear of the person or thing that inflicted the trauma. One of the advantages of video games is that the player is always fundamentally safe. Although games can induce real fear, revulsion, or dread, there is never any true risk to the player’s health or life. This shield from real consequences can create opportunities for people to explore dangerous or stressful situations in calm and rational ways through games. UX designers can make use of game design, for example, to help prepare emergency workers to handle disastrous events.
Behaviorists found that changing the frequency and timing of rewards and punishments creates specific, predictable patterns of behavior. Skinner defined several such schedules of reinforcement, each with its own distinct effects and its own implications for design.
Continuous reinforcement is the simplest kind of schedule, with each desired action resulting in a reward (technically it isn’t a schedule at all, but that’s academic). New behaviors are picked up fastest with continuous reinforcement, so it’s useful early in a game or within training levels. But because behaviors learned on a continuous schedule extinguish quickly if the reward suddenly stops coming, some things need to work reliably so that people can consistently count on them.
Continuous reinforcement is extremely important in the design of conventional user interfaces; in fact, it’s normally the only way we UX designers work. If users find that menu items don’t respond in a timely fashion when they’re clicked, they’ll stop trying pretty quickly. Game design, however, allows us to make use of other types of reinforcement schedules, each of which promotes different patterns of behavior.
A fixed-ratio schedule gives out the reward only after a predetermined number of tries. For example, a Skinner box might be set up to give the rat a pellet every fifth time it presses the lever. Rats respond to this schedule by working faster—pushing the lever very quickly until the pellet pops out, stopping to take a quick breather, and then going right back to tapping away at the lever (Figure 9-3). The same pattern happens in shooter games when enemies require a specific number of hits to be killed.
On the variable-ratio schedule, the reward is given after a number of tries that changes over time (although there isn’t a huge deviation from the average). The result is the fastest rate of response, with very little pausing after the reinforcement is given (Figure 9-4). Slot machines work on variable-ratio schedules. They pay out fairly regularly, but not predictably. You know a win is coming, but you don’t know how many times you’ll need to drop in a coin before it happens. This schedule of reinforcement is the reason players get hooked.
Rats in a box on a fixed-interval schedule receive a pellet only after a fixed amount of time. No matter how many times the lever is pressed, nothing will happen until the required time has elapsed. Once rats grow accustomed to this schedule, they lay off the lever for a while after getting a reward, and then start pressing it furiously right around the time they expect another pellet to become available (Figure 9-5).
In a variable-interval schedule, a reward is issued after an amount of time that keeps changing. From the rat’s perspective, this sort of schedule is indistinguishable from randomization. This schedule produces the slowest pattern of behavior (Figure 9-6). Because the rat doesn’t know exactly when the reward will come and tapping the lever faster doesn’t help anything, it makes the most sense to check in only every now and then.
You can use the principles of behaviorism to create a better game design. They enable you to create predictable patterns of behavior and craft an experience that will be more challenging, more interesting, and more engaging. I’ll wrap up this chapter with a couple of examples that demonstrate the effects that these behavioral tools, applied carefully, have had in real-world games.
When we look closely at games that a lot of people have played compulsively, we often find that these games contain examples of behaviorism in action. Pac-Man, for example, made use of several principles first observed in Skinner boxes.
Like many other arcade games, Pac-Man’s system of positive reinforcement is based on points. The points are just used as a common currency, providing a single way to reward several different types of actions commensurate with their difficulty. The small dots scattered over the board provide only 10 points each, but there are a lot of them. So clearing the board is a worthwhile long-term goal. Eating one ghost provides 200 points, with the reward doubling for each additional ghost eaten within a short period. But trying to eat multiple ghosts is risky, because their vulnerability may run out before the player gets to them. Forcing players to make a trade-off between earning points and staying safe makes the game more interesting and exciting.
Pac-Man also provides negative reinforcement. The game is continually stressful because the player is constantly being hunted by the ghosts and running to avoid being cornered, eaten, and short-changed on a perfectly good quarter. Eating a power pellet, however, temporarily alleviates the negative situation and allows the player to enjoy complete safety on the board. So the power pellets are very attractive, but only four of them are available. Consuming all of them too quickly leaves the player defenseless while still needing to clear the board. This design presents players with another meaningful choice: enjoy safety in the short term or maximize their chances of surviving in the long term.
Finally, the bonus fruits follow a fixed-interval schedule, appearing on the board at predetermined times (Figure 9-7). Players can gain the most points in the game by eating fruit, but again this strategy involves increased risk because the fruit appears in one of the most dangerous spots on the board and may disappear before the player can get to it. As players gain experience with the game, they become able to predict when the fruit will appear. As with all fixed-interval schedules, players can spend the time when they’re not expecting the fruit engaged in other things, and then start hanging out around the lower middle of the board when they sense the fruit is about to pop up. The most experienced players are able to minimize risk by arriving at the drop point just as the fruit appears.
Valve Software’s online multiplayer game Left 4 Dead was designed expressly with principles of behaviorism in mind. It’s set in a zombie apocalypse, with some players in the role of human survivors fighting other players in the role of the infected.
Figure 9-7. As with any fixed-interval schedule, the points offered by the opportunity to gobble up a piece of fruit in Pac-Man encourage players to start hanging out around the middle of the board when they think the next piece of fruit is about to appear.
One element of the design provides a great example of positive punishment being used to improve the gameplay experience. The game’s designers wanted to create a teamwork dynamic in which players representing the humans would stick together as a group and help one another (Figure 9-8). To promote this form of play, they designed the abilities of the infected specifically so that human players who wandered away from the group would be easy pickings. The players’ movements aren’t directly restricted in any way, but players who decide to go rogue find themselves attacked and killed more often. The winning strategy comes from sticking with the group, working collaboratively, and building relationships with your team members.
Figure 9-8. The designers of Left 4 Dead wanted to create a dynamic that encouraged players to stay together.
The item drops in Left 4 Dead make use of the behavior pattern that results from a variable-interval schedule of reinforcement. Every so often, a helpful item, like a weapon or health pack, will appear at fixed locations in the game world (Figure 9-9). All experienced players know where the items will show up, but they don’t know when. As a result, they need to keep checking these spots regularly or someone else will pick them up first. So it is more likely that players will be found at certain locations on the map at any given time—knowledge that their opponents can use to their own advantage by lying in ambush nearby.
Figure 9-9. Item drops use reinforcement schedules to increase the likelihood that players will revisit specific places frequently.
Note here that the game’s designers aren’t the only ones involved in creating the game experience. The players need to behave in a certain way for the gameplay to really work. The designers manipulate the system of play, and in reacting to that system the players construct the experience. This design promotes a deep level of engagement in the game.
Behaviorism can offend our sense of human dignity because it suggests that unseen actors can make our choices for us. Indeed, Skinner championed the ideas that our actions are determined by external forces rather than our own free will, and that our sense of self-determination is an illusion. Contemporary psychology has demonstrated that in real life, decision making is much more complex. People’s internal capacity to think and reason can resist efforts to shape their behavior from the outside. But behaviorism did reveal a framework underlying the actions of most animals, which can also bear an influence on us—especially when we’re inclined to allow it to.
Unlike Skinner’s rats, video game players are not tyrannically confined to a box. They enter a game freely because they value the experience of playing. When players adopt collaborative gameplay strategies in response to the behavioral design of Left 4 Dead, it’s not because their free will has been stolen away from them. They collaborate because it’s the winning strategy that the game’s designers have selected, and the players feel invested in winning. Such investment speaks well of the game’s fundamental appeal, because people wouldn’t feel so invested if they didn’t enjoy the experience in the first place. The decision to play is an expression of our free will. When we cross into a game’s magic circle, we acquiesce to its behavioral drivers in order to enable the experience.
Importantly, designs that don’t value the player experience cannot be made enjoyable simply by applying the behavioral tools discussed in this chapter. If this were true, then creating compelling game experiences would be as easy as it is in the online parody Progress Wars (Figure 9-10). However, designers can make great use of these tools to deepen engagement in a game that is fundamentally rewarding to play.
