CHAPTER 7

Matching Scoring to Learning Goals

In This Chapter

What are the basics of scoring for learning game design?

What scoring methods can you use in your game?

How do you create a scoring system?

Three scoring case studies

Guru game play opportunity

When players sit down to a game, most instinctively know that the goal of playing is to win, whether it is accomplished by collecting the most resources, crossing the finish line first, or finding the treasure before everyone else. Game designers don’t have to tell players the goal of the game is to win. However, good game designers do explain to players how they earn the points and achievements that allow them to win. For example, players need to know the conditions for crossing the finish line or finding the treasure. Success or failure at in-game activities feeds into both winning the game and determining the score. This chapter explores how to create scoring within your game (Figure 7-1).

Figure 7-1. Learning-Game Design Process: Scoring and Rewards

Points and achievements inform players where they are in the game, how close they might be to winning, how close other players are to winning, and how they should form a strategy. But scoring provides more than a record of who wins; it also provides a guideline for performance and activities within the game. In fact, scoring in a learning game needs to correlate to how well or poorly a player is performing in the game. The scoring algorithm you create as the game designer needs to tie to both learning and player progress in the game. It also should reflect on-the-job behaviors and thought processes you want to teach and reinforce.

Scoring Basics

Scoring in a learning game should revolve around six key concepts.

Keep the Scoring Simple

You do not want to create complex scoring algorithms that require players to consult the guidebook repeatedly. Instead, the focus needs to be on learning. It can be as simple as a card game in which players “score” by collecting cards, with the winner being the person with the most cards at the end of the game (Figure 7-2).

Figure 7-2. Challenge and Answer Cards for a Learning Game

Make the Scoring Transparent

Anyone playing the game should be able to figure out how to score and what it takes to win the game. If some mystery or uncertainty surrounds how a score is obtained, the learners may get confused or frustrated because they won’t know how their actions in the game are reflected in the score. Make it clear in the opening tutorial, rulebook, or explanation screen how scoring is achieved.

Tie the Scoring Directly to Learning Outcomes

If players are learning the content in the game, they should be doing well in terms of score; if they are not learning, their score should reflect this. You can’t let chance dictate winning in a learning game. The goal is mastery or reinforcement of the content through game play, which can be difficult elements to balance. However, you must ensure that learning is the ultimate outcome, and nothing drives a player’s actions more in a game than scoring.

De-Emphasize Winning in Learning Environments

If you are using competition as a game element, make it clear that winning or losing is low in importance compared with learning (Cantador and Conde 2010). You don’t want to make the losing experience so terrible that it turns off the loser, and you don’t want the game so focused on winning that everything else is forgotten. Learning games need to be about learning—not winning. So be careful when crafting the winning state of an instructional game so that even if players or their teams do not win, they still learn.

Add Variability to the Scoring

This usually means that scoring cannot be one-dimensional. For example, if you create a question-and-answer game and players get the same points for answering questions correctly, then they all could have identical scores at the end if no one misses a question. This scoring algorithm would make it difficult to create a leaderboard or competition because everyone can achieve the same score. So a good game designer might add another variable, such as time or number of tries required: The score could be a combination of how many questions a player answered correctly on the first try and how long it took to answer all the questions. This combination allows two players to learn the content but not necessarily obtain the same score.

Reinforce On-the-Job Realities Through the Scoring

Actions and activities required for an effective performance should be echoed or mimicked within the scoring. You do not want the scoring of the game to work against what the player is attempting to learn. For example, if the player’s job requires accuracy, the game should require accuracy to win. If the player’s job does not include time pressures, making time part of the scoring algorithm is not an effective game mechanic. The scoring should not hinder learning or transfer to the job. Design the scoring mechanism of the game to reward the behaviors and activities required for effective on-the-job performance.

Methods of Scoring

Scoring is about more than points; it provides feedback on how well a person is doing in the game. In a learning game, scoring should help players gauge their progress toward mastery of the content. As such, achievements and rewards can come in several different forms.

Earning Points

Points are feedback related to the level of effort, timeliness, correctness, or accuracy of responses to a question, scenario, or other instructional event. They are effective for providing a means of measuring progress against a standard, the maximum amount of points attainable, or peers. Typically, you want a high level of points. So instead of assigning one point for a correct answer, assign 100 points. The larger the point value, the more players will feel is at stake. Getting one point for answering a question does not seem as big of a deal as getting 100 points, even though the overall results might be the same. This is also true of losing one point versus 100 points. Additionally, adding a large number of points allows you to create more variability within the scoring.

Leveling Up

In many learning games, a level is a defined phase that requires certain actions to move to the next phase. Sometimes a level equates to mastery of content in one area, sometimes it represents a player completing a certain set of tasks, and sometimes each level in a game is tied to one overarching instructional objective. Leveling up, or progressing from the simplest level to more complex ones, creates a learning environment that mirrors moving from simple knowledge to more complex knowledge.

Levels also keep the learning space manageable. Developing a learning game with one vast level containing all the content and dozens of learning objectives would be daunting for both the player and the designer. A well-designed progression of levels accomplishes three goals:

1. Levels help players progress by providing new information or insights at each level. This in turn keeps players engaged and focused on a relatively small learning objective.
2. Levels build on and reinforce skills or knowledge developed in the previous level. As players progress and the levels become more difficult, they are required to recall and use knowledge or skills learned in previous levels to advance. However, at this point, they usually have to perform the skills more quickly or under greater pressure to make the application of the skill more challenging. Toward the end, players usually must use skills learned from previous levels in unique combinations.
3. Levels serve as motivation. Progression through the levels becomes a goal players want to attain. The different levels provide small, achievable goals that encourage players to do more activities so they can get to the next level. Progression through all levels means players have achieved terminal learning objectives.

Unlocking Content

Games can reward success by giving learners access to places within the game space that they earn through productive game play. This reward mechanism taps into people’s desire to explore. Another version of this reward structure is when in-game mysteries are revealed based on players’ performance, or they earn clues to solve in-game puzzles. If learners do well, they might earn the right to play more of the game. This might mean earning an extra life, an extra chance, or even a reward like a power-up, which gives players special powers for a short time.

Earning Achievements

Whether they are badges, trophies, or other visible signs of accomplishment, achievements in learning games can encourage players to perform a specific behavior or task, or to progress through different levels. Achievements that happen in a competitive game can also help keep players from feeling totally shut out if they lose. They can show players that they did indeed learn while playing the game.

Figure 7-3 is an example of an achievement case from a game. Notice there is space for numerous achievements, all of which players earn based on how well they perform within the game.

Figure 7-3. Scoring in a Game With Numerous Achievements

Learning games have two categories of achievements: measurement achievements and completion achievements. Learners receive measurement achievements for completing a task to a certain degree against other learners’ performance, their own performance, or another standard. For example, many games use a three-star rating system, which gives players one to three stars based on how well they performed on a level. One star indicates adequate performance on a level; three stars means the top level of performance.

Completion achievements do not tell players how well they have performed the task; instead, the game offers these achievements once learners complete a task. Completion achievements can be split into two subcategories: performance-contingent achievements, which require skill or knowledge to complete, and non-performance-contingent achievements, which are awarded for simply being present. The best practice from a learning-game perspective is to use measurement achievements instead of completion achievements to increase a player’s motivation to play the game. However, if you have to use completion achievements, use performance-contingent achievements.

Creating the Scoring Algorithm

Creating a scoring system is more difficult than it might appear. This is especially true if you are creating an online game. Because online games can calculate scores and keep track of variables, their scoring can get complicated quickly. To manage this complexity, use a spreadsheet when developing the scoring process. With a spreadsheet, you can see how manipulating points for one item affects an overall ideal score. Or you can see if a player is even capable of achieving enough points to level up or earn a critical badge.

Table 7-1 is an example spreadsheet showing scoring for different types of questions within an adventure game, with points awarded based on the degree of difficulty. These three questions represent the challenge in level 1. If players answer all questions for three question types correctly, they could earn 14,500 points; they need 10,000 to advance to level 2.

Creating a chart like this can help you create a fair system for awarding points for specific activities. Often a spreadsheet is the best way to arrange this type of information because you can then change the numbers for scoring and observe the corresponding results.

Table 7-1. Scoring and Levels of Difficulty

For example, one mistake that new learning-game designers make is that they devise the in-game challenges and scoring in such a way that unless players are perfect, they are unable to move to the next level. On the other hand, game designers sometimes allow players to earn too many points in the beginning, and the game becomes too easy. Creating a proper scoring structure is important to balance an instructional game.

Also, notice in Table 7-1 that players do not lose points for incorrect answers. If you add in penalties for incorrect answers, the spreadsheet and the scoring become more difficult. You would need to determine how many points can be lost for each incorrect answer to still achieve your learning goals.

For example, if players lost 1,000 points for each incorrect hard question above, they could still make it to the next level if they answered every other intermediate and easy question correctly. In this case, that may be acceptable. However, if the hard questions were a synthesis of knowledge to be learned, it may be important from a learning perspective that the hard questions be answered correctly. In that case, losing 1,000 points for each wrong answer would not be enough. The designer might want to make the penalty higher so the player cannot advance without answering at least one or two of the hard questions correctly.

Start with a simple spreadsheet with scores for correct answers or actions and then add complexity as you build the game. Keep in mind that you want to strive for simple scoring, but you must make scoring meaningful and motivating as well. The action or knowledge that leads to scoring will be the actions the players will focus on when playing the game.

As you consider crafting your scoring algorithm for your game, here are some best practices to keep in mind:

• Reward players for boring tasks and give them feedback for interesting ones.

• Make achievements challenging for the greatest returns in player performance.

• If you choose to give rewards, give them for performance rather than completion. Giving players a badge for completing a section isn’t a good idea. It’s better to give a reward if they complete the section to a certain standard of proficiency.

• Let your reward be a form of performance feedback to the player.

• For complex tasks requiring creativity or complicated strategies, or when onboarding a new player into the game, instill a mastery orientation. In other words, have players be concerned with improving their own skills and abilities during the game experience rather than comparing their abilities with others.

• For simple or repetitive tasks, instill a performance orientation. In other words, have players compare their scores or knowledge gained with others.

• Use expected achievements—achievements players know they are going to receive if they perform in a certain manner—as a method for players to set goals. Use unexpected achievements to encourage exploration within the game environment.

• If competitive achievements are used in the game, make them available only after the players have learned how to play and are comfortable with the game and scoring function.

• Consider adding cooperative achievements to encourage players to work together.

• Align the activities and actions on the job with the actions in the game for best learning and transferability.

• Play-test your game to find out if your scoring is increasing player motivation, decreasing it, or having no influence at all.

With some tips on scoring methods and algorithms in mind, let’s examine how game designers created the scoring framework for three learning games you’re now familiar with: Zombie Sales Apocalypse, Knowledge Guru, and TE Town.

Zombie Sales Apocalypse

The Zombie Sales Apocalypse game features many scoring elements simultaneously. First, the game bases players’ overall score on the number of questions they answer correctly minus the number of questions they answer incorrectly. The questions are embedded in dialogue, and a wrong choice in the dialogue reduces the players’ score.

Next, the game scores players across five dimensions of the sales model. In this case, points are not rewarded; instead, meters are filled for each selection of dialogue that correctly fulfills the element of the meter. When a meter is filled, players have successfully completed all the dialogue related to that element of the sales model. Table 7-2 shows what it looks like in a spreadsheet, and Figure 7-4 shows what it looks like on the screen.

Table 7-2 indicates how the scoring structure was designed. First, each element of the sales model was identified and listed and then linked to specific dialogue used within the game. The player selects the correct dialogue for the character to say from a list of three possible answers. Each right or wrong answer is assigned a score. Right answers increase the score; wrong answers decrease it. The total possible points and total possible penalty points are calculated to make sure the game is balanced in scoring. (This table changed many times while trying to create the right balance for scoring.)

Table 7-2. Zombie Sales Apocalypse Scoring Table

In Figure 7-4, you can see how the scoring looks in the game. On the left side of the screen, you can see meters and the initial score. The score increases or decreases based on how the player answers a particular question. At the end of the game, the score is provided to the player.

Figure 7-4. How Scoring From a Table Translates Into a Game

Knowledge Guru

Knowledge Guru has a relatively simple scoring system, but it still requires more complexity than you might imagine. Two needs drive this complexity: to ensure good variability in player scores for the leaderboards, and to align with the learning goals of the game and reward players for performance over time. Table 7-3 shows how each question is scored in Knowledge Guru. At this point, the scoring is simple. Table 7-4 details where greater complexity emerges. To ensure scoring variability, we created power-ups and other rewards.

The Guru game platform is based on the cognitive science concept that repetition builds long-term memory. Each learning objective within the game has at least one question set, which is a series of three questions that ask the player to recall or apply the same content to correctly respond to a question. On players’ initial exposure to the question content, which is the first question in the set, the value of a correct response is 1,000 points. The penalty for making a mistake is a 250-point deduction to their score. The game requires players to retry until they correctly respond. If players correctly respond on their first retry, they earn the full value of the question.

However, by players’ third exposure to the question content, the value of the question is much higher (10,000 points), and thus a mistake results in players losing 10,000 points. A correct response on a subsequent attempt only nets players 5,000 points. Tables 7-3 and 7-4 show what the full scoring algorithm looks like, including the bonus points for each “bonus gate” game and the possible values when players earn power-ups.

Table 7-3. Scoring for the Knowledge Guru Game

World A Actions	Points Gained and Lost
Respond correctly on first attempt at question	1,000
Respond incorrectly on first attempt at question	–250
Retry responding after reviewing misstep	1,000
Miss second attempt at question	–500
Respond correctly on third attempt at question	1,000
Miss any future attempts	–500
Answer correctly on fourth attempt or more	0
World B Actions	Points Gained and Lost
Respond correctly on first attempt at question	5,000
Respond incorrectly on first attempt at question	–2,500
Retry responding after reviewing misstep	2,500
Miss second attempt at question	–5,000
Respond correctly on third attempt at question	0
Miss any future attempts	–5,000
Answer correctly on fourth attempt or more	0
World C Actions	Points Gained and Lost
Respond correctly on first attempt at question	10,000
Respond incorrectly on first attempt at question	–10,000
Retry responding after reviewing misstep	5,000
Miss second attempt at question	–20,000
Respond correctly on third attempt at question	0
Miss any future attempts	–20,000
Answer correctly on fourth attempt or more	0

Table 7-4. Power-Up and Reward Scoring for Knowledge Guru Game

TE Town Scoring

TE Town went through five iterations of the scoring. Items we started with—but had deleted by the time we got to this version—included penalties for absence from the game and a variety of bonuses players could collect based on how well they performed in various mini games. The scoring was simply getting too complex to easily communicate to players. When we play-tested, we found players were not clear on how they were earning population or treasury dollars, so we kept refining the scoring and our in-game communication until we reached a point in which the majority of players easily understood what was happening. Table 7-5 shows the final scoring.

Table 7-5. Scoring Algorithm for TE Town Mobile Game

	Starting Population	Treasury
	10	$1,000.00
Mini Game 1: Hunt for Applications
Player Action	Population Gained or Lost	Treasury Gained or Lost
Shoot a correct application.	Max of 1,000; value decreases the longer player waits to shoot the application.	N/A
Shoot an incorrect application or miss one that is correct.	Lose a life; lose three lives and game over.	N/A
Mini Game 2: Meet the Customer
Player Action	Population Gained or Lost	Treasury Gained or Lost
Bid 1: Get it right.	0	250
Bid 2: Get both right.	0	500
Bid 3: Get all right.	0	1,000
Miss any question.	0	0
Mini Game 3: Product Picker
Player Action	Impact on Item’s Sales Value	Multiplier
Right on first try.	100% of sale value.	0.25 (25 cents) per townie
Second attempt.	Lose 60% of sale value.	0.1 (10 cents) per townie
Third attempt.	Lose 80% of sale value.	0.5 (5 cents) per townie
All subsequent attempts.	Lose 96% of sale value.	0.01 (1 cent) per townie
Mini Game 4: Seal the Deal
Player Action	Population Gained or Lost	Treasury Gained or Lost
Ask a good question without asking any bad questions.	0	0.25 (25 cents) per townie
Ask a good question after asking one bad question.	0	0.05 (5 cents) per townie
Ask a good question after asking two bad questions.	0	0.01 (1 cent) per townie
Mini Game 5: Sales Scramble
Game Action	Population Gained or Lost	Treasury Gained or Lost
People moving into town when game begins.	2,000	0
After 9 seconds elapse.	1,600	0
After 18 seconds elapse.	1,200	0
After 27 seconds elapse.	800	0
After 36 seconds elapse.	400	0
After 45 seconds elapse without success.	0	0

Guru Game Play Opportunity

We invite you to return to the Game Design Guru game and play the level covering scoring and rewards. Remember, go to www.theknowledgeguru.com/ATDGameDesignGuru and play to learn!

Wrap-Up

Scoring is one of the most difficult elements to get right when creating a learning game. As a designer, you must balance the need to reward appropriate skills or behaviors while trying to provide a disincentive for incorrect responses. You need to craft the scoring system so that it’s fair to the player but also easy to understand and master. By using a spreadsheet and mapping out scores that would result in various scenarios, you can work toward a fair and balanced scoring algorithm for your learning game.

But don’t make the scoring aspect of your game more complicated than it needs to be; otherwise, it will interfere with learning. In fact, the simpler, the better. Most important, scores in learning games must be an indication of learning.