CHAPTER 4
DEFINING SCENARIO OUTCOMES
All effective training begins with the end(s) in mind. As we do a traditional job analysis, we specify tasks in terms of their demonstrable outcomes or worker actions and write learning objectives to match. In scenario-based e-learning, the outcomes are often strategic and may require critical thinking skills. Defining the desired actions and decisions to reflect critical thinking skills is more challenging than defining outcomes for procedural skills. Furthermore, depending on your problem-solving domain, your outcomes often include not only an “answer” but an optimal path of accessing case data, evaluating data credibility, and/or stating hypotheses or a rationale for decisions made. One of the challenges of scenario-based e-learning is translating your learning objective into specific online interactions—behaviors such as clicking on objects, selecting multiple-choice options, or manipulating slider bars. The behaviors must reflect your intended learning outcomes, your technologies, and your learners. In this chapter I discuss in greater detail the task deliverable component of your lesson design, including:
- Defining outcomes for the different scenario-based e-learning domains
- Determining the level of complexity of your outcome deliverable(s)
- Translating learning objectives into learner actions supported by your multimedia environment
OUTCOME DELIVERABLES FOR LEARNING DOMAINS
Recall from Chapter 3 that the outcome deliverable is defined as the actions or decisions the learner must make as they investigate, resolve, and complete the scenario. You will define these outcomes by interviewing and observing experienced performers and identifying the physical and mental work flow, that is—how they resolve situations calling for judgment and decision making. For example, an automotive technician must verify the problem, select a series of tests, analyze the results, and identify the failure source and associated repair actions. A veterinarian in an emergency situation must rapidly collect and interpret data, determine treatment priorities, and monitor changes. Sometimes you will need to go beyond traditional job analysis observations and interviews in order to identify tacit knowledge and skills underlying expert performance. The process of eliciting tacit knowledge and skills is called cognitive task analysis. I’ll review some techniques you can use in Chapter 11.
After you write the learning objectives in your usual manner, for design and development purposes, you will need to translate those objectives into specific computer actions such as clicking on links or dragging and dropping objects. I’ve found that this translation can be a major hurdle for designers unfamiliar with e-learning design and development. The translation will depend on a number of factors that in turn will impact the complexity of your design. In the next section I will review some of the factors you need to consider as you convert higher-level learning objectives into specific online behaviors.
ASSESS THE COMPLEXITY OF YOUR OUTCOME RESPONSES
The learning domain, your technology, and the prior knowledge of your learners are among the major factors that will influence your translation from a task-specific learning objective to a scenario-based e-learning objective. In Figure 4.1. I summarize four of the most important factors to consider.
FIGURE 4.1. Factors Influencing the Complexity of Multimedia Design.
1. Number of Outcomes
If you have a learning objective with a single outcome verb, your lesson may focus on only one main result. For example, in a compliance scenario, the learner may simply indicate whether the action depicted does or does not violate policy or law. In more complex situations, you may have multiple objectives that require more than one outcome—not only a right or wrong answer but also an optimal process for collecting data and acting on it. For example, in the automotive troubleshooting lesson, it’s important to identify the failure, but it’s also important to do so in an efficient manner. The program tracks the learner’s path and gives feedback allowing the learner to compare his path with that of an expert, as shown in Figure 1.2. Identifying the failure and using an efficient testing process to diagnose the failure represent two different outcomes. In the wedding counselor course Bridezilla, the learner selects from three multiple-choice response options during the interview, selects pull-down options to fill in worksheets, and can type in notes. In Figure 4.2 you can see the multiple-choice interview options in the left panel and the worksheet with pull-down menus and note-taking window in the right.
FIGURE 4.2. Multiple-Choice Response Options on Left Panel and Pull-Down Type in Response Fields on the Right Worksheet.
In another example, shown in Figure 4.3 from Bioworld, a medical diagnostic lesson, the learner uses a drag-and-drop interaction to prioritize data supporting the diagnosis.
FIGURE 4.3. In Bioworld the Learner Prioritizes Data Using Drag-and-Drop Interactions.
With permission from Susanne LaJoie, McGill University.
As you consider the variety and level of complexity in response options, keep in mind your target audience, your technological constraints, such as authoring resources, and your project resources. For more novice learners and limited resources, opt for simpler solutions and a few consistent response formats. Set up interfaces that offer a limited set of options of which one is clearly correct. Even if you need to artificially constrain the real-world response options, requiring too much of novices will overload their memory capacity and depress motivation. In contrast, for more experienced learners, you can expand the learning environment to a more varied and complex set of outcome deliverables that include multiple on-screen actions or open responses.
2. Outcome Precision
By outcome precision I mean the degree of ambiguity in potential responses. High precision outcomes imply response options that are correct or incorrect. The scenario has a fairly clear-cut right or wrong answer. For example, in a compliance course, the learner must decide whether a client offer of two $300 tickets to the local basketball game does or does not violate policy. A yes or no option that reflects a clear-cut correct or incorrect answer makes for a relatively simple interface design.
In many critical thinking domains black and white answers are scarce—the options are grayer or open-ended. For example, in an interpersonal branched scenario course, the learner may face four possible responses to an angry customer, of which three are both plausible and somewhat effective. The authoring system can give scoring weights to each learner response and then at each decision point or after several decision points lead to a different outcome. Alternatively, in a scenario requiring engineers to design a well-fracturing fluid system, there may be multiple solutions, several of which may be acceptable. Scenarios with multiple acceptable outcomes will often require human mediation for evaluation and feedback.
3. Interface Response Options
Interface response options are the different methods learners use to engage with the interface, such as clicking links or dragging on-screen objects. You will need to consider the affordances of your technology—in terms of authoring software, programming expertise, and delivery hardware. For a more constrained environment, you might limit response options to a single choice from two or more options, as in a branched scenario. Alternatively, you might incorporate a slider bar to indicate a response that falls along a continuum. For example in Figure 4.4 I show a screen from Bioworld—a scenario-based e-learning program focusing on medical problem solving. On this screen the learner has selected an initial hypothesis of Salmonella from a pull-down menu and now slides the “Belief Meter” to indicate his or her level of confidence in the hypothesis.
FIGURE 4.4. Learners Indicate Confidence in Their Hypotheses with the Belief Meter Slider Bar (Upper Left Corner).
With permission from Susanne Lajoie, McGill University.
Interface response options, including checklists, pull-down menus, links, slider bars, or selection of on-screen objects, allow the learner to respond in a variety of ways to the scenario. In general, a simpler program will use only one or two response formats such as multiple choice and clicking on links. A more complex program may require multiple forms of input, including clicking on links, dragging objects, and typing in responses.
4. Social Presence
Scenario-based e-learning can be designed to be used in an instructor-led environment—for either traditional in-person or virtual online classes. At the other extreme, it can be designed as a stand-alone self-study module. Intermediate solutions involve a stand-alone package supplemented by social media—resources such as wikis or shared media pages where learners can collaborate, exchange outcomes, or obtain expert advice.
In a high social-presence environment such as a physical classroom, it is generally easier to accommodate outcomes that are more open-ended, more nuanced, and that involve multiple deliverables. For example, role-play exercises that always have multiple correct responses can be facilitated and critiqued by classmates and instructors. Likewise, design domains that require development of an original product can be more readily produced and critiqued in an environment with higher social presence. Because of their higher level of social presence, I classify synchronous facilitated environments at the lower end of multimedia design complexity. While the learner response may be more complex for a role play compared to selecting predefined on-screen options, access to peers or an instructor takes some of the burden off your multimedia design. In contrast, in environments with lower social presence, the scenario-based e-learning design may be more challenging, especially for open-ended outcomes.
WHAT DO YOU THINK?
Here is an opportunity for you to think about the complexity involved in several examples. Don’t want to do it now? Just move on to the next topic in this chapter.
For each learning objective below, select which outcome design options could be indicated. Naturally, your answers will depend on your assumptions about the scenario. When finished, you can compare your assumptions and responses to mine at the end of the chapter.
one or multiple high or low limited or multiple high medium low one or multiple high or low limited or multiple high medium low one or multiple high or low limited or multiple high medium lowTRANSLATE YOUR LEARNING OBJECTIVES
As you plan the design of your scenario-based e-learning lessons, you will usually need to reframe your learning objective into specifications of online learner actions that your environment can support. On the simpler end the response may be a yes-or-no box to click or a selection among three or four options. On the more complex end, it may involve clicking on any of a number of on-screen objects, selecting options from a pull-down menu, or selecting one answer from a lengthy list. In a virtual world or instructor-led environment, a design task or a role play may offer almost unlimited response options. Keep in mind, however, that the more complex outcomes will also require more complex evaluation criteria and feedback. A design task of any complexity warrants a debriefing or evaluation in an environment of high social presence, such as from subject-matter experts, instructors, or peers. Review and feedback of open-ended outcomes will require a rubric (checklist) to guide evaluation. You can read more about scenario evaluation in Chapter 9.
Table 4.1 lists some examples of learning objectives accompanied by higher and lower complexity translations for each of the major learning domains. For example, under the interpersonal domain, a less complex response format uses a branched scenario in which the learner makes one choice from three or four options on each screen. In contrast, a more complex response requires a role play. In a blended solution, you may decide to use the multimedia scenarios as prework for a face-to-face session. Recent research has shown that blended solutions that leverage the strengths of synchronous and asynchronous learning environments support optimal learning (U.S. Department of Education, 2012).
TABLE 4.1. Some Typical Objectives and Associated Response Options for Learning Domains
Multimedia Response Options
If you are familiar with e-learning, either as a designer, developer, or student, you are likely aware of the various options commonly available in a multimedia interface, summarized in Table 4.2. As you translate your learning objective into specific online behaviors, review this list—keeping in mind the options supported by your authoring environment, your desired learning outcomes, and learner prior knowledge. For example, in the Bridezilla lesson (see Figure 4.2) we used multiple-choice questions, links, pull-down lists and input fields. In Bioworld (see Figures 4.3 and 4.4), case research and solution activities use drag and drop, links, pull-down menus, and slider bars.
TABLE 4.2. Common Multimedia Response Options
| Option | Description | Use for |
| Yes-No | Learner makes one choice from two options | Selecting decisions or actions |
| Multiple-Choice | Learner makes one choice from several options | Selecting decisions or actions |
| Checklist | Learner makes several choices from a list of options | Selecting multiple reasons for a decision Selecting multiple interpretations of data |
| Links | Learner selects from several text links | Accessing scenario case data and resources Making scenario decisions |
| Pull-Down Menu | Learner selects one choice from several options | Selecting hypotheses, decisions, actions, or rationale |
| Drag and Drop | Learner moves content from one screen area to another | Classifying objects or data |
| Click on Object | Learner selects an on-screen object | Identifying object as source of problem Selecting data to be saved for review |
| Slider Bar | Learner slides bar to indicate a degree | Indicating confidence in a decision or hypothesis Indicating likely validity of a data source |
| Type in | Learner types in short text entry | Summarizing rationale |
| Virtual World | Learner moves and takes action in 3D world | Role playing responses Taking actions on equipment Designing product |
| Social Media | Learner can search or contribute to social media available in your organization | Building a shared media page such as Facebook Soliciting feedback from an online expert or team |
| Search Engines | Learner can use organizational or external search engines to identify facts or procedures relevant to scenario | Searching online technician manual to identify correct schematic Searching database to find background information relevant to scenario product |
One of the pitfalls of scenario-based learning is open ended environments that fail to provide the structure and guidance your learners need. One feature of your design that influences structure and guidance is your interface, including the type and number of response options allowed in the interface. I will return to this issue in greater detail in Chapter 6 on guidance in scenario-based e-learning.
SCENARIO OUTCOMES AND MULTIMEDIA INTERFACES
In Chapter 2 I introduced four navigational interfaces: branched scenario, menu, open object, and virtual world. As you define and translate outcomes and their associated response options, you may need to revisit your initial decisions. For example, a learning objective that focuses on the best work flow sequence to accomplish the goal must offer an interface and be supported by a program that allows multiple solution paths that can be tracked. An open design with on-screen objects to select is a common interface for that kind of task deliverable. In contrast, as I mentioned previously, an open-ended task deliverable such as a product design may require an instructor-facilitated environment. If you are getting started in scenario-based e-learning, I recommend starting with smaller and less complex lessons, learning from your experience, and gradually evolving to more complex levels as needed.
WHAT DO YOU THINK? REVISITED
Here are my answers. If you have different choices, don’t worry—there are multiple interpretations of each objective which could lead to different answers.
For each learning objective below, select which task deliverable design options could be indicated.
one or multiple high or low limited or multiple high medium lowIn this scenario, the main focus is on the final product, a news article with a single open-ended outcome with multiple solutions. However, the focus may be not only on the product itself but also on the search and decision process involved in identifying and verifying source content. That’s why I selected both one and multiple for number of outcomes. The outcome report will have low precision, that is, no single right or wrong answer. In addition, the learner will need a number of response format alternatives to select and search different databases. Because the news article is a very open product, this objective would likely benefit from high social presence in a virtual or physical setting.
As prework for an instructor-led session, the scenario-based e-learning lesson could focus on a subset of the final skills, such as identifying and searching relevant databases, assessing the credibility of the data, and assembling pre-constructed report “pieces” to build an online article. This type of lesson could support the basic knowledge and skills prior to instructor-led sessions that emphasize open-ended tasks, such as writing the article.
one or multiple high or low limited or multiple high medium lowFor this learning objective, I assume that learners will make multiple responses as they select tests, select interpretations, and ultimately select a treatment. The complexity of the scenario could vary, but for a straightforward case, I would incorporate treatment choices that are right or wrong and an outcome deliverable that would include not only the final selected treatment but the tests ordered and perhaps some indication of rationale. Depending on the complexity of the case, discussions among learners would promote learning. At the same time, for relatively straightforward situations, a lower level of social presence might suffice. I took a middle ground and selected medium.
one or multiple high or low limited or multiple high medium lowObjective 3 is the least complex of the three scenarios. I assume a fairly straightforward scenario requiring a single decision that is correct or incorrect and can be indicated with a yes or no response. This type of scenario-based e-learning could easily be implemented in a self-study multimedia learning environment. High social presence is not needed, although it could add value if learners wanted to discuss “What if’s?” or if scenarios were of sufficient complexity.
COMING NEXT
Now that you have specified your outcome deliverables and associated responses, you are ready to plan the trigger event as well as specify what scenario case data will be needed by the learner and how best to represent it in your multimedia interface.
ADDITIONAL RESOURCE
Horton, W. (2006). e-Learning by design. San Francisco: Pfeiffer.
In this comprehensive guide to design of all types of e-learning, you can review a wide variety of activities, some of which could be used in your scenario-based e-learning design.
Interpersonal skills Leadership and compliance Diagnosis and repair Research, analysis, and rationale Tradeoffs Operations Design Team coordination Other
one or multiple high or low limited or multiple high medium low
Yes or no Multiple choice Checklist Links Pull-down menu Drag and drop Object select Slider bar Type in Virtual world Classroom Virtual classroom Social media: __________________________________________ Search engines: ________________________________________ Other: _______________________________________________