One of the benefits of asynchronous e-learning is the potential to tailor training to just what the individual learner needs. If someone has already worked with a spreadsheet and knows the basics of cells and formulas, why should she spend time going through all of the introductory lessons on cells and formulas? Unlike the classroom, where learning mostly proceeds in a lock-step group pace and sequence, individualized instruction should in theory adjust instruction to each learner.

As we mentioned in Chapter 1, in the United States we invest over $50 billion a year in organizational training. But that figure does not incorporate the most expensive element of any training program—the time of the learners. If we can save learning time by tailoring instruction to each person's needs, there is a great potential for economic payoff in efficient instruction.

The ability to individualize instruction is called adaptive learning. You are probably most familiar with adaptive learning in the form of pretests that assign learners to the courses or lessons that match their skills. Depending on the size and scope of the course, the learner will typically complete a pretest that includes fifteen to twenty test questions followed by an assignment to specific lessons. However, in most cases, once they start, all learners work through the same lesson, regardless of their rate of learning. Everyone sees the same sequence of examples and works the same number of practice exercises. We know that students learn at different rates and, if we could track learning dynamically in the lesson, we could adjust the amount of instruction to learner needs. However, taking frequent tests throughout a course or lesson is too time-consuming. Therefore, dynamic adaptive instruction has generally not been implemented.

In Chapters 8, 9, and 10 we recommended that you use a series of gradually faded worked examples so that, as learners progress through a lesson, they receive completion examples that require them to fill in more and more steps. However, an implicit assumption behind this method is that all learners develop expertise at a linear rate correspondent with the progressively faded worked examples. A more accurate approach would adjust the sequence from worked examples through faded worked examples to full problems based on actual measures of learning. Some learners may need many fully worked examples to build a schema. Others will be ready for full practice exercises after only one or two worked examples. e-Learning offers a mechanism for tailoring instruction to each learner. However, for ongoing diagnosis of learning, we need a testing method that can quickly and accurately assess expertise as learners progress. Kalyuga and Sweller (2004, 2005) have recently reported a new rapid testing method that can serve this purpose.

What if, instead of requiring learners to provide complete problem solutions, as in traditional tests, the test only requires them to write down the first step they would take to solve a problem? Because experts have acquired and automated many schemas for solving problems in their domain, they can often jump from a problem statement directly to a solution with few or even no intermediate steps. In contrast, someone with a lower level of expertise might know the general approach to take but will need to invest considerable effort to write out several intermediate steps to solve the same problem. Of course, someone who is a novice may not even attempt the problem. Or he would write out an incorrect step. We illustrate this idea in Figure 11.1 with one algebra question from a rapid diagnostic test and several possible responses. Test takers are instructed to quickly respond by writing down the first step they would normally take to solve the problem. Experts who are very familiar with these types of equations are likely to draw on their existing schemas and immediately jump to the solution or final stages of solution. Novices are more likely to acknowledge they don't know or write out something incorrect. Learners with intermediate expertise will respond with a correct, intermediate step leading to the solution.

Figure 11.1. Alternative First Steps to Solve an Algebra Problem Among Learners of Diverse Experience.

Since learners are only writing out one step to a problem solution, to obtain a comprehensive picture of their range of skills, you would need to include problems at varying levels of complexity. For example, in Figure 11.2 we show four levels of algebraic problems along with their associated solution rules. If you work out the sample we label "advanced," you will see that its solution incorporates all of the rules of the problems that precede it. If you were to construct a fast test to monitor which rules your learners had acquired, you could develop a four-question test asking the learners to write out the first step they would take to solve each of these problems. Learners who miss the basic problem will most likely not be able to solve any of the more advanced problems, since they all incorporate the rule in Problem 1. Likewise, learners who solve Problem 1 but miss Problem 2 probably know Rule 1 but not Rule 2. By asking learners to write out the first step only, you can save a lot of testing time. By including several problems that incorporate the range of skills involved, you can infer your learners' skill levels.

Figure 11.2. Levels of Algebraic Equation Problems That Incorporate Increasing Numbers of Skills.

For example in Figures 11.3 and 11.4 you can see Questions 1 and 3 from our Excel demonstration e-lesson pretest on the CD. The first question involves a simple addition operation, whereas the third question requires constructing a formula that involves four different variables. Depending on which questions were answered accurately, the learner is branched to the appropriate topic in the lesson. You can try this out on the load managed web-based lesson on our CD.

Figure 11.3. The First of a Three-Item Pretest in an Excel Lesson on the CD.

Figure 11.4. The Third of a Three-Item Pretest in an Excel Lesson on the CD.

In order for a rapid testing method like this one to be practical, we need to know that it does give us an accurate picture of the learner's skills. One way to verify the value of such a rapid test is to compare it with test scores from a traditional test measuring the same skills.

Adaptive online training can be used to tailor instructional content and methods to each learner's level of expertise as they start a course and as they progress through the lessons. Such a program would use results from a rapid pretest to assign learners to the appropriate lesson or topic within a lesson. As the learner progresses through the training in a lesson or lesson topic, the instructional materials are tailored based on her learning.

For example, suppose after taking a lesson pretest, Marcy is assigned to Topic B since she demonstrated an understanding of the prerequisite skill in Topic A. Marcy receives some instruction on Topic B followed by a rapid diagnostic test. If she passes the diagnostic test, she moves on to Topic C. Otherwise, she receives additional training on Topic B and repeats the testing-training cycle until she demonstrates competency in Topic B. In this way, each learner moves through the lessons at his or her own pace and progresses only after demonstrating competency in each skill.

To consider a more specific adaptive scenario, our demonstration asynchronous Excel lesson on formulas included three topics on how to construct formulas for addition and subtraction, multiplication and division, and a combination of both. Three rapid pretest items like the ones shown in Figures 11.3 and 11.4 are used to assign the learner to the appropriate topic in the lesson. In that topic the learner views a worked example and one or two completion examples, ending with a full problem assignment. At that point the learners are tested again using an item like the one shown in Figure 11.5. If they respond correctly, they move to the next topic. Otherwise, they are assigned additional worked examples. You can try out our adaptive instruction when you view the asynchronous Excel lesson example on the CD.

Dynamic rapid testing sounds good, but we need evidence that it works as intended. In the next section, we summarize very recent research reported by Kalyuga and Sweller (2005) that compared the learning from this type of adaptive training to learning of a comparison group that did not receive adaptive training.

Figure 11.5. A Diagnostic Test Given After Completion of Topic 1.

WHAT THE RESEARCH SAYS ABOUT ADAPTIVE E-LEARNING BASED ON RAPID ASSESSMENT

Figure 11.6 illustrates a high-level overview of the adaptive training plan tested by Kalyuga and Sweller (2005). The lessons were designed to teach the algebraic skills summarized in Figure 11.2. As you can see in Figure 11.6, the learners assigned to the adaptive training group start with a rapid pretest. Based on their pretest scores, they are assigned to lesson topics Level 1 through Level 4. Figure 11.7 summarizes the instructional methods used at each level. The basic instruction at Level 1 included two worked examples. However, in the basic instruction at Levels 2 and 3, the worked examples were faded and presented as completion examples.

For example, if you started at Level 2 you would receive a faded worked example in which the first step, which reflects the new skill in Level 2, is already worked and you would complete the second step based on your prerequisite knowledge. Level 4 included full problem assignments. A novice who started at Level 1 would begin with fully worked examples and move through progressively faded completion examples, finishing at Level 4 with full problem assignments.

Figure 11.6. An Overview of an Adaptive Testing Learning Sequence.

Figure 11.7. A Summary of the Instruction Methods at Each Level of an Adaptive Training Plan.

A learner only progresses to the next level after demonstrating mastery of her current stage on a rapid diagnostic test. If the learner fails to demonstrate learning on the rapid diagnostic test, she is branched to one of two levels of additional training that includes more worked examples. After completing the additional training, she retakes the rapid diagnostic test. When her solutions on the rapid test showed she had acquired the relevant rule, she was branched to the next level. Once participants completed Level 4, they took a rapid post-test with different items but a similar structure to the pretest.

Kalyuga and Sweller (2005) compared learning from the adaptive training with learning from a comparison group. To ensure that learners in the comparison group received the same amount of instruction as individuals in the adaptive group, learners in the comparison group were randomly paired with an individual in the adapted group and received the same instruction as their partners. In that way, both groups of learners were exposed to the same amount of instruction. However, individuals in the comparison group received their instructional assignments independently of their actual learning. These individuals did not take any diagnostic tests and completed the pretest and post-test only.

Kalyuga and Sweller (2005) found that the average pretest/post-test gain was 2.27 for the adaptive group and .67 for the comparison group. This difference was both statistically and practically significant, with an effect size of .55, which is moderate.

In this research, the highly structured domain of algebraic equations allowed the researchers to cleanly develop rapid assessments and instructional treatments with faded worked examples matched to learner expertise. Our Excel formula lesson likewise involved sufficient structure in topics to permit us to easily apply the adaptive plan. We will need further research to determine how to best adapt this methodology to less structured learning goals. Since the rapid testing method is very new, we will need more research, including experiments and field trials, before recommending it for all types of instructional goals. In the next paragraphs, we discuss the benefits and drawbacks of adaptive learning.

From a practical perspective, adaptive testing always requires a greater investment of resources than alternative designs that incorporate the same learning methods for everyone. First, the diagnostic tests must be built. Second, the additional instruction must be built for those learners who do not reach competency after completing the basic instruction. The potential benefits of pretests are efficiency gains for trainee populations with diverse prior knowledge. The pretest will allow learners to bypass training topics that they already know. If you have a sufficiently large population of learners with diverse skill sets, you can likely make a strong economic case for diagnostic pretesting.

The potential benefits of dynamic in-lesson assessment are efficiency in learning for faster learners as well as confidence in equivalent skill achievement among learners requiring different levels of instructional support. The dynamic in-lesson test will ensure that learners do not progress until they have mastered prerequisite skills. If you have a trainee population with sufficiently diverse instructional support requirements and it's critical that all learners reach a similar performance level, you can likely make an economic case for dynamic assessment at the topic or lesson level. Naturally, the more fine-grained your adaptive learning, the more development work is involved. If you work at the lesson level, your diagnostic test items will be fewer than if you work at the topic level. Since e-learning lessons should be relatively short, you might gain most cost benefit at the lesson level.

The research is quite recent on rapid testing. However, based on results to date we conclude that:

Depending on your situation, you may want to use the rapid pretest only, the rapid diagnostic test only, or a combination of both. As you can see, the development of the test items and the extra instructional support will add considerably to the cost of the courseware and should be undertaken only when you anticipate sufficient return on investment. Whether using diagnostic assessment or not, you should consider the expertise reversal effect when designing more advanced lessons.

In Parts II and III, we summarized a number of proven techniques for reducing extraneous cognitive load and increasing germane load for novice and for experienced learners. In Part IV we have seen that, when it comes to experienced learners, many of the techniques that improve learning and learning efficiency in novice learners either have no effect or, in some cases, actually have a negative effect.

In the last part of the book we include two chapters that summarize and integrate all of the guidelines we have generated throughout the book. Chapter 12 organizes cognitive load management guidelines into an instructional application context. Chapter 12 won't present any new research or guidelines. It will integrate the guidelines we've discussed throughout the book in a context appropriate for individuals responsible for design, development, evaluation, and delivery of training products.