Chapter 3
Guided Cognition Effects in Learning Literature
Abstract
This chapter reports 11 experiments that were done to determine whether Guided Cognition-designed homework would facilitate learning in high school and middle school literature classes. The experiments were done in an authentic learning environment, which means that students were in their normal school environment and were not aware that their learning was being observed. Students never saw the researchers, and all materials were handled by their regular teachers. The experiments fit seamlessly into the regular curriculum and did not affect students' course grades. Typically, a student would experience Guided Cognition-designed homework only once or twice during the entire school year. Guided Cognition homework was found to improve students' performance on unexpected quizzes by about 10%, approximately one letter grade. The effects were replicated with various literature (Shakespeare's MacBeth, Conrad's The Secret Sharer, Anouilh's Becket, Hinton's The Outsiders), under controlled time conditions, with and without explicit teaching of the content, with repetition of Guided Cognition homework, for grades 7–12 (ages 12–18) and for students of average, and advanced abilities. These results demonstrate that the advantages of Guided Cognition study are not due to specific content, time-on-task, teaching effects, or novelty and are obtained across a range of ages, ability levels, and subject matter. Improved learning resulted from engaging in effective cognitive processes triggered by the cognitive events that were embedded into the homework questions and tasks.
Keywords
Authentic learning environments; Cognitive events; Cognitive processes; Effective learning; Guided Cognition; High school and middle school literature; Homework; Novelty effects; Student ability; Time-on-task
The experiments in this chapter explore the effects of Guided Cognition homework on learning literature. Literature was chosen as our first subject matter because it includes a great deal of interpretive thinking, as well as specific factual material.
Experiments 1 and 2: Can Guided Cognition-Designed Homework Facilitate Learning for Average- and Advanced-Ability Literature Students?
Experiment 1 (with average-ability students) and Experiment 2 (with advanced-ability students) were designed to determine whether learning literature can be facilitated by enriching homework with cognitive events commonly found in the classroom environment.
Method
Participants
Three classes of average-ability, 12th-grade students participated in Experiment 1, and two classes of advanced-ability (advanced placement), 12th-grade students participated in Experiment 2. The same teacher taught all students the same content on the same days. (Characteristics of participating schools and student populations are listed in Appendix I for all experiments.)
Materials
The subject matter under study was Shakespeare's Macbeth, Act III and Act IV. For five content areas of Act III, five Traditional (T) homework questions were prepared for all students. These Traditional homework questions were syntactically straightforward verbal questions about events and about the thoughts and actions of characters at specific points in the play. Students were free to answer these questions in any way they wished. For five content areas of Act IV, Traditional (T) homework questions were prepared for half
of the students, and corresponding Guided Cognition (GC) homework questions were prepared for the other half of the students.
The Guided Cognition homework questions addressed the same content as the Traditional homework questions, but each Guided Cognition question was constructed in such a way that the student would engage in one of the five content-focused cognitive events. These events include: relate to prior experience, in which the student answers the question within the context of previous knowledge; visualize and illustrate, in which the student creates a simple line drawing to help answer the question; consider divergent answers, in which the student answers the question from more than one point of view; brainstorm and evaluate, in which the student answers the question with a list of ideas and then rank orders them in terms of their importance or in terms of their positive or negative qualities; and role play, in which the student answers the question as if he or she is a character.
Each of these content-focused cognitive events was included once across the five Guided Cognition questions. An example of a Traditional homework question is, Why does Macbeth visit the witches? The corresponding Guided Cognition homework for the consider divergent answers cognitive event is, Give two opposite but potentially valid reasons for the visit by Macbeth to the witches. Students who received the five questions that included the content-focused cognitive events also received five process-focused cognitive events. These five process-focused cognitive events were: estimate and measure time, set goals, evaluate quality of work completed, seek validation, and evaluate quantity of work completed.
A half-hour multiple-choice and short-answer quiz was constructed to test the content knowledge addressed by the Act III and Act IV homework questions. The quiz was designed to be fairly difficult in order to avoid ceiling effects. In addition, a scaled perceptions and preferences survey was prepared that asked students to compare the Act III and Act IV homework on the following dimensions: enjoyment, learning effectiveness, interest, fun, amount of work, and amount of time. Additional scaled items prepared for students in the Guided Cognition condition asked them to rate how well each homework activity (i.e., cognitive event) helped them understand and learn the material.
Design and procedure
The experiments were exact replications across ability levels. In Experiment 1, one class was assigned to the control condition (T-T), and two classes were assigned to the experimental condition (T-GC). In Experiment 2, one class was assigned to the control condition (T-T), and one class was assigned to
the experimental condition (T-GC). Act III was taught on Monday and Tuesday, and Traditional homework was assigned to the control and experimental classes for Tuesday evening. Act IV was taught on Wednesday and Thursday, and for Thursday evening, Traditional homework was assigned to control classes (Condition T-T), and Guided Cognition homework was assigned to experimental classes (Condition T-GC).
For Act IV, students in Condition T-T answered five Traditional homework questions. Homework for the students in Condition T-GC asked students to estimate the time their homework would take and state two to four study goals. Then these students answered three homework questions that, in turn, included the consider divergent answers, visualize and illustrate, and brainstorm and evaluate content-focused cognitive events. Next, Condition T-GC students reviewed their work and evaluated quality by listing something that was done well. Then they answered two homework questions that, in turn, included the role play and relate to prior experience content-focused cognitive events. Following these questions, students evaluated the quantity of work completed, sought validation, and recorded the time spent doing the homework.
The Act IV homework was collected on Friday, there was no Macbeth-related classwork, and there was no homework assignment over the weekend. On Monday, all classes received a previously unannounced 3-day delayed quiz on Acts III and IV. On Tuesday, all classes were asked to provide opinions about the homework by answering questions on the perceptions and preferences survey.
The experiment timeline is summarized below:
| Monday | Taught Act III. |
|---|---|
| Tuesday | Taught Act III. |
| Assigned all students Traditional homework. | |
| Wednesday | Collected Act III homework. |
| Taught Act IV. | |
| Thursday | Taught Act IV. |
| Assigned either Traditional or Guided Cognition Act IV homework. | |
| Friday | Collected Act IV homework. |
| No Macbeth-related classwork. | |
| Weekend | No homework assignment. |
| Monday | Previously unannounced quiz on Acts III and IV of Macbeth. |
| Tuesday | Perceptions and preferences survey. |
Results
Performance data: Experiment 1 (average-ability students)
Quiz performance of Conditions T-T and T-GC students is shown in Figure 3.1.
As expected, there was no difference in quiz performance on content studied Tuesday evening with average scores of 50.8% (n
=
17) and 50.0% (n
=
31) for Conditions T-T and T-GC, respectively, t (46)
=
0.16, p
=
.878, ns. This result is predicted by three facts. First, Condition T-T and Condition T-GC students had very similar course grade-point averages (GPAs) halfway through the school year, 78% and 77%, respectively. Second, all students were taught the same material, Act III, by the same teacher on the same day. Third, all students received the same Traditional homework to do on Tuesday evening.
In sharp contrast, Condition T-GC students performed much better than Condition T-T students on the content studied Thursday evening with average scores of 62.7% (n
=
31) and 42.6% (n
=
17) for Conditions T-GC and T-T, respectively, t (46)
=
3.71, p
=
.001. The Guided Cognition homework students recalled 47% more (20.1 percentage points more) than the Traditional homework students.
Experiments 1 and 2 were designed such that the primary comparison is the interaction of homework type by homework day, and it is obvious that a strikingly significant effect was obtained, F (1,46)
=
14.74, p
<
.000. Well-known long-term recency effects predict that more recently studied content
will be better recalled on a quiz (e.g., Bjork & Whitten, 1974; Ebbinghaus, 1885; Whitten, 1978), so it may seem curious that the control group performed worse on Act IV than on Act III. In the sort of classroom experiments reported here, however, several counteracting factors may come into play. In the present experiments, for example, the teacher judged the Thursday homework content to be more difficult than the Tuesday homework content. This may account for the poorer performance by the control group on the Thursday content as compared with the less recent Tuesday content. Such factors are unimportant in Experiments 1 and 2, however, because they were designed to make the interaction effects of primary analytical importance.
Performance data: Experiment 2 (advanced-ability students)
This experiment provides the opportunity to determine whether the effects found with average students will be replicated by students who are already very high achievers in this subject matter. Figure 3.2 shows a clear replication of the effect, with an obvious difference being the higher level of overall performance on all material by both the control and experimental conditions, as compared to Experiment 1.
Condition T-T and Condition T-GC students had identical course grade-point averages halfway through the school year, 89% and 89% respectively. All students were taught the same material, Act III, by the same teacher on the same day, and all students received the same Traditional homework to
do on Tuesday evening. Although Condition T-T students appear to have performed somewhat better than Condition T-GC students on the content studied Tuesday evening with scores of 75.8% (Condition T-T, n
=
18) and 68.4% (Condition T-GC, n
=
26), the difference is not significant, t (42)
=
1.60, p
=
.117, ns. In contrast, performance on the content studied Thursday evening showed the striking benefit of performing Guided Cognition homework: Condition T-GC students performed much better than Condition T-T students with average scores of 86.7% and 71.4% for Conditions T-GC and T-T, respectively, t (42)
=
3.25, p
=
.002. Because the overall performance was higher, the relative improvement sounds less impressive, at 21% (15.3 percentage points more), but the strength of the effect, measured by the interaction, is at least as strong as in Experiment 1, F (1, 42)
=
15.68, p
<
.000. The generality across ability levels, as shown by the results of Experiments 1 and 2, suggests that a wide range of students can benefit from homework that includes supervised group learning cognitive events during unsupervised individual learning.
Perceptions and preferences survey data
The opinion surveys were included in our initial experiments to gain some insight into students' perceptions of Guided Cognition homework and to learn about their preferences for Traditional or Guided Cognition homework.
Comparison of Traditional and Guided Cognition homework. In this part of the perceptions and preferences survey, students were asked to respond to six statements that compared attributes of the Thursday homework with those of the Tuesday homework. The questions were of the form, “The Thursday homework was—more enjoyable, helped me learn better, was more interesting, was more fun, required more work, required more time—than the Tuesday homework.” Students selected ratings from a 7-point scale where 1
=
strongly disagree and 7
=
strongly agree. Each set of ratings was compared to the midpoint of the ratings scale. The ratings means and t-test results are shown in Table 3.1.
Because the T-T condition classes had the same style of homework on both evenings, we would expect to find their ratings to cluster around 4, indicating no perceived differences, and this was indeed the finding for five of six comparisons made by the Experiment 1 students, and for all six comparisons made by the Experiment 2 students. On the other hand, ratings from the T-GC condition classes can tell us how the students perceived the new style homework as compared to the Traditional homework, and several comparisons showed reliable differences.
Table 3.1
Compared to Traditional homework, advanced-ability students (Experiment 2) thought the Guided Cognition homework was more enjoyable and more fun, was more helpful for learning, was more interesting, and required more work and time. The ratings from the average-ability students (Experiment 1) show the same results with the exception that the Guided Cognition homework was not rated as more enjoyable or more fun than the Traditional homework.
Because Guided Cognition homework was thought to take more time, we need to control time-on-task to know whether we are observing superior learning that results from the engagement of more effective underlying cognitive processes, or whether we are seeing the less interesting effects of time-on-task. We do not think that time-on-task accounted for much of the Guided Cognition effect in Experiments 1 and 2 because students reported that most of the extra Guided Cognition homework time was spent on one task, making a diagram. The effects of time-on-task are evaluated in Experiments 3, 4, and 5 below.
Opinions about specific cognitive events. The process-focused cognitive events elicit metacognitive, planning, and evaluative cognitive processes, with little emphasis on specific to-be-learned content. In contrast, the content-focused cognitive events direct students' attention toward interpreting and learning specific content. To gain some insight for future research, we asked the students who had been in Condition T-GC to rate the extent to which each Guided Cognition homework task facilitated understanding and learning. The students were shown the 10 Guided Cognition tasks (i.e., the 10 cognitive events) with the following instructions: “For each of the homework exercises listed below, rate how well each activity helped you understand and learn the material.” Ratings were made on a 7-point scale where 7 indicated the most helpful. Each set of ratings was compared to the midpoint of the ratings scale. The ratings means and t-test results are shown in Table 3.2.
The average-ability students (Experiment 1) rated four of the five content-focused cognitive events as beneficial for learning (i.e., significantly greater than 4 on the 7-point scale). The advanced-ability students (Experiment 2) rated all five of the content-focused cognitive events as beneficial for learning. In contrast, the process-focused cognitive events were not generally perceived as helpful for learning. Average-ability students rated five of five and advanced-ability students rated four of five process-focused cognitive events as neutral (i.e., not significantly different from 4) or not beneficial (i.e., significantly less than 4). From these results, we can generalize that students, over a wide range of abilities, do not perceive much learning benefit
from the process-focused cognitive events, whereas they perceive the content-focused cognitive events to be helpful. Based on this information, we further investigated the effects of the five content-focused cognitive events in the next seven experiments.
Table 3.2
|
Process-Focused
Cognitive Events
|
Experiment 1 (Average Ability) | Experiment 2 (Advanced Ability) |
|---|---|---|
| Estimate and measure time | 3.21∗ | 2.50∗ |
| t(23) = −2.49 | t(25) = −6.16 | |
| p = .021 | p < .000 | |
| Set goals | 4.21 | 4.50 |
| t(23) = 0.62 | t(25) = 1.87 | |
| p = .540 | p = .073 | |
| Evaluate quality of work completed | 3.96 | 3.42 |
| t(23) = −0.14 | t(25) = −1.89 | |
| p = .890 | p = .070 | |
| Seek validation | 4.08 | 4.73∗ |
| t(23) = 0.24 | t(25) = 2.71 | |
| p = .811 | p = .012 | |
| Evaluate quantity of work completed | 4.50 | 4.27 |
| t(23) = 1.73 | t(25) = 0.83 | |
| p = .097 | p = .417 |
Experiments 3 and 4: Is Guided Cognition Effective When We Control Homework Study Time and When We Eliminate Teaching?
Experiment 3 (with average-ability students) and Experiment 4 (with advanced-ability students) were designed to determine whether the Guided Cognition effects of Experiments 1 and 2 were due to the cognitive events (and their underlying cognitive processes) even when time-on-task was strictly controlled. To control time, the “homework” was done in class under timed conditions. In other respects, the in-class homework retained most characteristics of unsupervised individual learning. Students were allowed to use their books and did their own work without interacting with other students or the teacher.
These experiments also were designed to eliminate the possibility that the observed Guided Cognition effects were somehow induced by the teacher or teaching, rather than by the homework study activities. For this reason, these experiments had neither teaching nor classroom discussion between the in-class, timed study activities and subsequent quizzes. To further control any possible teacher effects, students were randomly assigned to experimental and control groups within each classroom.
Method
Participants
Two classes of average-ability, 12th-grade students participated in Experiment 3, and three classes of advanced-ability (advanced placement), 12th-grade students participated in Experiment 4. All students were in classes conducted by the same teacher.
Materials
The content studied was “The Secret Sharer,” a 44-page novella by Joseph Conrad. Five Traditional homework questions were prepared for Part I of “The Secret Sharer.” These were syntactically straightforward verbal questions about events and characters. Five pairs of homework questions were
then prepared for Part II of “The Secret Sharer.” The questions in each pair addressed the same content, but one question was a Traditional question, and one was a Guided Cognition question. Each Guided Cognition study question included one of the five content-focused cognitive events used in Experiments 1 and 2: relate to prior experience, visualize and illustrate, consider divergent answers, brainstorm and evaluate, and role play. Each cognitive event was included once across the five questions.
As an example of these corresponding pairs of questions, the Guided Cognition question for the visualize and illustrate cognitive event was, Close your eyes and imagine the scene when Leggatt is escaping from the captain's ship. Draw a simple diagram that shows the positions of all the characters during this scene. Next to each character, write words that describe how he feels. The corresponding Traditional question was, How does each character feel during the part of the story when Leggatt is escaping from the captain's ship? All five pairs of Guided Cognition and Traditional homework questions are listed in Appendix II.
Question booklets were constructed with one question per page. This format allowed the teacher to pace the students' written responses by instructing students when to turn a page and begin answering the next question.
A 3-day delayed previously unannounced quiz was designed to test both parts of the story. The quiz was a standard format that had been used in these classes and consisted of 10 short-discussion questions. The first five quiz questions covered the content of the five Part I study questions, and the next five quiz questions covered the content of the five Part II study questions. To avoid ceiling effects, the quiz was designed to be fairly difficult.
A 31-day delayed previously unannounced quiz was constructed to retest the content of the five Part II study questions. This quiz also used a standard format that included short answers, sentence completions, and fill-in-the-blank responses.
Design and procedure
“The Secret Sharer” was introduced to students on Tuesday, and they began reading Part I in class. They were asked to complete the reading of Part I on Tuesday evening and were informed that on Wednesday there would be an open-book writing exercise in class to answer questions about Part I. On Wednesday, all students were given the same booklets of five Traditional questions. The booklets consisted of a cover sheet and five pages, each containing one question. The teacher explained that 7
minutes would be allocated for each question. The teacher informed the students when each 7-minute interval was completed, at which time the students turned to the
next question. During this writing exercise, students were allowed to use their textbooks but were not allowed to use other notes.
On Thursday, all students began reading Part II in class and completed the reading Thursday evening, again with the expectation of an open-book writing exercise the next day. On Friday, half of the students in each class were randomly assigned to the control group (T-T) and were given booklets of five Traditional questions. The remaining students were assigned to the experimental group (T-GC) and were given booklets of five Guided Cognition questions. As on Wednesday, the teacher paced the students' writing by allowing 7
minutes per question, and as on Wednesday, students were allowed to use their textbooks.
On Monday, all students received a 3-day delayed previously unannounced quiz on the content from the five Part I and the five Part II in-class writing exercises. Students were given 20
minutes to answer the Part I questions and 20
minutes to answer the Part II questions. To avoid any possible teaching effects, these writing exercises were collected and not discussed or reviewed. Later in the semester, all students received a 31-day delayed previously unannounced quiz on the content from the Part II in-class writing exercises. Students were given 40
minutes to complete this quiz.
The experiment timeline is summarized below:
| Tuesday | Students read Part I of “The Secret Sharer” in class. |
|---|---|
| Evening homework was to complete reading Part I. | |
| Wednesday | In-class homework to answer five Traditional questions. |
| Thursday | Students read Part II of “The Secret Sharer” in class. |
| Evening homework was to complete reading Part II. | |
| Friday | In-class homework to answer either five Traditional or five Guided Cognition questions. |
| Weekend | No homework for the weekend. |
| Monday | Previously unannounced quiz on Part I and Part II homework content. |
| 31 days later | Previously unannounced quiz on Part II homework content. |
Results
Data from students who answered each of the 10 homework questions and who were present for the 3-day delayed and the 31-day delayed quizzes were included in the analyses. For the 3-day delayed quiz, a rubric was constructed that listed the facts relevant to each question. A student's answer to
each question was compared with the rubric and graded for completeness on a 5-point scale. For the 31-day delayed quiz, each of the five sections corresponded to a study question and was graded on a 5-point scale. Thus, each cognitive event's contribution was equally weighted.
Experiment 3. In this experiment with average-ability students, time-on-task was very well controlled by the timed writing exercises. The teacher observed students during the 7-minute timed writing periods and noted any student who was finished early. In general, students used all the allotted time to write answers to questions, regardless of question format. Of 165 Part II questions answered, students completed only 13 in less than the full 7
minutes, and of those, 5 were Traditional and 8 were Guided Cognition questions.
The purpose of Part I questions on the 3-day delayed quiz was to check that the baseline performance of the control and experimental groups was well matched. Because all students had the same Part I questions, it was expected that quiz performance would be similar, and it was. The average scores for the T-T (n
=
18) and T-GC (n
=
15) students were 75.7% and 77.5%, respectively, t (31)
=
0.42, p
=
.678, ns. In addition, there was no significant difference in performance on the content of Part II. The average scores for the T-T and the T-GC students were 72.7% and 77.7%, respectively, in the expected direction, but not reliably so, t (31)
=
1.03, p
=
.310, ns. After 31
days, however, the students who had answered the Guided Cognition questions recalled 61.9%, and the students who had answered the Traditional questions recalled 43.7%, t (31)
=
3.99, p
<
.000. This 18.2 percentage-point advantage for Guided Cognition study, in practical terms, is the equivalent of scoring nearly two grades higher on the quiz.
Experiment 4. The advanced-ability students exhibited a similar pattern of performance. Time-on-task was again well controlled with only 1
T question and 1
GC question, of 305 asked, being answered in less than the allotted 7
minutes.
As expected, the control and experimental groups performed nearly identically on the 3-day delayed quiz's Part I questions. The average scores for the T-T (n
=
29) and T-GC (n
=
32) students were 84.8% and 81.6%, respectively, t (59)
=
1.62, p
=
.111, ns. In addition, as with the average-ability students of Experiment 3, there was no significant difference in performance on the content of Part II. The average scores for the T-T and the T-GC students were 90.9% and 90.8%, respectively, t (59)
=
0.05, p
=
.959, ns, and were high enough to suspect a ceiling effect. After 31
days, however, the students who had answered the Guided Cognition questions recalled
84.8%, and the students who had answered the Traditional questions recalled 79.2%, t (59)
=
2.34, p
=
.023. This 5.6 percentage-point gain is reliable and, in practical terms, could result in about one-half grade higher on the quiz.
Discussion
Experiments 3 and 4 did not show a Guided Cognition advantage on the 3-day delayed quiz but clearly demonstrated such an advantage after a 1-month delay. Students in these experiments were given instructions to prepare for in-class writing. Such preparation may have given students in Traditional and Guided Cognition conditions similar near-term recall, but in spite of this, it is clear that after a month's delay, the students who answered the Guided Cognition questions could recall more of the studied content than the students who answered the Traditional questions. In other words, Guided Cognition study produced a more gradual forgetting function compared to Traditional study. From these results, we conclude that Guided Cognition questions help students create more stable or more accessible representations of content than Traditional questions. These results cannot be attributed to differential in-class homework study time because time to answer the questions in class was controlled. Likewise, these results cannot be attributed to teacher or teaching effects because the content under study was not taught in class but was assigned as reading in preparation for the in-class homework writing exercises.
Experiment 5: Can Learning Be Predicted by Time Spent on Either Traditional or Guided Cognition Homework?
In Experiments 3 and 4, time to perform the unsupervised tasks was controlled. In Experiment 5, we returned to a more natural homework environment that allowed students to determine the amount of time used to answer questions. Our goal was to learn more about the relation of the amount of time spent on homework and later performance on unexpected quizzes.
Method
Participants
Three classes of advanced-ability 12th-grade students participated. All students were taught the same content on the same days by the same teacher.
Materials
The content studied was Becket, a play by Jean Anouilh. Five pairs of homework questions were prepared for the contents of Acts 1 and 2. The questions in each pair addressed the same content, with one question being a Traditional question and the other being a Guided Cognition question. Each Guided Cognition study question was constructed to include one of the five cognitive events used in Experiments 3 and 4: relate to prior experience, visualize and illustrate, consider divergent answers, brainstorm and evaluate, and role play. As in previous experiments, each cognitive event was included once across the five study questions.
As an example of these corresponding pairs of questions, the Guided Cognition question for the relate to prior experience cognitive event was, List five ways Becket protects other people following the battle in France (p. 43 and following). In one or two sentences explain how Becket is like a person from your experience who also protected others. The corresponding Traditional question was, List five ways Becket protects other people following the battle in France (p. 43 and following).
A homework sheet was prepared for each condition. Instructions at the top of each sheet reminded students to read all of Acts 1 and 2 before starting the homework. Additional instructions to the students were: “Write down the time when you start answering the questions. If possible, work straight through until you finish. If you have to take a break, record your stop and restart times. When you're finished, write down the clock time.” Labeled spaces were provided for each student to record the starting time, a break starting time (if needed), a resume time (if needed), and the final stop time.
A review activity (quiz) was constructed to test delayed retention of the content. This quiz used a standard format that included short answers, sentence completions, and fill-in-the-blanks. Each section of this quiz corresponded to a study question and was graded on a 5-point scale, so the total possible quiz score was 25 points. Thus, each cognitive event's contribution was equally weighted.
Design and procedure
Becket was introduced to students on Monday, and they were asked to read Act 1 that evening. Act 1 was discussed on Tuesday, and students started reading Act 2, which was then discussed on Wednesday. For Wednesday evening's homework, the assignment was to complete reading Act 2, to answer five questions on Acts 1 and 2, and to keep track of the time spent answering the questions. Students within each class were randomly assigned
to either the Traditional or Guided Cognition homework condition and were given the assigned condition's homework sheet without being informed of the other type of homework. Homework was collected on Thursday. On Thursday and Friday, students worked on poetry unrelated to the play, after which semester break began. On the second day of school following the break, a delay of 14
days, students were given 30
minutes to complete a previously unannounced review activity (quiz) on Acts 1 and 2 of Becket as a means to refresh their memories of the play that had been interrupted by the holiday break. The review activity was performed like a quiz (closed book, no notes), and students were informed that a serious effort to complete the activity would result in a fixed number of participation points toward their marking period grade.
The experiment timeline is summarized below:
| Monday | Becket was introduced by the teacher. |
|---|---|
| Evening homework was to read Act 1. | |
| Tuesday | Discussion of Act 1. |
| Students began reading Act 2. | |
| Wednesday | Discussion of Act 2. |
| Evening homework was to complete reading Act 2 and to answer either five Traditional or five Guided Cognition questions about Acts 1 and 2 while keeping track of the time spent doing so. | |
| Thursday | Homework was collected. |
| Classwork unrelated to the play. | |
| Friday | Classwork unrelated to the play. |
| Break | Ten-day semester break without homework about Becket. |
| Second day after break | Previously unannounced review activity (quiz) on Acts 1 and 2 of Becket. |
Results
Content retention
Data from students who completed the homework and were present for the quiz were included in the analyses. All students who were present for the quiz completed it within the allotted 30
minutes. Students who had completed Guided Cognition homework 2
weeks earlier performed significantly better on the quiz than students who had completed Traditional homework 2
weeks earlier, thus replicating the Guided Cognition advantage found in Experiments 1–4. Average scores for the GC condition (n
=
31)
and for the T condition (n
=
40) were 87.3% and 77.5% respectively, t (69)
=
3.55, p
=
.001. This gain of nearly 10 percentage points from the Guided Cognition homework is an improvement of almost a letter grade on many grading scales, where grades are 10 percentage points apart.
Time on homework
Self-recorded clock start and stop times were used to calculate the time each student spent completing the homework. Students in the Guided Cognition condition spent significantly more time doing their homework than did students in the Traditional condition, with means of 50.2 and 39.0
minutes, respectively, t (69)
=
2.67, p
=
.010. This is not unexpected because the Guided Cognition homework included activities, such as line drawings and relating to other experiences, which might encourage a more measured approach to answering the questions. Time-on-task may, of course, account for some of the Guided Cognition effect, but Experiments 3 and 4 demonstrated that when time is controlled, the advantage of GC homework is still quite strong.
Relation of time spent on homework and subsequent performance
The range of time spent on homework was quite large: For Traditional homework, reported times ranged from 12
minutes to 110
minutes; for Guided Cognition homework, reported times ranged from 23 to 80
minutes. If time spent on homework was an important factor in subsequent performance, we would expect, within each type of homework, a positive correlation between homework time and review activity (quiz) scores. We found, however, no significant correlations: For the Condition T students, r
=
−.106, ns; for the Condition GC students, r
=
−.092, ns. Figure 3.3 shows a scatter plot of data from both conditions.
Given the results of Experiments 3 and 4, we expected in Experiment 5 to find only a weak correlation between time spent on homework and performance on the 2-week delayed review activity (quiz). What we found, in fact, was no significant correlation of time on homework and subsequent performance. In contrast, we found the usual Guided Cognition effect. One interpretation of these results is that spending more time on homework to make it neat or more detailed may not result in more learning, but engaging in effective cognitive processes, as encouraged by Guided Cognition homework, does result in more learning. In other words, the more important variable is the engagement of learning effective cognitive processes, as encouraged by specific cognitive events that are designed into the homework.
During unsupervised study, students are in control of where they study and of how much time and effort they will invest, but Guided Cognition-designed tasks determine how they will think and which cognitive processes they will engage. Whether students answered Traditional or Guided Cognition questions, more time did not predict better achievement, yet students who answered Guided Cognition questions achieved significantly better scores than students who answered Traditional questions. Clearly, how students think during unsupervised study periods has a decisive effect on what they learn.
Experiments 6 and 7: Do the Benefits of Guided Cognition Persist, or Are They the Result of Novelty?
It has long been known that novelty can affect attention and memory (e.g., von Restorff, 1933) and recent neuroscience has identified brain regions and processes that are most likely stimulated by novelty (e.g., Kishiyama, Yonelinas, & Lazzara, 2004; Schomaker & Meeter, 2015). Because the Guided Cognition questions are different from typical homework questions, it is possible that their novelty could somehow influence students to focus more intensely on, and consequently to better encode, the studied content (e.g., Habib & Lepage, 1999). If so, we would expect repeated exposure to the Guided Cognition tasks to reduce their learning effectiveness.
Experiment 6 with average-ability students and Experiment 7 with advanced-ability students were designed to determine whether the Guided Cognition advantage is a durable, repeatable result, or whether it is a consequence of novelty. In Experiments 6 and 7, students read Shakespeare's Macbeth. Traditional homework was assigned for some students after each of Acts III and IV, whereas Guided Cognition homework was assigned for other students after each of Acts III and IV. A previously unannounced quiz was given to determine whether the Guided Cognition advantage was obtained for just the first Guided Cognition experience (Act III) or was also obtained for the repeated Guided Cognition experience (Act IV).
Method
Participants
Two classes of average-ability, 12th-grade students participated in Experiment 6, and three classes of advanced-ability (advanced placement), 12th-grade students participated in Experiment 7. The same teacher taught all students the same content on the same days.
Materials
Five Traditional homework questions and five corresponding Guided Cognition homework questions were prepared for each of Acts III and IV. For each Act, one of the five content-focused cognitive events (relate to prior experience, visualize and illustrate, consider divergent answers, brainstorm and evaluate, and role play) was incorporated into a Guided Cognition study question. Each cognitive event was included once across the five questions of each Act.
As an example of these corresponding pairs of questions, the Guided Cognition question for the brainstorm and evaluate cognitive event was, Brainstorm to determine the positive and negative effects of King Duncan's murder on Macbeth. List the positive effects on the left and the negative effects on the right. Number the items in the respective lists from most to least positive and from most to least negative. The corresponding Traditional question was, How has Macbeth been affected by murdering King Duncan?
A 3-day delayed previously unannounced quiz was constructed to test this content. The quiz used a standard format that included short answers, sentence completions, and fill-in-the-blanks.
Design and procedure
Within each class, students were randomly assigned to either Condition T or Condition GC. Condition T homework for half of the students in each class
consisted of five Traditional questions for each of Acts III and IV. Condition GC homework for the other students in each class consisted of five Guided Cognition questions for each of Acts III and IV. Act III of Macbeth was taught on Monday and Tuesday with Act III homework assigned for Tuesday evening and collected on Wednesday. Act IV was taught on Wednesday and Thursday with Act IV homework assigned for Thursday evening. On Friday, the Act IV homework was collected at the beginning of class, and work unrelated to the play filled the class period. No Macbeth-related homework was assigned for the weekend. The previously unannounced quiz was given on the following Monday. Students were given 20
minutes to answer five questions about Act III and 20
minutes to answer five questions about Act IV.
The experiment timeline is summarized below:
| Monday | Taught Act III. |
|---|---|
| Tuesday | Taught Act III. |
| Assigned Traditional or Guided Cognition homework for Tuesday evening. | |
| Wednesday | Collected Act III homework. |
| Taught Act IV. | |
| Thursday | Taught Act IV. |
| Assigned Traditional or Guided Cognition homework for Thursday evening. | |
| Friday | Collected Act IV homework. |
| Classwork was unrelated to Macbeth. | |
| Weekend | No Macbeth-related homework. |
| Monday | Previously unannounced quiz about Acts III and IV. |
Results
Data from students who answered each of the 10 homework questions and who were present for the quiz were included in the analyses. For each of Act III and Act IV, each section of this quiz corresponded to a study question. Each quiz section was graded on a 5-point scale, so the total possible score for each Act was 25 points. Thus, each cognitive event's contribution was equally weighted. Students' performance on the 3-day delayed previously unannounced quiz is shown in Figure 3.4.
Experiment 6. For the average-ability students, quiz performance on Act III was 64.4% for Condition GC (n
=
21) and 44.6% for Condition T (n
=
20), and quiz performance on Act IV was 53.2% for Condition GC and 32.9% for Condition T. An ANOVA with homework type (Guided Cognition vs. Traditional) as a between-subject variable and Act (III vs.
IV) as a within-subject variable was conducted and confirmed a significant effect of homework type, F (1,39)
=
22.55, p
<
.000, and a significant effect of Act, F (1,39)
=
17.64, p
<
.000. The interaction between these two variables was not significant, F (1,39)
=
0.007, ns. Clearly, the Guided Cognition homework was more effective, and the gain was not diminished by experience with the Guided Cognition style questions. (Lower overall performance on Act IV is likely a content effect; i.e., the material was more difficult.)
Experiment 7. For the advanced-ability students, quiz performance on Act III was 80.4% for Condition GC (n
=
35) and 70.9% for Condition T (n
=
37), and quiz performance on Act IV was 71.4% for Condition GC and 64.8% for Condition T. An ANOVA with homework type (Guided Cognition vs. Traditional) as a between-subject variable and Act (III vs. IV) as a within-subject variable was conducted and confirmed a significant effect of homework type, F (1,70)
=
8.89, p
=
.004, and a significant effect of Act, F (1,70)
=
27.49, p
<
.000. As in Experiment 6, there was no interaction between these variables, F (1,70)
=
0.98, ns. Thus, the pattern of results found with advanced-ability students follows exactly that of the average-ability students in Experiment 6.
In each experiment, there were significant advantages of Guided Cognition homework, and there were no interactions of homework type with
Act. This pattern of results shows that the advantage of Guided Cognition-designed homework is durable and is not due to novelty. It is also worth noting that the performance of average-ability students who experienced Guided Cognition homework in Experiment 6 approached the performance of advanced-ability students who experienced Traditional homework in Experiment 7.
Experiments 8 and 9: Is Guided Cognition of Unsupervised Learning Effective for Younger Students?
All experiments reported thus far have had high school seniors (ages 17–18) as participants. It is important to determine whether Guided Cognition techniques are effective for younger students, who may approach studying somewhat differently from older, more experienced, and more developmentally mature students. Experiments 8 and 9 were designed to evaluate the effectiveness of Guided Cognition homework for middle school students (ages 12–13).
Method
Participants
Three classes of average-ability, 7th-grade, middle school literature students, taught by one teacher, participated in Experiment 8. Two classes of advanced-ability (honors), 7th-grade, middle school literature students, taught by a second teacher, participated in Experiment 9.
Materials
Students in each experiment were reading The Outsiders by S.E. Hinton, and Chapters 4–6 were used as the content for the experiments. Five Traditional homework questions and five corresponding Guided Cognition homework questions were prepared using five cognitive events: relate to prior experience, visualize and illustrate, consider divergent answers, brainstorm and evaluate, and role play. Each of these five cognitive events was included once across the set of five Guided Cognition questions. As an example of the corresponding pairs of questions, the Guided Cognition question for the role play cognitive event was, Pretend you are Johnny. Speaking in the first person as Johnny (I …), tell why you decide that you and Ponyboy should turn yourselves in. The corresponding Traditional question was, Why does Johnny decide that he and Ponyboy should turn themselves in?
A quiz to evaluate learning from answering the questions was constructed. The quiz consisted of five test items, one corresponding to the
content associated with each of the five questions. For each test item, students were asked to state or list several ideas or facts so that we could evaluate the breadth of learning from each of the five questions and therefore could compare the learning from Guided Cognition homework to the learning from Traditional homework.
Design and procedure
For these experiments, the “homework” questions were completed in class to assure that students did their own work and to maximize completion rates. Students were asked to read Chapters 4–6 of The Outsiders before coming to class on Thursday. These chapters were not taught in class so that learning from the planned in-class homework questions could be clearly assessed. In Experiment 8, one class of students was assigned Traditional questions, and two classes were assigned Guided Cognition questions. In Experiment 9, one class of students was assigned Traditional questions, and one class was assigned Guided Cognition questions.
Students were given five Traditional or five Guided Cognition questions to answer in class on Thursday. They were allowed to use their books, as if this were a homework assignment, and did not discuss their work with other students or with the teacher, thus making this an instance of unsupervised individual learning. On Friday, there was no discussion of the book or of Thursday's questions. Students were assigned unrelated work for Friday and the weekend. On Monday, all students were given the previously unannounced quiz.
The experiment timeline is summarized below:
| Wednesday | Students were assigned homework to read Chapters 4–6 of The Outsiders. |
|---|---|
| Thursday | In-class homework to answer five Traditional or five Guided Cognition questions. |
| Friday | Unrelated classwork. |
| Weekend | Unrelated homework. |
| Monday | Previously unannounced quiz on content of Thursday's in-class homework. |
Results
For each of the five homework questions, students were given a check mark for an answer relevant to the content of the question. Students who received five check marks were included in the analyses.
Using students' pre-experiment course grades from the first 7
months of the 10-month school year, we compared the qualifying students to assure that they were well matched across the two conditions. In Experiment 8, the mean grades were 82.3% and 80.7% for the Traditional (n
=
23) and Guided Cognition (n
=
43) groups, respectively, and analysis confirmed that these averages were not significantly different, t (64)
=
0.83, p
=
.409, ns. In Experiment 9, the mean grades were 91.6% and 90.5% for the Traditional (n
=
19) and Guided Cognition (n
=
19) groups, respectively, and analysis confirmed that these means were not different, t (36)
=
1.33, p
=
.194, ns. Based on their pre-experiment course performance, the students were very well matched in literature achievement across the Traditional and Guided Cognition conditions in each experiment.
The content of each of the five in-class homework questions was evaluated by a corresponding quiz question. Each quiz question was graded on a 5-point scale for a total possible quiz score of 25 points. Thus, each cognitive event's contribution was equally weighted. Students' performance on the 4-day delayed quiz is shown in Figure 3.5.
Experiment 8
. Quiz performance was 64.1% for Guided Cognition homework and 56.0% for Traditional homework, t (64)
=
3.28, p
=
.002. This 8.1 percentage-point advantage confirmed that average-ability middle
school students benefit from Guided Cognition unsupervised individual study.
Experiment 9. Quiz performance was 71.0% for Guided Cognition homework and 61.3% for Traditional homework, t (36)
=
2.95, p
=
.005. This 9.7 percentage-point advantage confirmed that advanced-ability middle school students benefit from Guided Cognition unsupervised individual study.
Discussion
In Experiments 8 and 9, Guided Cognition in-class homework improved student performance nearly a letter grade, assuming that grades are 10 percentage points apart. Looking across the two experiments, it is interesting to note that the average-ability students who had Guided Cognition in-class homework performed as well as advanced students who had Traditional in-class homework. In Experiments 8 and 9, all students had the same materials and experimental conditions, except that different teachers conducted the two experiments. Remember, however, that Chapters 4–6 of The Outsiders (the content of the experiments) were read and studied, but not taught, making the teacher variable unimportant. Therefore, it is fair to conclude that using Guided Cognition homework materials makes it possible to bring the performance of average-ability students to the level of advanced-ability students.
Experiments 10 and 11: Do Process-Focused Metacognitive/Planning/Evaluative Cognitive Events Promote Learning as Do Content-Focused Cognitive Events? Which Individual Cognitive Events Are Effective for Learning During Homework? Does Immediate Experience with Guided Cognition Tasks Affect Subsequent Unguided Learning?
Recall that students in Experiments 1 and 2 did not think that the metacognitive/planning/evaluative process-focused cognitive events were helpful for learning (see Table 3.2). It is well known, however, that learners do not always know what is helpful and do not necessarily know what they know (Bjork, 1994, 1999; Bjork, Dunlosky, & Kornell, 2013; Dunlosky & Lipko, 2007; Kornell, 2015; Metcalfe, 1998, 2009; Schwartz & Metcalfe, 2017). Given these well-understood results, the first purpose of Experiments 10
and 11 was to assess the learning effectiveness of some process-focused cognitive events, compared to the learning-effectiveness of the set of five content-focused cognitive events that were evaluated in Experiments 3 through 9. Therefore, Experiment 10 (with average-ability high school seniors) and Experiment 11 (with advanced-ability high school seniors) included two categories of cognitive events. One category contained content-focused cognitive events that focused the student's attention on to-be-learned content. The other category consisted of metacognitive/planning/evaluative process-focused cognitive events that focused the student's attention on goals, effort, time, and accomplishments.
The second and major purpose of Experiments 10 and 11 was to assess the learning-effectiveness of individual content-focused cognitive events. Experiments 1 through 9 evaluated the effectiveness of Guided Cognition homework design on learning specific course content by looking at the total effect of 10 cognitive events (Experiments 1 and 2) or by looking at the total effect of five cognitive events (Experiments 3 through 9). These experiments have demonstrated the overall effectiveness of Guided Cognition homework under several conditions, but Experiments 1–9 were not primarily designed to assess the effectiveness of each individual cognitive event and did not have sufficiently large sample sizes to do so. Nevertheless, preliminary analyses of data from Experiments 6 and 7 (12th-grade students) and from Experiments 8 and 9 (7th-grade students) found that the mean quiz score associated with each individual cognitive event was in the expected direction (Guided Cognition better than Traditional), and in several cases, individual events were found to be significantly beneficial. Experiments 10 and 11 were designed to have larger sample sizes so we could directly assess the learning-effectiveness of the individual cognitive events.
The third purpose of Experiments 10 and 11 was to assess whether immediate experience with Guided Cognition tasks affects subsequent unguided learning. For this purpose, the experiments were designed to provide comparisons of the unguided learning of students who had experienced Guided Cognition homework with the unguided learning of students who had experienced Traditional homework.
In sum, the designs of Experiments 10 and 11 allow us (1) to measure the learning-effectiveness of process-focused cognitive events as a set and to compare that to the learning-effectiveness of content-focused cognitive events as a set, (2) to determine the learning-effectiveness of the individual content-focused cognitive events, and (3) to measure the proactive effects of Guided Cognition experience.
Method
Participants
Four classes of average-ability, 12th-grade, high school literature students, two taught by one teacher and two taught by another teacher, participated in Experiment 10. Five classes of advanced-ability (advanced placement and honors), 12th-grade, high school literature students, three taught by one teacher and two taught by another teacher, participated in Experiment 11.
Materials
Students in each experiment were reading Shakespeare's Macbeth, and Acts III and IV were used as the content for the experiments.
For Act III, five Traditional (T) homework questions and five corresponding content-focused cognitive event (GC) homework questions were prepared. As in previous experiments, the five content-focused cognitive events used for the Guided Cognition homework were: relate to prior experience, visualize and illustrate, consider divergent answers, brainstorm and evaluate, and role play. In addition, some students were asked to perform five metacognitive/planning/evaluative process-focused cognitive events (M). These events were: set goals, estimate time needed, measure time used, evaluate quality of work completed, and evaluate quantity of work completed.
For Act IV, five Traditional (T) homework questions were prepared.
A 10-question quiz was constructed to evaluate learning from the Act III and Act IV homework. The 10 quiz questions corresponded to the content areas of the 10 homework questions. For each quiz question, students were asked to state or list several ideas or facts so that we could evaluate the breadth of learning from each of the 10 homework questions.
Design and procedure
For Act III, each student was given one of four homework conditions. These were Traditional content questions (T), Traditional content questions plus process-focused cognitive events (T
+
M), content-focused cognitive event questions (GC), and content-focused cognitive event questions plus process-focused cognitive events (GC
+
M). Homework materials for all four conditions are shown in Appendix III.
The four Act III homework conditions were assigned as follows: Within each classroom, students were rank-ordered according to their literature course grades for the first quarter of the school year. Each of the four students in each successive quartet within the rank order was assigned to a different
one of the four conditions. The order of conditions within successive quartets was rotated so that the highest ranked students in successive quartets were assigned equally to each condition, the second-highest ranked students in successive quartets were assigned equally to each condition, etc.
For Act IV, all students received the same Traditional (T) content homework questions.
Act III of Macbeth was taught on Monday and Tuesday with Act III homework assigned for Tuesday evening and collected on Wednesday. Act IV was taught on Wednesday and Thursday with Act IV homework assigned for Thursday evening. On Friday, the Act IV homework was collected at the beginning of class, and work unrelated to the play filled the class period. No Macbeth-related homework was assigned for the weekend. The previously unannounced quiz was given on the following Monday. Students were given 20
minutes to answer five questions about Act III and 20
minutes to answer five questions about Act IV.
The experiment timeline is summarized below:
| Monday | Taught Macbeth Act III. |
|---|---|
| Tuesday | Taught Macbeth Act III. |
| Assigned Act III homework for evening. | |
| Wednesday | Collected Act III homework. |
| Taught Act IV. | |
| Thursday | Taught Act IV. |
| Assigned Act IV homework for evening. | |
| Friday | Collected Act IV homework. |
| Classwork unrelated to Macbeth. | |
| Weekend | No homework was assigned. |
| Monday | Previously unannounced quiz on Acts III and IV of Macbeth. |
Results
Quiz data evaluating learning from the Act III homework, where students received one of the four types of homework, are of primary interest. Students were given a check mark for each homework question that had an answer relevant to the content of the question. Students who received check marks for each of the five Act III homework questions, and who were present to receive the quiz, were included in the analysis of quiz performance on the Act III content. There was some attrition because of absences or incomplete homework, so we compared the pre-experiment grades of the qualifying students to assure that they were still well matched across the four conditions.
The mean course grades for Experiment 10 students were 83.2% (n
=
25), 83.7% (n
=
22), 84.3% (n
=
19), and 84.4% (n
=
19) for T, T
+
M, GC, and GC
+
M conditions, respectively. As expected, there was no condition assignment main effect of GC, F (1, 81)
=
0.22, p
=
.638, ns; there was no condition assignment main effect of M, F (1, 81)
=
0.02, p
=
.876, ns; and there was no GC by M assignment interaction, F (1, 81)
=
0.02, p
=
.899, ns. The mean course grades for Experiment 11 students were 89.1% (n
=
22), 88.5% (n
=
24), 89.3% (n
=
22), and 89.1% (n
=
20) for T, T
+
M, GC, and GC
+
M conditions, respectively. As expected, there was no condition assignment main effect of GC, F (1, 84)
=
0.27, p
=
.604, ns; there was no condition assignment main effect of M, F (1, 84)
=
0.33, p
=
.569, ns; and there was no GC by M assignment interaction, F (1, 84)
=
0.07, p
=
.792, ns. Students who qualified for the Act III analysis were, therefore, very well matched according to pre-experiment performance in the literature course.
Overall effects of Act III homework
The content of each of the five homework questions was evaluated by a corresponding quiz question. Each quiz question was graded on a 5-point scale for a total possible quiz score of 25 points. Thus, each cognitive event's contribution was equally weighted.
To determine whether we replicated the overall Guided Cognition effect with the content-focused cognitive events, and to determine whether we obtained an additional learning boost from the process-focused cognitive events, we performed a between-subjects analysis of the total Act III quiz score for the four homework conditions.
Experiment 10. Performance of the average-ability students shows a main effect of GC, F (1, 81)
=
22.20, p
<
.000, with quiz performance of 53.3% after Guided Cognition homework and 39.5% after Traditional homework, a learning advantage of Guided Cognition homework over Traditional homework of 13.8 percentage points. There was no main effect of M, F (1, 81)
=
0.003, p
=
.959, ns; and no GC by M interaction, F (1, 81)
<
.000, p
=
.988, ns.
Experiment 11. Performance of the advanced-ability students shows a main effect of GC, F (1, 84)
=
44.04, p
<
.000, with quiz performance of 68.3% after Guided Cognition homework and 54.4% after Traditional homework, a learning advantage of Guided Cognition homework over Traditional homework of 13.9 percentage points. There was no main effect of M, F (1, 84)
=
1.14, p
=
.289, ns; and no GC by M interaction, F (1, 84)
=
0.04, p
=
.835, ns.
In each experiment, therefore, the content-focused Guided Cognition homework raised quiz performance more than a letter grade, assuming that letter grades are 10 percentage points apart. Across experiments, it is again interesting to note that the average-ability students who had content-focused Guided Cognition homework performed as well on the quiz as the advanced-ability students who had Traditional homework. These results, the main effect of Guided Cognition homework compared with Traditional homework, are shown in Figure 3.6 and parallel those seen when comparing across Experiments 8 and 9 (see Figure 3.5) where younger students and different content were involved.
Although this equivalence does not always occur, it is reasonable to expect subsequent quiz performance of average-ability students who have had content-focused Guided Cognition homework to approach (see Experiments 1 and 2; 3 and 4; 6 and 7) or to equal (see Experiments 8 and 9; 10 and 11) the subsequent quiz performance of advanced-ability students who have had Traditional homework.
It is worth noting that the performance data of students in Experiments 10 and 11 validate the opinions of students in Experiments 1 and 2, regarding the value of process-focused cognitive events. Recall that students in Experiments 1 and 2 mostly judged the learning benefits of process-focused cognitive events to be neutral or negative (see Table 3.2). Several
years later, different students, studying the same content, demonstrated no additional learning from performing the process-focused cognitive events.
Given the vast literature on metacognition and its importance for learning (e.g., Bjork et al., 2013; Schwartz & Metcalfe, 2017), it may seem surprising that the process-focused, metacognitive/planning/evaluative cognitive events had no effect whatsoever on learning from the homework. The format of Experiments 10 and 11, however, did not require follow-up study that could have been directed effectively by the self-reflection activities. From the results of Experiments 10 and 11, it is fair to conclude that setting goals, estimating time, measuring time, evaluating quality of work completed, and evaluating quantity of work completed are not useful cognitive activities unless they are followed by subsequent study activities, which make use of the self-reflection for productive self-regulated learning processes. We hypothesize that students in these experiments did not voluntarily engage in follow-up study after the self-reflection activities and that such follow-up study might typically need to be externally motivated.
Effects of individual content-focused cognitive events in Act III homework
Because there was neither a main effect of the metacognitive/planning/evaluative, process-focused cognitive events nor an interaction of these cognitive events with the presence or absence of content-focused cognitive events, we combined the data of the GC and the GC
+
M conditions (hereafter referred to as GC∗) and compared them with the combined data of the T and T
+
M conditions (hereafter referred to as T∗) to evaluate the learning-effectiveness of the individual content-focused cognitive events.
Experiment 10. The t-tests evaluating quiz performance after GC∗ or T∗ confirmed that each of the five content-focused cognitive events was significantly beneficial for the average-ability students. Results are shown in Table 3.3.
Experiment 11. The t-tests evaluating quiz performance after GC∗ or T∗ confirmed that four of the five content-focused cognitive events were significantly beneficial for the advanced-ability students, and the fifth was nearly so. Results are shown in Table 3.4.
Summarizing across Experiments 10 and 11, it is clear that each of the five studied cognitive events can be significantly beneficial for learning. The effectiveness of a specific cognitive event is likely a function of the studied content and a function of the cognitive skills and habits of a student. As an example of how content may interact with the effectiveness of a cognitive event, suppose that the studied content is already highly visual in nature. In this case, adding a homework study requirement to visualize and illustrate
the content may not add much to the student's already engaged cognitive processes. On the other hand, adding visualization to content that is not as inherently visual may create a more memorable representation than would otherwise have been created. In particular, creating a simple line drawing of a scene from a story can help organize and unify various details of the scene, thereby improving recollection of those details.
Table 3.3
Table 3.4
As an example of how ability might interact with the effectiveness of a cognitive event, suppose that higher-ability students in a particular subject-matter area are in the habit of visualizing the material more than are lower-ability students. In such a case, we would expect the lower-ability students to gain more from a homework requirement to visualize and illustrate the content than would the higher-ability students who are already using their visualization capabilities.
Although effectiveness of individual cognitive events may vary with specific to-be-learned content and with the cognitive abilities and habits of individual students, each cognitive event has been shown to be effective for some content, and no cognitive event has produced lower performance than Traditional homework. Given these results, a practical strategy is to design homework tasks that include any or all of the five studied cognitive events in order to facilitate learning from the homework.
Proactive effects of Guided Cognition experience
Quiz data evaluating learning from the Act IV homework, where all students received the same Traditional homework, can be used to evaluate whether there is a proactive (or carryover) effect from Guided Cognition study of the Act III homework to quiz performance on the content of the Act IV homework. For homework on Acts III and IV, students were given a check mark for each homework question whose answer related to the question's content. Students were included in the analysis of quiz performance on the Act IV homework content if they had received check marks for each of the five Act III homework questions and for each of the five Act IV homework questions. Because there was additional attrition due to absences or incomplete homework, we compared the pre-experiment grades of the qualifying students to assure that they were still well matched across the four conditions. The average course grades for Experiment 10 students were 84.4% (n
=
21), 84.5% (n
=
15), 84.3% (n
=
19), and 85.9% (n
=
15) for T, T
+
M, GC, and GC
+
M conditions, respectively. As expected, there was no condition assignment main effect of GC, F (1, 66)
=
0.13, p
=
.723, ns; there was no condition assignment main effect of M, F (1, 66)
=
0.19, p
=
.666, ns; and there was no GC by M assignment interaction, F (1, 66)
=
0.15, p
=
.698, ns. The average course grades for Experiment 11 students were 89.5% (n
=
17), 88.6% (n
=
22), 89.2% (n
=
20), and 89.1% (n
=
18) for T, T
+
M, GC, and GC
+
M conditions, respectively. As expected, there was no condition assignment main effect of GC, F (1, 73)
=
0.003, p
=
.954, ns; there was no condition assignment main effect of M, F (1, 73)
=
0.44, p
=
.509, ns; and there was no GC by M assignment
interaction, F (1, 73)
=
0.23, p
=
.633, ns. Students who qualified for the Act IV analysis were, therefore, very well matched according to pre-experiment performance in the literature course.
As was true for Act III homework, the content of each of the five Act IV homework questions was evaluated by a corresponding quiz question. Each quiz question was graded on a 5-point scale for a total possible quiz score of 25 points. Because all students received the same Act IV Traditional homework, unless the effect of the Act III Guided Cognition homework carries forward or is proactive, we should observe the same Act IV quiz performance whether students had Guided Cognition or Traditional homework on Act III.
Experiment 10. Performance of the average-ability students shows a main effect of prior Guided Cognition experience, F (1, 66)
=
4.14, p
=
.046, with quiz performance of 43.8% on Act IV content after experiencing Act III Guided Cognition homework and 35.2% on Act IV content after experiencing Act III Traditional homework, a proactive learning advantage of prior Guided Cognition homework experience over prior Traditional homework experience of 8.6 percentage points, nearly a letter grade increment, assuming grades are 10 points apart. There was no main effect of M, F (1, 66)
=
0.01, p
=
.912, ns; and no GC by M interaction, F (1, 66)
=
0.52, p
=
.475, ns.
Experiment 11. Performance of the advanced-ability students shows no main effect of prior Guided Cognition experience, F (1, 73)
=
0.06, p
=
.816, ns, with quiz performance of 56.8% on Act IV content after experiencing Act III Guided Cognition homework and 55.8% on Act IV content after experiencing Act III Traditional homework. There was no main effect of M, F (1, 73)
=
0.32, p
=
.571, ns; and no GC by M interaction, F (1, 73)
=
2.30, p
=
.133, ns.
Discussion
A proactive Guided Cognition effect, such as observed in Experiment 10, could be due to Act III Guided Cognition experience influencing how students study Act IV, even though they received Traditional questions for Act IV. Or, a proactive Guided Cognition effect could be a retrieval phenomenon, such that better performance on Act III in some sense cued recall of some content of Act IV. We obtained a reliable proactive Guided Cognition effect for the average-ability students (Experiment 10), but not for the advanced-ability students (Experiment 11). These results point to a differential storage explanation: It is plausible that the average-ability students were influenced to study for Act IV in ways suggested by the Guided Cognition homework experience of Act III and that these study approaches were
more effective than their usual strategies. It is also plausible that the advanced-ability students already had fairly effective study strategies that they could fall back on and that they did not perceive the superiority of the Guided Cognition strategies over their usual ones. As a result, advanced-ability students may have been less influenced to use Guided Cognition strategies. The proactive Guided Cognition effect of just the average-ability students does not, on the other hand, support a differential retrieval explanation: If the proactive effect resulted from retrieval cuing, both ability levels should have shown the proactive effect because both ability levels had an Act III- content Guided Cognition effect of almost 14 percentage points.
Conclusions of Guided Cognition Research for Literature Homework Design
Several conclusions can be drawn from the results of these 11 literature experiments:
- 1. Guiding cognition through the wording of homework questions and tasks can engage students in specific cognitive events that elicit learning-effective cognitive processes, which significantly increase learning from homework.
- 2. Homework effectiveness is not a simple function of time spent on the tasks but is a function of the kinds of cognitive events (and their corresponding cognitive processes) that students engage in while studying.
- 3. Gains from Guided Cognition homework are found with and without explicit teaching, so the homework itself is a valuable part of the total learning effort.
- 4. Benefits from Guided Cognition homework are not due to novelty. This result implies that Guided Cognition can be used frequently without loss of effectiveness.
- 5. Guided Cognition benefits may become more apparent at longer retention intervals, so this form of homework may be of great benefit for very long-term retention.
- 6. Students generally perceive content-focused cognitive events as helpful for learning, but they generally perceive metacognitive/planning/evaluative process-focused cognitive events as not helpful for learning.
- 7. Content-focused cognitive events are effective for learning specific content, but metacognitive/planning/evaluative, process-focused cognitive events, under the conditions of our experiments, are not.
- 8. Each of the five content-focused cognitive events that were used in our experiments (relate to prior experience, visualize and illustrate, consider divergent answers, brainstorm and evaluate, and role play) was effective in improving learning when inserted into homework questions.
- 9. Immediate experience with Guided Cognition homework may have a proactive benefit for average-ability students on subsequent unguided study.
- 10. The benefit of Guided Cognition varies from content to content, but overall it can be expected to increase subsequent quiz performance approximately one grade or more, assuming that grades are 10 percentage points apart.
- 11. Average-ability students who receive Guided Cognition homework approach, and in some cases match, the performance of advanced-ability students who receive Traditional homework.
- 12 The Guided Cognition effect in studying literature has been replicated with a variety of content (Shakespearean tragedy, modern drama, classic novella, young adult novel); over a wide range of grades (7–12) and ages (12–18); and for average- and advanced-ability students.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.