Effective Communication to Improve the Quality of University Instruction
Ernesto Schiefelbein F. and Noel F. McGinn
This chapter describes the elements and outcomes of a strategy to transform instructional practices in a midsized, multicampus university in Latin America. A catalytic model approach was used to connect central university management with academic units, mobilize concern about student failure rates, and over time to reach an agreement on a small but significant change in course design. This change prompted further discussion of instructional practices eventually resulting in further changes and improvement of learning outcomes.
The University’s Problem
In Chile, as in other countries of Latin America, university enrollments in recent years have increased rapidly so that they now involve a significant proportion of the eligible population. The expansion of access has been greatly welcomed and also has made more evident serious problems of quality and inequality. These challenges have sparked an intense national debate (OECD, 2012).
The population of higher education students in Chile increased almost five times over two decades: from less than 250,000 in 1990 to more than 1.2 million at present. This increase implies that the net enrollment rate of the 18- to 24-year-old population soared from 12% in 1990 to 33% in 2011 (Mineduc, 2012). A prime beneficiary from this expansion has been youth from low income families. In the 1990s, only 25% of students in higher education came from families in the lower three-fifths of the income distribution. At present they constitute 40% of the total enrollment.
Students enter university with different levels of preparedness, linked to the secondary school they attended. Students in Chile attend one of three kinds of secondary schools: free municipal, subsidized private, or fee-charging private. Average scores on a national achievement test for students attending free schools are about one standard deviation below those of students attending fee-charging private schools (Mineduc, 2011). Most students in the lower three-fifths of the income distribution attend municipal schools, whereas most of those in the higher quintiles attend fee-charging schools.
Differences in achievement test scores are as a consequence attributable in part to differences in the quality of the secondary school attended. In Chile, teacher attendance rates are lowest in free and subsidized schools, as is actual time spent by the teacher in the classroom, compared with that of teachers in fee-charging schools. As a consequence, teachers in the free and subsidized schools cover less of the official curriculum on which the national achievement test is based (Arango, 2008). Estimates are that students in schools in lower income neighborhoods are taught about 50% of the official curriculum (Centro de Estudios, 2013).
Students’ levels of academic preparation are also influenced by methods of teaching. The predominant method of instruction in secondary schools in Chile is teacher-centered, directed at the whole class (frontal teaching), and emphasizes learning by memorization over experimentation or reasoning (Edwards & Calvo, 1995). The results of this ineffective instruction are reflected in reading comprehension: only 20% of the population aged 15–65 years has achieved a level of functional literacy sufficient to meet the demands of daily life and work in a complex and advanced society (IALS-OECD, 2000).
In effect, therefore, more than half of the current students entering higher education in Chile have difficulties in understanding what they read, a fundamental skill required for success in the university. In addition, as an increasing proportion of students represent the “first generation” of their families to attain this level of education, more students than in previous years have little understanding of the demands of the culture of the university (Larraín & Zurita, 2008). Taken together, these factors help explain why only half of the students who enter the university actually complete the requirements for graduation (Elacqua, 2012).
Student failure of courses and consequent failure to graduate was of great concern to the university, which is the subject of this chapter. Created in 1989, it was founded with a mission to enable children from lower income families to complete higher education. The university actively pursued recruitment of first-generation students. By 2011, total enrollment had grown to 20,000, distributed across 4 campuses and 30 programs. Two-thirds of the students in the university come from low-income families, attend secondary schools in lower income neighborhoods, and finance their university studies with loans.
In pursuit of its mission, the university seeks to prepare students who will play an important role in Chilean society. It offers a full range of professional career programs (e.g., medicine, psychology, industrial engineering), each of which grants a professional degree on completion. As in other universities in Chile, all students in a given program take the same fixed set of courses.
Although many of the university’s students are first generation, its professors are not. Most were raised in middle and upper income families and studied in secondary schools located in higher income neighborhoods. Generalizing from their personal experience, these professors expected that entering students would have the basic skills and knowledge necessary to master the curriculum in their field. They had no idea that many of the terms used in their lectures went completely “over the heads” of many students. A simple example: in mathematics students could grasp the meaning of a vernacular term such as “casual,” but miss the significance of more technically precise terms such as “fortuitous,” “chance,” or “random.”
The difference in social origins of students and professors poses a basic dilemma for the university. If it continues with current method of frontal teaching and high standards, students from lower quality schools will continue to be likely to fail to graduate and the university will perpetuate the social gap (which is a root cause of the students’ failure). If professors lower their standards to take into account low levels of prior knowledge, this will limit learning of the existing curriculum and produce graduates whose skills and knowledge are insufficient to compete in the labor market, again perpetuating the social gap.
In order to address this issue, the university first had to learn what was being taught in the various programs. Most professors were teaching without a detailed course description or syllabus (often providing only a list of recommended readings). In addition, most of the courses in the university are taught by part-time professors who also teach in other universities. In the absence of a syllabus, the university had no way to insure that what was being taught in one course was compatible with that taught in other courses in a given program.
Prior to this, the university had made several efforts to overcome the gap between prior knowledge and demands of the programs. The Governing Board of the university increased the proportion of professors with postgraduate degrees, encouraged professors to assess their own teaching, and sought advice from leaders of other universities that had been successful in their efforts to raise the quality of teaching. It also sought to improve the integration of first-generation students into the university culture. These actions had little effect on overall completion rates, however. As a consequence, in 2006 the Board created a working group to measure and explain more fully the problem of failure and non-completion, to establish priorities, and to identify possible strategies of solution.
Analysis of Paths to Solution
The success of students in higher education is explained in part by factors that are essentially unchangeable, such as experiences in early childhood. Students are, however, to a certain degree malleable. Actions taken, once a student is attending the university can improve academic performance. It is possible once students are enrolled to reduce gaps in student knowledge and to improve their study habits, to raise their reading ability and improve their use of time and in general increase their capacity to relate concepts and learn more. For example, through interventions that help students achieve academic successes, it is possible to build a positive feedback loop linking their sense of self-efficacy (in academic endeavors) with self-esteem resulting in greater effort (Bresó, 2011; Chan, 2008; Feria, 2010). Taking limitations of prior knowledge and learning ability into account, it is possible to employ incentives that increase the probability of academic success and eventual graduation.
On the other hand, few remedial programs have had success in Chilean higher education. Nor, have they had striking success in American universities (Bettinger & Long, 2009; Complete College America, 2012). Even though college students in the United States can maintain a regular course load by taking up to 40% of their courses as remedial without grades (NCSL, 2012; Peter D. Hart Research Associates, 2005), graduation rates fall well below expectations. In Chile, because all students in a given program take the same fixed set of courses those who fail a course must repeat it in the following semester along with the regular load. This separation of students from their cohort reduces the likelihood of eventual completion (Cornelius-White, 2007).
The remedial courses that are more often successful are those that involve students as active participants in the teaching–learning process. This can be achieved by asking students to prepare themselves before class, usually by appropriate reading, reflection on questions based on the material, or doing exercises or carrying out applications, followed up by discussion in class. These apparently simple actions have various positive effects on the teaching and learning process. The mental responses they engender include increased familiarity with the terminology (vocabulary) used by the professor; formulation of questions to clarify doubts and misunderstandings of the material; increased confidence in ability to participate in class (which facilitates the professor’s task); and increased self-esteem resulting in greater effort (Coulter & Smith, 2012; Herreid & Schiller, 2013; Koontz & Plank, 2011).
The university’s review of research found programs in several other universities that had succeeded in promoting active student participation in the teaching–learning process with consequent reduction in course failure rates. The review indicated that it is possible to get students to prepare their classes but that some effort is required, and that the use of adequate incentives requires commitment of professors and strong leadership by the administration. It did not seem that required methods and structures would work in all instances (Duranczyk et al., 2004; Wilcox & Angelis, 2011). An effort was made to understand the conditions in which this strategy would be effective with first-generation university students in Chile.
Near the end of 2011, two unexpected events galvanized the university into action. The first was a visit by university officials to two universities in Germany that recently had improved their completion rate. The experience of these universities demonstrated that, in order to improve academic performance, students had to spend more time on study, and that this requires overcoming the tradition of review of what the professor covered in the previous class. Second, a visit to the university by a professor from Harvard University introduced officials to the use of the “flipped class” method (Berrett, 2012; Mazur, 2012) to teach science. This professor also recommended getting students to read in advance about what would be covered in their next class. Given this background, the Governing Board in September 2011 approved the introduction of a method of Reading for Active Participation (RAP) at the beginning of the 2012 academic year. Immediately afterwards, 150 program directors and professors participated in a workshop operated as a “flipped class.”
In introducing RAP, officials expected that it would be relatively easy to get students to read a brief text before each class. A more serious challenge was how to prepare appropriate texts for the 5,000 new students who would begin classes in 30 different programs in March 2012. This would require describing the specific content of each weekly class session and its accompanying reading. These outlines or syllabuses would include some 60–70 pages for each of 156 distinct courses. The courses are distributed across four campuses and involve 800 sections. It was a large endeavor, but university officials were hopeful that given several experiments in previous years, and the outcome of the September 2011 workshop, that the innovation could be implemented successfully.
Designing a New Model for Teaching, Communicating, and Implementing
The challenge facing new (first-generation) students was highlighted by a diagnostic study of “study methods and habits” carried out by the Department of Undergraduate Teaching in March 2012. The study showed that 64% of the new students needed to improve their study habits and methods, and that 40% lacked adequate physical space for studying (Unpublished).
The working group decided to provide students with a maximum of two pages (1,000 words) of reading for each class included in the syllabus. The reading could be completed in 5–7 minutes (in many cases en route to the University from home or during a break). While the syllabus would describe the course themes and learning objectives, the anticipatory text would define for each session the specialized terms to be used, and introduce the critical knowledge required to understand the session. This would compensate for gaps in knowledge of students from those secondary schools that did not provide complete curriculum coverage.
The Department of Undergraduate Teaching and the program directors oversaw the preparation of the syllabus (course outline) and anticipatory reading material for each of the 156 courses given in the first semester of each program. Each syllabus was authored by a professor chosen from the regular faculty. The authors in turn coordinated their efforts with the professors who would teach one or more of the sections of the courses. Authors were compensated for their time once the syllabus was approved by the other professors offering the course.
Students learned about RAP when they first contacted the university to ask about programs and requirements for admission. They were informed again on enrollment, and during the orientation sessions in the first week of classes. On these occasions, students were told about how doing the preparatory reading would benefit them. They also learned how to access the syllabuses and anticipatory readings on SAGAF, the computer platform provided by the university.
The justification of the RAP method to professors was simple and direct—“when students read, they usually participate more actively in class and understand the answers professors give to their questions”—emphasizing that this benefit comes only if students actually do the reading at least once. In effect, students learn more when they accept (some measure of) responsibility for their own learning. To motivate students to read, research had shown, professors should begin each class with a question based on the anticipatory reading directed at a few students chosen at random, and followed by assignment of a grade based on the response given (Dunlap, 2012; Leamnson, 1999; Nilson, 2010; Weimer, 2011).
The introduction of RAP asked the professors to make six changes in their teaching. They were asked to:
1. cover, in each class, the topic scheduled in the syllabus and anticipatory reading (because otherwise the students’ reading would serve no purpose reducing their likelihood of preparing for the class);
2. dedicate the first 4 min of each session to asking one to three students chosen randomly to comment on a question based on the anticipatory text;
3. evaluate out loud the students’ response assigning a grade (that would contribute a small amount to the final grade);
4. follow by asking students for any doubts raised by the reading and providing clarification;
5. carry out the rest of the session in the usual manner (with no further reference to the students’ anticipatory reading); and
6. at the end of the class recommend that students download material for the next session from SAGAF to determine whether it raised any questions that the professor should answer at the next session.
While these changes seemed simple enough, previous pilot tests had revealed that some professors neglected to ask one or more students about the reading, and some neglected to evaluate the response. Some professors appeared to resist assigning a grade to motivate students to read. They agreed that reading is a habit acquired in early childhood in the family and that some students lacking that experience needed to learn its benefits for university study, but felt uncomfortable assigning a grade to the students’ response (even though the assigned grade had only a small impact on the final grade).
University officials anticipated, therefore, that the 700 professors would vary in their implementation of the six-part innovation. How would it be possible to determine the extent to which RAP was being put into practice and to persuade professors to implement the program as designed? Draconian regulations, classroom observation, and incentives or sanctions for compliance were ruled out.
A Catalytic Model to Change Instructional Practices Without Direct Intervention
The initial steps taken were intended to create conditions that over time will stimulate and facilitate changes in the behavior of both students and their professors. The interventions taken, while worthwhile, were not ends in themselves but instead intended as a catalyst for change (Christensen et al., 2006; Waddock & Post, 1991). Student study practices were deemed easier to change than professors’ instructional practices, but changes in the former (combined with other actions) would contribute to changes in the latter.
An effort was made to encourage family members to persuade students to adopt the practices of the RAP method. In January and February, at the time of enrollment, the Admissions Office introduced new students (and family members who accompanied them) to the syllabus and RAP, demonstrating how to use the information system to access the material for each class in every one of their first semester courses. Emphasis was placed on the importance of preparing for upcoming classes by reading the anticipatory texts.
Student response to the introduction of RAP was assessed by voluntary (anonymous) surveys of students distributed and returned using the university’s computerized information system. The first survey, in March 2012, was designed to detect startup problems students had with the method (e.g., access to anticipatory texts), and the extent to which professors and they were following the method. Discussion groups were held with professors on each campus to identify their perception of benefits and problems generated by the innovation, to elicit suggestions for improvements in the design for the second semester (August to December 2012), and to plan changes in the syllabuses and anticipatory texts prior to the first semester of 2013.
Surveys carried out at the end of the first and second semester assessed the implementation of the RAP method. This implementation involved seven different elements (e.g., explanation of the purpose of the method, access to material on SAGAF, training of professors in use of the method, etc.). As Table 5.1 indicates, student approval increased directly with the number of elements of the method that they experienced. At the end of the first semester, the “grade” students assigned to RAP (a measure of approval) increased directly with the number of elements implemented. The survey at the end of the second semester indicated that student reading of the anticipatory text increased directly with the number of elements carried out.
Implementation of the method improved significantly in each of the next 2 years. Table 5.2 reports on three elements that characterize the rate of improvement. Over the 3 years the proportion of professors asking questions about reading of the anticipatory text increased from 15.9% to 92.3%, and the proportion assigning grades increased from 7.6% in 2012 to 84.6% in 2014. During this period the deans and the program directors used the survey results to tell professors about student enthusiasm for the method and their increased reading of course material. Sessions held with professors on each campus discussed their experiences with the introduction of the method. The next step was to show that student enthusiasm for the Method reflected their awareness of improved learning (and higher course grades).
Table 5.1 Level of Implementation, Student Evaluation of RAP, and Reading of Text (2012)
Table 5.2 Opinion of Students About Classroom Implementation of RAP 2012–2014
Impact of Student Reading Before Class on Academic Performance (Grades and Course Completion)
Grades are dependent on several factors. Difficulty level varies from course to course and across disciplines and programs as well as from professor to professor. The importance attributed to performance on the final examination varies across courses. Some professors take into account student participation in class in assigning grades, whereas others do not. Grades can also vary as a function of university norms (e.g., grade inflation reflects a general relaxation of common standards). Grade point averages reflect all these factors. Even so, we can expect that students whose prior (course relevant) knowledge is high will do better than those whose knowledge is low. This is one reason why university grades are correlated with admission test scores and secondary school grade point averages.
The impact of prior knowledge is minimized by comparing changes in grade point averages over time. Students with low prior knowledge should be able to increase their knowledge as much (if not more) than those who start with high levels of knowledge. To observe the impact of reading the anticipatory text on learning (as measured by grades), we compared average grades across courses for each of the three succesive semesters. As we are comparing grades for the same individuals at three points time, constant factors (such as family income and education, admission test scores, and academic program) should not explain any differences that appear.
Table 5.3 presents the average grade point averages according to whether the student reports reading the anticipatory text. At the end of the first semester, students who said they read the text had an average grade point average of 4.80, while those who said they did not read it had an average of 4.71. This difference is not statistically significant. The difference in grade point averages is larger at the end of the second semester of enrollment (Spring 2012) but still not statistically significant. By the end of the third semester, however, the difference is large enough to be considered reliable. Students who say they read the anticipatory text have higher grade point averages than their classmates who do not read.
Table 5.3 Grades and Courses Passed by “Readers” and “Non-readers” Who First Enrolled in March 2012
A similar result is obtained by comparing the proportion of courses passed. The difference is insignificant at the end of the first semester, but by the end of the third semester, reading students pass more courses than those who do not read. Students who fail courses in Chile are obliged to repeat them, and students who repeat courses are more likely to end up leaving the university before graduation. Course failure is not the only reason for deserting, but a comparison of the number of students in the cohort of 437 who answered the survey in the first semester of 2012 with those still enrolled in the second and third semesters suggests that 7.7% of the read-before-class group dropped out by the end of the third semester, whereas 8.6% of the non-reading group dropped out. If the effect of reading before class is cumulative over time, by the end of the 8th or 10th semester of study, the two groups of students will be markedly different in their academic performance and degree completion.
The effect of the RAP method should increase over time as more and more professors implement all of its elements, and as more students understand the benefits of reading before class and learn to study effectively. A less tangible but important impact of RAP is changes in what occurs in the classroom. Implementation of the method required small changes in professors’ conduct of their classes. These changes set in motion other changes that may over time have a more profound impact. In various panel discussions, students reported that as a consequence of the anticpatory reading they felt more confident in class. This confidence led to asking questions during the professor’s lecture, which in some courses resulted in active discussions. In separate meetings, professors reported that student participation in class had increased; they felt this was a positive outcome, that students were now more interested in what the professor was teaching.
If the effects of RAP are in fact cumulative and the method continues to improve the quality of instruction in the University, this case can serve as an example for other institutions seeking to overcome educational inequalities by providing first-generation students with a more effective teaching program.
References
Arango, A. (2008, July 30). The failings of Chile’s educational system: Institutionalized inequality and a preference for the affluent. Retrieved February 13, 2013, from Council on Hemispheric Affairs: http://www.coha.org/the-failings-of-chile%E2%80%99s-education-system-institutionalized-inequality-and-a-preference-for-the-affluent/
Berrett, D. (2012). How ’flipping’ the classroom can improve the traditional lecture. The Chronicle of Higher Education, 58 (25), A16–A18.
Bettinger, E.P., & Long, B.T. (2009). Addressing the needs of under-prepared students in higher education: Does college remediation work? Journal of Human Resources (http://www.nber.org/papers/w11325), 44 (3).
Bresó, E.S. (2011). Can a self-efficacy-based intervention decrease burnout, increase engagement, and enhance performance? A quasi-experimental study. Higher Education, 61(4), 339–355.
Centro de Estudios. (2013). Implementación del currículum de Educación Media en Chile. Serie Evidencias. Santiago, Chile: Ministerio de Educación.
Chan, J.C.-F. (2008). Effects of different evaluative feedback on student’s self-efficacy in learning. Instructional Science, 38(1), 37–58.
Christensen, C.M., Baumann, H., Ruggles, R., & Sadtler, T.M. (2006). Disruptive innovation for social change. Harvard Business Review, 84(12), 94–101.
Complete College America. (2012). Remediation: Higher Education’s Bridge to Nowhere. Washington DC: Complete College America.
Cornelius-White, J. (2007). Learner-centered teacher-student relationships are effective: A meta-analysis. Review of Educational Research, 77(1), 113–143.
Coulter, C.J., & Smith, S. (2012). The impact of preclass reading assignments on class performance. Currents in Pharmacy Teaching and Learnng, 4(2), 109–112.
Dunlap, J. (2012). Encouraging Students to Read Before Class. Denver, CO: Center for Faculty Development, University of Colorado.
Duranczyk, I.M., Higbee, J.L., & Lundell, D.B. (2004). Best Practices for Access and Retention in Higher Education. Minneapolis MN: Center for Research on Developmental Education and Urban Literacy, Urban College, University of Minnesota.
Edwards, V., & Calvo, C. (1995). El liceo por dentro, estudio etnográfico sobre prácticas de trabajo en educación media. Santiago, Chile: Ministerio de Educación, Programa MECE.
Elacqua, G. (2012). Education: Chile’s students demand reform. Americas Quarterly, Winter.
Feria, J.V. (2010). Judgments of self-perceived academic competence and their differential impact on students’ achievement motivation, learning approach and academic performance. European Journal for the Psychology of Education, 25(4), 519–536.
Herreid, C.F., & Schiller, N.A. (2013). Case studies and the flipped classroom. Journal of College Science Teaching, 42(5), 62–65.
IALS-OECD. (2000). Literacy in the Information Age. Final Report of the International Adult Literacy Survey. Ottawa: OECD, Statistics Canada.
Koontz, T.M., & Plank, K.M. (2011). Can reading questions foster active learning? A study of six college courses. Journal on Excellence in College Teaching, 22(3), 23–46.
Larraín, C., & Zurita, S. (2008). The new student loan system in Chile´s higher education. Higher Education, 55(6), 683–702.
Leamnson, R. (1999). Thinking about Teaching and Learning: Developing Habits of Learning with First Year College and University Students. Baltimore, MD: Stylus Publishing.
Mazur, E. (2012). Farewell, lecture? Science, 323(5910), 50–51.
Mineduc. (2012). Análisis y Recomendaciones para el Sistema de Financiamiento Estudiantil. Santiago, Chile: Ministerio de Educación y Cultura, Comisión de Financiamiento Estudiantil.
Mineduc. (2011). Resultados Nacionales SIMCE 2010. Santiago, Chile: Ministerio de Educación y Cultura, SIMCE.
NCSL. (2012). Hot Topics in Higher Education: Reforming Remedial Education. Washington, DC, http://www.ncsl.org/research/education/improving-college-completion-reforming-remedial.aspx: National Conference of State Legislatures.
Nilson, L.B. (2010). Getting students to do the readings. In L.B. Nilson, Teaching At Its Best: A Research-Based Resource for College Instructors (3rd Edition) (pp. 211–222). San Francisco: Jossey-Bass.
OECD. (2012). Quality Assurance in Higher Education in Chile: Reviews of National Policies for Education. Paris: OECD, http://www.oecd.org/chile/Quality%20Assurance%20in%20Higher%20Education%20in%20Chile%20-%20Reviews%20of%20National%20Policies%20for%20Education.pdf.
Peter D. Hart Research Associates. (2005). Rising to the Challenge: Are High School Graduates Prepared for College and Work? Washington, DC: Achieve.
Waddock, S.A., & Post, J.E. (1991). Social entrepreneurs and catalytic change. Public Administration Review, 51(5), 393–401.
Weimer, M. (2011). A comparison of two strategies for getting students to do the reading. The Teaching Professor, 25(1), 4.
Wilcox, K.C., & Angelis, J.L. (2011). Best Practices from High-Performing High Schools: How Successful Schools Help Students Stay in School and Thrive. New York: Teachers College Press.