Recognize and Celebrate Successes

Not every piece of evidence demands improvement. When your college achieves its goals and meets its targets, celebrate these successes! (I have joked that accreditors should require that the “use” of such evidence be a mandatory pizza party.) Then focus on sharing these successes more effectively with your stakeholders.

But if your college’s culture is one of ongoing betterment, questions may be considered, even when results look great. Should your standards be raised? Can you achieve your goals or assess your results more efficiently? Should you turn your attention to enhancing relevance to your audiences? The answers may be no, not now, but the questions are still considered.

Use Evidence to Advance Quality and Effectiveness

The most common uses of evidence that I see are minor tweaks. Faculty who are dissatisfied with their students’ skills in citing research literature, for example, may agree to spend more time explaining this in their classes, to provide more examples and homework, and perhaps to address this skill in more courses in their program. This kind of fine-tuning is low-cost and requires the consensus of a relatively small number of faculty, but it does not lead to substantive advancements in your college’s overall quality and effectiveness.

Using evidence to make broader or more substantive changes is harder, but it can be done. The Education Trust has shared the stories of eight U.S. universities that used analyses of systematic evidence to implement changes that improved their student success rates dramatically (Yeado, Haycock, Johnstone, & Chaplot, 2014). Here are additional examples:

• After examining the retention rates of first-year, second-year, and transfer students, as well as those of students of color, along with results from the National Survey of Student Engagement (NSSE) (http://nsse.iub.edu) and the Faculty Survey of Student Engagement (FSSE) (http://fsse.iub.edu), the University of Wisconsin–Oshkosh redesigned its general education curriculum to foster earlier and more pervasive use of strategies that promote student learning and success (Kuh & O’Donnell, 2013).
• After assessing student learning in its writing-intensive, capstone, and service-learning courses, Daemen College hired a writing coordinator and writing-in-the-disciplines specialist, added an information literacy component to its first-year writing course, increased the proportion of first-year writing courses taught by full-time faculty from 35 to 90 percent, and offered workshops for faculty teaching writing-intensive courses (Morace & Hibschweiler, n.d.).
• After reviewing a variety of measures on its academic advisement programs, including NSSE results, student/advisor ratios, and a program review, the University of Tennessee–Knoxville increased the number of full-time advisors, restructured orientation advising for new first-time students, and implemented a new advising policy targeting at-risk students (NSSE, 2012).

Use Evidence to Deploy Resources Prudently

There is a common theme among these examples: meaningful advancements in quality and effectiveness require resource investments. One of the characteristics of good stewardship is that evidence is used to inform resource deployment decisions, and you can see that in these examples. An important way to link evidence to resource decisions is to require “business plans” for any initiative that is proposed or under review, as discussed in Chapter 20. Here are some other suggestions for linking evidence to resource decisions.

Include student learning evidence in the mix of evidence, as shown in the above examples. If you find this difficult, the problem may be that:

• Your college is not yet assessing truly important learning outcomes.
• Your college has not yet clearly defined what successful learning outcomes look like (see Chapter 15).
• Faculty do not see value in identifying areas for improvement in teaching and learning (see Chapter 18).

Use external as well as internal evidence. Enrollment management decisions may be informed more by market analysis than by evidence of student learning, for example.

Prepare for a time lag. I have joked that colleges need a one-month hiatus between the end of one fiscal year and the start of the next, to examine evidence from the previous year and use it to set or refine goals, plans, and resources for the next. The reality is that, at many colleges, next year’s annual budget must be put in place before evidence from the current year is in hand. For colleges whose fiscal year runs from July 1 to June 30, for example, budgets for the following year are often put together in the spring, whereas evidence of student learning is often collected at the end of the spring semester, and other evidence, such as final financial figures, may not be available until after the current fiscal year ends.

There are a few ways to deal with this, although none is entirely satisfactory. One is to acknowledge a lag, allowing decision-makers time to “make meaning of and to reflect on assessment results” (Baker, Jankowski, Provezis, & Kinzie, 2012, p. 7). Student learning evidence collected at the end of the 2017–2018 academic year, for example, might be analyzed after the 2018–2019 academic year begins and used to inform the 2019–2020 budget. Another option is to build into each annual budget a reserve for funding any pressing needs identified through evidence. If you find that your students’ writing skills are miserable, for example, you will want to fund immediate measures to improve them, rather than wait for the following fiscal year.

Use Evidence to Refine Goals and Targets

Sometimes, what needs attention is not your programs, services, and activities but your goals and targets. Suppose, for example, that your college aimed to raise $5 million this year toward a $20 million five-year target, but it brought in only $3 million. Yes, perhaps fundraising efforts could be improved, but perhaps the $20 million target was overly ambitious, especially if you raised only $1 million last year (in which case $3 million this year would be a terrific accomplishment). On the other hand, if you raised $8 million this year, you might want to consider stretching your $20 million five-year target to something more ambitious.

Before you cut back on what seems to be an overly ambitious goal, consider this: often the reason for falling short is not that your goal is too ambitious but that you have too many goals, causing your college and its faculty and staff to lose focus and diffuse energies. What can you scale back on or put on hold, freeing up time and resources to focus more on your critical goals?

This applies to goals for student learning as well. If your students are not writing as well as you would like, for example, look at all your other goals for them. Are you expecting them to learn too many things? Can time spent on a less-critical goal be scaled back, freeing faculty and student time to work on writing?

Use Evidence Fairly, Ethically, and Responsibly

A few principles (Suskie, 2009) are worth highlighting here.

Evidence should not make decisions for you; it should only advise you as you use your professional judgment to make suitable decisions.

Do not base a major decision on only one piece of evidence, such as the results of a single assessment of student learning.

Do not use evidence punitively. Past use of evidence—good or bad—affects people’s willingness to participate in gathering and using evidence now (Petrides & Nodine, 2005). Do not react to disappointing evidence by immediately eliminating a program or denying promotion or tenure to the faculty involved. Instead, provide an opportunity to address the problems identified through the evidence.

Be careful how you recognize and honor evidence of success. It is tempting to reward successful results with, say, merit pay increases, but this can backfire, tempting individuals to distort, if not outright falsify, evidence in order to look as good as possible. This kind of practice can also force faculty and staff to compete against one another for a limited pool of pay increase funds, which can destroy the culture of collaboration that is an essential component of quality. Yes, evidence of success should be celebrated, but so should efforts to understand and improve quality, especially collaborative efforts, even if those efforts initially fail or yield disappointing results.