What Is a Quality Program?

I use two frameworks for defining a quality program, service, or operation and thereby organizing a program review—pick whichever one you like better. The one that I used for many years has three fundamental criteria for reviewing academic programs (Shirley & Volkwein, 1978):

1. Quality: the quality of
   • Inputs: faculty credentials, student qualifications, facilities, library holdings, and so on
   • Processes: curriculum design, teaching methods, academic advisement, and so on
   • Outcomes: student learning outcomes, research and community impact, and so on

   Of these, the most important are outcomes, specifically student learning outcomes.

2. Need and demand: the number of potential students interested in the program and demand for graduates by employers, graduate programs, and the like
3. Cost and cost-effectiveness: how much the program costs and how efficiently and effectively it uses its resources

The second framework reorganizes these criteria into the five cultures of quality.

A culture of relevance. A quality program provides honest, balanced information and treats students and other stakeholders fairly and consistently. Its resources—such as facilities, technology infrastructure, library collection, administrative oversight, and faculty and staff with appropriate experience and expertise—are sufficient to ensure the program’s health and well-being, and it deploys those resources effectively, prudently, and efficiently. It meets stakeholder needs, especially those of its students, and there is enough demand and need for the program by students, employers, and others, now and continuing into the future, to ensure its viability. It is also effective in serving the public good, achieving its purpose and goals, and demonstrating that it is meeting its responsibilities.

A culture of community. A quality program’s culture is one of respect, communication, collaboration, growth and development, shared collegial governance, and documentation.

A culture of focus and aspiration. A quality program has a clear sense of purpose that supports the college’s mission. It has clear goals, along with programs and services designed to meet the needs of its students and other key stakeholders and to help the college achieve its overall purpose and goals.

A culture of evidence. A quality program regularly collects useful, good-quality evidence of its effectiveness, especially evidence of what its students have learned and how successful they are in later pursuits. It has clear, justifiable definitions of what successful students look like.

A culture of betterment. A quality program uses systematic evidence to advance its quality and effectiveness. Indeed, some accreditors now require that evidence from program reviews be incorporated in college planning and resource allocation decisions.

Specialized Accreditation as a Form of Program Review

Specialized accreditation can be the epitome of program review. Because specialized accreditation reviews are overseen by external organizations using consistent criteria and trained reviewers, specialized accreditation has a credibility that internally driven program reviews cannot achieve.

Specialized accreditors vary greatly in their purposes and requirements. Some specialized accreditations, especially those in medical disciplines, are mandatory if graduates are to be eligible for licensure or employment, while others are completely optional. Some specialized accreditors are Title IV gatekeepers (Chapter 2) recognized by the U.S. Department of Education (ED), while dozens more are recognized by the Council on Higher Education Accreditation (CHEA). Some are recognized by both bodies, and some are not recognized by either.

Specialized accreditors also vary considerably in their attention to the five cultures of quality. While all recognized accreditors now expect a culture of evidence, for example, some specialized accreditors place more emphasis on this than others do. Some still place considerable weight on inputs into the learning process, such as faculty credentials and facilities, even though the connection of those inputs to quality and effectiveness is not well established, as discussed in Chapter 4. As a result, the value of specialized accreditation in ensuring quality is inconsistent.

Unless specialized accreditation is required for licensure, for Title IV funds, or by your regional accreditor, consider the costs and benefits. Maintaining some specialized accreditations can cost tens of thousands of dollars or more every year. What are the benefits, and how do they weigh against those costs? What systematic evidence do you have that your key stakeholders demand specialized accreditation? How much does specialized accreditation affect enrollment and revenue? Are the benefits worth the investment in specialized accreditation, or could those resources be deployed in other ways that might more effectively advance the program’s quality and effectiveness? Consider developing business plans (discussed later in this chapter) for specialized accreditation that examines these questions.

Integrate multiple accreditations. If your college has more than one accreditation (for example, regional accreditation and several specialized accreditations), look for ways to integrate your accreditation work. While requirements understandably vary across accreditors (specialized accreditors, for example, require evidence of competencies specific to their professions), most share requirements related to the five dimensions of quality.

Many colleges prepare charts or “crosswalks” that align regional and specialized accreditation requirements; check with peer colleges that have recently undergone review to see whether they have one to share, or prepare your own. In many cases, the material prepared for a specialized accreditation review can be used for a regional accreditation review; it may simply need some updating and perhaps expansion.

View Program Reviews as Cousins of Grant Proposals and Business Plans

Effective program reviews have much in common with grant proposals and the business plans that entrepreneurs take to potential investors when seeking start-up funding. All three make an evidence-informed case for investment through systematic evidence and by showing that the nuts and bolts for getting things done have been thought through. Proposals for new initiatives at your college, such as adding a new program, introducing a new curricular requirement, offering a program online, or seeking specialized accreditation, should have a similar design and intent as well. Table 20.1 lists questions that might be addressed in program reviews and proposals for new initiatives.

The questions in Table 20.1 are not easy to answer! They suggest a team approach to program review, with information and evidence developed not just by the faculty and administrators in the program, but by others as well, including your college’s institutional research, budget, and marketing offices.

Ensure Program Review Integrity and Value

In good times, program reviews can become undisciplined wish lists, concluding that nothing is wrong that an infusion of resources would not cure. In difficult times, when program cutbacks and closures may be contemplated, they can become defenses of the status quo, full of assurances that everything is the best possible quality. And, in both good and bad times, some program reviews simply end up on a shelf. None of these outcomes helps a college or program advance its quality agenda.

Require a usable program review . . . and use it. Effective program reviews are used by college leaders as well as members of the program to inform decisions regarding the program’s vision, plans, and support. In order to accomplish this:

• Conclude program reviews with proposed action steps and timelines to advance the program’s quality and effectiveness that flow from and are supported by the systematic evidence provided in the program review.
• Require the program chair/director, dean, and vice president to review and jointly sign off on the completed program review, endorsing the proposed action steps and agreeing to provide needed support. These endorsements keep the proposed actions from turning into an unrealistic wish list that will never go anywhere.
• Make a college-level commitment that any final decisions to cut back or terminate a program will be preceded, if at all possible, by reasonable opportunities and resources to address shortcomings.
• Ask the program for annual updates on its progress in implementing the action steps, to ensure that follow-through is indeed happening.

Use an external reviewer to help ensure the program review’s integrity, quality, and value. Some disciplinary associations offer review processes or clearinghouses of screened reviewers. The American Chemical Society (ACS), for example, offers an approval program for bachelor’s degree programs, and the American Sociological Association (ASA) offers a Department Resource Group of consultants who can assist with program review.

But how is an external reviewer chosen when these kinds of resources are not available? Program faculty might argue that anyone outside their discipline, such as a dean, does not understand their discipline well enough to select an appropriate reviewer. But if the faculty select their own reviewer, they might be tempted to choose a colleague who will simply reinforce their own conclusions and not bring an independent perspective. Then there are truly one-of-a-kind programs, often interdisciplinary, that have no peers and, thus, few potential reviewers. Options for addressing all these challenges include the following:

• Ask program faculty to identify two or three potential reviewers and let the dean or provost make the final selection.
• Insist that prospective reviewers have credentials demonstrating their broad experience engaging with an array of comparable programs.
• Provide the reviewer with clear expectations for objectivity and evidence-supported conclusions. These expectations can be conveyed through a meeting with the dean, written guidelines, or a template for the reviewer’s report.