21

Competency-Based Design and Evaluation

For the entire history of training, the predominate approach to mastery in skill-based learning has been “watch, learn, practice, feedback, mastery.” The reason this approach has lasted so long is that it works for the purpose for which it was designed, essentially the immersion and training of new skilled tradepeople in psychomotor or tactile-based skills like masonry, pottery, carving, millwrighting, cooking, baking, and farming. Before World War II, it was not unusual for most workers to be cast in an occupation very early in life, either from family history or necessity, and then stay in that occupation all of their working lives. This meant that training happened as a natural evolution from apprentice to journeyman, with a few even rising to the title of master craftsman. During this time of training, the worker was slowly immersed in the skills necessary for the trade and evaluated by the more senior members of the craft. There was little in the way of formality in training because each craft had somewhat different traditions in the way of teaching new workers.

As the trades became more technical and complicated, they also encompassed many new areas of expertise, including the medical, legal, and military occupations. Training also became more formal, and it was a natural progression that mastery of these occupations was now accompanied by a formal evaluation process and the issuance of some form of licensure. In the building trades, a worker advanced from being an apprentice to being a licensed journeyman with a credential issued by the trade or a governmental authority. This formalization of trades also eventually extended to titles like accountant, paramedic, firefighter, police officer, and countless others. The licensure was now the currency required to get and hold an occupation, and these were almost always limited to individuals who went step by step through a formal training period that combined traditional classroom and on-the-job training for a set period of time before any mastery would or could be documented. There was no serious thought given to allowing the evaluation of mastery to take place without an associated and related formal learning period.

The traditional foundation of academic credit was the first column to fall when the first awards of experiential credit began to appear in community colleges after WWII. Veterans needed a fast track to a college degree, and because many training courses and work experiences from the war were somewhat or even totally duplicated in a formal setting, why not allow students to document this learning for possible licensure or even for college credits? The same logic applied to the noncredit world of training, but would take a slightly different and longer road to utilization.

How Competency-Based Evaluation Evolved

The first generation of this new system provided a way to fast track college credits to some level, but never allowed for these experiential credits to apply toward a specific major, because that was deemed the province of the issuing college or university. Regional academic accreditation still requires that students take all or most of their courses in a major with the issuing school. So students could shorten their time in college, but they couldn’t really take what they learned through previous informal learning and use it toward a specific credentialing for a particular skill set or profession without retaking courses that contained content they had already mastered. Enter competency-based evaluation (CBE).

CBE is essentially a system for allowing a formal and reliable way to evaluate previous learning from any source and turn that into some form of recognized currency with the same standing and level of acceptance as existing formal learning in specific skill and content areas. The early adapter to this new approach to learning and credentialing is the skilled-trades system of apprenticeship, which normally requires years of classroom and field experience before even being considered as a credentialed journeyman in a trade. CBE allows a path for informal or nondocumented learning to be evaluated for mastery toward a skilled-trades credential.

Competency-based education, training, and evaluation are each worth a book on their own. This is but a small subset of the competency-based suite of tools for an instructional designer. The most important element of this to remember is that there is an alternative to traditional approaches to earning licensure, certification, or a college degree. As the legacy world of “time in class” or “contact hours” starts to crumble, the new normal is finding credible, observable, and measurable ways to move learners through the system to productive careers based on the principle of experiential learning.

How CBE Works

The premise of competency-based evaluation is that a learner’s competency is not dependent on where or when it was obtained and there are no requirements for a formal classroom-training component. The only thing that matters, and should therefore be evaluated, is mastery of a specific learning objective. With this in mind, competency-based evaluation starts at the exact same place for a designer as building a course from scratch: determining and writing behavioral objectives.

For example, in the world of apprenticeship, a course might consist of training in how to complete a specific skill, such as laying brick to the line in a masonry apprenticeship. If students are in an apprenticeship course, they would be required to attend classroom lectures and demonstrations and then practice the skill set under supervision of an instructor. In the world of CBE, someone would only be required to pass a written test on the content and then perform the skill while being evaluated by one or more instructors. Instead of several days or weeks in the traditional classroom-based process, someone might be able to show mastery with CBE in several hours. Imagine the time savings over a four- or five-year apprenticeship for someone with experiential skills but no formal credential such as a journeyman standing.

This same approach works for any content area. First, determine the objectives, then provide a consistent evaluation process that ensures that mastery can be determined to the degree required and in the domain that is equivalent for determining traditional mastery. For example, if a learner is only required to complete a written evaluation at a specific degree or performance, then a CBE would probably require that a prospective learner complete the same written evaluation to the same degree of mastery.

In the case of a demonstrated skill like performing CPR, the CBE student would be required to perform CPR to the same standards expected of a course participant for credentialing while also passing the written portion of the credentialing process. The requirement is that CBE participants meet or exceed the original level of mastery in an objective domain consistent with the existing course.

How to Design CBE Evaluations

The process of designing CBE evaluations can be rather simple or as time-consuming as designing the basics of a course, including objectives and evaluation tasks. In an established course that already contains acceptable objectives and evaluation tasks, the existing evaluations will need to be migrated into a format that allows for the necessary mastery checks. In the case of a written exam, a proctored exam process will suffice. Simply implement the test and score the results.

In skill-based evaluations that require observation and scoring of a task or skill process, a designer will be required to develop a rubric that scores each objective. The scoring is best accomplished by the observations performed by two or more SMEs in the skill area. Each completes the rubric independently, and a score is determined by averaging the SMEs’ scores. In some situations, one SME entering a failing score for an individual stops the process and no average is necessary. Other options are to average in the failing score(s) and see if the final score passes the mastery threshold. Each SME is required to score and initial his rubric for recordkeeping purposes (Table 21-1).

Table 21-1. Example of Recordkeeping Rubric

Scoring in rubrics can be pass-fail; a score of A, B, C, and so forth; or even a numeric score from 1 to 100 or 1 to 5, depending on the needs of each organization and course. The process may be designed so that an individual can retake a section of an evaluation again after a set period of time, such as a week, a month, or whatever seems appropriate. There are also programs that do not allow second chances on the CBE exam process in order to discourage repeat test takers who might get lucky and pass at some point.

Because many CBE programs are sponsored by an organization that also awards a credential, the standards and process of scoring each section of a CBE will be at the organization’s discretion and may ultimately be the result of a governing body’s preferences to ensure consistency and rigor. Each of these situations is unique; do not be surprised if this is more work than actually designing the process of CBE.

The role of most CBE programs is not to eliminate the requirement for any formal training; it is primarily to reduce the amount of time spent in a program for those with abundant life experience in a specific skill set. This rests on the premise that there is no justification for learners taking courses in content areas that they have already mastered. CBE can cover an entire program’s content, but it is usually only a percentage of the mastery required for credentialing, and other content would need to be taken and mastery proved in a more traditional course-based context. Of course, there will be exceptions to this for individuals with many years of undocumented mastery.

How to Weigh Financials

Using a CBE approach raises financial questions in many environments. Schools and training programs are expensive to initiate and maintain. This is especially true if there is licensure and detailed recordkeeping associated with an occupation. Without some additional forms of significant monetary investment by other associated groups or individuals, a program is going to have to find a way to supplement traditional tuition and fees to reach credentialing using CBE. This can come from various approaches to making CBE at least a break-even proposition.

One approach is to have a fee for each CBE mastery evaluation regardless of outcomes. Another is to charge administrative fees and application fees as well as a charge for actually implementing the evaluation. Other approaches include a blanket charge for the credential regardless of the combination of traditional coursework and CBE mastery events.

Think about how this might work; be sure to include the financial elements of this process and outcomes to avoid creating a bottomless financial burden when trying to streamline the process of credentialing a learner. With some time and effort, a model can be advanced that will work in making a program at least solvent, given the options for learners and the organization.

In Conclusion

In the new age of efficiency in apprenticeship programs and in training in general, the practice of creating competency-based evaluation programs has become the new design protocol in content areas that are predominantly skills based. The use of evaluation schemes that allow for the rapid review of mastery enables a greater number of qualified learners to participate in courses only in those areas in which they have not yet achieved mastery. This allows for potentially quicker migration from entry to exit in a program without logging all of the usual associated classroom and practice hours required for certification. For learners with a large degree of nondocumented skills experience, this is an opportunity to quickly reach certification while still maintaining the quality and skills mastery required by an organization or licensure process.

Discussion Questions

1.  You are asked by the owner of a large electrical contractor to find out if several of their nonlicensed electricians can test out of the examination for being licensed. Where do you start?

2.  Do you think it is fair that learners with more experience can essentially test through some or all of a credential by just being evaluated?

3.  Is there any downside to using CBE?