APPENDIX B

Matching Objectives with Evaluation

Instructions

Take this simple test to check your knowledge about the different levels of objectives. Indicate the level of evaluation corresponding to each objective in the list. Each objective begins with the statement, “After completing this program or project, participants should do the following.” So, have a go, and see how well you know the levels. We’ll review responses afterward. Use the corresponding numbers below for your answers:

1. Reaction
2. Learning
3. Application
4. Impact
5. Return-on-Investment

Table B.1: Quiz: Identify the Level of Objective
Objective	Level
After completing this program or project, participants should:
1. Improve work group productivity by 20 percent in six months.
2. Initiate at least three cost-reduction projects in 15 days.
3. Achieve an average cost reduction of $20,000 per project in six months.
4. Increase the use of counseling skills in 90 percent of situations where work habits are unacceptable.
5. Achieve a 2:1 benefit-cost ratio (bcr) one year after the new gain-sharing program is implemented.
6. Identify the five critical steps an employee must take to access and use the employee assistance plan.
7. Increase the external customer satisfaction index by 25 percent in three months.
8. Address customer complaints with a five-step process in 95 percent of complaint situations.
9. Not have an adverse reaction to the absenteeism policy within 90 days of implementation.
10. Achieve a leadership-simulation score average of 75 out of a possible 100 in three weeks.
11. Conduct a performance review meeting with direct reports within two weeks to establish performance improvement goals.
12. Provide a 4 out of 5 rating on the appropriateness of a new ethics policy.
13. Decrease the time to recruit engineers from 35 days to 20 days in six months.
14. Complete all the steps on the action plans in three months.
15. Perceive the flextime schedule to be useful (70 percent “yes”).
16. Be involved in a career enhancement program at a rate of 15 percent of nonmanagerial staff.
17. Decrease the amount of time to complete a project.
18. Achieve a post-test score increase of 30 percent over pre-test.
19. Use new software daily as reflected in an 80 percent score of an unscheduled audit of use in three weeks.
20. Submit ideas or suggestions for improvement in the first year (10 percent target).

Responses to Quiz

1. Improve work group productivity by 20 percent in six months (4). This is an impact objective, clearly showing the consequences of doing something different in the work unit. That different approach, process, or application of new skills or knowledge has improved productivity.
2. Initiate at least three cost-reduction projects in 15 days (3). This objective represents action. Participants have initiated projects, but have not completed them. The projects will take several months to complete, and the fact that they are not yet complete means this is not an impact objective.
3. Achieve an average cost reduction of $20,000 per project in six months (4). This is an impact objective and is connected to the previous objective. It is the consequence of completing the project. The previous objective focused on getting participants to take action, while this one speaks to the consequences of that action.
4. Increase the use of counseling skills in 90 percent of situations where work habits are unacceptable (3). This use of skills represents application. Participants are applying what was learned in a basic supervisory skill-building program.
5. Achieve a 2:1 benefit-cost ratio (BCR) one year after the new gain-sharing program is implemented (5). This is ROI. The BCR is one way to show benefits versus costs. Incidentally, a 2:1 BCR is a 100 percent ROI, as detailed in chapter 8.
6. Identify the five critical steps an employee must take to access and use the employee assistance plan (2). This learning objective indicates what employees must learn or know to make the plan work for them.
7. Increase the external customer satisfaction index by 25 percent in three months (4). This is an impact objective—a consequence of doing something different with customers that results in a higher degree of satisfaction.
8. Address customer complaints with a five-step process in 95 percent of complaint situations (3). This application objective indicates use of a particular process. The objective involves a criterion as well.
9. Not have an adverse reaction to the absenteeism policy within 90 days of implementation (1). In this case, it’s important for the reaction to be at least neutral, if not positive. The key is to avoid a negative reaction when changing current policies or practices. Employees might perceive a loss of control or a loss of freedom they previously had. The focus of this reaction objective is keeping attitudes in check.
10. Achieve a leadership-simulation score average of 75 out of a possible 100 in three weeks (2). This is a learning objective. Simulation replicates a process, task, or procedure and measures the extent to which participants know how to do something. Consequently, a simulation score measures knowledge or skills gained.
11. Conduct a performance review meeting with direct reports within two weeks to establish performance improvement goals (3). This application objective is part of a performance management system introduced through a formal session where supervisors and managers learned how to set performance improvement goals. These supervisors were required to conduct the review meeting with direct reports, with action items and goals posted online in the performance management system.
12. Provide a 4 out of 5 rating on the appropriateness of a new ethics policy (1). This objective focuses on reaction to the policy. The concern is that employees should perceive that the actions taken to address ethics are appropriate.
13. Decrease the time to recruit engineers from 35 days to 20 days in six months (4). This is an impact objective, a consequence of changing the recruiting process. Essentially, we are trying something different (Level 3) and, consequently, the recruiting time decreases.
14. Complete all the steps on the action plans in three months (3). This is an application objective, as the steps must be completed. Sometimes action plans do contain impact consequences and can actually be involved at both Level 3 and 4 objectives; but, not knowing that, the most appropriate response would be 3, application.
15. Perceive the flextime schedule to be useful (70 percent “yes”) (1). This is a reaction objective, as individuals react to the new flex system. When these types of systems are implemented, it is important for participants to perceive the systems as useful to them personally.
16. Be involved in a career enhancement program at a rate of 15 percent of nonmanagerial staff (3). This is a voluntary career enhancement program where participants have a choice to be involved. If they want to be involved, they must complete an assessment, plan a career development program, and work on the process until it is completed.
17. Decrease the amount of time to complete a project (4). This impact objective reflects a consequence of applying a different process in the project management scenario. It could involve implementing software, conducting project management training, or establishing a project management office. One of the consequences of the new process after it is implemented and applied (Level 3) is that project management time decreases (Level 4).
18. Achieve a post-test score increase of 30 percent over pre-test (2). This is a bit tricky, but normally it would represent a learning objective. Pre- and post-testing usually measure learning; however, pre/post data can be developed at Levels 1, 2, 3, or 4. At Level 3, for instance, we could have pre/post measures on changes in skills. For example, a classic way to measure leadership development skills is a 360-degree feedback process, where behavioral data are taken from a variety of sources and integrated for a composite rating. This process could be conducted prior to a leadership development program as a pre-assessment and conducted three months after the program for a post-assessment. The pre/post difference shows a change in behavior; hence, it reflects application.
19. Use new software daily as reflected in an 80 percent score of an unscheduled audit of use in three weeks (3). This objective is an application objective dealing with actual use of the software. In this situation, an appointed person will come to the workplace and audit the use of the software by examining routine work and developing user profiles. This is an assessment of application of the software.
20. Submit ideas or suggestions for improvement in the first year (10 percent target) (3). During the launch of this employee suggestion program, employees are introduced to the program (measure reaction), they learn how to submit suggestions (measure learning), and then they submit suggestions (application). If the suggestion is accepted, any cost savings are realized, and an impact is realized. The goal is to have 10 percent of employees submit suggestions.

We hope you find that you have mastered the ability to create objectives. They are powerful and necessary and ensure programs and projects are positioned to achieve results!