Evaluation is essential to learning. Our society emphasizes right and wrong, good and bad, worthy and unworthy, winning and losing. Because of this, we tend to think that evaluation is about placing blame. Not true: the true significance of evaluation in organizations is that it provides information that will result in improved performance.
Evaluation means looking for indicators of progress toward and achievement of strategic goals, and understanding what it is about the learning process that is helping employees improve and sustain their performance over time. You must identify aspects of the organization that are barriers to learning and high performance, because this is how organizations change and improve themselves.
All members of the organization need to be continually asking:
• What are we doing?
• How well are we doing it?
• How can it be improved?
These reflective questions should be applied to all of the activities, processes, and systems within the organization.
Involve representatives of key stakeholder groups (employees, customers, clients, suppliers, business partners, and others who have a vested interest in the performance of the organization), because they are more likely to use the results for their own learning if they are involved from the outset. Collaborate with them to decide what should be evaluated, what questions should be asked, and how the information will be used once it is collected.
Evaluation of organizational learning measures the interaction of each element of the learning process. We want to know what elements of the system affect learning and how this learning results in enhanced organizational capacity to be successful.
Evaluation measures do not have to be rigorous, in fact, the simpler the better. The point is to collect data (numbers, descriptions, stories) that help the stakeholders understand the progress that employees and the organization are making toward business goals. These stakeholders need to know what is working well and what should be changed; if, for example, the managers of a manufacturing line are presented with evidence that their behavior is a barrier to the performance improvement of employees on that line and they don’t allow trainees to apply their new learning, they might decide to change their behavior and support performance improvement. Until they have this feedback, they are unlikely to change.
Once you decide what needs to be evaluated and why, and how you will use the information in the organization, then you are ready to select the appropriate method of measurement. For example, if you are trying to develop a high performance team, you might want to:
• Survey team members about their attitudes toward teamwork
• Interview team members about the effectiveness of communication among team members
• Observe the team members in action to assess how they work together
• Survey the team’s internal and external customers about their perceptions of the team’s effectiveness
• Examine team products for indicators of quality
Managers throughout the organization are important partners in the application of these methods and in the utilization of the information. The results of these measures should be reported to other managers who can use the information for process-improvement purposes. By bringing other managers into the performance-improvement process, you build a sense of organization-wide ownership and commitment to performance-improvement. The responsibility for learning and change shifts to the people in the organization who have significant influence on the organization’s success.
Measure the outcomes of the learning process. Any learning process has short-term outcomes that show progress towards strategic goals and long-term outcomes that show the impact of the learning on the organization. These outcomes include:
• New employee skills, knowledge, and attitudes
• Achievement of job behavior objectives
• Achievement of job output objectives
• Achievement of business process objectives
• Achievement of department/unit goals
• Achievement of business strategic goals
The following suggestions will make the evaluation of organizational learning useful to you and to your organization:
• Involve internal customers in deciding what to measure and how to measure it.
Ask them what they want to know and why. Ask them for their thoughts about the data-collection methods that you are recommending. Ask them to help pilot these methods to see if they will produce the information that you need. Often a workshop format that allows these stakeholders to work together as a group is useful. This process of answering measurement questions with input from key stakeholders can be more valuable to developing a high performance organization than the actual data that are collected.
• Choose the method of measurement only after deciding what to measure.
The tendency is to use a paper-and-pencil survey to measure just about everything that has to do with employee perceptions, but we have many different methods available to us: online surveys, focus groups, structured observation, logs and journals, and learning histories. The appropriateness of each of these methods depends on what kind of data are needed, the sources of that data, the circumstances for collecting the data, and how the data will be used. The method should make the thinking behind comments and behaviors explicit. Users of the evaluation want to know how and know-why.
• Report data that are credible and useful to the customer.
Line managers might accept the accuracy of employee interviews and focus groups, but senior executives might only listen to production and financial data. Know your audience so that you can collect measurement data and report findings that the key stakeholders will find convincing.
• Report findings so that the customer can hear them.
This has to do with how the information is reported. You will want all of the various customers to understand your findings and be able to act on the implications. Keep it simple, relate it to the goals that are important to the particular audience, and recommend what should be done about the results. Do not just report numbers; tell the story about what the numbers mean to the organization.
• Measure the process as well as the outcome.
Continuous improvement is achieved by regular assessment of where people are in the process of learning. Adjustments to the process can be made, especially as you find out more about employee needs and the organization becomes clearer about its goals.
• Provide just-in-time and just-enough information.
Give employees the information they need, when and where they need it. Performance is maximized when people are not overwhelmed with new information, when they can relate new information to their work, and when they can apply the learning to a problem on the job immediately.
• Measure to improve the process, not to blame or punish.
Our tendency is to feel threatened by anything that might reveal our personal incompetency. When we feel threatened, we become less cooperative and less willing to improve performance. Do everything that you can to assure participants that the measures are not being used to make judgments about individuals. Follow through on this promise. Use the data only to make changes in the learning process and to plan for additional activities that will make a difference in performance.
• Evaluation is a powerful tool for organizational learning.
Used to evaluate the link between learning and enhanced capacity to achieve strategic goals, it can help you make your organization more successful. But this requires moving beyond assessing learners’ immediate reactions to events and examining the entire process that facilitates individual, small-group, and whole-organization learning. The payoff comes when you use this information to improve the learning process and make sure the process has high impact on building the capacity of the organization.