My coach’s office desk was full of writing materials. The floor was covered with new training materials. Trophies and pictures of training camps and tournaments covered the walls. His office was a collection of many years living in the world of a top-level sport.
“Don’t wait for something you already can measure.”
—My coach, Koos Henneveld
My personal plan was scheduled on an old whiteboard. At one point, it showed 24 months until July 30, 1992. My coach had scheduled all of my measurements—speed, condition, strength, weight, fat percentage, knowledge of competitors—at certain milestones. My coach and I would discuss how I had changed my life to be an Olympic athlete: Did I feel comfortable with new skills? How well was I doing in my self-reflection? Had I grown as champion? Without using an assessment tool, he collected data to see when his interventions would be needed.
He always told the staff, “Don’t wait for something you already can measure.” He meant that every day counts. Why wait until the next tournament to figure out that my condition was not good enough? I agree. Why wait until the end of your reinforcement program to see whether it worked? Measurements are important, but the actions you take in response to these measurements will change the world.
As an Olympic athlete, I liked most of these measurements. How well did I grow? What skills had I mastered? How was I doing on my way toward my dream on July 30?
Of course, I hated some measurements. Although I knew they were good to have, I hated measuring my weight and fat percentages. They were a reflection on the way I lived the week before. I must admit, some weeks I did not live as an Olympic athlete, and my coach knew it the next week.
One of the things my coach always explained to his staff was, “If you measure at the perfect moment, each measurement is valuable. If you measure at the wrong moment, the outcome is worthless and your athlete is frustrated. So plan your measurements very carefully.” And that is also what you must consider for your reinforcement program.
A good measurement at the perfect moment is valuable. A good measurement at the wrong moment is worthless.
You have already created a strong foundation based on one or more reinforcement objectives. You used the first principle to focus on the 3 phases, and with the second principle you guaranteed the continuation of the change to close all five gaps. The last principle to build a strong foundation for your reinforcement program is to measure the behavior change (see Figure 8.1). When you use this principle, your reinforcement program will provide you continuous insights as to the learners’ progress and behavioral change.
This third principle, “Create measurable behavior change,” is a guide in how to use different types of measurements. You don’t want to take a measurement at the start of your program and another at the end; if you do, you may be rather surprised by the result. Furthermore, if you take measurements only at the start and end of the program, you won’t have an opportunity to influence the result. With a solid measurement plan, you collect lots of valuable data, gain insights, and see the possibilities to use measurement throughout the program.
THE MEASUREMENT PLAN
When you put together your plan, start by measuring the current behavior and deciding on the needed change. Then you can decide how to measure the conditions along the way. Because behavior change is based on a reinforcement objective, create a separate measurement plan for each objective. As you organize the final training, you can merge your measurement plans.
In your plan, list the needed and observable behavior and the desired timing. Based on the reinforcement objective and the verb you selected, you can determine what behavior you want to see at certain points in the program to guarantee success. You must measure a lot more often than at a starting and ending point. Use the proof of the results of a training event plus the reinforcement program. But you should also measure progress during the program and use this data to create your interventions.
Behavior change takes time. Your learners need to practice and become comfortable with any new behaviors over time. New skills are mastered in small steps, and learners have to move through the 3 phases (why, how, apply).
When you applied the second principle, you closed the five gaps to ensure that change was not blocked. The process of behavior change challenges these gaps. Is one (or more) of the gaps “open” and blocking the learner from changing?
Know what you want to measure and when you want to measure it.
The first step is to determine what you want to measure and when. When you create the framework of your measurement plan, use the five What and When questions:
What do your learners need to do at each point in your reinforcement program? (What mini steps will create the most impact?)
What level of progress do you want to see from your learners and over what time period?
What would you like to see your learners apply after one month? Another month?
What impact would you like to see in the organization and when?
What should the learners evaluate and reflect on and when?
If you want to check whether the framework of your measurement plan aligns with the behavior that is needed to create impact in your organization, use the reinforcement objectives. Use the thinking process and the outcome of the 10-step approach to determine a reinforcement objective.
Examine the preconditions for change.
After you have determined the points at which you want to see the defined behavior, it’s time to determine the next measurements. These questions are not specific to the behavior change but focus on the important conditions:
Do the learners understand why it’s important to change?
Is the needed knowledge on a correct level?
Do the learners feel comfortable using new knowledge and skills?
Does the environment still meet the requirements for growth?
The answers provide you with insight on how preconditions influence the learners’ success. An important precondition is knowledge. I see that in almost every reinforcement program. Remember, increasing knowledge does not automatically mean using it. It is a precondition!
To determine the knowledge level, ask:
At what point do your learners need to master new knowledge to make sure that a lack of knowledge is not blocking behavior change?
An important part of this question is: “At what point. . . .”
If your training event is well structured, your reinforcement program will consist of only 25 percent knowledge and skills checking. Reinforcement is not retraining or micro learning, so you only need to include knowledge and skills checkpoints if knowledge or skill level are preconditions.
Fine-tune your measurements.
It doesn’t make sense to measure knowledge level as a precondition for successful behavior change at the end of your program or to measure the learners’ awareness of the importance of behavior change after you check their knowledge levels. Follow the 3 phases of behavior change, collecting valuable data to use in your analysis and determining your interventions, as needed.
In practice, I have seen successful variations where measurements of the specific phases of behavior change are mixed. I often see designers who use knowledge measurements in the Apply phase to check current knowledge levels or to inspire learners to keep their knowledge levels up to date. I also see that designers use survey questions to measure the application of new knowledge and skills, not only in the Apply phase but also in the beginning of the program. In that case, the survey questions are meant to drive learners’ awareness rather than to gain insights about how well the learning has transferred.
If you mix and match, your role as a reinforcement specialist will be challenged. You’ve already determined what you want to measure and when, so the final step to complete your measurement plan is the How questions. Challenge yourself with these questions:
How will you ask the questions?
How will you avoid being predictable?
How flexible is your measurement plan?
How can you avoid creating an assessment tool?
How can you use connecting questions to determine progress?
How can you balance push and pull in your reinforcement program?
How will you create value for the learners and collect valuable data for yourself at the same time?
ACTIONABLE INTELLIGENCE
Using Principle 3, “Create Measurable Behavior Change,” makes your reinforcement program results-driven. Once your measurement plan has been implemented, you will collect a great deal of data. How valuable is that data? and How can you optimize the results of your analyses? I would like to introduce Actionable Intelligence.
Before you learn how to use Actionable Intelligence, let’s explore what it is. Figure 8.2 shows what to do with your data to take focused action within your organization.
During your reinforcement program, data is collected from carefully crafted and planned measurements. The first step is to organize this data.
After you’ve organized the data, it can be converted to information in the form of reports or overviews.
This information can then be reviewed. The next step is to analyze the information to convert it into knowledge.
If you use this knowledge to predict why and what will happen, knowledge then becomes intelligence.
Based on this intelligence, you can identify potential decisions.
When you make a decision, you can define the needed actions.
But the cycle doesn’t end there. It continues as you measure the results of these actions. Organize the data, convert it into information, and so on. This is how raw data from your measurement plan turns into actionable intelligence.
Measuring is so much more than just knowing.
Asking the right questions
During your reinforcement program, you want to gather as much data as possible. Apart from answering What, When, and How questions, you must also construct good questions. If you want to collect different data, you can use different types of questions. In Table 8.1, three types of questions often used in reinforcement programs are listed.
The learners have multiple choices from which to select a correct answer.
Measure the knowledge level of the learners.
Survey
The learners have multiple choices from which to select an answer. These questions seem like quiz questions, but the difference is that every answer is correct. There are no correct/incorrect answers.
Help you to collect ideas, opinions, frequency, reflections, priority, levels of difficulty, or levels of comfort.
In some cases, you may not be happy with certain answers, but that is different from an answer being incorrect.
Open-Field
The learner is not guided in the answers. After understanding the question, the learner is free to compile an answer.
Used for self-refection. The learner needs to think about the answer and write it down, which has a good effect on reinforcement results.
If you limit the text box to 1,000 characters, the learners have to focus on the key points.
The difference between open-ended and closed questions
What’s the difference between a closed question and an open-ended question? A closed question is one that can be properly and sensibly answered with a yes or no response. Closed questions always start with a verb.
Are you using. . .?
Did you meet. . .?
Do you use these. . .?
An open-ended question cannot be answered with a simple yes or no. These questions start with W’s and an H, as shown in Table 8.2.
Keep in mind that some open-ended questions are easy to answer, usually with only one word.
Who. . .?
Where. . ?
When. . ?
Some reinforcement specialists consider closed questions “bad” and open questions good, but that is false. Each type has its uses. Closed questions, because they stimulate short yes/no answers are ideal for creating a fast-paced interaction between you and the learners. Clearly, closed questions are not ideal to stimulate in-depth thinking, which is a major function of open-ended questions.
Combining question types
If you combine question types, you can collect valuable data. What data do you collect if you, for example, combine a survey question and an open-field question?
You can ask the learners to answer a survey question about their comfort level using the new knowledge or skills. The learners must pick an answer from the four or five choices. The next question is an open-field question in which the learners are asked to explain why they choose an answer to the previous survey question.
By using this method, you can collect data on a macro level, but by following up with an open-field question, you also gather micro-level data. The coach can analyze the survey as well as the explanation from the learner. In practice, I see this combination a lot because it provides a lot of good insights on both macro and micro levels.
Think about what other question combinations would be powerful in a reinforcement program. Try these combinations in your program and analyze their value. After you are familiar with Principle 4, “Provide a Perfect Push and Pull,” and Principle 6, “Follow the Reinforcement Flow,” you will be able to balance your questions and avoid building a boring assessment tool.
Obtaining valuable answers
A good question generates valuable answers that can be analyzed. For a reinforcement program, one of my clients created a reinforcement objective, used the 10 steps, and followed all of the principles to build a strong foundation. The behavior change that the client wanted to see from their leaders was to have weekly meetings. To increase the quality of these meetings, the training event provided the learners with a form that listed the steps they had to follow to have an effective meeting. The survey question they used was simple: Have you had a meeting in the past week?
In Table 8.3, you see two sets of answer choices for this question. Which one would you select and why?
With answer set A, you collect the perfect data. X percent had a meeting and Y percent did not have a meeting. Answer set B gives you much more data:
You can determine the number of leaders who had a meeting and did not have a meeting last week.
You can determine from all the leaders who had a meeting and what percentage used the meeting form.
You can determine from all leaders the percentage who had scheduled a meeting and who had not.
If you use survey questions to collect quantity data, I challenge you to add quality aspects. You will not only collect more valuable data for your purposes, but you will also focus the learners on quantity and quality. In a reinforcement process, the different answers influence the learners’ thinking process. Don’t create lazy brains!
Don’t miss an opportunity to collect additional valuable information. Principle 6, “Follow the Reinforcement Flow,” will teach you how to use specific answers to balance skills and challenges.
Using ranges as answer choices
Using a Likert-type scale when you create the answers on your survey questions will also help you build actionable intelligence. The Likert scale is the most widely used approach to scaling responses. Some examples:
Level of Agreement
1 – Strongly disagree
2 – Disagree
3 – Neither agree or disagree
4 – Agree
5 – Strongly agree
Priority
1 – Not a priority
2 – Low priority
3 – Somewhat priority
4 – Neutral
5 – Moderate Priority
6 – High priority
7 – Essential priority
Frequency
1 – Never
2 – Rarely
3 – Sometimes
4 – Often
5 – Always
Level of Difficulty
1 – Very difficult
2 – Difficult
3 – Neutral
4 – Easy
5 – Very easy
Level of Quality
1 – Poor
2 – Fair
3 – Good
4 – Very good
5 – Excellent
Level of Comfort
1 – Not comfortable
2 – Somewhat comfortable
3 – Somewhat comfortable
4 – Very comfortable
Five common failures in questions
Apart from the improvements you can make to the answers, the question itself often needs extra attention. Review these five common failures in the design of a question before you write questions for your learners.
The question is too complex. Designers often write an introduction to the question, making the question unclear.
Example: Implication questions are sometimes important for ensuring a shared understanding of a problem’s severity or urgency. As a result of well-focused implication questions, the customer articulates and thus sees the problem more clearly, along with the consequences of not addressing the problem. How confident are you in your ability to ask implication questions during a sales call?
Tip: Split the introduction and the question clearly.
One question contains two questions.
Example: How confident are you in your ability to ask implication questions during a sales call, and what consequences does this have on your conversation partner?
Tip: Have the learner answer one question at a time. Always!
The question is too vague.
Example: Sometimes confidence is important. How confident are you in your ability to ask lots of implication questions during most of your sales calls in the upcoming weeks?
Tip: Don’t use the words sometimes, some, many, most, several, really, good, lots, thing, very, big, a few, or enough.
The question could have multiple interpretations.
Example: How confident are you in your sales calls and ability to ask implication questions?
Tip: Write out your question first, send it to a colleague, and then discuss it face-to-face to make sure there is no miscommunication. In this example, are you confident in the sales calls, or are you confident in the sales calls and asking implication questions?
The question is not personal enough; the learner does not recognize his or her own situation in the question.
Example: How would you rate your confidence level of asking some implication questions during the calls our salespeople have with our customers?
Tip: Make every question specific to the learner’s personal situation. Avoid words like the market, the customer, our salespeople. Later in this book you will learn how to write in the second person, and use you and your effectively.
If you avoid these five common mistakes in your design of the questions and combine quantity and quality in your answers, you collect the right data for your actionable intelligence.
