STEP NINE

Measure Success

If you can’t measure it, you can’t manage it. 

—W. Edwards Deming

Measuring and evaluating the success and results of your change initiative is vital. Knowing what data to look for and how to interpret it is key to knowing whether you are on the path to success or failure. Often, organizations decide on a set of measures for their change initiative that have no relation to true performance. These measures focus on activity rather than productivity. An example would be interviewing 50 stakeholders to understand what they hope to see as a result of the change initiative, only to have them come back and say they do not like what they have seen. The focus should not be on interviewing 50 stakeholders—it should be related to how the stakeholders see progress toward reaching the goals agreed on at the beginning of the project. Measures need to give you information you can use to show progress toward a goal.

Measurement is important for a number of reasons:

• It allows for communicating the level of progress toward reaching a goal.
• It keeps stakeholders informed of what is working and potential issues that have come up.
• It shows investors the return on their investment.
• It allows others to conduct their change plans to work in conjunction with yours.
• It allows concerned parties to provide feedback according to what they see.
• It provides an opportunity to make changes or corrections to the plan.
• It acts as a learning tool to show others what works. 
• It allows an organization to build a culture of sharing.

How Do You Measure Success?

Helping your stakeholders identify and agree on areas to measure, and how often to measure them, builds credibility and allows them to see how things are actually progressing compared to the change plan. It allows for issues to be addressed early on before they turn into major problems. It also allows for contact on a regular basis, which is another important aspect in communicating with stakeholders to build strong relationships. Make sure your metrics are aligned with the goals of your change initiative.

What will you be measuring? How will you be measuring it? How often will you share information on progress toward the goals? The answers to these questions should be agreed to by both the stakeholders and the project sponsors. Any changes in plans or time line should be discussed and have their buy-in. Keep areas in mind that refer to quality, speed, and cost, sometimes referred to as the “better, faster, cheaper” model. Some areas to consider incorporating in your measures include the following:

• Budget—What budget was set for the project? How is the money being spent (burn rate), in what amounts, and during what period of time? Measures would show whether the change initiative is under or over budget and by how much at what point in time.
• Other Financials—How much will the change initiative save in production? From an internal perspective look at cost cutting, and from an external perspective consider revenue generation. Can you see an increase in existing customer spending or new customers coming onboard as a direct result of the change process?
• Time—What time line did you set before starting the change project? Identifying key project dates and specific project milestones gives individuals involved in the change initiative something to strive for as they make progress toward change.
• People—How have the skill sets of change team members affected other areas of the project and the overall goals? Measuring individuals’ contributions makes it easier to know what areas are lacking and what type of additional change team members are needed to fill these gaps.
• Recognition—How are change team members responding to the change process? Are they excited? Is the project acting as a motivator to drive further change, or do members of the change team need more support?
• Quality (Thoroughness)—Are there recognized industry standards that can help in the measurement process? To what extent and in what ways has your organization adhered to these standards?
• Functionality—Can the results of your change initiative be measured incrementally as you work to achieve your overall goal? One way to measure progress is to release a subset of a product line as it is completed, but before the whole product line is available.
• Resources—How effectively are you using available resources, and to what extent? Consider enlisting technology, knowledge, and other tools or supplies.
• Competition—Are you keeping up with, ahead of, or behind the competition? 

Keeping stakeholders informed of the progress toward reaching the change goals is important in maintaining their ongoing support. It is best to identify what will be measured, how, and how often at the start of the change effort, in your change management plan (see Step 5). A good plan gives stakeholders an opportunity to provide ongoing feedback from their perspective as well.

The Measurement Process

All of the above areas can be measured in a number of ways. When measurements are made during the change process, it is known as formative evaluation. This type of evaluation is done at regularly scheduled periods or at certain stages of a project and is recommended for organizational change.

Among the many terms and concepts used in the measurement process, the concept of performance measurement attempts to look at performance in terms of outcomes—such as the number of parts produced per day or financial performance for the quarter—and inputs—such as the amount of raw materials, the quality of the material input, and the skills of individuals involved in the process. The key is to use measurement to follow your progress toward your goal. Taking action on a small issue can save a project from much costlier problems down the road.

Kirkpatrick’s Levels of Evaluation

Closely related to individuals’ performance are the training and development they receive while participating in an organizational change initiative. One model often used in the process of assessing individuals’ experience is Kirkpatrick’s Levels of Evaluation (Margolis, 2009).

The model is based on a continuum from how individuals feel about the training they have received to whether any results can be derived from that training. Level 1 looks at whether participants enjoyed the initiative. It is often not enough to affect what they do. Level 2 looks at learning. Individuals can be tested to see how much they have learned. Level 3 involves making observations to see if individuals behave differently. Level 4 is the most revealing, but also takes the longest time to measure. It considers whether things have improved from an earlier state. An added challenge in Level 4 is making sure any changes can be linked back to the training. A fifth level has been proposed to measure the return on investment from the training. Basically, did the money spent on training pay off?

Remember that measured success in one change area may be offset by an issue caused by change in another. Take, for example, the case of a well-known computer company that, for years, received high marks in customer service. In an effort to cut costs, the company outsourced much of its customer service department, but this action, while drastically reducing costs and achieving the company’s goal by any standard of cost reduction, caused the reported quality of its customer service to decrease dramatically.

One customer’s online comment demonstrates the extent of the decline: “On my last purchase, the monitor was DOA [dead on arrival]. I called the customer support number and got a guy with a very heavy accent who directed me to the monitor manual on CD, provided with the computer. I asked him if he understood the irony of putting the monitor troubleshooting guide on a CD since if the monitor didn’t work you’d have no way of reading the guide. After 40 minutes of being on hold and talking to a supervisor, they said they’d have to send a new one—the very suggestion I’d made at the beginning of the call.”

Key Points to Keep in Mind

In the end, a measurement system should

• link to the goals of the organization
• provide feedback as to the effectiveness of the organization’s strategy
• be easy for individuals to understand
• look at areas relevant to what is being measured
• look at both short-term and long-term performance 
• be consistent with the reward structure of the organization
• be a constant, not just an occasional activity
• give insight into not only what is wrong but also how it could be better.

WORKSHEET 9.1

Areas of Change initiative Measurement