CHAPTER 7

Impact Objectives

For some stakeholders, impact objectives are the most important. These objectives represent both tangible and intangible measures. This chapter focuses on developing Level 4 (Impact) objectives. These objectives are based on business measures that are plentiful throughout organizations. Data developed at Level 4 are explored, along with the issues involved in writing the objectives. The development of these objectives is straightforward. Several examples are presented.

ARE IMPACT OBJECTIVES NECESSARY?

The chain of impact presented in chapter 1 presents the impact objectives as the consequence of a program or project. These objectives are reflected in measures of hard data, such as output, quality, cost, and time, as well as soft data, such as customer satisfaction, employee engagement, and brand awareness. These objectives are particularly important for senior executives. Some executives lack enthusiasm about a project or program unless the results reflect key business measures. These impact measures are found in operating reports, scorecards, dashboards, and key performance indicators. It is often the deficiency in performance of one or more impact measures that has led to the creation of the program. This was discussed in chapter 2. In terms of power to affect project design, delivery, and success, impact objectives are the most powerful. They provide the focus needed to drive the results that top executives desire.

In practice, many programs do not contain these impact objectives. This is particularly true in soft programs, such as learning and development, human resources, leadership development, executive coaching, communication, change management, public relations, and compliance. In reality, each of these types of programs can be connected to a business measure and thus have business impact objectives connected to them. The challenge is to perform the analysis described in chapter 2 and develop the objectives using the techniques in this chapter.

HOW TO CONSTRUCT IMPACT OBJECTIVES

Chapter 2 focused on how objectives are developed working through levels of needs assessment. A business measure reflects a business need. Job performance needs and the subsequent analysis ensure that the business measure is connected to the project or program. When this happens, the business need is clearly defined and is the primary driver in selecting or developing the program or project.

Precise Definition

It is important to ensure that the precise definition of the impact measure is offered. Definitions can vary considerably. For example, a broad definition of quality is unusable. Quality may be defined as errors, rework, warranty claims, or low scores on industry-wide customer satisfaction surveys. The specific quality measure must be defined, which occurs during the business needs analysis. Consider employee turnover. An objective for a project might be to reduce employee turnover. The wording here is too vague, however, and needs to be defined as involuntary turnover, avoidable turnover, or regrettable turnover. The definitions are critical so that the specific measure becomes a focus during the project or program.

A Consequence of Action

A business impact measure, the basis for the impact objective, is a consequence of a specific action. In chapter 2, the initial analysis included a step to determine what participants are doing or not doing on the job that is influencing the business measure. This translates into the specific action needed and is written as an application objective. Sometimes, there is confusion between Levels 3 and 4. Level 3 (Application) is always action (activity), and Level 4 (Impact) is the consequence of that action. Consider the objective to reduce project management time by 20 percent. Some may consider this a Level 3 objective, in which participants will simply reduce the amount of time. However, to accomplish the objective, participants must do something different. What has worked before is not working. Perhaps a new process, policy, tool, or even training is needed to manage the project. Any of these solutions can reduce time if implemented properly. The Level 3 objective would be the use of the process, policy, tool, or training. The time reduction is the consequence of the use of the process, policy, tool, or training (that is, reduction in time)— a Level 4 objective. This distinction should be reviewed constantly to ensure understanding of these two types of objectives.

Criterion

As with learning and application objectives, impact objectives should include a criterion. The criterion is usually defined as speed, accuracy, or quality and follows the same rules stated in chapters 5 and 6. For example, an objective for a safety project might be that government-imposed safety fines should decrease from $2.5 million per year to less than $500,000 within one year after project completion. This objective includes accuracy and time criteria. This degree of specificity is needed to measure success. For some projects, the specificity is at the individual level. For example, a project designed to increase sales in a new-product launch could vary with each individual customer relationship manager. In some territories, the new product has little competition, and the growth number can be set high. In others, where competition is intense, the growth number might be much smaller. In this circumstance, an overall objective might be more appropriate. For example, the objective might be that sales reach a 12 percent increase throughout the entire company. The key here is to make the objective specific enough to communicate what is desired.

Successive Impact Measures

A potentially confusing issue is the fact that some impact measures have a successive chain of impact. The difficulty lies in deciding whether one objective is an appropriate measure or if all measures are objectives. For example, Table 7.1 details five possible consequences of sexual harassment in the workplace. The victim of the harassment suffers stress; the victim’s job satisfaction drops; internal complaints of sexual harassment increase; the victim is increasingly absent from work; and employee turnover rises as victims seek employment elsewhere. The difficulty lies in determining which measures (if not all) are influenced by a sexual harassment prevention program. Most of this is sorted out in the up-front needs assessment to ensure that the particular program can indeed influence all these measures. Even if the principal focus of the program is to reduce complaints, it is important to determine whether the other measures are connected. If they are, they also could become objectives for the program.

Table 7.1: Successive Impact Measures
		Level		Data (one year)
		3	Sexual Harassment
Stress	Job satisfaction	4	Formal internal complaints	55	Absenteeism	Turnover
		4	External charges	24
		4	Litigated complaints	10
		4	Legal fees/ expenses	$632,000
		4	Settlements/ losses	$450,000
			Total cost
			Prevention/ investigation/
		4	Defense	$1,655,000
Adapted from “preventing sexual Harassment—Healthcare inc., ” chapter 1, Proving the Value of HR: ROI Case wStudies. P.P. phillips & J.J. phillips. birmingham, alabama: ROI institute and Mpi, 2007.

When considering complaints, there is a successive series of Level 4 (Impact) measures. A formal internal complaint, if not resolved, could convert to an external charge with the Equal Employment Opportunity Commission. If that charge is not resolved to the victim’s satisfaction, he or she has a right to sue the employer, creating a litigated complaint. Litigation leads to legal fees and expenses and also to settlements. Ultimately, all of this, from prevention to investigation to defense, represents a significant cost (Phillips & Phillips, 2007).

A program could actually have objectives for each of these. Although they should all improve in relative proportion, this might not be the case. Under U.S. law, employees have a right to sue an employer, even before an external charge is actually resolved or if it is resolved in favor of the victim. For those reasons, the focus might be on reducing the number of litigated complaints. The confusion comes when the monetary value of this program is calculated. Converting data to monetary value is critical to evaluation when an ROI calculation is developed. Using all Level 4 impact measures represents a tremendous amount of duplication. Perhaps it would be best to take one measure and use it in the conversion process. Still, they can all be objectives that are influenced by the sexual harassment prevention project. This series of successive impact measures can usually be uncovered when asking a series of “what if” questions. What if this measure happens? Does it lead to something else?

TOPICS FOR IMPACT OBJECTIVES

Impact objectives are measures located throughout the organization as common indicators. There are hundreds, if not thousands, of these measures in an organization. When identifying impact measures, consider the following issues.

Hard Versus Soft Data

To help set objectives for the desired measures, a distinction is made in two general categories of data: hard data and soft data, as described in chapter 2. Hard data are the primary measurements of improvement, presented through rational, undisputed facts that are easily collected. They are the most desirable type of data to collect. The ultimate criteria for measuring the effectiveness of management rest on hard data items, such as productivity, profitability, cost control, and quality control. Chapter 2 provided examples of hard data grouped into categories of output, quality, cost, and time.

Hard-data measures are often supplemented with interim assessments of soft data, such as brand awareness, satisfaction, loyalty, and teamwork. Although a program designed to enhance competencies or manage change should have an ultimate impact on hard-data items, measuring soft-data items may be more efficient. While soft data may be more difficult to analyze, they are used when hard data are unavailable. Soft data are more difficult to convert to monetary values than hard data; are subjectively based, in many cases; and are less credible as a performance measurement. Chapter 2 provided a list of typical soft-data items grouped into typical categories.

The preference of hard data in programs does not minimize the value of soft data. Soft data are essential for a complete evaluation of a program; success may rest on soft-data measurements. For example, in an empowerment program at a chemical plant, three key measures of success were identified: employee stress, job satisfaction, and teamwork. All were listed as intangibles.

Most programs have objectives that use a combination of hard- and soft-data items in the evaluation. For example, a project to install new technology in a manufacturing plant had the following impact objectives:

• Reduction of production costs.
• Improvement in productivity.
• Improvement in quality.
• Reduction in inventory shortages.
• Improvement in production capability.
• Increase in technology leadership.
• Increase in job satisfaction.

These improvements included both hard data (production costs, productivity, and quality) and soft data (capability, technology, leadership, satisfaction). Most programs include both types of objectives.

Tangible Versus Intangible

The confusion about the categories of hard and soft data and the often-reduced value placed on soft data was discussed in chapter 2. This leads to a critical definition in this book. While the terms hard data and soft data can be used to discuss impact data, the terms tangible and intangible can also be used and represent a more accurate depiction. Tangible data are those data that have been converted to monetary value. Intangible data are defined as data purposely not converted to monetary value (that is, if data cannot be converted to monetary value credibly with a reasonable amount of resources, then they are reported as intangibles). This approach has several advantages. First, it avoids the sometimes confusing labels of soft and hard. Second, it negates the argument that being soft equates to little or no value. Third, it brings definition to the situation. In some organizations, a particular data item may be converted to money already, and the conversion is credible because the measure is already tangible. However, in other organizations, the same measure may not been converted and cannot be converted with a reasonable amount of resources. Therefore, it is left as intangible. Fourth, use of the terms tangible and intangible provides a rule that enhances the consistency of the evaluation process. Having such a rule helps ensure that if two people conduct the same evaluation, they will get the same or similar results.

Scorecards

Scorecards, such as those used in sporting events, provide important measures fans can review to understand the position of their team. Similar scorecards are used by top executives and often form the basis for impact objectives. In Robert Kaplan and David Norton’s landmark book The Balanced Scorecard, this concept was brought to the attention of organizations (Kaplan & Norton, 1996). The authors suggested that data can be organized and reported from four perspectives: financial, customer, business processes, and learning and growth.

Scorecards come in a variety of types, such as Kaplan and Norton’s balanced scorecard and the scorecard set in the President’s Management Agenda, which uses the traffic light grading system (green for success, yellow for mixed results, red for unsatisfactory). Regardless of the type of scorecard, top executives place great emphasis on this concept. In some organizations, the scorecard concept has filtered down to various business units, and each part of the business has been required to develop scorecards.

The scorecard approach is appealing because it provides a quick comparison of key measures and examines the status of the organization. As a management tool, scorecards can be important in shaping and improving or maintaining the performance of the organization through the implementation of preventive programs. Scorecard measures often link to particular projects or programs. In many situations, a scorecard deficiency measure may have prompted the program in the first place.

Measures Linked to Specific Programs

An important issue that often surfaces when considering ROI applications is the understanding of specific measures that are often driven by specific types of programs. While there are no standard answers, Table 7.2 represents a summary of some typical measures for objectives for specific types of programs. The measures are quite broad for some programs. For example, leadership development may pay off in a variety of measures, such as improved productivity, enhanced sales and revenues, improved quality, cycle-time reduction, direct-cost savings, and employee job satisfaction. For other programs, the measures are quite narrow. Labor-management cooperation projects typically influence grievances, work stoppages, and employee satisfaction. Orientation, or onboarding, programs typically influence measures of early turnover (turnover in the first 90 days of employment), initial job performance, and initial productivity. The measures that are influenced depend on the objectives and the design of the program. Table 7.2 also illustrates the immense number of measures that can be driven or influenced.

A word of caution is needed. Presenting specific measures linked to a typical program may give the impression that these are the only measures influenced. In practice, a particular program can have many outcomes. Table 7.2 shows the most likely measures based on studies the ROI Institute has conducted or reviewed. In the course of a decade, we have been involved in more than 2,000 studies, and common threads exist among particular programs.

Relevant Measures

Existing performance measures should be thoroughly researched to identify those related to the proposed program. Several performance measures often are related to the same item. For example, the efficiency of a production unit can be measured in several ways:

• the number of units produced per hour
• the number of on-schedule production units
• the percentage of equipment used
• the percentage of equipment downtime
• the labor cost per unit of production
• the overtime required per unit of production
• total unit cost.

Table 7.2: Typical Impact Measures for Projects and Programs
Program	Key Impact Measurements
Absenteeism control/ reduction	Absenteeism, customer satisfaction, delays, job satisfaction, productivity, stress
Association meetings	Absenteeism, costs, customer service, job satisfaction, productivity, quality, sales, time, turnover
Business coaching	Costs, customer satisfaction, efficiency, employee satisfaction, productivity/output, quality, time savings
Career development/ career management	Job satisfaction, promotions, recruiting expenses, turnover
Communications programs	Conflicts, errors, job satisfaction, productivity, stress
Compensation plans	Costs, job satisfaction, productivity, quality
Compliance programs	Charges, losses, penalties/fines, settlements
Diversity	Absenteeism, charges, complaints, losses settlements, turnover
Employee retention programs	Engagement, job satisfaction, promotions, turnover
Engineering/technical/ training conferences	Costs, customer satisfaction, cycle times, downtime, job satisfaction, process time, productivity/output, quality, waste
Ethics programs	Fines, fraud, incidents, penalties, theft
E-learning	Cost savings, cycle times, error reductions, job satisfaction, productivity improvement, quality improvement,
Executive education	Absenteeism, costs, customer service, job satisfaction, productivity, quality, sales, time, turnover
Franchise/dealer meetings	Cost of sales, customer loyalty, efficiency, market share, quality, sales
Golfing events	Customer loyalty, market share, new accounts, sales, upselling
Labor-management cooperation programs	Absenteeism, grievances, job satisfaction, work stoppages
Leadership development	Cost/time savings, development, efficiency, employee satisfaction, engagement, productivity/output, quality (continued on next page)
Management development	Absenteeism, costs, customer service, job satisfaction, productivity, quality, sales, time, turnover
Marketing programs	Brand awareness, churn rate, cross-selling, customer loyalty, customer satisfaction, market share, new acounts, sales, upselling
Medical meetings	Compliance, efficiency, medical costs, patient satisfaction, quality
Orientation, onboarding	Early turnover, performance, productivity, quality of work, training time
Personal productivity/time management	Job satisfaction, productivity, stress reduction, time savings
Project management	Budgets, quality improvement, time savings
Quality programs	Costs, cycle times, defects, response times, rework
Retention management	Engagement, job satisfaction, turnover
Safety programs	Accident frequency rates, accident severity rates, first aid treatments
Sales training/meetings	Customer loyalty, market share, new accounts, sales
Self-directed teams	Absenteeism, customer satisfaction, job satisfaction productivity/output, quality, turnover
Sexual harassment prevention	Absenteeism, complaints, employee satisfaction, turnover
Six sigma/lean projects	Costs, cycle times, defects, response times, rework, waste
Software projects	Absenteeism, costs, customer service, job satisfaction, productivity, quality, sales, time, turnover
Stress management	Absenteeism, job satisfaction, medical costs, turnover
Supervisor/team leader programs	Absenteeism, complaints, costs, job satisfaction, productivity, quality, sales, time, turnover
Seam building	Absenteeism, costs, customer service, job satisfaction, productivity, quality, sales, time, turnover
Wellness/fitness programs	Absenteeism, accidents, medical costs, turnover

Each of these, in its own way, measures the effectiveness or efficiency of the production unit. Related measures should be reviewed to determine those most relevant to the program.

Existing Measures Converted to Usable Ones

Occasionally, existing performance measures are integrated with other data, and keeping them isolated from unrelated data may be difficult. In this situation, all existing, related measures should be extracted and tabulated again to make them more appropriate for comparison in the evaluation. At times, conversion factors may be necessary. For example, the average number of new sales orders per month may be presented regularly in the performance measures for the sales department. In addition, the sales costs per sales representative are also presented. However, in a particular program, the average cost per new sale is needed. The average number of new sales orders and the sales cost per sales representative are required to develop the data necessary for comparison.

New Measures

In a few cases, data needed to measure the success of a program are unavailable, and new data are needed. The project leader must work with the client organization to develop record-keeping systems, if economically feasible. In one organization, the sales staff’s delayed responses to customer requests were an issue. This problem was discovered based on customer feedback. The feedback data prompted a project to reduce the response time. To help ensure the success of the project, several measures were planned, including measuring the actual time to respond to a customer request. Initially this measure was not available. As the program was implemented, new software was used to measure the time.

When developing new measures, several questions need to be addressed:

• Which department/section will develop the measurement system?
• Who will record and monitor the data?
• Where will it be recorded?
• Will input forms be used?
• Who will report it?

These questions will usually involve other departments or a management decision that extends beyond the scope of the project. Often the administration, operations, or technology functions will be instrumental in helping determine whether new measures are needed and, if so, how they will be developed. However, this action should be a last resort.

Summary

Developing impact objectives is relatively easy when following a few specific guidelines. Impact measures are located throughout a typical organization. The measures are there; it is a matter of connecting them to the project. Table 7.3 shows the key steps to developing impact objectives.

HOW TO USE IMPACT OBJECTIVES

Impact objectives are even more powerful than application objectives when driving results. While application objectives help remove the mystery of what participants should be doing to make the project or program successful, impact objectives show the consequence of the application, expressed in measures that are precise, meaningful, and desired by top executives. Here are a few uses for these objectives.

Program Design

Like application objectives, impact objectives give additional guidance and direction to designers and developers. Impact objectives show those who develop the content the expected ultimate outcomes. Consequently, the activities, discussions, dialogue, exercises, skill practices, and problems focus on those outcomes. Impact objectives keep the ultimate focus on why a project is being pursued. The content may change dramatically with the inclusion of impact objectives. Designers are creative people; and in the absence of specific consequences, they will fill in the blanks with perhaps incorrect information.

Program Facilitation

Impact objectives are powerful for those who lead the team. For example, in formal learning and development sessions, the facilitator has clear direction of the results of the project. Teaching will focus on the ultimate consequences, the impact measure. Here, a facilitator will use previous experience and knowledge of the measure to teach with the end result in mind. For a project team leader, the ultimate result is clearly in focus and is a goal for the team to reach—a goal that is precise, measurable, and achievable. The team’s discussion will address these measures, the progress made, and the barriers that might be in the way.

Participants

Impact objectives are provided to participants so they understand how the project will benefit them personally, as well as the organization. Having an impact measure takes the mystery out of the ultimate consequence of their actions and implementation. It shows how the project will drive a particular and important measure in the organization. They see what is in it for them and what is in it for the organization. This also allows the organization to create expectations for these participants, sometimes with individual goals for the impact measure.

Marketing

Nothing sells a project or program more effectively than when impact measures are connected to the project or program. Impact objectives provide impressive information for prospective participants for routine learning and development, coaching, leadership development, team building, compliance, ethics, and even communications programs. The potential impact of the project helps to convince all interested stakeholders about the value of the project. The objectives can serve as the most important strategic marketing data possible.

Evaluators

The impact objectives guide the evaluation. The impact data sets are defined so that they are easily collected. They provide direction as to the options for isolation and data conversion. In fact, they provide the roadmap to overall success of the project or program.

EXAMPLES

Projects and programs should drive one or more business impact measures. Impact objectives represent key business measures that should be improved as the application and implementation objectives are achieved. Table 7.4 shows typical business impact objectives from a variety of projects and programs.

FINAL THOUGHTS

This chapter outlines the process of developing impact objectives. Business impact objectives are critical to measuring business performance because they define the ultimate expected outcomes of the program. They describe business-unit performance that should be connected to the program. Above all, impact objectives emphasize achieving bottom-line results that key client groups expect and demand. For some, these are the most powerful objectives. They are straightforward and based on measures that are plentiful in organizations. The objectives are based on needs identified in the up-front analysis covered in chapter 2. This chapter presents additional tips and techniques to develop and fine tune impact objectives, building on the material in chapter 2. Examples were presented in this chapter, and guidelines and issues were explored. The next chapter focuses on the highest level of objectives, ROI.

EXERCISE: WHAT’S WRONG WITH THESE IMPACT OBJECTIVES?

As presented in previous chapters, problematic objectives exist. After reading Table 7.5, indicate concerns about each objective. Responses to this exercise are provided in Appendix A.