3

Alignment of Engagement Programs and Evaluation Planning

In far too many situations, employee engagement programs are implemented without a complete picture of the reason for the program. Pursuing initiatives without knowing why or identifying clear performance and business needs up front can be disastrous in today’s economic climate. Evaluation is too often an afterthought. This chapter explores why it is important to plan ahead, how to use the V-Model to align programs with business needs, the process of integrating needs with the evaluation, and the steps and artifacts included in evaluation planning.

ACHIEVING THE PROPER ALIGNMENT

The basis for an employee engagement program adding value rests on the rationale for its existence and the extent to which it relates to a specific business need. This fundamental concept requires a thorough needs analysis, which is the beginning point in the ROI Methodology. As described in chapter 2, conducting a diagnosis or assessment of the organization’s needs allows the employee engagement practitioner to determine, with the help of the client, the necessary programs. This also sets the stage for collecting any necessary data and minimizing defensiveness and resistance.

There are a variety of methods for conducting a needs analysis, but integrating the ROI Methodology is critical when measuring at higher levels of evaluation. For example, let’s assume that during an analysis phase two findings emerge: high turnover and low productivity. This is where the analysis phase moves to inquiring about what behaviors, rewards, and helpful mechanisms may be contributing to the high turnover and decreased productivity. In this case, a lack of employee engagement is causing most of the problems, but the organization’s leaders are not aware of how serious the problem is. The solution involves increasing engagement and developing leaders who use and encourage engagement in productive ways. As you can see, this approach not only helps identify the right solution, but it sets into motion relevant goals and measures. There are two key questions that help quantify the gap that exists: What is the ideal or desired state? What is the current state?

The V-Model is a powerful method for ensuring business alignment because it maps the connection of needs analysis from programs to objectives and evaluation (Exhibit 3-1). It shows the important links between the evaluation and the initial problem or opportunity that created the need for the program. It also shows the three points at which business alignment occurs—the beginning of the program, during the program, and during the follow-up evaluation—in order to validate the alignment.

EXHIBIT 3-1. Alignment With the V-Model

From ROI Institute. All rights reserved.

The V-Model is based on the concept of the five levels of evaluation described in chapter 2. As we will explore throughout this book, employee engagement is a natural candidate for the ROI Methodology and its alignment process.

It’s best to think of the V-Model in terms of the evaluation side first. Evaluation moves through different levels of measurement:

•  reaction to the engagement concept (Level 1)

•  learning how to be engaged (Level 2)

•  application; becoming engaged (Level 3)

•  impact; the consequences of engagement measured through output, quality, and time (Level 4)

•  ROI; the financial value of engagement, a comparison of monetary benefits with the cost of the program (Level 5).

From the viewpoint of key stakeholders, such as the clients or sponsors, the higher levels are more important because they show the business contribution (Level 4) and ROI (Level 5). In terms of evaluation, Level 4 is where an isolation technique is applied to specify how much improvement comes directly from the program. This step ensures that business alignment is confirmed.

The measures that are captured at each level are defined in the objectives. There are five corresponding levels of objectives, as illustrated in Exhibit 3-1, which increase in importance as the levels progress; Levels 4 and 5 are often the most valuable from a client’s perspective. These objectives are developed during the needs assessment, which defines particular needs at each level. Here, the highest and most important level is the potential payoff of the program, followed by business, performance, learning, and preference needs.

Level 5, Payoff Needs

Needs assessment occurring at Level 5 addresses the potential payoff opportunity for an organization. This step is taken to determine whether an ROI will be possible if the program is pursued. The first part of the process is to decide whether the problem is worth solving or if the opportunity warrants serious consideration. This is obvious in cases where serious problems are affecting the organization’s operations and strategy. For example, reversing an annual 32 percent turnover rate of critical talent at a hospital is an obvious payoff opportunity. Another example is an organization for which new account growth is flat and customer loyalty is low based on industry standards. These types of payoff opportunities make it clear that there is a problem that needs to be solved or an opportunity that should be pursued with a clearly identified business need.

Others represent not-so-obvious payoff opportunities, such as a request to implement an employee engagement program or a succession management process. In this case, the business measures of significance become more evident during the Level 4 analysis.

During Level 5 analysis, it is important to not only identify the business measures that need to improve, but also convert them into monetary values when possible so the anticipated economic benefit of addressing the opportunity is evident. Determining the payoff’s monetary value is useful not only in identifying the scope of the opportunity, but also in forecasting the potential ROI. In many situations, this may not be a calculation but an “estimate” based on the perception of the cost of having disengaged employees. When the solution(s) is identified and the targets for improvement as a result of the solution are set, it is important to determine the approximate cost for the entire project. With the approximate program cost and the monetary value of the opportunity in hand, you can use the ROI forecast to indicate the potential payoff for investing in a particular program. While this practice may not be feasible for some programs, it is often important and necessary for very expensive, strategic, or critical programs.

Level 4, Business Needs

At Level 4, business data that indicate movement toward addressing the payoff need are examined to determine which measures are in most need of improvement. Ideally, employee engagement should improve the business measures listed in Exhibits 3-2 and 3-3. When cross-functional programs are anticipated, each participant could have a different business need. This may involve having participants bring one, two, or three measures (business needs) to improve, using the competencies with their teams. This is very powerful because employee engagement is now customized to the participants at the impact level.

EXHIBIT 3-2. Examples of Hard Data

This process may involve reviewing organizational databases to examine all types of hard and soft data. Sometimes the performance of one item triggers the employee engagement program. For example, it’s easy to pinpoint the business measure when sales are not as high as they should be, operating costs are excessive, product quality is deteriorating, or productivity is low. These key measures come directly from data in the organization and are often found in the operating databases.

EXHIBIT 3-3. Examples of Soft Data

Exhibit 3-2 shows these business needs arranged into four hard data categories: output, quality, costs, and time. Examples include sales, production, errors, waste, accidental costs, downtime, project time, and compliance fines. These measures exist in any type of organization—even in the public sector and among nonprofits and nongovernment organizations—and often attract the attention of executives and chief administrators because they represent business impact. It is important to connect a project to at least one of these measures. However, keep in mind that impact measures can also be subjective, such as customer service, image, work climate, customer satisfaction, job satisfaction, engagement, reputation, and teamwork.

In other cases, business alignment with employee engagement programs may involve a review of HR measures, such as employee engagement, job satisfaction, employee complaints and grievances, absenteeism and tardiness, teamwork, accidents and incidents, performance ratings, employee transfers and promotions, and talent retention, turnover, and turnover costs. Although these measures may not be as important as measures of output, quality, costs, and time, they are still important, and in some cases are the primary measures of interest for an employee engagement program. Soft data are sometimes reported as a program’s intangible benefits because they cannot always be converted to money credibly or with a minimum amount of resources. But they can be important business measures and, if improved, can help an organization take advantage of a payoff opportunity.

Level 3, Performance Needs

The Level 3 analysis involves determining performance needs or gaps that will contribute to improving the business measures. The task is to determine what is causing the problem (or creating the opportunity) identified at Level 4; for example, what is causing the business to be performing below the desired level? What should the organization be doing more or less of? Is there something the organization should be doing differently? Performance tools or systems, inadequate technology, lack of engagement, and broken or ineffective processes are all examples of performance needs.

The desired and current state should reveal performance needs because the reason for the inadequate performance will be the basis for the solution. For example, if customer complaints have increased, and it is discovered that the technology used by customer service is outdated and slow, then the technology that supports customer service representatives is the cause of the problem and needs to be resolved.

Performance needs can be uncovered using a variety of problem-solving or analysis techniques. This may involve the use of data collection techniques, such as surveys, questionnaires, focus groups, or interviews. The key is to determine the causes of the problem so that solutions can be developed. Sometimes there is one clear solution to address the performance need; other times there are multiple solutions and then a decision must be made as to which one to pursue, if pursuing all is not an option. ROI forecasting is one way to help make the decision.

Level 2, Learning Needs

During the Level 2 assessment, the specific information, knowledge, or skills required to address the performance needs are identified. An analysis may reveal learning deficiencies, in terms of knowledge and skills that can contribute to the problem. In other situations, the solution will need a learning component as employees learn how to implement a new process, procedure, or technology. For employee engagement, the solution typically involves the acquisition of knowledge or the development of skills necessary to become more engaged. In some cases, perceptions or attitudes may need to be altered before an employee engagement program can be successful. The extent of learning required will determine whether formalized training is needed, or if more informal, on-the-job methods can be used to create an increased engagement.

Level 1, Preference Needs

Finally, the Level 1 assessment describes the preferences for the engagement program. This involves determining the preferred way in which those involved in the process will need or want it to be implemented. Typical questions that surface include “Is this important?” “Is this necessary?” and “Is it relevant to me?” Preference needs may involve aspects of implementation, including decisions about when learning is expected, in what amounts, how it is presented, and the overall timeframe. Implementation involves timing, support, expectations, and other key factors.

Using the V-Model, Exhibit 3-4 shows an example of linking needs assessment with the evaluation of an employee engagement program involving a team-based process for making improvements. The target audience is composed of team members and team leaders. As the exhibit shows, the first step is to see if the problem is worth pursuing.

These five levels of needs analysis develop a comprehensive profile for determining how best to address an opportunity or problem worth solving. They also serve as the basis for the engagement program objectives.

EXHIBIT 3-4. Employee Engagement Program Example With the V-Model

Objectives

Objectives keep the employee engagement program and the business aligned by positioning that program with the outcomes in mind at every level. The levels of objectives are:

•  Level 0 Input objectives are the indicators that are generally tracked, such as the number of programs, people affected, hours, and so on. They are categorized as Level 0 because they do not reflect the outcome.

•  Level 1 Reaction and planned action objectives describe expected immediate satisfaction with the employee engagement program. They define important aspects, including the relevance of the change and the importance of the information or content shared through the process.

•  Level 2 Learning objectives describe the expected immediate outcomes in terms of knowledge acquisition, skill attainment, awareness, and insights gained through the employee engagement program. These objectives set the stage for transitioning to performance and behavior change.

•  Level 3 Application objectives describe the expected intermediate outcomes in terms of what behavior change, involvement, and actions are expected as a result of the employee engagement program. Objectives may target specific steps to be taken or specific behaviors that need to change.

•  Level 4 Impact objectives define the specific business measures that should improve as a result of the employee engagement. Improvements in these intermediate (and sometimes long-term) outcomes represent changes in output, quality, costs, and time, as well as customer satisfaction and employee satisfaction. Objectives at this level answer the question, “So what?” as it relates to the program. They describe to stakeholders the importance of the intervention.

•  Level 5 ROI objectives define for stakeholders the intended financial outcome. This single indicator sets the expectation for how the benefits of the employee engagement program will relate to the cost. These objectives must be identified and developed before the evaluation can be conducted; ideally this occurs early in the process when the program is being designed.

Each level represents a category of measures that describes how much progress is being made to address the various levels of need. Specific measurable objectives serve as the blueprint for building the employee engagement program. By aligning initiatives with the business through a thorough needs assessment and the development of measurable objectives, the implementation and evaluation of the employee engagement program becomes more systematic and reliable. Identifying stakeholder needs and developing relevant objectives make up the first two phases of business alignment and set the stage for planning the third phase: evaluation.

EVALUATION PLANNING

The evaluation must be planned—overall and individually—for each program. Not much planning is involved for an evaluation conducted only at the reaction levels, but as the evaluation moves up the value chain, increased attention and effort needs to be placed on planning. During the typical planning cycle, the purpose of the evaluation must be reviewed for specific solutions and to determine where the evaluation will stop on the value chain. The feasibility of evaluating at different levels should also be explored.

Employee engagement professionals understand the importance of planning for almost any type of undertaking. Most agree that thorough planning can lead to more effective implementation. The same holds true for ROI analysis. Careful planning for ROI analysis not only saves time and effort, but can also make a difference in the success or failure of the entire project. Planning involves the development of three documents: data collection plan, ROI analysis plan, and communication and implementation plan. These documents are described using the following case study.

CASE STUDY

A manufacturing company with 15,000 employees has experienced some recent problems. Revenues have increased during the past three years, but profit margins have declined to breakeven. The company has a solid and loyal customer base and aligns the business to client needs. However, the executive team is concerned about high turnover rates, low productivity, and low employee engagement. A recent analysis of the annual engagement survey determined several key outcomes:

•  To grow the business, more leaders need to ensure that employees are engaged.

•  Employees need to have a clear understanding of their goals.

•  Employees are not sure of their responsibilities.

The lack of engagement was also linked to business needs, including productivity, quality, retention, and cost reduction. The employee engagement team determined that it would be beneficial to the company to use team leaders to drive employee engagement, especially since the competencies identified for leadership included communication skills, business acumen, and the ability to engage employees.

The solution was a three-day intensive learning workshop for team leaders that focused on the concepts of engagement and how team leaders could get their employees more engaged. The employee engagement team agreed to implement a feedback process with the team leaders, which would provide an engagement score before the program and three months after the program to show the changes with this immediate group. The evaluation acted as a control group to isolate the program’s effects.

The workshop involved several key steps. Before participating, team leaders completed a one-hour online session to learn about the engagement process. Feedback was administered online and collected from team members and team leaders. This was followed by a feedback session conducted during the three-day workshop, along with a comprehensive report for each leader. The expected outcome was an action plan generated by the team leader.

Objectives

The employee engagement team created the following multilevel objectives for the employee engagement program:

•  Participants will rate the employee engagement program as relevant to their jobs.

•  Participants will rate the employee engagement as important to their success.

•  Participants must demonstrate acceptable performance on setting goals.

•  Participants must demonstrate acceptable performance on communicating responsibilities to team members.

•  Participants will use the engagement concepts with team members on a routine basis.

•  Participants will improve productivity, quality, retention, and costs.

Data Collection Plan

Exhibit 3-5 shows the completed data collection plan. Defining the objectives and measures at each level, including return on investment, is vital. Measures can sometimes be examined in different ways, so defining them up front eliminates confusion.

The data collection methods detailed in Exhibit 3-5 correspond to the different objective levels, using a range of options that are described in the next chapter. Next, the data sources were identified. Data can be collected from existing organizational databases, or by those participating in the employee engagement program. In some cases—as in the case of the 360-degree feedback assessment—team leaders, as well as their direct reports (team members), provide data.

Timing is important for determining when data should be collected from the different sources for each level. During implementation, data often come directly from those involved in the program. In other situations, the follow-up can be determined based on when the program is operational and successful.

Finally, the responsibilities were detailed, outlining specifically who should be involved in the data collection process.

ROI Analysis Plan

Exhibit 3-6 shows the completed ROI analysis plan, which is connected through business impact data. The first column shows the detailed definition of each impact data measure. The second column defines the method for isolating the effects of the program on each data item using one or more of the specific techniques available. The method of converting data to monetary values is listed in the third column using one or more available techniques.

The fourth column defines the cost categories for the specific program. Using a fully loaded cost profile, all the categories are detailed here. Completing this action during the planning stage is helpful for determining whether specific cost categories need to be monitored during implementation. The fifth column defines the intangible benefits that may be derived from the program. When listed here, the intangible benefits are only anticipated; they must be measured in some way to determine whether they have actually been influenced by the program. Finally, the last columns detail other influences that may affect implementation and offer a space for additional comments, respectively.

EXHIBIT 3-5. Data Collection Plan

EXHIBIT 3-6. ROI Analysis Plan

Communication and Implementation Plan

The communication and implementation plan details how the results will be communicated to various groups, which groups will receive the information, and the specific schedule of events and activities connected to the other planning documents. It should include the method of communicating, the content of the communication, and the timing for the communication. The plan also defines the rationale for communicating with the group and for anticipated payoffs, along with the individual responsibility for monitoring actions from the evaluation. It clearly delivers the information to the right groups to ensure that action occurs; in almost every impact study there are significant actions that can be taken.

FINAL THOUGHTS

This chapter explored the alignment of employee engagement and evaluation planning. It described in detail when and how an ROI analysis should be considered as a process improvement tool. Using the V-Model, a step-by-step explanation was provided to properly align employee engagement programs with business needs. Special attention was paid to integrating needs analysis with the ROI Methodology, ensuring that the employee engagement practitioner has adequate tools and understanding to conduct a needs assessment and plan for the evaluation. Finally, the role of planning for an ROI project was presented, detailing the key steps in the process through an actual case study and illustrating how planning documents are used.