You will not find a one-size-fits-all intervention to improving performance. So it follows that there is not a one-size-fits-all method or model to evaluating interventions. To illustrate this point, the following case study demonstrates the use of multiple interventions to address the performance goals of an organization. This case covers how to address each intervention of an evaluation project.
The case, which represents the work of Redwood Mountain Consulting in San Jose, CA, is excerpted from a case study by Jim Fuller in ASTD's Performance Interventions: Selecting, Implementing, and Evaluating the Results, edited by Brenda Sugrue and Jim Fuller.
The director of sales for a large corporation approached the training department requesting help to improve sales to new accounts. The trainers persuaded the director to set the training request aside until identifying the actual causes of the performance problem.
The formal performance analysis identified a number of issues (ranging from accounting processes to significant conflicts in the use of the rewards and recognition system) that created barriers to performance for the sales reps. Because of the extensive issues, numerous solutions were necessary to address the root cause barriers. The director of sales had a goal of increasing sales to new accounts by 15 percent. Twelve months after the implementation of the interventions, the organization had achieved a 30 percent increase.
The training manager used a standard performance analysis approach, which included the following tasks:
identifying the business needs
identifying required performance of employees
quantifying the performance gaps
identifying root causes of performance gaps.
Although the director only requested training, the trainers also knew what the organization needed to achieve: a 15 percent increase in sales to new accounts. Together, the director and trainers agreed that the evaluation metric for the performance-improvement effort would be sales into new accounts with a goal of 15 percent (or $150 million) in the first year.
To achieve this business need, a specific performance was required of the sales force: They needed to call on new accounts. Given the average rate for closing sales, each sales rep would need to call on three new accounts each week.
In addition, to obtain the time required to call on the new accounts, the sales reps needed to use the new automated quotation system to reduce their weekly paperwork. Finally, for the sales calls to have any chance for success, a strong advertising program needed to be in place in the sales territory.
The results of the performance gap analysis showed that sales reps were using the new quotation system extensively and were saving time. The quotation system was therefore removed from the analysis process. The gap analysis showed that the sales reps were implementing almost no advertising efforts. And because experience had shown it to be a waste of time without the advertising in place, reps were not making calls on new accounts.
The training manager randomly selected several sales reps to participate in a focus group discussion on the reasons why reps were not advertising. Their results identified three major performance barriers:
The sales reps clearly did not know anything about advertising. They did not know what constituted good advertising, how to create it, where to place it, how to get it placed, how frequently to run it, or how to create a plan. Advertising simply was in neither their educational nor experiential background.
The organization had a very cumbersome approval process for expenditures over $500. Because almost any advertising effort required more than $500, the sales reps would have to fill out several forms and obtain several signatures each time they wanted to advertise in their territory. Due to the long process, sales reps who did advertise usually had to place the advertising expenses on their personal credit card and wait for reimbursement from the organization.
The sales reps were compensated monthly based on sales performance compared to sales quota. A sales dollar was a sales dollar, regardless of what account it came from, or what product or service a rep sold. Most sales reps made quota with existing accounts. Selling into a new account required more time and effort than selling into an existing account. Given the compensation system, there was no incentive to sell new accounts.
The training manager examined the performance necessary to run a local advertising campaign. The original three-day training request fell far short of the training that would be required. After calling some training providers, the training manager put the actual training requirement at a full two weeks. The costs of such a program were simply prohibitive. Taking the sales reps out of the field for two weeks has a lost opportunity cost of approximately $40 million (if sales reps are in class, they are not selling).
The training manager proposed the following three alternative solutions that would eliminate the need for training.
By hiring two roving advertising experts, the sales reps could explain their advertising objectives, and the experts could create both the plan and materials as well as implement the plan. This solution freed up even more time for sales reps to call on new accounts, and the advertising specialists would be able to implement better advertising than any of the sales reps.
The cost for two specialists was far less than the training investment. For sales reps who insisted on managing their own advertising, the organization supplied them with turnkey advertising materials, which require no creative effort. Finally, a 20-page how-to booklet on advertising was developed, complete with examples to follow.
The expense approval solution was a bit more complex because the director of sales did not have control over the process. A quick conversation between the director of sales, the financial controller, and the training manager, however, resulted in a workable solution. For advertising expenses, the no signature required limit was raised from $500 to $5,000. For advertising expenses more than $5,000, an email message from the director to the finance department would suffice for authorization to reimburse the sales rep. Finally, the finance controller offered to have local TV advertising billed directly to the organization.
The compensation system was more difficult. The director of sales did not understand why the compensation system needed adjustment. He felt that his command to sell into new accounts should be sufficient motivation. After a long discussion about the human performance system, the director was able to see that the compensation system undermined his authority. The director of sales and the training manager developed a number of fixes for the compensation system, all within the control of the director. This adaptation did not require changes to the formal system.
New accounts require more time and effort than selling into established accounts. To compensate for this fact, sales into new accounts would count as 120 percent of quota credit. This did not fully cover the increased investment, but it did decrease the performance barrier significantly.
New accounts that were still doing business with the organization one year later would result in the sales rep receiving a $1,000 bonus. It was estimated that the average sales rep could easily earn a $20,000 bonus this way. In addition, a sales award was announced. The sales rep with the highest new account sales would receive a twoweek trip for two anywhere in the world. Finally, if a sales rep had poor new account performance, it disqualified him or her from earning other sales awards or bonuses, regardless of performance.
The interventions required to improve new sales performance spanned the three traditional categories of solutions:
The finance system clearly placed environmental barriers in the way of running advertising programs.
The sales reps lacked the skills and knowledge necessary to implement the training.
The compensation system did not create motivation to sell into new accounts.
The interventions also spanned the three levels of performance analyzed by HPI practitioners:
Performer: The compensation system was directly targeted at changing the motivation and performance of the individual sales rep.
Process: Processes were changed for the approval and reimbursement of advertising expenses.
Organization: The sales organization changed as a result of the creation and staffing of a new advertising department with roving advertising specialists (a completely new position within the company).
It took two full months to implement the interventions. The finance manager was able to make the changes in the process in less than a week. The changes to the compensation system took the longest. The information systems needed to track new account performance and determine pay. This required the longest amount of time.
The director of sales established a clear performance measure at the beginning of the performance improvement project. The organization wanted to achieve an overall 15 percent increase in sales through new accounts. The organization tracked new account sales (less than one year old) for two years prior to the performanceimprovement project and continued after the implementation of the project. This primary goal required neither a new data-collection strategy nor a reporting strategy. The sales reps exceeded the original goal of 15 percent, as new account sales went more than 30 percent in the first year.
To measure progress toward the sales goal, the training manager and the director of sales selected additional evaluation measures. These measures also were metrics that the organization was already capturing and reporting. They selected this metric because calling on new accounts is a known predecessor to new account sales.
Advertising was a necessary component of successful new account sales. The training manager also tracked the volume of advertising (measured in dollars) as a predecessor to new account sales.