5	When Measurement Obstructs Good Judgment

MEASURES AND THE MEASUREMENT process, especially badly designed or unnecessarily complex measurement systems, are among the biggest barriers to turning knowledge into action. In our field research, we encountered example after example of measurement processes that fueled destructive behavior inside organizations. What is even more striking, however, is that when we encountered counterproductive measurement practices, managers often recognized and complained bitterly about them and described to us why and how they should be changed. Yet the use of such unproductive measures persisted.

This chapter has three objectives. First, we document the problems that measurements create in turning knowledge into action. Second, we show reasons why organizations persist in using flawed measurement practices, even when their leaders are aware of doing so, and what this fact implies about closing the gap between knowing and doing. Third, we present examples of organizations that have used measurements that produce intelligent behavior, and from these examples infer some general principles that distinguish measurements that cause problems from those that help organizations solve problems. It is clear to us that merely knowing what measurement practices should be used does not, by itself, cause leaders to implement measures that produce intelligent, mindful, learning behavior rather than the reverse.

Measures That Create Problems

Everyone knows that measures focus attention on what is measured. Everyone also knows that because what is measured is presumed to be important, measures affect what people do, as well as what they notice and ignore. As a consequence, everyone knows that what gets measured gets done, and what is not measured tends to be ignored. The importance of measurement is news to virtually no one. Moreover, there is evidence that measures have powerful effects on behavior even when they are not coupled with rewards. People want to do well on dimensions that are important to their organizations, even if there is no immediate consequence, other than social status, that accrues as a result of scoring well on the measures.¹

You might think, given this common knowledge, that firms would routinely use measurement systems that cause their people to pay attention to issues that top managers say, and know, are important for performance given their specific business strategy. You might think that they would focus their measurements on elements of management practice, business strategy, and firm culture that truly matter for long-term performance. And, you might think that firms would recognize the commonsense wisdom expressed in a line from Otis Redding’s song “Sitting by the Dock of the Bay” on the need for fewer, focused measurements: “Can’t do what ten people tell me to do, so I guess I’ll remain the same.” Yet, firm after firm fails to implement these well-known and commonsense principles.

To illustrate, recruiting and retaining people with the right technical skills is critical for high-technology firms. People costs are a large proportion of total costs in these knowledge-intensive businesses. Top executives in the computer software and hardware industries regularly complain about the scarcity of talent and the shortage of skilled people. You might think that, given these business realities and the shortage of knowledge workers, there would be well-developed measures of the costs, benefits, and consequences of turnover, training, and other employment practices. But a study of workforce management practices in the Silicon Valley by A. T. Kearney found that “most employers do not understand their total cost of employment.” Moreover, “most employers do not specifically track their training or recruitment costs.”² Even though skills, training, and employment are critical to business success, most firms have virtually no measures relevant to their performance on these dimensions. Why is this neglect of measurement, or failing to measure the right things in the right ways, so pervasive? Frederick Reichheld, a Bain consultant who wrote The Loyalty Effect, said this about why firms neglect these measurement issues and about the problems this neglect creates:

The most aggressive minds in an organization rarely focus on measurement systems…. Leaders, they feel, should concentrate on important, exciting things like vision and strategy, and let the people with the green eye shades worry about measurement. The trouble with this attitude is that measurement lies at the very heart of both vision and strategy…. Most executives today work with inherited measurement systems that distort their business strategies.³

What follows is a small set of examples illustrating common measurement problems. These cases help us understand how measurement systems and practices contribute to the creation and persistence of knowing-doing problems. We see in these examples that measurement practices caused these organizations not only to have gaps between knowledge and action, but also to act in ways that were the reverse of what their executives believed should be done.

Focus on Short-Term Financial Performance: Problems at Hewlett-Packard

There is an ongoing struggle for the soul of Hewlett-Packard. On the one hand, the firm was founded and has flourished on the basis of strong values, the HP Way, and has long been recognized for its strong culture. Many management books have cited Hewlett-Packard as an outstanding organization and argued that the HP culture was an important source of the firm’s success.⁴ The company is still ranked quite high in surveys of the best places to work⁵ and enjoys turnover much smaller than the norm in the Silicon Valley. On the other hand, Hewlett-Packard has recently had difficulty in maintaining its growth and in meeting Wall Street’s earnings expectations. In fact, the company recently announced it was splitting up into two parts. Many of its businesses, such as computers and printers, have become increasingly price competitive, and the firm has been under pressure to increase its margins. The result is that the firm is being torn between its values and beliefs and a set of short-term financial pressures, made real by a set of measurement practices that emphasize results in the present at the expense of being prepared for the future.

In the very competitive labor market for engineers in the Silicon Valley, the ability to attract and retain talent is crucial to HP. Moreover, because HP is operating in very competitive, high-technology industries with shrinking product life cycles, intellectual capital and talent are critical to the firm’s continuing success. So, the firm knows that “morale governs motivation which is key to timely product development; strong culture fosters a healthy work environment” and that “employees need rewards for key contributions and successes.”⁶

Yet, there is pervasive evidence that the firm’s commitment to acting on the basis of what it knows has been sorely tested and that the company’s measurement practices have induced pressures inconsistent with its culture and with doing what it needs to do for long-term success. Numerous senior managers in Hewlett-Packard have argued that the company has drifted away from some of the practices and principles that made it a great firm and a great place to work. A survey in one division assessing employee morale on a five-point scale (in which 5 is high) showed a decline of 1.2 points, or almost 25 percent, for both engineers and managers over a 12-month period. The same survey revealed that 59 percent of the engineers and 75 percent of the managers thought that advancement opportunities had decreased in the last year, 76 percent of the engineers and 50 percent of the managers thought adherence to the HP Way had decreased, and 76 percent of the engineers and 75 percent of the managers felt that rewards had decreased.⁷

The measurement process, in which achieving targeted budget results in order to meet the quarterly expectations of Wall Street is paramount, has contributed to the company’s doing a number of things that are potentially inimical to its long-term business health. For instance, in the division that was the source of the data on morale, the preceding two years had seen two laboratory-wide reorganizations, a 40 percent reduction in the manufacturing workforce, imposition of stringent expense controls, and the virtual elimination of employee celebrations and rewards. People in the division claimed that HP was more concerned with the financials than with the people. But in a labor market that offered many options, less reward for accomplishment and lower morale increased turnover and decreased work performance, making the division’s prospects even worse.⁸

Some of Hewlett-Packard’s other current management practices, promoted by its measurement system, have also had negative effects. These practices include using a lot of outsourcing, including contracting out manufacturing and software programming, using temporary workers, and not recruiting for talent far enough in advance to ensure the company is able to attract the best people. We should be clear that these practices are, in fact, inconsistent with basic elements of the firm’s culture and values. The company explicitly talks about its tradition of not using contract and temporary help. For instance, during a half-day seminar on the company, its culture, and its founders, one summer intern “learned about HP’s historic avoidance of using temporary workers (outside contractors).” The same intern heard a similar account from long-time employees: The company had typically avoided hiring temporary employees in areas central to its business. There was a business case, not just cultural values, for this policy:

They [Hewlett and Packard] firmly believed that if a company provided a great work environment, trusted its employees, and gave employees the authority to make decisions, then the company’s employees would value this commitment and develop loyalty to the company resulting in higher morale and more productivity than could be accomplished by other companies.⁹

Recently, however, the company has outsourced more and more of its manufacturing activities and has come to rely more on temporary employees and contract programmers. At one point, in the early 1990s, the company was featured on a 60 Minutes television program for its use of illegal aliens furnished by a so-called body shop. Hewlett-Packard has been investigated by the U.S. Immigration and Naturalization Service and the Labor Department because of its use of contract workers from India.

What’s wrong with outsourcing manufacturing and using temporary help and contract labor (who typically earn much less and do not receive benefits) in order to make budget numbers? In the first place, “outsourcing important electronic components” means that “HP no longer has a manufacturing feedback mechanism in-house from which it can gain information about the quality of its hardware designing capabilities in terms of manufacturability, quality, defects and costs.”¹⁰ Outsourcing manufacturing can in the end inhibit the ability to learn about product design because of the separation of design from manufacturing across organizational boundaries. Perhaps even more important, as a case study concluded, this practice sends some important symbolic messages:

Another cost of increasing the use of outsourcing… is that it sends a subtle message to division managers throughout the company that costs, revenues, and efficiency metrics are more important than preserving employment opportunities within the company. It also means that short term planning becomes more important than long term planning…. Deciding to outsource… because of short run inadequacies in comparison with outside vendors… necessarily implies that these inadequacies will never be… addressed internally. In adopting a short term business view, this type of mentality lends itself to hiring temporary workers to fill short term demand fluctuations…. Temporary workers become more important when division managers begin to focus more on meeting short term financial and production goals than on long term planning on how to utilize permanent employees…. The focus becomes more tactical and less strategic.¹¹

The focus on making the budget numbers not only produces behavior that is inimical to developing long-term capabilities and contrary to the history and culture of the company, but also some behavior that is almost unethical. For instance, the case study also found:

There is a lot of counterproductive budgetary gaming behavior that occurs to make the numbers seem better than they otherwise would. An example… was one manager “parking” funds with an outside vendor during one quarter to be spent on projects in a future quarter. The purpose of doing this was to even out the costs from quarter to quarter, but what it amounted to was prepaying an outside vendor three to four months ahead of time…. Managers will [also] use temporary workers indefinitely, rotating new people in every so often, rather than hiring a permanent employee and blowing the head count budget.¹²

The lessons from Hewlett-Packard are sobering. Even an organization with a strong culture and long tradition and history of valuing people can get into trouble when there are intense pressures to “make the numbers” and a measurement system that emphasizes primarily those numbers. These pressures arise both outside the firm, in the form of demands from the capital markets, and inside, because of a divisionalized structure in which people compete with each other to meet their budget targets. Blind adherence to making the budget leads to all kinds of games in setting the budget targets in the first place and then to doing numerous things, some of which are harmful to the firm and its development of capabilities in the long term, to obtain short-term results that meet or exceed targeted expectations. And, a measurement system that is so strictly financially focused leads to neglecting other aspects of the company and its development. Interviews with numerous managers at various levels inside the company repeatedly come back to the budgeting system as a major cause of the firm’s loss of its culture, history, and traditions. Few have made convincing arguments as to what the company has gained to correspond to these losses, as many echo the sentiments of the quotations above concerning the company’s loss in terms of the ability to develop and build long-term knowledge and competence. What is also striking is how widespread these views are, as well as the pervasive sense of the inevitability of the situation.

The Hewlett-Packard example also illustrates another important point: The time scale of the measurements—how often the firm assesses results—helps to establish the time horizon that tends to govern behavior in the organization. A quarterly budget focus at HP has produced an emphasis on the short term and actions designed to manage performance that take a short-term perspective. Time horizons are not inevitable but are, in fact, the result of organizational measurement practices.

Overly Complex Measurements: Citibank and a Large Financial Institution

Citibank, along with many other companies, has embraced the idea of the balanced scorecard, as the company’s published material and cases written on the firm make clear. In principle, the balanced scorecard makes a great deal of sense. Rather than just measuring and evaluating managers on the financial performance of their units, which largely reflects what has happened in the past, the scorecard emphasizes getting ready for the future. Managers are assessed on dimensions such as customer satisfaction, employee attitudes and development, and new products and services—all dimensions that are important in affecting the current and future performance of the firm, not just on measures of short-term financial performance. A second advantage is that the balanced-scorecard approach recognizes that a firm’s measurement system reflects the theory of the business. What ought to be measured are those elements of organizational operations that are the most crucial for performance, reflecting the firm’s theory of the ultimate causes of performance. What does get measured is what the organization attends to in any event, so it becomes the theory under which the firm operates. So, the measurement system either reflects or becomes the firm’s implicit theory of what affects performance. This, too, is enormously sensible, because it is common sense to measure those aspects that are the most consequential for affecting the success of the organization.¹³

Nonetheless, even though it is great in theory, a number of problems often emerge with the balanced-scorecard approach. These problems include the following:

1. The system is too complex, with too many separate measures.
2. The system is often highly subjective in its actual implementation.
3. Precise metrics often miss important elements of performance that are more difficult to quantify but that may be critical to organizational success over the long term.

Consider, first, the issue of complexity. Most organizations use at least four categories of measures and have numerous indicators within each category. As a consequence, balanced-scorecard systems frequently become enormously complicated. For instance, a Harvard Business School case study reported that California branch managers at Citibank faced the following scorecard measures in 1996:

FINANCIAL:

Revenue

Expense

Margin

STRATEGY IMPLEMENTATION:

Total households

New to bank households

Lost to bank households

Cross-sell, splits, mergers households

Retail asset balances

Market share

CUSTOMER SATISFACTION

CONTROL:

Audit

Legal/Regulatory

PEOPLE:

Performance Management

Teamwork

Training/Development

Self

Other

Employee Satisfaction

STANDARDS:

Leadership

Business Ethics/Integrity

Customer Interaction/Focus

Community Involvement

Contribution to Overall Business¹⁴

Each component of the Scorecard was scored independently into one of three rating categories: “below par,” “par,” or “above par.” For those measures that could be measured quantitatively—financial, strategy implementation, customer satisfaction, and control—pre-defined performance thresholds determined where performance fell in this three-level scale. However, ratings related to people and standards lacked an appropriate objective indicator: in these cases, performance was determined subjectively by the branch manager’s superior.

In addition, the manager’s boss gave a global rating for each of the six components of the Scorecard and an overall rating for the branch manager….¹⁵

At Citibank, to ensure that the Scorecard was meaningful, bonuses were tied to the ratings:

… a branch manager’s bonus was tied to his or her final Performance Scorecard rating. A “below par” rating did not carry any bonus. A “par” rating generated a bonus of up to 15% of the basic salary…. An “above par” rating could mean as much as 30% bonus.

Without “par” ratings in all the components of the Scorecard, a manager could not get an “above par” rating.¹⁶

Note that there are more than 20 separate indicators of performance and six categories in the balanced scorecard as it was implemented at Citibank. Nor is this unusual for companies using the balanced scorecard. Mobil Oil’s balanced-scorecard system, used to determine compensation, has a similar level of complexity. This is a lot of information to keep in focus. Also, being rated below par even in one category means that regardless of how well a person did in other categories, he or she cannot obtain the highest performance bonus. This is particularly a problem because ratings in many of the categories are so subjective.

In a large financial institution we studied, one successful branch manager objected to the balanced scorecard’s inherent subjectivity even as it appeared to be objective:

It’s still very subjective. My boss who does my rating sits down and talks to me on [a] quarterly basis about team work, my people management skills, my ethics, things like that. For the first two quarters of this year, I’ve been rated “par” in people. But meanwhile, I’ve given up my sales manager to be a branch manager in another branch. I’ve given up a personal banker to be a branch manager in another branch. I just gave up another personal banker. So, I’m developing talent, and people are taking my talent, but as a people manager, I’m just par.

[My boss] came to that judgment because I have one person in my branch that she thinks I should be probably managing a little harder… but [my boss is] not here to visit and see what I do and… doesn’t really know how I manage these people, because maybe I don’t put them on documentation.

Because of the complexity of the balanced-scorecard system, people can’t really focus very well on a few key dimensions. Research indicates that human beings can keep only about seven things in their heads at any one time.¹⁷ Having more than 20 indicators of performance in six categories dilutes the attention employees can pay to any single issue or even a small set of issues. With so many indicators and measures, it is easy to simply ignore the whole thing because of the impossibility of comprehending so many dimensions at once. The behavioral effect of the scorecard is further diluted because of the appearance of subjectivity. People believe the ratings are subjective, biased, and not necessarily valid, so they don’t believe the ratings are fully under their control. They conclude that their performance does not necessarily translate into good ratings because of the subjective aspects of the system. Consequently, it is, again, easy to simply disregard the entire system. In response to a question about the effects of the scorecard, a person who had worked on developing the scorecard system and who was considered to be a good branch manager said:

I don’t think it’s really changed people’s behavior. In the beginning, people were excited about the scorecard because it seemed like an opportunity to make a lot more money. When that didn’t happen it kind of went: “this is just like SBL [another management change initiative at the bank]. I wasn’t getting money then, I’m not getting money now. I’m still getting my 4% merit increase. Life goes on.”

I think that some people are judged differently from others. When you allow someone subjectivity, what happens is that they tend to favor certain people not maybe knowing all the details. My boss manages probably 30 other people and can’t spend a whole lot of time interacting with me. We’ll have a meeting once a week for two or three hours. My boss doesn’t visit my branch all that much. We don’t have phone conversations. But yet, at the end of the quarter, [my boss] can look at my numbers which is half of my scorecard and then the other half is all… perceptions based on maybe 12 hours of interaction during the quarter. I don’t really think that’s fair—whether I benefit from it, which in most cases I do, or whether there’s a lot of other people who don’t benefit from it.

Finally, the balanced-scorecard examples illustrate another phenomenon we frequently observed: Hard measures drive out the soft, and precise metrics can miss important elements of performance. For instance, the branch manager quoted above was apparently reasonably successful in identifying and developing managerial talent, since people from his branch were often promoted to positions in other locations. This is an important part of being a good leader—being successful in the people development process. But he didn’t get much credit for this success. It would be possible, of course, to just add another quantitative indicator to the system, for instance, the number or proportion of people from one’s direct reports promoted to higher positions. But that would make the system even more complex and, in any event, misses the point. It is often difficult to specify in advance every dimension and every aspect of managerial performance. Trying to obtain precise metrics of performance can miss critical, subtle elements of performance, elements that vary across people and that cannot always be defined in advance, let alone objectively and precisely measured.

The lessons from several financial institutions’ experience with the balanced scorecard are that, at the end of the day, effective measurement systems that will drive behavior need to be simple enough to focus attention on key elements and fair enough so that people believe they can affect the measures. Also, the measurement systems cannot be so powerful in directing people that important elements of behavior and performance that are not, and cannot be, fully captured in the measures receive too little or no attention because of the unrelenting emphasis just on the quantitative measures incorporated in the system. No measurement system is going to perfectly capture all of the important elements of performance or all of the behaviors that people need to do for the organization to be successful. So, measurements should be guides, helping to direct behavior, but not so powerful in their implementation that they substitute for the judgment and wisdom that is so necessary to acquire knowledge and turn it into action. This “light touch” in the implementation of measures is particularly important because such systems are invariably going to be changed much less frequently than the rate at which new circumstances arise and new knowledge of how to enhance organizational performance develops. Given this difference in cycle times, firms need measurement systems that don’t restrict or inhibit the development and implementation of performance knowledge.

Although implemented in many companies with the best of intentions, the balanced scorecard often fails on these dimensions. Important elements of performance knowledge not captured in the formal measurement system get neglected, for instance, the people development process. Measures come to focus attention to such a degree that intuitive wisdom and tacit knowledge cannot be readily implemented. At that point, the good intentions don’t matter. Either behavior is largely unaffected, people are demoralized, the wrong behavior is encouraged, or some combination of these unanticipated outcomes occurs.

In-Process versus Outcome Measures: General Motors

Why is it so hard to implement good manufacturing practices, such as those embodied in lean or flexible production, even though virtually every automobile executive knows that such systems produce higher-quality cars at lower cost? As we reported in Chapter 1, a study of the diffusion of flexible manufacturing systems over a five-year period found only modest implementation of lean manufacturing systems, and in some plants, these management practices had been taken out. The measurements that are made—and not made—by automobile firms contribute to these implementation problems. Managers at General Motors told us that part of the problem is that the company uses too many end-of-process measures—measures that tell them how well they have done—and not enough in-process indicators and controls that help them understand what is going right and what is going wrong. So, learning at General Motors and elsewhere is inhibited because companies are measuring the wrong things and not gathering data that permit them to really understand, manage, and control the process. In that regard, budgetary figures, costs, and even the balanced-scorecard measures are too far removed from processes in many instances to guide behavior and permit knowledge to be developed and turned into action.

Tom Lasorda, a senior executive at General Motors charged with the responsibility of improving manufacturing operations by implementing flexible production techniques, was articulate on this issue:

We are outstanding at providing what I call end of process measures. So, what was your absenteeism? What was your first time quality number? What was your scheduled production? What was your cost per car, and all that stuff? Now walk it back from there and say, “give me your in-process measures.” And from an individual level of how you relate to those, you won’t find it as detailed as you will in other companies. Can I go to this station and find the SPC [statistical process control] charts that are controlled by the individual, and the standardized work charts that are being checked by the individual? If you had that level of detail what you will find is a far more robust organization with fewer measures, I might add, at the end of the process. Because the company that is not in control has far more measures because they’re not changing the basic management systems that are in place.

We’ve got to shrink the number of measures, get the organization focused on them, and then build a pyramid that says, “these are primary measures, progress measures, and in process measures,” so everybody can link their job to the overall measures.¹⁸

The irony is that the end-of-process measures cause those subjected to them to feel a great deal of pressure, to feel tightly controlled. The problem is that they are not being measured or controlled on the things that really matter and things that they can directly affect—their specific behaviors and actions on the job. So, people at General Motors may feel more tightly controlled than those at Toyota, for instance, even though the process is actually much less under control at GM and even though there is much more control over specific work practices at Toyota. The lesson is that measuring people on outcomes and not giving them in-process measurement can help create stress and frustration, but often does not result in a more controlled or effective operations process.

Lasorda had a specific plan to overcome the measurement problem in the manufacturing part of North American operations:

It’s a set of measures that is much more refined so that everybody understands how he or she links to them…. Any company has an enormous number of measures. The ones you might see are the end of business measures. What did you do on profitability? Go down to each level of the organization and say, “Can you show me your measures?” What I’ve typically seen is 15 to 18 measures. And then you’d say to them, “What are we doing in-process?” And there would be nothing.

So here is what I have said to the organization. We’ve developed what we call the Business Plan Deployment Strategy, which says what are the key business plan measures that are going to affect the viability of the enterprise. Let’s focus on those and how you drive those into the organization. And how do you engage people in business issues and work on trying to improve performance against the goals…. All of a sudden, you start engaging people in the business where before they really didn’t understand the measures and how they related to them.¹⁹

The General Motors example teaches us that real control does not come simply from having a plethora of outcome measures. Control and improvement come from measures that provide information about processes, measures that give people immediate and understandable information about how they need to act. In this instance, these lessons seem to have been learned and are being implemented. But the learning came with difficulty. Lasorda and others first needed to overcome the belief that the best measurement systems assess mostly end-of-process outcomes—a perspective that fails to take into account that measurements are only useful when they can actually guide behavior.

Why Poor Measurement Practices Persist

The negative consequences of the measurement systems we described above were widely known and understood by people in each of these firms, including top management. Similarly, in dozens of other cases that we need not repeat here, the story is the same: Numerous people in the organization understand and are able to explain in articulate detail that they are using a flawed set of measurement practices. So, the question becomes, If measurements create problems in developing and implementing knowledge, if some measurement practices make the knowing-doing gap worse, and if people in the firms involved know this, why do these ineffective measurement practices persist?

There are no easy answers to this question. But our conclusion from doing our own research and from reading pertinent work by others is that there are three interrelated processes that explain why these flawed measurement practices persist:

1. Many companies operate using an oversimplified or incorrect model of human behavior.
2. Because that model of behavior is widely shared, it has become institutionalized in certain types of measures and measurement systems, which have assumed a taken-for-granted quality, have become a signal of competent management (even if they are actually the opposite), and are so widely diffused that firms are reluctant not to follow them.
3. The primacy of the capital markets and shareholder concerns creates pressures for measurement practices that are relevant to shareholders’ interests but may be irrelevant or even counterproductive for the ultimate success of the business.

The model of behavior implicit in the measurement systems used by most firms is that individuals are atomistic and economic, rather than social, creatures. The atomistic view is captured in having measures for each individual. This procedure presumes that (1) individual results are the consequences of individual decisions and actions and that (2) individual outcomes and individual behaviors are under the control and discretion of these individuals, so that results and decisions can be reasonably reliably attributed to individuals. Think back to the Citibank balanced-scorecard system and its associated bonus plan. Each individual in a managerial role receives his or her own scorecard measures. That presumes that performance, as assessed by the scorecard or, for that matter, by any other measurement system, results from individual decisions and behaviors—which is why individual incentives are tied to the measures. For this measurement and incentive scheme to be sensible, it is also implicitly assumed that it is possible to make valid judgments about performance using dimensions of the scorecard. In other words, the process assumes that performance can be assessed and assigned to individuals.

But if there is one thing that we know for certain, it is that organizations are systems in which behavior is interdependent. What you are able to accomplish, and indeed, what you choose to do and how you behave, is not solely under your individual control. Rather, your behavior and performance are influenced by the actions, attitudes, and behaviors of many others in the immediate environment. Consider the Citibank balanced-scorecard example. Is teamwork, one dimension along which people are evaluated, just under the influence of the branch manager? Or is teamwork also a consequence of many organizational management practices, including rewards and measurements, that the branch manager can’t affect very much? How much effect does that person, by himself or herself, have on customer satisfaction? Customer satisfaction is the result of interactions with numerous other people and of decisions that the branch manager doesn’t even fully control. One branch manager, in commenting on the Gallup survey results used to assess customer satisfaction, noted:

They don’t tell you the questions but they tell you the answers, and even when they tell you the answers, it’s not very specific. They’ll say, the lines were too long. Well, the lines were too long at what time? Maybe there’s a reason. Today if you told me my lines were too long this afternoon I’d understand why. It is because I had to send two people home sick. So, if a customer came in and that customer actually got polled, and it’s someone who happened to come in today, the person is going to say that the lines were too long.

The fiction in most measurement systems is that individual performance measures assigned to individuals presumably reflect the effort and skill those people used in doing their jobs. But individual performance in an interdependent system will always be difficult or impossible to measure. Individual performance and behavior, even if they could be accurately assessed, are the result of many things over which the person has little or no control, as the above example nicely illustrates.

Measurement practices are heavily institutionalized, which means that they are taken for granted and used in a mindless way. There is a profession, called accounting, that has made a business of developing and institutionalizing certain measures, even if they are the wrong ones. Consider the following. How much variation do you see in companies’ business strategies? A lot. How much variation do you see in organizational cultures? A lot. How much variation do you see even in company incentive systems? Again, a great deal. Now consider, how much variation do you see, not just in public financial reporting, but even in management accounting measures and practices? Not much. Does that make sense to you? Does it seem sensible that firms that vary dramatically in their strategies, their cultures, their incentive schemes, and so forth, should all have and use essentially similar managerial measurement and reporting systems? As long as accountants have control of internal measurements, not much will change. We have nothing against accountants, but are simply noting that they are pursuing a different set of goals. Specifically, we have seen few accountants or controllers who worry about the effect of measurement systems on turning knowledge into action or on the organization’s ability to develop and transfer skill and competence.

And this brings us to our last, interrelated source of the problem—the primacy of capital market interests. Public accounting is first and foremost designed to ensure that investors have accurate and consistent financial information on which to base their decisions. Of course, management accounting is presumably separate and distinct from public financial reporting, and information systems and measures could, in theory, be designed to accomplish anything that managers wanted. But public companies have an audit committee, and the public accounting firms have to assure the audit committee that the company’s financial controls are sound. So even internal audits and financial controls soon become focused on the public, capital market aspects of financial reporting as developed and enforced by public accounting firms. For good or for ill, this capital market influence further ensures uniformity of practices. Since there is much less uniformity of business conditions, culture, and strategy, this uniformity in measurement practices almost guarantees measurements that have little connection to the particular business problems and issues confronting a specific organization. But even in the face of institutionalized pressures and generally shared models of behavior, some firms have been better able to get their measures right. The situation isn’t hopeless. Let’s consider some examples.

Using Measures to Enhance the
Development and Use of Knowledge

We found that a simple principle was applied in firms in which the measurement systems helped—rather than undermined—the ability to turn knowledge into action. Such firms measured things that were core to their culture and values and intimately tied to their basic business model and strategy, and used these measures to make business processes visible to all employees. For example, at Wainwright Industries, a privately owned manufacturing company that won the Baldrige Award in 1994, “measures are simple, visual indicator systems to operationalize the goals so everyone can tell at every moment whether or not their actions are producing the desired results.”²⁰ Also, in most instances the measures used were aggregate, measuring the results at a group, subunit, or organizational level rather than attempting to do the impossible by assessing the performance and contributions of individuals working in interdependent systems.

In a sense, the underlying premise of the balanced score-card is right—measures should embody a theory of organizational performance. At Wainwright Industries, the five key indicators are, in order of importance, (1) safety, (2) measures of employee (internal customer) satisfaction and continuous improvement, (3) customer satisfaction, (4) quality, and (5) financial performance. The relative importance of the various measures reflects leaders’ beliefs about how the company and its business works. Quality is comparatively low in importance because “at Wainwright, quality is an automatic result of employee satisfaction, training and involvement.” Similarly for financial performance: “Strong business performance results from employee satisfaction, training, involvement, customer satisfaction, and highest product quality.”²¹ There are two key differences between what these firms did and the balanced-scorecard approach: (1) The measurement systems are far simpler and understandable, and (2) they took mindful actions to build measures that suited their needs rather than mindlessly adopting measures that were well institutionalized in accounting firms. These organizations recognized the tradeoff between having a measurement system that captured the full complexity of the sources of performance and having a system that focused attention on fewer, but the most critical, aspects of operations and culture.

Measures Linked to Cultural Values and Philosophy: The Men’s Wearhouse

To understand the wisdom of a company’s measurement system, it is first necessary to know something about its business model and culture. The Men’s Wearhouse sees its salespeople as professionals—which is why it spends so much time and effort training them—whose goal is to develop an understanding of customers’ wardrobe needs and to cultivate a high-service relationship so that the company becomes the source to fill those needs. Their sales philosophy is described in their training materials:

The Men’s Wearhouse sales philosophy is consistent with the Company’s goal of creating Win-Win-Win situations for our customers, wardrobe consultants, and the Company…

The customer wins because we have consultants. As team-oriented, professional consultants, we seek to create a quality relationship with the customer…. Unlike other stores with “clerks,” The Men’s Wear-house is able to show and assist a gentleman with an entire wardrobe concept. This helps him to look and feel better, while saving time and money.

Since you have met the customer’s clothing needs well beyond those he initially identified, you have made a great sale and a customer for life…. The Men’s Wearhouse wins with every customer that is delighted by the quality of service, the value of the clothing and the unique shopping experience.²²

Charlie Bresler, one of the most senior executives at the company, in his orientation talk at Suits University, emphasizes the importance of team selling and personal relationships with other people in the company:

My most important job is really maintaining an environment in our stores which is a positive work environment…. As a wardrobe consultant, you are expected to define your success in part as only achieved when your teammates, the sales associates, the tailors, and other wardrobe consultants and management people in the store are also successful… and that you will, over time, define your success not only in terms of your own goals, but also the goals and aspirations of the other people in your store. And that you will come to really care about them as human beings and as people who finally realize their potential, too.²³

The company has over 400 stores throughout the United States. Maintaining this culture and philosophy is a challenge. But the firm has developed measures that help them accomplish this task. To sit with Charlie Bresler and look at sales figures for wardrobe consultants in each store is to learn how the measurement system helps the company enforce and track its culture. One important measure he and the other leaders use is the number of transactions for each wardrobe consultant in the store. This number should be approximately the same for all of the people in a given store. If it is not, it means that someone is not sharing walk-in business equally with the other people. To hog customers is to not display the proper team spirit, and people who persist in this behavior get fired, even if their sales volumes are high. The second number is the average size of the ticket, or sale. The company actually has a higher commission for transactions over $500 because of the belief that wardrobe consultants, not clerks, are able to learn about an individual’s wardrobe needs and fill more of those needs, particularly by selling accessories such as shoes, ties, and shirts. This ability to “accessorize,” measured from the sales tickets, helps managers assess training needs and the success of their efforts to produce sales professionals who truly are wardrobe consultants.

To reinforce the cultural values, the firm’s performance appraisal system is focused on specific behaviors—behaviors that can be coached and learned. For instance, in addition to specific sales figures and measures that indicate how much cross-selling of merchandise the person is doing, some of the other dimensions on the appraisal include “Participates in team selling; ensures proper alteration revenue collection; treats customers in a warm and caring manner; contributes to store maintenance and stock work; is familiar with merchandise carried at local competitors; greets, interviews, and tapes all customers properly; and contributes to store maintenance and stock work.”²⁴

The Men’s Wearhouse has developed quantitative and more qualitative but still behaviorally specific indicators that closely match its intended culture, values, and business model. The quantitative measures are produced routinely by the firm’s management information system and are used to identify problems and to recognize successes. By making the measures behaviorally specific, the company has come closer to actually having in-process measures that can be taught, learned, and implemented, as contrasted with the more typical outcome or end-of-process measures.

Few, Simple Measures: SAS Institute

SAS Institute, a software firm providing statistical analysis, data mining, data warehousing, and decision support software, has a business model that emphasizes relationships. “Rather than selling a product and then selling upgrades on a regular basis, SAS offers an annual licensing arrangement after a thirty-day free trial period that provides for free upgrades and customer support…. Over time, revenues from a given customer will be higher, as long as the customer renews the licensing agreement [emphasis added].”²⁵ It is also in an industry in which knowledge and intellectual capital are critical. Its measurement practices reflect both these business imperatives and the philosophy of the firm, as expressed by James Goodnight, one of the co-founders and currently the CEO: “What we tried to do was to treat people who joined the company as we ourselves wanted to be treated…. If you take care of your people, they will take care of the company.”²⁶

SAS Institute believes that people should work at a place because they enjoy the work and the people they work with—financial incentives are less important. Consequently, the company offers no one stock options or phantom stock. People are paid good, competitive salaries, and at the end of the year there is typically a small bonus (less than 10 percent of salary). Merit raises are given once a year. There are no short-term individual incentive pay schemes—no sales commissions for account executives, for instance. The company deemphasizes individual competition by not posting comparative sales data by name. Barrett Joyner, vice president of North American sales and marketing, had this to say about incentive schemes and their associated measures:

We have sales targets, but mostly as a way of keeping score…. We’re big on a long-term approach. I’m not smart enough to incent on a formula. People are constantly finding holes in incentive plans…. A lot of incentive plans represent ways of signalling to people what they were supposed to do and to emphasize…. Here, we just tell people what we want them to do and what we expect.²⁷

The firm has done away with its performance appraisal forms. David Russo, the vice president of human resources, commented:

If there were a good performance appraisal process, everybody would be using it…. I don’t think you can really manage someone’s performance. I think you can observe the results. I think you can give them the tools. I think you can set short and long-term goals. And you can sit back and see if it happens or it doesn’t happen…. Our idea is to have performance management be based on conversation instead of documentation.²⁸

Although apparently a radical idea, a number of other companies—including the GM Powertrain Group and Glenroy, Inc., a privately held manufacturer of packaging materials—have eliminated the annual performance review measurement with good success:

Too many leaders confuse feedback with paperwork. “Filling out a form is inspection, not feedback,” says Kelly Allan… “History has taught us that relying on inspections is costly, improves nothing for very long, and makes the organization less competitive.”²⁹

Jim Goodnight of SAS sees a simple one-page financial report each month. Managers are evaluated primarily on their ability to attract and retain people. The company monitors turnover very carefully, and in 1997 had voluntary turnover of only 3 percent—a fraction of what is typical for the software industry. The company also tries to ensure that it is a great place to work, and takes the results of surveys and studies such as Fortune’s list of the 100 best places to work in America (it ranked third) very seriously.

In a relationship-oriented business based primarily on intellectual talent, SAS encourages long-term relationship behavior through its measurements and through what it chooses not to measure and make public. In a place in which the attraction and retention of talent is key, turnover and factors related to the building of talent are what the firm measures. The emphasis, even in a geographically dispersed organization of 5,000 people, remains on interpersonal communication—an emphasis consistent with the relationship-oriented business model and philosophy. You have relationships with people, not with reports or numbers.

Using Measures to Maintain Focus on What’s Important: Intuit

Intuit is a large firm that develops and sells financial planning software for both individuals and small businesses—the best-known titles are TurboTax, Quicken, and QuickBooks— implemented initially in both the Macintosh and PC environments and now on the Internet as well.³⁰ The firm today has about $600 million in annual revenue and 3,000 employees. The company has a set of operating values that include the following:

• Integrity without compromise
• Do right by all our customers
• It’s the people
• Seek the best
• Continually improve processes
• Speak, listen, respond
• Teams work
• Customers define quality
• Think fast, move fast
• We care and give back

In 1998, three-quarters of the employees responding to the firm’s annual employee survey agreed with the statement “Intuit lives up to the corporate operating values.”

The company has faced numerous challenges, including increasing product market competition, the development of Internet-based financial planning services, an attempt by Microsoft to buy the company that was stopped by antitrust concerns, and the demands of recruiting and retaining a talented workforce in the Silicon Valley. The company is publicly traded and offers stock options as a recruitment and retention strategy, and so is concerned about its stock price. It would be easy to get diverted from doing what it knows it should. How has Intuit stayed focused? Mostly through its employee survey. Employees know the survey matters, so the response rate is quite high—73 percent in 1998. And, the company uses the survey results to guide its behavior. Between 1995 and 1998, the proportion of people agreeing with the statement “We care and give back” doubled to 77 percent. Of course, the survey by itself is of little value. Many companies have surveys and don’t do anything with the results. Intuit people and the firm’s cultural values are what make the surveys work. At the same time, generalized cultural values and intentions to be a certain way, without measures to assess how well the firm is doing, could easily become merely good intentions that do not drive action. The values, culture, and quality of the people provide the knowledge and intentions, and the surveys help turn that knowledge and values into action.

In 1995 employee morale was not where Intuit wanted it to be, so the company refocused on its people. Intuit moves around valuable employees, even at the cost of short-term efficiency, to reinforce its commitment to career development for its people. Moving people to different units also helps to reinforce the team culture by causing individuals to identify more with the company as a whole than with just small units. When an employee complained about not getting a raise, he received one a few days later after a thorough review by management. The company reemphasized its social activities and celebrations, and did not let financial challenges divert its focus from building morale and spirit.

Intuit uses both its formal employee survey and informal feedback to quickly identify gaps between its aspirations, embodied in its core operating principles, and what it is doing. Then, leaders move quickly to address these gaps. The measurement system at Intuit provides an ongoing check on how the company is living up to its values—values that reflect what managers know to be related to the firm’s ultimate financial success. The measurement, nothing more than an employee survey, but one that is taken very seriously, affords a way of focusing managerial effort on those dimensions of the culture that most need attention at a given moment. At Intuit, the measurement system helps to reinforce and build the culture and to implement practices that leaders know are vital to the firm’s success.

Measures That Produce Change: Sears

One response we often hear when we talk about firms that have fewer knowing-doing gaps is, The examples you have given are companies that have done things correctly from the beginning. Can firms ever change? How? These questions make the Sears story particularly interesting, because Sears confronted difficulties that were surmounted in large part through changes in measurement practices. But before we get to this case, we need to also confront a potential misperception: The examples we have provided, such as SAS Institute, Intuit, and The Men’s Wearhouse, did not implement the right measurement practices from the beginning. In each instance there was, and is, ongoing learning about what worked and what didn’t. In all these cases, mistakes occurred and were used as information about changes that needed to be made, not as bad news that should be swept under the rug or used as a reason to blame or punish individuals. So, rather than being used to instill fear in these organizations, both the process of developing the measurement systems and the measurement practices themselves are used to support learning and to drive out fear. Measurement practices and measures evolved as the organizations developed a better understanding of their business model, a better-defined culture, and more clarity concerning their guiding philosophy. What distinguishes these examples from many others is not that they did things perfectly from the beginning but their willingness to focus measurement practices on the important task of turning knowledge into action. And, the leaders of these firms were willing to break with convention and place less emphasis on what everyone else was doing and measuring.

In the case of Sears, the reinvention of measurement practices resulted from a financial crisis. In 1992, Sears had revenues of $52 billion, “nearly 9% lower than 1991 revenues of $57 billion, and lower than annual revenues in each of the three preceding years. Sears generated a net loss from continuing operations of $1.8 billion and a total net loss of $3.9 billion.”³¹ The financial problems were, in large measure, the result of Sears’ failure to put knowledge of successful retail practices into action. As for the knowledge of retailing part, Anthony Rucci, the company’s head of human resources, noted:

The basic elements of an employee-customer-profit model are not difficult to grasp. Any person with even a little experience in retailing understands intuitively that there is a chain of cause and effect running from employee behavior to customer behavior to profits, and it’s not hard to see that behavior depends primarily on attitude.³²

The problem was that as competition, from Wal-Mart, Kmart, and Target, among others, had intensified over the preceding years, Sears had made changes in response to competitive pressure and declining profits that were inconsistent with this knowledge of how retailing worked. These changes created a continuing cycle of poor performance and ineffective responses:

In the 1960s and 1970s, you could feel success as you walked through the stores. Managers and employees were loyal, energetic…. Employees were mostly full time and they really knew their product…. Then came the competition and profitability declined…. Everyone down to hourly-paid supervisors… had been on incentives, but over time those incentives were eliminated…. Corporate also reduced the number of central administrative support groups…. The stores got less support. They also eliminated several positions in the store. As a result, there were fewer knowledgeable salespeople out on the floor, and store managers ended up doing less merchandising and employee coaching and more paperwork and logistics.³³

Sears’ problems, manifested in an “intense internal focus,”³⁴ were magnified by a measurement system that was also internally focused and focused on the past. The traditional cost accounting and financial measurement system assessed costs and profit margins that had already been achieved, but did not register the diminishing levels of both employee and customer satisfaction.³⁵ So, the measurement system could not assess the damage being done to the firm’s long-term viability by actions being taken to increase profit margins and decrease costs in the short run, actions that included putting more part-time and less-experienced—but cheaper—people on the floor to serve customers. Most important, the measurement system did not reflect the essential retailing business model of the link between employees, customers, and profits. As a result, measurement practices hindered the implementation of what Sears knew about how to enhance its performance.

Arthur Martinez arrived in the fall of 1992 and led a remarkable transformation at Sears that drove up both operating results and its stock price. Martinez took a lot of short-term measures to enhance service, such as emphasizing training, putting the best people in the stores during evenings and weekends when the best customers were shopping, offering Sunday deliveries, and similar steps. Martinez also set in place a strategy of cultural transformation, based on a change in measurement practices, that would ensure that the changes at Sears would last. The company developed a vision called the 3Cs or the “three compellings”: Sears was to be a compelling place to shop, a compelling place to work, and a compelling place to invest. These objectives were translated into specific metrics. A compelling place to shop was measured by overall customer satisfaction and customer retention. A compelling place to work was measured by attitudes about the job and the company. And a compelling place to invest was measured by revenue growth, operating margins, asset utilization, and indicators of productivity improvement.³⁶ Sears’ measurement practices were based on the desire to develop leading, not lagging, indicators of performance. The measures were validated in econometric studies that quantified exactly how much effect improvements in various dimensions had on other indicators.

Refinement of the measurements and the model predicting performance is part of an ongoing process that continues to the present. Measures change as more knowledge becomes available. Like Intuit, Sears began to pay more attention to employee surveys, partly because the company wanted to retain and motivate good people and partly because of the demonstration of the empirical connection between employee attitudes, customer attitudes, and store performance. Executives in Sears believe that the changes in measurement practices were crucial to the firm’s turnaround:

The point is that we know vastly more than we once did, that all that information helps us run the company, and that some of it has given us a decided competitive edge…. Our model shows that a 5 point improvement in employee attitudes will drive a 1. 3 point improvement in customer satisfaction, which in turn will drive a 0.5% improvement in revenue growth…. These numbers are as rigorous as any others we work with at Sears. Every year, our accounting firm audits them as closely as it audits our financials.³⁷

The changes in the measures at Sears permitted the firm to recapture and implement what it once knew—the connections between management actions, employee attitudes, customer attitudes and loyalty, and profits. In a company that had for years taken measures of employee attitudes because of a commitment to the importance of its people, managers, with the help of the new measurement model, reaffirmed the importance of Sears’ people to the firm’s success. As one result, frontline people received much more training in understanding the business, the firm’s strategy, and the competitive environment. And, employees were given more say and authority in helping the company regain its competitive footing. Just as one set of accounting measures had driven the company off course, a different set of measures, more embedded in the actual business model and operating processes of the retail business, helped the firm recover.

Measurement That Turns Knowledge
into Action

Based on the examples of both good and bad practices we have seen in this chapter, we can conclude that measurement practices that help organizations develop knowledge and turn that knowledge into action typically have the following properties:

• The measurements are relatively global in their scope, focusing less on trying to assess individual performance, which is always difficult in interdependent systems, and more on focusing attention on factors critical to organizational success.
• The measures are often focused more on processes and means to ends, and less on end-of-process or final outcomes. This focus results in measures that facilitate learning and provide data that can better guide action and decision making.
• They are tied to and reflect the business model, culture, and philosophy of the firm. As a result, measurement practices vary from one firm to the other as the business imperatives, cultures, and philosophies vary. And, in measuring things such as adherence to values, recruitment and retention, and working cooperatively with others, the measures depart from conventional accounting-based indicators.
• The measures result from a mindful, ongoing process of learning from experience and experimentation. There is not the sense that the system is ever completed. Rather, there is the view that the measurement system can always be improved and, because the business environment is likely to change, practices that are effective now may be ineffective in the future. Measures evolve to serve a fundamental core business and operating philosophy or strategy that is more constant.
• The measurement process uses comparatively few metrics. Although these firms may collect a large amount of data, they emphasize and attend to a small set of measures that are believed to be especially crucial for supporting the company’s business model, philosophy, and culture. A focus on critical issues and processes is emphasized at the expense of comprehensiveness and complexity—things that dilute attention and that mix the important with the trivial.
• At its best, measurement closes the loop, auditing and assessing what the organization is doing, thereby ensuring, as in the case of Intuit, that the firm does what it knows.

Unfortunately, just knowing these guidelines for designing better measurement practices does not ensure that they will be implemented. We have shown that there are substantial barriers to implementing these principles, regardless of their wisdom or validity, not the least of which is the accounting industry and conventional accounting and measurement practices. But there is some cause for optimism. Some of the firms in which we have seen the most effective measurement practices are publicly traded. So, a company need not be private or even owner-controlled to implement measurements that help it build and implement knowledge. At The Men’s Wearhouse and SAS Institute, leaders are mindful that their philosophies and business models run contrary to the conventional wisdom in their industries, and believe that a focus solely on the short term will undermine long-term performance as well as sacrifice other values these firms hold dear. In the case of Sears, a financial crisis gave leaders an opportunity to break with the immediate past and to develop a measurement system that supported their business model. In other instances, the measurement approach arose from a particular business philosophy and culture that wouldn’t tolerate either rampant short-termism or measuring things that didn’t matter. In each of the instances in which effective measurement practices were used, knowing what to do, why it needed to be done, and having the persistence and courage to do it helped leaders turn knowledge about how to enhance performance into organizational action.