How Other Industries Measure
Baseball. Baseball is one of the industries that show us that we could be measuring the wrong indicators when we’re looking to see either how we’re doing now or how well we’re going to do in the future. Such metrics could mean that we’re focusing on talent as opposed to performance. Talent is ability or potential, but the focus should be on performance. When we generate metrics, we want to measure performance—how well we’re doing, but not talent—how well we might do in the future. This is because we can only gauge how well we will do in the future by taking an honest account of how we’re doing now.
This is one of the reasons why baseball is an industry in which the way performance is measured has come under fire. Measuring performance in baseball should be one of the easiest tasks out there. You have players who are batting against the same players on the same fields. This means that generating statistics in baseball is fairly simple. The problem, however, is that sometimes we don’t know which statistics indicate future success and which do not. That is, we don’t know which statistics are relevant.
Michael Lewis’s hugely successful Moneyball took the question of which statistics are relevant in baseball head on. The central argument of Moneyball is that the metrics that baseball insiders (including players, managers, coaches, scouts, and the front office) have used over the past century have been flawed. In Lewis’s words, “Traditional yardsticks of success for players and teams are fatally flawed” (Lewis 2004). Some of the statistics that Bill James, one of the heroes of the book, derides as irrelevant are 60-yard dash times, RBIs, and batting averages. James showed that on-base percentage and slugging percentage are better indicators of future success and that getting a hit is less important than avoiding an out. In the end, every play in baseball can be evaluated (meaning that it can be given a metric) according to how many runs it will statistically contribute. An example of this is that a strike on the second pitch of an at-bat may be worth –.75 runs. This new piece of information (available only because new managers are focusing on relevant metrics) might fly in the face of the conventional wisdom, which is why getting people to truly believe in Bill James’s new managerial system was extremely difficult.
However, when managers of teams started listening to James it paid off. When the Oakland Athletics (with $55 million/year) implemented the Moneyball strategy, they became competitive with the New York Yankees (with $205 million/year). (Lewis 2004) The manager of the Oakland A’s, Billy Beane, the real protagonist of Moneyball, was the one who turned the team around. When the conventional wisdom said that you needed buff hitters and pitchers, Beane realized he could use James’s system of metrics to come up with a winning and affordable team. Talent (at least in baseball) sometimes gets in the way of performance (which should be the real goal of a baseball team’s manager), but Beane found a way, using metrics, to turn the tide on this system. The Moneyball story shows us again that one of the keys to good metrics is that they be relevant to performance in the industry where they belong and not just generally appealing or shocking. It also shows that sometimes the results of performance metrics will be counter-intuitive, so it will take a lot to get people (especially managers) on board, but it is often worth the fight.
Government. Even the government uses metrics to guide it in its decision making processes. Currently employed government-wide is the Executive Branch Management Scorecard, which is “used to show both how well a department or agency is executing the management initiatives, and where it scores at a given point in time against the overall standards for success.” (OMB 2001) Each agency is graded on whether they meet a number of criteria.
The Office of Management and Budget (OMB) “assesses agency ‘progress’ on a case by case basis against the deliverables and time lines established for the five initiatives that are agreed upon with each agency as follows:
Green (Success): Implementation is proceeding according to plans agreed upon with the agencies.
Yellow (Mixed Results): Some slippage or other issues requiring adjustment by the agency in order to achieve the initiative objectives on a timely basis.
Red (Unsatisfactory): “Initiative in serious jeopardy. Unlikely to realize objectives absent significant management intervention” (U.S. Government 2006).
Another set of government performance metrics has been developed by Robert S. Kaplan and David P. Norton; it is called the “balanced scorecard.” In an Office of Personnel management newsletter Kaplan and Norton compare the balanced scorecard “to the dials and indicators in an airplane cockpit. For the complex task of flying an airplane, pilots need detailed information about fuel, air speed, altitude, bearing, and other indicators that summarize the current and predicted environment. Reliance on one instrument can be fatal. Similarly, the complexity of managing an organization requires that managers be able to view performance in several areas simultaneously. A balanced scorecard or a balanced set of measures provides that valuable information.” (OPM 1996) In order to generate a balanced scorecard, information is gathered from four perspectives: (1) the customer’s perspective, (2) the internal business perspective, (3) the innovation and learning perspective, and (4) the financial perspective. (OPM 1996) This is an important lesson to learn for generating metrics across the board: A variety of perspectives should be used so that all metrics are balanced and thus, in Clark’s framework “credible.”
However, of course, public performance metrics will be quite different from private performance metrics. In Translating Performance Metrics from the Private to the Public Sector, Paul Arveson writes that all “governmental agencies exist not for profit but to fulfill their charter or mission, which is an ‘inherently governmental function.’ Hence, unlike private-sector businesses that can change in any way they please, government agencies are constrained to work within their authorized mission. On the other hand, private corporations are prohibited from engaging in some activities that are authorized for the government only; these exclusions are described in the Constitution.” (Arveson 1999) Therefore, there will be many differences between metrics designed for the private sector and those designed for the public sector. Arveson writes that “The key metric for government (or nonprofit) performance, therefore, is not financial in nature, but rather mission effectiveness. But mission effectiveness is not a definite and static thing.” (Arveson 1999) Arveson goes on to translate the performance metrics from the private to the public world. “For instance, competitiveness in the private world translates into mission effectiveness in the public world; profit, growth, and market share in the private world translate into cost reduction and efficiency in the public world; and innovation, creativity, and good will in the private world translate into recognition, accountability to public, integrity, and fairness in the public world.” (Arveson 1999) The lesson for generating metrics in general (and for generating HR metrics more specifically) is that before you generate performance metrics you must know what the purpose of the firm is (and this is a special case of Clark’s “relevancy” condition) and whether it is a private firm that must respond to market conditions or a public agency that has to respond to the will of the people, regulations, and its governmentally defined mission.