When I told one of the readers of the original manuscript of The Tyranny of Metrics that I was writing a preface to this paperback edition, he suggested that we emblazon the cover with “Now with More Tyranny!” The proposal was made in jest. But the more I thought about it, the more plausible it seemed. Since the book’s original publication, I’ve received a steady stream of reactions from people from a variety of professions and walks of life, with the common theme, “The situation is even worse than you present.” They say that what I’ve termed “metric fixation” is more widespread, more deeply entrenched, and more oppressive than the book suggests.
The book is not about the evils of measurement of human performance, or of rewarding achievement, or of transparency. It is about “metric fixation”—a perversion of these, based upon beliefs that seem reasonable at first, but turn out to be unreasonable in practice. The title, The Tyranny of Metrics is not meant to convey the message that metrics are intrinsically tyrannical, but rather that they are frequently used in ways that are dysfunctional and oppressive.
The book tries to articulate in social scientific terms the sense of frustration and dissatisfaction that many people have with the organizations in which they work, even when—indeed, especially when—they are supportive of the fundamental goals and purposes of those organizations. It tries to give voice to what many say soto voce among colleagues, but are loathe to proclaim publicly, lest they be identified by their organizational superiors as enemies of progress—where progress is defined in terms of metric fixation.
The book is difficult to pigeonhole: in discussions with my editor about how to classify it, we made plausible cases for “current affairs” (since it deals with matters of public policy); for sociology, since it deals with social structures; for political science, since it deals with power and control; for economics, since it deals with incentives; and for organizational behavior, since it explores motivation within organizations. The book also draws upon history and the philosophy of the social sciences—since much of it is concerns changing beliefs about what counts as authentic and inauthentic knowledge, including the belief that numbers are objective, scientific, and reliable, whereas judgment is suspect.
It is also, in its way, a business management book. But as I came to realize after finishing the book, it is a book about management as seen in good part from the perspective of the managed, which sets it off from most books in that genre. And unlike most business books—which tend to be upbeat and chirpy, promising some new technique that will revolutionize one’s organization—this book is more critical in tone, promising less of a cure-all than an antidote to the snake oil sold under the labels of measurement, accountability and transparency.
The chapters in the middle of the book explore the proper use and frequent misuse of metrics in a variety of fields, including education, medicine, policing, the military, and business. Those chapters are intended to be illustrative of the broader processes and problems that the book anatomizes. They are by no means intended to be definitive. Once they’ve understood the broad patterns, informed readers from any of these fields will be able to add many more examples of their own, while others will identify additional areas in which metric fixation takes its toll.
Indeed, some readers of the book have brought to my attention its relevance to sports and advertising. Readers who were either professional baseball scouts or amateur lovers of the sport wrote to me about the transformation of major league baseball by the increasing dominance of metric fixation. Metrics have long played a role in the management and coaching of the sport, but their use has become ever more predominant in recent years. The role of “sabermetrics” was chronicled by Michael Lewis in his book Moneyball (2003), which recounted the way in which the management of the Oakland Athletics baseball team used metrics to raise the status of the team. Since then, it was determined by metric analysis that teams were more likely to score by means of home runs, rather than by a series of base hits. Instructed by analysts schooled in metrics, batters are now taught to focus on the “launch angles” that are most likely to result in a home run, as opposed to a base hit, and are willing to accept more strikeouts. The result has been the metric rationalization of professional baseball. The game is far more regular: there are fewer base hits, hence fewer runners from base to base, and fewer bases stolen. Yet it was the irregular elements of the game that provided much of its excitement—those men running around the bases, and the suspense of whether they would make it home. The result is a game that is more boring to watch, resulting in diminished audiences. As Alan Jacobs, an informed but now disillusioned fan of the sport, noted in a review of my book, major league baseball provides a telling example of the tendency of metric fixation to orient “our questions and thoughts and concerns in the direction of existing techniques of measurement and assessment.”1
Trends in contemporary advertising illustrate the phenomenon of “measurability bias,” the tendency to prefer options simply because they can more easily be measured. Rather than engage in brand development by advertising in a wide range of media, for example, companies increasingly prefer to advertise only in venues that provide “direct response” in the form of clicking on links, on the grounds that these can be measured, while the effect of billboards, television advertisements or newspaper ads cannot. Based on the logic that only that which is measureable is worth doing, companies are moving their ad expenditures to online platforms such as Google and Facebook, who control ad tracking on the internet. That, in turn, has led to massive ad fraud, in the form of “bots” which defraud advertisers by creating fake websites that draw clicks and hence ad revenue.2
Measurability bias is also ever more prominent in foreign aid, where as in so many areas, it leads to a focus on short-term results over long-term benefits. Take the case of the development of water resources, a pressing issue in many nations. At one time, American development aid was focused on helping establish water management systems—complex activities required to provide sustainable ongoing water supplies in countries with rising demands from rapidly growing urban populations. These included the development of dams and irrigation systems, and policies that would incentivize conservation and discourage hoarding, pollution, and the cultivation of crops that overtax existing water supplies. But under Congressional pressure for measurable results, and performance-based funding cycles, USAID was re-oriented toward the measurable rather than the important. Instead of funding long-term, complex water management activities, the State Department increasingly focuses on sanitation projects—taps and toilets—that are easily counted.3
The distorting effects of the pressure for measurable results is evident too in the realm of philanthropy, to which a brief chapter of the book is devoted, but where there is much more to be said. Philanthropy is increasingly dominated and influenced by foundations such as the Bill and Melinda Gates Foundation. Established by those who have made their fortune in technology and finance—fields in which measured performance is the coin of the realm—the founders carry over those preferences to their philanthropies, emphasizing measureable results. The effect is to aim primarily at what can be most readily measured, at the expense of improvements that are difficult to measure or are influential only in the long run. These philanthropic giants, in turn, set the tone not only for smaller philanthropies, but for government agencies in the field on international development.
As careful readers have noticed, while focused on contemporary trends in the use and misuse of measurement, The Tyranny of Metrics links up with much older themes and broader contemporary concerns. It touches upon the perennial issue of the role of practical wisdom acquired through experience, as opposed to techniques that can be taught. It addresses the distinction between human affairs given to systematization and abstraction, where decisions can be made based on general rules, versus situations where good decision-making demands particular, contextual knowledge. The book tries to cast a critical light on some contemporary trends, including the hazards of managerialism and scientism in contemporary organizational life. That is to say, it calls into question the notion of management conceived as a set of metric techniques that can be easily packaged, and champions management as a craft and art, learned in good part through practice and talent. Though it draws most of its specific examples from the United States and the United Kingdom, the trends in organizational culture that it critiques are increasingly international.
When numbers, standardized measurement of performance, and big data are seen as the wave of the future, professional judgment based upon experience and talent are seen as retrograde, almost anachronistic.
Human judgment—based on talent and experience—has become unfashionable. Measurement is in. Scholars in the field of behavioral psychology delight in demonstrating that biases lead us to misestimate numerical values and probabilities, casting doubt on judgment. Those who belong to groups that may have suffered from social bias identify judgment with prejudice, and prejudice with unwarranted discrimination. To them, “objective” metrics comprised of “hard” numbers may seem like an antidote. Measuring performance, making those metrics public, and rewarding persons and institutions accordingly is frequently embraced as the silver bullet that will solve problems in education, medicine, policing, and other public institutions.
From yet another direction, management gurus tout the unbeatable effectiveness of algorithms based on big data. Consultants are quick to recommend that the solution to organizational deficiencies demand more data.
Add to that the lure of information technology. The growing opportunities to collect data, and the declining cost of doing so, contribute to the belief that data is the answer, for which organizations have to come up with questions. There is an unexamined faith that amassing metric data and sharing it widely within an organization will result in improvements of some sort. So who needs judgment based upon experience and talent?
The contention of this book is that you do.
NOTES TO THE PREFACE
1. Alan Jacobs, “The Tyranny of Metrics,” The New Atlantis, March 10, 2018.
2. .I thank Professor David Carroll of the Parsons School of Design for these insights.
3. David Reed, “In Search of a Mission,” in David Reed (ed.), Water, Security and U.S. Foreign Policy (New York, 2017); and more broadly Dan Honig, Navigation by Judgment: Why and When Top-Down Management of Foreign Aid Doesn’t Work (New York, 2018).