As we’ll see in our case studies and explore at greater length in the final chapter, there are settings in which metrics, in its various forms, works well. But there are many circumstances in which metric accountability is more dysfunctional than functional, or in which its costs outweigh its benefits. How should we account for the gap between the effectiveness of the culture of measurement, accountability, and transparency, and its ubiquity? Given its many drawbacks, why is it so popular?
While there is no single answer, and no hard proofs, here are some informed guesses.
DISTRUST OF JUDGMENT
The demand for measured accountability and transparency waxes as trust wanes. There is an elective affinity between a democratic society with substantial social mobility and greater ethnic heterogeneity, and the culture of measured accountability. In societies with an established, transgenerational upper class, the members of that class are more likely to feel secure in their positions, to trust one another, and to have imbibed a degree of tacit knowledge about how to govern from their families, giving them a high degree of confidence in their judgments (whether or not that confidence is justified).1 By contrast, in meritocratic societies with more open and changing elites, those who reach positions of authority are less likely to feel secure in their judgments, and more likely to seek seemingly objective criteria by which to make decisions. And numbers convey the air of objectivity; they imply the exclusion of subjective judgment.2 Numbers are regarded as “hard,” and thus a safer bet for those disposed to doubt their own judgments.
Numerical metrics also give the appearance (if one does not analyze their genesis and relevance too closely) of transparency and objectivity. A good part of their attractiveness is that they appear to be readily understood by all. As the Cambridge literary scholar Stefan Collini has observed, “public debate in modern liberal democracies has come to combine utilitarian valuations with a distrust of procedures that are not mechanically universalizable.”3
The quest for numerical metrics of accountability is particularly attractive in cultures marked by low social trust. And mistrust of authority has been a leitmotif of American culture since the 1960s. Thus in politics, administration, and many other fields, numbers are valued precisely because they replace reliance on the subjective, experience-based judgments of those in power. The quest for metrics of accountability exerts its spell over those on both the political left and right. There is a close affinity between it and the populist, egalitarian suspicion of authority based on class, expertise, and background.
The demand for greater “accountability,” which we saw reflected in the Google Ngram, fed upon the growing distrust of institutions and resentment of authority based on expertise that marked the United States (and to a considerable degree, other Western societies) from the 1960s onward. “Every profession is a conspiracy against the laity,” wrote George Bernard Shaw in his play, The Doctor’s Dilemma. Beginning in the 1970s, what for Shaw had been a bon mot increasingly became the operative assumption of public policy. The right and left looked to metrics, though not always for the same reasons.
The suspicion of authority was intrinsic to the post-1960s political left: to rely upon the judgment of experts was to surrender to the prejudices of established elites. Thus, the left had its reasons for advancing an agenda that professed to make institutions accountable and transparent, using the purportedly objective and scientific standards of measured performance.
On the right there was the suspicion, sometimes well founded, that public-sector institutions were being run more for the benefit of their employees than their clients and constituents. In some schools, police departments, and other government agencies, time-serving was indeed a reality, even if not as predominant or universal as its critics alleged. The culture of metric accountability was an understandable attempt to break the stranglehold of entrenched gerontocracy. When institutional establishments came under populist attack, they too resorted to metrics as a means of defense to demonstrate their effectiveness.
In a vicious circle, a lack of social trust leads to the apotheosis of metrics, and faith in metrics contributes to a declining reliance upon judgment. In a series of books, Philip K. Howard has argued that the decline of trust leads to a new mindset in which “[a]voiding human choice in public decisions is not just a theory … but a kind of theology…. Human choice is considered too dangerous.” As a consequence, “Officials no longer are allowed to act on their best judgment”4 or to exercise discretion, which is judgment about what the particular situation requires.5 The result is overregulation: an ever tighter web of rules, including the proliferation of rules within organizations.6 Often enough, metrics provides the tools for tightening that web. Over-measurement is a form of overregulation, just as mismeasurement is a form of misregulation.
Another motive for measuring performance is the fear of litigation as a result of the expansion of liability in American tort law. In the course of the twentieth century, earlier doctrinal barriers against suing doctors, hospitals, manufacturers, and municipalities broke down. The expansion of civil rights and environmental law further encouraged litigation.7 In employment, civil rights laws put new burdens of record-keeping and red tape on private companies as well as government agencies.8 The result: more and more money is spent on lawyers. And the perception of the United States as a litigious society9 creates an anxiety about the possibility of being sued, leading to defensiveness and risk-aversion. The urge to document every decision in the most objective way possible, so that hiring and promotion decisions can be made transparent to regulatory authorities, or used in case of litigation, provides another motivation for measuring performance.
THE CRITIQUE OF THE PROFESSIONS AND THE APOTHEOSIS OF CHOICE
On the political right, the mistrust of public-sector institutions led to the oft-stated conviction that the problem with the nonprofit sectors (government, schools, universities) is that they have “no bottom line” and hence no way of accounting for success or failure. To this way of thinking, the solution is to create a substitute bottom line in the form of “objective”—and preferably numerical—measures of standardized processes.
A parallel trend came from advocates of women’s health and later movements that challenged established institutions (such as physicians) and sought to make them more responsive. They looked to give patients greater control over their medical care. That entailed giving them a greater choice of providers, and more information—including performance metrics—to inform those choices. The road to empowerment was paved with metrics.
In one field after another, the introduction of greater measurement in the name of accountability did shine light upon real problems, including variations in professional practice that were supposedly grounded in “science,” and gaps in performance that had previously gone unnoticed or undocumented. The impact of these revelations both diminished faith in professional judgment and created pressure to find solutions, solutions thought to entail greater measurement in order to monitor the professionals whose ethos had been cast into doubt.
Closely related to these trends was the rising influence of the ideology of consumer choice, the belief that once provided with information, people will make the right choice when it comes to medical care, education, retirement planning, and so forth. Often, indeed, individuals are most capable of deciding on the best provider of services. But not always, and in some domains choice is particularly fraught. In healthcare, for example, choices pertaining to physicians or hospitals are made either when patients are healthy and disinclined to bother with medical matters, or when they are sick and therefore more anxious about their decisions, which diminishes their ability to process complex and often conflicting metrics. Yet by the 1990s, despite a number of studies indicating that patient empowerment did nothing to contain costs or improve quality of care, the model of the patient as consumer in the marketplace of medical services became ever more popular among politicians and policymakers on both sides of the political spectrum10
THE COST DISEASE
Another impetus for accountability in the fields of medicine and education stems from the fact that the relative cost of these services has risen compared to the costs of most consumer goods. Part of the reason lies in “the cost disease,” a phenomenon first identified by the economists William Baumol and William Bowen in 1966. They observed that the past hundred years had seen steady increases in productivity in manufacturing, largely the product of improved technology.11 As technological developments and the intensification of global trade has led to ever-declining costs of most consumer goods, the relatively higher costs of medicine, education, and similar human services have become ever more salient—and a focus of increasing public discontent. Over the years, these trends have led to public pressure for greater efficiency and greater accountability—despite the difficulty of measuring inputs, outputs, and hence productivity in these fields.12 Add to this the fact that improvements in medical technology and more effective pharmaceuticals may legitimately add to costs: their added costs may well be worth it if people live longer or more pleasant lives and need to spend less time in the hospital.
LEADERSHIP AMID ORGANIZATIONAL COMPLEXITY
Other economic forces are also at play in the push for quantifiable measurements. As organizations (companies, universities, government agencies) become larger and more diversified, there is an ever greater remove between top management and those further down the organizational chain engaged in the actual activities to which the organization is dedicated. When institutions are particularly large, complex, and made up of dissimilar parts, that comprehension is simply impossible. Those at the top face to a greater degree than most of us a cognitive constraint that confronts all of us: making decisions despite having limited time and ability to deal with information overload. Metrics are a tempting means of dealing with this “bounded rationality,” and engaging with matters beyond one’s comprehension.
Imagine, for example, that you become the president of a large university, corporation, or cabinet department. You might, of course, rely on the informed opinion of experienced subordinates. But they are likely to have an intrinsic interest in the status quo: recall the dictum of the late poet and historian Robert Conquest—“Everyone is conservative about what they know best.” But what if you want to inject dynamism or change into an organization whose leadership you have just assumed (and this is the typical temptation of new cabinet secretaries, university presidents, and CEOs who long to “make a mark”)? Then getting your hands on “the numbers” seems like the most direct shortcut to comprehending your organization.
The problem is that management’s quest to get a handle on a complex organization often leads to what Yves Morieux and Peter Tollman have dubbed “complicatedness”: the expansion of procedures for reporting and decision-making, requiring ever more coordination bodies, meetings, and report-writing. With all that time spent reporting, meeting, and coordinating, there is little time left for actual doing.13
This drain on time and effort is exacerbated by the tendency of executives under the spell of metric fixation to distrust the experienced judgment of those under them. They are more willing to try to control subordinates through a variety of strategies, of which metrics is a central component. The demands for a constant stream of reports and standardized data have the effect, intended or inadvertent, of diminishing the autonomy of those lower in the organizational hierarchy—whose doubts about metrics-based innovations are dismissed as irrational or as a self-interested “resistance to change.”
Then there are the cultural peculiarities of some American bureaucracies (corporate, governmental, and nonprofit), which assume that each person can and should be rotated through an ascending hierarchy of posts, both within an organization and among organizations. This militates against developing a depth of expertise that would allow for meaningful evaluation of the significance and qualitative importance of work done by subordinates. Hence the attractiveness of relying on measurable, quantitative, criteria.
CEOs, university presidents, and heads of government agencies move from one organization to another to a greater degree now than in the past. A strange, egalitarian alchemy often assumes that there must be someone better to be found outside the organization than within it: that no one within the organization is good enough to ascend, but unknown people from other places might be.14 That assumption leads to a turnover of top leaders, executives, and managers, who arrive at their new posts with limited substantive knowledge of the institutions they are to manage. Hence their greater reliance on metrics, and preferably metrics that are similar from one organization to another (aka “best practices”). These outsiders-turned-insiders, lacking the deep knowledge of context that comes from experience, are more dependent on standardized forms of measurement. Not only that, but with an eye on their eventual exit to some better job with another organization, mobile managers are on the lookout for metrics of performance that can be deployed when the headhunter calls.
THE LURE OF IT
Yet another factor is the spread of information technology (IT). In the early 1980s the invention and rapid adoption of the electronic spreadsheet and the resulting ease of tabulating and manipulating figures had wide-ranging effects. As a prescient analyst of the phenomenon, Steven Levy, wrote in 1984,
The spreadsheet is a tool, but it is also a worldview—reality by the numbers…. Because spreadsheets can do so many important things, those who use them tend to lose sight of the crucial fact that the imaginary businesses that they can create on their computers are just that—imaginary. You can’t really duplicate a business inside a computer, just aspects of a business. And since numbers are the strength of spreadsheets, the aspects that get emphasized are the ones easily embodied in numbers. Intangible factors aren’t so easily quantified.15
Seth Klarman, among the most successful value investors of his generation, concurred, warning in 1991 that spreadsheets created the illusion of depth of analysis.16
Since then, the growing opportunities to collect data, and the declining cost of doing so, contribute to the meme that data is the answer, for which organizations have to come up with the questions. There is an often unexamined faith that amassing data and sharing it widely within the organization will result in improvements of some sort—even if much information has to be denuded of nuance and context to turn it into easily transferred “data.”