3

THE ORIGINS OF MEASURING AND PAYING FOR PERFORMANCE

“Accountability,” “metrics,” and “performance indicators” have become cultural memes. Embracing them promises a seat on the train of historical progress, and no politician, agency chief, university president, or school superintendent wants to be left behind. When metrics becomes the coin of the realm, to refuse to use it is to risk bankruptcy. There is pressure from elected officials and from foundation managers to pay up.

How and why did this tyranny of metrics come about?

SOME ORIGINS OF PAYING FOR MEASURED PERFORMANCE

The idea that organizations outside the free market would be more efficient if they were paid based on measured performance seems to have occurred first to policymakers in Victorian Britain. In 1862, Robert Lowe, a Liberal member of parliament who oversaw the committee on education, proposed a new method for government funding of schools, which would be based on “payment by results.” Lowe had distinguished himself in 1856 by shepherding through parliament a seminal piece of legislation in the history of capitalism. That was the Joint Stock Companies Act, which, together with legislation passed the previous year, the Limited Liability Act, set out a new law for corporations based on the principal of limited liability. From reforming the structure of business, Lowe turned to reforming government-supported schools.

Lowe’s scheme was based on the premise that “the duty of a State in public education is … to obtain the greatest possible quantity of reading, writing, and arithmetic for the greatest number.”¹ Schools were to be funded based on the performance of their students in the “3 Rs.” Each school was to be visited annually by a school inspector, who was to quiz every student in English language and arithmetic. For every student who failed to appear or to answer questions successfully, a small sum would be deducted from the school’s government funding. Lowe’s reform was intended in part to cut costs, but above all to make school funding dependent on measurable results in the most basic and practical of skills, and to bring education into accord with his market-oriented principles by linking payment to performance.²

Lowe’s scheme was challenged by Matthew Arnold, the great cultural critic, whose day job was as a government inspector of the very schools Lowe set out to transform. Arnold warned consistently against extending the criteria appropriate to the market to other areas of life. With a dose of bravery, Arnold launched a public salvo against his political superior. In an essay entitled “The Twice-Revised Code,” Arnold attacked the narrow and mechanical conception of education implied by the code. The ability to read intelligently, he pointed out, developed primarily not from narrowly tailored reading lessons, but from a more general cultivation, imbibed from the family or, failing that, from a school environment that created the mental desire to read. The goal of the schools, therefore, should be “general intellectual cultivation,” without which the skills of reading and writing would not develop.³ The government, he lamented, sought to fund only the most rudimentary of educations instead of responding to “the strong desire of the lower classes to raise themselves.”⁴ Since many impoverished students would inevitably be absent when the annual test was administered, or would fail the test itself, he predicted that the net effect of the proposed reform would be to reduce the funding of schools for the poor. The education of the people, he concluded, was to be sacrificed to “the friends of economy at any price.”⁵

Arnold frequently found himself inspecting schools where students ingested mountains of facts and arithmetic, but were bereft of analytic ability and utterly incapable of understanding sophisticated prose or poetry. They were taught not to reason but to cram.⁶ Both before and especially after the adoption of “payment for performance,” he criticized such education for being “far too little formative and humanizing … much in it, which its administrators point to as valuable results, is in truth mere machinery.”⁷ This conception of education as machinery, tailored to the measurable production of reading, writing, and computation, and capable of being rewarded based on measurable output, ebbed and flowed in the decades that followed, reaching a flood tide at the end of the twentieth century.

At each subsequent wave, we’ll encounter critics like Arnold, who pointed to the unmeasured costs of tying reward to standardized measurement.

MEASURING PERFORMANCE: TAYLORISM

There were traces of metric fixation in the school efficiency movement that rolled across the American educational landscape, starting in the 1910s and continuing for decades. In 1911, Simon Patten, an influential professor of economics at the Wharton School of Business, demanded that schools provide evidence of their contribution to society by showing results that could be “readily seen and measured.”⁸ Other would-be reformers sought to bring to the school system the fruits of the industrial efficiency movement, founded by Frederick Winslow Taylor, an American engineer who coined the term “scientific management” in 1911.⁹ Taylor analyzed the production of pig iron in factories by breaking down the process into its component parts (through time-and-motion studies) and determining standard levels of output for each job. Workers who carried out their tasks more slowly than the prescribed time were paid at a lower rate per unit of output; those who met the expectation were rewarded at a higher rate. Taylor also advocated an elaborate system for monitoring and controlling the workplace.¹⁰ His goal was to increase efficiency by standardizing and speeding up work on the factory floor to create mass production.

Specialization and standardization of tasks, recording and reporting of all activity, pecuniary carrots and sticks—these were the legacy of Taylor and his disciples to subsequent generations.

Taylorism was based on trying to replace the implicit knowledge of the workmen with mass-production methods developed, planned, monitored, and controlled by managers. “Under scientific management,” he wrote, “the managers assume … the burden of gathering together all of the traditional knowledge which in the past has been possessed by the workmen and then of classifying, tabulating, and reducing this knowledge to rules, laws, formulae…. Thus all of the planning which under the old system was done by the workmen, must of necessity under the new system be done by management in accordance with the law of science.”¹¹ According to Taylor, “It is only through enforced standardization of methods, enforced adoption of the best implements and working conditions, and enforced cooperation that this faster work can be assured. And the duty of enforcing the adoption of standards and enforcing this cooperation rests with management alone” (italics in original).¹²

Taylorist themes of the need for greater efficiency through standardization and monitoring were reflected in the widely influential textbook Public School Administration, published in 1916 by the dean of Stanford University’s School of Education, Ellwood P. Cubberley.¹³ The notion of judging teachers based on the test scores of their pupils was floated for decades thereafter. One education researcher, William Lancelot, tried to determine the contribution of teachers to their pupils’ learning by testing the students’ knowledge of mathematics at the beginning and end of the school year to arrive at a “pupil change” score. While some teachers were found to be more effective than others, the gains for pupils who studied with the best teachers were very modest.¹⁴ In the early twenty-first century, the same concept would be revived under the moniker of “value-added scoring” and then, in the Obama years, as “student growth.”¹⁵

Taylorist modes of organizing factory production were increasingly adopted in a wide range of manufacturing industries in the interwar period. By the 1950s they were the norm at companies like General Motors, where, as the sociologist Daniel Bell noted, the managerial “superstructure which organizes and directs production … draws all possible brain-work away from the shop; everything is centered in the planning and schedule and design departments.” The result reinforced the numbing routine for the workers at the bottom of the hierarchy.¹⁶ At the end of the century, metrics would bring these modes of organization out of manufacturing and into the service sector.

MANAGERIALISM AND MEASUREMENT

Taylorism was developed by engineers, but another contribution to the culture of accountability as standardized measurement came from the accounting profession. It was Robert McNamara, an accountant who at the age of 24 became the youngest professor at the Harvard Business School, who carried the message of metrics to the largest organization in the United States: the U.S. Army.

The decades in which McNamara rose from business school professor, to Ford Motor Company executive, to Secretary of Defense, and finally to president of the World Bank also saw the transformation of American business schools. In an earlier era, business schools had focused on preparing their students for jobs in particular industries and enterprises. From the 1950s onward, the business school ideal became the general manager, equipped with a set of skills that were independent of particular industries.

The core of managerial expertise was now defined as a distinct set of skills and techniques, focused upon a mastery of quantitative methodologies.¹⁷ Decisions based on numbers were viewed as scientific, since numbers were thought to imply objectivity and accuracy.¹⁸ Management theorists and gurus who dispensed this new wisdom ascended to the office once ascribed by Shelley to poets as “the unacknowledged legislators of mankind.”¹⁹

Before that, “expertise” meant the career-long accumulation of knowledge of a specific field, as one progressed from rung to rung within the same institution or business—accumulating what economists call “task-specific know-how.” Auto executives were “car guys”—men who had spent much of their professional life in the automotive industry. They were increasingly replaced by McNamara-like “bean counters,” adept at calculating costs and profit margins.²⁰

In time, this attempt to turn management into a science to prepare aspirants for executive positions in corporate America morphed into the gospel of managerialism. The role of judgment grounded in experience and a deep knowledge of context was downplayed. The premise of managerialism is that the differences among organizations—including private corporations, government agencies, and universities—are less important than the similarities. Thus the performance of all organizations can be optimized using the same toolkit of managerial techniques and skills.²¹ We might think of judgment and expertise based upon experience as the lubricant that makes organizations flourish by providing task-specific know-how. Managerialism under the spell of metrics tends to ignore, if it does not actually disdain, all that.

As secretary of defense in charge of prosecuting the war in Vietnam, McNamara championed the metric of “body counts” as a purportedly reliable index of American progress in winning the war. Yet few of the generals in the field considered the body count a valid measure of success, and many knew the counts to be exaggerations or outright fabrications.²² The result, in the pithy formulation of Kenneth Cukier and Viktor Mayer-Schönberger, was a “quagmire of quantification.”²³

McNamara’s Pentagon was characterized by what the military strategist Edward Luttwak called “the wholesale substitution of civilian mathematical analysis for military expertise. The new breed of the ‘systems analysts’ introduced new standards of intellectual discipline and greatly improved bookkeeping methods, but also a trained incapacity to understand the most important aspects of military power, which happen to be nonmeasurable.”²⁴ The various armed forces sought to maximize measurable “production”: the air force through the number of bombing sorties; artillery through the number of shells fired; infantry through body counts, reflecting statistical indices devised by McNamara and his associates in the Pentagon. But, as Luttwak writes, “In frontless war where there are no clear lines on the map to show victory and defeat, the only true measure of progress must be political and nonquantifiable: the impact on the enemy’s will to continue to fight.”²⁵

Luttwak’s critique of the American military establishment, published in 1984, focused on the fact that both its military and civilian leadership had become imbued with a managerial ethos, pursuing measureable “efficiencies” that were at odds with the sort of strategic thinking the military required. “Under the guidance of civilian officials—many of whom care little about their ignorance of strategy, operational craft, and tactics, and present themselves as managers capable of managing all things regardless of their content—the military establishment itself long ago accepted the pursuit of business efficiency as its supreme goal.” Military officers were themselves increasingly imbibing a managerial outlook, pursuing degrees in business administration, management, or economics. That led to what Luttwak called a “materialist bias,” aimed at measuring inputs and tangible outputs (such as firepower), rather than intangible human factors, such as strategy, leadership, group cohesion, and the morale of servicemen.²⁶ What could be precisely measured tended to overshadow what was really important. “[W]hile the material inputs are all hard facts, costs precisely stated in dollars and cents, the intangibles are difficult even to define and mostly cannot be measured at all,” he noted.²⁷

Whether or not Luttwak’s characterization was entirely fair, much of what he criticized in the American military establishment was about to transmigrate to a wide range of institutions in the United States and beyond.

One vector of the metric fixation was the rise of management consultants, outfitted with the managerial skills of quantitative analysis, whose first maxim was “If you can’t measure it, you can’t manage it.”²⁸ Reliance on numbers and quantitative manipulation not only gave the impression of scientific expertise based on “hard” evidence, it also minimized the need for specific, intimate knowledge of the institutions to whom advice was being sold.²⁹ The culture of management demanded more data—standardized, numerical data.