As we’ve seen, performance metrics readily become dysfunctional in governmental settings and in profit-making businesses, and that holds true for nonprofit organizations as well. As in the previous case studies, our purpose is not to survey the field as a whole but to offer some exemplary instances.
Like corporations and government agencies, charities are under pressure to be transparent in their workings and accountable to their donors, and it is often thought that the surest way to do so is by the use of performance metrics. Donors are presumed to want their contributions to be used efficiently, and for the stated goals of the nonprofit organizations to which they contribute. But how is that to be evaluated? And how are they to ensure that the funds are not siphoned off primarily for the benefit of the charity’s staff?
In recent decades, under the spell of metric fixation, funders—foundations, governments, and individuals—decided that the solution was to measure and publicize the percentage of each charity’s budget that was devoted to administrative and fund-raising costs (“overhead” or “indirect expenses”) as opposed to its activities or programs. Once again we see a pattern that we’ve encountered in regard to the use of metrics. What gets measured is what is most easily measured, and since the outcomes of charitable organizations are more difficult to measure than their inputs, it is the inputs that get the attention. At the extremes, the ratio of overhead-to-program costs can provide a useful indicator of fraud or of poor financial management. But too often, measured performance that may be useful in aberrant cases is extended to all cases.
For most charities, equating low overhead with higher productivity is not only deceptive but downright counterproductive. In order to be successful, charitable organizations need competent, trained staff. They need adequate computer and information systems. They need functional offices. And yes, the ability to keep raising funds. But the assumption that the effectiveness of charities is inversely proportional to their overhead expenses leads to underspending on overhead and the degradation of organizational capacities: instead of high-quality and well-trained staff, too many novices and too much staff turnover; computer systems that are out of date and inefficient; and as a result, less effectiveness in raising funds for ongoing activities or new programs. To make matters worse, the funders impose growing demands for reports, so that staff time devoted to documentation eats up an ever larger portion of the grant.
In response, the leaders of charitable organizations often end up trying to game the figures: by reporting that the time of leading staff members is devoted almost entirely to programs, or that there is no spending on fundraising. That response is understandable. But it feeds the expectations of funders that low overhead is the measure they should be looking at to hold charities accountable.1 Thus the snake of accountability eats its own tail.
THE TRANSFORMATIONAL VERSUS THE MEASUREABLE
Metric fixation is also evident in government foreign aid intended to promote social and economic development. There is a deeply ingrained and partly well-founded skepticism about foreign aid, which has too often been unproductive and indeed counterproductive.2 But some foreign aid programs do genuinely contribute to the health, education, economic development, and even political stability of poor countries. In trying to measure what works and what doesn’t, American government agencies have increasingly looked to metrics, with results that might by now be predictable to readers of this book.
Programs whose achievements are not easily measured in quantitative terms have been curtailed. It is easier to measure enrollment in primary schools and literacy rates, for example, than the sort of cultural education of future elites that comes from providing scholarships for students from poor countries to study in American universities. So when metrics becomes the standard of evaluation, programs that cannot demonstrate their short-term benefits are sacrificed. The U.S. Agency for International Development’s scholarship program, for example, was gutted by the White House Office of Management and Budget on the grounds that its benefits could not be put into dollar terms, and thus the government could not determine whether the program’s benefits exceeded its costs.3
Here, too, metrics promotes short-termism. Andrew Natsios, a distinguished public servant with long experience in international development, notes that the employees of government agencies in this field have “become infected with a very bad case of Obsessive Measurement Disorder, an intellectual dysfunction rooted in the notion that counting everything in government programs will produce better policy choices and improved management.” The emphasis on quantification leads to a neglect of programs with the longest-run potential benefits: those that improve the skills, knowledge, and norms of the civil service and judicial systems in underdeveloped nations. Those who suffer from Obsessive Measurement Disorder, Natsios writes, ignore “a central principle of development theory—that those development programs that are most precisely and easily measured are the least transformational, and those programs that are the most transformational are the least measureable.”4 High among those are the development of competent leadership and management.
Here, too, the urge to measure the most easily measureable leads to a focus not on outcomes but on measureable inputs, such as bureaucratic processes. As a USAID official confessed to one scholar, “No one has come up with a valid way to quantify the effectiveness of capacity building activities…. So instead of focusing on effectiveness in reporting, USAID focuses on what can be measured, such as the number of workshops held or the number of people who have participated in training.”5
The demand for more measurement and more quantification comes not only from congressional committees but also from executive agencies such as the Office of Management and Budget and from the Government Accountability Office, agencies staffed in good part by “accountants, economists, procurement officers, and legislative staffers who … bear the stamp of professors in public administration, business administration, or economics who overemphasized quantification in educating them.”6 These professional measurers are the vestal virgins of the sacred fire of metrics. They are also proselytizers, converting their senior management to the cult, which demands a substantial sacrifice of time and energy in the form of statistical reports with which to measure performance and ensure accountability.