Chapter 5
Goodhart’s “Underappreciated” Law and Benchmarking

It is impressive when you have a law named after you, even if it is a somewhat obscure law. If you study finance you may not have come across Goodhart’s Law. However, it is increasingly relevant as computers help to create more indexes, investors benchmark their investments off the indexes, and this distorts market data and corporate valuations. Charles Goodhart is a British professor of economics at the London School of Economics. He has done extensive and distinguished work on regulation, banking and financial systems. He has served governments and published textbooks on monetary economics.

He wrote a paper in 1975, which among other things, discussed the consequences of monetary management. From that paper emerged what is known as Goodhart’s Law. The quote from which the law is derived is, “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”18 There are numerous adaptations of his statement. For example, one summation states, once a social or economic measure is turned into a target for policy, it will lose the features that drove you to choose that target in the first place, as people try to anticipate that policy. Another interpretation is, when a measurement becomes a target it ceases to be a good measure.19

One of the most famous stories used to explain Goodhart’s Law is believed to be a bit of a fable that involves the nail industry in Soviet Russia. The central planners wanted to measure the productivity of the factories so they told the managers they will be measured by how many nails they make in a given year. To meet its goal a factory made the smallest nails possible to make the most nails out of each iron delivery. The nails were so small they were unusable and there was a great nail shortage in the Soviet Union. The next year the central planners tried to correct their mistake and set a goal that the nail factories would be measured by the weight of the nails they produce. The same nail factory then proceeded to order as much iron as it could get and produced one giant nail. The story does not tell what happened to the factory manager. This is a good illustration of how peoples’ actions to achieve a target distort the value of the target. This is also a pretty good story of why central planning fails.

Goodhart’s Law was derived with a focus on the impact of regulation, but it can be rephrased and reworked for many other things. For example, there are numerous mapping programs now that show drivers how to beat traffic back-ups, however, if too many people use the same program it can then create a traffic back-up on the alternative route. Near the George Washington Bridge that connects New Jersey to New York City, this became such a problem and drove so much traffic to one town that the mayor of the town tried to ban cars from out of town from cutting through their neighborhood streets.20

As there is more focus on measurable metrics for business and investments, Goodhart’s Law has become increasingly important in the investment world. On a trading desk the law might be simplified to the phrase, “there isn’t any value in a crowded trade.”

Market and valuation distortions happen when everyone is using the same benchmarks to measure investment success. Computing power and data storage have helped to create a proliferation of indexes and investment performance measurements. This has led to more demand from investors that money managers perform within a band relative to a benchmark (i.e., limiting tracking error to the benchmark).

Benchmarking has become a bit of an obsession. Trillions of dollars track benchmark indexes.21 The desire to copy these benchmark indexes has driven a whole industry of passive investing. This strategy involves investment managers simply trying to mimic the performance of an index. The goal is to do no better or worse than this index, no best ideas, and no specialized focus to meet the ultimate investors’ specific goals. Many of these passive investments are being made through exchange traded funds (ETFs).

One of the problems with benchmarking is that indexes often base their investment weighting on the market capitalization of the company. Therefore, stocks with bigger market capitalizations get a bigger portion of any new money that comes into the fund. This can create more momentum in the stock of the large capitalization companies, thus increasing their weighting in the indexed funds. Paul Woolley of the London School of Economics has done several studies in this area and is quoted as saying “When used as a benchmark for active management, market cap indices carry perverse incentive that impair fund returns, distort prices….”22

The process of chasing benchmarks can create risks in a portfolio. The benchmark chaser can end up running after investments where capital flows have gone, such as momentum stocks. These investments are made without analyzing or factoring in the potential cash flow of an investment or its value. Momentum investing can have success but it also has risks.

When market distortions happen, speculators and arbitrageurs weigh in to the market to exploit miss-pricings and opportunities caused by poorly structured investments. Assume that high risk investors see a stock that is in an index that has a high likelihood of having a positive near-term event; they may buy the stock, even though its long-term prospects may not be that good. Their purchases will likely push the price up. They know that this will make it a larger part of the index. Passive investors and benchmark huggers will have to buy that stock to stay balanced relative to the index. This passive buying will create a floor price for the risk investors and a potential exit after the news occurs. The passive investors may not realize they have just increased the risk in their portfolios.

As higher volatility stocks gain upward momentum benchmarking creates more of an upward bias in these higher volatility stocks (the same obviously can happen in bond markets, too). Mr. Woolley and Dimitri Vayanos have published models and studies23 that show that benchmarking distorts prices of stocks. If a positive shock occurs for a company and the stock rises based on the merit of the news, it can get a second boost when benchmark driven funds realize they are underweighting the stock and their buying adds to the momentum.

This benchmarking not only distorts prices but long term can create misallocation of capital to companies that may be performing better in the stock market regardless of how they are performing operationally. This trend may cause the equity markets to stop functioning as a gauge of success or underlying value and just as a measure of the size of a company’s market capitalization. The value of stocks now often moves not just on the inherent value of a company’s cash flow generation power, but on whether the company’s securities are included or excluded in an index.

Using an index’s return as a target for investment portfolios does not create the same problems as the Soviet nail factory but there are several risks if you chose a specific index as a “target.” Indexes are not static, they change, and you need to stay abreast of the rules. Index’s rules usually lead to changes in the make-up of the constituents over time. Indexes also can change their inclusion and exclusion rules periodically. Therefore, the targets that you had found attractive in each index may change. You need to carefully monitor the changes to an index, to make sure it is still a target you want to pursue as an investor.

Benchmarks have other idiosyncrasies that differ from trading real world portfolios. Benchmarks do not have transaction costs included in their returns. These costs occur when an investor trades the securities that are in the index. If other investors start to “game the system” and run in front of the benchmark inclusions and exclusion rules, as there is some evidence of, the costs of the investments can go up for the passive indexers and their respective benchmarks. Indexes also do not include any management or administrative fees that need to be incurred in a real-world investment. A strategy that just tries to copy or “hug” a benchmark can not outperform it meaningfully, especially after transaction costs and management fees.

Gains in computational power and access to digitized data coupled with the desire to develop better measurement of investment performance have led to an explosion in indexes and benchmark analysis. Understanding and analyzing investment performance is a necessity for asset allocators so that they can understand returns relative to risks and properly allocate capital. However, too much emphasis on benchmarks can lead to distortions and investors losing sight of what their real investment goals are.

Index creation has boomed so much during this century that in 2017 it was reported that there were more indexes than there were public stocks outstanding.30 In this desire to measure things using, benchmarks, investor may get performance or risks that they do not want. If an investment firm is hired because the client thinks they have a great investment system, a focus on benchmarking can neuter that. Sometimes investors should step away from indexes and really decide what they want from their investment portfolio.

There is no perfect investment for everyone. People have different time horizons for their investments, different return goals, and different risk tolerances. This sounds simple, but it is important to remember when investing. Does the investor want the fastest growth and appreciation in value, regardless of risk, or do they want to preserve capital and generate a healthy stream of income on which to live? The goals of investors and institutions are extremely varied and just chasing an index may not be the best way to achieve these, often complex goals.

Market indexes are used to describe and analyze broad movements in security valuations. Performance of stock and bond indexes can sometimes be viewed as a barometer for the overall economy. The analysis of subsectors within an index can give insights into which industries and types of companies can attract new capital. Indexes can also be used to compare returns of one type of security market to another. The proliferation of investment strategies that look to copy index performance is changing the quality and meaning of the data within indexes. When new money comes into index hugging strategies it is typically now being allocated based on market capitalization, not any relative value criteria. This is distorting the value of current index data. These factors must be considered today in any analysis that uses current index and stock price information relative to historical data.