Investing in the world’s financial markets is a varied, complex and important undertaking. Every investor should have a formal plan, in two parts: a philosophy or strategy for guiding the investment; and a methodology, or set of tactics, for carrying it out. A well-designed plan includes a viewpoint on how the financial markets should react to a range of developments in the real economy, what assets to invest in or avoid, and the likely range of returns a portfolio might earn, as well as yardsticks for measuring success or disappointment—a set of principles for monitoring the results, and signaling to the asset owners that they need to adjust the plan if something goes awry.¹

For institutional investors, a reasoned investment policy statement is often a legal requirement, and certainly part of the overseers’ fiduciary duty. For many other investors, however, an investment philosophy may develop over time through trial and error, even unconsciously—not unlike an organizational culture. But developing an informed investment plan presumes a knowledge of the financial markets that many people lack, and this shortage has created the need for an enormous industry of investment consultants, managers and advisers for institutions, businesses, and individuals.

Of the many options investors face, in this discussion we are most concerned with the final stages of this complex portfolio decision tree—the crucial, practical choice of how to manage the assets from day to day, with a focus on equities. Specifically, are investors better off in the long run by owning passive portfolios that aim to own every stock in the market, and therefore earn market-like results? Or should they invest in actively managed portfolios? That is, should they hire specialist firms pursuing more narrow portfolios intended to outperform the market?

The choice goes beyond philosophy. The investment outcomes can differ greatly, between “owning the market,” perhaps through an index fund that tracks the S&P 500 or the MSCI World Index by effectively owning every stock, versus the return on an actively managed strategy, where a manager narrows the portfolio to, say, 50 or 100 securities he believes will outperform the broad market. (The MSCI World Index, published by MSCI Inc., includes roughly 1,600 stocks in 23 markets in developed economies around the globe, and captures approximately 85 percent of the capitalization in each market. In much of the discussion that follows, we use the S&P 500 as a proxy for “the stock market.” A global index like the MSCI World Index is a better proxy for today’s investor, but we rely instead on the S&P 500 for its richer and more accessible history.) Consider a hypothetical $1,000 invested in the U.S. stock market in January 1990. From then through December 2015, the annualized return from the S&P 500 was 9.3 percent. Note that this is a hypothetical return, rather than an actual one. Performance of an index does not reflect actual results, although the managers of funds tracking indexes can closely approximate the hypothetical returns. Had the portfolio owned an S&P 500 index fund charging an annual management fee of 0.10 percent, its value would have grown to $9,814 at the end of 2015 (assuming the reinvestment of dividends, and no additions or withdrawals from the account, and no taxes). An actively managed fund that earned an average of 1.50 percent above the benchmark each year, even while charging an annual management fee of 0.75 percent, would have grown to $12,207, or about 24 percent greater.

But that example is simple arithmetic, calculated in a vacuum. A skillful active manager might earn an excess return averaging 1.50 percent over that many years, but inevitably that manager would outperform in some years and underperform in others—no manager can beat the market by the same margin every single year. The actual successes of a particular investor, working with particular managers, would vary according to many factors: the time periods considered; the behavior of the markets during those periods; the skill and consistency possessed by the active managers an investor hires to oversee the portfolio; which market sectors and securities those active managers emphasize; and the investor’s fortitude in staying invested during disappointing markets.

The Debate Is Timeless

The debate about active versus passive management is hardly new. Over 50 years ago, Benjamin Graham, the godfather of conservative equity investing, delivered what to some modern ears may sound like a surprising message.² In a November 1963 speech to the San Francisco Society of Security Analysts, Graham asserted:

I think the . . . most important reason why the investor should not be led to emphasize his selection of individual stocks, and to neglect the general level of the stock market is the fact that there is no indication that the investor can do better than the market averages by making his own selections or by taking expert advice. . . .

The outstanding support for that pessimistic statement is found in the record of the investment funds, which represent a combination of about the best investment brains in the country, and a tremendous expenditure of money, time, and carefully directed effort. The record shows that the funds have had great difficulty as a whole in equaling the performance of the 30 stocks in the Dow Jones Averages or the 500 Standard and Poor’s Index [sic]. If an investor had been able, by some rough across-the-board diversification to make up a portfolio approximating these averages he would have had every reason to expect about as good results as were shown by the very intelligent and careful stock selections by the investment-fund managers. But the great justification for the mutual funds is that very few individuals actually do follow such a sound and simple policy . . .

The underlying problem of selection is that the “good stocks”—chiefly the growth stocks with better than average prospects— tend to be fully priced and often overpriced.³

It may seem curious that one of the great investment minds, and an advocate of sober, methodical investment in a selected portfolio of stocks, should steer people away from trying their hands at active investment, and instead counsel investing somehow in the market as a whole (passive investing through index funds would be invented years later). What Graham was getting at was an intuitive concept that did not yet have a name, but has since been formalized into the Efficient Market Hypothesis. To understand what that hypothesis is, and what its implications are, calls for a review of financial history.

An Elegant Theory: The Capital Asset Pricing Model

Astute investors had always had an intuitive sense that the return to be expected from an investment was related to the risk they had to incur—the greater the risk, the greater the possible return. However, the concept of risk was a subjective one, until the 1950s, when financial economists began to develop what they believed was an objective definition—that is, a way to precisely quantify and measure risk. They focused on how an investment’s returns varied from one time period to another, based on the familiar statistical measures of standard deviation and variance. From this platform, economists were able to develop an expansive body of thought that came to be known as Modern Portfolio Theory, or MPT.

The first pillar of MPT is the Capital Asset Pricing Model, or CAPM, which was theorized in the 1960s by financial economists William Sharpe and John Lintner, and carried forward in the 1970s by Fischer Black.⁴ (Sharpe was awarded the Nobel Prize for his efforts in 1990.) The CAPM is an elegant theory, the objective of which is to describe the relationship between risk and reward for financial assets, and it offers a simple design for thinking about markets.

We will not describe the entire CAPM structure here, but one of its key tenets is particularly relevant to the debate about active versus passive investing. The CAPM starts from the assumption that all investors define risk in the model’s terms—the variability of returns. From that point, MPT demonstrates that there is one optimal portfolio of stocks which, when combined with cash, will provide the highest return possible for a given level of risk. (Alternatively, an optimal portfolio would experience the lowest possible level of risk for a given level of return.) The CAPM goes on to deduce that because all investors would hold this optimal portfolio, the market as a whole—the sum of all investors’ portfolios—would be that same optimal portfolio, although on a larger scale. Viewed from the reverse angle, this means that the optimal portfolio is simply a miniature of the overall market. In essence, CAPM relies on a logical inference: investors will all want to hold the optimal portfolio, but the only portfolio that it is possible for every investor to hold simultaneously is a miniature replica of the overall market, so the market must be the optimal portfolio.

It wasn’t long before investment practitioners devised products to bring theory into practice. Index funds allowing institutional investors to buy the entire stock market all in one go were invented in the early 1970s for that purpose—to meet MPT adherents’ perceived need for uniform market portfolios (Chapter 12).⁵

Further Elegance: The Efficient Market Hypothesis

The second pillar of MPT is the Efficient Market Hypothesis (EMH), which grew out of research conducted by Eugene Fama at the University of Chicago in the late 1960s. (Fama, too, would eventually win a Nobel Prize for his work.) In its purest form, the EMH holds that securities prices always reflect all available information about the economy, markets and companies, and that all investors interpret that information in the same rational way, resulting in rational prices for securities.⁶

If the EMH were valid, trying to outsmart the collective knowledge of the market by forecasting prices of a select group of stocks through an active strategy would be futile. Therefore, the notion of efficient markets held great appeal to those pioneering MPT economists: it would elegantly link the stock market with the economy, simplifying the process of investing and eliminating a lot of unnecessary effort (and commissions and fees for investors).

With the development of the CAPM and the EMH in rapid succession, the late 1960s and early 1970s were a heady time in the world of academic finance. “In the decade of the 1970s, I was a graduate student writing a PhD dissertation on rational expectations models, . . . and I was mostly caught up in the excitement of the time,” Robert J. Shiller, a professor at Yale University, and a winner of the Nobel prize in Economics, wrote in 2003.⁷

Reality Intrudes

Reality turned out differently, however. Further research refined the questions being asked, and before long cracks started to show in the two pillars of MPT. Shiller wrote:

One could easily wish that these models were true descriptions of the world around us, for it would then be a wonderful advance for our profession. . . . Wishful thinking can dominate much of the work of a profession for a decade, but not indefinitely. The 1970s already saw the beginnings of some disquiet over these models. . . . Browsing today again through finance journals from the 1970s, one sees some beginnings of reports of anomalies that didn’t seem likely to square with the efficient markets theory, even if they were not presented as significant evidence against the theory.⁸

Some of the anomalies were quirky and minor, such as the “January effect,” where stock prices tended to show better returns in the first month of the year. More troubling, Shiller notes, was the difficulty in explaining the persistently high and random level of volatility in a supposedly rational market: “The evidence regarding excess volatility seems, to some observers at least, to imply that changes in prices occur for no fundamental reason at all, that they occur because of such things as ‘sunspots’ or ‘animal spirits’ or just mass psychology.” The existence of these anomalies called into question the broader framework of the CAPM. And in fact, the empirical record of the CAPM turned out to be poor.

The Achilles’ heel of the CAPM is the notion of beta, which expresses the relationship between the expected return of an individual stock (or a portfolio), and the return of the market as a whole. In its simplest form, the CAPM assumes a systematic linear relationship, so that given a, say, 10 percent gain in the general market (over and above what is called the risk-free rate, usually set at the return on short-term U.S. Treasury bills), a stock with a beta of 1.5 would be expected to rise 15 percent more than the risk-free rate. Each security has an additional, unsystematic component to its return—alpha—embodying idiosyncratic factors about the stock and the underlying company. But alpha is assumed to make a small contribution over the long term, so that beta is the prime mover of a stock’s returns. (Although beta is not terribly useful for projecting expected returns, it has been invaluable in measuring performance after the fact, in its ability to link a portfolio’s actual returns to those of the market.)

By the 1990s financial economists had turned up a great number of theoretical arguments, as well as empirical anomalies in the markets, contrary to what the CAPM would explain. In 1993, after more than 20 years of struggle over the CAPM, the entire agenda of the prestigious annual conference of what is today known as the CFA Institute was devoted to the controversy. Yale professor Steven Ross, one of the pioneers of the CAPM and beta, presented a paper titled “Is Beta Useful?” wherein he concluded that for purposes of forecast returns, it is not:

My first acquaintance with the conflict between theory and reality happened years ago when a company asked me to help with a cost-of-capital issue. . . . We believed that, if this theory had merit, on average, over time, the stocks with betas in the bottom 10 percent would have the lowest return and those in the highest beta class would have the highest return; an upward-sloping line would connect them. What we got was a flat line, which means that having a low, middle, or high beta does not matter; the expected return is the same. This result is very depressing. . . .

For many years, we have been under the illusion that the CAPM is the same as finding that beta and expected returns are related to each other. That is true as a theoretical and philosophical tautology, but pragmatically, they are miles apart.⁹

The Problem with MPT

Why have the elegant theories that make up MPT proven to be poor descriptors of the real world of markets? As with many academic models, the issue has to do with what theorists refer to as simplifying assumptions. MPT holds that investors evaluate potential investments in two dimensions, and two dimensions only: expected return and expected risk. (This framework is known as “mean-variance”, referring to the mean of returns on securities, and their volatility as measured by the variance. The notion was developed by financial economics pioneer Harry Markowitz in the early 1950s.) MPT further assumes that all investors agree on the riskiness of a given investment or portfolio since, after all, risk is defined as a statistical measure of the volatility of returns—a number that can be calculated simply from past returns without any subjective input (or insight to the future). In addition, MPT posits that investors have a single objective: to maximize their expected return for any given level of risk.

In the past two decades, a new generation of “behavioral economists,” who combined insights into human psychology with the study of economics, have cast doubts on MPT, raising questions as to whether passive investing in the broad market is optimal, either in theory or in practice.