3

The Orwellian World of
“Rational Expectations”

THE SEARCH FOR fully predetermined accounts of individual decisionmaking and market outcomes actually predates economists' embrace of the Rational Expectations Hypothesis as the way to characterize rational forecasting behavior. Prior to this hypothesis, economists portrayed market participants' forecasting strategies with mechanical rules that made no explicit reference to how they reason about the way the economy works or how the causal process underpinning outcomes might change over time. John Muth proposed the Rational Expectations Hypothesis as a way to incorporate such considerations into models of forecasting. Criticizing pre-REH forecasting rules, he argued that

the character of dynamic processes is typically very sensitive to the way expectations are influenced by the actual course of events. Furthermore, it is often necessary to make sensible predictions about the way expectations would change when either the amount of available information or the structure of the system is changed. [Muth, 1961, pp. 315-16]

Muth's idea was that by relating participants' forecasting strategies to an economic model that purportedly captured the structure of the economy, economists would be able to make such sensible predictions over time. Consequently, he formulated the Rational Expectations Hypothesis as a hypothesis that market participants' forecasts “are essentially the same as the predictions of the relevant economic theory” (Muth, 1961, p. 316).

Muth was well aware that the term “rational expectations” suggests some notion of rationality. Indeed, he explicitly recognized the possibility that the name he picked for his hypothesis would create confusion He warned that the Rational Expectations Hypothesis should not be viewed as a normative hypothesis about how rational individuals should forecast the future. As he put it, “At the risk of confusing this purely descriptive hypothesis with a pronouncement as to what firms ought to do, we call such expectations ‘rational'” (Muth, 1961, p. 316, emphasis added).

Even viewed as a purely descriptive hypothesis, it is far from clear how the Rational Expectations Hypothesis should be used to describe forecasting strategies. To implement it, economists had to take a stand on the question of the relevant economic theory to which the hypothesis refers.

Muth did not discuss the difficulties inherent in selecting the relevant theory that should be used in implementing the Rational Expectations Hypothesis. In a fateful decision that triggered the Rational Expectations Revolution in macroeconomics and finance roughly a decade later, he embedded the hypothesis in a simple model of the agricultural market with a production lag. The model portrayed the price of produce at each point in time, t, as being dependent on farmers' expectations formed at some earlier time, t -1, when they had to decide what crop size to aim for. The implementation of the Rational Expectations Hypothesis in this model is particularly straightforward: farmers' expectations regarding the market price at t are set equal to the prediction of that price, implied by an economist's model, at t -1.

By using his own model as the relevant economic theory, Muth in effect ignored many other potentially relevant theories on offer. Indeed, not only do market participants have diverse views, but economists themselves are notorious for their disagreements about what underpins outcomes, particularly in financial markets and the macroeconomy. Thus, even if the relevant theory is a model based on economic theory, which is how economists have interpreted the Rational Expectations Hypothesis, each of their many extant models, as well as any combination of them, is in principle available to individuals forming their forecasts about the future. Moreover, as time passes, profit-seeking individuals and career-minded economists discover inadequacies in old models and attempt to formulate new ones, thereby expanding or contracting the set of relevant theories that market participants might use when forecasting the future.

Muth's idea that market participants pay attention to changes in the structure of the economy when forming their forecasts is compelling. However, the hypothesis that the relevant theory that captures how they use this information to think about the future is one particular economist's model is farfetched. Nevertheless, economists have used the Rational Expectations Hypothesis in exactly the way that Muth did: when an economist devises a model that relates market outcomes to participants' forecasts of these outcomes, he implements the hypothesis by equating these forecasts with the predictions generated by his own model.

In viewing the Rational Expectations Hypothesis as a description of how market participants forecast, disregarding the plurality of extant economic models and forecasting strategies is not the only serious shortcoming. An even more fundamental problem is that for contemporary economists, the relevant theory is a fully predetermined model. Indeed, the agricultural market model that Muth used to introduce the Rational Expectations Hypothesis is fully predetermined. Thus, like the pre-REH forecasting rules, the hypothesis excludes, by design, the possibility that individuals revise their forecasting strategies in nonroutine ways. Because a fully predetermined model implies an overarching forecasting strategy, the Rational Expectations Hypothesis amounts to assuming that one strategy specified in advance adequately characterizes how market participants will think about the future at every point in time.¹

By following the contemporary practice of modeling outcomes with fully predetermined models, Muth subverted his own insight. His idea that changes in the structure of the economy would generally alter market forecasting strategies merely morphed into another mechanical rule that presumed that participants never revise their forecasting strategies in ways that they, or an economist's Rational Expectations model, has not foreseen.²

Early criticism of the Rational Expectations Hypothesis focused on its epistemological flaws as a model of rational forecasting. In doing so, the critics also pointed out its behavioral implausibility as the purely descriptive hypothesis that Muth envisaged. The crux of the matter was shown in Frydman (1982): there is an inherent conflict between its presumption that people's beliefs can be adequately represented as one outcome of an economist's theorizing and the premise that market participants are motivated by self-interest. Simply put, profit-seeking individuals would not, in general, adhere to a single forecasting strategy.

Thomas Sargent, one of the most forceful early advocates of the Rational Expectations Hypothesis, acknowledged these critical arguments and recognized that treating the hypothesis as a (plausible) description of how market participants forecast the future is misleading:

Economists model this strategy with a single probability distribution. Frydman and Goldberg (2007, chapter 6) show rigorously that even if an economist allows for changes in forecasting strategies, as Hamilton (1988) does, he fully prespecifies such revisions, and thus in effect portrays forecasting with a single overarching strategy—a single probability distribution—that characterizes forecasting for all times: past, present, and future.

The idea of rational expectations is sometimes explained informally by saying that it reflects a process in which individuals are inspecting and altering their forecasting records….It is also sometimes said that [the Rational Expectations Hypothesis embodies] the idea that economists and the agents they are modeling should be placed on equal footing: the agents in the model should be able to forecast and profit-maximize and utility-maximize as well as…the econometrician who constructed the model.
        [T]hese ways of explaining things are suggestive, but misleading, because they make [the Rational Expectations Hypothesis] sound less restrictive and more behavioral than it really is. [Sargent, 1993, p. 21, emphasis added]

The implausibility of the Rational Expectations Hypothesis as a descriptive hypothesis undermines Muth's hope that it would enable economists to make sensible predictions about the way expectations change in response to changes in policy or any other change in their context.³ However, somewhat paradoxically, the implausibility of this hypothesis as a description of forecasting in real-world markets is entirely consistent with his warning that it should not be viewed as a hypothesis about what firms ought to do.

MUTH'S WARNING IGNORED

Lacking a normative or other justification for using the Rational Expectations Hypothesis to represent individual forecasting, macroeconomists working in the 1960s largely ignored it in modeling forecasting behavior. Indeed, when Edmund Phelps organized a milestone conference in 1969 on the role of expectations in modeling the microfoundations of macroeconomic theories, the papers collected in the conference volume (Phelps et al., 1970) made no use of the hypothesis, and it is not even listed in the index.

Lucas set out to justify the use of the Rational Expectations Hypothesis as the way to portray rational forecasting.⁴ His justification was based on the core belief that underpins the contemporary approach to macroeconomic theory and finance: fully predetermined models can provide adequate accounts of market outcomes.

Lucas observed that when an economist formulates a theoretical account of market prices, he, like every scientist, hypothesizes that his account adequately portrays how these outcomes unfold over time. Were an economist to impute to a representative agent forecasts that differed from the predictions of his own fully predetermined model, he would in effect be assuming that the agent is obviously irrational: she steadfastly adheres to a forecasting rule that generates forecasts that systematically differ from the model's hypothesized market prices.

Because Lucas took for granted the premise that a fully predetermined model could provide an adequate account of how actual prices evolve over time, he presumed that the ostensibly easily detectable forecast errors implied by such a model with a non-REH portrayal of forecasting pointed to obvious, yet unrealized, profit opportunities in real world markets. As he later emphatically put it, “if your theory reveals profit opportunities, you have the wrong theory” (Lucas, 2001, p. 13) of “actual prices.”

In a leap of faith that would change macroeconomics and finance for generations, Lucas brushed aside Muth's warning⁵ and presumed that the “right theory” is a fully predetermined model in which the Rational Expectations Hypothesis is used to characterize how individuals forecast future market outcomes. Lucas's odd claim gained wide acceptance among macroeconomists and finance theorists. The hypothesis was embraced by economists, spanning the Chicago free-market and the Massachusetts Institute of Technology (MIT) New Keynesian schools. Remarkably, the vast majority of economists came to believe that the Rational Expectations Hypothesis could finally turn macroeconomics and finance into an exact science.

THE RATIONAL EXPECTATIONS REVOLUTION:
MODEL CONSISTENCY AS A STANDARD
OF RATIONALITY

The imposition by the Rational Expectations Hypothesis of exact consistency between the predictions of market outcomes implied by an economist's own fully predetermined model and individuals' forecasting strategies quickly became the standard way to represent how rational individuals think about the future.

Because it could be applied in every fully predetermined model, the REH-based standard had much to recommend it to economists who believe that fully predetermined accounts of market outcomes are within reach of economic analysis. Faith in the divine apotheosis of economic theory led economists to hypothesize that every time one of them formulates his fully predetermined model, he has discovered such an account of market outcomes. Once an economist entertained such a fanciful hypothesis, it seemed reasonable to presume that profit-seeking would compel market participants to search for such a model, which they should do be able to discover, because, after all, an economist already had.

THE SPURIOUS NARRATIVE OF
RATIONAL EXPECTATIONS

Seeking to justify the Rational Expectations Hypothesis as a characterization of how profit-seeking market participants forecast the future, economists have assumed the existence of a fanciful World of Rational Expectations. In this world, a rational individual is supposed to believe at each point in time that she has found a true account of how market outcomes unfold into the indefinite future.

But nonroutine change and imperfect knowledge are not the only problems plaguing attempts to justify the Rational Expectations Hypothesis as a standard of rational forecasting. In contexts like financial markets, there is a two-way interdependence between how outcomes unfold over time and how market participants in the aggregate forecast. Consequently, it makes sense for rational individuals to incorporate into their forecasting strategies their views about how others are forecasting.

Additional assumptions that underpin a World of Rational Expectations, however, ensure that a rational individual does not need to worry about the forecasts of other participants. She is assumed to believe that all other rational individuals populating a World of Rational Expectations have discovered the same true process driving market outcomes over time.⁶ And, because an economist imputes his Rational Expectations model to every rational individual, he presumes that his model adequately captures the truth that they all have discovered. In a glaring example of how far research may be led astray by a logic based on fanciful assumptions, a World of Rational Expectations is constructed on the premise that if other participants are rational, their forecasting strategy will be identical to one's own.

In a World of Rational Expectations, the standard of rational forecasting purportedly provided by the Rational Expectations Hypothesis turns the very notion of rationality on its head. What economists imagine to be rational forecasting in this world would be considered blatantly irrational by anyone in the real world who is minimally rational. After all, a rational, profit-seeking individual understands that the world around her will change in nonroutine ways. She simply cannot afford to believe that, contrary to her experience, she has found a true overarching forecasting strategy, let alone that others have found it as well. Instead, she will look for new ways to forecast, which cannot be fully foreseen.

A WORLD OF STASIS AND
THOUGHT UNIFORMITY

What would a place look like in which economists, policymakers, or social planners (let alone profit-seeking market participants) could safely ignore nonroutine changes, imperfect knowledge, and diversity of views?⁷ Consider equity markets. Like all other asset prices, equity prices in the real world depend on the market's forecast, which is based on market participants' forecasting strategies and the causal variables—such as interest rates, inflation rates, and gross domestic product growth rates—that they believe are relevant. Moreover, some or all causal variables may impact prices directly, that is, independently of how they influence individuals' forecasts.

Most economists who rely on the Rational Expectations Hypothesis presume that both the impact of the market's forecast on prices and the direct effects of casual variables undergo absolutely no change.⁸ They also suppose that the social context, including technology, managerial practices, institutions, and economic policies, remains static. In this world, the processes governing the causal variables are constant over time, and news about them unfolds in a strictly routine way.

But even with the underlying structure of the market and economy irrevocably fixed, the process relating equity prices to a set of causal variables would, in general, still undergo change. Every time a significant number of participants altered their forecasting strategies, the causal process underpinning price movements would change. So to rule out nonroutine change, one would have to construct a world in which not only the social context is irrevocably fixed, but also profit-seeking participants adhere to one forecasting strategy endlessly.

A self-interested, rational individual would remain faithful to one forecasting strategy only if she believed that a better strategy was forever beyond her reach. A rational individual could believe this if she were persuaded that she had discovered a model that accurately captured the true unchanging structure of the economy. But, even believing that, she might not stick to one strategy, because what matters for prices is not the forecast of any particular individual, but the market's forecast.

To square this circle, every market participant in a World of Rational Expectations must not only believe that she knows the true underlying structure of the economy; she must also know that everyone else has this knowledge.⁹ Moreover, each participant must believe that everyone else bases their forecasting strategy on this knowledge. Only if an individual believes that she lives in a world populated by others who think exactly as she does about the process driving market prices and risk—and who forecast these outcomes according to that knowledge—would it be rational for her to use a fixed forecasting strategy based on her knowledge.¹⁰

The Rational Expectations Hypothesis thus leads economists to imagine a world of perfect knowledge and universal thought uniformity. In a World of Rational Expectations, individuals are able to predict perfectly the future prospects of all companies and investment projects, except for random forecasting errors that cancel each other out over time. And, because all rational individuals base their forecasts of prices and risk on these predictions and think alike, the market's forecast is also omniscient. Moreover, in a World of Rational Expectations, the market itself is nearly perfect: it sets prices equal to the true discounted values of the prospects of underlying assets, save for random errors that average to zero.¹¹

Imagining a world of thought uniformity and omniscience transports the economist quite far from the real world. But it still does not tell him how a rational individual would forecast. He must cross yet another ocean of epistemic turbulence before he reaches the untroubled shores of a World of Rational Expectations. He must disregard economists' lack of consensus concerning the relative validity of their existing theories and presume that his own Rational Expectations model, which he hypothesizes to be true, is in fact the model that everyone else believes to be true. Like the rational individuals in his imagined world, the economist believes that he, too, has “solved [his] ‘scientific problem'” (Sargent, 1993, p. 23): he has found the model that sharply predicts all possible future contingencies and their associated probabilities. The economist presumes that his model correctly predicts true fundamental values, as well as prices and risk at each future date.

This approach to modeling individual forecasting behavior assumes what any market participant would consider as simply unreasonable in the real world. No one in her right mind would think that she has discovered the true causal process behind market and economic outcomes, and that everyone thinks exactly as she does.¹² If thought uniformity and omniscience prevailed, there would be no need for markets to set prices and assess the prospects of companies and projects. All economists and each rational market participant would be capable of accomplishing this feat entirely on their own.

ECONOMISTS' RATIONALITY AND
SOCIALIST PLANNING

Economists believed that their presumption of thought uniformity in a World of Rational Expectations enabled them to model market outcomes that result from the decisions of many individuals by studying the decisions of one representative citizen. And, having confused this world with the real world, it was only a short step for economists to think that their fictitious citizen's objectives—self-interest or maximization of well-being—could be imputed to the market and society as a whole. Banishing nonroutine change from his model, an economist could simply prespecify all change and impute to society his own fixed understanding of the future. All he had to do was to solve the maximization problem of the representative individual, whose preferences and Rational Expectations forecasting strategy would become those of everyone.

Like a socialist planner, an economist thus believes that he can accomplish great feats, because he supposes that he has finally uncovered the fully predetermined mechanism that drives market outcomes and that his model adequately captures how market participants think about the future. He generally believes that his theory enables him to ascertain whether state intervention is warranted to, say, correct market failures or deal with swings in asset prices. He also uses his theory to prescribe how government should conduct macroeconomic policies and to study the consequences of those policies on society's welfare.

The problem that haunts the Rational Expectations Hypothesis is the same one that doomed socialist planning: no fully predetermined mechanism that drives market outcomes can, in principle, be uncovered. Basing the explanation of market outcomes on the decisionmaking of one individual who forecasts according to an economist's Rational Expectations model ignores the division of knowledge, which, as Hayek (1945) pointed out, is what distinguishes resource allocation by decentralized markets from a so-called “optimal” deployment of resources by a single individual.¹³Indeed, Lucas's (1995, 2001) account of how the Rational Expectations Hypothesis led him to embrace the representative-agent construct stands in stark contrast to Hayek's position.

In discussing market outcomes for a competitive industry, Lucas (2001, p. 13) argues that “one can show that an industry over time will operate so as to maximize a discounted, consumer surplus integral—a problem that is mathematically no harder than the present value maximizing problem faced by a single firm.” He then asks, “[W]ho, exactly, is solving this planning problem?” As Hayek did, he recognizes that the answer is “Adam Smith's ‘invisible hand,' of course, not any actual person' (Lucas, 2001, p. 13, emphasis added). Nevertheless, in a striking leap of faith, Lucas claims that an economist—that is, an actual person—can adequately represent what the invisible hand of the market does by solving the value-maximizing problem faced by a single firm.

For Hayek (1945, p. 520), the division of “knowledge which is not given to anyone in its totality” was the key to his argument that central planners could not, in principle, substitute for markets. In contrast, Lucas believes that because Rational Expectations models rule out nonroutine change and the division of knowledge, they enable an economist to make use of single-agent optimization techniques, and thus are the right tools to comprehend market outcomes: “[T]he mathematics of planning problems turned out to be just the right equipment needed to understand the decentralized interactions of a large number of producers” (Lucas, 2001, p. 14, emphasis added).

In effect, Lucas posits that Smith's invisible hand could be made visible and intelligible, after all. To understand markets, economists need only learn how to solve optimal allocation problems that a fictitious central planner confronts, but which actual planners could never solve. Indeed, graduate students in economics are instructed to devote most of their time to solving just such problems.

Lucas, of course, is not the first economist to ignore Hayek's arguments that thinking of society's resource-allocation problem as maximization by a single individual—an economist, a planner, or a policy official—is not “the right equipment” to analyze market outcomes. Oskar Lange considered markets to be ill equipped to solve the resource-allocation problem and thus advocated socialist planning (ultimately leaving the University of Chicago to become a senior official in Poland's Communist government). Having focused on the insufficiency of market prices for longer-term decisions, Lange argued, in ways reminiscent of Rational Expectations theorists, that planners' mathematical modeling and computers could do a better job than markets in allocating capital to longer-term projects:

After setting up an objective function…and certain constraints, future…prices can be calculated. These…prices serve as an instrument of economic accounting in long-term development plans. Actual market equilibrium prices do not suffice here, knowledge of preprogrammed future…prices is needed….Mathematical programming turns out to be an essential instrument of optimal economic planning. Here the electronic computer does not replace the market. It fulfills a function which the market never was able to perform. [Lange, 1967, p. 161, emphasis in the original]

As we have seen, Rational Expectations models similarly presume that markets do not play an essential role in allocating society's resources. Thus, it stands to reason that Lange thought that planners could use what later became the single-individual mathematics of Rational Expectations models of markets to abolish markets and fully predetermine the future.

¹Economists model this strategy with a single probability distribution. Frydman and Goldberg (2007, chapter 6) show rigorously that even if an economist allows for changes in forecasting strategies, as Hamilton (1988) does, he fully prespecifies such revisions, and thus in effect portrays forecasting with a single overarching strategy—a single probability distribution—that characterizes forecasting for all times: past, present, and future.

²For an alternative way to formalize the idea that economists' models might be useful in modeling market forecasting, see the Theories-Consistent Expectations Hypothesis in Frydman and Phelps (1990). Frydman and Goldberg (2007, chapters 10 and 15) show how this hypothesis can be used to examine empirically the movements of exchange rates using models that place imperfect knowledge at the center of the analysis.

³Central banks around the world, spurred by widespread acceptance of Lucas's (1976) arguments that Rational Expectations models, unlike the traditional pre-REH Keynesian econometric models, offer the means to examine the effects of changes in economic policy on market forecasting strategies, adopted the idea that Rational Expectations models provide an adequate description of forecasting behavior. See Frydman and Goldberg (2008) for a discussion of this milestone in macroeconomic policy analysis.

⁴Lucas (1995, 2001) provides a fascinating account of the way he arrived at this justification and its importance for subsequent developments in macroeconomics and policymaking.

⁵For an extensive discussion and a formal exposition of the reasons behind Lucas's normative interpretation of the Rational Expectations Hypothesis and the striking difference between his position and Muth's (1961, p. 316) warning that it should not be confused with “a pronouncement as to what firms ought to do,” see Frydman and Goldberg (2010a).

⁶Economists sometimes construct Rational Expectations models that do not make this so-called “common knowledge” assumption. However, dropping this assumption leaves unexplained why, even in the context of a fanciful World of Rational Expectations, an economist's model characterizes how profitseeking individuals forecast the future.

⁷This section and the following one build on arguments in Frydman (1982, 1983).

⁸Some Rational Expectations theorists contemplate a world in which such change does occur but is strictly mechanical. For this reason, the main thrust of our argument would not be affected by considering this slightly more complicated setup.

⁹On strictly logical grounds, it would be sufficient to attribute to an individual in a World of Rational Expectations the belief that only the average, rather than each, of the other participants' forecasting strategies is consistent with the supposedly true process driving outcomes. However, as we show in Chapter 4, this seemingly weaker assumption requires an economist to impose much stronger restrictions in defining a World of Rational Expectations: rigid, fully predetermined connections between the forecasts of the other participants.

¹⁰Despite their empirical findings, some seminal behavioral models continue to use the Rational Expectations Hypothesis in modeling forecasting behavior (DeLong et al., 1990a, 1990b). In these models, there is a subset of market participants who are uninformed and base their forecasts on erroneous considerations. The “smart” or rational individuals are assumed to have a full understanding of how the uninformed participants forecast. As we discuss in Chapter 6, this mixing of rational and irrational participants is even more incoherent than a World of Rational Expectations.

¹¹In a World of Rational Expectations, participants may fall prey to self-fulfilling price movements away from what they know are true fundamental values. We return to this possibility in Chapter 6.

¹²Even in a static world, there are potentially many causal variables that might be relevant for explaining outcomes. But convergence of market participants' learning to one “true” set of such variables requires even stronger assumptions about thought uniformity than those that define a World of Rational Expectations. See Frydman (1982), Frydman and Phelps (1983), Phelps (1983), and Frydman and Goldberg (2010a).

¹³This section builds on Frydman (1983), who shows that they apply directly to a large class of Rational Expectations models with decentralized information developed by Lucas (1973), which are fundamentally flawed for some of the same reasons that Hayek used to show that socialist planning is impossible in principle.