1.1 Background Motivation

Traditionally, U.S. electric power systems were organized as collections of regulated vertically-integrated utility companies.¹ Each utility company controlled all generation, transmission, and distribution of electric power in a designated geographical area for the purpose of servicing power consumers in this area; see Fig. 1. In return, the utility company was guaranteed a rate of return ensuring a suitable profit margin.

Starting in the mid-1990s, prodded by successive rule-making from the U.S. Federal Energy Regulatory Commission (FERC), wholesale power systems in many parts of the U.S. have been substantially restructured into partially decentralized systems based more fully on market valuation and allocation mechanisms. These restructuring efforts were driven by a desire to ensure efficient power production and utilization, reliable power supplies, and affordable power prices for consumers.

As depicted in Fig. 2, the basic design ultimately proposed by FERC (2003) for U.S. wholesale power systems envisions private Generation Companies (GenCos) who sell bulk power to private companies called Load-Serving Entities (LSEs), who in turn resell this power to retail consumers. These transactions between GenCos and LSEs take place within a wholesale power market consisting of a Day-Ahead Market (DAM) and a Real-Time Market (RTM), operating in parallel, which are centrally managed by an Independent System Operator (ISO) or Regional Transmission Organization (RTO); see Fig. 3. Any discrepancies that arise between the day-ahead generation schedules determined in the DAM based on estimated next-day loads and the actual needs for generation based on real-time loads are handled in the RTM, which thus functions as a real-time balancing mechanism.²

The physical power flows underlying these transactions take place by means of a high-voltage transmission grid that remains centrally managed by the ISO/RTO in order to ensure open access at reasonable access rates. Transmission grid congestion is managed in the DAM and RTM by Locational Marginal Pricing (LMP).³

The basic FERC design depicted in Figs. 2 and 3 has to date been adopted by seven U.S. regions encompassing over 60% of U.S. generating capacity; see Fig. 4. Six of these restructured wholesale power markets entail interstate commerce and hence are under FERC's jurisdiction. The seventh, the Electric Reliability Council of Texas (ERCOT), was deliberately restructured to avoid interstate commerce and hence Federal jurisdiction. Since retail transactions in all seven regions typically involve only local commerce over lower-voltage distribution grids, these transactions are regulated by state and local agencies. In consequence, by and large, retail prices paid by U.S. retail electric power users are regulated flat rates charged on a monthly basis that are not directly responsive to changes in wholesale power prices.

Nevertheless, attempts have repeatedly been made in the U.S. to restructure retail power system operations. The basic idea has been to replace LSEs by a competitive retail market enabling a large number of retailers to sell electric power to price-taking businesses and households at market-clearing prices. The design envisioned circa 2010 for a power system restructured at both the wholesale and retail levels is depicted in Fig. 5. A key aspect to note in Fig. 5 is that all power-flow arrows still point down, just as they do in Figs. 1 and 2, reflecting the traditional idea that businesses and households are passive loads that consume power taking prices/rates as given.

As it happens, however, the design in Fig. 5 has not come to pass; it has been overtaken by two related trends. First, “variable energy resources” are increasingly being substituted for thermal generators (e.g., coal-fired power plants) in response to growing concerns regarding environmental pollution from thermal generation (Ela et al., 2016). Second, technological developments such as advanced metering and intelligent devices are permitting businesses and households to become more active participants in power system transactions (Kabalci, 2016).

Variable energy resources (VERs) are renewable energy resources, such as wind turbines and solar (photovoltaic) panels, whose power generation cannot be closely controlled to match changes in demand or to meet other system requirements. For example, generation from wind turbines and solar panels can abruptly change due to sudden changes in wind speed and cloud cover. The growing penetration of VERs thus tends to increase the volatility of net power demands (i.e., customer power demands minus non-controllable generation) as well as the frequency of strong ramp events (i.e., rapid increases or declines in net power demands).

Volatility and strong ramp events make it difficult for system operators to maintain a continual real-time balance between power supply (i.e., generation net of transmission losses) and net power demands, an essential physical requirement for the reliable operation of power grids.⁴ Since large thermal generators (especially nuclear and coal-fired) tend to have slow ramping capabilities, they are not an effective means for countering real-time fluctuations in net power demands. Although large-scale energy storage devices could in principle be deployed to offset these real-time fluctuations, to date this deployment has not been cost effective. Consequently, power system researchers are turning their attention to the possible use of more nimble types of resources for balancing purposes.

In particular, power system researchers are investigating the possibility of introducing various forms of aggregators able to harness ancillary services⁵ from collections of distributed energy resources (DERs) owned by businesses and households connected to distribution grids. Examples of DERs include distributed generation (e.g., rooftop solar panels), plug-in electric vehicles, battery storage systems, and household appliances with energy storage capabilities and flexible power needs that permit collections of these appliances to function as prosumers, i.e., as entities that can produce or consume power depending on local conditions.

Harnessing of services from DERs for real-time operations requires voluntary participation by DER owners, hence it requires the creation of value streams that can be used to compensate DER owners appropriately for provided DER services. In addition, it requires technological developments such as advanced metering and intelligent devices that enable DERs to respond in a timely and accurate manner to real-time electronic signals.

Fig. 6 envisions a future transmission and distribution (T/D) system. An ISO/RTO-managed wholesale power market operating over a high-voltage (HV) transmission grid is tightly linked through DER/load aggregators to a distribution system operating over a lower-voltage (LV) distribution grid. The participants in the distribution system include DERs (distributed generation, storage, prosumers), locally managed by intelligent price-responsive software agents, as well as conventional (non-price-responsive) loads. An important characteristic of such a system is that data, signals, and power can flow up as well as down between the transmission and distribution levels.

Another important characteristic of such a system is a prevalence of contract-based transactions that permit transactor responsibilities to be expressed in clear legally-enforceable terms. Indeed, Fig. 6 is only a partial depiction of the many contractual relationships already engaged in by modern power system participants. Missing are directly negotiated bilateral contracts between wholesale power buyers and sellers, self-scheduled in the wholesale power market in order to secure transmission access, as well as various types of financial contracts (e.g., financial transmission rights) secured by system participants in order to hedge against price and quantity risks.

1.2 Chapter Scope

In view of the developments outlined in Section 1.1, a critical concern is how to design appropriate economic and control mechanisms to handle demand and supply transactions among the increasingly heterogeneous and dispersed collection of participants in modern electric power systems. Responding to this concern, the GridWise Architecture Council (2015) has formulated a new Transactive Energy System (TES) framework for power systems. This TES framework is defined (p. 11) to be “a set of economic and control mechanisms that allows the dynamic balance of supply and demand across an entire electrical infrastructure using value as a key operational parameter.”

Given the complexity of TES designs, their validation prior to real-world implementation requires many different levels of investigation ranging from conceptual analysis to field studies (Widergren et al., 2016). Moreover, to avoid adverse unintended consequences, this validation needs to include a careful consideration of behavioral incentives; TES participants should not perceive opportunities to game system rules for own advantage at the expense of system reliability and efficiency (Gallo, 2016).

Fortunately, agent-based modeling is well suited for these purposes. As detailed in IEEE (2016), Ringler et al. (2016), and Tesfatsion (2017a), TES researchers are increasingly turning to agent-based modeling tools in an attempt to bridge the gap between conceptual TES design proposals and validated real-world TES implementations.

This handbook chapter discusses current and potential uses of Agent-Based Modeling (ABM) as a support tool for TES research, with a stress on general methodological and practical concerns. Section 2 provides a broad overview of Agent-based Computational Economics (ACE), a specialization of ABM to the study of systems involving economic processes. Although the precise meaning of ABM continues to be debated in the literature, specific modeling principles have been developed for ACE that carefully distinguish it from other types of modeling and that highlight its particular relevance for TES research.

Seven specific modeling principles underlying ACE model design are presented and explained in Section 2.1. Taken together, they express the fundamental goal of many agent-based modelers: namely, to be able to study real-world systems as historical processes unfolding through time.

Section 2.2 divides the various objectives being pursued by ACE researchers into basic categories: empirical understanding; normative design; qualitative insight and theory generation; and method/tool advancement. Section 2.3 identifies distinct aspects of empirical validation that researchers tend to weight differently, depending upon their objectives: namely, input validation; process validation; in-sample prediction; and out-of-sample forecasting. Although differential weighting by objective is commonly done, it is argued that ACE modeling permits researchers to strive for a more comprehensive approach to empirical validation that simultaneously considers all four aspects.

Section 2.4 considers the increasingly important role that ACE models are playing as computational laboratories for the development and testing of policy initiatives in advance of implementation. A taxonomy of policy readiness levels (PRLs) is proposed for policy initiatives ranging from conceptual modeling (PRL 1) to real-world policy deployment (PRL 9). It is noted that ACE modeling is helping to bridge the difficult gap between the conceptual research (PRLs 1–3) typically undertaken at universities and the field studies and deployments (PRLs 7–9) typically undertaken by industry. Section 2.5 argues that this PRL taxonomy could facilitate the development of standardized presentation protocols for ACE policy models that appropriately take into account model purpose and level of model development.

Section 3 briefly reviews early ACE research on electric power systems, with an emphasis on seminal contributions. Section 4 provides a general overview of TES research undertaken to date. The next two sections focus on ACE computational laboratories as a support tool for the design of TES architectures. Section 5 discusses current and potential ACE support for TES research on demand-response initiatives designed to encourage more active demand-side participation in T/D system operations. Section 6 discusses current and potential ACE support for TES research on contract design, with a stress on the need for contracts that facilitate flexible service provision in a manner that is both incentive compatible and robust against strategic manipulation. Concluding remarks are given in Section 7.

2.1 ACE Modeling Principles

Agent-based Computational Economics (ACE) is the computational modeling of economic processes (including whole economies) as open-ended dynamic systems of interacting agents.⁶ The following seven modeling principles collectively characterize the ACE modeling approach:

(MP1) Agent Definition: An agent is a software entity within a computationally constructed world capable of acting over time on the basis of its own state, i.e., its own internal data, attributes, and methods.

(MP2) Agent Scope: Agents can represent individuals, social groupings, institutions, biological entities, and/or physical entities.

(MP3) Agent Local Constructivity: The action of an agent at any given time is determined as a function of the agent's own state at that time.

(MP4) Agent Autonomy: Coordination of agent interactions cannot be externally imposed by means of free-floating restrictions, i.e., restrictions not embodied within agent states.

(MP5) System Constructivity: The state of the modeled system at any given time is determined by the ensemble of agent states at that time.

(MP6) System Historicity: Given initial agent states, all subsequent events in the modeled system are determined solely by agent interactions.

(MP7) Modeler as Culture-Dish Experimenter: The role of the modeler is limited to the setting of initial agent states and to the non-perturbational observation, analysis, and reporting of model outcomes.

Considered as a collective whole, modeling principles (MP1)–(MP7) embody the idea that an ACE model is a computational laboratory permitting users to explore how changes in initial conditions affect outcomes in a modeled dynamic system over time. This exploration process is analogous to biological experimentation with cultures in Petri dishes. A user sets initial conditions for a modeled dynamic system in accordance with some purpose at hand. The user then steps back, and the modeled dynamic system thereafter runs forward through time as a virtual world whose dynamics are driven by the interactions of its constituent agents.

The explicit statement of these modeling principles permits ACE to be distinguished more clearly and carefully from other modeling approaches, such as standard game theory and general equilibrium modeling within economics, and standard usages of state-space modeling by economists, engineers, and physicists. It also permits more precise comparisons between ACE and important historical antecedents, such as system dynamics (Rahmandad and Sterman, 2008) and microsimulation (Richiardi, 2013).

Modeling principle (MP1) provides a concise definition of an ACE agent as a software entity capable of taking actions based on its own local state. Here, “state” refers to three possibly-empty categories characterizing an agent at any given time: data (recorded physical sensations, empirical observations, statistical summaries, ...); attributes (physical conditions, financial conditions, beliefs, preferences, ...); and methods (data acquisition, physical laws, data interpretation, logical deduction, optimization routines, learning algorithms, decision rules, ...). There is no presumption here that the data acquired by an agent are accurate or complete, or that the methods used by an agent to process data are without error.

An agent's state represents the potential of this agent to express various types of behaviors through its actions. The agent's actual expressed behaviors within its virtual world, while conditioned on the agent's successive states, are also constrained and channeled by interactions with other agents.

An important corollary of (MP1) is that agents in ACE models are encapsulated software entities, i.e., software entities that package together data, attributes, and methods. This encapsulation permits an agent's internal aspects to be partially or completely hidden from other agents. A person familiar with object-oriented programming (OOP) might wonder why “agent” is used in (MP1) instead of “object,” or “object template” (class), since both agents and objects refer to encapsulated software entities. “Agent” is used in ACE, and in ABM more generally, to stress the intended application to problem domains that include entities capable of various degrees of self-governance, self-directed social interactions, and deliberate masking of intentions. In contrast, OOP has traditionally interpreted objects as passive tools developed by a user to aid the accomplishment of a user-specified task.

The “state” conceptualization in (MP1) differs in two important ways from state depictions in standard state-space modeling:

Regarding (ii), an agent's state can evolve over time in open-ended ways. For example, an agent can continue to augment its data D(t)$\mathbb{D}(t)$$\mathbb{D}(t)$ over successive times t without need to rely on fixed-dimensional sufficient statistics, and its attributes $\alpha (t)$ can also vary over time. Moreover, the agent's methods $\mathbb{M}(t)$ might include a domain $\mathbb{R}(t)$ of possible decision rules plus a genetic algorithm g that involves mutation and recombination operations. When g operates on $\mathbb{R}(t)$, given $\mathbb{D}(t)$ and $\alpha (t)$, the result $g(\mathbb{R}(t);\mathbb{D}(t),\alpha (t))$ could be a modified decision-rule domain $\mathbb{R}(t+\mathrm{\Delta }t)$ that has different elements than $\mathbb{R}(t)$ and possibly also a different dimension than $\mathbb{R}(t)$.

Modeling principle (MP2) expresses the intended broad scope of the agent definition provided in (MP1). In particular, in contrast to many agent definitions proposed in the general ABM literature, (MP2) makes clear that ACE agents are not restricted to human representations. Such a restriction would require modelers to make unnecessary distinctions between human actions and the actions of all other kinds of entities. Instead, (MP1) is in accordance with the standard dictionary meaning of agent as any entity, whether person or thing, able to take actions that affect subsequent events.

Another way of viewing (MP2) is that it calls for a broad agent taxonomy, i.e., a broad classification of agents into ordered groups or categories. As illustrated in Fig. 7, agents in ACE models can span all the way from passive entities with no cognitive function (e.g., grid transformers) to active entities with sophisticated decision-making capabilities (e.g., DER/load aggregators). Moreover, agents (e.g., distribution systems) can have other agents (e.g., households) as constituent members, thus permitting the modeling and study of hierarchical organizations.

The remaining five ACE modeling principles (MP3)–(MP7) imply that ACE models are state-space models in initial value form (Tesfatsion, 2017d). Specifically, an ACE model specifies how an ensemble of agent states varies over time, starting from a given ensemble of agent states at an initial time. Modern economic theory also relies heavily on state-space modeling. However, modeling principles (MP3)–(MP7) sharply differentiate ACE models from standard economic state-space models.

Specifically, (MP3)–(MP7) require agent autonomy conditional on initial agent states. In contrast, standard economic state-space models incorporate modeler-imposed rationality, optimality, and equilibrium conditions that could not (or would not) be met by agents acting solely on the basis of their own local states at each successive point in time. For example, rational expectations assumptions require ex ante agent expectations to be consistent with ex post model outcomes. Consequently, the derivation of rational expectations solutions is a global fixed-point problem requiring the simultaneous consideration of all time periods.

The seven modeling principles (MP1)–(MP7) together require an ACE model to be fully agent based. That is, all entities capable of acting within an ACE computationally-constructed world must be modeled as some form of agent. This requirement has two key advantages. First, it enhances conceptual transparency; all factors affecting world events must be clearly identified as an agent or agent component. Second, it facilitates plug-and-play model scalability. The number of previously-typed agents can easily be increased, since this does not require changes to the interfaces between agent types. Also, high-level architectures (HLAs)⁷ can be designed for ACE models that facilitate enlargement of their scope through inclusion of new agent types.

For ACE researchers, as for economists in general, the modeling of decision methods for decision-making agents is a primary concern. Here it is important to correct a major misconception still being expressed by some commentators uninformed about the powerful capabilities of modern software: namely, the misconception that ACE decision-making agents cannot be as rational (or irrational) as real people.

To the contrary, the constraints on agent decision making implied by modeling principles (MP1)–(MP7) are constraints inherent in every real-world economic system. As seen in the ACE learning research linked at Tesfatsion (2017e), the decision methods used by ACE agents can range from simple behavioral rules to sophisticated anticipatory learning algorithms for the approximate achievement of intertemporal objectives. A concrete demonstration of this assertion within a macroeconomic context is provided by Sinitskaya and Tesfatsion (2015).

A second common misconception is the incorrect belief that (MP1)–(MP7) rule out any consideration of stochasticity. To the contrary, stochastic aspects can easily be represented within ACE models. Agent data can include recorded realizations for random events, agent attributes can include beliefs based on probabilistic assessments, and agent methods can include pseudo-random number generators (PRNGs). A PRNG is an algorithm, initialized by a seed value, that is able to generate a number sequence whose properties mimic the properties of a random number sequence.

PRNGs can be included among the methods of decision-making agents, thus permitting these agents to “randomize” their behaviors. For example, a decision-making agent can use PRNGs to choose among equally preferred actions or action delays, to construct mixed strategies in game situations to avoid exploitable predictability, and/or to induce perturbations in action routines in order to explore new action possibilities.

PRNGs can also be included among the methods of other types of agents, such as physical or biological agents, in order to model stochastic processes external to decision-making agents. For example, the Weather agent in Fig. 7 might use a PRNG to generate a weather pattern for each simulated year that affects the operations of Appliance agents and Solar Panel agents. These operational effects could in turn induce changes in the decisions of the DER/Load Aggregator agent and Household agents.

An important constraint affecting the ACE modeling of stochasticity is that the modeling principles (MP1)–(MP7) require an ACE model to be dynamically complete. Thus, ACE modelers must identify the sources of any stochastic shocks affecting events within their modeled worlds, not simply their impact points, because all such shocks must come from agents actually residing within these worlds. This requirement encourages ACE modelers to think carefully about the intended empirical referents for any included stochastic shock terms. It also facilitates successive model development. For example, a Weather agent represented as a highly simplified stochastic process in a current modeling effort can easily be modified to have a more empirically compelling representation in a subsequent modeling effort.

Another key issue is whether modeling principles (MP1)–(MP7) imply ACE models are necessarily pre-statable. As stressed by Longo et al. (2012) and Koppl et al. (2015), the real world “bubbles forth” with an ever-evolving state space, driven in part by random (acausal) events. This renders infeasible the pre-statement of accurate equations of motion for real-world state processes.

ACE modeling addresses this issue in two ways. First, there is no requirement in ACE modeled worlds that the agents residing within these worlds be able to accurately depict laws of motion for their states in equation form, or in any other form. Second, data can be streamed into ACE models in a manner that prevents even the modeler from being able to accurately pre-state future model outcomes.

More precisely, suppose an ACE model has no run-time interaction with any external system during times $t\in [t^o,T]$ for some finite horizon T. Then, in principle, the modeler at time $t^o$ could pre-state all model outcomes over the time interval $[t^o,T]$, conditional on a given specification of agent states at time $t^o$, in the same manner that he could in principle pre-state all possible plays of a chess game with a given closure rule.

Nevertheless, (MP1)–(MP7) do not imply that ACE agents have complete state information. Consequently, ACE agents can experience events over time that they have no way of knowing in advance. For example, suppose an ACE model consists of a Weather agent interacting over times $t\in [t^o,T]$ with a variety of other agents, as depicted in Fig. 7. At the initial time $t^o$ the modeler might know the weather pattern that the Weather agent will generate over $[t^o,T]$, or be able to pre-state this weather pattern based on the modeler's time-$t^o$ knowledge (or control) of the Weather agent's data, attributes, and methods. However, if other agents have no access to the Weather agent's internal aspects, they will experience weather over $[t^o,T]$ as a stochastic process.

Alternatively, an ACE model can have run-time interactions with an external system. For example, as discussed by LeBaron and Tesfatsion (2008, Section III) and Borrill and Tesfatsion (2011, Section 2.1), an ACE model can be data driven; that is, it can include conduit agents permitting external data to be streamed into the model during run-time that are unknown (or unknowable) by the modeler at the initial time $t^o$. In this case the modeler at time $t^o$ will not be able to pre-state future model outcomes, even in principle.

A particularly intriguing case to consider is when the data streamed into an ACE modeled world include sequences of outcomes extracted from real-world processes. For example, real-world weather data could be streamed into the Weather agent in Fig. 7 that this agent then uses to generate weather patterns for its computational world. These weather data could include thermal or atmospheric noise data accessible to decision-making agents, such as Household agents, enabling them to use “truly random” numbers in place of PRNGs to randomize their decision-making processes.

2.2 ACE Objectives and Scope

Current ACE research divides roughly into four strands differentiated by objective. One primary objective is empirical understanding: What explains the appearance and persistence of empirical regularities? ACE researchers seek possible causal mechanisms grounded in the successive interactions of agents operating within computationally-rendered virtual worlds. A virtual world capable of generating an empirical regularity of interest provides a candidate explanation for this regularity.

A second primary objective is normative design: How can ACE models be used as computational laboratories to facilitate the design of structures, institutions, and regulations resulting in desirable system performance over time? The ACE approach to normative design is akin to filling a bucket with water to determine if it leaks. A researcher constructs a virtual world that captures salient aspects of a system operating under a proposed design. The researcher identifies a range of initial agent state specifications of interest, including seed values for agent PRNG methods. For each such specification the researcher permits the virtual world to develop over time driven solely by agent interactions. Recorded outcomes are then used to evaluate design performance.

One key issue for ACE normative design is the extent to which resulting outcomes are efficient, fair, and orderly, despite possible attempts by strategic decision-making agents to game the design for personal advantage. A second key issue is a cautionary concern for adverse unintended consequences. Optimal design might not always be a realistic goal, especially for large complex systems; but ACE models can facilitate robust design for increased system resiliency, a goal that is both feasible and highly desirable.

A third primary objective of ACE researchers is qualitative insight and theory generation: How can ACE models be used to study the potential behaviors of dynamic systems over time? Ideally, what is needed is a dynamic system's phase portrait, i.e., a representation of its potential state trajectories starting from all feasible initial states. Phase portraits reveal not only the possible existence of equilibria but also the basins of attraction for any such equilibria. Phase portraits thus help to clarify which regions of a system's state space are credibly reachable, hence of empirical interest, and which are not. An ACE modeling of a dynamic system can be used to conduct batched runs starting from multiple initial agent states, thus providing a rough approximation of the system's phase portrait.

A fourth primary objective of ACE researchers is method/tool advancement: How best to provide ACE researchers with the methods and tools they need to undertake theoretical studies of dynamic systems through systematic sensitivity studies, and to examine the compatibility of sensitivity-generated theories with real-world data? ACE researchers are exploring a variety of ways to address this objective ranging from careful consideration of methodological principles to the practical development of programming, visualization, and empirical validation tools.

2.3 Enabling Comprehensive Empirical Validation

Modelers focused on the scientific understanding of real-world systems want their models to have empirical validity (“consistency with real world data”). Below are four distinct aspects of empirical validation which, ideally, a model intended for scientific understanding should simultaneously achieve:

EV1. Input Validation: Are the exogenous inputs for the model (e.g., functional forms, random shock realizations, data-based parameter estimates, and/or parameter values imported from other studies) empirically meaningful and appropriate for the purpose at hand?

EV2. Process Validation: How well do the physical, biological, institutional, and social processes represented within the model reflect real-world aspects important for the purpose at hand? Are all process specifications consistent with essential scaffolding constraints, such as physical laws, stock-flow relationships, and accounting identities?

EV3. Descriptive Output Validation: How well are model-generated outputs able to capture the salient features of the sample data used for model identification? (in-sample fitting)

EV4. Predictive Output Validation: How well are model-generated outputs able to forecast distributions, or distribution moments, for sample data withheld from model identification or for data acquired at a later time? (out-of-sample forecasting)

In practice, economists relying solely on standard analytical modeling tools do not place equal weight on these four aspects of empirical validation. Particularly for larger-scale economic systems, such as macroeconomies, analytical tractability issues and a desire to adhere to preconceived rationality, optimality, and equilibrium ideals have forced severe compromises.

In contrast, an ACE model is an open-ended dynamic system. Starting from an initial state, outcomes are determined forward through time, one state leading to the next, in a constructive manner. This process does not depend on the determination, or even the existence, of equilibrium states. ACE thus provides researchers with tremendous flexibility to tailor their agents to their specific purposes.

In particular, ACE researchers can match modeled biological, physical, institutional, and social agents to their empirical counterparts in the real world. This ability to match modeled agents to empirical counterparts, important for scientific understanding, is also critical for normative design purposes. Robustness of proposed designs against strategic manipulation can only be assured in advance of implementation if the modeled decision-making agents used to test the performance of these designs have the same degree of freedom to engage in strategic behaviors as their empirical counterparts.

ACE modeling thus permits researchers to strive for the simultaneous achievement of all four empirical validation aspects EV1 through EV4. This pursuit of comprehensive empirical validation will of course be tempered in practice by data limitations. Even in an era of Big Data advances, data availability and quality remain important concerns. Computational limitations, such as round-off error, truncation error, and error propagation, are also a concern. Advances in computer technology and numerical approximation procedures are rapidly relaxing these limitations. In the interim, however, as expressed by Judd (2006, p. 887), numerical error must be traded off against specification error:

“The key fact is that economists face a trade-off between the numerical errors in computational work and the specification errors of analytically tractable models. Computationally intensive approaches offer opportunities to examine realistic models, a valuable option even with the numerical errors. As Tukey (1962) puts it, ‘Far better an approximate answer to the right question ... than an exact answer to the wrong question ...’.”

Empirical validation of ACE models in the sense of EV1 through EV4 is a highly active area of research. Extensive annotated pointers to this research can be found at Tesfatsion (2017f).

2.4 Avoiding Premature Jumps to Policy Implementation

Ideally, changes in a society's current institutional and regulatory policies should be guided by research that is strongly supported by empirical evidence. Reaching a point where a proposed new policy is ready for real-world implementation will typically require a series of modeling efforts at different scales and with different degrees of empirical verisimilitude. Moving too soon to policy implementation on the basis of over-simplified models entails major risk of adverse unintended consequences.

Consider, for example, the Policy Readiness Levels (PRLs)⁸ proposed in Table 1 for research directed towards the normative design of institutional and/or regulatory policies. Due to relatively limited data and computational capabilities, policy researchers at universities tend to work at PRLs 1–3. In contrast, policy researchers within industry, government, and regulatory agencies tend to work at PRLs 7–9.

The interim PRLs 4–6 thus constitute a “valley of death” that hinders the careful step-by-step development and testing of policy proposals from conceptual formulation all the way to real-world implementation. Fortunately, ACE modeling is well suited for bridging this valley because it facilitates the construction of computational platforms⁹ permitting policy model development and testing at PRLs 4–6. This will be illustrated in Section 5, which focuses on the use of ACE platforms for the design and evaluation of transactive energy system architectures.

All nine levels in the PRL taxonomy are essential for ensuring conceptual policy ideas are brought to real-world fruition. Explicit recognition and acceptance of this tiered model valuation could encourage policy researchers to become more supportive of each other's varied contributions.

Another important point is that the PRL taxonomy does not necessarily have to represent a one-way road map from initial concept to completed application. Rather, PRLs 1–9 could constitute a single concept-to-application iteration in an ongoing Iterative Participatory Modeling (IPM) process. Extensive annotated pointers to IPM studies can be found at Tesfatsion (2017f).

2.5 Towards Standardized Presentation Protocols for ACE Policy Models

The classification of policy models in accordance with policy readiness levels (PRLs), as proposed in Section 2.4, could also help resolve another key issue facing ACE policy researchers. Specifically, how can ACE models and model findings undertaken for policy purposes be presented to stakeholders, regulators, and other interested parties in a clear and compelling manner (Wallace et al., 2015, Sections 3–4, 6)?

Most ACE models are not simply the computational implementation of a model previously developed in equation form. Rather, ACE modeling often proceeds from agent taxonomy and flow diagrams, to pseudo-code, and finally to software programs that can be compiled and run. In this case the software programs are the models. On the other hand, it follows from the modeling principles presented in Section 2.1 that ACE models are initial-value state space models. Consequently, in principle, the software program for any ACE model can equivalently be represented in abstract form as a system of discrete-time or discrete-event difference equations, starting from user-specified initial conditions. These analytical representations become increasingly complex as the number of agents increases.

The practical challenge facing ACE policy researchers then becomes how best to present approximations of their models to interested parties who are unable or unwilling to understand these models in coded or analytical form. Most ACE policy researchers resort to verbal descriptions, simple graphical depictions for model components and interactions, Unified Modeling Language (UML) diagrams,¹⁰ and/or pseudo-code expressing the logical flow of agent interactions over time. Anyone wishing to replicate reported results is referred to the original source code.

The lack of presentation protocols for ACE policy models (and for ACE models more generally) has been severely criticized by economists who directly specify their models in analytical or statistical terms using commonly accepted approaches. At the very least, it complicates efforts to communicate model features and findings with clarity, thus hindering the accumulation of knowledge across successive modeling efforts.

Fortunately, the development of presentation protocols for agent-based models is now an active area of research (Tesfatsion, 2017d,g). For example, the ODD (Overview, Design concepts, and Details) protocol developed by Grimm et al. (2006, 2010a) has been widely adopted by ecologists who use agent-based models. To date, however, proposed protocols such as ODD have attempted to provide “one size fits all” requirements for the presentation of models, regardless of purpose and development level.

For policy research, a better way to proceed would seem to be the adoption of multiple standardized presentation protocols tailored to the PRL of a modeling effort. For example, a protocol for PRL 1–3 models could require a complete model presentation within the confines of a typical journal article. In contrast, a protocol for PRL 4–7 models could consist of two sets of presentation requirements: one set for a summary model presentation to be reported within the confines of a typical journal article; and a second set for a complete model presentation (source code, documentation, and test-case simulation data) to be reported at a supplementary website repository.

5.1 Overview

In traditional U.S. electric power systems based on vertically integrated utilities (Fig. 1), the power usage of residential, commercial, and industrial customers was generally assumed to be highly unresponsive to price changes. Utilities typically charged their customers a flat hourly rate for power usage, plus additional fixed charges, on an extended (e.g., monthly) basis. A critical utility operator function, referred to as “load following,” was then to ensure that real-time customer power usage was continually balanced by real-time power generation, whatever form this power usage took.

Thus, customers became accustomed to extracting power from the grid without any consideration of its actual production cost or environmental impact. Paraphrasing Bunn (2004, p. 5), customers were essentially the holders of power call options that were unconstrained in volume up to fuse box limits and that could freely be exercised at customer convenience.

This traditional conception of customer power usage as externally determined load in need of balancing has been carried forward into U.S. restructured wholesale power markets (Fig. 2). Although, in principle, LSEs participating in day-ahead markets are permitted to submit hourly demand bids for the next-day power needs of their customers in two parts – a price-responsive demand schedule and a fixed power amount – most LSE hourly demand bids currently take a fixed form.

As far back as 2002, power economists have forcefully argued the need for participants on both sides of a power market, buyers and sellers, to be able to express their reservation values¹³ for power in order to achieve an efficient pricing of power; see, e.g., Stoft (2002, Chapter 1-1), Kirschen (2003), Rassenti et al. (2003), and Tesfatsion (2009). However, given the relatively primitive state of metering technology, it was not feasible for power customers to adjust their power usage in real time in response either to system operator commands or to automated signals. Consequently, power customers continued to play a largely passive role in power system operations.

Fortunately, recent breakthroughs in metering technology, referred to as Advanced Metering Infrastructure (AMI), have radically improved the potential for more active customer participation (Kabalci, 2016). AMI broadly refers to an integrated system of meters, communication links (wired or wireless), and data management processes that permits rapid two-way communication between power customers and the agencies (e.g., utilities) that manage their power supplies.

In particular, AMI enables the implementation of demand-response initiatives designed to encourage fuller demand-side participation in power system operations. Power system researchers are currently exploring three basic types of demand-response initiatives¹⁴:

(i) Incentive-Based Load Control: Down/up adjustments in the power usage of business and household devices are made either in response to direct requests from designated parties¹⁵ or via device switches under the remote control of designated parties, with compensation at administratively set rates.

(ii) Price-Responsive Demand: Down/up power usage adjustments are undertaken by businesses and/or households in response to changes in power prices communicated to them by designated parties.

(iii) Transactive Energy: Demands and supplies for power and ancillary services by businesses and households are determined by decentralized bid/offer-based transactions within a power system organized as a TES.

The implementation of these demand-response initiatives can result in curtailments (or increases) in total power withdrawal from the grid, or in shifts in the timing of power withdrawals from the grid with no significant change in total power withdrawal. In some cases, demand response resources might be willing and able to offset curtailments (or increases) in their power withdrawals from the grid by resorting to local “behind the meter” generation and storage facilities, such as an onsite wind turbine or a small-scale battery system with no grid connection.

A key goal of type-(i) initiatives is to permit ancillary services to be extracted from demand-side resources in support of system reliability. A key goal of type-(ii) initiatives is to enhance system efficiency by permitting business and household customers to express their reservation values for power at different times and locations. A key goal of type-(iii) initiatives is to enhance the reliability and efficiency of system operations by enabling a balancing of demands and supplies for power and ancillary services across an entire electrical infrastructure on the basis of business and household reservation values.

Researchers focusing on type-(i) and type-(ii) demand response initiatives have primarily stressed metering, control, and planning aspects for system operators and power customers. For example, Bitar et al. (2014, Chapters 2–3) and Parvania et al. (2013) investigate the ability of type-(i) demand response programs to provide reserve services for system operators. Yoon et al. (2014) study various forms of control strategies for type-(i) initiatives designed to maximize the net benefits of building residents subject to constraints. Thomas et al. (2012) propose a type-(ii) intelligent air-conditioning system controller able to determine optimal next-day power usage for a household based on the household's comfort/cost trade-off preferences, conditional on price signals for next day retail power usage and a forecast for next-day environmental conditions. Wang and He (2016) propose a two-stage co-optimization framework for a customer whose stage-1 decision is to install a battery energy storage system and whose stage-2 decision is to join one of several offered type-(i) and type-(ii) demand response programs.

However, some work has explored the effects of type-(ii) demand response initiatives on power system operations over time. For example, Thomas and Tesfatsion (2016, 2017) use an ACE computational platform to investigate the effects of price-responsive household demands for power on integrated T/D system operations over time by means of systematic computational experiments.¹⁶ Zhou et al. (2011) develop an agent-based computational platform to study the effects of price-responsive power demand by commercial buildings, modeled as autonomous agents with reinforcement learning capabilities, that compete to offer demand response services into a wholesale power market operating over a transmission grid. Making use of the Policy Readiness Levels (PRLs) proposed in Section 2.4, this agent-based work can roughly be classified as PRL 4.

TES researchers focusing on type-(iii) demand response initiatives are interested in understanding the potential effects of these initiatives on the end-to-end operations of entire T/D systems. The work of these TES researchers can roughly be divided into three categories:

[PRLs 1–3] Conceptual discussion supported by graphical depictions and/or by relatively simple analytic modeling: e.g., Bitar et al. (2014, Chapter 4), Rahimi et al. (2016).

[PRLs 4–6] Performance studies making use of agent-based computational platforms: e.g., Broeer et al. (2014), Kahrobaee et al. (2014) Karfopoulos et al. (2015, Sections 1–4), Kok (2013, Section III), Pinto et al. (2011), Santos et al. (2015).

[PRLs 7–8] Relatively large-scale performance tests in laboratory or field settings: e.g., AEP (2014), Karfopoulos et al. (2015, Sections 5–6), Kok (2013, Section IV), Kok and Widergren (2016), PNNL (2015).

As stressed by Ringler et al. (2016, Section 5), the application of agent-based modeling to the study of integrated T/D system operations with smart grid capabilities, such as TES architectures, is still relatively new. Moreover, the source code developed for most of these studies is not publicly available.

A key exception is work based on GridLAB-D (2017), an open-source agent-based computational platform developed by researchers with the U.S. Department of Energy at Pacific Northwest National Laboratory. As explained by Chassin et al. (2014), GridLAB-D permits the customized simulation of a distribution system populated with residential, commercial, and industrial customers that own a wide range of electrical devices.

The first illustration discussed below is a type-(ii) price-responsive demand study by Thomas and Tesfatsion (2016, 2017). This study is conducted in part by means of an ACE computational platform referred to as the Integrated Retail and Wholesale (IRW) Test Bed (Tesfatsion, 2017h). The IRW Test Bed simulates the successive daily operations of an electric power system consisting of retail and wholesale power sectors subject to distribution and transmission grid constraints. GridLAB-D is used to model non-price-responsive electrical devices owned by distribution system households.

The second illustration discussed below is a type-(iii) transactive energy study focusing on PowerMatcher (2016), a TES architecture based on agent-based modeling principles that has been tested in multiple field settings. An exceptionally clear, detailed, and thought-provoking report on PowerMatcher is provided by Kok (2013), the original developer.

5.2 ACE Demand Response Study Undertaken with the IRW Test Bed

5.2.1 The IRW Test Bed

As illustrated in Fig. 9, the IRW Test Bed models the integrated grid-constrained operations of retail and wholesale power sectors over time. The IRW Test Bed has four key components:

• •  C1: Wholesale power sector, implemented by means of the AMES Wholesale Power Market Test Bed (AMES, 2017).
• •  C2: Retail power sector, implemented in part by GridLAB-D (2017).
• •  C3: C++ modeling of price-responsive loads.
• •  C4: MySQL database server to facilitate data storage and transfer among C1–C3.

AMES (Agent-based Modeling of Electricity Systems)¹⁷ is an open-source ACE computational platform that permits the simulation over successive 24-hour days of a wholesale power market adhering to standard practices in U.S. ISO/RTO-managed wholesale power markets. Fig. 10 depicts the principal types of agents comprising AMES (V4.0). These agents include a physical entity (transmission grid), institutional entities (day-ahead and real-time markets), and decision-making entities (ISO and market participants). Each AMES agent is characterized at any given time by its internal state (data, attributes, methods). For visual clarity, Fig. 10 only lists some of the more important attributes and methods for each agent, not the agent's complete state.

More precisely, the decision-making agents in AMES include an Independent System Operator (ISO), Load-Serving Entities (LSEs), and Generation Companies (GenCos). The methods of the AMES decision-making agents can include, in particular, learning methods for updating current methods on the basis of past events.¹⁸ This permits AMES users to investigate the degree to which changes in power market rules or other exogenously specified power system aspects would open up market power opportunities that AMES decision-making agents could exploit for their own personal advantage at the expense of overall system performance.¹⁹

The LSEs and GenCos participate in an ISO-operated two settlement system consisting of a Day-Ahead Market (DAM) and a Real-Time Market (RTM) operating over a high-voltage transmission grid. Transmission grid congestion in each market is managed by Locational Marginal Prices (LMPs). The daily DAM and RTM operate in parallel with each other and are separately settled; cf. Fig. 3.

During the morning of each day D the GenCos and LSEs submit into the DAM a collection of supply offers and demand bids, respectively, for all 24 hours H of day $\mathrm{D}+1$. For each hour H, these offers and bids take the following general form²⁰:

(1) $\text{GenCo Price-Responsive Supply:}\pi =a+2bp$

(1)

(2) $\text{LSE Price-Responsive Demand:}\pi =c-2dp$

(2)

(3) $\text{LSE Fixed Demand:}p=\text{FD(H)}$

(3)

where π ($/MWh) denotes price, p (MW) denotes power, FD(H) (MW) denotes a fixed (non-price-responsive) demand for power, and a ($ /MWh), b ($/(MW)²h), c ($/MWh), and d ($/(MW)²h) are positive constants. The power levels in (1) through (3) represent constant power levels to be maintained during the entire hour H, either as injections into the grid (power supplies) or as withdrawals from the grid (power demands).

Given these offers and bids, the ISO solves Security-Constrained Unit Commitment (SCUC) and Security-Constrained Economic Dispatch (SCED) optimization problems subject to standard system constraints²¹ in order to determine: (i) generation unit commitments; (ii) scheduled generation dispatch levels; and (iii) a price LMP${}^{DA}$(B, H, $\mathrm{D}+1$) ($ /MWh) at each transmission grid bus B for each hour H of day $\mathrm{D}+1$. A generator located at bus B is paid LMP${}^{DA}$(B, H, $\mathrm{D}+1$) for each MW it is scheduled to inject at B during hour H of day $\mathrm{D}+1$, and an LSE located at a bus B must pay LMP${}^{DA}$(B, H, $\mathrm{D}+1$) for each MW its household customers are scheduled to withdraw at bus B during hour H of day $\mathrm{D}+1$.

The RTM runs each hour of each day.²² At the start of the RTM for hour H on day $\mathrm{D}+1$, the ISO is assumed to know the actual household power usage for hour H of day $\mathrm{D}+1$. The ISO then solves a SCED optimization problem to resolve any discrepancies between the generation scheduled in the day-D DAM for dispatch during hour H of day $\mathrm{D}+1$, which was based on day-D LSE demand bids, and the generation needed during hour H of day $\mathrm{D}+1$ to balance actual household power usage. Any needed adjustments in the DAM-scheduled power supplies and demands at a bus B for hour H of day $\mathrm{D}+1$ are settled at the RTM LMP at bus B for hour H of day $\mathrm{D}+1$.

GridLAB-D (2017) is an open-source agent-based computational platform for the study of distribution systems that provides detailed physically-based models for a wide variety of appliances and devices owned by residential, commercial, and industrial customers. The distribution system for the IRW Test Bed currently consists of residential households with both conventional (non-price-responsive) and price-responsive loads. The conventional loads are generated by means of GridLAB-D.

As detailed in Thomas et al. (2012), the price-responsive loads in the IRW Test Bed are currently generated as price-responsive demands for air-conditioning (A/C) power usage. The price-responsive A/C demand for each household is determined by an intelligent A/C controller operating in accordance with physical and economic principles. Each day the controller determines the household's optimal next-day A/C power usage based on the household's comfort/cost trade-off preferences, next-day power prices announced today by the LSEs, and a forecast for next-day environmental conditions.

Fig. 11 depicts the feedback loop process by which real-time power usage is determined over time in the IRW Test Bed.

5.2.2 A Demand-Response Study Using the IRW Test Bed

Thomas and Tesfatsion (2016, 2017) study the effects of price-responsive A/C load on the integrated operation of wholesale and retail power sectors using a simplified version of the IRW Test Bed. Hereafter this study is referred to as the IRW Test Case. This section reports some of the findings for this IRW Test Case in order to illustrate more concretely the capabilities of the IRW Test Bed for demand-response research.

The transmission grid lines for the IRW Test Case are specified with sufficiently large capacity that congestion does not arise. Hence, the grid effectively consists of a single bus B. There is only one GenCo and one LSE, each located at bus B. The GenCo is automatically committed in the DAM each day without need for unit-commitment optimizations.

The LSE serves as a wholesale power purchasing agent for 500 households populating the distribution system. The power usage of each household arises solely from air conditioning (A/C). The A/C load of each household is managed by a smart controller (Thomas et al., 2012), responsive to both price and environmental conditions, that reflects the specific comfort/cost trade-off preferences of the household. The GenCo has sufficient capacity to meet the power needs of the 500 households. Line capacities for the distribution grid are large enough to avoid grid congestion.

The feedback loop determining retail prices and real-time loads over time for the IRW Test Case is depicted in Fig. 12. A more detailed description of this loop will now be provided.

The supply offer submitted by the GenCo to the day-D DAM for any specific hour H of day $\mathrm{D}+1$ takes the price-elastic form (1). This supply offer gives the GenCo's reservation value (marginal production cost) for each successive megawatt (MW) of power it might be required to generate during hour H of day $\mathrm{D}+1$; that is, it represents the GenCo's marginal cost function.

The demand bid submitted by the LSE to the day-D DAM for any specific hour H of day D + 1 takes the fixed-demand form (3).²³ This fixed-demand bid is a forecast of household load for hour H of day $\mathrm{D}+1$. The LSE sets this forecast equal to the actual household load observed for hour H on day $\mathrm{D}-1$.

The RTM runs 24 hours of each day. The ISO is correctly able to forecast actual household power usage for hour H of day $\mathrm{D}+1$ at the start of the RTM for hour H of day $\mathrm{D}+1$. The ISO uses the RTM for hour H of day $\mathrm{D}+1$ to resolve any discrepancies between the generation scheduled in the day-D DAM for dispatch during hour H of day $\mathrm{D}+1$ and the actual amount of generation needed during hour H of day $\mathrm{D}+1$ to balance actual household power usage for hour H of day $\mathrm{D}+1$. These discrepancies are settled at the RTM LMP for hour H of day $\mathrm{D}+1$.

The retail contracts offered by the LSE to the 500 households take one of two possible forms: (i) a flat-rate retail contract with a flat rate R ($/MWh) set to ensure the LSE breaks even over time; or (ii) a dynamic-price retail contract with one-way communication (LSE to households) in which DAM LMPs, marked up by a non-negative percentage m, are passed through to households as next-day retail prices. More precisely, under dynamic-price retail contracts, the retail price charged by the LSE to each household for power usage during hour H of day $\mathrm{D}+1$ is given by

(4) $r\text{(H,\hspace{0.17em}}\mathrm{D}+1\text{)}=[1+m]{\text{LMP}}^{DA}\text{(B,\hspace{0.17em}H,\hspace{0.17em}}\mathrm{D}+1\text{)}$

(4)

where LMP${}^{DA}$(B, H, $\mathrm{D}+1$) ($/MWh) is the LMP determined in the day-D DAM at bus B for hour H of day $\mathrm{D}+1$.²⁴

As detailed in Thomas et al. (2012), the net benefit (benefit minus cost) attained by household h from the purchase and use of electric power during any hour H of any day D is given by

(5) ${\text{NB}}^h(\mathrm{H},\mathrm{D})={\text{Comfort}}^h(\mathrm{H},\mathrm{D})-{\alpha }^h{\text{EnergyCost}}^h(\mathrm{H},\mathrm{D})$

(5)

In (5), the comfort (Utils) attained by household h depends on the interior thermal conditions experienced by h during hour H of day D. Also, the energy cost ($) charged to h depends on two factors: (i) h's power usage during hour H of day D; and (ii) the form of h's retail contract, either flat-rate or dynamic-price.

The key parameter ${\alpha }^h$ (Utils/$) in (5) is a trade-off parameter measuring the manner in which household h trades off comfort against cost. The higher the value of ${\alpha }^h$, the greater the weight that h places on energy cost savings relative to thermal comfort.²⁵

A simplified version of the IRW Test Case with a postulated download-sloping aggregate demand curve for households is first used to derive, analytically, a set of necessary and sufficient conditions for market efficiency and system stability under both dynamic-price and flat-rate retail contracting. A complete analysis of short-run welfare outcomes under each form of contracting is also provided. A key finding is that the use of dynamic-price retail contracts induces braided cobweb dynamics consisting of two interweaved cobweb cycles for power and price outcomes. These braided cobweb cycles can exhibit either point convergence, limit cycle convergence, or divergence depending on a small set of structural parameters characterizing power supply and demand conditions.

Simulation studies are then conducted for the full IRW Test Case to examine the form of the aggregate demand curve for the 500 households with price-responsive A/C controllers under varying price conditions, all else equal. It is shown that this aggregate demand curve is well-approximated by a linear downward-sloping curve in the power-price plane. A systematic sensitivity study is then conducted for the full IRW Test Case with the following three aspects taken as treatment factors: (i) the form of retail contracts, either flat-rate or dynamic-price; (ii) the mark-up m in (4) that determines the percentage by which retail prices are marked up over wholesale prices in the case of dynamic-price retail contracts; and (iii) the household comfort-cost trade-off parameter ${\alpha }^h$ in (5). Four values are tested for m: $[0.0,0.2,0.4,0.6]$. Also, four values are tested for ${\alpha }^h$: [0, Low, Medium, High].

Fig. 13 compares welfare outcomes realized for the LSE, the GenCo, and the households for a range of treatments exhibiting point-convergent cobweb dynamics. For simplicity, only treatments with α and m values commonly set across all households are shown. A key finding indicated by the outcomes reported in Fig. 13 is that dynamic-price retail contracts with a positive mark-up m result in worse welfare outcomes for the GenCo and for household residents with $\alpha >0$ than flat-rate retail contracts.

5.3 PowerMatcher: A TES Architecture for Distribution Systems

TES researchers stress the desirability of two-way communication between retail customers and distribution system operators (or their intermediaries). In particular, they assert that distribution system operations should be based more fully on demand bids and supply offers from retail customers, not simply on the responsiveness of price-taking retail customers to signaled prices. The negative results reported for dynamic pricing in Section 5.2.2 suggest that bid/offer-based designs for retail customer transactions should indeed be carefully considered.

This section reports on PowerMatcher (2016), a TES architecture for distribution systems developed by Koen Kok in collaboration with industry partners (Kok, 2013). PowerMatcher is fully based on agent-based modeling principles; it implements a decentralized bid/offer-based mechanism for the real-time management of collections of customer-owned distributed energy resource (DER) devices by means of intelligent software agents.

Specifically, PowerMatcher organizes a collection of DER devices into a network of “transaction nodes.” Communication is restricted to two-way exchanges of information between adjacent transaction nodes, where this information must be required for the support of transactions between these adjacent nodes. Thus, local information specific to individual DER devices that is not needed to support nodal transactions is not included in these nodal communications.

This communication protocol permits a dramatic simplification in overall information and storage requirements. In addition, it facilitates other desirable TES design attributes, such as: privacy protection for DER owners; an ability to accommodate a wide variety of DER devices; and an ability to incorporate an increasing number of DER devices over time.

As detailed in Kok (2013), the transaction nodes envisioned by PowerMatcher for a particular collection of DER devices consist of four distinct types of software agents participating in real-time transactions: Objective Agent; Local Device Agent (LDA); Concentrator; and Auctioneer. An Objective Agent is a software agent that carries out external control actions for specific types of business applications. An LDA is a software agent that communicates bids for power to an adjacent Concentrator or to the Auctioneer (if adjacent) on behalf of a DER device, where each bid indicates the amount of power the device stands ready to consume (absorb) or produce (supply) as a function of the power price.²⁶

A Concentrator aggregates bids for power from adjacent Concentrators and/or from adjacent LDAs and communicates these aggregated bids either to an adjacent Concentrator or to the Auctioneer (if adjacent). The Auctioneer is a single software agent that functions as the top-level aggregator for the entire collection of DER devices. The Auctioneer aggregates bids for power from adjacent Concentrators and immediately communicates a price signal back to these adjacent Concentrators. Each of these Concentrators then immediately communicates a price signal back to the adjacent Concentrators or adjacent LDAs from which it received bids. Each LDA uses its received price signal, together with its bid, to determine the amount of power it is required to consume (absorb) or produce (supply).

The price signal communicated by the Auctioneer to its adjacent Concentrators, conditional on the bids it has just received from them, can be designed to achieve particular objectives. For example, the collection of DER devices could constitute a microgrid in island mode,²⁷ and the price signals that the Auctioneers sends to its adjacent Concentrators, conditional on received bids, could be power prices selected to ensure a continual balance between power supply and power demand on this microgrid.

As reported by Kok (2013, Table 1.1, p. 8) and Widergren et al. (2016), PowerMatcher has performed well in both agent-based simulation and field studies. These studies have included a wide variety of DERs, such as: industrial freezing houses; large industrial Combined Heat and Power (CHP) systems; residential micro-CHP systems; heat pumps; electrical vehicles; and battery storage devices. In these studies, PowerMatcher has demonstrated the following abilities: maintain end-user privacy; ensure incentive compatibility for participating DER owners; reduce peak-load demand, important for system reliability; perform congestion management in distribution networks; integrate renewable energy sources (e.g., wind power); and scale to systems involving more than a million DER device owners.

To facilitate understanding of PowerMatcher's decentralized layered market architecture, it is useful to consider a concrete example. Specifications and outcomes will briefly be reported for a PowerMatcher test case conducted at Iowa State University (Tesfatsion et al., 2017).

The distribution grid for this test case, depicted in Fig. 14, is a 13-bus grid populated with 180 households by instantiating 12 households at each of 15 phase loads. Each of these 180 households has two types of load: (i) conventional (non-price-responsive) load; and (ii) price-responsive load arising from an electric air-conditioning (A/C) system locally managed by an A/C controller with bang-bang (ON/OFF) control settings. To ensure household diversity, households are initialized with randomly distributed structural attributes (e.g., different floor sizes) and with randomly distributed inside air temperatures.

The current state of each household is measured by its current inside air temperature, $T_a$, determined by current weather conditions, past A/C control settings, and structural household attributes. The goal of each household is to ensure that $T_a$ is maintained between a lower temperature level $T_1$ and an upper temperature level $T_2$.

The distribution system is managed by a Distribution System Operator (DSO) tasked with ensuring the reliability of system operations. The goal of the DSO is to ensure that aggregate household A/C power usage for each day D closely tracks a target 24-hour load profile for A/C power usage during day D. In pursuit of this goal, the DSO uses a PowerMatcher design based on two-way communication to manage aggregate household A/C power usage.

More precisely, the DSO and households engage in the following iterative seven-step process:

The demand bid for A/C power usage reported to the DSO by each household at each bid refresh point takes one of three possible forms, depending on the household's current inside air temperature state $T_a$²⁸:

• •  ON ($T_a\ge T_2$): The house is too hot. The A/C system must stay (or be switched) on, regardless of price; hence, the A/C system has no power usage flexibility.
• •  OFF ($T_a\le T_1$): The house is too cold. The A/C system must stay (or be switched) off, regardless of price; hence, the A/C system has no power usage flexibility.
• •  May Run ($T_1<T_a<T_2$): The A/C system stays (or is switched) on if and only if the power price Π does not exceed the maximum price ${\mathrm{\Pi }}^{⁎}(T_a)$ that the household is willing to pay for its ON power usage level $P^{⁎}$, where ${\mathrm{\Pi }}^{⁎}(T_a)$ is an increasing function of $T_a$; see Fig. 15.

The procedure used by the DSO to aggregate household A/C power demand bids at any given time is illustrated in Fig. 16. For simplicity, only two households are depicted, each in a May Run state ($T_1<T_a<T_2$). The maximum acceptable ON-power price function for each household takes the simple form

(6) ${\mathrm{\Pi }}^{⁎}(T_a)=\theta \cdot \left[\frac{T_a-T_1}{T_2-T_1}\right]\text{for}{T}_1<T_a<T_2,$

where $\theta >0$. Household one has a lower inside air temperature $T_a$ than household two. Hence, the value of (6) for household one (labeled A) is smaller than the value of (6) for household two (labeled B).

Fig. 17 reports illustrative outcomes for the ISU PowerMatcher Test Case. As indicated, given suitable time-step specifications for the data check rate, bid refresh rate, aggregate bid refresh rate, price signal rate, and power control rate, the DSO is able to use price signals to ensure that the aggregate A/C load arising from the A/C power usage demands of the 180 households populating the distribution grid closely tracks the DSO's target 24-hour A/C load profile.

This simple test case raises a number of additional issues in need of careful study. For example, if current U.S. distribution systems were to implement a PowerMatcher design managed by a DSO, would this improve the efficiency and reliability of their operations? What precise form would the DSO's objective(s) and constraints have to take to ensure this improvement? And could this be done in a way that ensures revenue sufficiency for the DSO, i.e., coverage of all incurred costs by incoming revenues?

More generally, a DSO with PowerMatcher Auctioneer capabilities could function as an intermediary between a collection of business and/or household device owners on a lower-voltage distribution grid and a wholesale power market operating over a high-voltage transmission grid. In this case broader DSO objectives could be considered, such as the appropriately compensated harnessing of ancillary services from the devices for the support of integrated T/D operations.

For this broader case, another key issue arises: namely, how might the participation of the DSO in the wholesale power market be facilitated by an appropriate form of contract? This issue is addressed below in Section 6.2.

6.1 Overview

A contract is an agreement between two or more parties. Contract design is the study of contract formulation in relation to intended contract purpose. Interestingly, the 2016 Nobel Prize in Economics²⁹ was awarded to two economists, Oliver Hart and Bengt Holmström, for their work on contract design (Royal Swedish Academy of Sciences, 2016).

Markets organized as persistent institutions often rely on explicit legally-binding contracts to provide a careful statement of participant responsibilities as well as the penalties that would apply should participants fail to carry out these responsibilities. Market institutions can take many forms, including: bilateral exchanges permitting individually negotiated bilateral trades; over-the-counter markets managed by a geographically dispersed collection of dealers; and auction markets for which submitted bids/offers are centrally cleared by an auctioneer.

Modern power systems encompass a wide variety of market institutions ranging from bilateral exchanges for financial risk-hedging instruments to ISO/RTO-managed auction markets for the purchase and sale of electric power and ancillary services (NAS, 2016, Chapter 2). These various market forms are based explicitly or implicitly on contractual arrangements. For example, the business practice manuals published by U.S. ISOs/RTOs are legal documents that set out, in excruciating detail, the roles, responsibilities, information access rights, and compensation rules governing the various participants in ISO/RTO-managed markets and supporting processes.

To date, most studies focusing on contract design for electric power systems have focused on the operations of a single wholesale or retail market.³⁰ However, by definition, a Transactive Energy System (TES) is a set of economic and control mechanisms to ensure the balancing of demands and supplies for power across an entire electrical infrastructure. Consequently, TES contract design research requires a broader scope.

TES researchers are well aware of the need to put TES transactions on a secure contractual footing. Consider, for example, the following assertions by the GridWise Architecture Council (2015, p. 12):

“A TE system must clearly define transactions within the context of that system. The following questions (and possibly others not anticipated here) must be able to be answered: Who are the transacting parties, what information is exchanged between them to create a transaction, and what is exchanged between them to execute a transaction? What are the rules governing transactions? What is the mechanism(s) for reaching agreement?”

TES researchers are also well aware of the need to ensure incentive compatibility for their proposed TES architectures, in the sense that these architectures should ensure sustainable business models for all TES participants.

Nevertheless, most TES researchers to date have not explicitly focused on contract design as an important component of TES architectural design. In particular, insufficient attention has been paid to the need to ensure that TES contractual relationships are robust against strategic behavior, i.e., robust against the possibility that transactors with learning capabilities might discover ways to manipulate these contractual relationships for personal advantage at the expense of overall system performance.

The next section uses an illustrative example to indicate how ACE modeling tools could facilitate the development of TES contract designs for flexible power and ancillary service provision that are both compatible with transactor incentives and robust against transactor strategic manipulation.

6.2 ACE Support for TES Contract Design: Illustrative Example

As discussed in Section 1.1, the participation of non-controllable generation (e.g., wind, solar) in U.S. electric power systems is rapidly growing. As discussed in Section 5, the implementation of various forms of demand response in these systems is encouraging more active demand-side participation. The combined effect of these two developments has been a substantial increase in the uncertainty and volatility of net load, i.e., load minus non-controllable generation.

Consequently, system operators tasked with the real-time balancing of net load by means of controllable generation are now seeking ways to ensure greater flexibility in the provision of this controllable generation. However, three issues have impeded these efforts.

First, rigidity in product definitions is hindering appropriate compensation for valuable forms of flexibility in controllable generation, such as flexibility in ramp-rate and duration. Second, eligibility restrictions are preventing the achievement of an even playing field for potential providers of controllable generation. Third, the required payment of market-cleared controllable generation in advance of actual real-time generation,³¹ plus variously required out-of-market make-whole payments for controllable generation (e.g., uplift payments for unit commitment costs), are providing opportunities for strategic manipulation.

Several recent studies (Tesfatsion et al., 2013; Heo and Tesfatsion, 2015; Li and Tesfatsion, 2018) have explored the possibility that all three of these issues could be ameliorated through the use of standardized contracts for power and ancillary services permitting swing (flexibility) in their contractual terms. This swing contract design is intended for use by any dispatchable resource participating in a centrally-managed wholesale power market. These participants could include, for example, entities managing large collections of Distributed Energy Resource (DER) devices as dispatchable virtual power plants (generators) or as dispatchable virtual batteries (prosumers).

The swing contract proposed in these studies permits a resource to offer the availability of power paths with variously specified attributes, such as start-location, start-time, power level, ramp rate, and duration. Moreover, each of these attributes can be offered as a range of values rather than as a point value, thus permitting greater flexibility in real-time implementation.

For illustration, consider the following swing contract that permits a dispatchable resource to offer power paths with swing (flexibility) in both their power level and their ramp rate:

(7) $\begin{array}{cc}\hfill & \text{SC}=[b,t_s,t_e,\mathcal{P},\mathcal{R},\varphi ]\hfill \\ \hfill & b=\text{location where service delivery is to occur;}\hfill \\ \hfill & t_s=\text{power delivery start time;}\hfill \\ \hfill & t_e=\text{power delivery end time;}\hfill \\ \hfill & \mathcal{P}=[P^{min},P^{max}]=\text{range of power levels }p\text{;}\hfill \\ \hfill & \mathcal{R}=[-R^D,R^U]=\text{range of down/up ramp rates }r\text{;}\hfill \\ \hfill & \varphi =\text{performance payment method for real-time services.}\hfill \end{array}$

In (7), $t_s$ and $t_e$ denote specific calendar times expressed at the granularity of time periods of length Δt (e.g., 1 h, 1 min), with $t_s<t_e$. The power interval bounds $P^{min}\le P^{max}$ can represent pure power injections (if $0\le P^{min}$), pure power withdrawals or absorptions (if $P^{max}\le 0$), or bi-directional power capabilities (if $P^{min}\le 0\le P^{max}$). The down/up limits $-R^D$ and $R^U$ for the ramp rates r (MW/Δt) are assumed to satisfy $-R^D\le 0\le R^U$. The location b, the start time $t_s$, and the end time $t_e$ are all specified as single values in (7). However, the power levels p and the down/up ramp rates r are specified in swing form with associated ranges $\mathcal{P}$ and $\mathcal{R}$.

As discussed at greater length in Heo and Tesfatsion (2015), the performance payment method ϕ in (7) that designates the mode of ex post compensation for actual real-time service performance can take a wide variety of forms, such as a specified flat rate for energy and/or a power-mileage compensation for ramping. Moreover, ϕ can include penalties or incentive payments to encourage accurate following of real-time dispatch instructions.

To understand the obligations of the seller and buyer of this swing contract, should it be cleared, a more concrete example might be helpful. Consider a DER aggregator that manages a large collection of DER devices owned by retail customers. Assume the number and diversity of these devices permits the DER aggregator to function as a dispatchable virtual power plant with controllable down/up power within predictable ramp rate limits.

Suppose the DER aggregator offers a swing contract into an ISO-managed day-ahead market (DAM) at an availability price $\alpha =\mathrm{\$}100$, with $\mathrm{\Delta }t=1$ h. Under this contract the DER aggregator offers to inject power into the transmission grid at bus b from 8:00 am to 10:00 am on the following day. The offered power levels range from 10 MW to 40 MW, but the required down/up ramp rates r to achieve these power levels must satisfy $-38\text{MW}/\mathrm{h}\le r\le 28$ MW/h.

Suppose, also, that the performance payment method ϕ appearing among the terms of this swing contract requires the DER aggregator to be paid the price $\varphi =\mathrm{\$}35/\text{MWh}$ for each MWh of energy it delivers. This payment is a pay-for-performance obligation. That is, the payment is not due until after the actual delivery of energy has been made. However, if the swing contract is cleared by the ISO, the DER aggregator is immediately entitled to receive its availability price $\alpha =\mathrm{\$}100$.

Fig. 18A summarizes the contractual terms of this swing contract, and Fig. 18B depicts one possible power path that the ISO could dispatch in real-time operations in accordance with these contractual terms. The darkened (green) area under this power path is the corresponding energy (MWh) delivery, to be compensated ex post at the rate of $35/MWh.

Li and Tesfatsion (2018) develop an analytic model of an ISO/RTO-managed DAM for which any market participant with dispatchable resources is able to offer services from these resources using a swing contract. They demonstrate that the ISO/RTO can determine the optimal market clearing of these swing contracts by means of a mixed integer linear programming (MILP) formulation, solvable using standard MILP solution software.

In particular, as depicted in Fig. 19, this optimal market clearing of swing contracts accomplishes both Security Constrained Unit Commitment (SCUC) and Security-Constrained Economic Dispatch (SCED) subject to the usual types of physical constraints. Moreover, it does so without requiring any performance payments to be made in advance of actual performance, or any out-of-market uplift payments to be made for unit commitment costs.

Tesfatsion et al. (2013) and Heo and Tesfatsion (2015) discuss in detail a long list of potential advantages that could result from the use of swing contracts in a linked sequence of ISO/RTO-managed power markets with planning horizons ranging from years to minutes. These assertions are supported by simple analytical swing-contract examples. Hence, in terms of the Policy Readiness Levels (PRLs) defined in Section 2.4, these studies would be classified as PRL 2.

In contrast, Li and Tesfatsion (2018) provide a specific analytic optimization formulation for achieving the optimal market clearing of DAMs permitting swing contracting for all dispatchable resources, and they demonstrate the effectiveness of this formulation by means of a numerical application. Thus, this work is at PRL 3. However, any real-world implementation (PRL 9) of these swing-contract ideas will require substantial preliminary work to be carried out at each of the intervening PRL levels 4–8. In particular, it will require a crossing of the “valley of death” (PRLs 4–6).

Researchers at Iowa State University are currently developing an ACE computational platform at PRL 5 for the express purpose of studying the performance of swing contracts for DER aggregators within integrated transmission and distribution (ITD) systems. This platform, referred to as the ITD Test System, is an extended version of the IRW Test Bed discussed in Section 5.2.1; see Fig. 8 for a depiction of its intended final form. The ITD Test System will retain a key ability of the IRW Test Bed: namely, the ability to model decision-making agents as strategic agents with learning capabilities and local objectives.

An ACE computational platform, such as the ITD Test System, would permit many critical issues to be explored for swing contract market designs at successively greater scales and with successively greater empirical fidelity. A number of these issues are highlighted by Gallo (2016, Sections 2–3).

For example, would the two-part pricing permitted by swing contracting, in particular the ex-ante payment for service availability versus the ex-post payment for actual service performance, result in more appropriate compensation for the full range of services provided by thermal generation, including ancillary service performance? If so, this would alleviate the well-known “merit order effects” that have arisen in ISO/RTO-managed wholesale power markets due to the increased penetration of renewable energy resources with subsidized or naturally low marginal production costs.³²

Also, would this two-part pricing help to resolve the “missing money” problem, i.e., the inability of some resource owners participating in current electric power markets to recover their full costs without various types of out-of-market payments? Would it eliminate the need for capacity markets in which payments are made for capacity availability without any direct tie-in to real-time performance? Would the elimination of payments in advance of performance, and out-of-market payments, reduce opportunities for participants to exercise market power at the expense of overall system reliability and efficiency?

Of course, subsequent to these PRL 4–6 studies, prototype large-scale studies and field work at PRLs 7–8 would have to be undertaken to test the performance of swing contract market designs in expected field conditions. Presumably such studies would be undertaken in collaboration with industry and/or national laboratory partners.