Structural Contingency Theory

Structural contingency theorystructural contingency theory (SCT) is probably the first clearly articulated organization theory that pulled the environment in as a major factor in the structure and actions of a given organization. Three notions central to SCT are starting points for any consideration of this theory. First, there is no one best way to organize and manage activity. By contrast, the notion of a “best” way to organize activity had been an essential part of organization theory throughout the early part of the twentieth century and may be seen in the work of such authors as Taylor (1911) and Fayol (1917). Second, although there may be no best way of organizing, not all ways are equally effective. That is, in various circumstances, there may exist better ways to organize among the options available. Third, the “better” ways to organize probably depend on the nature of the task environment to which the organization relates and in which it is embedded.

In its earliest manifestations, SCT pivoted on the question of the role of different technologies in the work processes and structures within an organization, and there arose a sort of technological determinism: if the technology of a particular health service was experimental, innovative, complicated, and with uncertain results, then the structure of work was posited to be “organic,” nonhierarchical, fluid, and egalitarian (Hage and Aiken, 1969). Contrariwise, if the technology were standardized, simple, uncomplicated, and with almost certain results, then the structure of work was posited to be “mechanistic,” hierarchical, and rule bound. As an example, in the early days of kidney dialysis and kidney transplants, when there was high uncertainty about the effectiveness of various procedures and a good deal of error in applying the procedures, the structure of the provider teams was loose, changeable, and egalitarian (Fox, 1959). Today, with the enormous progress in medical knowledge surrounding kidney disease and the high level of certainty of the treatment outcomes, kidney treatment procedures are highly standardized and generally delivered in structured settings with tight role definitions. There is virtually nothing experimental about these treatments. In short, as the technology evolved from an experimental and uncertain mode to one that is highly certain and reliable with predictable outcomes, so has the evolution of the structures within which this clinical work is performed moved from loose and collegial relationships to tighter, rule-driven, and hierarchical relationships. Such transitions of and progress in diagnostic and treatment approaches across the spectrum of medical care are common and are usually accompanied by the switch in organizational structures depicted in the kidney dialysis example.

SCT accepted the basic premise of earlier approaches to the structure of organizations: that the nature of the technology (in its broadest sense) explained the design or structure of work in organizations, and there were good ways and bad ways to organize work. Perrow's (1967) framework for analyzing organizations, adapted for health care organizations by Smith and Kaluzny (1975), provided a fundamental statement and propositions espoused by SCT. By describing the clinical tasks and their accompanying technology according to two different dimensions—few versus many exceptional cases and clearly understood and specified clinical tasks versus not well understood or specified procedures—one may cross-hatch them to create four distinct categories: (1) few exceptional cases and clearly understood clinical tasks, (2) many exceptional cases and clearly understood clinical tasks, (3) few exceptional cases and poorly understood clinical tasks, and (4) many exceptional cases and poorly understood clinical tasks. Examples of organizations or organizational units that reflect the first situation tend to be a more bureaucratic form with a high level of standardization, vertical authority structures, and formalization of relationships generally. Public health departments, immunization programs, and routine testing clinics fall into this domain.

Examples in the second category are organizational units that provide urgent care services. A more professional organizational model tends to prevail in which there is some formal structure but a more peer-oriented set of interpersonal relationships. Authority is somewhat more shared than the instance of the first category, although some bureaucratic tendencies persist. In the third category, the few exceptional cases, which would ordinarily lead to bureaucratic organizational traits, tend to be offset by the poor understanding of the clinical tasks, which requires a more informal organizational form to allow an experimental, risk-taking, and often unpredictable approach to patient care. Hence, a more horizontal, egalitarian, and professional set of relationships tends to exist for this organization approach. Examples include care provided in mental health services for schizophrenia and other mental disorders. Finally, in the fourth category, the organizational arrangements strongly tend to be professional and nonbureaucratic, as described in the way people interrelated during the early days of treatment for kidney problems. Examples abound today: for instance, genomic medicine, where every patient is different and where the connection between genetic disposition and disease is still very uncertain, and treatment settings for breast cancer, given that there are so many different and subtle types of cancer of this part of the body, for which many treatment modalities are completely ineffective.

The overriding point is that there tends to be a correspondence or fit of the contingency, in this case, the specific diseases or injuries of the presenting patient and the organizational form most appropriate for the care delivery. This is the classical SCT paradigm. It argues that certain features of structure should be given greater or lesser emphasis depending on the task technologies involved in patient care. The idea was and still is that some structures fit these task technologies better than others. The “goodness” or “badness” had to do with the fit between the technology and the structure, which led to the slogan, “Form follows function.” That is, the structure must be related to and fit with the function under consideration, and the function was a proxy for the technology involved in the task.

Studies in health care using SCT include Argote's (1982) work on input uncertainty and organizational coordination in hospital emergency departments, Leatt and Schneck's (1984) study of ways of grouping nursing subunits in hospitals, Alexander and Randolph's (1985) assessment of fit between technology and the organization of nursing subunits, Mark and colleagues' (2008) work on medication errors and patient safety, Swofford's (2011) empirical work on rural hospitals' affiliation with larger systems and their fit or lack of fit with the contingencies of the environment, Van de Ven and colleagues' (2012) examination of performance across a large medical group practice and its subsidiary clinics, and Shay and Ozcan's (2013) work examining the relation of fit to inpatient rehabilitation facility performance following regulation. (The perspective's contemporary relevance is the subject of chapter 9.) The authors suggest the pertinence of an adapted and updated SCT approach to explore the relationship of new organizational forms and processes given recent technological advances in genomic medicine.

A final word about SCT is that it is a theoretical perspective that examines not only the effect of the degree of fit between organizational forms and impinging contingencies on performance but also the role of management in finding and keeping the best degree of fit possible. Often thought of as a perspective stressing the deterministic forces of technology in the environment, SCT's position on managerial discretion is clear: management matters. Lawrence and Lorsch (1967) spend at least one entire chapter in their groundbreaking book outlining what management should and should not do to foster improved performance. For example, they write: “Any attempt to use the decentralization and integration approach systematically would have to begin with a diagnostic study of the organization and its immediate environment” (p. 215). In other words, it is management's responsibility to do the strategic assessments necessary to determine the best organizational form given one or another context. These authors call for a “more intelligent tailoring of organizations to their task and environments” (p. 212). They further discuss the need for various approaches to conflict resolution that they argue will inevitably arise when restructuring occurs. In short, the SCT perspective, although emphasizing the strong deterministic impact of technologies on the work setting, simultaneously accords to management the importance of the quality of its interventions to improve fit and therefore performance (Donaldson, 2001). Connected to this work, chapter 8 in this book addresses the rise of hospital-based systems in local and regional markets (also referred to as clusters) and uses Lawrence and Lorsch's (1967) perspectives on differentiation and integration as a launching pad to describe and explain these clusters' varied organizational forms.

Resource Dependence Theory

Whereas SCT was establishing itself as an important way to examine the interplay of organizational behavior and the environment, particularly its technological features, Pfeffer and Salancik (2003) introduced a different perspective on this question through resource dependence theory (RDT). In contrast to SCT, RDT's underlying approach was based on ways that power was acquired, maintained, and exercised. Technological shifts were not unimportant, but the way technology was treated was within a power perspective. RDT's basic premise is that organizations, dependent on resource pools, glean them from their relevant environments, which consist of many other organizations. Many of these other organizations have the resources required by the focal organization, and because of this resource asymmetry, power accrues to the organizations that hold the needed resources. Thus, the relationship between any two organizations is defined by the degree to which resources are symmetrical, and levels of interdependence between any two organizations are a function of the degree to which each organization has resources that the other requires. In other words, RDT describes such relationships as the uneven distribution of power between or among organizations.

In addition, the greater the asymmetry there is between a focal organization and others in its environment, the more there is environmental uncertainty. The key asymmetries are the concentration of resources outside the organization (the distribution of power or authority in the environment), the interconnectedness of the organization with others, and the level of munificence of these resources. The importance of any given resource to an organization is based on the magnitude of the resource used and how critical the resource is to the performance of the organization's core business. However, it is not just an issue of how critical a resource is to an organization so long as there is predictability and stability in the supply of the resource in question. The difficulties begin when the resource is conditioned by uncertainty.

Under conditions of uncertainty, organizational survival depends on an organization's ability to procure and maintain key resources (Alexander and Morrissey, 1989; Sheppard, 1995; Casciaro and Piskorski, 2005), and organizations enter into exchange relationships to manage the demands of groups on which the organization depends for these resources, thereby ensuring survival (Sheppard, 1995; Pfeffer and Salancik, 2003; Levine and White, 1961). RDT initially focused on board interlocks, alliances, and mergers, but more recent work has turned to the adaptation of organizations to their environment to find the least constraining approach to coordinating external relationships (Scott and Davis, 2007). Organizations take action in a number of ways that include buffering strategies designed to control the flow of resources to the technical core, bridging strategies where links to other important organizations are developed, and adaptation to gain legitimacy in the eyes of external stakeholders and resource providers.

An important way to influence exchange relationships includes increasing the size of the organization, thereby increasing its power and ability to survive (Pfeffer and Salancik, 2003; Kotter, 1979; Sheppard, 1995; Scott and Davis, 2007). Large organizations possess the power and resources to accommodate environmental demands, and their size and slack resources also permit them to “experiment with new strategies” with a relatively lower risk of failure during times of environmental change (Zinn, Weech, and Brannon, 1998, p. 264; Zinn and Flood, 2009). In addition, bridging mechanisms have been identified as key methods to manage organizational interdependence, including cooptation, alliances, and mergers and acquisitions (Sheppard, 1995; Scott and Davis, 2007). As organizations evaluate these opportunities to manage organizational interdependence, they must weigh the trade-off that exists between autonomy and environmental stability (Pfeffer and Salancik, 2003; Scott and Davis, 2007). Studies suggest that organizations' tactics for managing interorganizational relationships are often based on their power positions, and they will “adopt the least constraining structures … sufficient to maintain autonomy and ensure access to critical resources” (Scott and Davis, 2007, p. 244).

Instances of RDT's applicability to questions in health care show its wide relevance. Banaszak-Holl, Zinn, and Mor (1996) studied the impact of market and organizational characteristics on nursing care facility service innovations. A study by Zinn et al. (1998) supported some of RDT's propositions concerning the adoption of Total Quality Management (TQM) programs among nursing homes. Alexander and Morrissey (1989) found that market conditions, regulatory factors, and munificence were significantly associated with hospital contract management. Kazley and Ozcan (2007) uncovered RDT factors that determine prevalence of electronic medical record adoption in acute care hospitals. Fareed and Mick (2011) applied RDT to provide a series of hypotheses predicting which hospitals will adopt certain patient safety measures.

Like SCT and, though somewhat understated, an important feature of RDT is that it allows for some human agency: the ability and the potency of human decisions in altering the course of organizational action. Although it is easy to think of RDT as a perspective that completely limits the options of organizations in the face of uncertainty and that emphasizes the paradox that securing resources may mean giving up some of their independence, it is actually the case that Pfeffer and Salancik (2003) emphasized three critical roles of management in developing effective strategies. Symbolic, responsive, and discretionary roles are each available and, to a greater or lesser degree, may allow an organization to express itself through acquired power to effect change. Pfeffer and Salancik (2003) write:

Although organizations were obviously constrained by their situations and environments, there were opportunities to do things, such as coopting . . . sources of constraint, to obtain, at least temporarily, more autonomy and the ability to pursue organizational interests. . . . In other words . . . strategic choice was both possible . . . and sometimes, although not inevitably, efficacious because the strategies to overcome constraint sometimes worked. (p. xii)

Hence, in explaining how organizations manage their relationships with other organizations, RDT posits that (1) organizations do respond to the environment that they find themselves in, (2) they can develop varied strategies to enhance their autonomy and pursue their interests, and (3) power rather than efficiency is important for understanding what goes on inside organizations and what external actions they take. Although organizations try to maintain their autonomy and reduce dependence on their environment, they also recognize the need to form certain coalitions or networks to pool resources and increase power in negotiating with dominant organizations (Shortell and Kaluzny, 2000). In short, RDT assumes that managers can actively and strategically influence their environment to reduce undesired dependencies and enhance survivability (Aldrich and Pfeffer, 1976).

Institutional Theory

In contrast to the two previous theories, which posit a rationalistic view of organizations, institutional theory rests on a so-called naturalistic view of organizations (Scott and Davis, 2007). That is, organizations may engage in behavior that does not actually improve the effectiveness and efficiency of outcomes but improves its chances of survival through engagement with legitimizing structures and processes. This view is based on a broad set of observations that much organizational activity seems only remotely related to improved performance in the economic sense and that ostensibly poorly performing organizations manage to survive and thrive in ways in which both SCT and RDT, not to mention economic perspectives, would predict failure.

Institutional theory recognizes that organizations are fraught with internal and external pressures, and in order to survive, they must account for such pressures and convince others of their legitimacy (Scott and Davis, 2007; Mizruchi and Fein, 1999; Selznick, 1996; Meyer and Rowan, 1977). The development of institutional theory is often divided into two periods: “old” institutionalism developed in the 1940s and 1950s, and “new” institutionalism, also referred to as neoinstitutional theory, that emerged from seminal work in the late 1970s and 1980s.

Institutional theory's roots are often traced to the collective contributions of Philip Selznick (1948 1949, 1952 1957), who described the process of “institutionalization,” in which distinct organizational “forms, processes, strategies, outlooks, and competences” become infused with value as they “emerge from patterns of organizational interaction and adaptation” (Selznick, 1996, p. 271). As these patterns are adopted and replicated by other organizations and throughout society, they are given common meaning, furthering their institutionalization (Scott and Davis, 2007; Scott, 1987).

Whereas early institutional theorists acknowledged the normative and regulative pressures influencing organizations, neoinstitutionalists such as Meyer and Rowan (1977), Zucker (1977), and DiMaggio and Powell (1983) expanded the theory by bringing attention to the importance of cultural, cognitive, and symbolic elements that also affected organizations' behaviors (Scott, 2004; Zucker, 1987). New institutional theory also emphasizes the effects of the institutionalization process, including the widespread adoption of ceremonial practices (Meyer and Rowan, 1977) and the evident mimicry among organizations (DiMaggio and Powell, 1983).

Although these two camps of institutional thought have criticized and competed with one another, recent attempts to reconcile the two paradigms have focused on their collective contributions and shared insights (Abrutyn and Turner, 2011; Hirsch and Lounsbury, 1997; Greenwood and Hinings, 1996; Selznick, 1996). When elements of both old and new institutionalisms are incorporated, a balanced institutional theoretical approach offers a valuable perspective that speaks to the social factors influencing organizations' behaviors.

Institutional theory recognizes that organizations work within two distinct types of environments: technical and institutional (Alexander and D'Aunno, 1990). Technical environments yield economic rewards to organizations exhibiting efficiency, effectiveness, and coordination; in contrast, institutional environments offer social rewards and support to organizations exhibiting conformity to legitimated practices and cultural rules, even in the absence of efficiency (Scott, 1987; Zucker, 1987; Scott and Meyer, 1983; Meyer and Rowan, 1977; Selznick, 1957). Legitimacy is one of the key constructs of institutional theory, which Suchman (1995) broadly defined as “a generalized perception or assumption that the actions of an entity are desirable, proper, or appropriate within some socially constructed system of norms, values, beliefs, and definitions” (p. 574). Within institutional environments, organizations desire to appear legitimate in their practices and structures, and such legitimacy yields access to needed resources and ensures survival (Meyer and Rowan, 1977).

Institutional theorists have identified three types of institutional pressures that organizations face in their environments: regulative, normative, and cultural-cognitive pressures (Scott, 2004). Regulative pressures include those created by various rules, laws, and governance systems that coerce organizations to comply with institutionalized behaviors. Normative pressures include the adopted traditions and norms that define the social obligations, appropriate behaviors, and common morals and values espoused by society, and such pressures often extend from professional groups, religion, educational systems, and family structures (Scott and Davis, 2007; Zucker, 1987; DiMaggio and Powell, 1983). Cultural-cognitive pressures include internalized beliefs and taken-for-granted social realities that dictate social life (Meyer and Rowan, 1977; Zucker, 1977). In light of regulative, normative, and cultural-cognitive elements, an organization gains legitimacy to the extent that its behavior “conforms to existing rules and laws,” is “consistent with widely shared norms defining appropriate behavior,” and conforms to “mental constructs” as well as “common symbolic systems and shared meanings,” respectively (Scott and Davis, 2007, pp. 259–260).

In response to these institutional pressures, organizations have been observed to resemble one another more and more over time, a process of homogenization known as isomorphism (Mizruchi and Fein, 1999; DiMaggio and Powell, 1983; Meyer and Rowan, 1977). DiMaggio and Powell (1983) distinguished institutional isomorphism, emphasizing adaptation, from the competitive form of isomorphism espoused by population ecologists such as Hannan and Freeman (1977), which emphasized selection. Such institutional isomorphic change occurs as organizations strive for legitimacy and compete for market survival, and DiMaggio and Powell (1983) specified three mechanisms of institutional isomorphism evident among organizations: coercive isomorphism, normative isomorphism, and mimetic isomorphism. Coercive isomorphism occurs as organizations conform to political pressures and expectations exerted by other organizations and societal groups on which they depend; normative isomorphism occurs as organizations are composed of individuals shaped by their education, professional values, and social networks; and mimetic isomorphism occurs as organizations respond to pervasive uncertainty by modeling themselves after other organizations (DiMaggio and Powell, 1983). Organizations displaying isomorphic change and adopting legitimate structures and behaviors do so “not necessarily because [such] forms are technically appropriate but rather because they conform to socially accepted notions of what is appropriate” (Mizruchi and Fein, 1999, p. 679).

As organizations respond to various institutional elements, they also begin to adopt and perpetuate constructed stories and ceremonial activities, which Meyer and Rowan (1977) refer to as “rationalized myths.” Such symbolic practices, despite often being separate from an organization's actual behaviors, attract the attention and approval of the public, eventually defining the organization (Mizruchi and Fein, 1999). These rationalized myths are adopted to ensure an organization's legitimacy and survival, leading organizations to decouple their structures and activities, adopt elaborate displays of satisfaction and success, and reduce inspection and evaluation activities to ceremonial procedures (Meyer and Rowan, 1977).

Scholarship based on institutional theory has focused on the development and influence of institutional logics, which drew attention from neoinstitutionalism's interest in organizational fields (Scott and Davis, 2007; Hirsch and Lounsbury, 1997). Organizational fields are defined as diverse organizations that are linked through supporting, governing, and competing relationships and together “constitute a recognized area of institutional life” (DiMaggio and Powell, 1983, p. 148). Within these fields, institutional logics are formed and shared, serving as the practices, social assumptions, symbolic constructions, established values, and organizing principles of a given field that help define organizational reality and provide guidance to organizational behaviors (Thornton and Ocasio, 1999; Friedland and Alford, 1991). At any given time within an organizational field, multiple institutional logics may exist and even compete, with logics stemming from institutions such as professions, markets, religion, government, and corporations (Lounsbury, 2007; Thornton, 2002; Thornton and Ocasio, 1999). The existence of multiple logics that conflict and compete is not without consequence. In some instances, this leads to the transformation of institutional logics over time as previously dominant logics succumb to newer prevailing logics, thereby producing continual change, reinstitutionalization, variation in behaviors, and variation in the adoption rate of practices (Lounsbury, 2002 2007; Arndt and Bigelow, 2006). In other instances, rival institutional logics may be managed through collaboration to allow their continued existence, with future change the result of collaborative activities (Reay and Hinings, 2009).

Institutional theory has proven to be a valuable perspective within health care organization studies. Hospitals and health care organizations today simultaneously face strong technical and institutional environments, as well as an array of conflicting institutional pressures and logics (White, 2003; Arndt and Bigelow, 2000; D'Aunno, Succi, and Alexander, 2000). For example, following the “corporatization” of health care, today's health care organizations are pressured to operate under a business-driven logic of efficiency (Arndt and Bigelow, 2006; Reay and Hinings, 2005) while also adhering to societal expectations that health services be provided first for public benefit rather than corporate profit (Alexander and D'Aunno, 2003). This conflict is particularly evident among Catholic-sponsored health care organizations, which are held to expectations to be equal in their provision of efficient, high-quality care yet distinct in their Catholic identity and values (White, 2000). As the health care sector is frequently seen as highly complex, confusing, dynamic, and with an uncertain nature, institutional theory is particularly valuable in its explanation of how a “range of accepted organizational practices and templates” may abound from an array of institutional pressures and logics (D'Aunno et al., 2000, p. 682).

Finally, as Thomas D'Aunno writes in chapter 4 of this book, institutionalists have had to come to terms with changes in institutional forces themselves. For the most part, the causal direction asserted in classical institutional and neoinstitutional theory has been from environmental or exogenous institutional forces as they impinge on organizations. But it has become clearer that organizations can and do influence their institutional environments, and how and when this happens is a new domain of study. Institutional forces may be influenced by specific strategic activity by an organization or alliances of organizations; political lobbying is a clear example. Or demographic change may rupture a traditional institutional force. For example, throughout US history, immigrant groups have been pressured to learn English to accomplish socialization and acculturation as “Americans.” In the early 2000s, reversing this long tradition, learning the minority language has actually been part of a new institutional force, particularly in communities and states with a large number of Hispanic immigrants. In short, institutional forces can and do change (chapter 4 presents a thorough treatment of this issue).

Population Ecology

Population ecologypopulation ecology (PE) in organization theory is a perspective that was initially viewed as perhaps the most radical statement about environmental determinism in forming and destroying organizations. This perspective focuses on how social conditions and environmental changes affect the rates at which new organizations form, the rates at which they change, and the rates at which they die (Hannan and Freeman, 1989). In addition, it is not individual organizational activity that is studied, but whole fields or populations of organizations form the subject matter.

A key idea in the PE framework is that organizations exist or do not exist almost solely as a function of exogenous forces well beyond the control of any individual organization, organizational actors, or group of organizations (Aldrich, 1979). This apparently extreme statement makes better sense once it is realized that PE speaks not about individual organizations but about entire organizational types or fields. For example, one may cite various organizational types that have disappeared or are in the process of disappearing: small private physician practices in primary care and specialty areas, independent freestanding nonprofit community hospitals, and individually owned freestanding pharmacies. There is even evidence that some religious-sponsored health systems are disappearing or converting their ownership (e.g., Catholic Healthcare West converted to a nondenominational system renamed Dignity Health). One may also cite examples of organizational types that are increasing in number: retail chain store pharmacies, retail health care services generally, freestanding ambulatory surgery centers, and geographically clustered hospital systems. Experimental organizational forms like patient-centered medical homes and accountable care organizations may well become the future norm of organizational delivery forms that new environmental circumstances favor.

A second key issue surrounding the life and death of organizations is the question of why they tend to resemble one another through their lifetimes. This is also a question that institutional theorists ask: Why is there this sameness that appears over time? PE theorists have a different answer than institutional theorists do.

According to the PE perspective, the existence of similar organizational forms is determined by an evolutionary process in which forms that fit environmental requirements are retained, while forms that do not exhibit such fit are selected out (Lewin, Weigelt, and Emery, 2004; Baum and Rowley, 2002; Aldrich, 1979; Hannan and Freeman, 1977). Thus, as certain forms are retained and others do not survive through an evolutionary selection process, the remaining organizations are those with similar forms that are best suited for survival. This does not mean that a single type of organizational form, once it is retained, will always be assured of survival. The environment and its requirements are subject to continual change, and whether an organization meets such requirements may depend on a variety of elements, including abrupt technological changes, an organization's demographic traits (e.g., size and age), ecological processes, and environmental pressures. As change occurs within environments such that new requirements and demands favor new organizational forms, the evolutionary process continues (Lewin et al., 2004; Scott, 2004).

From an empirical standpoint, one may think of this relationship of organizational forms, their fit with environmental requirements, and survival in terms of dependent and independent variables. PE's dependent variables of interest are the birth, change, and death of organizations, particularly at the population level. By population level, we refer to an entire type of organization, such as small, rural, general acute care hospitals. Independent variables might include aspects of the environment and organization that are summarized by these large groupings (Scott and Davis, 2007): technology, demographic factors, ecological factors, and environmental processes.

The idea relating to technology's influence on organizational formation, change, and death is that new organizations arise with abrupt changes in technology. For example, in health care, freestanding imaging clinics emerged due to the wide and rapid diffusion of computed tomography scans and magnetic resonance imaging machines. There is hardly any reason for these technologies to be sequestered inside a hospital. So within the PE perspective, interest has focused on what are called periods of rapid and discontinuous change, which theorists refer to as “punctuated equilibrium.” The other side of this is the often observed tendency of organizations not to change due to sunk costs, inertia, levels of comfort, imprinting at birth, hard-earned learning, and competencies that they are loath to cast off. Such organizations face dim prospects for survival, and PE recognizes the extreme difficulty organizations experience in adapting to change (Lewin et al., 2004; Baum and Rowley, 2002).

Demographic factors refer to the demography of the organizational population itself, with a focus on size and age. Age is seen in a mixed way: although there are clear benefits to increasing age, there is also the problem of inertia. But up to a point, organizational learning and experience override the “liability of newness.” Some have shown that age follows an inverted U distribution (Wholey, Christianson, and Sanchez, 1992) with midsized organizations—here health maintenance organizations—being at greatest risk of failure compared to small or large organizations. In general, larger size may also be a marker for organizational slack, which provides a cushion and resources to deal with external pressures that a smaller organization cannot withstand (Zinn and Flood, 2009).

Two types of ecological factors are of general interest: niche width (or specialist versus generalist) and density dependence. In terms of niche width, the resources within the environment allow the development of niches that can support a certain population size, also called carrying capacity (Aldrich, 1987). In a niche, the specialist organization takes advantage of holes in the environment in which an organization can thrive due to its filling a small breach that larger and more general organizations do not fill. Specialist organizations often do a much better job of what their central core competency is than generalist organizations do. But operating as a generalist organization provides a capacity to deal with more forces and stakeholders, being many things to many others and thus being less vulnerable to any specific threat. Density dependence refers to the number of previous foundings and failures in an organizational population. This idea presents a paradox in its signaling about the likelihood that new entrants to the field will be able to exist for very long. Many failures release resources that new organizations can sop up, but they also indicate that something is wrong with the resources and elements in the environment or that the environment itself has changed.

Finally, environmental processes refer to the broader processes in the social, political, and economic realms and is a sort of grab-bag category to cover population-level forces not captured by the notions already discussed. These processes and pressures are manifold and legion, public and private, large and small, short-lived and enduring. In health care, we can identify a host of public and private regulatory and policy bodies, payment organizations, and professional associations. In general, this category refers to the mix of regulatory and market forces that are at play all the time in health care but are increasing in complexity, diversity, and demands for accountability.

PE theorists are interested mainly in processes that occur over time, or longitudinally, rather than cross-sectional relationships. These processes take place between the independent and dependent variables previously described, and they include variation, selection, retention and diffusion, and competition (Aldrich, 1979 1987). First, there is variation, which can occur among organizations, and the variation can be planned or unplanned. In health care, this variation is frequently caused by new diagnostic and therapeutic technologies that radically change the prospects for new organizational forms. As a result of variation, new and raw material is provided for the second population process, selection, and circumstantial forces that cause selection to happen. Given that the environment contains resources for which organizations compete, “some variations prove more beneficial than others in acquiring resources in a competitive environment” (Aldrich, 1987, p. 261). Thus, the positive selection of forms is based on criteria that favor certain variations over others or even eliminate other variations entirely. The third stage of the change process is the retention and diffusion stage in which favored variations are preserved, duplicated, and reproduced among surviving organizations and new generations of organizations (Aldrich, 1987). Finally, competition occurs as surviving organizations within and between populations battle for needed resources.

In health care, a good example of these stages is the rise of multihospital systems, which are discussed in a different context in chapter 8. From the late 1960s into the 1970s, hospitals combining into systems were but one variant of several interorganizational forms built from the classical community-based stand-alone hospital, whether for-profit or nonprofit. Other interorganizational forms included cooperatives, alliances, and more loosely integrated organizational forms. Hence, variation existed in interorganizational arrangements among hospitals. In the second stage of selection, the multihospital system quickly gained superiority over the other forms. The relatively rapid decision-making ability of these systems, the control of key resources, the development of organizational slack, and the economies of scale favored these systems over other arrangements. The third stage of retention and diffusion, after the selection and replication of the form had taken place, has now led to multihospital systems as an entrenched organizational form across the geography of the United States, showing every sign of being the most effective way that acute care inpatient services can survive the withering forces emanating from market and regulatory pressures. And regarding the fourth stage of competition, we see multihospital systems today as fierce and dominant competitors within their local markets. More generally, organizations left standing are considered those that the forces behind organization ecology have favored with “fit.” Illustrating PE's relevance to multihospital systems, Lewis and Alexander (1986) applied the PE perspective in their development of a taxonomy of these systems. Other applications of PE within health care organization studies include Fennell's (1980) analysis of the geographical grouping of competing hospitals, a study of rural primary care clinics by Ricketts and colleagues (1987), Alexander and Amburgey's (1987) examination of changes in the American hospital industry, and a series of studies by Wholey and colleagues (Wholey and Sanchez, 1991; Wholey et al., 1992 1993) examining health maintenance organizations.

Some charge the PE approach as too reliant on a biological model of organizations (Scott and Davis, 2007), emphasizing a conflict-laden and confrontational social Darwinism that fails to consider the numerous ways in which organizations engage in cooperative, consensual, and mutual relationships. A more balanced approach may incorporate both conflict and cooperation as outcomes of organizations involved in resource changing environments. However, from PE's historical perspective that views entire types of health care organizations over a long period of time, the rise and fall of organizational forms find a ready explanation in the macrolevel processes of ecological selection.

Principal-Agent Theory

Principal-agent theoryprincipal-agent theory (called simply “agency theory” in this chapter) is a major contribution by economics to a perspective that is both highly rational and focused on how organizations can improve their efficiency, the major outcome of interest. Some observers would not automatically turn to economics for insights into the nature and behavior of organizations because economists often depict organizations as black boxes, which are of less interest than the markets in which they find themselves. Furthermore, there is the belief among many economists that organizations are epiphenomena of market imperfections: if people had perfect or near-perfect information about markets such that exchanges were made with full knowledge by the buyer of the seller and vice versa, organizations as we know them would not need to exist because most exchanges could be made contractually among small-sized groups. But because the costs of making exchanges can increase rapidly through market contracting under conditions such as “asset specificity” and “opportunism,” such expenses may outweigh the costs of simply producing the goods or services of interest inside an organizational structure (Akerlof, 1970; Klein, Crawford, and Alchian, 1978). So there is a tradition of accepting organizations as a necessary evil that is fundamentally inferior to market-based exchanges. To paraphrase Nobel laureate and economist Oliver Williamson (1975), “In the beginning, there were markets” (p. 20).

In health care, the antiorganizational sentiment is sometimes explicit, as when James Robinson (1997) compared contractual exchanges among medical providers to internal organizational exchanges in integrated delivery systems. He found the former superior to the latter and characterized the organizational alternative for physicians as leading to a loss of “the entrepreneurial spirit, risk taking spirit and developing the civil service mentality of the hospital employee” (p. 17).

Notwithstanding these somewhat antiorganizational views, useful insights have derived from an economic perspective on organizations, including principal-agent theory, which revolves around the relationship between a principal and an agent, as governed by a contract. Eisenhardt's (1989) definition is an excellent place to begin: “The focus of [agency] theory is on determining the most efficient contract governing (controlling) the principal-agent relationship given assumptions about people” (p. 58). These assumptions include self-interest, bounded rationality, and risk aversion, among others. There are two types of contracts at the broad conceptual level (Pontes, 1995): behavior-oriented contracts and outcome-oriented contracts. The former consists of contracts emphasizing salaries and hierarchical control; the latter consists of contracts containing language about commissions, stock options, property rights, and market control. What is of interest in the contract is the relationship established between the principal and the agent, who works on behalf of the former.

Generally the problem is how the principal (e.g., an owner or a shareholder) motivates the agent to act faithfully and reliably in the principal's interest. Because the two parties usually have different interests and the principal lacks sufficient information on the behavior of the agent, the theory predicts a lack of adequate and congruent action on the part of the agent. Also, the agent presumably possesses under this circumstance an incentive to misrepresent his or her work, shirk his or her duties, and otherwise engage in less-than-optimal behavior consistent with the interests of the principal. These problems are normally seen as falling into two categories: moral hazard and adverse selection. Moral hazard stems from the principal's imperfect information about agents and their actions. This serves as a temptation for agents to act more in their own self-interest than in those of the principal. Adverse selection is the phenomenon of agents exaggerating their qualifications, skills, and abilities in order to be hired at as much an advantage as possible.

There are various “solutions” to this problem, including everything from commissions to profit sharing to performance measurement and rewards. The central mechanism for the enforcement of solutions is the dyadic contract between principal and agent, and in many countries, contract law is the major institutional forum for writing and enforcing the content of contracts. In its most extreme form, an organization is seen as nothing more than a bolus of contracts written to define, enforce, and control acceptable and desirable agent behavior.

Although many different propositions may be deduced from agency theory, as Eisenhardt (1989) illustrates, several are key. For example, if an outcome is hard to measure and uncertain in its accomplishment, there will be a tendency to try behavior-based contracts, and vice versa. These kinds of contracts emphasize procedures and processes, and in health care, their application is becoming more and more common through best practices, clinical guidelines, and any system that emphasizes the preferred way of delivering a service. Since so many medical regimens and activities are high risk with unclear outcomes, the relationship between physician and patient or physician and insurer will emphasize that x, y, and z are done to the best of the physician's ability, knowing full well that the outcome cannot be predicted with much certainty.

Contrariwise, if one knows with some certainty that predicted outcomes will almost always be produced in a particular service, then an outcome-based contract will suffice. In health care, one can cite an optician's filling a prescription for eyeglass lenses (Dranove and White, 1987). The client can offer to pay a fee for grinding the proper lens or, if the glasses are not of the quality expected, the client can pay nothing. In this instance, there is an outcome-oriented contract. The client is not concerned with the process through which the glasses are produced; he or she only cares whether his or her vision is improved.

In sum, one can see why health care providers have been subjected to behavior-based contracts in so many instances. It is because the outcome of much medical care is uncertain. If medicine becomes more programmable, as the recent push toward evidence-based medicine and standardization would indicate, there will continue to be an emphasis on behavior-based contracts. But as improved technology renders modern medicine more certain in its outcomes, agency theory predicts that there will be an emphasis on outcomes-based contracts.

Transaction Cost Economics

Building on the premises of principal-agent theory, transaction cost economics (TCE) concerns itself less with the microlevel nature of contractual exchanges between individuals and more with the larger relationship of organizations in their respective markets. Given the belief that markets are the preferred locations for economic exchanges, with their inherent efficiency through the price mechanism as the most effective medium of what a buyer needs to know about a product or service (as noted in the discussion about agency theory), the basic question is why organizations exist in the first place. This may seem to be an odd question: Do not most organizations, at least in the for-profit sector, participate in exchanges in markets? Why, then, would anyone be concerned about exchanges involving organizations “replacing” market exchanges?

The issue is subtle. The idea is that in a perfect world of little uncertainty, highly reliable information about products and services, and little or no opportunistic behavior of individuals involved in market exchanges (an idealized and abstract state of affairs), all economic exchanges would involve trading among just individuals or very small groups of individuals. In health care, one might imagine a state of affairs where small solo or partnership physician practices dominate a market in which patients interact on a one-on-one basis with a given practice, moving from one to the next until a satisfactory exchange of medical care at the right price and the right quality is found. There would be no need for larger, complex organizations that internalize market exchanges into their structures with their inevitable bureaucratic attributes (e.g., formality, hierarchy, centralization). So although it is true that organizations participate in market exchanges, organizations also tend to usurp these exchanges through horizontal and vertical integration. But we still see markets with large bureaucratic organizations playing active roles in them. How can this be?

The answer lies in the observation that markets have varying levels of imperfection, so what makes markets imperfect? Different writers have different approaches to this question, but Oliver Williamson (1975), the acknowledged advocate of TCE, focuses on a matrix of factors that distort market exchanges such that organizations will arise to deal with the imperfections. The two essential problems confronting people making market exchanges are opportunism and bounded rationality. Like principal-agent theory, TCE assumes that people are essentially self-centered, opportunistic, and potentially deceitful. Simultaneously, bounded rationality—the inability of market actors to know all that they need to know about their exchange partners due to a lack of timely and relevant information, rapid change in market circumstances, and chance—limits people from making well-informed purchasing decisions. The idea of bounded rationality comes from Simon (1957), who demonstrated that it is virtually impossible to construct coherent decision trees that illuminate all possible rational paths of market behavior. These two factors are considered axioms in TCE, and together they yield three more specific problems that a market actor must confront: market uncertainty and complexity, small numbers bargaining, and information “impactedness.”

Uncertainty and complexity in the environment lead to efforts to create certainty and reduce complexity through internalization of whatever components of the market appear to be at the root of the confusion. The idea is that an organizational structure (Williamson uses the term hierarchy) allows decisions to be made in a coherent sequence, enacts processes of explicit control, and forces convergent goals among all players. Together these features are seen as ways to reduce the impact that uncertainty and complexity have on economic exchanges. Small numbers bargaining is simply the problem of a market actor having to confront a limited number of choices for a product or service, which places the buyer at a disadvantage to be exploited because of the lack of alternatives. The buyer can avoid this dilemma by producing the product or service. Finally, information impactedness, or information asymmetry, is the situation in which the actual circumstances of a potential exchange are not well known to a market actor.

The forces of market uncertainty and complexity, small numbers bargaining, and information impactedness produce a situation such that the costs of engaging in a market exchange are so high that internalization of the exchange inside an organizational structure will be less expensive—hence, the rise and importance of organizations. The putative advantages of intraorganizational exchanges outweigh the advantages of trading in markets. Put another way, overcoming these limits exacts such high transaction costs (the costs of seeking, making, monitoring, and enforcing contracts) that it is less expensive to incur the administrative costs of internally managing the provision or production of the given services or goods (Mick, 1990).

TCE applications abound in the general organization and management literature (David and Han, 2004) but are less common in health care. Empirical examples include studies of nurse staffing patterns and hospital efficiency (Bloom, Alexander, and Nuchols, 1997), of the outsourcing of electronic information systems in US hospitals (Diana, 2009), and of health plans' decisions either to use disease management programs or redesign care within physician practices in an effort to improve chronic illness management (Leeman and Mark, 2006). Conceptual examples of TCE applied to health care organizations include Fareed and Mick's (2011) proposal to predict whether hospitals will select in-house patient safety remedies or purchase these solutions from market vendors and Shay and Mick's (2013) discussion of whether post–acute care organizations will increasingly integrate vertically with hospitals as an effect of the passage of the Patient Protection and Affordable Care Act of 2010.

There are problems with the application of a strict TCE approach to health care organizations. First, the traditional depiction of TCE may be too narrow because its advocates are typically silent on the issue of transaction costs inside the organization. Mick and Conrad (1988) argue that these interior costs must be weighed against the costs of exterior market exchanges. Such interior costs include planning, implementing, directing, monitoring, evaluating, and enforcing the production of goods and services inside the organization. In a well-managed health care organization, these costs are likely to be relatively low as compared to those in a poorly managed health care organization. It is only through a comparative assessment of market and interior transaction costs that a decision about whether to “make or buy” a good or service can be made as rationally as possible, notwithstanding the difficulty inherent in measuring transaction costs, a major criticism of the perspective.

Second, there are concerns that TCE's depiction of human nature is too rigid (Ghoshal and Moran, 1996). A basic problem resides in the assumption that human behavior and dispositions are constant and are almost always opportunistic and deceptive. The possibility that human behavior is malleable and shaped by its context is not considered. The kind of managerial action that TCE commonly calls for tends to be by fiat, that is, highly directive, hierarchical, and formal. But this approach can undermine the establishment of trust as a prerequisite to motivate organizational actors and can end up by creating the kind of behavior that management wanted to neutralize in the first place. TCE advocates thus often ignore the importance of social control and informality in its approach. The more that management relies on fiat and rational controls, the more it must continue to do so because it reinforces any tendency toward opportunism that might exist and destroys positive socialization processes and the informal controls that are intertwined with them. And in a final irony, as rational control increases, the advantage of hierarchy is lost because administrative transaction costs will also increase, and the exchanges that were intended to take place inside the organization would be more efficiently produced in the market.

Despite these criticisms, as Stiles, Mick, and Wise (2001) argue, “the vocabulary and concepts of transaction costs . . . provide a completely appropriate depiction of the barriers to successful intra-organizational management” (pp. 86–87). Thus, an extended view of TCE actually offers a more practical approach than its original depiction. By relaxing some of the more rigid assumptions that some scholars have made, TCE has become a useful approach for analysis of such phenomena as the decision to integrate or deintegrate vertically in health care organizations.

Other Theories

Other theoretical postures that have gained currency in more recent times include social network theory, also known as social network analysis, and complexity theory, also referred to as complexity science, complex adaptive systems theory, or chaos theory. These perspectives are the subject of chapters 10 and 11, respectively.

In chapter 10, Timothy R. Huerta and Roberto Dandi explain the primary concepts behind social network theory, which portrays the actions and environments of organizations as directly shaped by the formal and informal, interpersonal and interorganizational relationships maintained within their networks (Scott and Davis, 2007). Chapter 11 offers James W. Begun and Marcus Thygeson's updated assessment of complexity science in health care organization theory. Complexity theory proposes that within complex, nonlinear, “living” systems such as health care organizations, unpredictable properties emerge as the result of dynamic interaction, and what initially appears to be chaotic produces creativity and order (Scott and Davis, 2007; Begun, Zimmerman, and Dooley, 2003). Both of these chapters present clear expositions of these theories and their unique approaches to understanding and explaining organizations. Therefore, rather than subject readers to redundancy and restate the issues surrounding these theories, we refer readers to these portions of the book.