CHAPTER TWO
UNDERSTANDING THE STUDY OF ORGANIZATIONS: A HISTORICAL REVIEW

Large, complex organizations have existed for many centuries, but within the past two centuries they have proliferated tremendously. This chapter reviews major developments in the research and theory about organizations and management. This book's analysis of public organizations begins with this chapter's review that illustrates the generic theme mentioned in the previous chapter. Major contributors to this field have usually treated organizations and management as generally similar in all contexts, with little distinction between the public and private sectors. This generic emphasis has great value, and this book draws upon it. Later chapters present evidence supporting the claim that public organizations are distinct. Leaders and analysts need to be aware of the historical developments summarized in this chapter. The review covers terms, ideas, and names that serve as part of the vocabulary of management. Leaders in organizations often refer to Theory X and Theory Y, span of control, and other concepts that the review covers.

Theoretical perspectives on how to organize and manage have strongly influenced, and have been influenced by, the way managers and organizations behave. Some of the general trends involve profoundly important beliefs about the nature of human motivation and of successful organizations. The review shows that management theory and practice have evolved over the past two centuries. Theories about the motives, values, and capacities of people in organizations have evolved, and this evolution has in turn prompted additional theories about how organizations must look and behave in response to the increasing complexity of the contexts in which they operate. Theories and practice have moved away from emphasis on organizations with strong chains of command, very specific job responsibilities, and strong controls over the people in them. While such topics remain important, the field has moved toward emphasis on more flexible, adaptive organizations; horizontal communications; and more participation, empowerment, and teamwork.

The Systems Metaphor

Figures 1.1 and 1.2 and the accompanying definition of organization in Chapter One reflect one major theme for the past century's major developments in the field: how the field has moved from early approaches (now considered “classical” views) that emphasized a single appropriate form of organization and management, toward more recent approaches that reject this “one best way” concept. Recent perspectives emphasize the variety of organizational forms that can be effective under the different conditions that organizations face. This trend in organizational analysis draws on general systems theory. This body of theory developed the idea that there are various types of systems in nature that have much in common. Analyzing these systems, according to systems theorists, can provide insights about diverse entities (Daft, 2020; Katz and Kahn, 1966, pp. 19–29).

A system is an ongoing process that transforms inputs into outputs. The outputs in turn influence subsequent inputs into the system in a way that supports the continuing operation of the process. One can think of an organization as a system that takes in various resources and transforms them in ways that lead to attaining additional supplies of resources (the definition in Chapter One includes this idea). Systems have subsystems, such as communications systems or production systems within organizations, and throughput processes, which are sets of internal linkages in the transformation process. The outputs of the system lead to feedback – that is, the influences that the outputs have on subsequent inputs. The systems theorists, then, deserve credit (or blame) for making terms such as input and feedback part of our everyday jargon. Management analysts have used systems concepts – usually elaborated far beyond the simple description given here – to examine management systems.

A major trend among organizational theorists in the past century has been to distinguish between closed systems and open or adaptive systems. Some systems are closed to their environment; the internal processes remain the same regardless of environmental changes. A thermostat is part of a closed system that transforms inputs, in the form of room temperature, into outputs, in the form of responses from heating or air-conditioning units. These outputs feed back into the system by changing the room temperature. The system's processes are stable and machinelike. They respond consistently in a programmed pattern. One can think of a human being as an open or adaptive system. Humans transform their behaviors to adapt to their environment when there are environmental changes for which the system is not programmed. Thus, the human being's internal processes are open to the environment and able to adapt to shifts in it. Some organization theorists found the systems approach helpful as a metaphor for describing how organization theory has evolved during the past century. These theorists say that the earliest, “classical” theories treated organizations and employees as if they were closed systems.

Classical Approaches to Understanding Organizations

The earliest classical theories, and the advice they gave to managers, emphasized stable, clearly defined structures and processes, as if organizational goals were clear and managers' main challenge was to design the most efficient, machine-like procedures to maximize attainment of the organization's goals. Some organization theorists have also characterized this as the “one best way” approach to organization.

Frederick Taylor and Scientific Management

Frederick Taylor (1919) is usually cited as one of the pioneers of managerial analysis and as the major figure in the scientific management school. In Taylor's own words scientific management involved the systematic analysis of “every little act” in tasks to be performed by workers. Taylor asserted that the role of management was to gather detailed information on work processes, analyze it, and derive rules and guidelines for the most efficient way to perform the required tasks. Workers were then to be selected and trained in these procedures so they could maximize their output and their own earnings.

Procedures similar to those that Taylor and others developed for analyzing and designing tasks are still in use today. They conducted time–motion studies, which involved the detailed measurement and analysis of physical characteristics of the workplace, such as the placement of tools and machinery in relation to the worker and the movements and time that the worker had to devote to using them. The objective was to achieve the most efficient physical layout for the performance of a specified task. Analytical procedures of this sort are still used in government and industry.

Taylor's determination to find the “one best way” to perform a task was such that he even devoted himself to finding the best way to design golf greens and golf clubs. He designed a putter that the golfer could stabilize by cradling the club in his or her elbows. The putter proved so accurate that the US Golf Association banned it (Hansen, 1999).

Taylor's emphasis on the efficient programming of tasks and of workers' activities provoked controversy. Critics attacked his work for its apparent inhumanity and its underestimation of psychological and social influences on worker morale and productivity. Taylor contended that his methods would benefit workers by helping them to increase their earnings and the quality of their work. In his own accounts of his work he said that he became interested in ways of encouraging workers without supervisors' having to place pressure on them. As a manager, he had been involved in a very unpleasant dispute with workers, which he attributed to the obligation to put them under pressure (Burrell and Morgan, 1980, p. 126). He wanted to find alternatives to such situations.

Yet, Taylor did emphasize pay as the primary reward for work. He stressed specialization of worker activities, as if the worker were a rather mindless component of a mechanistic process. He did not improve his image with later organizational analysts when he illustrated his techniques with a description of his efforts to train a Scandinavian worker, whom he described as “dumb as an ox,” in the most efficient procedures for shoveling pig iron. While contemporary organizations still use variations of scientific management in valuable ways, as a guiding conception of analyzing organizations it too often oversimplified the complexity of human needs and motives in the workplace.

Max Weber: Bureaucracy as an Ideal Construct

In the early decades of the twentieth century, Max Weber's writings became influential, in a related but distinct way. Organization theorists often treat Weber as one of the founders of organizational sociology – the analysis of complex organizations. His investigations of bureaucracy as a social phenomenon provided the most influential early analysis of the topic (Gerth and Mills, 1946).

The proliferation of organizations with authority formally distributed among bureaus or subunits is a recent development in human history. Weber undertook to specify the defining characteristics of the bureaucratic form of organization, which he saw as a relatively new and desirable form in society. He saw the spread of such organizations as part of a movement toward more legal and rational forms of authority and away from authority based on tradition (such as inherited monarchical power) or charisma (such as that possessed by a ruler like Napoleon). In traditional feudal or aristocratic systems, Weber said, people's functions were assigned by personal trustees or appointees of the ruler. Further, their offices were more like avocations than modern-day jobs; authority was discharged as a matter of privilege and the bestowing of a favor.

The bureaucratic form was distinct in its legalistic specification of the authorities and obligations of office. Weber wrote that the fully developed version of bureaucracy had the following characteristics:

  1. Fixed, official jurisdictional areas are established by means of rules. The rules distribute the regular activities required by the organization among these fixed positions or offices, prescribing official duties for each. The rules distribute the authority to discharge the duties, and they also establish specified qualifications required for each office.
  2. There is a hierarchy of authority, involving supervision of lower offices by higher ones.
  3. Administrative positions in the bureaucracy usually require expert training and the full working capacity of the official.
  4. Management of subunits follows relatively stable and exhaustive rules, and knowledge of these rules is the special expertise of the official.
  5. The management position serves as a full-time vocation, or career, for the official.

Weber regarded this bureaucratic form of organization as having technical advantages compared with administrative systems in which the officials regarded their service as an avocation, often gained by birthright or through the favor of a ruler, to be discharged at the official's personal discretion. In Weber's view, qualified career officials, a structured hierarchy, and clear, rule-based specifications of duties made for precision, speed, clarity, and consistency. In addition, the strict delimiting of the authority of career officials and the specification of organizational duty meant that duties would be performed consistently, and clients would be treated without favoritism. With officials placed in positions on the basis of merit rather than birthright or political favoritism, constrained by rules defining their duties, and serving as career experts, bureaucracies represented the most efficient organizational form yet developed, from Weber's perspective.

Weber did express concern that bureaucratic routines could oppress individual freedom (Fry, 1989) and that problems could arise from placing bureaucratic experts in control of major societal functions. Nevertheless, he described bureaucracy as a desirable form of organization, especially for efficiency and the fair treatment of clients and employees. He thus emphasized a model of organization involving consistent rules and hierarchy of authority. For this reason, Weber is often grouped with the other classic figures as a proponent of what later analysts would characterize as the closed-system view of organizations.

The Administrative Management School: Principles of Administration

Analysts also began to develop the first management theories that encompassed a broad range of administrative functions and the proper means of discharging those functions. They sought to develop principles of administration to guide managers in such functions as planning, organizing, supervising, controlling, and delegating authority. This group became known as the administrative management school (March and Simon, 1958).

The proponents of this school of thought advanced principles that focused on providing effective organization. The flavor of their work and their principles are illustrated in prominent papers by two of the leading figures in this group, Luther Gulick and James Mooney. In “Notes on the Theory of Organization,” Gulick (1937) discussed two fundamental functions of management: the division of work and the coordination of work. Concerning the division of work, he discussed the need to create clearly defined specializations. Specialization, he said, allows the matching of skills to tasks and the clear, consistent assignment of tasks.

Once tasks have been divided, coordinating the work becomes imperative. On this matter, Gulick proposed principles that were clearer than his general points about specialization. Coordination should be guided by several principles. First is the span of control – the number of subordinates reporting to one supervisor. The span of control should be kept narrow, limited to between six and ten subordinates per supervisor. The supervisor's attention should not be divided among too many subordinates. The principle of one master stated that each subordinate should have only one superior. There should be no confusion about who the supervisor is. A third principle, technical efficiency through the principle of homogeneity, required that tasks must be grouped into units on the basis of their homogeneity. Dissimilar tasks should not be grouped together.

In the same paper, Gulick sought to define the job of management through what became one of the most influential acronyms in general management and public administration: POSDCORB. The letters stand for planning, organizing, staffing, directing, coordinating, reporting, and budgeting. These are the functions, he said, for which principles needed to be developed in subsequent work.

In “The Scalar Principle,” Mooney (1930) presented a generally similar picture of the effort to develop principles. He said that an organization must be like a scale, a graded series of steps, in levels of authority and corresponding responsibilities. The principle involved several component principles, including leadership. Under this principle, Mooney said, a “supreme coordinating authority” at the top must project itself through the “scalar chain” to coordinate the entire structure. The scalar chain relied on the principle of delegation, under which higher levels assign responsibility to lower levels. These processes accomplished the third principle of functional definition, under which each person is assigned a specific task.

These two examples illustrate the nature of the administrative management school. These two authors emphasize formal structure in the organization and the hierarchical authority of administrators. Although some of the principles are only vaguely discussed, others are quite clear. Tasks should be highly specialized. Lines of hierarchical authority must be very clear, with clear delegation down from the top. Span of control should be narrow. A subordinate should be directly accountable to one superior. Like Weber and Taylor, these authors tended to emphasize consistency, rationality, and machine-like efficiency. Critics would later contend that they wrote about organizations as if they could operate most effectively as closed systems, designed according to one proper form of organization.

The historical contribution of this group is undeniable; the tables of contents of many contemporary management texts reflect the influence of these theorists' early efforts to conceive the role of management and administration. In some highly successful corporations, top executives have made this literature required reading for subordinates (Perrow, 1970b).

Gulick identified very strongly with public administration. He and other members of the administrative management school played important roles in the work of various committees and commissions on reorganizing the federal government, such as the Brownlow Committee in 1937 and the Hoover Commission in 1947. The reforms these groups proposed reflected the views of the administrative management school. They aimed at such objectives as grouping federal agencies according to similar functions, strengthening the hierarchical authority of the chief executive, and narrowing the executive's span of control.

The immediate influence of these proposals on the structure of the federal government was complicated by political conflicts between the president and Congress (Arnold, 1995). They had a strong influence, however, especially on the development of an orthodox view of how administrative management should be designed in government. Some scholars have argued that structural developments in public agencies and the attitudes of government officials about such issues still reflect an orthodox administrative management school perspective (Knott and Miller, 1987; Warwick, 1975, pp. 69–71). The influence of the administrative management school on these reform efforts can be considered the most significant direct influence on practical events in government that organization theorists have ever had. Nevertheless, critics later attacked the views of the administrative management theorists as too limited for organizational analysis. As described later, researchers began to find that many successful contemporary organizations violate the school's principles drastically and enthusiastically.

The administrative management theorists' work was related to a broad progressive reform movement (Knott and Miller, 1987). Those reformers sought to eradicate corruption in government, especially on the part of urban political machines and their leaders. They sought to institute more professional forms of administration through such means as establishing the role of the city manager. In addition, the growth of government over the earlier part of the century had led to disorganization among the agencies and programs of government; there was a need for better organization. In this context, the administrative management theorists' emphasis on basic organizational principles appears not only well justified but absolutely necessary.

It is also important to acknowledge that these early theorists did not advance their ideas as simplemindedly as some later critics depict it. Although Luther Gulick came to be characterized in many organization theory texts as one of the foremost proponents of highly bureaucratized organizations, he wrote a reflection on administrative issues from World War II in which he drew conclusions about the efficiency of democracy. He argued that the democratic system of the United States actually gave it advantages over the more authoritarian axis powers. The more democratic process required more participation and cooperation in problem solving and thus led to better planning and implementation of plans than in the authoritarian regimes (Van Riper, 1998). Gulick thus suggested that more democratic processes may look less efficient than more authoritarian ones, even though they can produce more efficient and effective results.

Another very original thinker, Mary Parker Follett, wrote approvingly of administrative principles, and scholars sometimes classify her as a member of this school. She wrote, however, a classic essay on “the giving of orders” (Follett, [1926] 1989) that had very original and forward-looking implications. In the essay, she proposed a cooperative, participative process for giving orders, in which superiors and subordinates develop a shared understanding of the particular situation and what it requires. They then follow the “law” of the situation rather than having a superior impose an order on a subordinate. Follett's perspective both foreshadowed later movements and influenced them in the direction of participatory management that received more and more emphasis among organizational analysts and in organizational practice. It also foreshadowed contemporary developments in feminist organization theory (Guy, 1995; Harquail, 2020; Hult, 1995; Morton and Lindquist, 1997).

Still, these contributions concentrated on a limited portion of the framework for organizational analysis given in Figures 1.1 and 1.2. They emphasized the middle and lower parts of the framework, particularly organizational structure. They paid attention to tasks and to incentives and motivation, but they were limited in comparison with the work of later authors. Additional developments rapidly began to expand the analysis of organizations, with increasing attention to the other components in Figures 1.1 and 1.2.

Reactions, Critiques, and Subsequent Developments in Analysis of Organizations and the People in Them

Developments in the emerging field of industrial psychology led to dramatic changes in the way organizational and managerial analysts viewed organizations and the people in them. Researchers studying behavior in industry began to develop more insight into psychological factors in work settings. They analyzed the relationships among such factors as fatigue, monotony, and worker productivity. They studied working conditions, analyzing variables such as rest periods, hours of work, methods of payment, routineness of work, and the influence of social groups in the workplace (Burrell and Morgan, 1980, p. 129).

The Hawthorne Studies: Elaborating the Nature of People in the Workplace

A series of experiments beginning in the mid-1920s at the Hawthorne plant of the Western Electric Company provided a more subtle view of the psychology of the workplace than previous analysts had produced. The Hawthorne studies involved experiments and academic and popular reports over a number of years. Controversies ensued over the interpretation of these studies (Burrell and Morgan, 1980, pp. 120–143); however, organizational analysts consider them pathbreaking illustrations of the influence of social and psychological factors on work behavior. An employee's work-group experiences, a sense of the importance of the work, and concern on the part of supervisors are among the important social and psychological influences on workers.

The leaders of the project identified several experiments as particularly significant (Roethlisberger and Dickson, 1939). In one experiment, the researchers lowered the level of illumination in the workplace and found that productivity nevertheless increased, because the workers responded to the attention that the researchers paid to them and their work. In another study, they improved the working conditions in a small unit through numerous alterations in rest periods and working hours. Increases in output were at first taken as evidence that the changes were influencing productivity. When the researchers tested that conclusion by withdrawing the improved conditions, however, they found that, rather than falling off, output remained high. In the course of the experiment, the researchers had consulted the workers about their opinions and reactions and displayed concern for their physical well-being. Their experiment on the physical conditions of the workplace had actually altered the social situation in the workplace, and that appeared to account for the continued high output.

In observing another work group, the researchers found that it enforced strict norms regarding group members' productivity. To be a socially accepted member of the group, a worker had to avoid being a “rate buster,” who turns out too much work; a “chiseler,” who turns out too little; or a “squealer,” who says something to a supervisor that could be detrimental to another worker. This suggested a distinction between the formal organization, as it is officially presented in organization charts and rules, and the informal organization. The informal organization develops through unofficial social processes within the organization, but can involve norms and standards as influential on the worker as formal requirements.

The Hawthorne studies provided a significant demonstration of the importance of social and psychological factors in the workplace up to that time, and they contributed to a major shift in research on management and organizations. The emphasis on social influences, informal processes, and the motivating power of attention from others and a sense of significance for one's work constituted a major counterpoint against the principles of administrative management and scientific management.

Chester Barnard and Herbert Simon: The Inducements–Contributions Equilibrium, The Limits of Rationality, and Behavioral Theory of Organizations

Chester Barnard, a successful business executive turned organization theorist, and Herbert Simon, an academic who would become a Nobel laureate, provided major contributions that moved research in new directions. These contributions added substantially to the attention that organization theorists paid to organizational processes (especially decision making), people, environments, leadership, and goals and values.

Encouraged by the members of the Harvard group who were responsible for the Hawthorne studies and related work (Burrell and Morgan, 1980, p. 148), Chester Barnard wrote The Functions of the Executive (1938). It became one of the most influential books in the history of the field. Barnard's definition of an organization – “a system of consciously coordinated activities or forces of two or more persons” (1938, p. 73) – illustrates the sharp difference between his perspective and that of the classical theorists. Rather than emphasizing the formal authority of executive leaders, Barnard focused on how leaders induce the cooperative activities fundamental to an organization. He characterized an organization as an “economy of incentives,” in which individuals contribute their participation and effort in exchange for incentives that the organization provides. The executive must keep the economy in equilibrium by providing incentives to induce contributions from members; those contributions in turn earn the resources for continuing incentives. (Notice that the definition of organization in Chapter One speaks of leaders and organizations seeking to gain resources from the environment to translate into incentives. This reflects the influence of Barnard's perspective.) Barnard offered a rich typology of incentives, including not just money but also power, prestige, fulfillment of ideals and altruistic motives, participation in valuable work, and many others. (Table 10.2 in Chapter Ten describes the incentives that he named.)

Barnard also saw the economy of incentives as interrelated with other key functions of the executive, including communication and persuasion. The executive must use communication and persuasion to influence workers' valuations of incentives. The executive can, e.g., emphasize major organizational values. The persuasion process thus requires an effective communication process. He discussed the communication and persuasion processes as essential to the “informal organization” which fosters the cooperative activity essential for effective organization. Some authors now cite Barnard's ideas on these topics as an early recognition of the importance of organizational culture, a topic that has received extensive attention in organizational analysis and practice in recent decades (and which is covered in more depth in Chapter 11).

Barnard's divergence from the classical approaches is obvious. Rather than stating prescriptive principles, he sought to describe the empirical reality of organizations. He treated the role of the executive as central, but he placed less emphasis on formal authority and formal organizational structures. Barnard's analysis drew interest from many researchers, including one preeminent social scientist, Herbert Simon. Simon attacked the administrative management school much more directly than Barnard had. In an article titled “The Proverbs of Administration” in Public Administration Review (1946), he criticized the administrative management school's principles of administration as vague and contradictory. He compared them to proverbs because he saw them as prescriptive platitudes. The principle of specialization, e.g., never specified whether one should specialize by function, clientele, or place. The principle of span of control conflicts with the principle of unity of command. In a large organization, narrow spans of control require small work units, with a supervisor for each. Then there must be many supervisors above those supervisors to keep the span of control narrow at that level, and so on up. This makes communication up, down, and across the organization very cumbersome, and it makes it difficult to maintain clear, direct hierarchical lines of authority.

Simon called for a more systematic examination of administrative processes to develop concepts and study their relationships. Researchers, he said, should determine when individuals in administrative settings should choose one or the other of the alternatives represented by the principles. As indicated by his critique, such choices are seldom clear. Such limits on the ability of organizational members to be completely rational are major determinants of organizational processes and their effects. Simon argued that these limits on rationality must be more carefully analyzed through a more empirical approach, with decision making as the primary focus.

Simon's critique of Gulick and others in the administrative management school underestimated strengths of that approach (Hammond 1990). The administrative management school did address challenges that managers constantly face, such as decisions about the appropriate span of control, that have a continuing influence on organizational structures in government. Still, most organization theorists agree that Simon's rejection of the school's principles had the stronger influence on subsequent work in the field and changed its direction.

Simon pursued his ideas further in Administrative Behavior (1948). As the title indicates, he emphasized analysis of actual behavior rather than prescribing principles. He drew on Barnard's idea of an equilibrium of inducements and contributions and extended it into a more elaborate discussion of an organization's need to provide sufficient inducements to members, external constituencies, and supporters. (The definition and framework in Chapter One also reflect the influence of Simon's perspective.)

Like Barnard, Simon was concerned with the process of inducement and persuasion and with abstract incentives such as prestige, power, and altruistic service in addition to material incentives. He emphasized the uncertainties that administrative decision makers face. Faced with uncertainty and complexity, how do administrators make choices and decisions? The classical principles of administration were based on the assumption that administrators could be rational in their choice of the most efficient mode of organization. Much of economic theory assumed that firms and individuals are strictly rational in maximizing profits and personal gain. Simon observed that in administrative settings, there are usually uncertainties that impose cognitive limits on rationality. Strictly rational decisions are impossible in complex situations, because information and time for making decisions are limited, and human cognitive capacity is too limited to process all the information and to consider all of the alternatives. Whereas most economic theory assumed maximizing behavior in decision making, Simon coined a new concept. Rather than maximize, administrators “satisfice.” Satisficing involves choosing the best of a limited set of alternatives so as to optimize the decision within the constraints of limited information and time. Administrators often cannot make maximally rational decisions. The administrator makes the best possible decision within the constraints imposed by the available time, resources, and cognitive capacity.

This conception of the decision-making process challenged a fundamental tenet of economic theory. It influenced subsequent research on decision making in business firms, as amplified by A Behavioral Theory of the Firm by Richard Cyert and James March (1963). It provided a major step toward more recent approaches to organizational decision making. Chapters Six and Seven describe Cyert and March's conclusions and why prominent organizational analysts consider this book one of the most influential in the history of the field. With James March, Simon published another influential book, Organizations (March and Simon, 1958), in which they elaborated the theory of an equilibrium between inducements and worker contributions. They presented an extensive set of propositions about factors influencing a person's decision to join and stay with an organization and, once in it, to produce. In these and other works, Simon's insights about decision making in administrative settings appears to be one of the main reasons that he was later awarded the Nobel Prize in economics.

Social Psychology, Group Dynamics, and Human Relationships

In the 1930s Kurt Lewin, a psychologist, arrived in the United States as a refugee from Nazism, to continue his very influential research. He developed field theory and topological psychology, which sought to explain human actions as functions of both the characteristics of the individual and forces impinging on the individual at a given time. These ideas differed from other prominent approaches of the time, such as Freudian psychology, which emphasized unconscious motives and past experiences.

Lewin's emphasis on the field of forces influencing an individual's actions drew on his interest in group behaviors and change processes in groups (Back, 1972, p. 98). He studied power, communication, influence, and “cohesion” within groups, and he developed a conception of change that analysts of group and organizational change still find valuable.

Lewin argued that groups and individuals maintain a “quasi-stationary equilibrium” in their attitudes and behaviors. This equilibrium results from a balance between forces pressing for change and those pressing against change. To change people, you must change these forces. Groups exert pressures and influences on the individuals within them. One must alter the total field of group pressures, through a three-phase process. The first phase is “unfreezing,” or weakening the forces against change and strengthening the forces for change. Next, the “changing” phase moves the group to a new equilibrium point. Finally, the “refreezing” phase firmly sets the new equilibrium through such processes as expressions of group consensus.

One of Lewin's experiments in group dynamics illustrates his meaning. Lewin conducted research on social processes such as race relations and group leadership. During World War II, Lewin sought to aid the war effort by conducting research on methods of encouraging consumption of underutilized foods as a way of conserving resources. He conducted an experiment about getting housewives to decide that they should use more beef hearts in preparing meals. He assembled the housewives in groups and presented them with information favoring the change. They then discussed the matter, aired and resolved their concerns about the change (“unfreezing”), and came to a consensus that they should use more beef hearts (change). In groups in which the housewives made a public commitment to do so, more of them adopted the new behavior than in groups in which the members made no such public commitment. The group commitment and its effect provide an example of “refreezing,” or setting group forces at a new equilibrium point.

As the intellectual leader of a group of social scientists interested in research on group processes, Lewin was instrumental in establishing the Research Center for Group Dynamics at MIT and the first National Training Laboratory, which served for years as a leading center for training in group processes. These activities produced an interesting set of diverse, sometimes opposing influences on later work in the field.

Lewin's efforts were among the first to apply experimental methods (such as using control groups) to the analysis of human behavior. The work of Lewin and his colleagues set in motion the development of experimental social psychology, which led to extensive experimentation on group processes. In an influential experiment conducted by members of this group, Lester Coch and John R. P. French (1948) compared different factory work groups faced with a change in their work procedures. One group participated fully in the decision to make the change, another group had limited participation, and a third group was simply instructed to make the change. The participative groups made the change more readily and more effectively, with the most participative group doing the best. These findings contributed to making participative decision making (PDM) a widely utilized technique in management theory and practice. Numerous experiments of this sort contributed to the growing literature on industrial psychology and organizational behavior.

Interestingly, Lewin's influence also led to an opposing trend in applied group dynamics. The National Training Laboratory conducted training in group processes for governmental and industrial organizations. After Lewin's death, the group dynamics movement split into two movements. In addition to the researchers who emphasized rigorous experimental research on group concepts, a large group continued to emphasize industrial applications and training in group processes. This work contributed to the development of the field of organization development (described in Chapter Thirteen). It also led to the widespread use of T-groups, sensitivity sessions, and encounter-group techniques during the 1960s and 1970s (Back, 1972, p. 99). The work of Lewin and his colleagues (Lewin, 1947; 1948; Lewin and Lippitt, 1938; Lewin, Lippitt, and White, 1939) substantially influenced analysts' conceptions of the components of Figures 1.1 and 1.2, especially those concerned with change, decision making, and groups.

The Human Relations School

The Hawthorne experiments and related work and the research on group dynamics were producing insights about social and psychological factors in the workplace. They emphasized the value of participative management, enhancing employee self-esteem, and improving human relations in organizations.

The psychologist Abraham Maslow developed a theory of human needs that became one of the most influential theories ever developed by a social scientist. Maslow argued that human needs fall into a set of major categories, arranged in a “hierarchy of prepotency.” The needs in the lowest category dominate a person's motives until they are sufficiently fulfilled, then those in the next-highest category dominate, and so on. The categories, in order of prepotency, were physiological needs, safety needs, love needs, self-esteem needs, and self-actualization needs. The self-actualization category referred to the need for self-fulfillment, for reaching one's potential and becoming all that one is capable of becoming. Thus, once a person fulfills his or her basic physiological needs, such as the need for food, and then fulfills the needs at the higher levels on the hierarchy, he or she ultimately becomes concerned with self-actualization. This idea of making a distinction between lower and higher-order needs was particularly attractive to writers emphasizing human relations in organizations (for more detail on Maslow's formulation, see Chapters Nine and Ten).

Douglas McGregor, e.g., published a book whose title foretells its message: The Human Side of Enterprise (1960). McGregor had been instrumental in bringing Kurt Lewin to MIT, and the influence of both Lewin and Maslow was apparent in his conceptions of “Theory X” and “Theory Y.” He argued that management practices in American industry were dominated by a view of human behavior that he labeled Theory X. This theory held that employees were basically lazy, passive, and resistant to change and responsibility. Hence management must direct, control, and motivate employees. McGregor felt that Theory X guided organizational practices in most industrial organizations and was at the heart of classic approaches to management, such as scientific management.

Theory Y involved a diametrically different view of employees. Drawing on Maslow's conception of higher-level needs for self-esteem and self-actualization, McGregor defined Theory Y as the view that employees are capable of self-direction and self-motivation. Underutilized though this theory was at the time, management based on this approach would be more effective, because individual self-discipline is a more effective form of control than authoritarian supervision. McGregor advocated management approaches that would allow more worker participation and self-control, such as decentralization of authority, management by objectives, and job enlargement.

Theory Y rejected the classical approach to organization. Other major authors of the time placed a similar value on releasing human potential in the workplace. Argyris (1957), e.g., argued that there were inherent conflicts between the needs of the mature human personality and the needs of organizations. When management applies the classical principles of administration, healthy individuals will experience frustration and conflict. Healthy individuals desire independence, activeness, and use of their abilities. These motives clash with the classical principles, such as those that call for hierarchical authority and narrow specialization. These principles foster dependence on superiors and organizational rules, promote passiveness due to reduced individual discretion, and limit workers' opportunities to use their abilities. Argyris, too, called for further development of such techniques as participative leadership and job enlargement to counter this problem.

Like the classical theorists before them, the proponents of human relations theories in turn became the targets of criticism. Critics complained that they concentrated too narrowly on one dimension of organizations – the human dimension – with too little attention to other dimensions, such as organizational structure and environmental pressures. They argued that the human relations theorists were repeating the mistake of proposing one best way of approaching organizational analysis, because they always treated interpersonal and psychological factors as the crucial issues. Critics also complained that the human relations approach lacked empirical support due to weak evidence that improved human relations would lead to improved organizational performance (Perrow, 1970b). Research in the 1950s and 1960s produced evidence of considerable conflict in some very successful organizations. Research also produced little evidence of a strong relationship between individual job satisfaction and productivity (see Chapter Ten).

Like the criticisms of the classical approaches, these criticisms often overlooked the historical perspective of the writers, underestimating the significance of what they were trying to do at the time. The insights that these human relations theorists provided remain valuable – and dangerous to ignore. Examples still abound of management practices that cause damage because of inattention to the factors emphasized by the human relations theorists. E.g., when improperly implemented, scientific management techniques have created situations in which workers slow down or disguise their normal behaviors when management analysts try to observe them.

Open-Systems Approaches and Contingency Theory

Concerns over the limitation of the earlier approaches intensified as researchers found increasing evidence that organizations successfully adopt different forms under different circumstances. Organizational analysts became convinced that different forms of organization can be effective under certain contingencies of tasks and technology, organizational size, and operating environment. The analysis of these contingencies and their influence on organizations made “contingency theory” the dominant approach in organizational analysis in the 1960s and 1970s and organizational analysts still find it useful (Daft, 2020, pp. 28–34).

Around the middle of the twentieth century, researchers associated with the Tavistock Institute in Great Britain analyzed “sociotechnical systems,” emphasizing the relationships among technical factors and social dimensions in the workplace (Burrell and Morgan, 1980, pp. 146–147). E.g., Trist and Bamforth (1951) published an analysis of a change in work processes in a coal-mining operation that found that the technical changes in the work process changed the social relationships within the work group. They depicted the organization as a system with interdependent social and technical subsystems. In response to disturbances, the system moves to a new point of equilibrium – a new pattern of interrelated social and technical processes. Additional studies by the Tavistock researchers further developed this view that organizations are systems with group and individual characteristics, task requirements, and interrelations among them that must be properly accommodated in the design of the organization.

Tavistock researchers also devoted attention to the external environments of organizations. In a widely influential article titled “The Causal Texture of Organizational Environments,” Emery and Trist (1965) noted the increasing flux and uncertainty in the political, social, economic, and technological settings in which organizations operate. They discussed the influence on the internal operations of organizations of the degree of “turbulence” in their environment. Thus the emphasis moved toward analysis of organizations as open systems facing the need to adapt to environmental variations. (The case of “The Funeral in the Public Service Center” available in the Instructor's Guide describes a situation in a government agency in which a manager made a change in work group assignments. He intended for the change to improve technical operations and efficiency. The change disrupted social relations in the group affected, however, and invoked sharp resistance to the change. The manager had not taken into account the seemingly simple insight of sociotechnical systems theory: the important relationship between technical and social factors in organizations.)

In the United States, the most explicit systems approach to organizational analysis appeared in a very prominent text by Daniel Katz and Robert L. Kahn (1966), The Social Psychology of Organizations. They applied the systems language of inputs, throughputs, outputs, and feedback to organizations. In analyzing throughput processes, e.g., they differentiated various major subsystems, including maintenance subsystems, adaptive subsystems, and managerial subsystems. Scholars regard Katz and Kahn's effort as a classic in the organizational literature (Burrell and Morgan, 1980, p. 158), but it also provides an example of the very general nature of the systems approach. Because of its very general concepts, organizational researchers increasingly treated systems theory as a broad framework for organizing information (Kast and Rosenzweig, 1973, p. 16), but not as a clearly articulated theory. The metaphor of organizations as open, adaptive systems remained powerful, however, as an expression of the view of organizations as social entities that adapt to a variety of influences and imperatives.

Besides the efforts to apply systems concepts to organizations, research results indicated that organizations adopt different forms in response to contingencies. (Chapter Eight provides further discussion of contingency theories.) In England, Joan Woodward (1965) conducted a pathbreaking study of British industrial firms. She found that the firms fell into three categories on the basis of the production process or “technology” they employed: small-batch or unit production systems were used by such organizations as shipbuilding and aircraft manufacturing firms; large-batch or mass-production systems were operated by typical mass manufacturing firms; and continuous production systems were used by petroleum refiners and chemical producers. Most importantly, she concluded that the successful firms within each category showed similar management-structure profiles, but those profiles differed among the three categories. The successful firms within a category were similar on such dimensions as the number of managerial levels and the spans of control, yet they differed on these measures from the successful firms in the other two categories. This indicated that the firms within a category had achieved a successful fit between their structure and the requirements of the particular production process or technology with which they had to deal. The firms appeared to be effectively adapting structure to technology.

Burns and Stalker (1961) in The Management of Innovation, further contributed to the conclusion that effective organizations adapt their structures to contingencies. Burns and Stalker analyzed firms in the electronics industry in Great Britain. The industry was undergoing rapid change, with new products being developed, markets for the products shifting, and new technology becoming available. The firms faced considerable flux and uncertainty in their operating environments. Burns and Stalker classified the firms into two categories on the basis of their managerial structures and practices: organic and mechanistic organizations. Their descriptions of the characteristics of these two groups depict mechanistic organizations as bureaucratic organizations designed along the lines of the classical approaches. The name of the category also has obvious implications: these were organizations designed to operate in machine-like fashion. Burns and Stalker concluded that the organic type, named to underscore an analogy with living, flexible organisms, performed more successfully in the rapidly changing electronics industry. In these organizations there was less emphasis on communicating up and down the chain of command, on the superior controlling subordinates' behavior, and on strict adherence to job descriptions. There was more emphasis on networking and lateral communication, on the supervisor as facilitator, and on flexible and changing work assignments. Such organizations adapted and innovated more effectively under changing and uncertain conditions because they had more flexible structures, communication, supervision, and role definition. The mechanistic form can be more successful under stable environmental and technological conditions, however, where its emphasis on consistency and specificity makes it more efficient than a more loosely structured organization. Thus, Burns and Stalker also emphasized the need for a proper adaptation of the organization to contingencies.

Another important research project heavily emphasized organizations' environment as a determinant of effective structure. Paul Lawrence and Jay Lorsch (1967) studied US firms in three separate industries that confronted varying degrees of uncertainty, complexity, and change. The researchers concluded that the firms that were successfully operating in uncertain, complex, changing environments had more highly differentiated internal structures. By differentiated structures, they meant that the subunits differed a great deal among themselves, in their goals, time frames, and internal work climates. Yet these highly differentiated firms also had elaborate structures and procedures for integrating the diverse units in the organization. The integrating structures included task forces, liaison officers, and committees. Successful firms in more stable, certain environments, on the other hand, showed less differentiation and integration. Lawrence and Lorsch concluded that successful firms must have internal structures as complex as the environments in which they operate.

Other researchers continued to develop the general contingency perspective and to analyze specific contingencies. Perrow (1973) published an important analysis of organizational technology. He proposed two basic dimensions for the concepts of technology: the predictability of the task (the number of exceptions and variations encountered) and the analyzability of the problems encountered (the degree to which, when one encounters a new problem or exception, one can follow a clear program for solving it). Routine tasks are more predictable (there are fewer exceptions or variations) and more analyzable (exceptions or variations can be resolved through an established program or procedures). Organizations with routine tasks have more formal, centralized structures. They use more rules, formal procedures, and plans. Organizations with nonroutine tasks, where tasks have more exceptions and are harder to predict and where exceptions are harder to analyze and resolve, must have more flexible structures. They use more formal and informal meetings than rules and plans. (Chapter Eight describes a study confirming these relationships in public organizations.)

During this period, James Thompson (1967) published Organizations in Action, a very influential book that further developed the contingency perspective. Drawing on Herbert Simon's ideas about bounded rationality and satisficing, Thompson depicted organizations as reflecting their members' striving for rationality and consistency in the face of pressures against those orientations. He advanced propositions about how organizations use hierarchy, structure, units designed to buffer the environment, and other arrangements to try to “isolate the technical core” – that is, to create stable conditions for the units doing the basic work of the organization. Thompson suggested that organizations will try to group subunits on the basis of their technological interdependence – that is, their needs for information and exchange with each other in the work process (see Chapter Eight). Organizations, he proposed, will also adapt their structures to their environment. Where environments are shifting and unstable, organizations will adopt decentralized structures, with few formal rules and procedures, to provide flexibility for adapting to the environment (Chapter Four provides further description). One of Thompson's important achievements was to provide a driving logic for contingency and open-systems perspectives by drawing on Simon's ideas. Organizations respond to complexity and uncertainty in their technologies and their environments by adopting more complex and flexible structures. They do so because the greater demands for information processing strain the bounded rationality of managers and the information processing capacity of more formal bureaucratic structures. Clear chains of command, vertical communication, and strict rules and task assignments can be too slow and inflexible in adapting to complex information and rapid change.

Through the 1960s and 1970s, an upsurge in empirical research on organizations extended and tested the open-systems and contingency theory approaches. Many of these studies took place in public and nonprofit organizations. Peter Blau and his colleagues (Blau and Schoenherr, 1971) reported studies of government agencies that showed relationships between organizational size and structure. These and other studies added size to the standard set of contingencies. Hage and Aiken (1969) reported on a series of studies of social welfare agencies that provided evidence that routineness of tasks, joint programs among organizations, and other factors were related to organizational structure and change. In England, a team of researchers (Pugh, Hickson, and Hinings, 1969) conducted what became known as the Aston studies – a major effort at empirical measurement of organizations – and developed an empirical taxonomy, grouping organizations into types based on the measured characteristics. They interpreted differences in their taxonomic categories as the results of differences in age, size, technology, and external auspices and control. (Chapter Eight discusses important implications of these studies for theories about public organizations.) Child (1972) pointed out that in addition to the other contingencies that contingency theorists emphasized, managers' strategic choices play an important role in adapting organizational structure. These and numerous other efforts had by the mid-1970s established the contingency approach – the argument that organizational structures and processes are shaped by contingencies of technology, size, environment, and strategic choice – as a central topic in organization theory. Authors began to translate the contingency observations into prescriptive statements for use in “organizational design” (Daft, 2020; Galbraith, 1977; Mintzberg, 1989; Starbuck and Nystrom, 1981).

Like the other theories covered in this review and in later chapters, contingency theory soon encountered criticisms and controversies. Researchers disputed how the key concepts should be defined and measured. Some studies found a relationship between technology and structure; some did not (Tolbert and Hall, 2009). The basic idea that organizations must adapt to conditions they face, through such responses as adopting more flexible structures as they contend with more environmental uncertainty, still serves as a central theme in organization theory (Daft, 2020; Donaldson, 2001; Scott and Davis, 2015) and management practice (Peters, 1987). The developments in organizational research reviewed here have produced an elaborate field, with numerous professional journals carrying articles reporting analyses of a wide array of organizational topics. These journals and a profusion of books cover organizational structure, environment, effectiveness, change, conflict, communication, strategy, technology, interorganizational relations, and related variables.

More Recent Developments in Organization Theory and Research

In the past two decades, the field has moved in new directions, many of which represent extensions of contingency and open-systems theories, with increased emphasis on organizations' relations with their contexts (Scott and Davis, 2015). Later chapters describe how organization theorists have developed natural selection and population ecology models for analysis of how certain organizational forms survive and evolve in certain environmental settings (Aldrich, Ruef, and Lippmann, 2020; Hannan and Freeman, 1989; Scott and Davis, 2015; Tolbert and Hall, 2009). Other theorists have analyzed external controls on organizations, with emphasis on organizations' dependence on their environments for crucial resources (Pfeffer and Salancik, 1978). Principal-agent perspectives have been used to understand how actors use information to their advantage in relationships (Heinrich and Marschke, 2010). Political scientists have found this perspective useful in analyzing relations between government agencies and the political authorities seeking to control them (e.g., Wood and Waterman, 1994). Political scientists, however, have also emphasized that the relationships involve mutual influences that work both ways, rather than simply involving control by the political authorities (Krause, 1999; Waterman and Meier, 1998). Transaction cost theory emphasizes the costs involved when organizations provide a good or service for themselves, as opposed to attaining it from an internal source – a topic that has obvious relevance to the major trend toward outsourcing by government organizations. Institutional theory concerns the processes by which organizational characteristics become institutionalized – accepted as legitimate and enduring – in the organization, often due to influences of external institutions (Scott and Davis, 2015). The many perspectives create a sprawling theoretical landscape, but the various theoretical orientations provide insights for researchers and administrators who seek to understand public organizations. Later chapters discuss the relevance of those theories.

The research and theory on people and groups in work settings described earlier have similarly led to a proliferation of closely related work in organizational behavior and organizational psychology, including a similar trend toward elaborate empirical studies and conceptual development during the 1960s and 1970s. Thousands of articles and books have reported work on employee motivation and satisfaction, work involvement, role conflict and ambiguity, organizational identification and commitment, professionalism, leadership behavior and effectiveness, task design, and managerial procedures such as management by objectives and flextime.

As the different fields have progressed, new topics have emerged. Among many examples, the topic of organizational culture has received attention and is emphasized in Figures 1.1 and 1.2. Advances in technology – especially computer, information, and communications technology – have presented organizations and managers with dramatic new challenges and opportunities, and researchers have been pressing to develop the theoretical and research grounding needed to manage these developments. The increasing presence of women and other underrepresented groups in the workforce has given rise to literature focusing on representation and diversity in organizations (Fernandez, 2020; Riccucci, 2018) and feminist organization theory (Harquail, 2020). Subsequent chapters discuss these and other rapidly developing topics.

The Role of Public and Nonprofit Organizations and Their Management in Organization Theory

The rich field of organization theory provides many valuable concepts and insights on which this book draws. It also raises an important issue for those interested in public and nonprofit organizations and management: Have the characteristics of public and nonprofit organizations and their members been adequately analyzed in this voluminous literature? As mentioned in Chapter One and further described in later chapters, there has been a literature on public bureaucracies for many years (e.g., Downs 1962; Niskanen 1971), but the historical review provided here illustrates how little attention has been devoted to this literature by most of the organization theorists. Implicitly, many organization theorists convey the message that the distinction lacks importance.

The analysts discussed in the preceding historical review have either concentrated on industrial organizations or sought to develop generic concepts and theories that apply across all types of organizations. E.g., even though Peter Blau, a prominent organization theorist, published an organizational typology that included a category of “commonweal organizations” very similar to what this book calls public organizations, he published empirical studies that downplayed such distinctiveness of organizational categories (Blau and Scott, 1962). Blau and Schoenherr (1971) examined government agencies for their studies of organizational size, but they drew their conclusions as if they applied to all organizations. So have replications of Blau's study (Beyer and Trice, 1979), even though Argyris (1972, p. 10) suggested that Blau may have found the particular relationship he discovered because he was studying organizations governed by civil service systems. Such organizations might respond to differences in size in different ways than do other organizations, such as business firms. When the contingency theorists analyzed environments, they typically concentrated on environmental uncertainty, especially as a characteristic of business firms' market environments, and placed little emphasis on political or governmental dynamics in organizational environments.

Providing a more classical example of this tendency, Weber argued that his conception of bureaucracy applied to government agencies and private businesses alike (Meyer, 1979). Major figures such as James Thompson (1962) and Herbert Simon (Simon, Smithburg, and Thompson, 1950) have stressed the commonalities among organizations and have suggested that public agencies and private firms are more alike than different. The contributions to organization theory and behavior described in this review were aimed at developing theory that would apply generally to all organizations. With exceptions (Blau and Scott, 1962; Scott, 2003, pp. 341–344), the theorists repeatedly implied or aggressively asserted that distinctions such as public and private, market and nonmarket, and governmental and nongovernmental offered little value for developing theory or understanding practice. Herbert Simon continued to offer such observations until the end of his life. He contended that public, private, and nonprofit organizations are essentially identical on the dimension that receives more attention than virtually any other in discussions of the unique aspects of public organizations – the capacities of leaders to reward employees (Simon, 1995, p. 283, n. 3). He also bluntly asserted that it is false to claim “that public and nonprofit organizations cannot, and on average do not, operate as efficiently as private businesses” (Simon, 1998, p. 11). So one of the foremost social scientists of the twentieth century attached little importance to the distinction we seek to develop in the next chapter.

Even so, research and writing about public organizations had been appearing for many decades, which were related to organizational sociology and psychology in various ways. Political scientists analyzing public “bureaucracies” usually emphasized the relationship between the bureaucracy and other elements of the political system. Economists emphasized the absence of economic markets for the outputs of public bureaucracies (Downs, 1967; Niskanen, 1971).

Authors interested in the management of public organizations began to point to this gap between the two literatures (Rainey, 1983). Authors cited in this book mounted a critique of the literature on organization theory, saying that it offered an incomplete analysis of public organizations and their political and institutional environments (Hood and Dunsire, 1981; Meyer, 1979; Perry and Kraemer, 1983; Pitt and Smith, 1981; Wamsley and Zald, 1973; Warwick, 1975). Yet many of these also criticized the writings on public bureaucracy as too anecdotal and lacking in the empirical and conceptual analyses common in organization theory. Also, the literature on government organizations showed too little concern with internal structures, behavior, and management, topics that had received extensive attention from researchers on organization theory. Researchers began to provide more explicit organizational analyses of public organizations, of the sort described in this book. As Chapter One mentioned, books and articles have since then provided many additional contributions. This activity has not fully resolved whether we can clarify the meaning of public organizations and public management and show evidence that such categories have significance for theory and practice. The next chapter turns to the challenge of formulating a definition and drawing distinctions.

Instructor's Guide Resources for Chapter Two

  • Key terms
  • Discussion questions
  • Topics for writing assignments or reports
  • Exercises
  • Case Study: Moving the Maisenbacher House

Available at www.wiley.com/go/college/rainey.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.144.96.159