13.7. EVALUATING ORGANIZATIONAL CHANGE

The several kinds of change outlined above need to be evaluated. The organization needs to know whether the cost of particular changes can be justified. It needs to assess what techniques of change work and what techniques are ineffective. Since each organization is to some extent unique and with its own culture, the fact that one technique worked in another organization does not mean it will be effective in your own.

The most important attitude that managers need to develop about organizational change is that every change is an experiment. When we do an experiment we modify some independent variables and measure some dependent variables. Similarly when we undertake an organizational change (our independent variable) we must measure its effects on the organization (our dependent variable). The reason we do experiments is to unlock the secrets of nature. Organizations are also part of nature. In fact, they are a very complex part of nature. Designing changes for organizations that will improve them is much more difficult than doing a chemistry experiment. So we must be modest in our expectations and not anticipate miracles. But with systematic change and careful measurement, we should be able to sort the changes that are effective from those that are ineffective or hurt the organization.

Unfortunately, administrators often have the wrong attitudes about organizational change. Since they are the ones who approved the change, they feel ego-involved. Since they want the change to succeed, they are unable to take an objective, open, experimental approach. Yet that is exactly the approach that is needed. It is important that managers train themselves to see organizational change supportively, but also critically. If it fails, try again. In other words, the correct attitude is to view change as an experiment, which may or may not work. If all our experiments came out the way we expected, there would be no point in doing them! It is exactly because we get unanticipated results that we keep experimenting. Similarly, we should not assume that every innovation will benefit the organization, and we should not put down our peers or subordinates who fail to introduce successful innovations. In fact, if one never fails, this may be a sign of too low a level of risk. The bold innovator has more failures than successes, but the few successes often change the world.

In sum, we must not expect that our first idea about organizational change is going to work. Try, and try again. Keep measuring what happens and you will gradually find out what works.

When doing evaluation research following organizational change, we need to use many and very different methods. For example, one may wish to measure job satisfaction, turnover rates, productivity rates, quality of publications, and many other dependent variables before deciding that a particular reform has or has not been successful.

In evaluating reforms, one must also consider that any particular reform continues over time and thus cannot always be evaluated at only one point. In fact, people who are specialists in evaluation research have distinguished between formative and summative evaluation. Formative evaluation examines the effects of the change as it happens. Thus one modifies the organizational change to take into account the results of the evaluation. In the case of summative evaluation, on the other hand, one waits until the change has occurred, and has been in place for some time, before making the evaluation.

Specialists in evaluation have used a variety of ways to make their evaluations. For example, some people advocate self-study as a means of getting the group affected by the change to assess how the particular reform has worked for them. Another approach consists of forming a blue-ribbon committee, usually consisting of people outside of the administrative unit in which the change has occurred, who come in, ask a lot of questions, and make a judgment as to whether the particular reform has been effective. Still another method is to look at particular data sets as criteria for an effective reform. These data sets may include the dollar amounts of grants and contracts obtained to support the research, the number of publications, judgments about the quality of the publications, the extent to which the persons or groups who were part of the changes are being quoted, or the reputation of the group. Still other possibilities include bringing in a specialist with an adversarial view, whose role is to discover that the reform is ineffective. This specialist is usually a critic of the reform and may often uncover problems that the participants may not see. Still another approach is goal-free evaluation, in which the evaluator simply tries to find out what "really" is happening or has happened. The idea in this case is that the evaluator is an unbiased spectator who can evaluate the change most appropriately. There are also classical evaluation specialists, who utilize "experimental" and "control" groups or look at the results of change over time. They use the particular group's performance prior to the introduction of a reform as a control for the evaluation of the change that has occurred since the reform was introduced. Still another approach is the one used by anthropologists who look at what is happening and describe it as well as they can from the perspective of the "natives" (the members of the organization). This is done without any idea about the antecedents or the correlates of the observations they make.

It is often the case that a combination of these approaches may be optimal in order to gain a really good understanding of the effects of the reform.

Another issue in evaluation research is the question of whose perspective to take more seriously. For example, in a department in a particular organization, the members of the department may represent one perspective, top management may represent a different perspective, the supervisor of the department may have a third perspective, and the peers in other departments may have a fourth perspective. Who is to say whose perspective should be taken more seriously? Should one weigh the various perspectives to get a single index that reflects the particular reform?