3.1. ORGANIZATION EFFECTIVENESS
Effectiveness can be determined by a number of different criteria. Table 3.1 lists some criteria that may be used; the reader will think of others. To some extent, the type of R&D organization will determine the criteria. The criteria listed in Table 3.1 are self-evident, but some comments are needed concerning the congruence of individual and organizational goals and the use of profit as a criterion.
| Criterion | Measurement Instrument |
|---|---|
| Quantity of output | Numbers of reports, publications, new products |
| Quality of the work | Number of patents obtained, number of times publications of lab members are quoted, number of refereed publications per member of lab |
| Increases in the size of organization | Obtaining more research funds |
| Absenteeism | Number of persons out of the total work force who are absent without a valid excuse on an average day (counted inversely) |
| Level of stress | Measured with physiological indexes, number of visits to hospital, frequency of peptic ulcers, etc. (counted inversely) |
| Level of job satisfaction | Measured with a standardized questionnaire, such as the Job Descriptive Index. Components: satisfaction with pay, supervisor, organization or company, job, co-workers, working conditions |
| Pride in the organization | Feelings of pride measured via questionnaires |
| Congruence of individual and organizational goals | The extent individual goals are consistent with goals as they are reflected in employee and management statements |
| Profits | Direct profits or return on investment studies where returns are determined from implementation of research products |
[] good case can be made for each organization developing its own criteria of effectiveness through participation of organization members in a debate that considers (1) different criteria, (2) how they should be measured, and (3) how they should be weighted. Such a debate has the advantage of involving the key members of the organization in the development of its goals. They become committed and ego-involved. The criteria that need to be debatsed are listed in Table 3.1. Other criteria might be suggested during these debates.
First, consider the congruence of individual and organizational goals. If the individual's activities are quite consistent with the activities and goals of the organization, this will result in a better organization than one in which individuals try to do "their own thing" and are not really concerned with what happens to the organization.
Next, consider profit. For a profit-oriented organization, revenues or earnings may provide a good measure of its productivity or effectiveness. However, for a research organization (or a nonprofit organization), other measures are needed. Nevertheless, a good way to integrate individual and organizational goals is to pay some bonus based on total organizational performance.
International Business Machines Corporation (IBM) is regarded as a leader in its industry for a variety of reasons. Out of 35,000 total research laboratories in the United States, IBM has set "the gold standard for turning research into corporate profits" (Feder, 2001). In 2000 IBM was awarded 2,922 patents in the United States, which was 43 percent more than second place NEC of Japan. Through patent licensing they have been able to generate $1.7 billion in fees. John Seely Brown, the former director of the Xerox Palo Alto Research Center, noted that IBM is the research center other company's benchmark themselves against. IBM invests heavily in research and development, which provides new product innovations to keep itself ahead of competitors. They attribute their profitability to the corporate culture, specifically through the fellows program. Being a "fellow" is a prestigious position within the company and brings financial bonuses and freedom in research. IBM also has a flair for public relations, helping to get their name out and be viewed as a technological pioneer. Linking monetary rewards to company profitability has also proved worthwhile in motivating employees.
In summary, R&D organization output measures can be subjective or objective, discrete or scalar, and quantitative or nonquantitative, and there can also be qualitative aspects associated with them. The relationship of output measures to organizational goals must also be included. An interesting categorization of output measures in terms of result, process, and social indicators has been proposed by Anthony and Young (2002).
Different organizations (governmental, commercial, and educational) will weigh the available criteria differently. It may be a useful exercise for the key teams of a lab to devote some time to a discussion of how the various criteria should be weighted. Agreement on how to do that is likely to increase the congruence of individual and organizational goals, and possibly reduce role conflict within the organization. Of course, in pure research the publication criterion is weighted more heavily, and in applied research the product that has been invented or developed is the key output that must meet certain specifications. These specifications themselves can be stated as criteria (e.g., product should cost less than a certain amount, should weigh less than a certain amount, should have certain performance characteristics, and so on). Sessions that are devoted to goal clarification and how specific criteria will be used to determine their attainment by the individual or the organization will take time, but will be of great value in creating a good climate of cooperation within the organization.
Commenting on organization effectiveness, Blake (1978, p. 260) suggests that the criterion for evaluating the effectiveness of an R&D organization should be the record of its success or failure in meeting its objectives. He recommends a set of questions that would form a basis for determining R&D organization effectiveness:
Are project cost schedules met?
Are project time schedules met?
Are time schedules kept that show both original estimated costs and actual costs of the projects?
Are records kept that show both the estimated completion time and actual completion time for the projects?
Is there clear delineation between overruns and cost increases caused by change in the scope of the projects or other proper causes?
Is there significant scientific fallout?
Szakonyi (1994) proposes ten R&D activities and six operating levels to measure organizational effectiveness. The proposed activities are as follows:
Selecting R&D
Planning and managing projects
Generating new product ideas
Maintaining the quality of the R&D process and methods
Motivating people
Establishing cross-disciplinary teams
Transferring technology
Fostering collaboration between R&D and finance, and
Linking R&D to business planning
These activities are evaluated in terms of their level of operation. The six proposed levels are as follows:
The issue is not recognized.
Initial efforts are made toward addressing the issue.
The right skills are in place.
Appropriate methods are used.
Responsibilities are clarified.
Continuous improvement is underway.
The proposed activities and levels of operation can be modified to suit organizational needs. This approach can be used as a diagnostic tool to identify deficiencies and ways to improve organization effectiveness.
Clearly, there are a number of ways of looking at organization effectiveness. Viewing organization effectiveness, and thus productivity, as a vector, the following relationship is proposed:
Productivity = Effectiveness = Output × Quality
3.1.1. Output Measures
Output has three categories: process measures, result measures, and strategic indicators. A quantitative or a qualitative measure can be assigned where possible; and these measures, where appropriate, relate to organizational objectives. Further description of these proposed measures follows.
3.1.1.1. Process Measures
These measures are process-oriented and relate to activities carried out by an organization or its subunit; they also relate to short-term, day-to-day activities of the organization. Some examples in a research organization may be
Number of times technical assistance was provided to an operational unit
Number of responses sent to enquiries from outside scientific or internal units
Number of visitors to the organization
Number of administrative types of actions handled
3.1.1.2. Result Measures
These would be tangible, measurable outputs expressed in terms of an organization's objectives and goals. Some examples are:
Number of technical reports published
Number of refereed papers published
Number of major innovations developed and adapted for commercialization
Return on R&D investment
3.1.1.3. Strategic Indicators
These indicators would focus on long-term and strategic aspects of the organization. Examples include:
Reputation of the research organization
Ability to attract highly qualified scientists
The degree of customer (sponsor organization) satisfaction with research output
Ability to attract research support for new high-risk research projects
Job satisfaction level of the employees
It is clear that there are many ways in which one can assess the effectiveness of an organization. Thus, when thinking of an effective organization we should include all the variables because any one of them is likely to be biased or contaminated by extraneous factors. Any one criterion could be biased because of the way it is confounded with other variables of which we are not aware, or because the way it is measured may not be as accurate as possible. On the other hand, if we utilize many criteria and there is some degree of convergence across these criteria, then we are reasonably sure that we are dealing with a meaningful overall criterion (e.g., the weighted sum of the above-mentioned criteria), which can be used as a means of assessing the effectiveness of an organization.