Chapter 1

Introduction to Research and Evaluation in Public Relations

Contemporary public relations practice has developed since the mid-20th century from the weak third sister in the marketing, advertising, and public relations mix to gain status as a full and equal player in the corporate suite. Part of that development can be traced to a change in the way public relations is practiced. The early days of public relations functions—limited to media relations and “press agentry”—have evolved into a sophisticated array of communications where public relations is no longer an afterthought, but is an integral part of the communications mix.

A central reason for this change in the perceptions of and stature of public relations in the communications world is the inclusion of research, measurement, and evaluation as a core part of the practice—tools that have been integral to the practice of marketing and advertising for decades. The purpose of this book is to provide the business reader and communications professional with the necessary and practical understanding of the problems and promises of public relations research, measurement, and evaluation—and more importantly, to act as a guide to the effective use of methods, measures, and analysis in providing grounded evidence of the success (or failure) of public relations campaigns.

Defining Public Relations and Its Objectives

What exactly is this profession called public relations? For many it is simply one of three promotional areas that management uses to get its message out: marketing, advertising, and public relations. What has differentiated them in the past can be viewed in terms of (a) what a business expects it to do and (b) the kinds of outcomes it produces. In too many eyes, public relations only includes dealing with media relations. That is, the objective of public relations to get coverage of the business—preferably positive—through the “placement” of articles and the like as endorsed by journalists.

But public relations is much more than “press agentry” or media relations. It is better seen as an umbrella term for any number of departments in a business or corporation that seeks to get its messages out to various publics or audiences by managing the flow of information between an organization and its publics or audiences.¹

A public is a part of a population that has been selected for study; an audience is a specifically targeted group within that public that has been targeted for a company’s messages. What then is public relations? First and foremost, public relations serves to manage the credibility, reputation, trust, relationship, and confidence of the general public in relation to the company.² As Professor Donald K. Wright noted, “Public relations is the management function that identifies, establishes, and maintains mutually beneficial relationships between an organization and the various publics on which its success or failure depends.”³

How is public relations practiced if it is an umbrella concept? Its practice can be defined by its function in the organization. Public relations takes on the following functions, sometimes alone and at other times as a combined function. The following list is neither complete nor is it by importance of function:

• Community relations
• Corporate communications
• Customer relations
• Employee relations
• Financial relations
• Government relations
• Media relations
• Public affairs
• Strategic communications

What then are public relations’ objectives? There three major objectives any public relations campaign seeks to accomplish: (a) to ensure that the messages get out to their intended audiences and that they are understood (“informational objective”); (b) to monitor the campaign so that benchmarks regarding the acceptance of messages by target audiences in terms of cognitive, affective, and behavioral attitudinal or belief acceptance, rejection, or maintenance (“motivational objective”); and (c) predicting what the target audience will actually do based on the campaign (“behavioral objective”). As Stacks points out, each objective must be met and then monitored before the next objective can be obtained.⁴ In forthcoming chapters we will introduce a number of ideas on how a public relations campaign should operate. We will look at how traditional public relations campaigns experienced unintended problems due to a lack of research and how the failure to establish measurable objectives, baselines, and benchmarks limits the effectiveness of public relations. More importantly, we will introduce a practical approach to public relations research that will result in better and more effective communications programs.

A Brief History of Public Relations Research

The formal origins of public relations research can be traced to the 1950s.⁵ During that period, a company called Group Attitudes Corporation was acquired by Hill & Knowlton.⁶ The primary focus of Group Attitudes Corporation was to function as a stand-alone yet captive arm of the parent agency. Its work included research for the Tobacco Institute,⁷ as well as for other Hill & Knowlton clients. The primary focus of this research, taken from a review of several published reports, was to assess reaction to communications messages and vehicles using processes that appear similar to the research methods employed by the advertising industry during this same period. This industry model was followed over the next 25 years with the establishment of research arms at several other public relations agencies. In addition to Hill & Knowlton, the major public relations agencies that have had research departments include Burson-Marsteller (Penn Schoen Berland), Ruder Finn (Research & Forecasts), Ketchum, Weber Shandwick (KRC), Edelman (Strategy One), Ogilvy Public Relations, APCO, Golin Harris, and GCI Group. For the most part, the primary function of these agency-based research departments was similar to the work initially conducted by Group Attitudes Corporation. Most of these research departments were created internally, with the notable exception of Penn Schoen Berland, which was acquired by WPP and later merged into Burson-Marsteller.

As early as the 1930s, methods were also being developed by advertisers and their agencies that linked exposure and persuasion measures to actual store sales. In essence, testing, measurement, analysis, and evaluation systems became an integral part of the advertising industry. These systems became so institutionalized by mid-decade that an academic journal—The Journal of Advertising Research—as well as an industry association—The Advertising Research Foundation—were established in 1936. Other journals followed and formal academic programs in marketing research were established at major universities throughout the United States.

During the late 1970s, it became increasingly apparent that public relations differed considerably from other communications disciplines, and advertising in particular, in its ability to be measured and evaluated. At the time, advertising testing was dominated by a variety of measurement and evaluation systems, of which the “day after recall” method (DAR), popularized by Burke Marketing Research in its work with Procter & Gamble, was one of the most common systems in use. These advertising-focused methods took on a source orientation and assumed that the message was completely controlled by the communicator.⁸ Therefore the ability to test message recall and message efficacy were highly controllable and, in theory, projectable as to what would occur if the advertising were actually to be placed.

With the recognition that public relations needed a different set of measures because of the unique nature of the profession, senior management at several major public relations agencies charged their research departments with the task of finding more credible and reliable methods to measure the effectiveness of public relations activities. While a number of experiments were undertaken at that time, the primary benefit derived from this experimentation was a heightened awareness of the overall value of measuring public relations.

This heightened awareness, along with advances in specific technologies, led to the founding of a number of research companies during the 1980s and 1990s that specialize in measuring and evaluating the outcome of public relations activities as well as the founding of a trade association (International Association for Measurement and Evaluation of Communication formerly known as the Association of Media Evaluation Companies, AMEC http://www.amecorg.com),⁹ the Commission on Public Relations Research and Evaluation and the Research Fellows both of which are affiliated with the Institute for Public Relations (http://www.instituteforpr.org).¹⁰ Currently approximately two dozen companies offer services that measure and evaluate public relations activities.¹¹ These companies have traditionally focused on evaluating only the outcomes of public relations, most commonly as media or press coverage that is a direct result of media relations activities (outputs). Few of their staff have formal or academic research training outside of “on-the-job” training in content analysis, and unlike other forms of communications research, these companies typically place little emphasis on formative, programmatic, or diagnostic research or research that is used to develop communications strategies and evaluate the impact of communications activities on target audiences.

The primary limitation of these companies is their focus on an intermediary in the public relations process—the media—rather than on the target audiences for these communications activities.

While the legacy of these public relations research agencies, as well as the services they provide the public relations industry, is noteworthy, for the most part they have failed to significantly advance either the science or the art of public relations measurement and evaluation because of their strong emphasis on media relations.

This lack of advancement occurred despite an insistence and commitment by the leadership of the profession that research function as a key and essential element in the creation of effective and successful public relations programs. Industry leaders who demanded the use of research in the development and evaluation of public relations programs included luminaries such as Harold Burson (Burson-Marsteller), Daniel Edelman (Edelman Worldwide), and David Finn (Ruder Finn), each of whom established dedicated research functions in their respective agencies.

The most significant commitment of the industry leadership to this nascent discipline was the founding in 1956 of the Foundation for Public Relations Research and Education (now operating as the Institute for Public Relations [IPR]) in conjunction with the Public Relations Society of America (PRSA). Over the past 50 years, the foundation has continued to emphasize the critical importance of research in the public relations process and has dedicated itself to “the science beneath the art of public relations.” Yet even with this dedicated effort, the IPR struggles to get the public relations profession and public relations professionals to recognize the importance of research and measurement as an essential element in the development of effective public relations programs. This struggle continues in spite of ongoing programs, conferences, and educational forums that focus exclusively on this agenda.

Moving Toward Excellence in Public Relations Research

While the IPR has been a continuing beacon on issues surrounding the inclusion of research in all public relations efforts, the shift toward using research to establish the foundation of public relations practice achieved its most significant support during the 1980s. In 1984, the International Association of Business Communicators (IABC) Foundation (now the IABC Research Foundation) developed a request for proposals for the landmark study of Excellence in Public Relations and Communication Management—a project that produced three books, many reports, and dozens of seminars for professionals on creating excellence in the practice of public relations.

In its request for proposals, the IABC board asked for proposals for research that would demonstrate how, why, and to what extent communication contributes to the achievement of organizational objectives and how the public relations function should be organized to best achieve those objectives. The foundation awarded a $400,000 grant to a team that included Professors James E. Grunig and Larissa Grunig of the University of Maryland and Professor David Dozier of San Diego State University.

This team, among others, produced numerous publications on excellence in public relations practice that include five major volumes.¹² Excellence in Public Relations and Communication Management by James Grunig; Manager’s Guide to Excellence in Public Relations and Communication Management by David Dozier, Larissa Grunig and James Grunig; Excellent Public Relations and Effective Organizations: A Study of Communication Management in Three Countries by James Grunig, Larissa Grunig and David Dozier; Managing Public Relations by James Grunig and Professor Todd Hunt; and The Future of Excellence in Public Relations and Communication Management: Challenges for the Next Generation by Professor Elizabeth Toth.

In addition, the Arthur W. Page Society underwrote one of the earliest and most business-oriented research volumes, Using Research in Public Relations: Applications to Program Management by Professor Glen Broom and David Dozier.¹³

Twenty-five years later, this remains the single largest grant in this field for the development of research protocols and practices. Yet even with this effort, the inclusion of research as a basic tool in the day-to-day practice of public relations remains an elusive goal.

During this period the profession has seen growth that is best represented by the multitude of companies specializing in this area, as well as a growing amount of academic literature in the field. Yet even with the increased attention paid, significant variations continue to exist in the range of approaches to public relations measurement and evaluation. These variations have resulted in a lack of standard measures that can be used to gauge the success of a public relations program, as well as in an uneven overall quality of the research being conducted.

There are likely many reasons why research in support of public relations activities has failed to progress significantly over the past 60 years. The reasons cited for this lack of advancement range from a genuine lack of commitment by the profession to a lack of resources, to a proprietary approach to research as a business edge, to changes in the practice of public relations, among others. However, there is another area where public relations research has failed and which is most likely the greatest contributor to its limited growth. That failure has been a systematic lack of understanding and application of the best practices necessary to achieve the levels of excellence required to advance the use of research to support the practice of public relations and, in turn, advance the overall practice of public relations.

The Concept of Best Practices

The history of “best practices” originated in the business literature during the origins of the industrial era.¹⁴ The concept was that, while there are multiple approaches that can be used to achieve a task or a goal, there is often a single technique, method, or process that is more effective than others in reaching an established goal. In essence, a best practice is a technique, a method, a process, or an activity that is more effective at delivering a particular outcome than any other technique, method, process, or activity. By using best practices, projects, tasks, and activities can be accomplished more effectively and with fewer problems and complications.

There is an essential relationship between public relations research and practice. In particular, there is the relationship between evaluation and measurement and successful public relations practices. The focus of this book will be on what has been labeled “best practices in public relations measurement and evaluation systems.”¹⁵ Public relations best practices include (1) clear and well-defined research objectives, (2) rigorous research design, and (3) detailed supporting documentation. The quality and substance of the research findings (1) demonstrate effectiveness, (2) link outputs (tactics) to outcomes, (3) develop better communications programs, (4) demonstrate an impact on business outcomes, (5) demonstrate cost-effectiveness, and (6) are applicable to a broad range of activities.

As Figure 1.1 demonstrates, there is a strong interrelationship between the organization setting communication objectives, messages sent by the organization, how those messages are received, and how the outtakes from those messages impact on the objectives and goals set by the organization.

As noted in a commentary from PricewaterhouseCoopers, “Best practices are simply the best way to perform a business process. They are the means by which leading companies achieve top performance, and they serve as goals for other companies that are striving for excellence.”¹⁶

While the concept of best practices is often applied to the operations of a specific company, the logical extension of best practices is its application to an overall industry through the establishment of standards against which assessments can be made. The goal of this book is to present best practices as they apply to public relations research, measurement, and evaluation. This presentation of best practices is not to provide definitive answers to business problems associated with communication. Rather, these best practices are meant to be sources of creative insight for improving the application of public relations research and, in turn, for improving the overall quality and effectiveness of public relations activities.

Figure 1.1 Best practices in public relations.

Used with permission of Public Relations Journal.

What This Book Will Cover

Specifically, this book will provide business readers with a basic understanding of the problems and promises of public relations research, measurement, and evaluation, while providing public relations professionals, present and future, with a guide to the effective use of research methods and measures, and analytical insight that leads to meaningful evaluation in providing grounded evidence of the success (or failure) of public relations campaigns, as well as the necessary information to plan an effective campaign.¹⁷

A Practitioner’s Guide to Public Relations Research, Measurement, and Evaluation is divided into five broad parts broken into short chapters:

• Part I—Business and the Practice of Public Relations

This part covers three key areas that are essential to the creation of any effective public relations research. The first is a review of basic public relations theory and how public relations activities can be tied to predicting measureable business outcomes (Chapter 1).

The second is a detailed examination of public relations goals and objectives in light of measureable business objectives. Included in this review is a discussion of what public relations goals and objectives can be realistically achieved and acceptable measures for each of these basic goals and objectives (Chapter 2).

The third is a discussion of the elements of establishing achievable public relations goals in light of overall business objectives and then reviewing the processes for setting communications objectives that are active, measurable, and can be evaluated (Chapter 3).

• Part II—Qualitative Methods for Effective Public Relations Research Measurement, and Evaluation

This part reviews the four major methodological areas that are commonly used in public relations research, measurement, and analysis. This part covers historical/secondary research (Chapter 4); qualitative research, including in-depth interviews, focus groups, and participant observation and quantitative research (Chapter 5); and content analysis (Chapter 6). This part will also introduce the use of baseline and benchmark measurements and their effective application, historical research methods, and how to best use secondary research sources as part of a complete public relations research program. Particular emphasis will be placed on content analysis as one of the most commonly used and misused public relations methodologies, as well as on quantitative methods.

• Part III—Quantitative Methods for Effective Public Relations Research, Measurement, and Evaluation

This part focuses on the quantitative dimension of public relations research, beginning with an emphasis on survey research (Chapter 7). Descriptive statistics and the presentation of data (Chapter 8) are examined as the baseline for understanding quantitative analysis as well as the concepts of probability and generalizing to larger audiences or populations. Finally, a discussion of sampling (Chapter 9) rounds out the part.

• Part IV—Best Practices of Public Relations Research Measurement, and Evaluation

The final part of this book looks at the future of public relations research and the specific practices that will ensure the value of research in creating effective and valuable public relations programs (Chapter 10).

Finally, the appendices include a bibliography of research and measurement sources, the Dictionary of Public Relations Research and Measurement.

Taken together, each of these parts will provide practitioners as well as nonpractitioners with a basic understanding of what is needed to evaluate public relations campaigns and ensure that research, measurement, and evaluation “toolkits” are up to date and complete.