Introduction to Research and Evaluations in Public Relations
Contemporary public relations practice has developed since the mid-20th century from the weak third sister in the marketing, advertising, and public relations mix to gain status as a full and equal player in the corporate suite. Part of that development can be traced to a change in the way public relations is practiced. The early days of public relations functions—limited to media relations and “press agentry”—have evolved to a sophisticated array of communications where public relations is no longer an afterthought, but is an integral part of the communications mix.
A central reason for this change in the perceptions of and stature of public relations in the communications world is the inclusion of research, measurement, and evaluation as a core part of the practice—tools that have been integral to the practice of marketing and advertising for decades. The purpose of this book is to provide the business reader and communications professional with the necessary and practical understanding of the problems and promises of public relations research, measurement, and evaluation—and more importantly as a guide to the effective use of methods, measures, and analysis in providing grounded evidence of the success (or failure) of public relations campaigns.
Defining Public Relations and Its Objectives
Why exactly is this profession called public relations? For many it is simply one of the three promotional areas that management uses to get its message out: marketing, advertising, and public relations. What has differentiated them in the past can be viewed in terms of (a) what a business expects it to do and (b) the kinds of outcomes it produces. In all too many eyes, public relations only includes dealing with media relations. That is, public relations’ objective is to get coverage of the business—preferably positive—through the placement of articles and the like as endorsed by journalists.
But public relations is much more than press agentry or media relations. It is better seen as an umbrella term for any number of departments in a business or corporation that seeks to get its messages out to various publics or audiences by managing the flow of information between an organization and its publics or audiences (Grunig and Hunt 1984). Public is a part of a population that has been selected for study; an audience is a specifically targeted group within that public that has been targeted for a company’s messages. What then is public relations? First and foremost, public relations serves to manage the credibility, reputation, trust, relationship, and confidence of the general public in relation to the company (Stacks 2017). As Professor Donald K. Wright noted, “Public relations is the management function that identifies, establishes, and maintains mutually beneficial relationships between an organization and the various publics on which its success or failure depends” (Wright 1990).
How is public relations practiced if it is an umbrella concept? Its practice can be defined by its function in the organization. Public relations takes on the following functions, sometimes alone and at other times as a combined function. The following list neither is complete nor is it listed by importance of function:
Community relations
Corporate communication
Customer relations
Employee relations
Financial relations
Governmental relations
Media relations
Public affairs
Strategic communication
What then are public relations’ objectives? There are three major objectives any public relations campaign seeks to accomplish. The first is to ensure that the messages get out to their intended audiences and that they are understood (informational objective). The second is to monitor the campaign so that benchmarks are set regarding the acceptance of messages by target audiences in terms of cognitive, affective, and behavioral attitudinal or belief acceptance or rejection or maintenance (“motivational objective”). And, the third is predicting what the target audience will actually do based on the campaign (“behavioral objective”). As Stacks (2011) points out, each objective must be met and then monitored before the next objective can be obtained. In essence, public relations is a broad scale function that encompasses the full range of communication from message development, to message delivery, receipt of message, impact on target audiences, and effect on business outcomes.
In forthcoming chapters we will introduce a number of ideas of how a public relations campaign should operate and be measured across the full range of public relations. We will look at how traditional and digital public relations campaigns experienced unintended problems due to a lack of research and how the failure to establish measurable objectives, baselines, and benchmarks limits the effectiveness of public relations. More importantly, we introduce a practical approach to public relations research that will result in better and more effective communications programs. Finally, we will move from a focus on best practices to standards. Best practices are sufficient conditions to research and they have been in the profession surpassed by an understanding that standards must be set that allow for evaluation against set research and measurement metrics. It is only then that results can be evaluated and transmitted to the client.
A Brief History of Public Relations Research
The formal origins of public relations research can be traced to the 1950s (The New York Times 1990). During that period, a company called Group Attitudes Corporation was acquired by Hill & Knowlton (The New York Times 1990). The primary focus of Group Attitudes Corporation was to function as a standalone yet captive arm of the parent agency. Its work included research for the Tobacco Institute (Legacy Tobacco Document Library n.d.), as well as for other Hill & Knowlton clients. The primary focus of this research, taken from a review of several published reports, was to assess reaction to communication messages and vehicles using processes that appear similar to the research methods employed by the advertising industry during this same period. This industry model was followed over the next 25 years with the establishment of research arms at several other public relations agencies. In addition to Hill & Knowlton, the major public relations agencies that have had research departments include Burson-Marsteller (Penn Schoen Berland), Ruder Finn (Research & Forecasts), Ketchum, Weber Shandwick (KRC), Edelman (Edelman Insights), Ogilvy Public Relations, APCO, Golin Harris and Cohn & Wolfe. For the most part, the primary function of these agency-based research departments was similar to the work initially conducted by Group Attitudes Corporation. Most of these research departments were created internally, with the notable exception of Penn Schoen Berland that was acquired by WPP and later merged into Burson-Marsteller.
As early as the 1930s, methods were also being developed by advertisers and their agencies that linked exposure and persuasion measures to actual store sales. In essence, testing, measurement, analysis, and evaluation systems became an integral part of the advertising industry. These systems became so institutionalized by mid-decade that an academic journal—The Journal of Advertising Research—as well as an industry association—The Advertising Research Foundation—were established in 1936. Other journals followed and formal academic programs in marketing research were established at major universities throughout the United States.
During the late 1970s, it became increasingly apparent that public relations differed considerably from other communications disciplines, and advertising in particular, in its ability to be measured and evaluated. At the time, advertising testing was dominated by a variety of measurement and evaluation systems of which the day after recall (DAR) method, popularized by Burke Marketing Research in its work with Procter & Gamble, was one of the most common systems in use. These advertising-focused methods took on a source orientation and assumed that the message was completely controlled by the communicator (Miller and Levine 2009; Miller and Burgoon 1974). Therefore, the ability to test message recall and message efficacy were highly controllable and, in theory, projectable as to what would occur if the advertising were actually to be placed.
With the recognition that public relations needed a different set of measures because of the unique nature of the profession, senior management at several major public relations agencies charged their research departments with the task of finding more credible and reliable methods to measure the effectiveness of public relations activities. While a number of experiments were undertaken at that time, the primary benefit derived from this experimentation was a heightened awareness of the overall value of measuring public relations.
This heightened awareness, along with advances in specific technologies, led to the founding of a number of research companies during the 1980s and 1990s that specialize in measuring and evaluating the outcome of public relations activities as well as a trade association (International Association for Measurement and Evaluation of Communication, formerly known as the Association of Media Evaluation Companies, AMEC; www.amecorg.com), the Commission on Public Relations Research and Evaluation and the Research Fellows both of which are affiliated with the Institute for Public Relations (IPR) (www.instituteforpr.org). Currently, numerous companies offer services that measure and evaluate public relations activities. These companies have traditionally focused on evaluating only the outcomes of public relations, most commonly as media or press coverage that is a direct result of media relations activities (outputs). Few of their staff have formal or academic research training outside of “on the job” training in content analysis and, unlike other forms of communication research, these companies typically place little emphasis on formative, programmatic or diagnostic research, or research that is used to develop communication strategies and evaluate the impact of communication activities on the target audiences.
The primary limitation of these companies is their focus on an intermediary in the public relations process—the media—rather than on the target audience(s) for these communication activities.
While the legacy of these public relations research agencies, as well as the services they provide the public relations industry, is noteworthy, for the most part they have failed to significantly advance either the science or the art of public relations measurement and evaluation because of their strong emphasis on media relations.
This lack of advancement occurred despite an insistence and commitment by the leadership of the profession that research functions as a key and essential element to the creation of effective and successful public relations programs. These industry leaders demanding the use of research in the development and evaluation of public relations programs included luminaries such as Harold Burson (Burson-Marsteller), Daniel Edelman (Edelman Worldwide), and David Finn (Ruder Finn), each of whom established dedicated research functions in their respective agencies.
The most significant commitment of the industry leadership to this nascent discipline was the founding in 1956 of the Foundation for Public Relations Research and Education (now operating as the Institute for Public Relations [IPR]) in conjunction with the Public Relations Society of America (PRSA). Over the past six decades, the Foundation has continued to emphasize the critical importance of research in the public relations process and has dedicated itself to “the science beneath the art of public relations” (Institute for Public Relations 2011). Yet even with this dedicated effort, the IPR struggles to have the public relations profession and public relations professionals recognize the importance of research and measurement as an essential element in the development of effective public relations programs. This struggle continues in spite of ongoing programs, conferences, and educational forums that focus exclusively on this agenda.
Moving Toward Excellence in Public Relations Research
While the IPR has been a continuing beacon on issues surrounding the inclusion of research in all public relations efforts, the shift toward using research to establish the foundation of public relations practice achieved its most significant support during the 1980s. In 1984, the International Association of Business Communicators (IABC) Foundation (now the IABC Research Foundation) developed a request for proposals for the landmark study of “Excellence in Public Relations and Communication Management”—a project that produced three books, many reports, and dozens of seminars for professionals on creating excellence in the practice of public relations.
In its request for proposals, the IABC Board asked for proposals for research that would demonstrate how, why, and to what extent communication contributes to the achievement of organizational objectives and how the public relations function should be organized to best achieve those objectives. The Foundation awarded a $400,000 grant to a team that included Professors James E. Grunig and Larissa Grunig of the University of Maryland and Professor David Dozier of San Diego State University.
This team, among others, produced numerous publications on excellence in public relations practice that include five major volumes: Excellence in Public Relations and Communication Management by James Grunig (Grunig 1992); Manager’s Guide to Excellence in Public Relations and Communication Management by David Dozier, Larissa Grunig, and James Grunig (Dozier, Grunig, and Grunig 1995); Excellent Public Relations and Effective Organizations: A Study of Communication Management in Three Countries by James Grunig, Larissa Grunig, and David Dozier (Grunig, Grunig, and Dozier 2002); Managing Public Relations by James Grunig and Professor Todd Hunt (Grunig and Hunt 1984); and The Future of Excellence in Public Relations and Communication Management: Challenges for the Next Generation by Professor Elizabeth Toth (Toth 2009).
In addition, the Arthur W. Page Society underwrote one of the earliest and most business-oriented research volumes, Using Research in Public Relations: Applications to Program Management by Professors Glen Broom and David Dozier (Broom and Dozier 1990).
Almost 30 years later, this remains the single largest grant in this field for the development of research protocols and practices. Yet even with this effort, the inclusion of research as a basic tool in the day-to-day practice of public relations professionals remains an elusive goal.
During this period, the profession has seen growth that is best represented by the multitude of companies specializing in this area, as well as a growing academic literature in the field. Yet, even with the increased attention paid, significant variations continue to exist with the varying range of approaches to public relations measurement and evaluation. These variations have resulted in a lack of standard measures that can be used to gauge the success of a public relations program as well as in an uneven overall quality of the research being conducted. We will cover the argument for standards and standardization of measurement and evaluation in Chapter 2.
There are likely many reasons why research in support of public relations activities failed to progress significantly over the past 60 years. These reasons cited for this lack of advancement range from a genuine lack of commitment by the profession to a lack of resources to a proprietary approach to research as a business edge to changes in the practice of public relations among others. However, there are two other areas where public relations research has failed and these are most likely the greatest contributors to its limited growth. Those failures have been due to a systematic lack of understanding and application of standards for measurement and evaluation as well as a lack of knowledge of the best practices necessary to achieve the levels of excellence required to advance the overall practice of public relations.
The Concept of Best Practices
The history of best practices originated in business literature during the origins of the industrial era (Taylor 1919). The concept was that, while there are multiple approaches that can be used to achieve a task or a goal, there is often a single technique, method, or process that is more effective than others in reaching an established goal. In essence, a best practice is a technique, a method, a process, or an activity, which is more effective at delivering a particular outcome than any other technique, method, process, or activity. By using best practices, projects, tasks, and activities can be accomplished more effectively and with fewer problems and complications.
There is an essential relationship between public relations research and practice. In particular, there is the relationship between evaluation and measurement and successful public relations practices. The focus of this book is on what has been labeled “best practices in public relations measurement and evaluation systems” (Michaelson and Macleod 2007). Public relations best practice entails (1) clear and well-defined research objectives, (2) rigorous research design, and (3) detailed supporting documentation. Second, is the quality and substance of the research findings that (1) demonstrate effectiveness, (2) link outputs (tactics) to outcomes, (3) develop better communication programs, (4) demonstrate an impact on business outcomes, (5) demonstrate cost effectiveness, and (6) is applicable to a broad range of activities.
As the following chart demonstrates (see Figure 1.1), there is a strong interrelationship between the organization setting communication objectives, messages sent by the organization, how those messages are received, and how the outtakes from those messages impact on the objectives goals set by the organization.
As noted in a commentary from PriceWaterhouseCoopers, “Best practices are simply the best way to perform a business process. They are the means by which leading companies achieve top performance, and they serve as goals for other companies that are striving for excellence” (www.globalbestpractices.com/Home/Document.aspx?Link=Best+practices/FAQs&Idx=BestPracticesIdx).
While the concept of best practices is often applied to the operations of a specific company, the logical extension of best practices is its application to an overall industry through the establishment of standards against which assessments can be made. The goal of this book is to present best practices as they apply to public relations research, measurement, and evaluation. This presentation of these best practices is not to provide definitive answers to business problems associated with communication. Rather, these best practices are meant to be sources of creative insight for improving the application of public relations research and, in turn, improving the overall quality and effectiveness of public relations activities overall.
Figure 1.1 Best practices in public relations
Before we can move to best practices, it is important to understand why we need to apply measurement to public relations activities. The role of public relations is to aid in achieving positive business outcomes. These outcomes are typically changes in attitude that impact behavior.
These attitudes are affected by communication to target audiences that contains messages which provide a motivation for audiences to take a desired action and in turn advocate for others to follow their behavior. These messages originate with the communicator and are delivered through paid, earned, shared, and owned media that is also referred to as P.E.S.O.
The challenge for most communicators is to understand where along this chain of communication challenges occur so that corrective actions can be taken which assure communication programs achieve their business objectives.
It is important to understand that applied public relations research, like all research in the business world, is a process. This process begins at a particular point as determined at the research onset and then moves from objective-to-objective with a focus on end-to-end measurement. It is a process in which findings add to our knowledge base and require us to both review prior research in the light of these findings and to use this information in strategic decision making.
Public relations measurement takes place at five distinct points where each is a critical juncture that can determine the success or failure of a communication program. These five points include creation of the message, distribution of the message, presence of the message through intermediaries, commentary on the messages and impact of the messages on behavior.
In the second edition of this book, we established that best practice public relations research was becoming the norm rather than the exception. We then advanced the notion that the profession must next look at standards against which measures of public relations performance are compared. In the next chapter we focus on three sets of standards as published in the industry-wide journal, Public Relations Journal. Chapter 2 discusses the communication stages that public relations activities can be evaluated against. These standards include measurement and evaluation, research ethics, and campaign effectiveness or excellence (Michaelson and Stacks 2011; Michaelson, Wright, and Stacks 2012; Bowen and Stacks 2013b).
What This Book Covers
Specifically, this book provides business readers with a basic understanding of the problems and promises of public relations research, measurement, and evaluation, while providing public relations professionals in present and future a guide to the effective use of the research methods, measures, and analytical insight that leads to meaningful evaluation in providing grounded evidence of the success (or failure) of public relations campaigns as well as for the necessary information to plan an effective campaign.
A Professional and Practitioner’s Guide to Public Relations Research, Measurement, and Evaluation is divided into five broad parts broken into short chapters on:
Part I: Business and the Practice of Public Relations
The part covers three key areas that are essential to the creation of any effective public relations research.
The first is a review of basic public relations theory and how public relations activities can be tied to predicting measureable business outcomes (Chapter 1).
The second is a detailed analysis of the need for and standards against which public relations research can be evaluated and the application of these standards in evaluating the research and then communicating it to the client (Chapter 2).
The third is a detailed examination of public relations goals and objectives in light of measureable business objectives. Included in this review is a discussion of what public relations goals and objectives can be realistically achieved and acceptable measures for each of these basic goals and objectives (Chapter 3).
The fourth is a discussion of the elements of establishing achievable public relations goals in light of overall business objectives and then reviewing the processes for setting communication objectives that are active, measurable, and can be evaluated (Chapter 4).
Part II: Qualitative Methods for Effective Public Relations Research, Measurement, and Evaluation
Part II reviews the four major methodological areas that are commonly used in public relations research, measurement, and analysis. This part covers historical or secondary research (Chapter 5); qualitative research including in-depth interviews, focus groups, and participant-observation and quantitative research (Chapter 6); and content analysis (Chapter 7).
This part also introduces the use of baseline or benchmark, or both, measurements and their effective application, historical research methods, and how to best use secondary research sources as part of a complete public relations research program.
Particular emphasis is placed on content analysis as one of the most commonly used and misused public relations methodologies, as well as on quantitative methods.
Part III: Quantitative Methods for Effective Public Relations Research Measurement and Evaluation
Part III focuses on the quantitative dimension of public relations research, beginning with an emphasis on sampling (Chapter 8).
The application of survey research of data and data collection methods are reviewed in the next chapter (Chapter 9). Finally, a discussion of statistical reasoning (Chapter 10) rounds out Part III where we examine further understanding quantitative analysis and the concepts of probability and generalizing to larger audiences or populations. Here we discuss the differences between datasets defined as “big,” “large,” and “small” and how the dataset reflects the goal of the research.
Part IV: Best Practices of Public Relations Research, Measurement, and Evaluation
This final part of this book looks to the future of public relations research and the specific practices that will ensure the value of research in creating effective and valuable public relations programs (Chapter 11).
Finally, Appendix B includes a bibliography of research and measurement sources, the Dictionary of Public Relations Research and Measurement (3rd ed.).
Taken together, each of these parts provide professionals as well as nonprofessionals with a basic understanding needed to evaluate public relations campaigns and ensure that research, measurement, and evaluation toolkits are up-to-date and complete.