Chapter 4
All research methods begin with the gathering of information or data available on a given topic or problem. This gathering of information is called secondary research, and although extremely important, may be the most overlooked of the public relations research methodologies available to professionals. Why? In part it may stem from the 24/7 nature of public relations—seldom, it would seem, do public relations professionals have the necessary lead time to conduct systematic secondary research on some topic. Second, a great many public relations professionals have little academic coursework in public relations, and a great many have never attempted a research methods course (the perception is that public relations is focused primarily on getting the message out, not evaluating its impact—a perception that by now should be understood as wrong and backward). And, finally, public relations only recently began trying to connect public relations programming to business goals and objectives.
Interestingly, if one looks at public relations budgets as compared to advertising and marketing budgets, very little money by comparison is provided for research; most of that is allocated to the actual collection of campaign data.1 This, then, begs the question: What do we know about a given topic and where can we gather existing information or data on a particular outcome variable of interest—and how can we do it as cheaply as possible? The answer: Use existing information and data available from the client, from within the organization, or from public documentation found in a variety of places. This is not say that these data will be free, but they will be less expensive than if gathered by one of the methodologies in the following chapters, and they certainly will drive the questions asked when collecting data using other methodologies.
Understanding Secondary Research
Secondary research is the gathering and analyzing of information and data that have already been published in some manner or reside in personal “libraries.” Secondary research, then, takes a “second look” at information and data relevant to a particular goal or objective.2 This information often leads to the creation of personal libraries that contain source material that the public relations professional—or any professional for that matter—will go back to time and time again. Perhaps this volume will become part of the reader’s personal library.
What constitutes a library? Materials found in libraries are generally classified into five categories: books, periodicals (newspapers, magazines, professional journals, academic journals), unpublished papers, videos/films, and databases. At one time all would be physically present in the library. In today’s libraries, however, with the exception of books (and this is changing as more and more books—like this one—are being “published” as electronic copy), a great majority of these are now filed away electronically. Historically a library was a physical location; today a library can be a physical location, but it can also be accessible to researchers through the Internet or it may reside on special websites.
Case Studies as a Special Category
Case studies, which can include any of the five information sources, are an important element of the public relations library. According to Stacks, public relations case studies can take two forms: historical and strategic (grounded).3 The most common case study is the historical case. The historical case study presents a campaign or research program in a linear fashion, from beginning to end. It assesses what the problem was, the background research, the objectives set, the communication plan put in place, and the evaluation of that plan. The strategic case study focuses on strategy and is modeled after case studies found in business. It provides a case history, but does not provide a complete history; instead, it asks the reader to make strategic decisions and then produces what is often called a “teaching note,” which evaluates what was done and why it worked or did not. There is a push in the profession for more strategic case studies as public relations professionals are being asked more and more for strategic input into business decisions. Public relations case studies using both the historical and strategic approaches can be found on the Arthur W. Page Society and Public Relations Society of America websites.
Secondary research requires that the researcher have access to it. Although this may sound obvious, many people do not know where to start when they begin to study a problem or are given an objective. Sources of information play an important role in the research program, are located in many places, come in at least three types, have their own validity and reliability, and can be analyzed qualitatively and quantitatively. In this chapter we will examine each not only as part of the methodology, but also as a way of determining which methodologies should be used when collecting data.
Planning to Conduct Research
Although it seems self-evident, planning to conduct research is often an overlooked part of the research program. As noted in Chapter 2, the developmental stage of a campaign requires considerable research to define goals and objectives, establish outcomes relevant to those goals and objectives, understand what has been done previously to inform the decision-making process, and choose the appropriate methodologies and analytical approaches. Secondary research is also a vital methodology for choosing measurement systems.
Perhaps one of the most important tasks completed in the development phase of a research program is the stating of questions that will be answered and inform the active research. What secondary research does is provide the researcher with the information necessary to answer four questions relevant to all research programs. These questions are typically considered in order, as one informs the next; however, each is singularly important in a research program. We will quickly review the four questions discussed in detail in Chapter 1: questions of definition, questions of fact, questions of value, and questions of policy.4
Questions of Definition
Since the concern of public relations is to establish that the outcomes of a campaign have met targeted objectives and that these outcomes also correlate to business objectives, it is important that the outcomes be evaluated by data from valid and reliable measures. As noted in Chapter 3, measurement, especially measurement that focuses on nonfinancial indicators of success or failure, is heavily dependent on definition. How the outcome variables are defined creates the base for all future research—whether that research focuses on methods or measurement. For instance, what is “trust” in relation to the public relations campaign? Is it the outcome variable (hence it is affected by strategies that target credibility, confidence, reputation, and relationship) or is it one of the variables that public relations focuses on to change perceptions about the product? Further, there are numerous definitions of “trust,” just as there are for other nonfinancial indicator variables; some are already defined (termed “reportative,” such as those found in the Dictionary of Public Relations Research and Measurement or Communication Research Measures: A Sourcebook5), while others are defined or “stipulated to” in relation to the campaign. Thus “trust” may be defined differently than commonly defined as reflected in the needs of the campaign. Secondary research helps to establish which definitions are relevant to the campaign and which need to be stipulated.
Questions of Fact
Questions of fact follow from the definitions and seek to establish whether the outcome variables do indeed exist. For financial and physical variables, answering a question of fact is quite easy—observe whether the variable exists. For nonfinancial and social variables, the question is not as straightforward or easy to answer. Nonfinancial variables are indicators that, by definition, cannot be “seen.” They can be inferred through the measurement of attitudes and beliefs, but their correlation to actual behavior is never 100%. Hence measurement enters into the answer—and measurement instrument or scale reliability and validity are paramount. If you have a reliable and valid measure of the concept of “trust” and it correlates well with other demonstrated measures, then you have answered the question of fact. Secondary research provides a history of measures and their reliability and validity as well as cases from previous campaigns to evaluate for effectiveness in measuring what is intended to measure.
Questions of Value
Not all research is factually oriented. Sometimes a researcher is interested in how good or sufficient an outcome is. This involves answering what has been called a “qualitative” question, where the value of something is being evaluated. It may be that the research is more interested in the quality of the responses rather than the number of responses; what was left out of a news release, not what was in it; or the quality of the relationship between important opinion leaders and a product or company. Questions of value answer such questions, and secondary research provides the researcher with an understanding of the communication’s value based on previous campaigns and cases.
Questions of Policy
What happens when you kick off a campaign? What information can the researcher find that indicates strategies were appropriate for the problem or client? Questions of policy answer the “should” question: Should the measurement scale be adapted because of audience differences not identified in the developmental stage? Should a different data gathering methodology be employed? Should a different sampling technique be used? Should there be more planned assessment of tactics as the campaign develops? Answers to questions like these are found in being able to look back and evaluate previous research through cases histories and other campaigns. Questions of policy are not typically answered until after the campaign is over and a complete campaign evaluation undertaken; however, secondary research can help answer such questions in advance of the campaign and in its developmental stage.
Benchmarking
It should be clear that secondary research is required when developing the research program supporting a public relations campaign. What secondary research ultimately provides the researcher is a benchmark or series of benchmarks for later comparison during the campaign. It also helps to plan for assessment during the campaign: What methodologies should be used? How many times should data be gathered? What policies should be in place if analysis indicates that objectives are not being met? Too few public relations campaigns actually set benchmarks and then test against them, even though such testing provides a continuous evaluation of a campaign from kickoff to completion. Further, many public relations campaigns cannot address the question of effectiveness because they fail to establish initial benchmarks—baselines—against which to compare results at the campaign’s end.
Information Types
What types of information does secondary research seek? In general, there are three types that differ in terms of their authenticity and reliability. All deal with the source of the information. Primary sources of information are the actual documents, journal articles, books, news reports, videos, and so forth as produced or printed. They are it and there is no question as to interpretation, as they come from the individual or organization that created them. Sometimes primary sources are not available for a number of reasons and we have to rely on secondary sources, or reports of the primary source through the eyes of someone else. Without access to the primary source, the researcher needs to ensure that what the secondary source is reporting is what was actually in the primary source. This is often done through cross-referencing of several secondary sources, ensuring that the reporting is reliable and also providing evidence of any bias in the secondary reporting. Finally, there are tertiary sources, which are reports of the secondary source. There are times, even with today’s Internet accessibility to many information sources, when primary and secondary sources are simply not available. The primary and secondary sources may be confidential reports, may be in restricted access sites, or may simply no longer exist and the researcher is left with a report of a report of a report.
Clearly the goal of anyone conducting secondary research is to gain access to primary sources. When primary sources are not available, secondary sources should be approached with caution and tested for reliability of reporting. Tertiary sources should be examined but rarely used; however, they may provide new avenues to a problem or help if working through definitional problems.
Information Sources
Where does a researcher turn when searching for information while conducting secondary research? Traditionally the answer to that question was easy and clear-cut: the library, or actually “libraries.” Today the researcher has a myriad of sources to select from and access to more sources than ever before. This creates a great opportunity, but also is fraught with problems, mainly the ability to establish the reliability and validity of information. Compare a traditional source for beginning secondary research when little is known about the campaign object: a printed encyclopedia such as the Encyclopedia Britannica and the electronic encyclopedia, Wikipedia. Both are accessible electronically, but only Britannica is truly “peer reviewed” (has editors who review the entries for factual errors), while Wikipedia allows users to add new entries and edit older entries. Wikipedia’s advantage of being updated as events change is offset by entries that have been added to promote a particular point of view or are truly wrong. The problem is that with Wikipedia, the researcher cannot gauge the accuracy or intent of the entry. In all fairness, Wikipedia has tried several workarounds to these problems, but still the researcher can never be sure that the content is accurate, unbiased, or valid. In spite of this, Wikipedia can be an excellent starting point, especially when trying to understand a product or client from a contemporary and “up-to-date” perspective.
Traditional Sources
The traditional source for secondary research is the physical library, of which there are two major types: public and private. Public libraries are found in almost every city and town in the United States, are open to all, and are the repository of books, magazines, and newspapers (local, national, and sometimes international). There are also libraries that are not open to all; these can usually be found on university and college campuses. These libraries are generally reserved for faculty, students, and other researchers who are given access for “serious” research. They are generally research-oriented but often contain many of the same books, magazines, and newspapers the public library does. In addition, they may have special collections associated only with that particular library.
Private libraries severely restrict access to their holdings. Private universities and colleges often have large holdings, but permission is required for the general public to access them. Still, they are often more accessible than other private libraries, including professional associations, corporations, and personal libraries. Trade association libraries often contain reports and data on a particular sector of the economy and are generally available with little difficulty, but mainly for paid members. Organizational and corporate libraries are much more difficult to access, and access may even be limited within the company to specific people. Many contain confidential reports and sensitive data that the organization does not want made public. Public relations professionals who work for companies should have access to corporate libraries and those working with public relations agencies should have access to information relevant to the problem they have been hired to work on. The final type of library is the personal library. As the name suggests, this is the library of an individual and is usually specific to that individual’s interests. Most academics have their own libraries of books and academic and professional journals to aid them with their research. All professionals should build their own working libraries to refer to as they gain expertise in particular areas or for general source material (this book, and others in this series, should be in a personal library).
The preceding discussion should not leave the impression that libraries are no different than they were in the Middle Ages. The modern library is a true repository of information, and almost all libraries are highly interconnected via the Internet so that public access is available without actually ever setting foot in a physical library. What makes the traditional library important, however, is that someone has made decisions on what information is available and checked it out for reliability and validity.
Contemporary Sources
The Internet has opened up secondary research, giving researchers unparalleled access to all types and sources of information. This access, however, does not mean that all sources are valid or reliable. As more and more reputable groups put their information on the Internet, validity and reliability concerns should diminish. Most of what researchers find on the Internet initially comes from search engines, and search engine optimization (SEO) and search engine marketing (SEM) have become hot topics. SEO deals with Internet searches to websites via search engines through unpaid searches; SEM deals with paid Internet searches.6 Regardless of which path a researcher chooses, working with search engines is no different than using the Dewey decimal system 20 years ago.
Search Engines
Using a search engine takes some thought. A simple search engine query using Google, Yahoo!, and Bing found more than 100,000 possible results for the phrase “public relations.” This did not include “PR” as an acronym, which would have yielded even more sites. To use a search engine effectively, the researcher must know exactly what she is looking for—meaning that she has carefully defined what it is she is seeking. Second, it is possible to refine the search using “Boolean operators,” terms that help to limit or expand searches. Phrases such as “and,” “or,” “not,” and “else” help the search engine expand or reduce the search. Although not Boolean operators specifically, the use of quotation marks also serves to provide input to the search engine, thus further refining the search. Thus using “public” and “relations” would yield many more than 100,000 results. Also, some search engines allow one to search for specific terms within so many words of each other. For instance, searching for all references to George Walker Bush, knowing they may be “George W. Bush” or “George Bush,” helps to refine the search. Third, the results that come up on the computer screen are not always listed by relevance, importance, or even number of times accessed. Companies can pay to have their websites listed early, and if you look at the right side of any search, you will find particular sites that are paid as advertisers. Knowing this and how Boolean operators work makes searching much more efficient.
Although there are many search engines available, some are more central to the secondary research process in that they may be more specialized. For instance, using Bing, Google, or Yahoo! is like using a dictionary or encyclopedia. Factiva and LexisNexis are more media-oriented search engines, as are the search engines associated with major newspapers, such as the New York Times Wall Street Journal, and Washington Post, to name but a few “national” newspapers with search engines that allow researchers to access articles from what used to be physical news “morgues.” If you want to search for case studies of public relations, advertising, or marketing, you can access Warc (http://www.warc.com), a site case study website. More specific search engines, such as those searching for medical terms or advice, might include WebMD. Legal questions and findings can be found in Westlaw, if the researcher has access to it. And finally, anyone wanting to search for U.S. government information can use LexisNexis and at least two U.S. government sites: archives.gov or gpoaccess.gov.
Finally, given the popularity of social media—blogs and tweets—search engines have been developed for social media communications. Three top-line search engines are SocialMention, WhosTalkin, and TopRank, with more being developed by the research departments of various public relations agencies.
Databases
Where have search engines come from? Years ago when you wanted to search online for something you went to a database, such as the 1970s HUMRO, which provided one- or two-line notations for mainly unpublished documents that were searched via a series of keywords, such as Shakespeare or Abraham Lincoln. Public relations–specific databases that have developed into powerful websites include the previously mentioned LexisNexis, PR Newswire, Business Wire, Cision, Medialink, and ABI/INFORM.
Assessing Source Reliability and Validity
As noted earlier, much of today’s information can be found on the Internet. While the Internet has made access to information easier than ever, it has also made establishing that information’s reliability and validity more difficult. Secondary researchers should always be wary of information that comes from the Internet, especially when the website sponsor is not known, is not listed, or has no contact information listed. Assuming that the information appears to come from a credible source, the secondary researcher must then establish its validity and reliability in three subjective ways: content, authority, and through established critical standards.7 Assessing content focuses on answering the following questions positively:
- Does the content deal with what you need?
- Does the content match what you already know?
Assessing authority focuses on answering the following questions:
- Who actually wrote the material? What is his/her credibility in the area?
- Has the material been subjected to editing, fact checking for accuracy, or been submitted to a panel of judges for review prior to publishing?
- Is the source of the information clearly stated and contact information provided?
Assessing by critical standards is more difficult and comes after the information has been read. McCormick suggests the following five questions be answered:8
- Are the main points and issues clearly identified?
- Are the underlying assumptions or arguments generally acceptable?
- Is evidence presented adequate, evaluated clearly, and supportive of the conclusions?
- Is there bias, and is that bias addressed?
- Is it well written and/or edited?
A second way to assess secondary sources that include data, such as those found in professional associations or governmental sources, is to actually conduct statistical tests on those data. As Hocking, Stacks, and McDermott point out, there are statistical techniques that can be used on summarized or aggregated data, and if the actual data sets are available, researchers can run confirmatory tests on the data and compare results to those that are published and interpreted.9
Secondary Research in Measurement and Evaluation
As noted in Chapter 3 and earlier in this chapter, secondary research is particularly important in creating measurement instruments to assess nonfinancial outcomes, collecting data to check against benchmarks, and evaluating communication program success. From a best practices approach, secondary research is essential in carrying out and evaluating a communication program or campaign. It begins with an ability to understand clients’ and products’ past experiences, critically evaluate similar programs through case studies, and find information that can be used as benchmarks, or if not available, to go into the field to collect that information. Best practices secondary research also dictates what type of data (qualitative, quantitative, or both) should be collected and how. Further, it establishes outcome expectations along the campaign timeline that can be tested against or sets critical benchmarks for testing. Finally, secondary research provides the research program with the necessary background against which to conduct final evaluations and to identify and correlate against other business objectives, including advertising and marketing.
Secondary Research Case
Media Assessment of Saudi Arabia’s Reputation and Foreign Perceptions Between September 1, 2007, and November 9, 200710
As part of opening up the Kingdom of Saudi Arabia to the outside world, the Saudis admitted a large number of journalists to cover the Saudi-hosted 2007 Organization of the Petroleum Exporting Countries (OPEC) summit in Riyadh. The Saudi Ministry of Foreign Affairs wanted an international perspective on what the journalists might cover and what effect OPEC’s recent oil price increases had on OPEC’s reputation and from that prepare anticipatory pointers on how the media agenda might play out.
Echo Research was engaged to conduct secondary research on OPEC, the Kingdom of Saudi Arabia, and worldwide perceptions of both. This secondary research was necessary to prepare Echo’s team to establish current perceptions of OPEC and Saudi Arabia, as well as to identify regions of the world, types of media to follow, and potential coding problems. It also sought to identify key opinion leaders and leading analyst companies. The goal was to present the Saudis with strategic recommendations for media relationships and key messaging strategies. The study’s objectives were to be able to identify foreign journalists’ key concerns, areas where the Saudis and OPEC could expect questions, and potential story areas relating not only to oil but also to stories driven by outside perceptions of Saudi culture.
The study analyzed 584 stories obtained from the general media and social media citizen blogs from the United States, United Kingdom, France, Germany, Spain, Italy, and China—most of the leading economic powers and countries with major interests in OPEC and Saudi oil. This required that the coding of media extracts be done in multiple languages, which added a complication in terms of understanding the meaning of words and phrases. An analysis of secondary data suggested that several areas would be examined, including perceptions of Saudi business, society, government, human rights abuses, and sponsorship of terrorism. Second, they examined perceptions of OPEC, including oil supplies, oil-based economics, and concerns about Middle East stability. In addition, a large “dictionary” of terminology across languages was set up for coder training.
Based on the secondary research, the study produced a number of findings and suggestions. The report found a number of assets and liabilities as seen through the media lens; it identified what it labeled “big, bad issues” that journalists would arrive with and which could drive coverage. This included fringe stories—stories not related to oil—about societal differences and potential misunderstandings based on such factors as treatment of women and a lack of freedom of the press. The study also assessed OPEC from opinion leader analysis and analyst reports, yielding 12 key factors cited for the increase in oil prices.
The study also identified “clusters” of journalists and their reporting on key issues for both Saudi Arabia and OPEC. The clusters were defined by topic and message tone (positive, negative, neutral).
Finally, based on the secondary research’s findings, recommendations were made regarding key messaging strategies and strategic media management.
Secondary research played a key role in this study by establishing expectations for events and providing key messaging strategies and media recommendations. Through a thorough search of the existing literature and media stories, Echo was well prepared to conduct the actual gathering of data and subsequent media analysis.
Summary
All public relations practice should begin with secondary research. Best practices require secondary research be as complete as possible. Many times, however, secondary research is overlooked as a major research methodology. Past public relations practices, which did not set benchmarks against which to test for success or failure of public relations efforts, made secondary research seem an afterthought. The 24/7 nature of reactive public relations—of tidying up or fixing a crisis—may also figure into the historical lack of secondary research. It should be noted that advertising and marketing programs make secondary research an essential part of their research programs. Contemporary public relations, from a strategic or proactive approach, requires that professionals have an understanding of previous research, competitors, and expected business goals and objectives, and continually add to their personal, corporate, and agency libraries secondary materials that can be easily accessed. Contemporary public relations has made this a necessary requirement for best practices.