Chapter 11. Gathering Citizen Data, Input, and Feedback

Marketing research as a tool for decision making is not usually questioned as much for its benefits as it is for its costs. You may have witnessed or even participated in tense debates reflected in the following comments aimed at discouraging a marketing research endeavor:

• By the time we get the survey designed and through the approval process, we will have missed the deadline to use the funds allocated for the campaign in the first place!

• The $35,000 you want to spend on research to determine who’s littering and where and when and why will have to come out of the media budget. That could mean as many as ten fewer billboards carrying our anti-litter message.

• The last time I was involved in a study like this, the report that came back had two inches of computer tables, twenty five pages of explanation about statistical technique, and a three-page summary of findings. Where’s the beef?

As you read this opening story, consider the high opportunity cost that would have been experienced if project managers had let challenges such as these get them down.

Opening Story: Building a Healthy Nation through Research in South Africa

Black women are a strategically important market for tobacco companies in South Africa, comprising over 39 percent of the population and constituting a key market for fast-moving consumer products like cigarettes and snuff. Traditionally, tobacco use has been perceived to be taboo for these women, and those who use it do so secretly or only with trusted others. Fortunately, as a result, black African women have had some of the lowest rates of smoking in South Africa. Unfortunately, tobacco marketers understand this and aim to change these societal norms.

Challenges

This targeting of black African women is particularly disturbing not only because of the diseases and early death tobacco use produces but also because these women have a significant influence on the consumption behavior of their children, family, and community members. This is compounded by the harmful effects on other household members from exposure to secondhand tobacco smoke and the increased incidence of tobacco-related harm for children during their mothers’ pregnancies.

The challenge and balancing act program planners faced was to help female smokers and snuff users quit and, with the same precious resources, combat temptations among nonusers to slip into a smoking lifestyle. Interventions needed to be carefully targeted—right on.

The solution employed was a rigorous marketing research effort to understand the target markets—better than the competition did.

Strategies

A research project in 1999 surveyed 1,314 black Xhosa-speaking women ages 15 to 64 living in the Cape Town metropolitan area to study the knowledge, attitudes, practices, and behaviors of this audience regarding tobacco use. This effort, also known as a KAPB study, was designed to interview women who were smokers and/or snuff users and to compare findings with nonusers. A quarter (25 percent) of those interviewed were smokers, 27 percent were snuff users, 2 percent used both cigarettes and snuff, and just under half (46 percent) did not use tobacco.

Findings revealed that tobacco marketing had indeed been successful in reaching black women, with the majority of respondents reporting personal exposure to tobacco marketing. A third of all the women had recently seen cigarette advertisements, with even more (42 percent) of smokers reporting this. Three-quarters had purchased cigarettes as children on behalf of an adult, and the vast majority knew the current price of a single cigarette. All women had a worrying lack of awareness of the health hazards of tobacco use.

Understanding Nonusers at Risk: More than half (58 percent) of women who did not use tobacco were at risk of converting. Five percent were found to be pro-tobacco in orientation and on the verge of smoking. These women had a more urban and less traditional identity, were less sensitive to others’ disapproval, were more exposed to cigarette advertising, had female friends who were smokers, and were more hedonistic and less health-oriented than the other women nonusers. Another 53 percent were found to have only a shallow commitment to remaining non-smokers. They were the most economically disadvantaged among the nonusers and in the midst of social transition. They were significantly desensitized to social taboos against black women smoking and were surrounded by smokers in their families and among female acquaintances.

Understanding How to Help Users: Tobacco users were found to be somewhat older and less educated. They were much more likely to be surrounded by other smokers such as husbands or partners and other family members who smoked. Over half the smokers said that they had smoked during pregnancy, and over a third of the snuff users said they took snuff while pregnant. Conservative estimates indicate that, on average, the women who were smokers spent at least 10 percent of their disposable income on their own cigarettes. Over half of their close female friends were smokers, and they had strong rationales for why tobacco use was beneficial—insights key to intervention messages. They believed smoking could help them lose weight. Snuff users thought that snuff had medicinal value, including pain relief.

Rewards

This rigorous research effort led to clear findings and recommendations.

It reinforced current tobacco-control legislation in South Africa, helping to ensure that regulations would be implemented, including banning advertising and smoking in public places as well as increased monitoring of sales to minors.

It led to recommendations that new efforts should be undertaken to develop strategies to counter smokers’ beliefs about the benefits of smoking, increase awareness of health risks, and enable tobacco users to identify and understand which of their own health problems are related to their tobacco usage.

It provided insight that efforts need to delink smoking from modernity and a progressive orientation and link being tobacco-free with the things black women value most, whether it is personal dignity, family welfare, upward mobility, or access to personal and social development.¹

Marketing Research in the Public Sector

Marketing research defined formally is “the systematic design, collection, analysis, and reporting of data relevant to a specific marketing situation facing an organization.”² Those among you who use citizen data, input, and feedback for marketing decision making and evaluation on a regular basis probably can’t even imagine how you could be effective without it. Those among you who have not will be amazed at how much easier this tool makes it for you to make decisions, obtain approvals, and report results.

Data on citizen behaviors and profiles can be critical in the marketing planning process. A utility, for example, will benefit from detailed census data on neighborhoods where it plans to work with single-family household residents to take steps to reduce rainwater runoff from roofs into sewers and drains. A Department of Transportation will count on using data on causes of automobile accidents when deciding among several options for reducing traffic-related injuries and deaths. And the Office of National Drug Control Policy will no doubt need to utilize data regarding current major distribution channels for drugs to develop effective intervention plans.

Citizen input will be especially insightful when developing the strategic marketing mix (4Ps). It can help you to determine the features of products, programs, and services (e.g., when developing a strategic plan for a school district), decide what incentives or disincentives will be most persuasive (e.g., what it will take to curtail tax evasion), choose among options for distribution channels and citizen access (e.g., when considering Sunday hours for liquor stores), and it will be crucial when developing campaign messages, choosing messengers, and selecting media channels. And it can help avoid producing and launching products and programs not likely to be a hit with citizens (e.g., the Susan B. Anthony $1 coin that looked like a quarter and had no special section in cash register drawers).

Feedback from citizens is a core component of program and campaign evaluation and is often used to assist in assessing satisfaction with programs and services, recall of and reaction to campaign messages and ways to do better “the next time” (e.g., Medicare drug bill in the U.S.), or when a campaign is rolled out.

This chapter presents a summary of marketing research terminology, primarily serving as a reference point for a common language, one that will assist you in future discussions with colleagues and contractors. It may also serve as a starting point in decision making regarding the appropriateness of a variety of potential research methodologies.

Research Defined by When It Is Conducted

Marketing research terminology sometimes refers to when research is conducted and used in the planning process. Three such terms are common: formative research, pretesting, and monitoring and evaluation.³

Formative Research

As the label implies, formative research refers to research that is used to help form strategies, especially understanding a target audience and developing the marketing mix. In March 2005, the U.S. Census Bureau, for example, promoted on their American FactFinder Web site an opportunity for users to participate in an online usability study to gain citizen ideas on how to improve the Web site.⁴ This information could then be used to inform a variety of product improvement efforts such as determining what additional data citizens wanted to be able to access on the site, improving navigation, and meeting additional needs for producing custom reports. In social marketing efforts, formative research most often focuses on identifying audience barriers and motivators to behavior change (e.g., reasons why citizens might not want to have their children walk to school one day a week and what might assuage their concerns).

Pretest Research

This research at a subsequent phase in the planning process is conducted to test draft strategies and tactics prior to production and implementation. Most commonly, it is used to help choose among a short list of alternative strategies (e.g., which of three slogans for Homeland Security are most likely to resonate with citizens). It can help assure you (or not) that chosen strategies and tactics have no major flaws (e.g., discovering that a recommendation from a health department to exercise one hour a day, seven days a week raised eyebrows as to whether officials were “out of touch with reality”). It is particularly useful in determining whether a certain approach will reach and influence your particular target market (e.g., whether the bookmobile hours and locations being considered will appeal to seniors).

Monitoring and Evaluation

After a program or campaign has been launched, marketing research can be used to gauge performance relative to your goals and objectives. When this research is conducted while the campaign is still underway, it is often referred to as monitoring, and it helps you to make course corrections “midstream.” After a project is completed, research can be conducted to provide a final assessment of the effort and is then referred to as evaluation.⁵ (Chapter 12, “Monitoring and Evaluating Performance,” provides more detail on monitoring and evaluation techniques.)

Research Defined by Source of Information

Information to inform your campaign is typically found in one of two ways. It may already exist somewhere, having been collected at another time for another purpose, in which case it is referred to as a secondary research source. Primary research, on the other hand, refers to data gathered for a specific research project for the first time.

Secondary Research

It is wise to start your research investigation by examining some of the rich variety of secondary data that is most likely currently available to determine whether you can glean enough insights and information without conducting more costly primary research. Secondary data certainly can provide a starting point and includes specific sources such as your agency’s internal records, databases, prior surveys, and even anecdotal comments (e.g., reports that a customer service department provides on types of calls received). Looking outside the organization, a rigorous search and literature review can be conducted for journal articles and surveys conducted by other organizations with your target audience and/or relative to your specific issue.

Many cities across the U.S., for example, are interested in increasing commercial recycling rates. A program manager at a city utility faced with this challenge might begin this journey by reviewing internal data regarding business customers and their current recycling rates and utilization of city recycling services and how this varies by type of business (i.e., retail vs. manufacturing). Next, a search would be undertaken to identify journal articles and surveys conducted by others that provide insight into barriers that businesses have to recycling and experiences that others have had with strategies such as education, new services, and offering incentives and disincentives.

Primary Research

When the needed data does not exist or is outdated, inaccurate, incomplete, or unreliable, you’ll want to consider the potential benefits and costs of conducting a primary research project.⁶

That city manager interested in increased commercial recycling rates has a variety of primary research options to learn more about the customer. He or she could ride along with waste management collectors and observe the actual locations and types of dumpsters being used, conduct in-person interviews with supervisors to explore barriers and solutions, commission anonymous surveys with owners to get candid information, and then follow this up with a focus group to pretest specific strategies. He or she will weigh these options and their potential costs against insights they will provide and the value placed on recyclable materials diverted from landfill.

Research Defined by Technique

Primary data can be collected using a variety of techniques. The following brief descriptions and examples simply introduce the range and nature of tools available and can serve as a reference point for further exploration.

Observational Research—Primary data is gathered by observing target markets in action. This technique can be used to enrich understanding of your audience (e.g., watching what warehouse employees do with leftover cardboard packaging), or it might be used as a means to evaluate campaign efforts (e.g., recording the number of drivers wearing seat belts before and after a campaign in a specific geographic area).

Ethnographic Research—This technique often includes observation as well as face-to-face interviews with study participants. As an anthropological discipline, it is considered a holistic research method, founded in the idea that to truly understand target markets, the researcher will need an extensive immersion in their natural environment. For example, a researcher might spend several days on a farm observing farmers’ activities, seeking to understand what it would take to influence them to collect and cover manure piles to protect water quality.

Experimental Research—As one of the most scientifically valid research techniques, the purpose of this approach is to capture cause-and-effect relationships by selecting similar groups of subjects, exposing them to different treatments, controlling extraneous variables, and then checking whether response differences are statistically significant.⁷ This approach might assist border patrols, for example, in deciding whether kiosks actually reduced or increased processing and wait time.

Behavioral Data—Data related to actual citizen behaviors is collected and analyzed, as in the case where data on taxpayers is used to understand differences between those submitting their forms and payment online versus by mail, in an effort to increase online utilization.

Focus Group Research—This popular methodology provides useful insights into customers’ thoughts, feelings, and even recommendations on potential strategies and ideas for future efforts. Thought of by some as a group interview, a focus group typically involves inviting eight to ten people to gather for a couple hours with a trained moderator who uses a discussion guide that focuses the discussion—hence the name “focus group.” Participants are carefully selected based on certain demographic, psychographic, or other considerations and are often paid a small sum for attending. This technique would be useful, for example, for a school district interested in understanding more about how retired citizens would like to be involved in classroom volunteering.

Mail Surveys—This survey technique offers the advantage of collecting large amounts of information at a low cost per respondent and is sometimes the best way to reach people not wanting to participate in personal, less anonymous interviews. Its disadvantages are that response rates are often very low and the researcher has little control over the quality of the mailing list sample (i.e., who fills out the survey at the mailing address). This technique might be appropriate and useful, for example, for providing ideas on what programs residents want most in a new community center.

Telephone Surveys—This technique is one of the best methods for gathering information quickly and has higher response rates than most mail questionnaires. Interviewers can explain difficult questions and even skip ones that don’t apply. On the downside, costs per respondent are higher. The Centers for Disease Control and Prevention utilize telephone surveys, for example, to track national trends in health-related behaviors such as levels of physical activity.

Online/Internet Surveys—This technique is the fastest-growing option, providing opportunities to send surveys through email, include a questionnaire on a Web site, introduce questions to a chat room or bulletin board, hold an online focus group, and learn about individuals by following how they clickstream through your Web site and move to others.⁸ Although it can be inexpensive, faster, and more versatile than some techniques, it can be prone to technological problems, and samples can be small and skewed.

In-person Individual Interviews—This most flexible but expensive method typically involves an arranged interview where the respondent is often contacted for an appointment and sometimes paid a small amount for their time. It offers the advantages of a focus group such as being able to show participants sample ads, but it also removes any concerns for group influence and provides a greater opportunity for in-depth questioning.

Intercept Interviews—Sometimes individual interviews are not prearranged and involve stopping people in places such as shopping malls, street corners, airports, or post office lobbies. Though less expensive, respondents may be more hurried, and extra care needs to be taken so that the sample is not biased in some way (e.g., interviewing people who look more approachable). A state department of business and economic development might make use of this tool at an airport gate, for example, to understand the demographic profile of visitors and explore their purposes for visiting and related behaviors such as shopping and entertainment while visiting.

Mystery Shopping—Often used as a technique to measure customer satisfaction, testers pose as customers or potential customers and report on strong and weak points experienced when interfacing with an agency’s personnel, applying for programs and services, or purchasing products.

Finally, you may also hear the terms “qualitative” and “quantitative” mentioned in research discussions. Qualitative techniques refer to studies where samples are typically small and are not projectable to the greater population, with a focus on identifying and seeking clarity on issues. You won’t be disappointed with this strategy if you and others are clear that it is a powerful technique for understanding issues in more depth and do not expect to use it when you need quantifiable and scientifically representative findings. Focus groups, personal interviews, observation, and ethnographic studies are often qualitative in nature. Quantitative tools, on the other hand, are used when you need hard counts, representative samples, and the ability to project results to larger populations with statistical reliability. This is when telephone, mail, and online interviews are more typical.

Steps in the Research Process

Nine steps are traditionally followed in a primary research project. As you will read, you begin with the end in mind.

#1: Determine Purpose—What decisions will this help us make? Why are we doing this?

Primary research should be conducted when it will help you make better decisions than the ones you would make without the research. A community center interested in offering (and filling) exercise classes for seniors, for example, will need to make smart decisions regarding length of class, time of day offered, type of instructor, preferred type of exercises, what to charge for the class, a name for the class, key messages to be used in communications, and ways to get the word out.

#2: Identify Informational Objectives—What specific information do we need to make this decision or answer the questions we have?

This step gets you closer to determining questions you will want to ask your target audience. For the senior exercise class, for example, program managers would benefit from knowing specifics such as what would motivate seniors to attend the class and what might keep them from enrolling. They would also want to test which of several pricing options, times of day for the class, instructor qualifications, and types of exercises are most appealing. They would also want to test several potential names, key messages, and media channels.

#3: Determine Audience—Who do we need the information from to answer the questions?

At this point, the target population that will be sampled is defined. Having articulated responses to steps one and two in this process, the target audience may seem obvious. The community center, for example, will want to conduct research with seniors in their community they have identified as the most likely to enroll in such a class, and they may be described by age, income, gender, ethnicity, physical abilities, benefits sought and/or related behaviors (e.g., already attend some other event at the community center).

#4: Select Research Technique—What is the most efficient and effective way to gather this information?

As presented in prior sections of this chapter, a variety of techniques are available. A specific method should be selected that will best meet the informational objectives at the lowest possible cost. One or more techniques will be chosen, based on whether results need to be statistically reliable and projectable to larger populations. For the community center, a focus group to identify issues and a telephone or mail survey to provide projectable findings would most likely help make decisions regarding the features, pricing, hours, and communications for the class.

#5: Develop Sample Plan—How many respondents should we survey? Where do we get names? How do we select (draw) our sample so that our results are representative of our target market?

A sample is the segment of the population you select for marketing research purposes to represent (at varying levels of reliability) the population you are studying. Clearly, large samples give more reliable results than small ones, although it is not necessary to sample the entire target market or even a large portion to get projectable results. An important decision at this step is how the sample is chosen. In a probability sample, each population member has a known chance of being included in the sample, and researchers can calculate confidence limits for sampling error. When this technique costs too much or takes too much time, a nonprobability sample is used, even though the sampling error cannot be measured. Five major types of samples are described in Table 11.1, with examples referring back to the community center interested in offering senior exercise classes.⁹

Table 11.1. Types of Samples¹⁰

#6: Pretest Draft Instruments—When tested with a small sample of respondents, how well does the instrument work?

A research instrument is now developed (e.g., a draft telephone survey or self-administered questionnaire) and pretested with a small sample of intended respondents. Of special interest are whether questions make sense, responses can be easily recorded, skip patterns are smooth, and the length is as intended. Is the introduction or explanation of the survey successful in engaging potential respondents, or does it instead turn them away or cause them to hang up? Appropriate revisions are then made. For the community center, the questionnaire used for the telephone survey could be tested with a sample of four to six respondents and then revised if needed.

#7: Field the Research—Who will conduct the research, and what is the timing?

You are now ready to collect the data. Options for implementation range from hiring a marketing research firm to doing it yourself. A fielding plan should be developed, outlining milestones such as timing for a targeted number of completed interviews, recruitment timeframes, dates for focus group discussions, and desired completion date.

#8: Analyze Data—How will data be analyzed and by whom to meet the needs of planners and other audiences?

Each methodology (technique) suggests its own type of analysis. Qualitative research (e.g., focus groups) is analyzed more in terms of issues that participants raised, preferences that they expressed, and ideas that they generated. Caution is often needed to refrain from quantifying or projecting these findings, such as concluding that half the participants loved Option A for the class name and only a fourth preferred Option B. A more appropriate analysis in this case is why they preferred the options they did. On the other hand, quantitative analysis of large sample surveys focuses on more statistical techniques and decision models to interpret and represent the findings and often requires professional research assistance.

#9: Write a Report and Present Recommendations—What information should be included in the report, and what format should be used for reporting?

Reports should be designed with the audience in mind. A typical report will begin with a standalone executive summary that leads with a statement of purpose for the research, a brief description of methodology, a summary of findings, and concludes with recommendations relating back to decisions and questions that the research was intended to address. A separate section of the report then presents more detailed findings, tables, and perhaps specific verbatim comments made by respondents. An appendix includes information on the sample and a copy of the survey instruments and any documents shared with respondents.

A Word about Low-Cost Research

Timeframes, price tags, and lack of funding for research projects often stop them in their tracks, especially in the public sector where managers are challenged to show “profitable” rationales for the undertaking. Although clarity of purpose and intended use for the research can assist you in gaining support, consideration should be given to the many inexpensive and quick approaches that, as Andreasen asserts, may not lead to certainty but will at least improve the decision to a degree that justifies the cost.¹¹ Among those mentioned in his book on Marketing Research That Won’t Break the Bank are the use of available data and techniques such as systematic observation, streamlined surveys, and small-scale samples.¹²

Additional options to explore include shared-cost surveys, where you can pay to add a few questions to a survey being conducted by a research firm for a variety of organizations, targeting an audience you are interested in. Additionally, professors and students at universities may be willing to volunteer their assistance because they may find your research proposal of interest and of benefit to their current projects and studies.

Summary

Terminology used in research sometimes refers to when the research is conducted in the planning process (formative, pretest, monitoring, or evaluation), the source of the information (secondary versus primary research), or the technique used to collect the data (observation, ethnographic studies, experimental approaches, behavioral data, focus groups, mail surveys, telephone surveys, online/Internet surveys, in-person interviews, intercept interviews, and mystery shopping). It was further explained that qualitative research refers to studies where samples are typically small and not representative of larger populations. On the other hand, quantitative tools are used when you need hard counts, representative samples, and the ability to project results with statistical reliability. Traditional steps in the planning process were described, beginning with a clear statement as to what decisions the research will support:

#1: Determine Purpose for the Research

#2: Identify Informational Objectives

#3: Determine Audience to be Surveyed

#4: Select Research Technique

#5: Develop Sample Plan

#6: Pretest Draft Instruments

#7: Field the Research

#8: Analyze Data

#9: Write a Report and Present Recommendations