Chapter 8
Quantitative research, by its very nature, is closely associated with numbers and coming to conclusions based on those numbers. When researchers use a quantitative method they take numerical data and interpret them through statistical reasoning. Statistical reasoning can be approached in two ways, depending on the nature of the research and its uses. The first way is to use the numerical data to simply describe variables in the study. This is called descriptive statistical reasoning. Often surveys will report what was found through the research in terms that simply describe how many respondents answered statements or as levels of a particular variable, such as sex (48 females or 48% of the sample as compared to 52 males or 52% of the sample felt the product was a good buy for its price). The statistics simply describe the results. The second way is to use the numerical data to infer differences between levels of a variable. This is called inferential statistical reasoning. Here the numerical data are used to establish the probability that groups of people are truly different. In the preceding example, for instance, is 48 actually smaller than 52? Or is there no real difference between female and male perceptions on that product? Inferential statistical reasoning provides a way to test—to infer—for differences.
This chapter will cover both descriptive and inferential statistical reasoning from a practical perspective. It will not require the understanding of complex statistical formulas, but instead will demonstrate how numerical data can be used to present findings and then interpret those findings in terms of their probability of differences. As such, there is very little mathematics involved. Indeed, statistical analysis, especially today with computerized statistical packages such as IBM® SPSS® Statistics,1 is more about understanding how to read a map and what that map represents. The analogy to a map is appropriate in that statistics provide the directions from which inferences can be made about numerical data, but each “map” is slightly different, just as maps used to travel differ between geographical and political representations of the same territory. Before we turn to descriptive statistical reasoning, a quick review of “data” and how we define and label data is necessary.
Describing Data
In Chapter 3 we noted that data can take many different forms. For statistical purposes, however, data are associated with numbers and “numerical thinking.” Because of this, people often assume that data—numbers—have some direct meaning. From the outset, let us agree that numbers have no meaning in and of themselves, but meaning is interjected into them by people collecting, analyzing, and reporting those numbers. That is, a number is simply an indicator—an often imprecise indicator—that provides us with the ability to make comparisons. In Chapter 3, numbers were defined by the type of data being collected or observed. We noted that there were two major types of data—categorical and continuous—and within each type there were two subclasses.
Categorical and Continuous Data
When employing statistical reasoning, especially when we are attempting to infer differences from data, how the data are initially defined becomes important. As covered in detail in Chapter 3, categorical analyses can result from either nominal or ordinal data—which in the former simply differentiates levels of an object or variable and in the latter proposes an ordering effect for levels of the object or variable. For instance, nominal data for sex would be defined as female or male, with each being equated equally and simply a way to distinguish the levels. However, for ordinal data, such as socioeconomic status, the variables lower class, middle class, and upper class are not only different but also ordered in terms of lowest to highest status. Note, too, that categorical data are always categorical data, even when analyses may appear to be continuous (e.g., percentages). Continuous analyses resulting from interval (e.g., age) or ratio (e.g., monetary data such as dollars or pounds sterling) data put the data on a range or continuum. As noted in Chapter 3, continuous data can be reduced to categorical data, but categorical data cannot be modified to become continuous data.
The first step in statistical reasoning is to understand how the data were defined before being collected. This provides the basic information required to determine which statistic is most appropriate to report and interpret. The second step is to actually compute the statistic by “running” the data either by hand or via a computer.
Using Computer Programs to Calculate Statistics
The computer has made all of us statisticians. This is not necessarily a good thing. A computer can compute any statistic asked for, whether or not it is the appropriate statistic for the data or problem. Further, new statistical packages often “help” in deciding what statistic should be run. The computer is actually only as smart as the user, so understanding what statistic to run should come from the research objectives, which of course reflect the larger public relations and business objectives. Some statistical packages have evolved into large programs that not only run the statistics requested but also have fairly good graphing programs. Other computer programs simply run the statistics requested and provide output for analysis. There are a large number of computerized statistical packages available, including IBM® SPSS® Statistics and SAS (large, comprehensive programs) and Minitab and Statistics With Finesse (smaller, less comprehensive programs).2 In addition, there are analytical packages that combine both the analytics, such as content analysis, and basic statistical analyses. These programs include survey packages such as Zoomerang and SurveyMonkey, as well as dedicated statistical analysis programs such as Mentor and statistical add-on programs that can be used in conjunction with Microsoft Office applications like Excel. Analyse-it and SPC XL are two of the most commonly used of these add-on programs.
The computer also allows researchers to provide clients with sophisticated visual presentations of the data. These dashboards or scorecards are simply descriptive statistics that are updated at intervals ranging from daily to monthly to quarterly. Figures 8.1 and 8.2 present two such visual representations of a number of variables representing verbal descriptions, categorical data, and continuous data. We will refer back to these when discussing visualizing descriptive data.
Figure 8.1. Sample scorecard with pie, column, and bar charts.
Used with permission of Echo Research.
Descriptive Statistical Reasoning
Descriptive statistics, as noted earlier, simply describe or summarize the data. All quantitative research and some qualitative research describe data. The simplest level of description is to summarize the data for one variable, or to conduct a univariate descriptive analysis. More complex analyses summarize the data for two or more variables, or conduct bivariate or multivariate descriptive analyses. Bivariate and multivariate analyses summarize the data by each variable’s levels, such as sex (male or female) and socioeconomic status (lower, middle, and upper):
Figure 8.2.Scorecard with Summarized Bivariate Tables and Stacked Column and Bar Charts.
Used with permission of Echo Research.
Figure 8.2. (Cont.)Scorecard with Summarized Bivariate Tables and Stacked Column and Bar Charts.
Used with permission of Echo Research.
Categorical Statistical Analysis
Categorical statistical analysis basically deals with frequency counts—the actual number of observations found in each level of a categorical variable. It is important to remember that the frequency count is the basis for all other descriptive statistics, the most common of which is the percentage. A percentage is simply the number of observations in a category divided by the total number of observations. Percentage data are often used when the data are to be reported as quartiles (25% segments) or deciles (10% segments). A ratio can be calculated from the frequency counts. A ratio is a comparison of two frequencies; for example, say we had 10 males and 5 females, the ratio of males to females would be 2 to 1, stated as 2:1.
Univariate
Univariate descriptive statistics deal with a single variable. For instance, if a content analysis is run with the following categories for story rating, “favorable,” “neutral,” and “unfavorable,” there would be three categories (plus the “other” category, which we will ignore for now). If there are 100 placements, the descriptive statistics might break down as follows: 50 favorable, 30 neutral, and 20 unfavorable. The data can also be described in terms of percentages, with the number of observations per category divided by the total observations: 50% favorable, 30% neutral, and 20% unfavorable. Percentages are often reported with small sample sizes, which may make interpretation difficult. For instance, it seldom happens in large samples that the results come out at 50% or 33.3%, unless they were rounded off. If so, the descriptive analysis should state this. A third descriptive statistic is the ratio. If a survey’s results found that males represented 40 out of 400 respondents, while females represented 360 respondents, the proportion of females to males would be 9 to 1, or 9:1.
Visualizing. Categorical descriptive statistics are usually visualized as a univariate table, a bar or column chart (bar charts are horizontal, columns are vertical), or a pie chart, although there are other formats that can be used (e.g., surface, donut, bubble, radar, or spider web). The following table visually presents the story placement results as a univariate table:
Many public relations research firms now provide visualizations of statistics. Several different univariate statistics are visualized in Figures 8.1 and 8.2.
Bivariate and Multivariate
Bivariate and multivariate descriptive statistics describe the relationships between two or more categorical variables. Of the two, bivariate analyses are most common in public relations research reports. A bivariate analysis on the story placement by respondent sex would result in a two column (sex) by three row (story rating) table:
A multivariate descriptive analysis would produce extra tables. If a third variable were being analyzed, say socioeconomic status of the respondent (low, middle, high), there would be three tables, one for each level of the three variables. In the previous example, we would have a table for low socioeconomic respondents, a table for middle socioeconomic respondents, and a table for high socioeconomic respondents.
Visualizing. Visualizing bivariate and multivariate analyses is typically done through bar or column charts, although the data can be visualized through other types of charts (e.g., pie, spider). Bivariate and multivariate charts are visual representations of a table, as shown in Figures 8.1 and 8.2.
Continuous Statistical Analysis
Continuous data are found on a continuum, hence the label “continuous.” Continuous data are considered “stronger” than their categorical counterparts because of the statistical procedures and what they tell researchers. As noted in Chapter 3, continuous data are either interval, where the distance between data points is considered to be equal, or ratio, where the distance between data points is absolute. The demographic variable “age,” when calculated from year born, would be interval data (e.g., if I were to respond to a survey question, “In what year were you born?” and filled in 1949, when subtracted from 2009, it would yield my age as 60 years). My bank account, which would be in dollars and cents, would be ratio data.
What makes continuous data so powerful is that along its continuum, the data will fall under some type of curve, which is a function of the distribution of all data points gathered for whatever continuous variable is being examined. All continuous data have their own normal distribution. The hypothetical normal curve is shown in Figure 8.3. Of importance is the area under the curve, which can be expressed in “deviations” from the mean or average of all data points. This underlies the concept of continuous data having a central tendency—to distribute around that mean. Without going into statistical detail, all data can be demonstrated to fall within x number of standard deviations (based on the mean and its variance, or distribution from the mean). All curves find that 34% of all data will fall 1 standard deviation from the mean, or 68% of the data will fall ±1 standard deviation from the data’s mean. This is powerful in terms of description in that it provides much more information than simple frequencies, percentages, or ratios. While there are many continuous statistics available, we will concentrate on six.
Figure 8.3.The normal curve and standard deviations.
Univariate
The five most commonly employed continuous statistics are the mean, median, mode, variance, and standard deviation. The mean typically refers to the average of data points for a variable. There are, however, a number of different means used by statisticians,3 but the mean usually found in public relations research is the average. Means are highly influenced by data points that are extremely far from the average. Outliers are data points that influence the mean. Take 10 data points: 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10. The mean for this data set would be 5.5. If one of the data points were an outlier, say instead of 6 it was 22, the mean would be much larger (7.1). When there are outliers, it is essential that the median, or the data point that is 50% of the data set scores, be examined. When calculating the median, the data are lined up in order and the middle data point is the mean (for data sets with an even number of data points, the median is the average of the two scores around the 50th percentile). In the case of the 10 scores, the median would be 5.5, same as the mean. In the case of the outlier, the median would be 6.0. The mode is the data point(s) that reoccurs most in the data set. In this case, there are no recurring numbers, each number is unique. However, if the data points were 1, 2, 3, 3, 4, 5, 6, 7, 8, 9, the mode would be 3. A data set where the mean, median, and mode are identical would indicate adherence to the hypothetical normal curve, as shown in Figure 8.3. When the mean, median, and mode differ, the shape of the curve flattens. Part of this is due to the variance or distribution of scores.
As noted earlier, all data for a continuous variable are distributed around the mean for that data set. The variance provides an indicator of the distribution of data points around the mean. Interestingly, the variance is typically larger for small data sets than larger data sets. Why? Think of an auditorium with 100 seats. The first 10 people come in and find a seat—the distribution will be large, as there are many seats and few people, so their seating may be anywhere. As the number of people increases, the distribution decreases as fewer and fewer seats are left. The variance describes how “normal” the data set is, but it is unique to the data set. The standard deviation, which is the square of the variance, normalizes the data and can be used to compare data sets of different variables, even if those variables are measured differently (e.g., 5- or 7-point measure) through the standardized score for each variable, which is expressed in terms of the number of standard deviations each score is from the mean. Thus from these continuous statistics, we know the distribution of data around a mean. For instance, the average age for a sample might be 21.2 years, with a standard deviation of 3.2 years. This would tell us that 68% of the sample is 18.0 to 24.4 years old.
Visualizing. Univariate continuous statistics typically are reported as numbers in a table. It is difficult to create a graph of only one variable. When we do, however, we typically find that a line graph is used to visually portray the data (see Figures 8.1 and 8.2). For that we need to turn to bivariate and multivariate variable analyses.
Bivariate and Multivariate
As with categorical variables, more than one variable can be described in relation to another. This is typically done by describing the relationship between means for two or more variables; examining the correlation between the two variables; however, one of those variables must be categorical, which provides points of reference for the analysis. For instance, age and sex can be described by looking at the mean and standard deviation for males and females. When we look at two continuous variables, we typically describe their correlation. A correlation is the relationship between the variables data points. A correlation can only reflect the relationship between the two variables, and ranges from a perfect correlation of +1.00 through no correlation at all at 0.00 to a perfect negative correlation of –1.00. According to Hocking, Stacks, and McDermott,4 correlations below ±.30 are “weak,” ±.30 to ±.70 are “moderate,” ±.70 to ±.90 are “high,” and greater than ±.90 are “very high.” In communications research, most correlations are typically less than ±.50, and if higher, they may be restricted by controlling the data range in some way, often making the relationship unclear.
Visualizing. Visualizing bivariate and multivariate relationships is easier than univariate relationships because there is a comparison. The usual visualization is via the line or “fever” graph, with separate lines indicating different variables in relationship to each other (see Figure 8.1). A correlation is visualized as a scatter graph, where one variable is found on the x-axis and the other on the y-axis. Figure 8.4 shows that the relationship between sales and consumer ratings of an advertisement are positively related. If you were to draw a line through the data points it would go from the lower left corner just above the 6 on the sales axis and continue at an angle upward toward the 6 on the advertisement ratings axis.
Figure 8.4.Scatter graph.
Using Categorical and Continuous Data to Describe Simple Relationships
Categorical and continuous data are now commonly intermingled in visualizing data from several variables, providing the public relations practitioner with an indication or snapshot of the relationships between variables of interest. Note in the first visual on the second row of Figure 8.1 that some variables are univariate and some are bivariate (and, like comparing means, one axis is categorical and the other continuous).
At this point it is important to emphasize that descriptive statistics do not analyze the data or the relationships between variables. To make judgments about variables being larger or smaller than others, or that the relationships are truly “significant,” requires that inferences be made. This moves us to a more advanced set of statistics that allows researchers to state the strength of a relationship or lack of relationship within certain degrees of confidence. We turn next to understanding inferential statistical analysis.
Inferential Statistical Analysis
Inferential statistics are seldom reported in public relations research and, if reported, are generally put in footnotes. As a matter of fact, if the study’s data represent a true census of all respondents or sources, inferential statistics are not needed—any descriptive statistics represent what is found in the study. However, when dealing with larger populations drawn from even larger universes, it is not possible to collect data from all respondents or sources. We then sample, and as we know, certain amounts of error are built into the sampling process. Thus the researcher is left to wonder whether the descriptive statistics are true representations of the larger population and differences among variables are real or whether they are due to chance—or error.
One of the reasons survey researchers run inferential statistics on their demographics is to test sampling error. If there were 400 randomly selected respondents to a survey, the researcher is willing to accept up to 5% sampling error—but this does not mean that there was sampling error, just that there might be. Therefore, running inferential tests on the data against known data (and hoping for no differences) gives a researcher the ability to say that she is 90%, 95%, or 99% confident that the sample is representative of the larger population. Inferential statistics provide the researcher with a confidence level, or the amount the variables differ from each other.
In survey research, the second form of error is measurement error. Inferential statistics allow us to test for measurement error among the outcome variables when analyzing the descriptive statistics. For instance, if purchase intent of a product is the outcome of interest and the study used measures of product awareness, product knowledge, and liking as indicators of purchase intent, the researcher needs to test whether the variables truly indicate purchase intent, and if so, how much confidence the researcher can have in the results of the tests. The accepted confidence that there are differences is put at 95%. This means that 95 times out of 100 the results obtained are due to the variables indicating purchase intent and not to measurement or other errors.
There are many inferential statistics that can be run on data. Some simply look to see if levels of a categorical variable (e.g., sex) describe differences in the outcome variable. For instance, do males intend to purchase the product more than females, or is the difference in purchase intent due to error? Others try to predict from a number of variables (almost always categorical) that are related to the outcome of interest. For instance, which of three variables, operationalized as dichotomous variables (i.e., high awareness, high knowledge, high liking versus low awareness, knowledge, and liking) best predicts purchase intent? Those statistics that look at differences are fairly simple, and we will look at the chi-square (c2) as representative of categorical variables and the t-test and analysis of variance (ANOVA) as representative of continuous tests. Regression is what most use to try to model predictive tests. As noted in the first part of this chapter, it is not our intent to make statisticians out of our readers, but we hope that after the short descriptions that follow you will have a basic understanding of what each test does.
Categorical
The chi-square (c2) test is one of the most utilized of categorical inferential statistical tests. The chi-square test determines whether the frequency of observations in a given category or level of a variable is different from what would be expected. Chi-square is often used in surveys to test demographic variables against known results, typically against census or industry data. For instance, suppose a random survey of shoppers in a large metropolitan area finds that 56% of the respondents are female and 44% are male. Going to the U.S. Census data (easily available through the U.S. government), we find that for this particular area females constitute 51% of the metropolitan area and males 49%. Chi-square can be used to test the female (56%) and male (44%) data obtained against the census data (51% and 49%, respectively). A chi-square test, if run, would find that the sample differed significantly (the probability of differences was confirmed with 95% confidence) from the census data (there is only a 5% chance that the data were due to sampling error).
Continuous
While categorical inferential statistics look for differences between categories, continuous inferential statistics look for differences between means for different categories. The most commonly used inferential test is the t-test. The t-test looks at differences in continuous outcome variables between variables with two groups (e.g., male/female, high/low, expensive/inexpensive). This is a major limitation, but t-tests are easy to interpret. The t-test has another limitation—it is sensitive to differences in small samples (less than 150 samples) because of the way it is calculated.5 Suppose we have two means, male purchase intent on a 5-point scale was found to be 4.5, while female purchase intent was found to be 2.5. The question the t-test answers is whether 4.5 is truly different from 2.5 (notice that it is not stated as larger or smaller, but as different). A t-test run on the means would find that, yes, the two groups are significantly different and that the researcher can be 95% confident in making that claim.
For larger samples, or for studies where there are more than two levels of a variable or more than one variable, a different test is run. The analysis of variance (ANOVA) or F-test is a more general test that is not sensitized to large samples. Instead of using a special variance measure, it simply looks for the variance that can be explained by being in a category (high, moderate, low awareness) as compared against the variance of being in any group (called “between-group” or systematic variance for the category and “within-group” or error variance for all respondents across groups) for the outcome variable (purchase intent). The ANOVA then tests to see if there is less variance for groups as compared against all respondents. With a larger sample, the results for dichotomous variables are similar to the t-test. However, with three or more groups, a significant overall finding (that being in a group did make a difference in the outcome variable) does not indicate how the groups differed. In that case, more tests must be run, which are a specialized form of the ANOVA—the ONE-WAY ANOVA.6
Finally, there are times when researchers want to predict which variables best predict an outcome. The most commonly used statistical test for this purpose is a simple or multiple regression. The regression first looks for differences between variables and then looks for how the variables correlate. By drawing a line through the correlation matrix, outcomes can be predicted for individual variables or you can test to see which variables of several will best predict outcomes.7 The variables that are examined are the “dependent variable” and the “independent variables,” which impact or affect the dependent variable. An example of a dependent variable is “willingness to purchase” a product or service. What regression does is examine a series of other variables that impact or “drive” a “willingness to purchase” and determine the overall contribution each of these independent variables on that decision. Examples of independent variables include overall awareness of a product, levels of knowledge about the product, or affinity toward or prior relationship between the company and the purchaser.
Best Practices
Best practices public relations employs descriptive and inferential statistics in analyzing the effect of public relations variables on the outcomes of interest. Running appropriate statistical tests and being able to discuss the descriptive findings is the first step in best practices statistical analysis. The second step is to run the appropriate inferential statistics to test for confidence in those differences and which variables best predict specific outcomes.
Summary
Continuous inferential statistics are powerful tests of differences between and potential predictors of variables relating to some outcome. With the increased emphasis on proving value, public relations research is moving quickly to employing more inferential tests. In many instances, public relations researchers do employ inferential tests, but to simplify presentations and final reports, they are often not included. The inferential test provides the public relations professional with the information necessary to state with confidence whether or not variables in a program or campaign are effective in demonstrating return on expectations, and ultimately return on investment.