STATISTICAL PROCEDURES UNIT A: WRITING THE STATISTICAL RESEARCH SUMMARY
When you summarize the results of your research, remember that these summaries are designed to be written for a research audience. The settings might include a professional association at which you are highlighting your findings (as in a poster session), a brief blurb for a book, the findings section of a team research report, or a proposal to company representatives as part of a consulting project.
Create the summary keeping the following points in mind, and use established writing guidelines (e.g., Chicago, American Psychological Association, Modern Language Association). Include figures and tables only when they are necessary in explaining or describing the findings. Unnecessary visual summaries can be distracting and redundant.
The following guidelines are organized by the steps you need to take to complete the statistical analysis. After you have stated the hypotheses, address each of the following as ingredients for your decision whether to reject the null hypotheses and how the findings relate to the overall research problem.
1. State the general research question or theory, and then list the hypotheses and study variables.
2. Describe the
purpose of the statistical test(s) you will use to address whether to reject the null hypothesis, accept the alternate hypotheses, and so on. The following are two examples of using different statistical procedures:
A one-sample t test was conducted on PWPVS scores of postal workers in a large suburban postal terminal to evaluate whether their mean score was significantly different from a score of 45, the assumed population mean of the PWPVS for U.S. postal workers.
A multiple linear regression analysis was conducted to examine whether worker satisfaction is predicted by type of job, satisfaction with coworkers, perceived social support, and satisfaction with supervision. Results of this analysis will show the proportion of variance in worker satisfaction contributed by the combination of predictor variables, and it will measure the unique contribution of each predictor to the variance in worker satisfaction.
3. Report the descriptive statistics (mean, standard deviation, standard error, etc.).
4. Report and briefly discuss the assumptions of the test and whether your data met these assumptions. Cite the evidence for meeting the assumptions (e.g., whether Levene’s test results indicate violations of equal variance, whether skewness and kurtosis figures are beyond acceptable limits and indicate different statistical tests). Each statistical procedure includes a separate set of assumptions, but most all (parametric tests) focus on whether outcome variable groups are normally distributed, group variances are equal, and interval data are used (for the outcome variable particularly, although specific tests can differ).
5. Report the results of the
omnibus statistical test including test ratio, df, and sig. levels. The omnibus test is the overall test that indicates a statistically significant finding among all components of the data. An example of this is the ANOVA test in which an overall significance test establishes whether all paired group differences among a factor, taken together, indicate nonchance differences in the dependent variable. The following examples rel
ate to ANOVA and multiple regression respectively:
- The omnibus F ratio (14.55) was significant (p = 0.02) indicating that we can reject the null hypothesis of equal group differences on the dependent variable.
- The model was significant (F = 99.07, p < 0.001) indicating an R2 of 0.128. The resulting regression equation was Ypred = 1.17 + 0.14X1 + 0.11X2
6. Report on any post hoc or individual predictor tests that are part of the statistical procedure. In the case of ANOVA, this includes tests like the Tukey’s HSD, Scheffe, LSD, Bonferroni, and so on, in which the researcher must determine if pairwise differences among independent variable groups are significantly different. For multiple regression, the researcher should note whether each predictor variable is significant as determined by individual t tests. In chi-square tests, follow-up tests would include the results of a two variable analysis within each condition of a separate control variable.
7. Report and discuss the effect size. This is a crucial part of every statistical analysis and must be reported to indicate the impact of the test variable(s) on the dependent variable. Each statistical test has a separate method for determining effect size, so consult a statistical text (e.g., see Abbott 2011) for the appropriate procedures.
For example, in t tests, you might report, “According to the criteria for Cohen’s d, the effect size of 0.367 is considered small to medium. This means …”
For regression results, you might conclude, “Both predictors explain 27 percent of the variance in the outcome variable (R2 = 0.272), and the squared part correlation for predictor one (part r2 = 0.15) indicates that it uniquely explains 15 percent of the variance in the outcome variable.
For chi square, you could note that “Cramer’s V (0.120) results for the first panel of the multivariate test indicates small differences between the two test variables in the first category of the control variable.”
8. Interpret the statistical results. All of the foregoing steps apply to the statistical findings. The researcher must now translate these technical findings into the language of the study so that the audience can understand how they apply to their problem of interest. Many people do not understand how technical findings like “the pairwise differences among factor A indicate significant differences between low and high levels” translates into real words!
In a regression study, for example, we might conclude that predictor one is a significant predictor of the outcome variable and results in a 5 percent change in the outcome measure with every 10 percent change in the predictor (when other predictors are not allowed to affect the results). Stated simply, “When pay (predictor one) increases by 10 percent, there is a 5 percent increase in worker job satisfaction (among workers with similar levels of job tenure (predictor two).”
Elaborate briefly what conclusions you might draw regarding the research question. The greatest danger in this step is to over-conclude. That is, researchers (even seasoned researchers) make the mistake of exceeding the boundaries of what the data actually report. Here is an example of over-concluding by a student reporting significant results in a test linking a new instructional procedure in a district of 10 schools to increased math achievement in those schools: “Therefore this procedure should be promoted in all schools and teachers not using them should be retrained.”
In this case, the student was stretching the findings just a bit! The problem of the ecological fallacy (see earlier sections in this book) is a particular problem in the interpretation section.
Here are some brief examples (from Abbott 2011) of interpretation sections for three statistical procedures:
ANOVA: “Taken together, the data analyses indicate that groups of FR have an impact on schools’ reading achievement results. In particular, as the percentages of students qualified for FR increase in the schools, the percentage of students meeting the reading achievement assessment declines … [The] F test (24.22) is significant (p < 0.05) and the effect size substantial (η2 = 0.47), indicating that 47 percent of the schools’ reading achievement scores are affected by grouping on the FR variable (Low, Medium, and High). A Tukey HSD analysis indicated that each of the three FR groups’ reading achievement percentages were significantly different from the other groups.”
Chi square: “As you can see from the figure, when the respondents have been victims of a crime, they are much more likely to have a high fear of crime. Over 54 percent of victims have a fear of crime compared to 29.4 percent of those not victimized. This difference in fear persists even when crime is on the decline as it was in our study.”
Multiple regression: “The findings indicate that the two predictors, eskills and eaccess, are significant predictors of eimpact. Both predictors explain about 27 percent of the variance (i.e., R2 = 0.275) in eimpact, a finding considered large. … Each predictor is a significant predictor … but eskills is the more powerful of the two in explaining the variance of eimpact. According to the sample data, it is important for elementary teachers to have access to technology for making an impact on the classroom, but is it much more important that they perceive that they have technology skills. Having technology skills explains almost ten times more variance in eimpact than having access to technology.”
