Some Possible Results from a Two-Way Mixed-Design ANOVA

Before showing how to analyze data from a mixed design, it is instructive to first review some of the results that are possible with this design. This will help illustrate the power of this design and will lay a foundation for concepts to follow.

A Significant Interaction

In Chapter 10, “Factorial ANOVA with Two Between-Group Factors,” you learned that a significant interaction means that the relationship between one predictor variable and the criterion is different at different levels of the second predictor variable. (With experimental research, the corresponding definition is “the effect of one independent variable on the criterion variable is different at different levels of the second independent variable.”). To better understand this definition, refer back to Figure 13.3, which illustrates the interaction between TIME and GROUP.

In this study, TIME is the variable that coded the repeated-measures factor (Time 1 scores versus Time 2 scores versus Time 3 scores). GROUP, on the other hand, is the variable that coded the between-subjects factor (experimental group versus control group). When there is a significant interaction between a repeated-measures factor and a between-groups factor, it means that the relationship between the repeated-measures factor and the criterion is different for the different groups coded under the between-subjects factor.

This is illustrated by the two lines in Figure 13.3. To understand this interaction, begin by focusing on just the solid line in the figure, which illustrates the relationship between TIME and investment scores for just the experimental group. Notice that the mean for the experimental group is relatively low at Time 1 but substantively higher at Times 2 and 3. This shows that there is a relationship between TIME and perceived investment for the experimental group.

Next, focus only on the dashed line that illustrates the relationship between TIME and perceived investment for just the control group. Notice that this line is flat (i.e., there is little change from Time 1 to Time 2 to Time 3) indicating that there is no relationship between TIME and investment size for the control group.

Combined, these results illustrate the definition for an interaction: the relationship between one predictor variable (TIME) and the criterion variable (investment size) is different at different levels of the second predictor variable.

You determine whether an interaction is significant by consulting the appropriate statistical test in your SAS output. However, it is also sometimes possible to identify a likely interaction by reviewing a figure that illustrates group means. For example, consider the solid line and the dashed line of Figure 13.3, particularly the first segment of each line that goes from Time 1 to Time 2. Notice that the solid line (for the experimental group) is not parallel to the dashed line (for the control group), the solid line has a much steeper angle. This is the hallmark of an interaction: nonparallel lines. Whenever a figure shows that a line segment for one group is not parallel to the corresponding line segment for a different group, it might mean that there is an interaction between the repeated-measures factor and the between-subjects factor.

In some (but not all) studies that employ a mixed design, your central hypothesis might require that there be a significant interaction between the repeated-measures variable and the between-subjects variable. For example, in the present study that assesses the effectiveness of the marriage encounter program, a significant interaction would be required to show that the experimental group displays a greater increase in investment size compared to the control group.

Significant Main Effects

If a given predictor variable is not involved in any significant interactions, you are free to determine whether or not that variable displays a significant main effect. When a predictor variable displays a significant main effect, it means that there is a difference between at least two of the levels of that predictor variable with respect to scores on the criterion variable.

The number of main effects that are possible in a study is equal to the number of predictor variables. The present investment model study includes two predictor variables; so, two main effects are possible: one for the repeated-measures factor (TIME); and one for the between-subjects factor (GROUP). These main effects can take a variety of different forms.

A Significant Main Effect for TIME

For example, Figure 13.4 shows one possible main effect for the TIME variable: an increasing linear time trend. Notice that the investment scores at Time 2 are higher than the scores at Time 1, and that the scores at Time 3 are somewhat higher than the scores at Time 2. Whenever the values of a predictor variable are plotted on the horizontal axis of a figure, a significant main effect for that variable is indicated when the line segments display a relatively steep angle.

In Figure 13.4, the values of TIME are plotted on the horizontal axis. (These values are identified as “Baseline Survey, Time 1,” “Post-Treatment Survey, Time 2,” and “Follow-Up Survey, Time 3.”) There is a relatively steep angle in the line that goes from Time 1 to Time 2, and also a relatively steep angle in the line that goes from Time 2 to Time 3. These results will typically suggest a significant main effect. Of course, you always check the appropriate statistical test in the SAS output to verify that the main effect is, in fact, significant.

Remember that the main effect for TIME is averaged over the two groups in the study. In the present case, this means that there is an overall main effect for TIME after collapsing the experimental and control groups (i.e., considering both as combined grouping).

A Significant Main Effect for GROUP

You would expect to see a different pattern in a figure if the main effect for the other predictor variable was significant. Earlier, it was stated that values of the predictor variable TIME are plotted as three separate points on the horizontal axis of the figure. In contrast, the values of the predictor variable GROUP are coded by drawing separate lines for the two groups under this variable: the experimental group is represented with a solid line, and the control group is represented with a dashed line.

When a predictor variable (such as GROUP) is represented in a figure by plotting separate lines for its various levels, a significant main effect for that variable is evident when at least two of the lines are relatively separate from each other. For example, consider Figure 13.5.

Begin your review of Figure 13.5 by first determining which effects are probably not significant. You can see that all line segments for the experimental group are parallel to their corresponding segments for the control group; this tells you that there is probably not a significant interaction between TIME and GROUP. Next, you can see that none of the line segments in Figure 13.5 display a relatively steep angle; this tells you that there is probably not a significant main effect for TIME.

However, notice that the solid line that represents the experimental group appears to be separated from the dashed line that represents the control group. Now, look at the individual data points. At Time 1, the experimental group displays a mean investment score that appears to be much higher than the one displayed by the control group. This same pattern of differences appears at Time 2 and Time 3. Combined, these are the results that you would expect to see if there were a significant main effect for GROUP. Figure 13.5 suggests that the experimental group consistently demonstrated higher investment scores than the control group.

A Significant Main Effect for Both Predictor Variables

It is possible to obtain significant main effects for both predictor variables simultaneously. Such an outcome appears in Figure 13.6. Notice that the line segments display a relatively steep angle (indicative of a main effect for TIME) and the lines for the two groups are also relatively separated (indicative of a main effect for GROUP).

Nonsignificant Interaction, Nonsignificant Main Effects

Of course, there is no law that says that any of your effects have to be significant (as every researcher knows all too well!). This is evident in Figure 13.7. In that figure, notice that the lines for the two groups are parallel suggesting a probable nonsignificant interaction. There is also no angle to the line, suggesting that the main effect for TIME is nonsignificant. Finally, the line for the experimental group is not really separated from the line for the control group; this suggests that the main effect for GROUP is likewise nonsignificant.