Example with Significant Differences between Experimental Conditions
To illustrate MANOVA, assume that you replicate the study that examined the effect of rewards on commitment in romantic relationships. This time, however, the study is modified so that you can obtain scores on three different dependent variables rather than just one.
It was hypothesized that the rewards people experience in romantic relationships have a causal effect on their commitment to those relationships. In Chapter 9, this hypothesis was tested by conducting a type of analogue experiment. All 18 participants in the experiment were asked to engage in similar tasks: to read the descriptions of 10 potential romantic “partners;” for each partner, imagine what it would be like to date this person and rate how committed they would be to a relationship with that person. For the first nine partners, each participant saw the same description. For partner 10, however, the different experimental groups were presented with a slightly different description of this person. For example, for the six participants in the high-reward condition, the last sentence in their description of partner 10 read as follows:
This person enjoys the same recreational activities that you enjoy and is very good-looking.
However, for the six participants in the mixed-reward condition, the last part of the description of partner 10 read as follows:
Sometimes, this person seems to enjoy the same recreational activities that you enjoy and sometimes s/he does not. Sometimes, partner 10 seems to be very good looking and sometimes s/he does not.
And for the six participants in the low-reward condition, the last sentence in the description of partner 10 read in this way:
This person does not enjoy the same recreational activities that you enjoy and is not very good-looking.
In your current replication of the study from Chapter 9, you manipulate this “type of rewards” independent variable in exactly the same way. The only difference between the present study and the one described earlier involves the number of dependent variables measured. In the earlier study, you obtained data on just one criterion variable: commitment. Commitment was defined as the participants’ rating of their commitment to remain in the relationship. Scores on this variable were created by summing participant responses to four questionnaire items. In the present study, however, you also obtain two additional dependent variables: participants’ rating of their satisfaction with their relationship with partner 10 as well as their ratings of how long they intend to stay in the relationship with partner 10. Here, satisfaction is defined as the participants’ emotional reaction to partner 10, whereas the intention to stay is defined as a rating of how long they intend to maintain the relationship with partner 10. Assume that satisfaction and intention to stay are measured with multiple items from a questionnaire, similar to those used to assess commitment.
You can see that you will obtain four scores for each participant in the study. One will simply be a score on a classification variable that indicates whether the participant is in the high-reward group, the mixed-reward group, or the low-reward group. The remaining three scores will be the participant’s scores on the measures of commitment, satisfaction, and intention to stay. In conducting the MANOVA, you need to determine whether there is a significant relationship between the type of rewards predictor variable and the three criterion variables, taken as a group.
Writing the SAS Program
Creating the SAS Dataset
There is one predictor variable in this study, type of rewards. This variable could assume one of three values: participants were either in the high-reward group; the mixed-reward group; or the low-reward group. Since this variable simply codes group membership, it is measured on a nominal-scale. You give participants a score of “1” if they are in the low-reward condition, a “2” if they are in the mixed-reward condition, and a score of “3” if they are in the high-reward condition. You need a short SAS variable name for this variable, so call it REWGRP (for “reward group”).
There are three criterion variables in this study. The first criterion variable, COMMITMENT, is given the SAS variable name, COMMIT. The second criterion, SATISFACTION, is called SATIS. The third, intention to stay, is called STAY. With each criterion variable, possible scores could range from a low of 4 to a high of 36.
This is the DATA step of the program that performs the MANOVA. It contains fictitious data from the preceding study:
1 DATA D1; 2 INPUT #1 @1 REWGRP 1. 3 @3 COMMIT 2. 4 @6 SATIS 2. 5 @9 STAY 2. ; 6 DATALINES; 7 1 13 10 14 8 1 06 10 08 9 1 12 12 15 10 1 04 10 13 11 1 09 05 07 12 1 08 05 12 13 2 33 30 36 14 2 29 25 29 15 2 35 30 30 16 2 34 28 26 17 2 28 30 26 18 2 27 26 25 19 3 36 30 28 20 3 32 32 29 21 3 31 31 27 22 3 32 36 36 23 3 35 30 33 24 3 34 30 32 25 ; 26 RUN;
The data are entered so that there is one line of data for each participant. The format used in entering the preceding data is summarized in the following table:
| Line | Column | Variable Name | Explanation |
|---|---|---|---|
| 1 | 1 | REWGRP | Codes group membership, so that
1 = low-reward condition 2 = mixed-reward condition 3 = high-reward condition |
| 2 | blank | ||
| 3-4 | COMMIT | Commitment ratings obtained when participants rated partner 10 | |
| 5 | blank | ||
| 6-7 | SATIS | Satisfaction ratings obtained when participants rated partner 10 | |
| 8 | blank | ||
| 9-10 | STAY | Intention to stay ratings obtained when participants rated partner 10 |
Testing for Significant Effects with PROC GLM
 | In the following example, you write a SAS program to determine whether the overall MANOVA is significant and follow it with univariate ANOVAs and Tukey HSD tests. Remember that the results for all of these tests will appear in the output regardless of the significance of the multivariate test. However, you should interpret the univariate ANOVAs and Tukey tests only if the overall MANOVA is significant. |
Here is the general form for the SAS program to perform a MANOVA followed by individual ANOVAs and Tukey’s HSD tests:
PROC GLM DATA=filename; CLASS predictor-variable; MODEL criterion-variables = predictor-variable; MEANS predictor-variable / TUKEY; MEANS predictor-variable; MANOVA H = predictor-variable / MSTAT = exact; RUN;
Here is the actual program (including the DATA step) that you would use to analyze data from the investment-model study just described:
1 DATA D1; 2 INPUT #1 @1 REWGRP 1. 3 @3 COMMIT 2. 4 @6 SATIS 2. 5 @9 STAY 2. ; 6 DATALINES; 7 1 13 10 14 8 1 06 10 08 9 1 12 12 15 10 1 04 10 13 11 1 09 05 07 12 1 08 05 12 13 2 33 30 36 14 2 29 25 29 15 2 35 30 30 16 2 34 28 26 17 2 28 30 26 18 2 27 26 25 19 3 36 30 28 20 3 32 32 29 21 3 31 31 27 22 3 32 36 36 23 3 35 30 33 24 3 34 30 32 25 ; 26 RUN; 27 28 PROC GLM DATA=D1; 29 CLASS REWGRP; 30 MODEL COMMIT SATIS STAY = REWGRP; 31 MEANS REWGRP / TUKEY; 32 MEANS REWGRP; 33 MANOVA H = REWGRP / MSTAT = exact; 34 RUN;
Notes Regarding the SAS Program
The data are input in lines 6 through 26 of the preceding program, and the PROC GLM step requesting the MANOVA appears in lines 28 through 34. In the CLASS statement on line 29, you should specify the name of the nominal-scale predictor variable (REWGRP, in this case). The MODEL statement in line 30 is similar to that described in Chapter 9, except that all of your criterion variables should appear to the left of the “=” sign (COMMIT, SATIS, and STAY in this case). The name of the predictor variable should again appear to the right of the “=” sign.
The MEANS statement on line 31 requests that the Tukey multiple comparison procedure be performed for the REWGRP predictor variable. The second MEANS statement on line 32 merely requests that the means for the various treatment groups be printed.
The only new statement in the program is the MANOVA statement that appears on line 33. To the right of the term “H =” in this statement, you should specify the name of your study’s predictor variable. The present program specifies “H = REWGRP.”
Results from the SAS Output
Specifying LINESIZE=80 and PAGESIZE=60 in the OPTIONS statement (first line) causes the preceding program to produce nine pages of output. The following table indicates which specific results appear on each page:
| Page | Information |
|---|---|
| 1 | Class level information |
| 2 | Univariate ANOVA results for COMMIT criterion |
| 3 | Univariate ANOVA results for SATIS criterion |
| 4 | Univariate ANOVA results for STAY criterion |
| 5 | Tukey HSD test results for COMMIT criterion |
| 6 | Tukey HSD test results for SATIS criterion |
| 7 | Tukey HSD test results for STAY criterion |
| 8 | Group means for each criterion variable |
| 9 | Results of the multivariate analysis: characteristic roots and vectors; Wilks’ lambda; and results of the multivariate F test |
The output produced by the preceding program is reproduced here as Output 11.1.
Output 11.1. Results of the Multivariate Analysis of Variance for the Investment Model Study, Significant Results
Interpretation of pages 1 through 8 (the univariate ANOVA results for the three criterion variables) would proceed in exactly the same manner as described in Chapter 9 in the section “Steps in Interpreting the Output.” Because these steps have already been described, they are not covered again in this chapter. You are encouraged, however, to review these results to make sure that everything looks right prior to interpreting the multivariate results. You should make sure that all participants were included in the analyses, that the input statements were written correctly, and so forth. Again, remember that these univariate ANOVAs and Tukey tests should be interpreted only if the overall MANOVA is significant.
Steps in Interpreting the Output
1. Review Wilks’ Lambda, the Multivariate F Statistic, and Its Associated p Value
Once you have reviewed the univariate statistics on pages 1 through 8 to verify that there are no obvious errors in entering data or in writing the program, you can review your MANOVA results on page 9 of the output.
You can state the multivariate null hypothesis for the present study as follows:
In the population, there is no difference across groups for participants in the high-reward group, the mixed-reward group, and the low-reward group when compared simultaneously on commitment, satisfaction, and intention to stay in the relationship.
In other words: in the population, the three groups are equal on all criterion variables. Symbolically, you can represent the null hypothesis this way:
Each “M” represents a sample mean for one of the treatment conditions on one of the criterion variables. With the preceding notation, the first number in each subscript identifies the criterion variable, and the second number identifies the experimental group. For example, M31 refers to the sample mean for the third criterion variable (the intention to stay) in the first experimental group (the low-reward group), whereas M12 refers to the sample mean for the first criterion variable (commitment) in the second experimental group (the mixed-reward group).
Below the heading “Statistic,” the first subheading you find is “Wilks’ Lambda.” Where the row headed “Wilks’ Lambda” intersects with the column headed “Value,” you see the computed value of Wilks’ lambda for this analysis. For the current analysis, lambda is approximately .03. Remember that small values (closer to 0) indicate a relatively strong relationship between the predictor variable and the multiple criterion variables, and so this obtained value of .03 indicates that there is a very strong relationship between REWGRP and the criterion variables. This is an exceptionally low value for lambda, much lower than you generally encounter in actual research.
This value for Wilks’ lambda indicates that the relationship between the predictor and the criteria is strong, but is it statistically significant? To determine this, see where the row headed “Wilks’ Lambda” intersects with the column headed “P-Value.” You can see that this significance value is very small (i.e., p < .01). Because this p value is less than the threshold value of .05, you can reject the null hypothesis of no differences between groups. In other words, you conclude that there is a significant multivariate effect for type of rewards.
2. Summarize the Results of Your Analyses
This section shows how to summarize just the results of the multivariate test discussed in the previous section. When the multivariate test is significant, you can proceed to the univariate ANOVAs and summarize them using the format presented in Chapter 9.
You can use the following statistical interpretation format to summarize the multivariate results from a MANOVA:
Statement of the problem
Nature of the variables
Statistical test
Null hypothesis (H0)
Alternative hypothesis (H1)
Obtained statistic
Obtained probability (p) value
Conclusion regarding the null hypothesis
You could summarize the preceding analysis in the following way.
A) Statement of the problem: The purpose of this study was to determine whether there was a difference across groups of participants in a low-reward relationship, mixed-reward relationship, and high-reward relationship with respect to their commitment to the relationship, their satisfaction with the relationship, and their intention to stay in the relationship.
B) Nature of the variables: This analysis involved one predictor variable and three criterion variables. The predictor variable was “type of rewards,” which was measured on a nominal-scale and could assume three values: a low-reward condition (coded as 1); a mixed-reward condition (coded as 2); and a high-reward condition (coded as 3). The three criterion variables were “commitment,” “satisfaction,” and “intention to stay.” All were measured on an interval/ratio scale.
C) Statistical test: Wilks’ lambda, derived through a one-way MANOVA, between-subjects design.
D) Null hypothesis (H0):
In the population, there is no difference across groups for participants in the high-reward group, the mixed-reward group, and the low-reward group when compared simultaneously on commitment, satisfaction, and intention to stay in the relationship.
E) Alternative hypothesis (H1): In the population, there is a difference between participants in at least two of the conditions when they are compared simultaneously on commitment, satisfaction, and intention to stay.
F) Obtained statistic: Wilks’ lambda = .03.
G) Obtained probability (p) value: p < .01.
H) Conclusion regarding the null hypothesis: Reject the null hypothesis.
Formal Description of Results for a Paper
You could summarize this analysis in the following way for a scholarly journal:
Results were analyzed using one-way MANOVA, between-subjects design. This analysis revealed a significant multivariate effect for type of rewards, Wilks' lambda = .03, and p < .01.