Example: Analysis of the Prosocial Orientation Inventory
Assume that you have developed an instrument called the Prosocial Orientation Inventory (POI) that assesses the extent to which a person has engaged in helping behaviors over the preceding six-month period. The instrument contains six items and is presented here.
Instructions: Below are a number of activities in |
which people
sometimes engage. For each item, please indicate
When this instrument was first developed, you intended to administer it to a sample of participants and use their responses to the six items as separate predictor variables in a multiple regression equation. As previously stated, however, you learned that this is a questionable practice and have decided, instead, to perform a principal component analysis on responses to the six items to see if a smaller number of components can successfully account for most of the variance in the dataset. If this is the case, you will use the resulting components as the predictor variables in your regression analyses.
At this point, it might be instructive to review the content of the six items that constitute the POI to make an informed guess as to what is likely to be observed from the principal component analysis. Imagine that, when you first constructed the instrument, you assumed that the six items were assessing six different types of prosocial behavior. Inspection of items 1 through 3, however, shows that these three items share something in common: they all deal with the activity of “going out of one’s way to do a favor for someone else.” It would not be surprising to learn that these three items hang together empirically in the principal component analysis to be performed. In the same way, a review of items 4 through 6 shows that all of these items involve the activity of giving money to those in need. Again, it is possible that these three items will also group together in the course of the analysis.
In summary, the nature of the items suggests that it might be possible to account for variance in the POI with just two components: a “helping others” component and a “financial giving” component. At this point, this is only speculation, of course; only a formal analysis can determine the number and nature of the components measured by the POI.
(Remember that the preceding instrument is fictitious and used for purposes of illustration only; it should not be regarded as an example of a good measure of prosocial orientation. Among other problems, this questionnaire obviously deals with very few forms of helping behavior.)
Preparing a Multiple-Item Instrument
The preceding section illustrates an important point about how not to prepare a multiple-item measure of a construct. Generally speaking, it is poor practice to throw together a questionnaire, administer it to a sample, and then perform a principal component analysis (or factor analysis) to determine what the questionnaire is measuring.
Better results are much more likely when you make a priori decisions about what you want the questionnaire to measure and then take steps to ensure that it does. For example, you would have been more likely to obtain desirable results if you:
had begun with a thorough review of theory and research on prosocial behavior;
used that review to determine how many types of prosocial behavior probably exist;
wrote multiple questionnaire items to assess each type of prosocial behavior.
Using this approach, you could have made statements such as “there are three types of prosocial behavior: acquaintance helping; stranger helping; and financial giving.” You could have then prepared a number of items to assess each of these three types, administered the questionnaire to a large sample, and performed a principal component analysis to see if the three components did, in fact, emerge.
Number of Items per Component
When a variable (such as a questionnaire item) is given a weight in constructing a principal component, we say that the variable loads on that component. For example, if the item “Went out of my way to do a favor for a coworker” is given a lot of weight in creating the “helping others” component, we say that this item loads on that component.
It is highly desirable to have at least three (and preferably more) variables loading on each retained component when the principal component analysis is complete (see Clark & Watson, 1995). Because some items might be dropped during the course of the analysis (for reasons to be discussed later), it is generally good practice to write at least five items for each construct that you want to measure. This increases the likelihood that at least three items per component will survive the analysis. Note that we have unfortunately violated this recommendation by apparently writing only three items for each of the two a priori components constituting the POI.
One additional note on scale length: the recommendation of three items per scale should be viewed as an absolute minimum and certainly not as an optimal number of items per scale. In practice, test and attitude scale developers normally desire that their scales contain many more than just three items to measure a given construct. It is not unusual to see individual scales that include 10, 20, or even more items to assess a single construct (e.g., O’Rourke & Cappeliez, 2002). Up to a point, the more items in the scale, the more reliable it will be. The recommendation of three items per scale should therefore be viewed as a rock-bottom lower limit, appropriate only if practical concerns (such as total questionnaire length) prevent you from including more items. For more information on scale construction, see Spector (1992).
Minimally Adequate Sample Size
Principal component analysis is a large-sample procedure. To obtain reliable results, the minimal number of participants providing usable data for the analysis should be the larger of 100 participants or five times the number of variables being analyzed (Streiner, 1994).
To illustrate, assume that you want to perform an analysis on responses to a 50-item questionnaire. (Remember that, when responses to a questionnaire are analyzed, the number of variables is equal to the number of items on that questionnaire.) Five times the number of items on the questionnaire equals 250. Therefore, your final sample should provide usable (complete) data from at least 250 participants. Of note, however, any participant who fails to answer just one item does not provide usable data for the principal component analysis and is therefore excluded from the final sample. A certain number of participants can always be expected to leave at least one question blank. To ensure that the final sample includes at least 250 usable responses, you would be wise to administer the questionnaire to perhaps 300 to 350 participants.
These rules regarding the number of participants per variable again constitute a lower limit, and some have argued that they should apply only under two optimal conditions for principal component analysis: when many variables are expected to load on each component; and when variable communalities are high. Under less optimal conditions, even larger samples might be required.