Should you rotate or not? As stated previously, rotation helps in the interpretation of the principal components by modifying the loading of each variable, but makes the results technically no longer a principal component. The overall variation explained by the rotated number of components will not change, but the contributions to the total variance explained by each component will change. What you will find by rotation is that the loading values will either move farther or closer to zero, theoretically aiding in identifying those variables that are important to each principal component. This is an attempt to associate a variable to only one principal component. Remember that this is unsupervised learning, so you are trying to understand your data, not test some hypothesis. In short, rotation aids you in this endeavor. I have seen both the non-rotated and rotated components used to calculate the loading. I like to use the rotated components.
The most common form of principal component rotation is known as varimax. There are other forms, such as quartimax and equimax, but we will focus on varimax rotation. In my experience, I've never seen the other methods provide better solutions. Trial and error on your part may be the best way to decide the issue.
Perhaps the best way to demonstrate this is via another simple illustration. Let's assume that we have a dataset of variables A through G and we have two principal components. Plotting this data, we will end up with the following diagram:
For the sake of argument, let's say that variable A's loading are -0.4 on PC1 and 0.1 on PC2. Now, let's say that variable D's loading are 0.4 on PC1 and -0.3 on PC2. For point E, the loading are -0.05 and -0.7, respectively. Note that the loading will follow the direction of the principal component. After running a varimax procedure, the rotated components will look as follows:
The following are the new loading on PC1 and PC2 after rotation:
- Variable A: -0.5 and 0.02
- Variable D: 0.5 and -0.3
- Variable E: 0.15 and -0.75
The loading have changed but the data points have not. With this simple illustration, we can't say that we have simplified the interpretation, but this should help you understand what is happening during the rotation of the principal components.