Conclusion

While it’s good to consider these questions carefully, you shouldn’t become obsessed with making the right choice. For most applications, a reasonable case can be made for two or more of these methods. Furthermore, the methods are so similar in their underlying philosophy that they usually give similar results. When they differ, it’s typically in situations where the evidence is not strong for any conclusion. If you have the time, it’s always worthwhile trying out two or more methods on the same data. If they lead you to the same conclusions, then your confidence is increased. If they are discrepant, your confidence is appropriately reduced. If they are widely discrepant, you should carefully investigate the reasons for the discrepancy. Perhaps you made an error in setting up one method or another. Maybe there’s a serious peculiarity in the data. Search for outliers and examine residuals and influence diagnostics. Discrepancies are often a powerful indication that there is something important to be learned.