Models should be designed as approximations to help us understand a particular problem, or a class of related problems. Models are not designed to be verbatim copies of the real world. Thus, all models are wrong in the same sense that maps are not the territory. Even when a priori, we consider every model to be wrong, not every model is equally wrong; some models will be better than others at describing a given problem. In the foregoing chapters, we focused our attention on the inference problem, that is, how to learn values of parameters from the data. In this chapter, we are going to focus on a complementary problem: how to compare two or more models that are used to explain the same data. As we will learn, this is not a simple problem to solve and at the same time is a central problem in data analysis.

In this chapter, we will explore the following topics:

• Posterior predictive checks
• Occam's razor—simplicity and accuracy
• Overfitting and underfitting
• Information criteria
• Bayes factors
• Regularizing priors