No system—computer program or human—has a basis to reliably predict outcomes for new examples beyond those it observed during training. The only way out is to have some additional prior knowledge or make assumptions that go beyond the training examples. We covered a broad range of algorithms from Chapter 7, Linear Models and Chapter 8, Time Series Models, to non-linear ensembles in Chapter 10, Decision Trees and Random Forest and Chapter 11, Gradient Boosting Machines as well as neural networks in various chapters of part 4 of this book. 

We saw that a linear model makes a strong assumption that the relationship between inputs and outputs has a very simple form, whereas the models discussed later aim to learn more complex functions. While it's probably obvious that a simple model will fail in most circumstances, a complex model is not always better. If the true relationship is linear but the data is noisy, the complex model will learn the noise as part of the complex relationship that it assumes to exist. This is the basic idea behind the No Free Lunch Theorem that states no algorithm is universally superior for all tasks. Good fit in some instances comes at the cost of poor performance elsewhere.

The key tools to tailor the choice of the algorithm to the data are data exploration and experiments based on an understanding of the assumptions the model makes.