A simple linear regression is a model that can be used to predict and/or explain one variable from another one. Using machine learning language, this is a case of supervised learning. From a probabilitic perspective, a linear regression model is an extension of the Gaussian model where the mean is not directly estimated but rather computed as a linear function of a predictor variable and some additional parameters. While the Gaussian distribution is the most common choice for the dependent variable, we are free to choose other distributions. One alternative, which is especially useful when dealing with potential outliers, is the Student's t-distribution. In the next chapter, we will explore other alternatives.

In this chapter, we also discussed the Pearson correlation coefficient, the most common measure of linear correlation between two variables, and we learned to compute a Bayesian version of it from the data and posterior predictive samples using a multivariate Gaussian distribution. One useful way to expand a linear regression model is by doing a hierarchical version of it, which provides the benefits of shrinkage. This is very simple to achieve with PyMC3. We also briefly discussed the importance of not interpreting correlation as causation, at least in the absence of a mechanistic model. As surprising as it may sound, we can use linear models to fit curves. We showed this with two examples; a polynomial regression, and by taking the square root of the independent variable. Another extension for simple linear regression is to deal with more than one independent variable with what is usually called multiple linear regression. Some precautions are necessary to avoid errors and problems interpreting these type of models, and we used a few examples to demonstrate this. Other ways to use the linear motif is to model interactions, and yet another one is to deal with a non-constant variance for the dependent variable.