Once the support for the prior distribution has been determined, we need to decide what the actual shape of the prior will look like (by choosing the right parameters). For example, if we think that a certain prior should be a Gaussian distribution, it really matters whether the mean is 4 or 20. In general, these are tuned so the peak/mode of the prior is very near the value that we have in mind. For example, if we expect the promotion effect on sales to be equal to 2, we should put a prior that has a mode near 2.

However, it's not just the mode, but also the asymmetry and the variance. For example, if we think that this promotional variable has an impact around two, but we are equally unsure whether it should be 1.8 or 2.2, we should choose a symmetric distribution centered at 2 (see the left side of the following screenshot). On the other hand, if we think that large values such as four and six are quite likely, we might want to choose an asymmetric one.

The following screenshot shows two gamma options that we could use for non-negative priors: 

Bear in mind that when we choose a normal prior, we cannot control the asymmetry (since it is equal to zero), but we can control the variance. The way Gaussian distributions are assigned is that we first think of the expected value that we think is correct, and we put the sigma in order to reflect our certainty about it. Here we have two examples, both centered at 2: variance = 0.5, which means that we are quite certain that the parameter should be near 2 (left), and variance = 2, which means that we are not very sure about it (see the right side of the preceding screenshot).

The following screenshot shows two Gaussian distributions: one with low variance (left), and another with high variance (right): 

So, what's the impact of the prior?

To begin with, it will define the support for the posterior density. A prior between zero and ∞ will generate a posterior in the (zero and ∞) area. The specific shape will depend on both the data, and the prior; but in general, for small datasets, the prior will dominate, whereas in big datasets, the data will dominate. In some way, the prior gets diluted as we add more data, and that makes perfect sense.

In the following example, we will load the house price dataset that we used in the previous recipe, and we will study how different priors impact the results.