Some remarks

Now, we will briefly discuss some key facts about the marginal likelihood. By carefully inspecting the definition of marginal likelihood, we can understand their properties and consequences for their practical use:

  • The good: Models with more parameters have a larger penalization than models with fewer parameters. Bayes factor has a built-in Occam's Razor! The intuitive reason for this is that the larger the number of parameters, the more spread the prior with respect to the likelihood. Thus, when computing the integral in the preceding formula, you will get a smaller value with a more concentrated prior. 
  • The bad: Computing the marginal likelihood is, generally, a hard task since the preceding formula is an integral of a highly variable function over a high dimensional parameter space. In general, this integral needs to be solved numerically using more or less sophisticated methods.
  • The ugly: The marginal likelihood depends sensitively on the values of the priors.

Using the marginal likelihood to compare models is a good idea because a penalization for complex models is already included (thus preventing us from overfitting). At the same time, a change in the prior will affect the computations of the marginal likelihood. At first, this sounds a little bit silly—we already know that priors affect computations (otherwise, we could simply avoid them), but the point here is the word sensitively. We are talking about changes in the prior that will keep the inference of θ more or less the same, but could have a big impact on the value of the marginal likelihood. You may have noticed in the previous examples that, in general, having a normal prior with a standard deviation of 100 is the same as having one with a standard deviation of 1,000. Instead, Bayes factors will be affected by these kind of changes.

Another source of criticism regarding Bayes factors is that they can be used as a Bayesian way of doing hypothesis testing. There is  nothing wrong with this per se, but many authors have pointed out that an inference approach, similar to the one used in this book and other books like Statistical Rethinking by Richard McElreath, is better suited to most problems than the hypothesis testing approach (whether Bayesian or not).

Having said all this, let's look at how we can compute Bayes factors.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.19.63.106