There is a family of related methods, collectively known as MCMC methods. These stochastic methods allow us to get samples from the true posterior distribution as long as we are able to compute the likelihood and the prior point-wise. While this is the same condition that we need for the grid-approach, MCMC methods outperform the grid approximation. The is because MCMC methods are capable of taking more samples from higher-probability regions than lower ones. In fact, an MCMC method will visit each region of the parameter-space in accordance to their relative probabilities. If region A is twice as likely as region B, then we are going to get twice as many samples from A as from B. Hence, even if we are not capable of computing the whole posterior analytically, we could use MCMC methods to take samples from it.

At the most fundamental level, basically everything we care about  in statistics is about computing expectations like:

Here are some particular examples of this general expression:

• The posterior, equation 1.14
• The posterior predictive distribution, equation 1.17
• The marginal likelihood given a model, equation 5.13

With MCMC methods, we approximate equation 8.9 using finite samples:

The big catch with equation 8.10 is that the equality only holds asymptotically, that is, for an infinite number of samples! In practice, we always have a finite number of samples, thus we want the MCMC methods to converge to the right answer as quickly as possible—with the least possible number samples (also known as draws).

In general, being sure that a particular sample from an MCMC has converged is not easy, to put it mildly. Thus, in practice, we must rely on empirical tests to make sure we have a reliable MCMC approximation. We will discuss such tests for MCMC samples in the Diagnosing samples section. It is important to keep in mind that other approximations (including the non-Markovian methods discussed in this chapter) also need empirical tests, but we will not discuss them as the focus of this book is MCMC methods.

Having a conceptual understanding of MCMC methods can help us to diagnose samples from them. So, let me ask, what's in a name? Well, sometimes not much, sometimes a lot. To understand what MCMC methods are, we are going to split the method into the two MC parts; the Monte Carlo part and the Markov Chain part.