Sampling-based stochastic inference

Sampling is about drawing samples, X=(x1, ..., xn), from a given distribution, p(x). Assuming the samples are independent, the law of large numbers ensures that for a growing number of samples, the fraction of a given instance, xi, in the sample (for the discrete case) corresponds to its probability, p(x=xi). In the continuous case, the analogous reasoning applies to a given region of the sample space. Hence, averages over samples can be used as unbiased estimators of the expected values of parameters of the distribution.

A practical challenge consists in ensuring independent sampling because the distribution is unknown. Dependent samples may still be unbiased, but tend to increase the variance of the estimate so that more samples will be needed for an equally precise estimate as for independent samples.

Sampling from a multivariate distribution is computationally demanding as the number of states increases exponentially with the number of dimensions. Numerous algorithms facilitate the process (see references for an overview). Now, we will introduce a few popular variations of MCMC-based methods.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
100.26.135.252