Part II

Integration and Simulation

Statisticians attempt to infer what is and what could be. To do this, we often rely on what is expected. In statistical contexts, expectations are usually expressed as integrals with respect to probability distributions.

The value of an integral can be derived analytically or numerically. Since an analytic solution is usually impossible for all but the simplest statistical problems, a numerical approximation is often used.

Numerical quadrature approximates the integral by systematically partitioning the region of integration into smaller parts, applying a simple approximation for each part, and then combining the results. We begin this portion of the book with coverage of quadrature techniques.

The Monte Carlo method attacks the problem by simulating random realizations and then averaging these to approximate the theoretical average. We describe these methods and explore a variety of strategies for improving their performance. Markov chain Monte Carlo is a particularly important simulation technique, and we devote two chapters to such methods.

Although we initially pose Monte Carlo methods as integration tools, it becomes increasingly apparent in these chapters that such probabilistic methods have broad utility for simulating random variates irrespective of how those simulations will be used. Our examples and exercises illustrate a variety of simulation applications.