Importance sampling is a method to study one distribution while sampling from another. Essentially, it performs biased sampling to generate pseudorandom numbers more sparingly. For example, instead of taking the points from a uniform distribution, the points are chosen from a distribution that concentrates the points in the region of interest of the function being integrated, thus focusing on the most important samples. The formula for the integrand can be written as follows:

To perform importance sampling, we need to do the following:

For example, let's estimate the integral using this method.

Using the truncated exponential sampling distribution (λ = 0.65) truncated at T = 1, we can write:

Hence, becomes and the integral can be estimated as .

Now implement this in R as follows:

> set.seed(23564)
> n <- 100000
> T  <- 1
> lambda <- 0.65
> Rand.num <- runif(n,0,1)*(1-exp(-T*lambda))
> x = -log(1-Rand.num)/lambda
> f<-exp(-x)
> g<-lambda*exp(-lambda*x)/(1-exp(-T*lambda))
> fg<-f/g
> sum(fg)/n
[1] 0.6320664

Now let's see what we get using the standard Monte Carlo integration:

> set.seed(23564)
> X <- runif(100000, 0, 1)
> Y <- exp(-X)
> mean(Y)
[1] 0.6319162

Although both methods produce similar results, the estimate using the importance method is much closer to the true value of . Let's compare the variance of the two sample distributions using the following lines of code:

# Importance Sampling
> var(fg)
[1] 0.003894969
#Standard Monte Carlo Integration 
> var(Y)
[1] 0.03267516

As you can see, we achieve a much better estimation using the importance sampling method; we obtain 8.4 times less variance than with simple Monte Carlo integration.