The delta method

The delta method is a statistical approach for deriving an approximate probability distribution for a function of an asymptotically normal estimator using the Taylor series approximation.

Let x = be a random variable asymptotically distributed as N(μ, σ²) with p.d.f. f(x), and g(x) be a single-valued function. If the integral ${\int }_{-\infty }^{+\infty }g(x)\text{d}F$ exists, it is called the mathematical expectation of g(x), denoted by E[g(x)]. The mean and variance of g(x) of the random variable x may not be readily obtainable. If x has the standard normal density, the integral corresponding to E[g(x)] does not have a tractable expression. In these situations, approximate mean and variance for g(x) can be obtained by taking a linear approximation of g(x) in the neighborhood of μ. With μ = E(x) and g(x) being continuous, the Taylor series expansion of the function g(x) about μ is given by

$\text{g}(x)=\text{g}(\mu )+{\text{g}}^{\prime }(\mu )(x-\mu )+{\text{g}}^{″}(\mu )\frac{{(x-\mu )}^2}{2!}+\mathrm{...}\text{,}$

(B.1)

$\text{g}(x)\approx \text{g}(\mu )+{\text{g}}^{\prime }(\mu )(x-\mu )\text{.}$

(B.2)

Let g(x) = y about μ, and the g(x) can be expressed as

$y=\text{g}(x)\approx \text{g}(\mu )+{\text{g}}^{\prime }(\mu )(x-\mu )\text{.}$

(B.3)

Taking the variance of both sides of Equation (B.3) yields:

$V(Y)=V[\text{g}(x)]\approx {[{\text{g}}^{\prime }(x)]}^{2}V(x)\text{.}$

(B.4)

So, if Y, or g(x), is any function of a random variable x, only the variance of x and the first derivative of the function need to be considered in calculating the approximate for the variance of g(x).

The delta method for the multivariate case is simply the extension of the above specifications. Suppose that $x=(x_1,\mathrm{...},x_n)\prime$ is a random vector with mean μ and variance–covariance matrix $\Sigma (x)$, and g(x) is a vector-valued transform of x where g is the link function. Let $\widehat{x}={\left({\widehat{x}}_1,\mathrm{...},{\widehat{x}}_n\right)}^{\prime }$ and $\text{g}\left(\widehat{x}\right)$ be the estimates of x and g(x), respectively. Then, the first-order Taylor series expansion of $\text{g}\left(\widehat{x}\right)$ yields approximation of mean

$\text{E}[\text{g}(\widehat{x})]\approx \text{g}(\mu )\text{,}$

(B.5)

and the variance–covariance matrix $V\left[\text{g}\left(\widehat{x}\right)\right]$

$V[g(\widehat{x})]\approx {\left[\frac{\partial g(\widehat{x})}{\partial \widehat{x}}\left|\widehat{x}=\mu \right.\right]}^{\prime }\Sigma (\widehat{x})\left[\frac{\partial g(\widehat{x})}{\partial \widehat{x}}\left|\widehat{x}=\mu \right.\right]\text{,}$

(B.6)

$\frac{\partial g(x)}{\partial x}=\left[\frac{\partial g_1(x)}{\partial x},\frac{\partial g_2(x)}{\partial x},\mathrm{...}\right]\text{,}$

$\Sigma (\widehat{x})\left(\begin{array}{cccc}\hfill V({\widehat{x}}_1)\hfill & \hfill Cov({\widehat{x}}_1,{\widehat{x}}_2)\hfill & \hfill \cdots \hfill & \hfill Cov({\widehat{x}}_1,{\widehat{x}}_n)\hfill \\ \hfill \hfill & \hfill V({\widehat{x}}_2)\hfill & \hfill \cdots \hfill & \hfill Cov({\widehat{x}}_2,{\widehat{x}}_n)\hfill \\ \hfill \hfill & \hfill \hfill & \hfill \ddots \hfill & \hfill \hfill \\ \hfill \hfill & \hfill \hfill & \hfill \hfill & \hfill V({\widehat{x}}_n)\hfill \end{array}\right)\text{.}$

The delta method depends on the validity of the Taylor series approximation, and therefore, some caution must be exercised when using it before its adequacy is verified with simulation.