Quasi-likelihood functions and properties

The term quasi-likelihood function was initially introduced by Wedderburn (1974). While providing a way of allowing for overdispersion in statistical analysis, quasi-likelihood estimating equations are mostly applied for grouped binary or count data. In theories underlying this model, the quasi-likelihood function is shown to have similar properties to the log-likelihood function. Briefly, in this perspective, only relationship between the mean and the variance is specified in the form of a variance function.

Suppose we have independent observations Y_i (i = 1, …, N) with expectation μ_i and variances V(μ_i), where V(·) is some known function. Let each observation μ_i be some known function of parameters $\beta =({\beta }_1,\mathrm{...},{\beta }_M)\prime$. Then for each observation the quasi-likelihood function, denoted by $Q(Y_i,{\mu }_i)$, is defined by the relation

$\frac{\partial Q(Y_i,{\mu }_i)}{\partial {\mu }_i}=\frac{Y_i-{\mu }_i}{V({\mu }_i)}\text{,}$

(C.1)

$\frac{\partial }{\partial {\mu }_i}Q(Y_i,{\mu }_i)=\text{.}{\int }_{}^{{\mu }_i}\frac{Y_i-{\mu }_i^{\prime }}{V({\mu }_i^{\prime })}\text{d}{\mu }_i^{\prime }+\text{function}\text{of}{Y}_i\text{.}$

For analytic convenience, the subscript i will be dropped in the succeeding texts, so that Y and μ will refer to an observation and its expectation, respectively.

Following immediately from the definition of $Q$, then $Q$ has the following first characteristic:

$\text{E}\left(\frac{\partial Q}{\partial \mu }\right)=0\text{.}$

(C.2)

By applying the chain rule, we have

$\frac{\partial Q}{\partial {\beta }_m}=\left(\frac{\partial Q}{\partial \mu }\right)\left(\frac{\partial \mu }{\partial {\beta }_m}\right)\text{,}$

$\text{E}\left(\frac{\partial Q}{\partial {\beta }_m}\right)=0\text{.}$

(C.3)

Given V(μ) = var(Y), we have

$\begin{array}{ll}\text{E}\left(\frac{\partial Q\partial Q}{\partial {\beta }_m\partial {\beta }_{m^{\prime }}}\right)\hfill & =\text{E}{\left(\frac{\partial Q}{\partial \mu }\right)}^2\frac{\partial \mu }{\partial {\beta }_m}\frac{\partial \mu }{\partial {\beta }_{m^{\prime }}}\hfill \\ \hfill & =\text{E}\left\{\frac{{(Y-\mu )}^2}{{[V(\mu )]}^2}\right\}\frac{\partial \mu }{\partial {\beta }_m}\frac{\partial \mu }{\partial {\beta }_{m^{\prime }}}\hfill \\ \hfill & =\frac{1}{V(\mu )}\frac{\partial \mu }{\partial {\beta }_m}\frac{\partial \mu }{\partial {\beta }_{m^{\prime }}}\hfill \end{array}\text{.}$

(C.4)

Also, the second partial derivative of the quasi-likelihood function with respect to β can be written by

$\begin{array}{ll}-\text{E}\left(\frac{{\partial }^{2}Q}{\partial {\beta }_m\partial {\beta }_{m^{\prime }}}\right)\hfill & =-\text{E}\left\{\frac{\partial }{\partial {\beta }_{m^{\prime }}}\left[\frac{Y-\mu }{V(\mu )}\frac{\partial \mu }{\partial {\beta }_m}\right]\right\}\hfill \\ \hfill & =-\text{E}\left\{(Y-\mu )\frac{\partial }{\partial {\beta }_{m^{\prime }}}\left[\frac{1}{V(\mu )}\frac{\partial \mu }{\partial {\beta }_m}\right]-\frac{1}{V(\mu )}\frac{\partial \mu }{\partial {\beta }_m}\frac{\partial \mu }{\partial {\beta }_{m^{\prime }}}\right\}\hfill \\ \hfill & =\frac{1}{V(\mu )}\frac{\partial \mu }{\partial {\beta }_m}\frac{\partial \mu }{\partial {\beta }_{m^{\prime }}}.\hfill \end{array}$

(C.5)

Let l denote the log-likelihood and the distribution of Y be specified in terms of μ. Then, from the Cramér–Rao inequality (Stuart and Ord, 1994), we have

$\mathrm{var}(Y)\ge -\frac{1}{\text{E}\left(\frac{{\partial }^{2}l}{\partial {\mu }^2}\right)}\text{.}$

(C.6)

Summarizing the above inferences, the quasi-likelihood $Q$ has the following properties:

$\begin{array}{ll}\text{Property}(\text{i}):\hfill & \text{E}\left(\frac{\partial Q}{\partial \mu }\right)=0,\hfill \\ \text{Property}(\text{ii}):\hfill & \text{E}\left(\frac{\partial Q}{\partial {\beta }_m}\right)=0,\hfill \\ \text{Property}(\text{iii}):\hfill & \text{E}{\left(\frac{\partial Q}{\partial \mu }\right)}^2=-\text{E}\left(\frac{{\partial }^{2}Q}{\partial {\mu }^2}\right)=\frac{1}{V(\mu )},\hfill \\ \text{Property}(\text{iv}):\hfill & \text{E}\left(\frac{\partial Q\partial Q}{\partial {\beta }_m\partial {\beta }_{m^{\prime }}}\right)=-\text{E}\left(\frac{{\partial }^{2}Q}{\partial {\beta }_m\partial {\beta }_{m^{\prime }}}\right)=\frac{1}{V(\mu )}\frac{\partial \mu }{\partial {\beta }_m}\frac{\partial \mu }{\partial {\beta }_{m^{\prime }}},\hfill \\ \text{Property}(\text{v}):\hfill & -\text{E}\left(\frac{{\partial }^{2}Q}{\partial {\mu }^2}\right)\le -\text{E}\left(\frac{{\partial }^{2}l}{\partial {\mu }^2}\right),\hfill \end{array}$

where Property (iii) is a special case of (iv), and for a one-parameter exponential family, the inequality in Property (v) becomes equality. Wedderburn (1974) mathematically proves that the precision of maximum quasi-likelihood estimates can be estimated from the expected second derivatives of $Q$ in the same fashion as the precision of maximum likelihood estimates is estimated from the expected second derivatives of the log likelihood. In practice, using the Newton–Raphson method with the expected second derivatives of $Q$ to calculate $\widehat{\beta }$ is equivalent to iteratively calculating a weighted linear regression of the residuals on the quantities of the scores by linear least squares, with the residuals and the score being calculated from the current estimate of $\widehat{\beta }$.