4.2.1. MLE bias in variance estimate in general linear models

${\widehat{\sigma }}^2=\left\{\begin{array}{l}\frac{\sum _{i=1}^N{\left(Y_i-\mu \right)}^2}{N}\hspace{0.17em}\hspace{0.17em}\text{if}\hspace{0.17em}\mu \hspace{0.17em}\text{is}\hspace{0.17em}\text{known}\\ \frac{\sum _{i=1}^N{\left(Y_i-\overline{Y}\right)}^2}{N-1}\hspace{0.17em}\hspace{0.17em}\text{if}\hspace{0.17em}\mu \hspace{0.17em}\text{is}\hspace{0.17em}\text{unknown}\end{array}\right\}.$

The above equation indicates that when μ is unknown there is one degree of freedom lost in computing the sample mean $\overline{Y}$. Such a loss in the degree of freedom needs to be corrected for, deriving an unbiased variance estimate. The corrected variance estimate given unknown μ is statistically defined as the mean square error.

$Y=X^{\prime }\beta +e\text{,}$

where e ∼ N(0,Σ) and Σ is defined as an N × N positive-definite variance–covariance matrix of random errors. The matrix Σ depends on the unknown parameters organized into a vector $\tilde{\Gamma }$, and given the specification of β, this matrix is often simplified as σ²I, where I is the identity matrix, assuming random errors to be conditionally independent in the presence of the fixed-effect parameters. For Σ = σ²I, $\tilde{\Gamma }\equiv {\sigma }^2$. It follows that the classical MLE of σ², given the observed outcome data y, is given by

${\widehat{\sigma }}^2=\frac{{\left(y-X^{\prime }\widehat{\beta }\right)}^{\prime }\left(y-X^{\prime }\widehat{\beta }\right)}{N}\text{.}$

(4.6)

If the population mean μ is unknown, β needs to be estimated from the data first for deriving the sample mean $X^{\prime }\widehat{\beta }$. Given M degrees of freedom lost in $\widehat{\beta }$, the expectation of σ² becomes

$\text{E}\left({\widehat{\sigma }}^2\right)=\frac{N-M}{N}{\sigma }^2\text{,}$

(4.7)

By contradiction, suppose $\text{E}\left({\widehat{\sigma }}^2\right)={\sigma }^2$, given by

$\text{E}\left[\frac{{\left(y-X^{\prime }\widehat{\beta }\right)}^{\prime }\left(y-X^{\prime }\widehat{\beta }\right)}{N}\right]={\sigma }^2\text{.}$

(4.8)

$\text{E}\left[\frac{{\left(y-X^{\prime }\widehat{\beta }\right)}^{\prime }\left(y-X^{\prime }\widehat{\beta }\right)}{N}\right]=\frac{N-M}{N}{\sigma }^2<{\sigma }^2=\text{E}\left[\frac{{\left(y-X^{\prime }\widehat{\beta }\right)}^{\prime }\left(y-X^{\prime }\widehat{\beta }\right)}{N}\right]\text{.}$

(4.9)

Obviously, Equation (4.9) contradicts the condition that $\text{E}\left({\widehat{\sigma }}^2\right)={\sigma }^2$. Therefore, the ML estimator for ${\sigma }^2$ is biased downward due to failure to account for loss of the degrees of freedom in the estimation of μ given the estimates of β. It is also clear from Equation (4.9) that if N is large and M is relatively small, bias in MLE ${\widehat{\sigma }}^2$ becomes negligible and thus can be overlooked.

4.2.2. Specification of REML in general linear models

As indicated earlier, MLE of ${\sigma }^2$ is biased downward without an adjustment factor (N − M)/N. For making a correction on this bias, the REML approach first specifies an N-dimensional vector containing only ones, denoted by 1_N. The distribution of Y then can be written by $\text{N}\left(\mu 1_N,{\sigma }^2{\text{I}}_N\right)$ (notice that 1 and the identity matrix I are two different concepts). Let $\stackrel{⌢}{A}=\left\{{\stackrel{⌢}{a}}_1,\mathrm{...},{\stackrel{⌢}{a}}_{N-1}\right\}$ be any N × (N − 1) matrix with N − 1 linearly independent columns orthogonal to the vector $1_N$. It follows that an N × 1 vector of error contrasts is created, defined as $\stackrel{⌢}{U}={\stackrel{⌢}{A}}^{\prime }Y$. By definition, $\stackrel{⌢}{U}$ follows a normal distribution with mean 0 and covariance matrix ${\sigma }^2{\stackrel{⌢}{A}}^{\prime }\stackrel{⌢}{A}$. Maximizing the corresponding likelihood with respect to the only remaining parameter ${\sigma }^2$ gives

${\widehat{\sigma }}^2=\frac{Y^{\prime }\stackrel{⌢}{A}{\left({\stackrel{⌢}{A}}^{\prime }\stackrel{⌢}{A}\right)}^{-1}\stackrel{⌢}{A}\prime Y}{\left(N-1\right)}\text{.}$

(4.10)

Equation (4.10) can be readily adapted to the estimation of ${\sigma }^2$ in general linear models. Given a linear regression $Y=X^{\prime }\beta +e,$ where Y is an N-dimensional vector and X is an N × M matrix with known covariate values. All elements in e are assumed to be independent and identically distributed (iid) with zero expectation and variance ${\sigma }^2$. Given the inclusion of the covariates, $\stackrel{⌢}{A}=\left\{{\stackrel{⌢}{a}}_1,\mathrm{....},{\stackrel{⌢}{a}}_{N-M}\right\}$ is now any N × (N − M) matrix with N − M linearly independent columns orthogonal to the columns of the design matrix X. Accordingly, in general linear models, the vector of error contrasts $\stackrel{⌢}{U}$ is an (N − M) × 1 vector, given by

$\stackrel{⌢}{U}=\left\{\begin{array}{l}{\stackrel{⌢}{u}}_1\\ ⋮\\ {\stackrel{⌢}{u}}_{N-M}\end{array}\right\}=\left\{\begin{array}{l}{{\stackrel{⌢}{a}}^{\prime }}_{1}y\\ ⋮\\ {{\stackrel{⌢}{a}}^{\prime }}_{N-M}y\end{array}\right\}={\stackrel{⌢}{A}}^{\prime }Y\text{,}$

(4.11)

where a given element in $\stackrel{⌢}{U}$ is ${{\stackrel{⌢}{a}}^{\prime }}_{i}y$, which, by definition, is an error contrast if $\text{E}\left({{\stackrel{⌢}{a}}^{\prime }}_{i}y\right)=0$, and thereby ${{\stackrel{⌢}{a}}^{\prime }}_{i}x=0^{\prime }$.

The vector $\stackrel{⌢}{U}$ follows a normal distribution with mean 0 and the newly structured covariance matrix ${\sigma }^2{\stackrel{⌢}{A}}^{\prime }\stackrel{⌢}{A}$, where ${\sigma }^2$ remains the only unknown parameter. Maximizing the corresponding likelihood with respect to ${\sigma }^2$ gives rise to the following estimator:

${\widehat{\sigma }}^2=\frac{{\left[y-X{\left(X^{\prime }X\right)}^{-1}{X}^{\prime }y\right]}^{\prime }\left[y-X{\left(X^{\prime }X\right)}^{-1}{X}^{\prime }y\right]}{N-M}\text{,}$

(4.12)

which is the mean square error in the context of general linear models, unbiased for ${\sigma }^2$. As a result, underestimation of MLE ${\sigma }^2$ is corrected. Therefore, the REML estimator is essentially a maximum likelihood approach on residuals. Additionally, from the specification $\stackrel{⌢}{U}={\stackrel{⌢}{A}}^{\prime }Y$, the following inference can be derived:

$\begin{array}{c}\stackrel{⌢}{U}={\stackrel{⌢}{A}}^{\prime }Y={\stackrel{⌢}{A}}^{\prime }\left(X^{\prime }\beta +e\right)\\ ={\stackrel{⌢}{A}}^{\prime }X\prime \beta +{\stackrel{⌢}{A}}^{\prime }e\\ =0+{\stackrel{⌢}{A}}^{\prime }e\\ ={\stackrel{⌢}{A}}^{\prime }e,\end{array}$

where ${\stackrel{⌢}{A}}^{\prime }e\sim N\left(0,{\stackrel{⌢}{A}}^{\prime }\Sigma \stackrel{⌢}{A}\right)$.

$l_R\left(\Sigma \left|\stackrel{⌢}{U}\right.\right)=-\frac{N-M}{2}\log \left(2\pi \right)-\frac{1}{2}\log \left|{\stackrel{⌢}{A}}^{\prime }\Sigma \stackrel{⌢}{A}\right|-\frac{1}{2}{\stackrel{⌢}{U}}^{\prime }{\left({\stackrel{⌢}{A}}^{\prime }\Sigma \stackrel{⌢}{A}\right)}^{-1}\stackrel{⌢}{U}\text{.}$

(4.13)

4.2.3. REML estimator in linear mixed models

The above REML estimator for general linear models can be easily extended to linear mixed models by following the same approach of error contrasts. Combining vectors of $Y_i$, $b_i$, and ${\varepsilon }_i$ and the matrices $X_i$ into Y, b, ɛ, and X leads to

$Y=X^{\prime }\beta +\mathrm{Zb}+e\text{,}$

where Z is the block diagonal matrix with blocks $Z_i$ on the main diagonal and zeroes off-diagonally. As formalized in Chapter 3, the marginal distribution of Y is normal with mean vector $X^{\prime }\beta$ and covariance matrix $V\left(R,G\right)$ with blocks $V_i$ on the main diagonal and zeroes off-diagonally. After some simplification, the classical log-likelihood function associated with MLE is

$l\left(G,R\right)=-\frac{N}{2}\log \left(2\pi \right)-\frac{1}{2}\log \left|V\right|-\frac{1}{2}\stackrel{⌢}{r}\prime V^{-1}\stackrel{⌢}{r}\text{,}$

(4.14)

where $\stackrel{⌢}{r}=y-X^{\prime }\widehat{\beta }$, defined as the vector of marginal residuals, which will be further discussed in Chapter 6.

Maximization of the above log-likelihood function on the data yields an estimate of β and $V\left(G,R\right)$. Analogous to the specification for general linear models, the diagonal elements in $V\left(G,R\right)$ are underestimated for small samples due to loss of the degrees of freedom in estimation of the marginal mean $X^{\prime }\beta$ first. By extending the REML estimator for general linear models, such underestimation in variance estimates can be corrected.

The REML estimator for the variance components R and G in linear mixed models can be obtained from maximizing the log-likelihood function of a set of contrasts $\stackrel{⌢}{U}={\stackrel{⌢}{A}}^{\prime }Y$, where $\stackrel{⌢}{A}$ is any [N × (N − M)] full-rank matrix with columns orthogonal to the columns of the X matrix. The $\stackrel{⌢}{U}$ follows a normal distribution with mean vector 0 and covariance matrix ${\stackrel{⌢}{A}}^{\prime }V\left(R,G\right)\stackrel{⌢}{A}$ that integrates β out. After some algebra, the joint restricted log-likelihood function with respect to parameter vectors R and G for a random sample of size N is

$l_R\left(G,R\right)=-\frac{N-M}{2}\log \left(2\pi \right)-\frac{1}{2}\log \left|V\right|-\frac{1}{2}\stackrel{⌢}{r}\prime V^{-1}\stackrel{⌢}{r}-\frac{1}{2}\log \left|X^{\prime }{V}^{-1}X\right|\text{,}$

(4.15)

$\begin{array}{c}\stackrel{⌢}{r}=y-X^{\prime }\widehat{\beta }\\ =y-X{\left(X^{\prime }{V}^{-1}X\right)}^{-1}{X}^{\prime }{V}^{-1}y.\end{array}$

Maximization of the above residual log-likelihood function on the transformed data of error contrasts yields an unbiased estimate of $V\left(G,R\right)$. In turn, Equation (3.23) can be applied to estimate the vector β (Fitzmaurice et al., 2004).

$-\frac{1}{2}\log \left|X^{\prime }{V}^{-1}X\right|=\frac{1}{2}\log \left|{\left(X^{\prime }{V}^{-1}X\right)}^{-1}\right|=\log {\left|\mathrm{cov}\left(\widehat{\beta }\right)\right|}^{\frac{1}{2}}\text{,}$

which is the covariance of $\widehat{\beta }$. This additional term can be regarded as an adjustment factor for bias in $V\left(G,R\right)$ from MLE. Correspondingly, the above restricted log-likelihood function can be expressed in terms of MLE plus an adjustment factor, given by

$l_R\left(G,R\right)\approx l\left(G,R\right)+\log {\left|\mathrm{cov}\left(\widehat{\beta }\right)\right|}^{\frac{1}{2}}\text{.}$

(4.16)

4.2.4. Justification of the restricted maximum likelihood method

There is some debate concerning the validity of the REML estimator. At the first glance, some information seems lost by basing inferences of G and R on l_R rather than on l. Patterson and Thompson (1971) contend that for any vector of N − M linearly independent error contrasts, the log-likelihood function for the transformed data vector $\stackrel{⌢}{U}=\stackrel{⌢}{A}\prime Y$ is proportional to the log-likelihood function for y, so that inferences on $\stackrel{⌢}{A}\prime Y$ are valid as on y. It actually makes no difference which N − M linearly independent contrasts are used because the log-likelihood function for any such set differs by no more than an additive constant (Harville, 1977).

Let ${{\stackrel{⌢}{a}}^{\prime }}_{i}y$ be a linear combination of error contrasts such that $\text{E}\left({{\stackrel{⌢}{a}}^{\prime }}_{i}y\right)=0$ and ${{\stackrel{⌢}{a}}^{\prime }}_{i}x=0^{\prime }$ and the density of $\stackrel{⌢}{U}$ be $f_{\stackrel{⌢}{U}}\left({\stackrel{⌢}{A}}^{\prime }Y\left|G,R\right.\right)$. Given the familiar statistical expression of β in linear regression models

$\beta ={\left(X^{\prime }{V}^{-1}X\right)}^{-1}{X}^{\prime }{V}^{-1}Y\text{,}$

$\widehat{\beta }={\stackrel{⌢}{H}}^{\prime }y\text{,}$

(4.17)

where $\stackrel{⌢}{H}$ is defined as

$\stackrel{⌢}{H}=V^{-1}X{\left(X^{\prime }{V}^{-1}X\right)}^{-1}\text{.}$

(4.18)

$\begin{array}{c}\left|\text{det}\left(\stackrel{⌢}{A},\stackrel{⌢}{H}\right)\right|={\left\{\text{det}\left[{\left(\stackrel{⌢}{A},\stackrel{⌢}{H}\right)}^{\prime }\left(\stackrel{⌢}{A},\stackrel{⌢}{H}\right)\right]\right\}}^{\frac{1}{2}}\\ ={\left[\text{det}\left(1\right)\right]}^{\frac{1}{2}}{\left[\text{det}\left({\stackrel{⌢}{H}}^{\prime }\stackrel{⌢}{H}-{\stackrel{⌢}{H}}^{\prime }\stackrel{⌢}{A}{1}^{-1}{\stackrel{⌢}{A}}^{\prime }\stackrel{⌢}{H}\right)\right]}^{\frac{1}{2}}\\ ={\left[\text{det}\left(X^{\prime }X\right)\right]}^{-\frac{1}{2}}.\end{array}$

(4.19)

Let $f_{\widehat{\beta }}\left(\cdot \left|\beta ,G,R\right.\right)$ and $f_y\left(\cdot \left|\beta ,G,R\right.\right)$ be the probability density functions (p.d.f.) of $\widehat{\beta }$ and y, respectively. Given the well-known relationship

${\left(y-X^{\prime }\beta \right)}^{\prime }{V}^{-1}\left(y-X^{\prime }\beta \right)={\left(y-X^{\prime }\widehat{\beta }\right)}^{\prime }{V}^{-1}\left(y-X^{\prime }\widehat{\beta }\right)+{\left(\beta -\widehat{\beta }\right)}^{\prime }\left(X^{\prime }{V}^{-1}X\right)\left(\beta -\widehat{\beta }\right)\text{,}$

the Bayes-type expression about the likelihood of $\stackrel{⌢}{U}$ is then given by

$\begin{array}{c}{f}_{\stackrel{⌢}{U}}\left({\stackrel{⌢}{A}}^{\prime }y\left|G,R\right.\right)=\int f_{\stackrel{⌢}{U}}\left({\stackrel{⌢}{A}}^{\prime }y\left|G,R\right.\right)f_{\widehat{\beta }}\left({\stackrel{⌢}{H}}^{\prime }y\left|\beta ,\right.G,R\right)\text{d}\beta \\ ={\left[\text{det}\left(X^{\prime }X\right)\right]}^{\frac{1}{2}}\int f_y\left(y\left|\beta ,\right.G,R\right)\text{d}\beta \\ ={\left(2\pi \right)}^{-\frac{1}{2}\left(N-M\right)}{\left[\text{det}\left(X^{\prime }X\right)\right]}^{\frac{1}{2}}{\left[\text{det}\left(V\right)\right]}^{-\frac{1}{2}}{\left[\text{det}\left(X^{\prime }{V}^{-1}X\right)\right]}^{-\frac{1}{2}}\\ \exp \left[-\frac{1}{2}{\left(y-X^{\prime }\widehat{\beta }\right)}^{\prime }{V}^{-1}\left(y-X^{\prime }\widehat{\beta }\right)\right].\end{array}$

(4.20)

Compared to Patterson and Thompson’s expression, Equation (4.20) provides a more convenient perspective to derive the likelihood equations, thereby being preferable for numerical computation of the density $f_{\stackrel{⌢}{U}}\left({\stackrel{⌢}{A}}^{\prime }y\left|G,R\right.\right)$.

Let $f\left(G,R\right)$ be the prior p.d.f. for the variance–covariance components G and R. When only the error contrasts are used, the posterior probability density for G and R is then written as

$p\left(G,R\left|y\right.\right)\propto .f\left(G,R\right)f_{\stackrel{⌢}{U}}\left({\stackrel{⌢}{A}}^{\prime }y\left|G,R\right.\right)\text{.}$

(4.21)

Equation (4.21) literally states that assuming β and the variance components $\left(G,R\right)$ to be statistically independent and the components of β to be independent and identically distributed, the posterior density for $\left(G,R\right)$ is proportional to the product of the prior density for $\left(G,R\right)$ and the likelihood function of an arbitrary set of (N −M) linearly independent error contrasts. Therefore, from the standpoint of Bayesian inference, using only error contrasts to make inference on $\left(G,R\right)$ is equivalent to ignoring any prior information on β and using all the data to make these inferences. In other words, the REML estimator is identical to the mode of the marginal posterior density for $\left(G,R\right)$, formally integrating β out.

4.2.5. Comparison between ML and REML estimators

In the application of linear mixed models, the REML estimator is often considered preferable over MLE as it corrects for loss of the degrees of freedom in MLE of the fixed effects (Davidian and Giltinan, 1995; Fitzmaurice et al., 2004; Harville, 1977). If the sample size is sufficiently larger than the number of model parameters, however, the bias in MLE is negligible, and in such situations, the two estimators usually yield very close parameter estimates. Harville (1977) compares the size of the estimated variances derived from the two estimators in the application of general linear models. The ML estimator of the variance ${\sigma }^2$ consistently yields smaller mean square errors than the REML estimator when M = rank(X) ≤ 4; by contrast, the REML estimator of the variance ${\sigma }^2$ has smaller mean square errors than MLE when M = rank(X) ≥ 5 and the size of N − M is sufficiently large. Therefore, when more fixed effects are specified, the difference between the ML and REML estimators of variance widens.

4.3.1. Newton–Raphson algorithm

The NR algorithm is an iterative method for finding estimates for the parameters by minimizing −2 times a specific log-likelihood function. In applying this algorithm, both ML and REML log-likelihood functions can be used to estimate the variance components (Laird and Ware, 1982; Ware, 1985). The first step of the NR algorithm is to simplify the computational procedure by solving the ML and REML estimates of ${\sigma }^2$ as a function of β, G, and R. In statistics, the procedure for reducing the dimension of an objective function by analytic substitution is referred to as profiling. In the NR algorithm, profiling is applied to ensure that the numerical optimization can be performed only over the model parameters. Given quations (3.22) and (4.15), the estimate of ${\sigma }^2$ can be simplified by

${\widehat{\sigma }}_{\text{ML}}^2\left(\beta ,G,R\right)=\frac{1}{N}{r}^{\prime }V{\left(G,R\right)}^{-1}r\text{,}$

(4.22)

${\widehat{\sigma }}_{\text{REML}}^2\left(\beta ,G,R\right)=\frac{1}{N-M}{r}^{\prime }V{\left(G,R\right)}^{-1}r\text{,}$

(4.23)

where $N=\sum _{i=1}^N{n}_i$ and $r=y-X^{\prime }\beta$, both defined earlier.

$P_{\text{ML}}\left(\beta ,G,R\left|y\right.\right)=-\frac{N}{2}\log \left(r^{\prime }V{\left(G,R\right)}^{-1}r\right)-\frac{1}{2}\log \left|V\left(G,R\right)\right|\text{,}$

(4.24)

$\begin{array}{c}{P}_{\text{REML}}\left(\beta ,G,R\left|y\right.\right)=-\frac{N-M}{2}\log \left(r^{\prime }V{\left(G,R\right)}^{-1}r\right)-\frac{1}{2}\log \left|V\left(G,R\right)\right|\\ -\frac{1}{2}\log \left|x^{\prime }V{\left(G,R\right)}^{-1}x\right|\text{,}\end{array}$

(4.25)

where $P_{\text{ML}}\left(\cdot \right)$ and $P_{\text{REML}}\left(\cdot \right)$ are the profile log-likelihood functions for computing the ML and the REML estimates, respectively. According to Lindstrom and Bates (1988), the NR optimization can be based either on the original log-likelihood or the profile log-likelihood function, but the latter is recommended as the profile log-likelihood usually requires fewer iterations for deriving estimates for β, G, and R.

In the NR algorithm, a series of formulas are specified for the derivatives of the profile log-likelihoods in Equations (4.24) and (4.25) and with respect to β, G, and R. These formulations enable an NR implementation. The derivatives for linear mixed models are computed with the conditionally independent errors, given by $V_i={\sigma }^2{V}_i\left(G,R\right)$. The chain rule (a rule in mathematics for differentiating compositions of functions) is used to find the derivatives for a specific mixed-effects model with respect to the transformed G and R. The inverse of the second derivatives of the profile log-likelihood function at iterative convergence yields an estimate for the marginal variance–covariance matrix over the parameter space. If the residual variance ${\sigma }^2$ is a part of the mixed model, it can be profiled by analytically solving for the optima ${\sigma }^2$ and then plugging this expression back into the likelihood formula (Wolfinger et al., 1994). The exact procedures for these formulas are complex, and the reader interested in the computational details concerning the first and second derivatives of $P_{\text{ML}}\left(\beta ,G,R\left|y\right.\right)$ and $P_{\text{REML}}\left(\beta ,G,R\left|y\right.\right)$ is referred to the original articles in this regard (Lindstrom and Bates, 1988; Wolfinger et al., 1994).

Operationally, the estimation of G and R can be performed through a conventional NR iterative scheme, given some specified initial values for the parameter estimates. With regard to linear mixed models, the series of parameter estimates in the iterative scheme is generally denoted by ${\widehat{\theta }}^{\tilde{j}}$ $(\ddot{j}=1,2,\dots )$, where θ consists of β, G, and R. The iterative scheme terminates when ${\widehat{\theta }}^{\tilde{j}+1}$ is sufficiently close to ${\widehat{\theta }}^{\tilde{j}}$ given some statistical criterion. As a result, the maximum likelihood estimate of θ can be operationally defined as $\widehat{\theta }={\widehat{\theta }}^{\tilde{j}+1}$.

4.3.2. Expectation–maximization algorithm

${\widehat{b}}_i=\widehat{G}{Z^{\prime }}_i{\widehat{V}}_i^{-1}\left(y_i-{X^{\prime }}_i\widehat{\beta }\right)\text{.}$

(4.26)

In Equation (4.26), $V_i$ is the overall variance–covariance matrix of $y_i$, and its inverse, $V_i^{-1}$, is often used as a weight matrix in the estimation process to yield efficient and robust parameter estimates. In this equation, ${\widehat{b}}_i$ is computed as the proportion of the overall variance–covariance matrix that comes from the between-subjects variability times the overall residual from the marginal mean. Therefore, given the availability of $\widehat{G}$ and ${\widehat{V}}_i$, ${\widehat{b}}_i$ can be readily computed. This estimator is efficient and robust because the fixed-effects estimate maximizes the likelihood based on the marginal distribution of the empirical data. The inference of Equation (4.26) will be further presented in Section 4.4.

Let $R_i={\sigma }^2\text{I}$ and G is an arbitrary q × q nonnegative-definite covariance matrix. The variance ${\sigma }^2$ can then be written as

${\widehat{\sigma }}^2=\frac{\sum _{i=1}^N{e^{\prime }}_i{e}_i}{\sum _{i=1}^N{n}_i}=\frac{{\tilde{t}}_1}{\sum _{i=1}^N{n}_i}\text{,}$

(4.27)

$\widehat{G}=N^{-1}\sum _{i=1}^N{b^{\prime }}_i{b}_i=\frac{{\tilde{t}}_2}{N}\text{,}$

(4.28)

where $e_i$ is an n_i × 1 column vector of model residuals for subject i, as defined in Chapter 3. According to Laird and Ware (1982), the two terms of sum of squares, ${\tilde{t}}_1=\sum _{i=1}^N{e^{\prime }}_i{e}_i$ and ${\tilde{t}}_2=\sum _{i=1}^N{b^{\prime }}_i{b}_i$, are the sufficient statistics for R_i and G, respectively.

Let Θ be an available estimate of R_i and G (notice that Θ differs from θ). The statistics ${\tilde{t}}_1$ and ${\tilde{t}}_2$ can then be calculated by using their expectations conditionally on the observed data vector y, given by

$\begin{array}{c}{\widehat{\tilde{t}}}_1=\text{E}\left\{\sum _{i=1}^N{e^{\prime }}_i{e}_i\left|y_i,\widehat{\beta }\left(\widehat{\Theta }\right),\widehat{\Theta }\right.\right\}\\ =\sum _{i=1}^N\left\{{\widehat{e}}_i{\left(\widehat{\Theta }\right)}^{\prime }{\widehat{e}}_i\left(\widehat{\Theta }\right)+\text{tr}\text{var}\left[e_i\left|y_i,\widehat{\beta }\left(\widehat{\Theta }\right),\widehat{\Theta }\right.\right]\right\},\end{array}$

(4.29)

$\begin{array}{c}\widehat{\tilde{t}}2=\text{E}\left\{\sum _{i=1}^N{b^{\prime }}_i{b}_i\left|y_i,\widehat{\beta }\left(\widehat{\Theta }\right),\widehat{\Theta }\right.\right\}\\ =\sum _{i=1}^N\left\{{\widehat{b}}_i{\left(\widehat{\Theta }\right)}^{\prime }{\widehat{b}}_i\left(\widehat{\Theta }\right)+\text{var}\left[b_i\left|y_i,\widehat{\beta }\left(\widehat{\Theta }\right),\widehat{\Theta }\right.\right]\right\},\end{array}$

(4.30)

${\widehat{e}}_i\left(\widehat{\Theta }\right)=\text{E}\left(e_i\left|y_i,\widehat{\beta }\left(\widehat{\Theta }\right),\widehat{\Theta }\right.\right)=y_i-{X^{\prime }}_i\widehat{\beta }\left(\widehat{\Theta }\right)-Z_i{b}_i\left(\widehat{\Theta }\right)\text{.}$

Operationally, the EM algorithm requires suitable starting values for ${\widehat{R}}_i$ and $\widehat{G}$ and then iterate between Equations (4.29) and (4.30), referred to as the E-step (expectation step), and between Equations (4.27) and (4.28), referred to as the M-step (maximization step). At convergence of values in ${\tilde{t}}_1$ and ${\tilde{t}}_2$ after a series of iterations, the ML estimates of both the fixed and the random effects, ${\widehat{\Theta }}^{\text{ML}}$, $\widehat{\beta }\left({\widehat{\Theta }}^{\text{ML}}\right)$, and $\widehat{b}\left({\widehat{\Theta }}^{\text{ML}}\right)$, can be obtained from the last E-step.

The EM algorithm can also be applied to compute the REML estimates, denoted by ${\widehat{\Theta }}^{\text{REML}}$, $\widehat{\beta }\left({\widehat{\Theta }}^{\text{REML}}\right)$, and ${\widehat{b}}_i\left({\widehat{\Theta }}^{\text{REML}}\right)$, respectively, through an empirical Bayes approach. In the EM iterative series for the REML estimator, the M-step remains the same as iterating between Equations (4.27) and (4.28) because the estimation of Θ is still based on y, β, and e. The E-step in the REML estimator, however, differs from the procedure in MLE. While the E-step for MLE depends on y and β, the expectation with REML is only conditional on y as β is integrated out of the likelihood. Consequently, in REML Equations (4.29) and (4.30) are replaced with the following formulas:

$\begin{array}{c}\widehat{\tilde{t}}1=\text{E}\left\{\sum _{i=1}^N{e^{\prime }}_i{e}_i\left|y_i,\widehat{\Theta }\right.\right\}\\ =\sum _{i=1}^N\left\{{\widehat{e}}_i{\left(\widehat{\Theta }\right)}^{\prime }{\widehat{e}}_i\left(\widehat{\Theta }\right)+\text{tr}\text{var}\left[e_i\left|y_i,\widehat{\Theta }\right.\right]\right\},\end{array}$

(4.31)

$\begin{array}{c}\widehat{\tilde{t}}2=\text{E}\left\{\sum _{i=1}^N{b^{\prime }}_i{b}_i\left|y_i,\widehat{\Theta }\right.\right\}\\ =\sum _{i=1}^N\left\{{\widehat{b}}_i{\left(\widehat{\Theta }\right)}^{\prime }{\widehat{b}}_i\left(\widehat{\Theta }\right)+\text{var}\left[b_i\left|y_i,\widehat{\Theta }\right.\right]\right\},\end{array}$

(4.32)

where ${e^{\prime }}_i\left(\widehat{\Theta }\right)$ is still defined as $y_i-{X^{\prime }}_i\widehat{\beta }\left(\widehat{\Theta }\right)-{Z^{\prime }}_i{b}_i\left(\widehat{\Theta }\right)\text{.}$

The expectations computed in the above E-step in the REML estimator involve the conditional means and variances of $b_i$ and $e_i$. Theoretically, the variance estimates obtained from REML can differ from their ML counterparts that are biased downward, as indicated in Section 4.2. For large samples, however, the bias in the ML estimates tends to be negligible, and therefore, the ML and the REML estimates are equally robust if the sample size is sufficiently large.

4.4.1. Best linear unbiased prediction

$\widehat{\beta }={\left(\sum _{i=1}^N{X^{\prime }}_i{V}_i^{-1}{X}_i\right)}^{-1}\sum _{i=1}^N{X^{\prime }}_i{V}_i^{-1}{y}_i\text{,}$

${\widehat{b}}_i=\widehat{G}{Z^{\prime }}_i{\widehat{V}}_i^{-1}\left(y_i-{X^{\prime }}_i\widehat{\beta }\right)\text{.}$

In the second equation, ${\widehat{b}}_i$ is computed as the proportion of the overall variance–covariance matrix that comes from the between-subjects variability times the overall residual from the marginal mean. Given this perspective, the covariance matrix of ${\widehat{b}}_i$ as a parameter in a linear mixed model can be written as

$\mathrm{var}\left({\widehat{b}}_i\right)=\widehat{G}{Z^{\prime }}_i\left\{{\widehat{V}}_i^{-1}-{\widehat{V}}_i^{-1}{X}_i{\left(\sum _{i=1}^N{X^{\prime }}_i{\widehat{V}}_i^{-1}{X}_i\right)}^{-1}{X^{\prime }}_i{\widehat{V}}_i^{-1}\right\}{Z}_i\widehat{G}\text{.}$

(4.33)

If β, G, and R are known, the best linear predictor for ${X^{\prime }}_0\beta +{Z^{\prime }}_{0}b$, where $X_0$ and $Z_0$ are matrices with some specific values, is

${X^{\prime }}_0\beta +{Z^{\prime }}_{0}G{Z}^{\prime }{V}^{-1}\left(y-X^{\prime }\widehat{\beta }\right)\text{,}$

where $\widehat{\beta }$ can be viewed as, in the general term, any solution for the following generalized least squares (GLS) equation

$X^{\prime }{V}^{-1}{X}^{\prime }\widehat{\beta }=X^{\prime }{V}^{-1}y\text{.}$

$\left(\begin{array}{cc}\hfill \begin{array}{l}{X}^{\prime }{R}^{-1}X\\ \end{array}\hfill & \hfill \begin{array}{l}{X}^{\prime }{R}^{-1}Z\\ \end{array}\hfill \\ \hfill Z^{\prime }{R}^{-1}X\hfill & \hfill Z^{\prime }{R}^{-1}Z+G^{-1}\hfill \end{array}\right)\left(\begin{array}{c}\hfill \begin{array}{l}\widehat{\beta }\\ \end{array}\hfill \\ \hfill \widehat{b}\hfill \end{array}\right)=\left(\begin{array}{c}\hfill \begin{array}{l}{X}^{\prime }{R}^{-1}y\\ \end{array}\hfill \\ \hfill Z^{\prime }{R}^{-1}y\hfill \end{array}\right)\text{.}$

(4.36)

$X^{\prime }{R}^{-1}{X}^{\prime }\widehat{\beta }+X^{\prime }{R}^{-1}Z\widehat{b}=X^{\prime }{R}^{-1}\mathrm{y,}$

(4.37a)

$Z^{\prime }{R}^{-1}{X}^{\prime }\widehat{\beta }+\left(Z^{\prime }{R}^{-1}Z+G^{-1}\right)\widehat{b}=Z^{\prime }{R}^{-1}y\text{.}$

(4.37b)

Henderson (1975) proves that in Equation (4.36), $\widehat{\beta }$ is a solution to Equation (4.35) and $\widehat{b}$ is equal to $G{Z}^{\prime }{V}^{-1}\left(y-X^{\prime }\widehat{\beta }\right)$ of Equation (4.34). The advantage of applying Equation (4.36) over Equation (4.34) is that neither V nor its inverse are required in the computation as R is usually specified as an identify matrix and G is often a diagonal matrix.

$\text{E}\left(\begin{array}{c}\hfill \begin{array}{l}\widehat{\beta }-\beta \\ \end{array}\hfill \\ \hfill \widehat{b}-b\hfill \end{array}\right){\left(\begin{array}{c}\hfill \begin{array}{l}\widehat{\beta }-\beta \\ \end{array}\hfill \\ \hfill \widehat{b}-b\hfill \end{array}\right)}^{\prime }={\left(\begin{array}{cc}\hfill \begin{array}{l}{X}^{\prime }{R}^{-1}X\\ \end{array}\hfill & \hfill \begin{array}{l}{X}^{\prime }{R}^{-1}Z\\ \end{array}\hfill \\ \hfill Z^{\prime }{R}^{-1}X\hfill & \hfill Z^{\prime }{R}^{-1}Z+G^{-1}\hfill \end{array}\right)}^{-1}{\sigma }^2\text{.}$

(4.38)

In summary, the BLUP procedure maximizes the sum of two-component log-likelihoods given the joint distribution of y and b. In this expression, the log-likelihood is not a classical likelihood function in general linear models due to the specification of b. Instead, BLUP selects estimates, or more accurately predictors, of β, b, and σ² to maximize the log-likelihood function. For the continuous response data, the BLUP estimators, denoted by $\tilde{\beta }$ and $\tilde{b}$, are the values that make the derivatives of the log-likelihood function with respect to β and b equal 0 (McGilchrist, 1994).

Statistically, the BLUP estimates have desirable distributional properties and may differ from those derived from the generalized least squares estimator. In the context of linear mixed models, BLUP estimates are linear in the sense that they are linear functions of the data y; unbiased because the average value of the estimate is equal to the average value of the quantity being estimated, written by $\text{E}\left({X^{\prime }}_0\beta +{Z^{\prime }}_{0}b\right)={X^{\prime }}_0\beta$; best in view of the fact that they have minimum mean squared error within the class of linear unbiased estimators; and prediction as to distinguish the estimates from the estimation of the fixed effects (Henderson, 1975; Robinson, 1991). For more details regarding BLUP’s derivations, justifications, and links to the other statistical techniques, the reader is referred to Henderson (1950, 1975), Liu et al. (2008), McGilchrist (1994), and Robinson (1991).

4.4.2. Shrinkage and reliability

Equations (4.36) and (4.38) imply that the estimation of model parameters depends on the relative size of the random effects per subject because ${\widehat{b}}_i$ is equal to $G{Z^{\prime }}_i{V}_i^{-1}\left(y_i-{X^{\prime }}_i\widehat{\beta }\right)$. In the expression of ${\widehat{b}}_i$, the term $G{Z}^{\prime }{V}^{-1}$ is the proportion of the overall covariance matrix that comes from the random effect components (Verbeke and Molenberghs, 2000). Therefore, if between-subjects variability is relatively low, thereby indicating greater residuals, much weight will be given to the population predictor $X^{\prime }\widehat{\beta }$. In contrast, if between-subjects variability is relatively high, much weight will be bestowed to the observed data y. Consequently, the predicted outcome for subject i, denoted by ${\widehat{y}}_i$, can be regarded as a weighted average of the model-based population mean and the observed data for subject i. In statistics, this estimating technique is referred to as shrinkage, also called the borrowing of strength approach. In longitudinal data analysis, this technique not only plays an important role in the estimation of parameters in mixed-effects regression models, but it is also impactful in missing-data analysis and in the study on misspecification of the assumed distribution of predictions (McCulloch and Neuhaus, 2011).

The concept of shrinkage might sound somewhat unfamiliar to the reader not highly familiar with advanced statistics. Given the importance of shrinkage in longitudinal data analysis, it may be useful to link shrinkage with the reliability analysis of the random effects applied in statistical inference of multilevel regression modeling (Raudenbush and Bryk, 2002). The reliability of a measurement is defined as the ratio between the variance of true scores and the variance of observed data. In multilevel regression models, in which longitudinal data analysis can be regarded as a special case, reliability can be estimated by the ratio between the level-2 variance, denoted by ${\sigma }_{00}$, and the sum of the level-2 and the level-1 variance components, with the latter divided by the number of observations within a level-2 unit i. The equation is

${\rho }_{0i}=\frac{{\sigma }_{00}}{{\sigma }_{00}+{\sigma }^2\text{/}{n}_i}\text{,}$

(4.39)

where ${\rho }_{0i}$ is the reliability score for level-2 unit i, ${\sigma }^2$ is the level-1 variance, and n_i is the sample size within level-2 unit i. Clearly, the smaller the size of n_i, the lower the reliability for level-2 unit i. Given Equation (4.39), it is easy to calculate the reliability score of the random effects for each level-2 unit using the parameter estimates from a multilevel model.

$\begin{array}{c}{\widehat{y}}_i={X^{\prime }}_i\widehat{\beta }+{Z^{\prime }}_i{\widehat{b}}_i\\ ={X^{\prime }}_i\widehat{\beta }+Z_i\widehat{G}{Z^{\prime }}_i{{\widehat{V}}_i}^{-1}\left(y_i-{X^{\prime }}_i\widehat{\beta }\right)\\ =\left(I_{n_i}-Z_i\widehat{G}{Z^{\prime }}_i{{\widehat{V}}_i}^{-1}\right){X^{\prime }}_i\widehat{\beta }+Z_i\widehat{G}{Z^{\prime }}_i{{\widehat{V}}_i}^{-1}{y}_i\\ ={\widehat{R}}_i{V_i}^{-1}{X^{\prime }}_i\widehat{\beta }+\left(I_{n_i}-{\widehat{R}}_i{{\widehat{V}}_i}^{-1}\right)y_i\text{.}\end{array}$

(4.40)

In the above BLUP estimation, prediction of the outcomes for subject i is expressed as a weighted average of the population mean ${X^{\prime }}_i\widehat{\beta }$ and the observed data y_i, with weights $R_i{V_i}^{-1}$ and $I_{n_i}-R_i{{\Sigma }_i}^{-1}$, respectively. When within-subject variability is relatively sizable, the observed data are shrunk toward the prior average ${X^{\prime }}_i\widehat{\beta }$ because the weight component $I_{n_i}-R_i{{\Sigma }_i}^{-1}$ is relatively low.

$\text{BLUP}\left({\widehat{b}}_i\right)=\frac{{\widehat{\sigma }}_b^2}{{\widehat{\sigma }}_b^2+{\widehat{\sigma }}_{\varepsilon }^2/n_i}\left(y_i-{X^{\prime }}_i\widehat{\beta }\right)\text{,}$

(4.41)

where the term ${\widehat{\sigma }}_b^2/\left({\widehat{\sigma }}_b^2+{\widehat{\sigma }}_{\varepsilon }^2/n_i\right)$ is referred to as the shrinkage factor in longitudinal data analysis. By the inclusion of the shrinkage factor, the BLUP of b_i has mean 0 and variance $\left[{\widehat{\sigma }}_b^2/\left({\widehat{\sigma }}_b^2+{\widehat{\sigma }}_{\varepsilon }^2/n_i\right)\right]{\widehat{\sigma }}_b^2$. Therefore, the variance of $\text{var}\left|\text{BLUP}\left({\widehat{b}}_i\right)\right|<\mathrm{var}\left(b_i\right)$.

${\widehat{y}}_i={X^{\prime }}_i\widehat{\beta }+\frac{{\widehat{\sigma }}_b^2}{{\widehat{\sigma }}_b^2+{\widehat{\sigma }}_{\varepsilon }^2/n_i}\left(y_i-{X^{\prime }}_i\widehat{\beta }\right)\text{.}$

(4.42)

$\text{BLUP}\left({\widehat{y}}_i\right)=y_i-\frac{{\widehat{\sigma }}_b^2}{{\widehat{\sigma }}_b^2+{\widehat{\sigma }}_{\varepsilon }^2/n_i}\left(y_i-{X^{\prime }}_i\widehat{\beta }\right)\text{.}$

(4.43)

In the above random intercept model, the prediction of y_i depends on ${\sigma }_{\varepsilon }^2$, ${\sigma }_b^2$, and $n_i$. Other things being equal, the number of observations for the subject determines the amount of shrinkage in linear estimation and prediction. In other words, if $n_i$ is small, within-subject variability is relatively large, and consequently, the observed data are more severely shrunk toward the population average ${X^{\prime }}_i\widehat{\beta }$. This simple case can be readily extended to BLUP predictions in the linear random coefficient model.