9.1.1. Specifications of “naïve” model with independence hypothesis

Prior to the introduction of the classical GEE model, I begin with the specification of a “naïve” model that assumes repeated measurements of the response for the same subject to be conditionally independent given the specified fixed effects θ. Let $y_i=\left\{y_{i1}\text{,}\mathrm{....}\text{,}{y}_{i{n}_i}\right\}\prime$ be the n_i × 1 vector of the longitudinal data for subject i and $X_i=\left\{X_{i1}\text{,}\mathrm{....}\text{,}{X}_{i{n}_i}\right\}\prime$ be the n_i × M matrix of covariate values associated with subject i. From Equation (8.16), the marginal density of the outcome for subject i at time point j, denoted by y_ij, can be written as

$f\left(y_{ij}\left|{\theta }_{ij},\phi \right.\right)=\exp \left[\frac{y_{ij}{\theta }_{ij}-\stackrel{⌢}{a}({\theta }_{ij})}{\phi }+b(y_{ij};\phi )\right]\text{,}$

(9.1)

where, in the longitudinal setting, θ_ij is an element in vector θ_i, and functions $\stackrel{⌢}{a}(\cdot )$, b(·), and the scale factor φ are defined previously. According to Equations (8.4) and (8.5), the first two moments of y_ij are

$\text{E}(y_{ij})={\mu }_i=\frac{\partial \stackrel{⌢}{a}({\theta }_{ij})}{\partial {\theta }_{ij}}\text{,}$

(9.2)

$\mathrm{var}(y_{ij})=\frac{{\partial }^2\stackrel{⌢}{a}({\theta }_{ij})}{\partial {\theta }_{ij}^2\phi }\text{.}$

(9.3)

From the relevant description in Chapter 8, the score equation for subject i, denoted by ${\tilde{U}}_i(\beta )$ and mathematically defined as the first partial derivative of the log-likelihood function with respect to b, is written as

${\tilde{U}}_i(\beta )=\frac{\partial }{\partial \beta }{l}_i(\beta )\text{.}$

${\tilde{U}}_i(\beta )={\left(\frac{\partial {\mu }_i}{\partial \beta }\right)}^{\prime }\frac{\partial {\theta }_i}{\partial {\mu }_i}\frac{\partial l_i(\beta )}{\partial {\theta }_i}\text{.}$

$\frac{\partial l_i(\beta )}{\partial {\theta }_i}=y_i-\text{E}(Y_i)=y_i-{\mu }_i\text{,}$

$\frac{\partial {\mu }_i}{\partial {\theta }_i}=\mathrm{cov}(Y_i)\text{.}$

${\tilde{U}}_i(\beta )=\frac{\partial l_i(\beta )}{\partial \beta }={\left(\frac{\partial {\mu }_i}{\partial \beta }\right)}^{\prime }{V}_i^{-1}(Y_i)[y_i-{\mu }_i]\text{.}$

In GLMs, ${\theta }_i=X_i^{\prime }\beta$, so that we have

${\left(\frac{\partial {\mu }_i}{\partial \beta }\right)}^{\prime }=X_i^{\prime }{A}_i\text{,}$

where A_i is the n_i × n_i diagonal variance matrix evaluated at X_i, defined by Equation (8.32). Consequently, the maximum likelihood estimate $\widehat{\beta }$ is the solution to

$\begin{array}{ll}\tilde{U}(\beta )\hfill & =\sum _{i=1}^N{\left(\frac{\partial {\mu }_i}{\partial \beta }\right)}^{\prime }{V}_i^{-1}(Y_i)(y_i-{\mu }_i)\hfill \\ \hfill & =\sum _{i=1}^N{X}_i^{\prime }{A}_i(y_i-{\mu }_i)=0\text{,}\hfill \end{array}$

(9.4)

where ${\mu }_i=g^{-1}\left(X_i^{\prime }\beta \right)$ is specified as the conditional mean for subject i. The previous specifications do not differ significantly from a standard generalized linear model given the assumption that repeated measurements of the response for the same subject are conditionally independent with the specified model parameters of the fixed effects.

9.1.2. Basic specifications of GEEs

In the ordinary maximum likelihood estimator, the parameter estimates $\widehat{\beta }$ are considered to be consistent and asymptotically normal. In view of the large-sample asymptotic property, the variance of $\widehat{\beta }$ can be consistently estimated by the inverse of the observed information matrix given the hypothesis of conditional independence. In longitudinal data analysis, such an independence hypothesis can result in the loss of efficiency and robustness in the estimation of regression coefficients, particularly when intraindividual correlation is strong and some of the covariates are time-varying (Fitzmaurice, 1995). More serious, the inverse of the observed information matrix ${\widehat{I}}^{-1}(\widehat{\beta })$ does not provide an adequate variance–covariance matrix for $\widehat{\beta }$, thereby indicating an inefficient, biased variance estimator. Therefore, this GLM approach based on the independence hypothesis is referred to as the “naïve” variance estimator in longitudinal data analysis. For correcting the “naïve” approach, Liang and Zeger (1986) and Zeger and Liang (1986) proposed the GEE model.

According to Liang and Zeger (1986) and Zeger and Liang (1986), given the desirable large-sample property, the asymptotic limit of $\sqrt{N}(\widehat{\beta }-\beta )$ is 0 for large samples, where β is the true regression coefficient. This proposition is considered valid even in the presence of considerable dependence of longitudinal data. The authors contend, however, that the inverse of the observed information matrix ${\widehat{I}}^{-1}(\widehat{\beta })$ can result in inconsistent estimates of the asymptotic variance of $\widehat{\beta }$. Given the independence hypothesis, the variance estimates are less efficient because they are more widely scattered around the true population value than they will be when intraindividual correlation is considered (Diggle et al., 2002; Fitzmaurice, 1995). Therefore, a robust covariance matrix for $\widehat{\beta }$ needs to be developed to account for covariance in the clustered data. Accordingly, Liang and Zeger (1986) and Zeger and Liang (1986) create a robust variance–covariance estimator externally to take intraindividual correlation into account. Statistically, the classical GEE model does not specify the joint distribution of the subject’s observations; rather, it reduces to the score equations for multivariate normal outcomes (Liang and Zeger, 1986; Zeger and Liang, 1986).

Let $V_i({\mu }_i,\tilde{a})$ be an n_i × n_i variance–covariance matrix of y_i, defined as

$V_i({\mu }_i,b)=\phi A_i^{1/2}{R}_i(\tilde{a})A_i^{1/2}\text{,}$

(9.5)

where A_i is an n_i × n_i diagonal variance matrix containing elements ν, and R(ã) be an n_i × n_i matrix of $\tilde{\alpha }$ that reflects the pattern of correlation among observations for subject i, as also defined in Chapter 8. In GEEs, R(ã) is referred to as the working correlation matrix to address dependence of repeated measurements. The classical GEE model for the estimation of β starts with the form

${\tilde{U}}_I(\beta )=\sum _{i=1}^N{\left(\frac{\partial {\mu }_i}{\partial \beta }\right)}^{\prime }{V}_i{({\mu }_i,\tilde{a})}^{-1}[y_i-{\mu }_i(\beta )]=0\text{,}$

(9.6)

where ${\tilde{U}}_I(\beta )$ is the total score statistic specified for the classical GEE model (GEE1).

Let ${\widehat{\beta }}_{\text{GEE}}$ be a vector of the estimated regression coefficients from the classical GEE model. It follows then that $\sqrt{N}({\widehat{\beta }}_{\text{GEE}}-\beta )$ is asymptotically multivariate normal with mean 0 and variance–covariance matrix

$V_{\text{GEE}}({\widehat{\beta }}_{\text{GEE}})=I_0^{-1}{I}_1{I}_0^{-1}\text{,}$

(9.7)

$\begin{array}{l}{I}_0=\sqrt{N}\sum _{i=1}^N{\left(\frac{\partial {\mu }_i}{\partial \beta }\right)}^{\prime }{V}_i{({\mu }_i,\tilde{\alpha })}^{-1}\frac{\partial {\mu }_i}{\partial \beta },\hfill \\ I_1=\sqrt{N}\sum _{i=1}^N{\left(\frac{\partial {\mu }_i}{\partial \beta }\right)}^{\prime }{V}_i{({\mu }_i,\tilde{\alpha })}^{-1}\text{cov}(y_i)V_i{({\mu }_i,\tilde{\alpha })}^{-1}\left(\frac{\partial {\mu }_i}{\partial \beta }\right).\hfill \end{array}$

Equation (9.7) specifies the classical GEE, also referred to as the “sandwich” variance estimator, or GEE1, with $I_0^{-1}$ being the “bread” and I₁ the “meat.” In the equation, I₀ is the conventional, model-based Fisher information matrix, and in the context of longitudinal data analysis, its inverse, $I_0^{-1}$, is referred to as the “naïve” variance estimator given its reliance on the “naïve” assumption. The term I₁ is the covariance matrix of the score statistic, and its inclusion in Equation (9.7) accounts for intraindividual correlation in longitudinal data. Therefore, the sandwich variance estimator results in a robust variance–covariance matrix for longitudinal data analysis.

Given the empirical adjustment on dependence of repeated measurements for the same subject, longitudinal data can be adequately assumed to be conditionally independent given the specified fixed effects. As a result, the random vector $\sqrt{N}(\widehat{\beta }-\beta )$ is asymptotically normal with mean 0 and a covariance matrix that can be estimated by ${\widehat{V}}_{\text{GEE}}(\widehat{\beta })$ (Liang et al., 1992). With the specification of a robust variance–covariance matrix, the valid Wald score can be derived to perform hypothesis tests on parameter estimates.

There are some advantages of the GEE approach. First, the asymptotic variance–covariance matrix of $\widehat{\beta }$ from the “sandwich” approach does not depend on the choice of an estimator for $\tilde{\alpha }$ and φ (Liang and Zeger, 1986; Zeger and Liang, 1986). Second, when cov(y_i) is well approximated, $\widehat{\beta }$ is reasonably efficient (Liang et al., 1992). Third, when cov(y_i) is correctly specified, Equation (9.6) is the score equation for b that corresponds to a log linear model (Fitzmaurice and Laird, 1993).

In GEE1, $\widehat{\beta }$ can be solved by applying the Gauss–Newton method, in which one calculates a regression of residuals on the quantities of the scores with linear least squares (Wedderburn, 1974). Operationally, GEE1 uses an iterative process between a modified Fisher scoring for b and the moment estimation of $\tilde{\alpha }$ and φ (Liang and Zeger, 1986; Zeger and Liang, 1986). The series of $\widehat{\beta }$ in the iterative scheme is generally denoted by ${\widehat{\beta }}^{\ddot{j}}$ ($\ddot{j}=1,2,\dots \text{.}$). The iterative scheme terminates when ${\widehat{\beta }}^{\ddot{j}+1}$ is sufficiently close to ${\widehat{\beta }}^{\ddot{j}}$. Therefore, the GEE1 iterative procedure for $\widehat{\beta }$ can be written as

${\widehat{\beta }}^{\ddot{j}+1}={\widehat{\beta }}^{\ddot{j}}-{\left[\sum _{i=1}^N{\left(\frac{\partial {\mu }_i}{\partial {\widehat{\beta }}^{\ddot{j}}}\right)}^{\prime }{V}_i^{-1}({\widehat{\beta }}^{\ddot{j}})\frac{\partial {\mu }_i}{\partial {\widehat{\beta }}^{\ddot{j}}}\right]}^{\text{-1}}\left\{\sum _{i=1}^N{\left(\frac{\partial {\mu }_i}{\partial {\widehat{\beta }}^{\ddot{j}}}\right)}^{\prime }{V}_i^{-1}({\widehat{\beta }}^{\ddot{j}})\left[y_i-{\mu }_i({\widehat{\beta }}^{\ddot{j}})\right]\right\}\text{.}$

(9.8)

Given the iterative process, the maximum likelihood or weighted least squares estimate of b can be operationally defined as $\widehat{\beta }={\widehat{\beta }}^{\ddot{j}+1}$. Lipsitz et al. (1994) contend that in terms of bias, mean squared error, and power, a one-step estimator, obtained by performing a single step of the GEE model with the use of ${\widehat{\beta }}_{\text{GEE}}$ as the starting value, is qualitatively similar to that of the fully iterated estimator.

At a given iteration, the correlation parameters $\tilde{\alpha }$ and the scale parameter φ can be estimated from the standardized residuals, given by

${\widehat{\varepsilon }}_{ij}=\frac{\left[y_{ij}-\partial \stackrel{⌢}{a}({\widehat{\theta }}_{ij})\right]}{\sqrt{{\partial }^2\stackrel{⌢}{a}({\widehat{\theta }}_{ij})}}\text{,}$

where ${\widehat{\theta }}_{ij}$ depends on the current value for β. Given the specification of ${\widehat{\varepsilon }}_{ij}$, the scale φ can be estimated as

${\widehat{\phi }}^{-1}=\frac{\sum _{i=1}^N\sum _{j=1}^{n_i}{\widehat{\varepsilon }}_{ij}^2}{N-M}\text{,}$

(9.9)

where, in longitudinal data analysis, $N=\sum n_i$. Given the choice of R(ã), ã can be estimated by a simple function of the equation

${\widehat{R}}_{j^{\prime }{j^{\prime }}^{\prime }}=\frac{\sum _{i=1}^N{\widehat{\varepsilon }}_{i{j}^{\prime }}{\widehat{\varepsilon }}_{i{j^{\prime }}^{\prime }}}{N-M}\text{.}$

(9.10)

To summarize, the classical GEE is a marginal regression model, in which the regression coefficients of covariates and the intraindividual correlation parameters are modeled separately (Liang and Zeger, 1986; Zeger and Liang, 1986). The estimation of each of the parameters b, $\tilde{\alpha }$, and φ depends on the other two, obtained operationally by an iterative procedure. Practically, the operational process of GEE1 consists of the following steps. First, fit a standard, univariate generalized linear regression model based on the independence hypothesis on repeated measurements of the response for the same subject (the so-called naïve model). Second, use the residuals from the naïve model to estimate the correlation parameters for repeated measurements within the subject. Third, fit the regression model again using a modified algorithm that consists of a correlation matrix estimated from the second step. Finally, iterate between the second and the third steps until the estimates stabilize given a specified statistical criterion.

9.1.3. Specifications of working correlation matrix

The second scenario is to specify the working correlation matrix as $R_i(\tilde{a})=\rho ;j\ne s$, where j and s are time points. This specification is referred to as the “exchangeable” correlation structure because values of y_i are assumed to vary equally across observations for the same subject. In this specification, ã is a scalar and is estimated by the regression. In continuous longitudinal normal data, this scenario is analogous to the specification of CS in the random intercept linear model.

The third scenario is given by $R_i(\tilde{a})={\rho }^{\left|j-s\right|};j\ne s$. This correlation pattern is analogous to the AR(1) correlation matrix if time intervals are equally spaced or to the SP(POW) covariance pattern model if time intervals are unequally spaced. In this specification, intraindividual correlation is patterned as an exponential function of the lag length. By an extension of this scenario, higher-order autoregressive structures can also be specified.

Finally, the fourth scenario specifies an unstructured working correlation structure, defined as $R_i(\tilde{a})={\tilde{a}}_{sj};j\ne s$. In this specification, no constraints are assumed on the pattern of intraindividual correlation. As described in Chapter 5, such a pattern model of correlation can be estimated without restriction on the correlation structure. Correspondingly, ã is specified as an n_i × n_i matrix containing [n_i × (n_i − 1)]/2 unique pair-wise correlation coefficients for all combinations of time points.

There are many other possible correlation structures, and in some special cases, ã can even be specified as depending on subject-specific covariates. For example, one can specify the working correlation matrix as ${\text{g}}^{-1}[R_i(\tilde{a})]=Z_i^{\prime }\tilde{a}$, where g(·) is a given link function and Z_i is a subset of subject-specific covariates. As they are rarely used in empirical analyses, those complex working correlation matrices are not further described in this text. For details concerning other working correlation matrices, the interested reader is referred to Fitzmaurice et al. (1993) and Liang and Zeger (1986).

Theoretically, the selection of a working correlation matrix closer to the true correlation structure can yield increased statistical efficiency and consistency (Fitzmaurice, 1995). Liang and Zeger (1986) and Zeger and Liang (1986) prove, however, that one can obtain a consistent and asymptotically Gaussian estimate $\widehat{\beta }$ or a consistent estimate of V_R for large samples, even if the working correlation matrix R(ã) is incorrectly specified. According to those scientists (1986), in longitudinal data analysis inferences about the regression coefficients are generally robust to misspecification of the model on dependence of observations due to the desirable large-sample properties.

9.1.4. Quasi-likelihood information criteria for GEEs

Let each observation μ_ij be some known function of parameters $\beta =({\beta }_1,\mathrm{....},{\beta }_M)\prime$. It follows then that the quasi-likelihood function for each observation is denoted by $Q(Y_{ij},{\mu }_{ij})$. With the independence working correlation (scenario 1), the quasi-likelihood evaluated with the parameter estimates is defined as

$Q[\beta (R),\phi ]=\sum _{i=1}^N\sum _{j=1}^{n_i}Q[\widehat{\beta }(R),\phi ;(Y_{ij},X_{ij})]\text{.}$

(9.11)

Let ${\widehat{V}}_R$ be the robust covariance estimate of R and $\widehat{\Omega }$ be the model-based covariance estimate under the independence working correlation evaluated at $\widehat{\beta }(R)$. The quasi-likelihood information criterion, or QIC(R), is then given by

$\text{QIC}(R)=-2Q[\beta (R),\phi ]+2\text{trace}({\widehat{\Omega }}^{-1}{\widehat{V}}_R)\text{.}$

(9.12)

When the GEE specifications are correct, ${\widehat{V}}_R$ and $\widehat{\Omega }$ are asymptotically equivalent, thereby leading to the approximation that $\text{trace}({\widehat{\Omega }}^{-1}{\widehat{V}}_R)\approx \tilde{M}$, where $\tilde{M}$ is the number of regression parameters. Given this property, Pan (2001) also develops an approximation to QIC(R), written as

$\text{QI}{\text{C}}_{\text{u}}(R)=-2Q[\beta (R),\phi ]+2\tilde{M}\text{,}$

(9.13)

9.2.1. Prentice’s GEE approach

Prentice (1988) expanded the classical GEE model based on the argument that the original GEE1 is not designed to estimate the marginal response probability or the pair-wise correlations. Therefore, a second set of estimating equations should be added to the GEE modeling to specify parameters of the correlation matrix with a binary data structure. Correspondingly, a sample correlation vector for subject i is proposed, denoted by ${\tilde{Z}}_i=({\tilde{Z}}_{i12},{\tilde{Z}}_{i13},\mathrm{....},{\tilde{Z}}_{i,n_{i-1},n_i})\prime$, consisting of time occasions (j_1, j₂). A typical element in vector ${\tilde{Z}}_i$ is defined as

${\tilde{Z}}_{ij1.j2}=\frac{(y_{ij1}-{\mu }_{ij1})(y_{ij2}-{\mu }_{ij2})}{\sqrt{{\mu }_{ij1}(1-{\mu }_{ij1})}{\mu }_{ij2}(1-{\mu }_{ij2})}=\frac{(y_{ij1}-{\mu }_{ij1})(y_{ij2}-{\mu }_{ij2})}{\sqrt{{\tilde{\pi }}_{ij1}{\tilde{q}}_{ij1}{\tilde{\pi }}_{ij2}{\tilde{q}}_{ij2}}}\text{,}$

where, in the case of the binary data taking value 1 or 0, ${\tilde{\pi }}_{ij}=\text{pr}(y_{ij}=1\left|X_i,\beta \right.)=\text{E}(y_{ij})$ and ${\tilde{q}}_{ij}=1-{\tilde{\pi }}_{ij}$. By definition, ${\tilde{Z}}_{ij1.j2}$ has mean ρ_ij1.j2, which is the pair-wise correlation, and variance

${\tilde{w}}_{ij1.j2}=1+(1-2{\tilde{\pi }}_{ij1})(1-2{\tilde{\pi }}_{ij2})\sqrt{{\tilde{\pi }}_{ij1}{\tilde{q}}_{ij1}{\tilde{\pi }}_{ij2}{\tilde{q}}_{ij2}}{\rho }_{ij1.j2}-{\rho }_{j1.j2}^2\text{.}$

Prentice (1988) presents that given the specification of ${\tilde{Z}}_i$, a GEE estimator for b and ã can be defined as a solution to

$\sum _{i=1}^N{\left(\frac{\partial {\mu }_i}{\partial \beta }\right)}^{\prime }{V_i}^{-1}\left[y_i-p_i\left(\beta \right)\right]=0\text{,}$

(9.14)

$\sum _{i=1}^N{\tilde{E}}_i^{\prime }{\tilde{W}}_i^{-1}[{\tilde{Z}}_i{-}_i{\rho }_i]=0\text{,}$

(9.15)

where ${\rho }_i=({\rho }_{i12},\mathrm{....},{\rho }_{i1n_i},{\rho }_{i23},\mathrm{...})\prime$, ${\tilde{E}}_i=\partial {\rho }_i/\partial \tilde{\alpha }$, and ${\tilde{W}}_i=\text{diag}({\tilde{w}}_{i12},\mathrm{....},{\tilde{w}}_{i1n_i},{\tilde{w}}_{i23},\mathrm{...})$. In this GEE approach, $\tilde{W}$ is specified as an [n_i × (n_i − 1)/2]-dimensional square diagonal matrix used as the working correlation structure for ${\tilde{Z}}_i=({\tilde{Z}}_{i12},\mathrm{....},{\tilde{Z}}_{i1n_i},{\tilde{Z}}_{i23},\dots )\prime$. In this expansion, the third- and fourth-order correlations are set to be zero. As a result, the variance–covariance matrix of y_i, denoted by V_i, does not serve as the working correlation matrix in this GEE model. With the specification of the second-order correlations, Prentice’s expansion is also referred to as the higher-order independence working assumption (Molenberghs and Verbeke, 2010).

Given the expansions, the joint asymptotic distribution of $\sqrt{N}(\widehat{\beta }-\beta )$ and $\sqrt{N}(\widehat{\tilde{\alpha }}-\tilde{\alpha })$ is multivariate normal with mean 0 and a variance–covariance matrix that can be estimated consistently by

$N\times \left(\begin{array}{cc}\hfill A\hfill & \hfill 0\hfill \\ \hfill B\hfill & \hfill C\hfill \end{array}\right)\left(\begin{array}{cc}\hfill {\Lambda }_{11}\hfill & \hfill {\Lambda }_{12}\hfill \\ \hfill {\Lambda }_{21}\hfill & \hfill {\Lambda }_{22}\hfill \end{array}\right)\left(\begin{array}{cc}\hfill A\hfill & \hfill B^{\prime }\hfill \\ \hfill 0\hfill & \hfill C\hfill \end{array}\right)\text{,}$

(9.16)

$A={\left[\sum _{i=1}^N{\left(\frac{\partial {\mu }_i}{\partial \beta }\right)}^{\prime }{V}_i^{-1}\frac{\partial {\mu }_i}{\partial \beta }\right]}^{-1}\text{,}$

$\begin{array}{c}\hfill B={\left(\sum _{i=1}^N{{\tilde{E}}_i}^{\prime }{\tilde{W}}_i^{-1}{{\tilde{E}}_i}^{\prime }\right)}^{-1}\left(\sum _{i=1}^N{{\tilde{E}}_i}^{\prime }{\tilde{W}}_i^{-1}\frac{\partial {\tilde{Z}}_i}{\partial \beta }\right)\hfill \\ \hfill \times {\left[\sum _{i=1}^N{\left(\frac{\partial {\mu }_i}{\partial \beta }\right)}^{\prime }{V}_i^{-1}\frac{\partial {\mu }_i}{\partial \beta }\right]}^{-1}\text{,}\hfill \end{array}$

$C={\left(\sum _{i=1}^N{{\tilde{E}}_i}^{\prime }{\tilde{W}}_i^{-1}{{\tilde{E}}_i}^{\prime }\right)}^{-1}\text{,}$

${\Lambda }_{11}=\sum _{i=1}^N{\left(\frac{\partial {\mu }_i}{\partial \beta }\right)}^{\prime }{V}_i^{-1}\text{cov}(y_i)V_i^{-1}\text{,}$

${\Lambda }_{12}=\sum _{i=1}^N{\left(\frac{\partial {\mu }_i}{\partial \beta }\right)}^{\prime }{V}_i^{-1}\text{cov}(y_i,{\tilde{Z}}_i){\tilde{W}}_i^{-1}{\tilde{E}}_i\text{,}$

${\Lambda }_{21}={\Lambda }_{12}\text{,}$

${\Lambda }_{22}=\sum _{i=1}^N{{\tilde{E}}_i}^{\prime }{\tilde{W}}_i^{-1}\text{cov}({\tilde{Z}}_i){\tilde{W}}_i^{-1}{\tilde{E}}_i\text{.}$

The corresponding covariance matrices, cov(y_i), $\mathrm{cov}(y_i,{\tilde{Z}}_i)$, and $\mathrm{cov}({\tilde{Z}}_i)$, can be obtained by following the standard GEE procedure, given by

$\begin{array}{l}\mathrm{cov}(y_i)=(y_i-p_i){(y_i-p_i)}^{\prime },\hfill \\ \mathrm{cov}(y_i,{\tilde{Z}}_i)=(y_i-p_i){({\tilde{Z}}_i-{\rho }_i)}^{\prime },\hfill \\ \mathrm{cov}({\tilde{Z}}_i)=({\tilde{Z}}_i-{\rho }_i){({\tilde{Z}}_i-{\rho }_i)}^{\prime }.\hfill \end{array}$

By using the GLM estimates for b and ã as the starting values, a simple iterative procedure can be applied to produce statistically efficient, consistent, and robust results from GEEs. Prentice (1988) indicates that by formalizing the second-moment parameters in GEEs, the model can improve the quality of parameter estimates by drawing simultaneous inferences about the regression and the random components. Liang et al. (1992) present that the solutions to Equations (9.11) and (9.12) yield highly efficient estimates for b and ã. When the cluster size is high, as is often the case in multilevel analytic studies, the solution of Prentice’s approach is computationally difficult (Carey et al., 1993). For example, if n = 50, there will be 1275 × 1275 elements in the covariance matrix of y and $\tilde{W}$.

9.2.2. Zhao and Prentice’s GEE method (GEE2)

$\tilde{U}(\beta ,\tilde{a})={\tilde{D}}_i^{\prime }{\tilde{V}}_i^{-1}{\tilde{f}}_i=0\text{,}$

(9.17)

${\tilde{D}}_i^{\prime }=\left(\begin{array}{cc}\hfill \partial {\mu }_i/\partial \beta \hfill & \hfill 0\hfill \\ \hfill {\partial }_i{\tilde{a}}_i/\partial \beta \hfill & \hfill \partial {\rho }_i/\partial {\tilde{a}}_i\hfill \end{array}\right)\text{,}$

${\tilde{V}}_i=\left[\begin{array}{cc}\hfill \mathrm{var}(y_i)\hfill & \hfill \mathrm{cov}(y_i,{\tilde{Z}}_i)\hfill \\ \hfill \mathrm{cov}({\tilde{Z}}_i,y_i)\hfill & \hfill \mathrm{var}({\tilde{Z}}_i)\hfill \end{array}\right]\text{,}$

$\tilde{f}=\left(\begin{array}{c}\hfill y_i-{\mu }_i\hfill \\ \hfill {\tilde{Z}}_i-{\rho }_i\hfill \end{array}\right)\text{.}$

The previous GEE specification uses information about intracluster correlations for the improvement of efficiency to estimate b, ã, and the corresponding standard errors. If the first- and second-order models are correctly specified, b and ã can be computed by using a Fisher scoring algorithm. It follows then that the asymptotic process $\sqrt{N}(\widehat{\beta }-\beta )$ in this expanded GEE approach is asymptotically multivariate normal with mean 0 and variance–covariance matrix

$\widehat{V}(\widehat{\beta })={\left(\sum _{i=1}^N{\widehat{{\tilde{D}}^{\prime }}}_i{{\widehat{\tilde{V}}}_i}^{-1}{\widehat{\tilde{D}}}_i\right)}^{-1}\left(\sum _{i=1}^N{\widehat{{\tilde{D}}^{\prime }}}_i{{\widehat{\tilde{V}}}_i}^{-1}{\widehat{\tilde{f}}}_i{{\widehat{\tilde{f}}}_i}^{\prime }{{\widehat{\tilde{V}}}_i}^{-1}\widehat{\tilde{D}}\right){\left(\sum _{i=1}^N{\widehat{{\tilde{D}}^{\prime }}}_i{{\widehat{\tilde{V}}}_i}^{-1}{\tilde{D}}_{^i}\right)}^{-1}\text{.}$

(9.18)

9.2.3. GEE models on odds ratios

${\tilde{\pi }}_{ij{j}^{\prime }}=\text{pr}(Y_{ij}=1,Y_{i{j}^{\prime }}=1)=\text{E}(Y_{ij{j}^{\prime }})\text{,}$

$Y_{ij{j}^{\prime }}=\text{I}(Y_{ij}=1,Y_{i{j}^{\prime }}=1)=Y_{ij}{Y}_{i{j}^{\prime }}$

Given the previous specification, the authors of this work (1991) display the expression of the joint probability, denoted by ${\tilde{\pi }}_{ij{j}^{\prime }}$, as a function of ${\tilde{\pi }}_{ij}$, ${\tilde{\pi }}_{i{j}^{\prime }}$, and the OR between the responses at time points j and j′. By defining a vector of pair-wise marginal ORs for subject i, written by ${\Gamma }_i=({\gamma }_{i12},{\gamma }_{i13},\mathrm{....},{\gamma }_{i(J-1)J})$, the OR of a given probability pair at time points j and j′, respectively, can be expressed in terms of ${\tilde{\pi }}_{ij}$, ${\tilde{\pi }}_{i{j}^{\prime }}$, and ${\tilde{\pi }}_{ij{j}^{\prime }}$, given by

${\gamma }_{ij{j}^{\prime }}=\frac{{\tilde{\pi }}_{ij{j}^{\prime }}(1-{\tilde{\pi }}_{ij}-{\tilde{\pi }}_{i{j}^{\prime }}+{\tilde{\pi }}_{ij{j}^{\prime }})}{({\tilde{\pi }}_{ij}-{\tilde{\pi }}_{ij{j}^{\prime }})({\tilde{\pi }}_{i{j}^{″}}-{\tilde{\pi }}_{ij{j}^{\prime }})}\text{,}\hspace{0.17em}j\ne j^{\prime }\text{.}$

(9.19)

The specified OR is not constrained by the means, and therefore, in many situations, the OR is considered preferable to express the pair-wise correlations for binary data. The natural logarithm of the OR, denoted by $\log ({\gamma }_{ij{j}^{\prime }})$ and referred to as the log OR, is often used as a more convenient measurement of association. The log OR has desirable properties as it can take any value in the range (−∞,∞), with $\log ({\gamma }_{ij{j}^{\prime }})=0$ indicating no association between Y_ij and $Y_{i{j}^{\prime }}$. Given the log transformation on the OR, the association between the measurements at two data points can be interpreted in a conventional fashion, while not constrained by the means.

Given the quadratic formula, ${\tilde{\pi }}_{ij{j}^{\prime }}$ can be solved in terms of the OR ${\gamma }_{ij{j}^{\prime }}$ and the two marginal probabilities, ${\tilde{\pi }}_{ij}$, ${\tilde{\pi }}_{i{j}^{\prime }}$, given by

${\tilde{\pi }}_{ij{j}^{\prime }}=\left\{\begin{array}{ll}\frac{{\xi }_{ij{j}^{\prime }}-\sqrt{[{\xi }_{ij{j}^{\prime }}^2-4{\gamma }_{ij{j}^{\prime }}({\gamma }_{ij{j}^{\prime }}-1){\tilde{\pi }}_{ij}{\tilde{\pi }}_{i{j}^{\prime }}]}}{2({\gamma }_{ij{j}^{\prime }}-1)}\hfill & \text{for}({\gamma }_{ij{j}^{\prime }}\ne 1),\hfill \\ {\gamma }_{ij{j}^{\prime }}{\tilde{\pi }}_{ij}{\tilde{\pi }}_{i{j}^{\prime }}\hfill & \text{for}({\gamma }_{ij{j}^{\prime }}=1),\hfill \end{array}\right.$

(9.20)

where ${\xi }_{j{j}^{\prime }}=[1-(1-{\gamma }_{ij{j}^{\prime }})({\tilde{\pi }}_{ij}+{\tilde{\pi }}_{i{j}^{\prime }})]$. In Equation (9.20), ${\tilde{\pi }}_{ij{j}^{\prime }}={\tilde{\pi }}_{ij{j}^{\prime }}(\beta ,\tilde{\alpha })$ is a function of b and $\tilde{\alpha }$ in the GEE formulation, where $\tilde{\alpha }$, in this specific case, is the correlation parameter associated with the ORs.

In this GEE model, the [n_i × (n_i − 1)/2]-dimensional square diagonal matrix $\tilde{W}$, used as the working correlation matrix, is specified as

${\tilde{W}}_i=\text{diag}[\text{var}(Y_{ij{j}^{\prime }})]=\text{diag}[{\tilde{\pi }}_{i{j}^{\prime }}(1-{\tilde{\pi }}_{i{j}^{\prime }})]\text{.}$

Given the specification of $\tilde{W}$, the second set of estimating equations in terms of Prentice’s expansion is

$\tilde{U}(\tilde{a})=\sum _{i=1}^N{{\tilde{E}}_i}^{\prime }{\tilde{W}}_i^{-1}[{\tilde{Z}}_i{-}_i{\rho }_i(\beta ,\tilde{a})]=0\text{.}$

(9.21)

Assuming Y_i to be correctly specified, the joint process $\sqrt{N}(\widehat{\beta }-\beta )$ and $\sqrt{N}(\widehat{\tilde{\alpha }}-\tilde{\alpha })$ has an asymptotic distribution that is multivariate normal with mean vector 0 and variance–covariance matrix

${\tilde{V}}_i(\beta ,\tilde{a})=\underset{N\to \infty }{\lim }\left(\begin{array}{cc}\hfill B_{11}^{-1}\hfill & \hfill 0\hfill \\ \hfill B_{21}\hfill & \hfill B_{22}^{-1}\hfill \end{array}\right)\left(\begin{array}{cc}\hfill {\Sigma }_{11}\hfill & \hfill {\Sigma }_{12}\hfill \\ \hfill {\Sigma }_{21}\hfill & \hfill {\Sigma }_{22}\hfill \end{array}\right)\left(\begin{array}{cc}\hfill B_{11}^{-1}\hfill & \hfill B_{21}^{\prime }\hfill \\ \hfill 0\hfill & \hfill B_{22}^{-1}\hfill \end{array}\right)\text{,}$

(9.22)

$B_{11}=N^{-1}\sum _{i=1}^N{\left(\frac{\partial {\mu }_i}{\partial \beta }\right)}^{\prime }{V}_i^{-1}\frac{\partial {\mu }_i}{\partial \beta }\text{,}$

$B_{22}=N^{-1}\sum _{i=1}^N{{\tilde{E}}_i}^{\prime }{\tilde{W}}_i^{-1}{{\tilde{E}}_i}^{\prime }\text{,}$

$B_{21}=B_{22}^{-1}\left[N^{-1}\sum _{i=1}^N{{\tilde{E}}_i}^{\prime }{\tilde{W}}_i^{-1}(-\partial {\rho }_i/\partial \tilde{\alpha })\right]B_{11}^{-1}\text{,}$

${\Sigma }_{11}=N^{-1}\sum _{i=1}^N{\left(\frac{\partial {\mu }_i}{\partial \beta }\right)}^{\prime }{V}_i^{-1}[\text{cov}(y_i)]V_i^{-1}\left(\frac{\partial {\mu }_i}{\partial \beta }\right)\text{,}$

${\Sigma }_{22}=N^{-1}\sum _{i=1}^N{{\tilde{E}}_i}^{\prime }{\tilde{W}}_i^{-1}[\text{cov}({\tilde{Z}}_i)]{\tilde{W}}_i^{-1}{\tilde{E}}_i\text{,}$

${\Sigma }_{12}=N^{-1}\sum _{i=1}^N{\left(\frac{\partial {\mu }_i}{\partial \beta }\right)}^{\prime }{V}_i^{-1}[\text{cov}(y_i,{\tilde{Z}}_i)]{\tilde{W}}_i^{-1}{\tilde{E}}_i\text{.}$

As a typical GEE model, the asymptotic variance–covariance matrix of $\sqrt{N}(\widehat{\beta }-\beta )$ in this approach is a sandwich estimator, given by

$V(\beta )=\underset{N\to \infty }{\lim }(B_{11}^{-1}{\Sigma }_{11}{B}_{11}^{-1})\text{.}$

(9.23)

$\begin{array}{ll}{\gamma }_{ij{j}^{\prime }}\hfill & =\frac{\text{pr}(Y_{ij}=1,Y_{i{j}^{\prime }}=1)\text{/pr}(Y_{ij}=1,Y_{i{j}^{\prime }}=0)}{\text{pr}(Y_{ij}=0,Y_{i{j}^{\prime }}=1)/\text{pr}(Y_{ij}=0,Y_{i{j}^{\prime }}=0)}\hfill \\ \hfill & =\frac{\text{pr}(Y_{ij}=1,Y_{i{j}^{\prime }}=1)\text{pr}(Y_{ij}=0,Y_{i{j}^{\prime }}=0)}{\text{pr}(Y_{ij}=1,Y_{i{j}^{\prime }}=0)\text{pr}(Y_{ij}=0,Y_{i{j}^{\prime }}=1)}\text{.}\hfill \end{array}$

(9.24)

Equation (9.22) can be applied to measure the degree of association between two measurements. For example, if the OR defined in the equation is one, there is no association between Y_ij and $Y_{i{j}^{\prime }}$. Based on this algorithm, some more complex combinations of the correlated binary data can be modeled by an extension of the two-moment perspective. The reader interested in those GEE algorithms is referred to Carey et al. (1993), Fitzmaurice and Laird (1993), and Liang et al. (1992).

9.3.1. Comparison between the two approaches

In the longitudinal setting, the marginal models are sometimes referred to as population-averaged regression approaches, with GEEs being a typical paradigm in this regard. When the response data are categorical, the marginal models are thought to be useful, with the main focus being on the difference in the transformed normal response between two population subgroups. In this application, population subgroups are identified by different values of covariates. With regard to dependence of longitudinal data, the GEE models account for intraindividual correlation by adjusting the covariance matrix of repeated measurements for the same subject, thereby achieving conditional independence of observations. As a result, the regression coefficients $\widehat{\beta }$ estimated for GEEs are robust, providing reliable statistical information about the effects of covariates on the transformed normal response variable. Nevertheless, only in some special situations can the estimated regression coefficients from GEEs translate into interpretable results on the response of a nonlinear scale.

Given the different analytic focuses, the interpretation of the regression coefficients differs markedly between the conditional and the marginal perspectives in longitudinal data analysis. For the conditional models, the regression coefficient of a covariate indicates the change in the transformed response variable (e.g., log odds) with a one-unit increase in the covariate within a subject. In the population-averaged approach, the regression coefficient of the covariate represents an average effect on the linear predictor, which cannot directly be retransformed to the population-averaged effect on the untransformed scale. For example, in logistic regression models, the mean of the difference in the log-odds of the response probability is not equal to the difference between two means due to variability in the inherent random effects. These two sets of coefficients are equal if and only if intraindividual correlation is zero. In the general case, where the random effects follow a normal or a multivariate normal distribution, the absolute value of a given regression coefficient in the marginal model, termed $\left|{\beta }_{\text{M}}\right|$, is smaller than the corresponding coefficient in the conditional model, termed $\left|{\beta }_{\text{C}}\right|$. In the analysis of nonlinear longitudinal data, the coefficient β_C can be retransformed to an interpretable effect on the nonlinear response by a complex retransformation process, whereas its marginal counterpart, β_M, cannot. In the succeeding chapters, the conditional longitudinal models on different data types will be described, in which the issue regarding the interpretation of regression coefficients will be further discussed.

9.3.2. Use of GEEs to fit a conditional model

$\text{E}(y_i)=\int g^{-1}(X_i^{\prime }\beta +Z_i^{\prime }{b}_i)\text{d}F(b_i)\text{,}$

(9.25)

$\text{var}(y_i)=\mathrm{var}[\text{E}(y_i\left|b_i\right.)]+\text{E}[\mathrm{var}(y_i\left|b\right.)]\text{.}$

(9.26)

$\begin{array}{ll}\text{var}(y_i)\hfill & \approx \mathrm{var}\left[g^{-1}(X_i^{\prime }\beta )+\frac{\partial g^{-1}(X_i^{\prime }\beta )b_i}{\partial b_i}\right]+\phi \text{E}\left\{\upsilon \left[g^{-1}(X_i^{\prime }{\beta }_i)+\frac{\partial g^{-1}(X_i^{\prime }\beta )b_i}{\partial b_i}\right]\right\}\hfill \\ \hfill & \approx {\text{L}}_i{Z}_{i}G{Z}_i^{\prime }{\text{L}}_i+\phi A_i({\mu }_i)\hfill \\ \hfill & \approx {\tilde{V}}_i(y_i)\text{,}\hfill \end{array}$

(9.27)

${\text{L}}_i=\text{diag}\left[\frac{\partial g^{-1}(X_i^{\prime }\beta )}{\partial X_i^{\prime }\beta }\right]\text{.}$

$G\approx {(Z_i^{\prime }{Z}_i)}^{-1}{Z}_i^{\prime }{\text{L}}_i[\text{var}(y_i)-\phi A_i({\mu }_i)]{\text{L}}_i^{-1}{Z}_i{(Z_i^{\prime }{Z}_i)}^{-1}\text{.}$

$\widehat{G}=\frac{1}{N}\sum _{i=1}^N{(Z_i^{\prime }{Z}_i)}^{-1}{Z}_i^{\prime }{\widehat{\text{L}}}_i^{-1}[(y_i-{\widehat{\mu }}_i)(y_i-{\widehat{\mu }}_i)\prime -\widehat{\phi }{\widehat{A}}_i({\widehat{\mu }}_i)]{\widehat{\text{L}}}_i^{-1}{Z}_i{(Z_i^{\prime }{Z}_i)}^{-1}\text{,}$

(9.28)

$\widehat{\phi }=\sum _{i=1}^N\sum _{j=1}^{n_i}\frac{{(y_{ij}-{\widehat{\mu }}_{ij})}^2-{({\widehat{\text{L}}}_i)}^2{Z}_{ij}^{\prime }\widehat{G}{Z}_{ij}}{\upsilon ({\widehat{\mu }}_{ij})}\text{.}$

(9.29)

In Equation (9.28), the information about b is not directly specified. According to Zeger et al. (1988), the GEE iterative procedure can be performed to compute $\widehat{\beta }$, G, and φ simultaneously. Consequently, the parameters in GLMMs, including the between-subjects random components, can be estimated from the specification of a marginal model without completely specifying the joint distribution of the multivariate response. Therefore, the application of this approach is based on the proposition that the score statistic specified in the marginal model accounts for the entire magnitude and dimension of the random components, including both the between-subjects random effects and within-subject random errors.