3.7. Fixed Effects Methods for Multinomial Responses

So far, this chapter has dealt only with binary response variables. We now consider a categorical response variable y_it that can take on more than two values. Without loss of generality, let's suppose that those values are the integers ranging from 1 to J. Let p_ijt = Prob(y_it = j). We now want a model for the dependence of this probability on predictors x_it and z_i.

We begin with the simpler case in which we assume an ordering of the J categories. The most popular model for ordered categorical data is the cumulative logit model which, in its conventional form, is available in both PROC LOGISTIC and PROC GENMOD. A fixed effects version of the model can be written as

where is the "cumulative" probability of being in category j or higher. Unfortunately, this model does not have reduced sufficient statistics for the α_i parameters. Thus, conditional maximum likelihood is not an option for estimation. One approach to estimation would be the approximate conditional method proposed by Waterman and Lindsay (1996), but that's not available in any commercial software. However, we can apply the hybrid method discussed in the previous section using robust standard errors to adjust for the lack of independence in the repeated observations for each individual.

As an example, we'll analyze data on a sample of 396 people who survived residential fires in the Philadelphia area (Keane et al. 1996). They were interviewed at 3 months, 6 months, and 12 months after the fire. The outcome variable FORGIVE is a response to the question "Have you had feelings that it is difficult to forgive yourself for anything that happened during the fire?" The possible responses were coded as follows:

1

Not at all

2

A little

3

Somewhat

4

Very much

The working data set contains 1,188 records, three for each person, corresponding to the three interviews. The predictor variables, measured at each interview, are

DEPRESS

A depression scale with values ranging from 1 to 4

RELSER

A measure of attendance at religious services:1=never, 2=occasional, 3=regular

SEVENT

Number of stressful events that have occurred since the fire or since the last interview, ranging from 0 to 5

There is also a variable SUBJID which is the id number for each person and is common to all three records for each person.

To implement the hybrid method, we must calculate person-specific means for each of the variables, merge those into the original data set, and then calculate deviations from those means:

PROC MEANS DATA=my.forgive NWAY NOPRINT;
   CLASS subjid;
   VAR relser control depress sevent ;
   OUTPUT OUT=means MEAN= mrelser mcontrol mdepress msevent;
RUN;
DATA forgive;
   MERGE my.forgive means;
   BY subjid;
   dcontrol=control-mcontrol;
   ddepress=depress-mdepress;
   drelser=relser-mrelser;
   dsevent=sevent-msevent;
RUN;

Now we're ready to estimate the cumulative logit model with PROC GENMOD:

PROC GENMOD DATA=forgive;
   CLASS time subjid;
   MODEL forgive= ddepress drelser dsevent
         mdepress mrelser msevent time / D=MULTINOMIAL;
   REPEATED SUBJECT=subjid / TYPE=IND;
   CONTRAST 'Fixed vs. Random Effects' ddepress 1 mdepress −1,
             drelser 1 mrelser −1, dsevent 1 msevent −1;
RUN;

The D=MULTINOMIAL option specifies that the dependent variable FORGIVE has a multinomial distribution with a cumulative link to the predictor variables. The default is a cumulative logit link (as opposed to probit or complementary log-log). Note that because I did not put the DESC option on the PROC statement, this model predicts the probability of the response variable having a lower value (less likelihood of difficulty forgiving oneself).

The REPEATED statement invokes GEE estimation, with standard errors and associated statistics calculated using the robust method of White (1980). Note that TYPE=IND is the only correlation structure allowed whenever D=MULTINOMIAL is specified. This means that GEE estimation presumes that there is no correlation among the repeated measures, and implies that GEE coefficient estimates are identical to those produced by conventional maximum likelihood. Nevertheless, the standard errors and test statistics are adjusted for dependence among the observations. Finally, the CONTRAST statement tests whether the coefficients for the deviation variables are the same as the coefficients for the corresponding mean variables. As we've seen before, this is equivalent to testing for fixed effects versus random effects.

Table 3.14. Output 3.13 Hybrid Estimates for Cumulative Logit Model with PROC GENMOD

Analysis Of GEE Parameter Estimates
Empirical Standard Error Estimates
Parameter		Estimate	Standard Error	95% Confidence	Limits	Z	Pr>|Z|
Intercept1		−3.7853	0.3653	−4.5012	−3.0693	−10.36	<.0001
Intercept2		−2.9643	0.3492	−3.6487	−2.2799	−8.49	<.0001
Intercept3		−2.1085	0.3422	−2.7791	−1.4378	−6.16	<.0001
ddepress		0.3001	0.1389	0.0278	0.5724	2.16	0.0308
drelser		0.1462	0.1600	−0.1673	0.4598	0.91	0.3606
dsevent		0.1728	0.0798	0.0165	0.3292	2.17	0.0303
mdepress		0.7831	0.1258	0.5365	1.0298	6.22	<.0001
mrelser		−0.3619	0.1572	−0.6700	−0.0538	−2.30	0.0213
msevent		0.0579	0.1294	−0.1957	0.3116	0.45	0.6543
time	1	0.7816	0.1712	0.4460	1.1172	4.56	<.0001
time	2	0.4850	0.1696	0.1526	0.8175	2.86	0.0042
time	3	0.0000	0.0000	0.0000	0.0000	.	.

Contrast Results for GEE Analysis
Contrast	DF	Chi-Square	Pr>ChiSq	Type
Fixed vs. Random Effects	3	11.87	0.0078	Score

Results are displayed in Output 3.13. The coefficients for the three deviation variables (those whose names begin with D) can be interpreted as if they were fixed effects coefficients. Because these coefficients depend only on variation over time within persons, they control for all stable covariates. Among these coefficients, we see significant effects of depression and number of stressful events. As expected, those who are more depressed and who had more stressful events are more likely to have trouble forgiving themselves. It's clear, on the other hand, that the deviation score for attendance at religious services does not have an effect, even though the mean score is significant at the .02 level. The results from the CONTRAST statement indicate that we should reject the null hypothesis that the deviation coefficients are the same as the corresponding mean coefficients. The implication is that we should focus our attention on the deviation coefficients, since they control for all stable covariates.

As we have previously observed, one limitation of the GEE method is that the coefficients are population averaged rather than subject specific, implying that they are attenuated toward zero because of population heterogeneity. If you're willing to put in some additional programming effort and computer time, you can avoid this problem by fitting a random effects model with PROC NLMIXED. Because the cumulative logit model is not built into NLMIXED, however, the programming is somewhat more involved:

PROC NLMIXED DATA=forgive;
   eta=bddepress*ddepress + bdrelser*drelser + bdsevent*dsevent
      + bmdepress*mdepress + bmrelser*mrelser + bmsevent*msevent +
        t1*(time EQ 1) + t2*(time EQ 2) + alpha;
   IF forgive=1 THEN p=1/(1+EXP(-b1-eta));
   ELSE IF forgive=2 THEN p=1/(1+EXP(-b2-eta))−1/(1+EXP(-b1-
           eta));
   ELSE IF forgive=3 THEN p=1/(1+EXP(-b3-eta))−1/(1+EXP(-b2-
           eta));
   ELSE p=1−1/(1+EXP(-b3-eta));
   ll=LOG(p);
   MODEL forgive~GENERAL(ll);
   RANDOM alpha~NORMAL(0,var) SUBJECT=subjid;
   CONTRAST 'Test of fixed vs. random' bddepress-bmdepress,
             bdrelser-bmrelser,bdsevent-bmsevent,bdsevent-
             bmsevent;
   PARMS  bddepress=0 bdrelser=0 bdsevent=0 bmdepress=0
          bmrelser=0 bmsevent=0 t1=0 t2=0 b1=1 b2=2 b3=3 var=1;
RUN;

The first statement after the PROC statement defines ETA to be a linear function of the explanatory variables, including a random disturbance term ALPHA. As with previous NLMIXED programs, I've chosen names for the coefficients that are the same as the names for the covariates, except prefixed by B, making it easier to interpret the output. The next four IF and ELSE IF statements specify the probability P of observing each outcome of the response variable, as it depends on ETA. The log-likelihood LL is defined to equal the log of the probability P. The MODEL statement says simply that the response variable FORGIVE has a log-likelihood given by LL. The RANDOM statement declares ALPHA to be normally distributed with a mean of 0 and a variance parameter VAR. The CONTRAST statement tests whether the coefficients for the deviation variables are the same as the coefficients for the corresponding mean variables. Finally, the PARMS statement assigns starting values to all the parameters.

The coefficient estimates in Output 3.14 generally follow the same pattern as those in Output 3.13 produced by PROC GENMOD, but their magnitudes are all noticeably larger. This is just what we would expect from a subject-specific method. The increased magnitude does not always imply an increased level of statistical significance, however, because the standard errors also increase. For example, the coefficient for DSEVENT has a p-value of .03 in Output 3.13 and .09 in Output 3.14. Another thing to keep in mind is that PROC GENMOD took 0.2 seconds to estimate the model on my PC, whereas PROC NLMIXED took 8 seconds.

Table 3.15. Output 3.14 Estimates for Cumulative Logit Model with PROC NLMIXED

Parameter Estimates
Parameter	Estimate	Standard Error	DF	tValue	Pr>|t|	Alpha	Lower	Upper	Gradient
bddepress	−0.5128	0.1948	395	−2.63	0.0088	0.05	−0.8957	−0.1298	0.000115
bdrelser	−0.2621	0.2324	395	−1.13	0.2602	0.05	−0.7191	0.1949	−0.00007
bdsevent	−0.2319	0.1380	395	−1.68	0.0935	0.05	−0.5032	0.03927	0.000148
bmdepress	−1.2357	0.2041	395	−6.05	<.0001	0.05	−1.6369	−0.8344	−0.00013
bmrelser	0.6043	0.2408	395	2.51	0.0125	0.05	0.1310	1.0777	−0.00004
bmsevent	−0.09638	0.2235	395	−0.43	0.6665	0.05	−0.5357	0.3430	−0.00006
t1	−1.1166	0.2639	395	−4.23	<.0001	0.05	−1.6355	−0.5978	0.000016
t2	−0.6750	0.2521	395	−2.68	0.0077	0.05	−1.1706	−0.1794	0.000035
b1	3.2332	0.5851	395	5.53	<.0001	0.05	2.0829	4.3835	−0.00004
b2	4.5390	0.6096	395	7.45	<.0001	0.05	3.3406	5.7374	−0.00004
b3	5.6948	0.6374	395	8.93	<.0001	0.05	4.4416	6.9480	0.00005
var	4.1569	0.8213	395	5.06	<.0001	0.05	2.5422	5.7716	−4.26E-6

Contrasts
Label	Num DF	Den DF	F Value	Pr>F
Test of fixed vs. random	3	395	4.60	0.0035

Now let's consider the more complicated situation in which the J categories are unordered. The most popular model for unordered dependent variables is the multinomial logit model (also known as the generalized logit model), which we now extend to include fixed effects:

In essence, this is a set of binary logistic regression equations that simultaneously compare each category to the last category. Notice that the fixed effects α_ij vary both over individuals and over response values.

As with a single binary logistic model (a special case of the multinomial model), there are reduced sufficient statistics for the α_ij terms, namely the frequency counts over time of the different response values for each individual. By conditioning on those counts, this model can be estimated by conditional maximum likelihood. Unfortunately, there are no SAS procedures that are designed to do this. For certain simple cases in which the time-varying covariates are categorical, the model can be reformulated as a log-linear model that can be estimated with PROC GENMOD or PROC CATMOD (Tjur 1982; Conaway 1989; Darroch and McCloud 1986; Kenward and Jones 1991), but I will not pursue those methods here. Alternatively, one could break the multinomial model into a set of binary models, one model for each comparison of a particular category with a reference category (Begg and Gray 1984; Allison 1999). Each binary model could then be estimated using the conditional logistic regression methods we have already discussed in this chapter. While this approach produces consistent estimates (in the statistical sense) of the coefficients, results will differ depending on the choice of the reference category. Furthermore, there is no overall test for the effect of each variable on the response variable.

As with the cumulative logit model, it seems that the best available approach in SAS is to use the hybrid method. In fact, we can use the same data set that we created earlier for estimating the cumulative model. Unfortunately, PROC GENMOD will not estimate an unordered multinomial model, and PROC LOGISTIC does not allow for dependence among the repeated observations. Instead, we shall use PROC SURVEYLOGISTIC, which does conventional maximum likelihood estimation of the coefficients but produces standard errors and test statistics that allow for dependence among the repeated observations. Here's the code:

PROC SURVEYLOGISTIC DATA=forgive;
   CLASS time;
   MODEL forgive(REF='1')= ddepress drelser dsevent
         mdepress mrelser msevent time / LINK=GLOGIT;
   CLUSTER subjid;
   CONTRAST 'Fixed vs. Random Effects' ddepress 1 mdepress −1,
            drelser 1 mrelser −1, dsevent 1 msevent −1;
RUN;

The syntax of PROC SURVEYLOGISTIC is nearly identical to that of PROC LOGISTIC except, in this example, for the CLUSTER statement. The CLUSTER statement specifies an id variable that defines groups within which observations are allowed to be dependent. In the MODEL statement, the REF= '1' option specifies that the reference category for the dependent variable will be 1, which is the value for 83% of the cases. The default is to choose the highest value (in this case 5) as the reference category, but only about 4% of the cases have that value. Although the generalized logit model is fundamentally invariant to the choice of the reference category, choosing a reference category with few cases can make it appear as though none of the coefficients is statistically significant.

The LINK=GLOGIT option tells SAS that this is a generalized (unordered) logit model rather than the cumulative logit model, which is the default. The CONTRAST statement tests the null hypothesis that all the deviation coefficients are identical to all the corresponding mean coefficients. Although the syntax is the same as the CONTRAST statement for the cumulative model in GENMOD, the consequences are somewhat different. In our unordered model, each predictor variable has three coefficients. Each coefficient measures the effect of the variable on being in one particular category rather than the reference category. When the CONTRAST statement specifies, say, DDEPRESS 1 MDEPRESS −1, each of the three coefficients for DDEPRESS is compared with the corresponding coefficients for MDEPRESS. Thus, the chi-square statistic produced by our CONTRAST statement has nine degrees of freedom (three variables × three coefficients).

Results are shown in Output 3.15. The "Type 3 Analysis of Effects" panel gives chi-squares for the null hypotheses that all three coefficients for each variable are zero, controlling for all the other variables. These chi-squares are invariant to the choice of the reference category. We see that none of the fixed effects deviation variables is statistically significant.

The "Analysis of Maximum Likelihood Estimates" panel reports the individual coefficient estimates and associated statistics. As already noted, these coefficients are conventional maximum likelihood estimates under the assumption that all observations are independent. Hence, they are the same numbers that would be produced by PROC LOGISTIC with the LINK=GLOGIT option. The standard errors, on the other hand, are adjusted for dependence among the repeated observations for each person. In fact, they are not all that different from the standard errors produced by PROC LOGISTIC (or PROC SURVEYLOGISTIC without the CLUSTER statement). Finally, the results for the CONTRAST statement provide marginal evidence that the deviation coefficients are different from the mean coefficients.

Table 3.16. Output 3.15 Estimates for Generalized Logit Model with PROC SURVEYLOGISTIC

Type 3 Analysis of Effects
Effect	DF	Wald Chi-Square	Pr>ChiSq
ddepress	3	6.0613	0.1087
drelser	3	1.4103	0.7031
dsevent	3	5.5606	0.1351
mdepress	3	37.0442	<.0001
mrelser	3	7.3868	0.0605
msevent	3	0.9435	0.8149
time	6	24.4000	0.0004

Analysis of Maximum Likelihood Estimates
Parameter		forgive	DF	Estimate	Standard Error	Wald Chi-Square	Pr>ChiSq
Intercept		2	1	−2.0965	0.3992	27.5840	<.0001
Intercept		3	1	−3.3333	0.5462	37.2401	<.0001
Intercept		4	1	−3.4458	0.5699	36.5541	<.0001
ddepress		2	1	0.2869	0.2154	1.7746	0.1828
ddepress		3	1	0.0597	0.2625	0.0518	0.8200
ddepress		4	1	0.5162	0.2418	4.5566	0.0328
drelser		2	1	0.0979	0.2933	0.1114	0.7386
drelser		3	1	0.1264	0.4082	0.0958	0.7569
drelser		4	1	0.3057	0.3016	1.0275	0.3107
dsevent		2	1	0.1197	0.1450	0.6820	0.4089
dsevent		3	1	0.0869	0.1907	0.2076	0.6487
dsevent		4	1	0.2961	0.1505	3.8730	0.0491
mdepress		2	1	0.6285	0.1362	21.3044	<.0001
mdepress		3	1	0.8142	0.1783	20.8480	<.0001
mdepress		4	1	1.0347	0.2091	24.4928	<.0001
mrelser		2	1	−0.4500	0.1715	6.8824	0.0087
mrelser		3	1	−0.2040	0.2703	0.5694	0.4505
mrelser		4	1	−0.3816	0.2572	2.2015	0.1379
msevent		2	1	0.1280	0.1404	0.8313	0.3619
msevent		3	1	−0.00520	0.2073	0.0006	0.9800
msevent		4	1	0.0326	0.2266	0.0207	0.8856
time	1	2	1	0.1745	0.1571	1.2343	0.2666
time	1	3	1	0.5163	0.1682	9.4245	0.0021
time	1	4	1	0.5069	0.1911	7.0390	0.0080
time	2	2	1	0.1984	0.1404	1.9957	0.1577
mdepress		4	1	1.0347	0.2091	24.4928	<.0001
mrelser		2	1	−0.4500	0.1715	6.8824	0.0087

Contrast Test Results
Contrast	DF	Wald Chi-Square	Pr>ChiSq
Fixed vs. Random Effects	9	16.0781	0.0653