Chapter 6 Understanding Linear Models Concepts

6.1 Introduction

6.2 The Dummy-Variable Model

6.2.1 The Simplest Case: A One-Way Classification

6.2.2 Parameter Estimates for a One-Way Classification

6.2.3 Using PROC GLM for Analysis of Variance

6.2.4 Estimable Functions in a One-Way Classification

6.3 Two-Way Classification: Unbalanced Data

6.3.1 General Considerations

6.3.2 Sums of Squares Computed by PROC GLM

6.3.3 Interpreting Sums of Squares in Reduction Notation

6.3.4 Interpreting Sums of Squares in the -Model Notation

6.3.5 An Example of Unbalanced Two-Way Classification

6.3.6 The MEANS, LSMEANS, CONTRAST, and ESTIMATE Statements in a Two-Way Layout

6.3.7 Estimable Functions for a Two-Way Classification

6.3.7.1 The General Form of Estimable Functions

6.3.7.2 Interpreting Sums of Squares Using Estimable Functions

6.3.7.3 Estimating Estimable Functions

6.3.7.4 Interpreting LSMEANS, CONTRAST, and ESTIMATE Results Using Estimable Functions

6.3.8 Empty Cells

6.4 Mixed-Model Issues

6.4.1 Proper Error Terms

6.4.2 More on Expected Mean Squares

6.4.3 An Issue of Model Formulation Related to Expected Mean Squares

6.5 ANOVA Issues for Unbalanced Mixed Models

6.5.1 Using Expected Mean Squares to Construct Approximate F-Tests for Fixed Effects

6.6 GLS and Likelihood Methodology Mixed Model

6.6.1 An Overview of Generalized Least Squares Methodology

6.6.2 Some Practical Issues about Generalized Least Squares Methodology

6.1 Introduction

The purpose of this chapter is to provide detailed information about how PROC GLM and PROC MIXED work for certain applications. Certainly, this is not a complete documentation. Rather, it provides enough information to you for a basic understanding and a basis for further reading.

Both GLM and MIXED utilize “dummy” variables, which are also called “indicator” variables in mathematics. They are created whenever a CLASS statement is specified. The primary distinction between GLM and MIXED in this regard is that MIXED separates the sets of dummy variables into a group for fixed effects and a group for random effects, whereas the primary computations of GLM use the dummy variables as representing fixed effects. The general linear model approach uses dummy variables in a regression model. Although this technique is useful in all situations, it is primarily applied to analysis of variance with unbalanced data, where the direct computation of sums of squares fails, and to analysis of covariance and associated techniques.

While the dummy variable approach is capable of handling a vast array of applications, it also presents some complications that must be overcome. Two of the principal complications regarding fixed effects are

❏ specifying model parameters and their estimates

❏ setting up meaningful combinations of parameters for testing and estimation.

Both of these are concerned with estimable functions. These complications must be dealt with in computer programs using general linear models. The purpose of this chapter is to explain, with the use of fairly simple examples, how the GLM procedure deals with the complications. A more technical description of GLM features is given in the SAS/STAT User’s Guide, Version 8, Volume 2.

This chapter describes the essence of general linear model and mixed-model computations. It is more or less self-contained, and you will notice some overlap with previous and subsequent chapters. In particular, the CONTRAST and ESTIMATE statements are discussed in Chapter 3, “Analysis of Variance for Balanced Data,” and the RANDOM statement is discussed in Chapter 4, “Analyzing Data with Random Effects.” This present chapter delves more deeply into some of the same topics. Section 6.2 provides essential concepts of using dummy variables in the context of a one-way classification. Section 6.3 does the same for a two-way classification with both factors fixed. Then Section 6.4 discusses technical issues for mixed models.

6.2 The Dummy-Variable Model

This section presents the analysis-of-variance model using dummy variables, methods for specifying model parameters, and the methods used by PROC GLM. For simplicity, an analysis-of-variance model with one-way classification that results from a completely randomized design illustrates the discussion. In application, however, such a structure might be adequately (and more efficiently) analyzed by using the ANOVA procedure (see Section 3.4, “Analysis of One-Way Classification of Data”).

6.2.1 The Simplest Case: A One-Way Classification

Data for the one-way classification consist of measurements classified according to a one-dimensional criterion. An example of this kind of structure is a set of student exam scores, where each student is taught by one of three teachers. The exam scores are thus grouped or classified according to TEACHER. The most straightforward model for data of this type is

y_ij = μ_i + ε_ij

where

yij	represents the jth measurement in the ith group.
μi	represents the population mean for the ith group.
εij	represents the random error with mean=0 and variance=σ2.
i = 1,..., t	where t equals the number of groups.
j = 1,..., ni	where ni equals the number of observations in the ith group.

This is called the means or μ -model because it uses the means μ₁,..., μ_t, as the basic parameters in the mathematical expression for the model (Hocking and Speed 1975). The corresponding estimates of these parameters are

$\begin{array}{l}{\widehat{\mu }}_1={\bar{y}}_{1.}\\ .\\ .\\ .\\ {\widehat{\mu }}_t={\bar{y}}_{t.}\end{array}$

where ${\bar{y}}_{i.}=\left({\Sigma }_j{y}_{ij}\right)/n_i$ is the mean of n_i observations in group i.

In these situations, the statistical inference of interest is often about differences between the means of the form (μ_i − μ_i,) or between the means and some reference or baseline value μ. Therefore, many statistical textbooks present a model for the one-way structure that employs these differences as basic parameters. This is the familiar analysis-of-variance model illustrated in Section 2.3.4:

y_ij = μ + τ_i + ε_ij

where μ equals the reference value and

τ_i = μ_i − μ

Thus, the means can be expressed as

μ_i = μ + τ_i

This relates the set of t means μ_i,..., μ_t, to a set of t+1 parameters, μ, τ₁,..., τ_t. Therefore, this model is said to be overspecified. Consequently, the parameters, μ, τ₁,..., τ_t are not well defined. For any set of values of μ₁,..., μ_t, there are infinitely many choices for μ, τ₁,..., τ₁, which satisfy the basic equations μ₁ = μ + τ₁, i = 1,..., t. The choice may depend on the situation at hand, or it may not be necessary to fully define the parameters.

For the implementation of the dummy-variable model, the analysis-of-variance model

y_ij = μ + τ_i + ε_ij

is rewritten as a regression model

y_ij = μ + τ₁x₁ + ... + τ_tx_t + ε_ij

where the dummy variables x₁,...,x_t are defined as follows:

x1	equals 1 for an observation in group 1 and 0 otherwise.
x2	equals 1 for an observation in group 2 and 0 otherwise.
·
·
·
xt	equals 1 for an observation in group t and 0 otherwise.

In matrix notation, the model equations for the data become

$Y=\left[\begin{array}{l}{y}_{11}\\ .\\ .\\ .\\ y_{1n_1}\\ y_{21}\\ .\\ .\\ .\\ y_{2n_2}\\ .\\ .\\ .\\ y_{t1}\\ .\\ .\\ .\\ y_{t{n}_t}\end{array}\right]=\left[\begin{array}{l}110...0\\ ....\\ ....\\ ....\\ 110...0\\ 101...0\\ ....\\ ....\\ ....\\ 101...0\\ .\text{}...\\ ....\\ ....\\ 100...1\\ ....\\ ....\\ ....\\ 100...1\end{array}\right]\left[\begin{array}{l}{\beta }_0\\ {\beta }_1\\ .\\ .\\ .\\ {\beta }_t\end{array}\right]+\left[\begin{array}{l}{\epsilon }_{11}\\ .\\ .\\ .\\ {\epsilon }_{1n_1}\\ {\epsilon }_{21}\\ .\\ .\\ .\\ {\epsilon }_{2n_2}\\ .\\ .\\ .\\ {\epsilon }_{t1}\\ .\\ .\\ .\\ {\epsilon }_{t{n}_t}\end{array}\right]=X\beta +\epsilon$

Thus, the matrices of the normal equations are

$X^{\prime }X=\left[\begin{array}{l}{n}_{.}{n}_1{n}_2...n_t\\ n_1{n}_{1}0...0\\ n_{2}0n_2...0\\ ....\\ ....\\ ....\\ n_{t}00...n_t\end{array}\right],X^{\prime }Y=\left[\begin{array}{l}{Y}_{\mathrm{..}}\\ Y_{1.}\\ Y_{2.}\\ .\\ .\\ .\\ Y_{t.}\end{array}\right]$

where Y_i. and Y_.. are totals corresponding to y̅_i. and y̅_... The normal equations (X′X) β̂ = X′Y are equivalent to the set

$\begin{array}{lll}\widehat{\text{μ}}\hfill & +\hfill & {\widehat{\text{τ}}}_1={\overline{y}}_1.\hfill \\ \widehat{\text{μ}}\hfill & +\hfill & {\widehat{\text{τ}}}_2={\overline{y}}_2.\hfill \\ .\hfill & \hfill & \hfill \\ .\hfill & \hfill & \hfill \\ .\hfill & \hfill & \hfill \\ \widehat{\text{μ}}\hfill & +\hfill & {\widehat{\text{τ}}}_t={\overline{y}}_t.\hfill \end{array}$

Because there are only t- equations, there is no unique solution for the (t+1) estimates $\widehat{\mu },{\widehat{\tau }}_1,...,{\widehat{\tau }}_t.$ Corresponding to this, the X′X matrix describing the set of normal equations is of dimension (t+1) x(t+1) and of rank t. In this model the first row of X′X is equal to the sum of the other t-rows. The same relationship exists among the columns of X′X. Therefore, X′X is said to be of less than full rank.

6.2.2 Parameter Estimates for a One-Way Classification

There are two popular methods for obtaining estimates with a less-than-full-rank model. Restrictions can be imposed on the parameters to obtain a full-rank model, or a generalized inverse of X′X can be obtained. PROC GLM uses the latter method. This section reviews both methods in order to put the approach used by PROC GLM into perspective.

The restrictions method is based on the fact that any definition of one of the parameters in the model (say the reference parameter) causes the other parameters to be uniquely defined. The definition can be restated in the form of a restriction. Another view of the term restriction is to define the parameters to have a unique interpretation. The corresponding estimates are then required to coincide with the definition of the parameters.

One type of restriction is to define one of the τ_i equal to 0, say τ_t = 0. In this case, becomes the mean of the tth group μ_τ = μ + τ_t = μ and τ_i becomes the difference between the mean for the ith group and the mean for the tth group, τ_i = μ − μ = μ_i − μ_t.

The corresponding restriction on the solution to the normal equations is to require ${\widehat{\tau }}_i=0.$ Requiring ${\widehat{\tau }}_t=0.$ leads automatically to a unique set of values for the remaining set of estimates $\widehat{\mu },{\widehat{\tau }}_1,...,{\widehat{\tau }}_{t-1}.$. This occurs because ${\tau }_t$ is dropped from the linear model. Consequently, the column corresponding to τ_t is dropped from the X matrix, producing the following model equation:

$\left[\begin{array}{l}{y}_{11}\\ .\\ .\\ .\\ y_{1n_1}\\ .\\ .\\ .\\ y_{t-1,1}\\ .\\ .\\ .\\ y_{t-1,n_{t-1}}\\ y_{t1}\\ .\\ .\\ .\\ y_{t{n}_t}\end{array}\right]=\left[\begin{array}{l}11...0\\ ...\\ ...\\ ...\\ 11...0\\ ...\\ ...\\ ...\\ 10...1\\ ...\\ ...\\ ...\\ 10...1\\ 10...0\\ ...\\ ...\\ ...\\ 10...0\end{array}\right]\left[\begin{array}{l}\mu \\ {\tau }_1\\ .\\ .\\ .\\ {\tau }_{t-1}\end{array}\right]+\left[\begin{array}{l}{\epsilon }_{11}\\ .\\ .\\ .\\ {\epsilon }_{1n_1}\\ .\\ .\\ .\\ {\epsilon }_{t-1,1}\\ .\\ .\\ .\\ {\epsilon }_{t-1,n_{t-1}}\\ {\epsilon }_{t-1}\\ .\\ .\\ .\\ {\epsilon }_{t{n}_t}\end{array}\right]$

The solution to the corresponding normal equation $\left({\mathbf{X}}^{\prime }\mathbf{X}\right)\widehat{\beta }={\mathbf{X}}^{\prime }\mathbf{Y}$, where X′X is now nonsingular, results in

$\begin{array}{l}\widehat{\mu }={\overline{y}}_{t.}\\ {\widehat{\tau }}_1={\overline{y}}_{1.}-{\overline{y}}_{t.}\\ {\widehat{\tau }}_2={\overline{y}}_{2.}-{\overline{y}}_{t.}\\ .\\ .\\ .\\ {\widehat{\tau }}_{(t-1)}={\overline{y}}_{(t-1).}-{\overline{y}}_{t.}\end{array}$

Another approach defines μ to be equal to the mean of μ₁, μ₂,..., μ_t; — that is, μ = (μ₁ + μ₂ +...+ μ_t)/t. Then μ is called the grand mean and the τ_i are called the group effects. From this definition of μ, it follows that Σ_i τ_i = 0. Consequently,

τ₁ = − τ₁ − τ₂ −…− τ_t−1

Therefore, observations y_tj = μ + τ_t + ε_ij in the tth group can be written

y_tj = μ − τ₁−τ₂−…−τ_t−1 +ε_ij

The parameter τ_t is dropped from the model, which now becomes

$\left[\begin{array}{l}{y}_{11}\\ \\ \\ \\ y_{1n_1}\\ \\ \\ \\ y_{t-1,1}\\ \\ \\ \\ y_{t-1,n_{t-1}}\\ y_{t1}\\ \\ \\ \\ y_{t{n}_t}\end{array}\right]=\left[\begin{array}{l}1100\\ \\ \\ \\ 1100\\ \\ \\ \\ 1001\\ \\ \\ \\ 1001\\ 1-1-1-1\\ \\ \text{}\text{}\text{}\\ \\ 1-1-1-1\end{array}\right]\left[\begin{array}{l}\mu \\ {\tau }_1\\ {\tau }_2\\ \\ \\ \\ {\tau }_{t-1}\end{array}\right]+\left[\begin{array}{l}{\epsilon }_{11}\\ \\ \\ \\ {\epsilon }_{1n_1}\\ \\ \\ \\ {\epsilon }_{t-1,1}\\ \\ \\ \\ {\epsilon }_{t-1,n_{t-1}}\\ {\epsilon }_{t1}\\ \\ \\ \\ {\epsilon }_{t{n}_t}\end{array}\right]$

The solution to the corresponding normal equation yields

$\begin{array}{l}\widehat{\mu }=({\overline{y}}_{1.}+\mathrm{...}+{\overline{y}}_{t.})/t\\ {\widehat{\tau }}_1={\overline{y}}_{1.}-{\overline{y}}_{\mathrm{..}}\\ {\widehat{\tau }}_2={\overline{y}}_{2.}-{\overline{y}}_{\mathrm{..}}\\ .\\ .\\ .\\ {\widehat{\tau }}_{t-1}={\overline{y}}_{(t-1).}-{\overline{y}}_{\mathrm{..}}\end{array}$

and the implementation of the condition ${\tau }_1=-{\tau }_1-{\tau }_2--{\tau }_{t-1}$ yields

${\widehat{\text{τ}}}_t={\overline{y}}_{t.}-{\overline{y}}_{\mathrm{..}}$

The use of generalized inverses and estimable functions may be preferable for a variety of reasons. In the restrictions method, it might not be clear which particular restriction is desired. In cases of empty cells in multiway classifications, it can be difficult to define the parameters. In fact, it is often hard to identify the empty cells in large, multiway classifications, let alone to define a set of parameters that adequately describe all pertinent effects and interactions. The generalized-inverse approach partially removes the burden of defining parameters from the data analyst.

Section 2.4.4, “Using the Generalized Inverse,” showed that there is no unique solution to a system of equations with a less-than-full-rank coefficient matrix and introduced the generalized inverse to obtain a nonunique solution. Although the set of parameter estimates produced using the generalized inverse is not unique, there is a class of linear functions of parameters called estimable functions for which unique estimates do exist. For example, the function (τ_i − τ_j) is estimable: its least-squares estimate is the same regardless of the particular solution obtained for the normal equations. For a discussion of the definition of estimable functions as it relates to the theory of linear models, see Graybill (1976) or Searle (1971).

PROC GLM uses a generalized inverse to obtain a solution that produces one set of estimates The technique, in some respects, is parallel to using a set of restrictions that set some of the parameter estimates to 0 Quantities to be estimated or comparisons to be made are specified, and PROC GLM determines whether or not the estimates or comparisons represent estimable functions PROC GLM then provides estimates, standard errors, and test statistics.

For certain applications, there is more than one set of hypotheses that can be tested To cover these situations, PROC GLM provides four types of sums of squares and associated F-statistics and also gives additional information to assist in interpreting the hypotheses tested.

6.2.3 Using PROC GLM for Analysis of Variance

Using PROC GLM for analysis of variance is similar to using PROC ANOVA; the statements listed for PROC ANOVA in Section 3.3.2, “Using the ANOVA and GLM Procedures,” are also used for PROC GLM. In addition to the statements listed for PROC ANOVA, the following SAS statements can be used with PROC GLM:

CONTRAST ‘label’ effect values< . . . effect values> < / options>;
ESTIMATE 'label’ effect values< . . . effect values> < / options>;
ID variables;
LSMEANS effects< / options>;
OUTPUT <OUT=SAS-data-set> keyword= names < . . . keyword=names>;
RANDOM effects< / options>;
WEIGHT variable;

The CONTRAST statement provides a way of obtaining custom hypotheses tests. The ESTIMATE statement can be used to estimate linear functions of the parameters. The LSMEANS (least-squares means) statement specifies effects for which least-squares estimates of means are computed. The uses of these statements are illustrated in Section 6.2.4, “Estimable Functions in the One-Way Classification,” and Section 6.3.6, “MEANS, LSMEANS, CONTRAST, and ESTIMATE Statements in the Two-Way Layout.” The RANDOM statement specifies which effects in the model are random (see Section 6.4.1, “Proper Error Terms”). When predicted values are requested as a MODEL statement option, values of the variable specified in the ID statement are printed for identification beside each observed, predicted, and residual value. The OUTPUT statement produces an output data set that contains the original data set values along with predicted and residual values. The WEIGHT statement is used when a weighted residual sum of squares is needed. For more information, refer to Chapter 24 in the SAS/STAT User’s Guide, Version 8, Volume 2.

Implementing PROC GLM for an analysis-of-variance model is illustrated by an example of test scores made by students in three classes taught by three different teachers. The data appear in Output 6.1.

Output 6.1 Data for One-Way Analysis of Variance

In terms of the analysis-of-variance model described above, the τ_j are the parameters associated with the different teachers (TEACH)—τ₁ is associated with JAY, τ₂ with PAT, and τ₃ with ROBIN. The following SAS statements are used to analyze SCORE2:

proc glm;
   class teach;
   model score2=teach / solution xpx i;

In this example, the CLASS variable TEACH identifies the three classes. In effect, PROC GLM establishes a dummy variable (1 for presence, 0 for absence) for each level of each CLASS variable. In this example, the CLASS statement causes PROC GLM to create dummy variables corresponding to JAY, PAT, and ROBIN, resulting in the following X matrix:

INTERCEPT JAY PAT ROBIN

$\mathbf{X}=\left[\begin{array}{l}\mu {\tau }_1{\tau }_2{\tau }_3\\ 1100\\ ....\\ ....\\ ....\\ 1100\\ 1010\\ ....\\ ....\\ ..\text{}\text{}..\\ 1010\\ 1001\\ ....\\ ....\\ ....\\ 1001\end{array}\right]\begin{array}{c}\begin{array}{l}\text{6 rows for Jay's group}\\ \\ \\ \end{array}\\ \begin{array}{l}\\ \text{14 rows for Pat's group}\end{array}\\ \begin{array}{l}\\ \\ \\ \\ \text{11 rows for Robin's group}\end{array}\end{array}$

Note that the columns for the dummy variables are in alphabetical order; the column positioning depends only on the values of the CLASS variable. For example, the column for JAY would appear after the columns for PAT and ROBIN if the value JAY were replaced by ZJAY.¹

The MODEL statement has the same purpose in PROC GLM as it does in PROC REG and PROC ANOVA. Note that the MODEL statement contains the SOLUTION option. This option is used because PROC GLM does not automatically print the estimated parameter vector when a model contains a CLASS statement. The results of the SAS statements shown above appear in Output 6.2.

Output 6.2 One-Way Analysis of Variance from PROC GLM

The first portion of the output, as in previous examples, shows the statistics for the overall model.

The second portion partitions the model sum of squares (MODEL SS) into portions corresponding to factors defined by the list of variables in the MODEL statement. In this model there is only one factor, TEACH, so the Type I and Type III SS are the same as the MODEL SS. Type II and Type IV have no special meaning here and would be the same as Type I and Type III.

The final portion of the output contains the parameter estimates obtained with the generalized inverses. Specifying XPX and I in the list of options in the MODEL statement causes the X′X and (X′X)^– matrices to be printed. Results appear in Output 6.3.

Output 6.3 XX and (XX) Matrices for a One-Way Classification

For this example, the matrix X′Y is

$\mathbf{\text{X′Y=}}\begin{array}{llll}\text{SCORE2}\hfill & \text{total}\hfill & \text{overall}\hfill & \hfill \\ \text{SCORE2}\hfill & \text{total}\hfill & \text{for}\hfill & \text{JAY}\hfill \\ \text{SCORE2}\hfill & \text{total}\hfill & \text{for}\hfill & \text{PAT}\hfill \\ \text{SCORE2}\hfill & \text{total}\hfill & \text{for}\hfill & \text{ROBIN}\hfill \end{array}=\begin{array}{l}2280\hfill \\ 456\hfill \\ 1027\hfill \\ 797\hfill \end{array}$

Taking (X′X)^‑ from the PROC GLM output and using X′Y above, the solution $\widehat{\mathbf{\beta }}={\left({\mathbf{X}}^{\prime }\mathbf{X}\right)}^{-}{\mathbf{X}}^{\prime }\mathbf{Y}$ is

$\left[\begin{array}{l}{\widehat{\beta }}_0\\ {\widehat{\beta }}_1\\ {\widehat{\beta }}_2\\ {\widehat{\beta }}_3\end{array}\right]=\left[\begin{array}{r}.0909-.0909-.0909.0000\\ -.0909.2575.0909.0000\\ -.0909.0909.1623.0000\\ .0000.0000.0000.0000\end{array}\right]\left[\begin{array}{r}2280\\ 456\\ 1027\\ 797\end{array}\right]=\left[\begin{array}{r}72.45\\ 3.54\\ 0.90\\ 0.00\end{array}\right]$

As pointed out in Section 2.4.4., the particular generalized inverse used by PROC GLM causes the last row and column of (X′X)^- to be set to 0. This yields a set of parameter estimates equivalent in this example to the set given by the restriction that τ₃ = 0. Using the principles discussed in Section 6.2.2, “Parameter Estimates for a One-Way Classification,” it follows that the INTERCEPT μ is actually the mean for the reference group ROBIN. The estimate ${\widehat{\tau }}_1$ labeled JAY is the difference between the mean for Jay’s group and the mean for Robin’s group, and similarly, the estimate ${\widehat{\tau }}_2$ labeled PAT is the mean for Pat’s group minus the mean for Robin’s group. Finally, the estimate ${\widehat{\tau }}_3$ labeled ROBIN, which is set to 0, can be viewed as the mean for Robin’s group minus the mean for Robin’s group.

Remember that these estimates are not unique—that is, they depend on the alphabetical order of the values of the CLASS variable. This fact is recognized in the output by denoting the estimates as biased, which is explained in the note after the listing of estimates.

The other MODEL statement options (P, CLM, CLI, and TOLERANCE), as well as the BY, ID, WEIGHT, FREQ, and OUTPUT statements, are not affected by the use of CLASS variables and may be used as described in Section 2.2.4, “The SS1 and SS2 Options: Two Types of Sums of Squares” and Section 2.2.5, “Tests of Subsets and Linear Combinations of Coefficients.”

6.2.4 Estimable Functions in a One-Way Classification

It is often the case that the particular parameter estimates obtained by the SOLUTION option in PROC GLM are not the estimates of interest, or there may be additional functions of the parameters that you want to estimate. You can specify such other estimates with PROC GLM.

An estimable function is a member of a special class of linear functions of parameters (see Section 2.2.4). An estimable function of the parameters has a definite interpretation regardless of how the parameters themselves are specified. Denote with L a vector of coefficients (L₁, L₂,…, L_t, L_{t + 1}). Then Lβ = L₁μ + L₂τ₁ +…+ L_{t + 1} τ_t is a linear function of the model parameters and is estimable (for this example) if it can be expressed as a linear function of the means μ₁,…,μ_t. Let β̂ be a solution to the normal equation. The function Lβ is estimated by Lβ̂, the corresponding linear function of the parameters. If Lβ is estimable, then Lβ̂ will have the same value regardless of the solution obtained from the normal equations. In the example,

$\widehat{\mathbf{\text{β}}}=\begin{array}{l}\widehat{\text{μ}}\hfill \\ {\widehat{\text{τ}}}_1\hfill \\ {\widehat{\text{τ}}}_2\hfill \\ {\widehat{\text{τ}}}_3\hfill \end{array}=\begin{array}{l}72.454\hfill \\ 3.545\hfill \\ 0.902\hfill \\ 0.000\hfill \end{array}\begin{array}{l}\text{INTERCEPT}\hfill \\ \text{JAY}\hfill \\ \text{PAT}\hfill \\ \text{ROBIN}\hfill \end{array}$

To illustrate, define

L = [1 1 0 0]

Then $\mathbf{\text{L}}\widehat{\mathbf{\text{β}}}=\widehat{\text{μ}}+{\widehat{\text{τ}}}_1={\widehat{\text{μ}}}_1=76.0,$ which is the estimate of the mean score of Jay's group Alternately, let

L = [0 + 1 −1 0]

Then Lβ̂ = (τ̂₁ − τ̂₂) = μ̂₁ – μ̂₂ = 2.643, the estimated mean difference between Jay’s and Pat’s groups. Because both of these are estimable functions, identical estimates would be obtained using a different generalized inverse—for example, if different names for the teachers changed the order of the dummy variables.

Variances of these estimates can be readily obtained with standard formulas that involve elements of the generalized inverse (see Section 2.2.4).

You can obtain the general form of the estimable functions with the E option in the MODEL statement

model score2 = teach / e ;

Output 6.4 shows you that L4, the coefficient or τ₃, must be equal to L1 – L2 – L3. Equivalently, L1 = L2 + L3 + L4. That is, the coefficient on μ must be the sum of the coefficients on τ₁, τ₂ and τ₃.

Output 6.4 Obtaining the General Form of Estimable Functions Using the E Option

PROC GLM calculates estimates and variances for several special types of estimable functions with LSMEANS, CONTRAST, or ESTIMATE statements as well as estimates of user-supplied functions.

The LSMEANS statement produces the least-squares estimates of CLASS variable means—these are sometimes referred to as adjusted means. For the one-way structure, these are simply the ordinary means. In terms of model parameter estimates, they are $\widehat{\mu }+{\widehat{\tau }}_i$. The following SAS statement lists the least-squares means for the three teachers for all dependent variables in the MODEL statement:

lsmeans teach / options;

The available options in the LSMEANS statement are

STDERR	prints the standard errors of each estimated least-squares mean and the t-statistic for a test of the hypothesis that the mean is 0
PDIFF	prints the p-values for the tests of equality of all pairs of CLASS means
E	prints a description of the linear function used to obtain each least-squares mean; this has importance in more complex situations
E=	specifies an effect in the model to use as an error term
ETYPE=	specifies the type (1, 2, 3, or 4) of the effect specified in the E= option
SINGULAR=	tunes the estimability checking

For more information, refer to Chapter 24 in the SAS/STAT User's Guide, Version 8, Volume 2. Output 6.5 shows results from the following SAS statement:

lsmeans teach / stderr pdiff;

Output 6.5 Results of the LSMEANS Statement

The least-squares mean for JAY is computed as $\widehat{\mu }+{\widehat{\tau }}_1=\text{72}\text{45}+\text{3}\text{54}$. Note that this linear function has coefficients L1=1, L2=1, L3=0 and L4=0, so it meets the estimability condition L1 = L2 + L3 + L4.

Least-squares means should not, in general, be confused with ordinary means, which are available with a MEANS statement. The MEANS statement produces simple, unadjusted means of all observations in each class or treatment. Except for one-way designs and some nested and balanced factorial structures that are normally analyzed with PROC ANOVA, these unadjusted means are generally not equal to the least-squares means. Note that for this example, the least-squares means are the same as the means obtained with the MEANS statement. (The MEANS statement is discussed in Section 3.4.2.)

A contrast is a linear function such that the elements of the coefficient vector sum to 0 for each effect. PROC GLM can be instructed to calculate a sum of squares and associated F-test due to one or more contrasts.

As an example, assume that teacher JAY used a special teaching method. You might then be interested in testing whether Jay’s students had mean scores different from the students of the other teachers, and whether PAT and ROBIN, using the same method, produced different mean scores. The corresponding contrasts are shown below:

	Multipliers for TEACH
Contrast	JAY	PAT	ROBIN
JAY vs others	–2	+1	+1
PAT vs ROBIN	0	–1	+1

Taking $\text{β}=(\mu ,{\tau }_1,{\tau }_2,{\tau }_3)$ the contrasts are

$\begin{array}{l}\text{L}\text{β}=-2{\text{μ}}_1+{\text{μ}}_2+{\text{μ}}_3(\text{JAY vs others})\\ =-2{\text{τ}}_1+{\text{τ}}_2+{\text{τ}}_3\end{array}$

and

$\begin{array}{l}\mathbf{\text{L}}\mathbf{\text{β}}=-{\text{μ}}_2+{\text{μ}}_3(\text{PAT vs ROBIN})\\ =-{\text{τ}}_2+{\text{τ}}_3\end{array}$

The corresponding CONTRAST statements are as follows:

contrast 'JAY vs others' teach -2 1 1;
contrast 'PAT vs ROBIN' teach 0 -1 1;

The results appear in Output 6.6.

Output 6.6 Results of the CONTRAST and ESTIMATE Statements

Keep the following points in mind when using the CONTRAST statement:

❏ You must know how many classes (categories) are present in the effect and in what order they are sorted by PROC GLM. If there are more effects (classes) in the data than the number of coefficients specified in the CONTRAST statement, PROC GLM adds trailing zeros. In other words, there is no check to see if the proper number of classes has been specified.

❏ The name or label of the contrast must be 20 characters or less.

❏ Available CONTRAST statement options are

E	prints the entire L vector.
E=effect	specifies an alternate error term.
ETYPE=n	specifies the type (1, 2, 3, or 4) of the E=effect.

❏ Multiple degrees-of-freedom contrasts can be specified by repeating the effect name and coefficients as needed. Thus, the statement

contrast 'ALL' teach -2 1 1, teach 0 -1 1;

produces a two DF sum of squares due to both contrasts. This feature can be used to obtain partial sums of squares for effects through the reduction principle, using sums of squares from multiple degrees-of-freedom contrasts that include and exclude the desired contrasts.

❏ If a non-estimable contrast has been specified, a message to that effect appears in the SAS log.

❏ Although only (t–1) linearly independent contrasts exist for t classes, any number of contrasts can be specified.

❏ The contrast sums of squares are not partial of (adjusted for) other contrasts that may be specified for the same effect (see the fourth point above).

❏ The CONTRAST statement is not available with PROC ANOVA; thus, the computational inefficiency of PROC GLM for analyzing balanced data may be justified if contrasts are required. However, contrast variables can be defined in a DATA step and estimates and statistics can be obtained by a full-rank regression analysis.

The ESTIMATE statement is used to obtain statistics for estimable functions other than least-squares means and contrasts, although it can also be used for these. For the current example, the ESTIMATE statement is used to re-estimate the least-squares means.

The respective least-squares means for JAY, PAT, and ROBIN estimate ${\mu }_1=\mu +{\tau }_1$, ${\mu }_2=\mu +{\tau }_2$, and ${\mu }_3=\mu +{\tau }_3$. The following statements duplicate the least-squares means:

estimate 'LSM JAY' intercept 1 teach 1;
estimate 'LSM PAT' intercept 1 teach 0 1;
estimate 'LSM ROBIN' intercept 1 teach 0 0 1;

Note the use of the term INTERCEPT (referring to) and the fact that the procedure supplies trailing zero-valued coefficients. The results of these statements appear after the listing of parameter estimates at the bottom of Output 6.6 for convenient comparison with the results of the LSMEANS statement.

6.3 Two-Way Classification: Unbalanced Data

The major applications of the two-way structure are the two-factor factorial experiment and the randomized blocks. These applications usually have balanced data. In this section, the two-way classification with unbalanced data is explored. This introduces new questions, such as how means and sums of squares should be computed.

6.3.1 General Considerations

The two-way classification model is

$y_{ijk}=\text{μ}+{\text{α}}_i+{\text{β}}_j+{\left(\text{α}\text{β}\right)}_{ij}+{\text{ε}}_{ijk}$

where

yijk	equals the kth observed score for the (i, j)th cell.
αi	equals the effect of the ith level of factor A.
βj	equals an effect of the jth level of factor B
(αβ)ij	equals the interaction effect for the ith level of factor A and the jth level of factor B.
εijk	equals the random error associated with individual observations.

The model can be defined without the interaction term when appropriate. Let n_ij denote the number of observations in the cell for level i of A and level j of B. If μ_ij denotes the population cell mean for level i of A and level j of B, then

μ_ij = μ + α_i + β_j + (αβ)_ij

At this point, no further restrictions on the parameters are assumed.

The computational formulas for PROC ANOVA that use the various treatment means provide correct statistics for balanced data—that is, data with an equal number of observations (n_ij=n for all i, j) for each treatment combination. When data are not balanced, sums of squares computed by PROC ANOVA can contain functions of the other parameters of the model, and thereby produce biased results.

To illustrate the effects of unbalanced data on the estimation of differences between means and computation of sums of squares, consider the data in this two-way table:

		B
		1	2
A	1	7,9	5
	2	8	4,6

Within level 1 of B, the cell mean for each level of A is 8—that is, ${\bar{y}}_{\text{11}}=\text{(7}+\text{9)/2}=\text{8 }$ and ${\bar{y}}_{\text{21}}=\text{8}$. Hence, there is no evidence of a difference between the levels of A within level 1 of B. Similarly, there is no evidence of a difference between levels of A within level 2 of B, because ${\bar{y}}_{\text{12}}=\text{5 }$ and ${\bar{y}}_{\text{22}}=\text{(4}+\text{6)/2}=\text{5}$. Therefore, you may conclude that there is no evidence in the table of a difference between the levels of A. However, the marginal means for A are

${\bar{y}}_{\mathrm{1..}}=(7+9+5)/3=7$

and

${\overline{y}}_{\mathrm{2..}}=(8+4+6)/3=6$

The difference of 7–6=1 between these marginal means may be erroneously interpreted as measuring an overall effect of the factor A. Actually, the observed difference between the marginal means for the two levels of A measures the effect of factor B in addition to the effect of factor A. This can be verified by expressing the observations in terms of the analysis-of-variance model $y_{ijk}=\mu +{\alpha }_i+{\beta }_j$. (For simplicity, the interaction nd error terms have been left out of the model.)

		B
		1	2
A
	1
		7 = μ + α1 + β1
			5 = μ + α1 + β1
		9 = μ + α1 + β2
	2
			4 = μ + α2 + β1
		8 = μ + α2 + β2
			6 = μ + α2 + β2

The difference between marginal means for A₁ and A₂ is shown to be

$\begin{array}{ccc}{\overline{y}}_{\mathrm{1..}}-{\overline{y}}_{\mathrm{2..}}& =& (1/3)\left[\left({\text{α}}_1+{\text{β}}_1\right)+\left({\text{α}}_1+{\text{β}}_1\right)+\left({\text{α}}_1+{\text{β}}_2\right)\right]\\ & & -(1/3)\left[\left({\text{α}}_2+{\text{β}}_1\right)+\left({\text{α}}_2+{\text{β}}_2\right)+\left({\text{α}}_2+{\text{β}}_2\right)\right]\\ & =& \left({\text{α}}_1-{\text{α}}_2\right)+(1/3)({\text{β}}_1-{\text{β}}_2)\end{array}$
${\text{H}}_{\text{0}}:{\text{α}}_1-{\text{α}}_2=0$

Thus, instead of estimating $\left({\alpha }_1-{\alpha }_2\right)$, the difference between the marginal means of A estimates $\left({\alpha }_1-{\alpha }_2\right)$ plus a function of the factor B parameters $\left({\beta }_1-{\beta }_2\right)/3$. In other words, the difference between the A marginal means is biased by factor B effects.

The null hypothesis about A that would normally be tested is

H₀: α₁ – α₂ = 0

However, for this example, the sum of squares for A computed by PROC ANOVA can be shown to equal $3{\left({\bar{y}}_1-{\bar{y}}_2\right)}^2/2$. Hence, the PROC ANOVA F-test for A actually tests the hypothesis

$H_0:({\alpha }_1-{\alpha }_2)+({\beta }_1-{\beta }_2)/3=0$

which involves the factor B difference $\left({\beta }_1-{\beta }_2\right)$ in addition to the factor A difference $\left({\alpha }_1-{\alpha }_2\right)$.

In terms of the μ model $y_{ijk}={\mu }_{ij}+{\epsilon }_{ijk}$, you usually want to estimate $\left({\mu }_{11}+{\mu }_{12}\right)/2$ and $\left({\mu }_{21}+{\mu }_{22}\right)/2$ or the difference between these quantities. However, the A marginal means for the example are

${\bar{y}}_1=(2{\mu }_{11}+{\mu }_{12})/3+{\bar{\epsilon }}_1$.

and

${\bar{y}}_2=({\mu }_{21}+2{\mu }_{22})/3+{\bar{\epsilon }}_2$.

These means estimate $2\left({\mu }_{11}+{\mu }_{22}\right)/3$ and $\left({\mu }_{21}+{\mu }_{22}\right)/3$, which are functions of the cell frequencies and might not be meaningful.

In summary, a major problem in the analysis of unbalanced data is the contamination of differences between factor means by effects of other factors. The solution to this problem is to adjust the means to remove the contaminating effects.

6.3.2 Sums of Squares Computed by PROC GLM

PROC GLM recognizes different theoretical approaches to analysis of variance by providing four types of sums of squares and associated statistics. The four types of sums of squares in PROC GLM are called Type I, Type II, Type III, and Type IV (SAS Institute). The four types of sums of squares are explained in general, conceptual terms, followed by more technical descriptions.

Type I sums-of-squares retain the properties discussed in Chapter 2, “Regression.” They correspond to adding each source (factor) sequentially to the model in the order listed. For example, the Type I sum of squares for the first factor listed is the same as PROC ANOVA would compute for that effect. It reflects differences between unadjusted means of that factor as if the data consist of a one-way structure. The Type I SS may not be particularly useful for analysis of unbalanced multiway structures but may be useful for nested models, polynomial models, and certain tests involving the homogeneity of regression coefficients (see Chapter 7, “Analysis of Covariance”). Also, comparing Type I and other types of sums of squares provides some information on the effect of the lack of balance.

Type II sums of squares are more difficult to understand. Generally, the Type II SS for an effect U, which may be a main effect or interaction, is adjusted for an effect V if and only if V does not contain U. Specifically, for a two-factor structure with interaction, the main effects, A and B, are not adjusted for the A*B interactions because the symbol A*B contains both A and B. Factor A is adjusted for B because the symbol B does not contain A. Similarly, B is adjusted for A, and the A*B interaction is adjusted for the two main effects.

Type II sums of squares for the main effects A and B are mainly appropriate for situations in which no interaction is present. These are the sums of squares presented in many major statistical textbooks. Their method of computation is often referred to as the method of fitting constants.

The Type II analysis relates to the following general guidelines often given in applied statistical texts. First, test for the significance of the A*B interaction. If A*B is insignificant, delete it from the model and analyze main effects, each adjusted for the other. If A*B is significant, then abandon main-effects analysis and focus your attention on simple effects.

Note that for full-rank regression models, the Type II sums of squares are adjusted for cross-product terms. This occurs because, for example,

$y={\beta }_0+{\beta }_1{x}_1+{\beta }_2{x}_2+{\beta }_3{x}_1{x}_2+\epsilon$

where the product x₁x₂ is dealt with simply as another independent variable with no concept of order of the term.

The Type III sums of squares correspond to Yates’s weighted squares of means analysis. Their principal use is in situations that require a comparison of main effects even in the presence of interaction. Type III sums of squares are partial sums of squares. In this sense, each effect is adjusted for all other effects. In particular, main effects A and B are adjusted for the interaction A*B if all these terms are in the model. If the model contains only main effects, then Type II and Type III analyses are the same. See Steel and Torrie (1980), Searle (1971), and Speed et al. (1978) for further discussion of the method of fitting constants and the method of weighted squares of means.

The Type IV functions were designed primarily for situations where there are empty cells. The principles underlying the Type IV sums of squares are quite involved and can be discussed only in a framework using the general construction of estimable functions. It should be noted that the Type IV functions are not necessarily unique when there are empty cells, but the functions are identical to those provided by Type III when there are no empty cells.

You can request four sums of squares in PROC GLM as options in the MODEL statement. For example, the following SAS statement specifies the printing of Type I and Type IV sums of squares:

model . . . / ss1 ss4;

Any or all types may be requested. If no sums of squares are specified, PROC GLM computes the Type I and Type III sums of squares by default.

The next two sections interpret the different sums of squares in terms of reduction notation and the μ-model.

6.3.3 Interpreting Sums of Squares in Reduction Notation

The types of sums of squares can be explained in terms of the reduction notation that is developed for regression models in Chapter 2. This requires writing the model as a regression model using dummy variables, with certain restrictions imposed on the parameters to give them unique interpretation.

As an example, consider a 2×3 factorial structure with n_ij observations in the cell in row i, column j. The equation for the model is

$y_{ijk}=\mu +{\alpha }_i+{\beta }_j+\alpha {\beta }_{ij}+{\epsilon }_{ijk}$

where i=1, 2, j=1, 2, 3, and k=1, . . . , n_ij. Assume n_ij>0 for all i, j. An expression of the form $\text{R}\left(\alpha \text{|}\mu \text{,}\beta \right)$ means the same as $\text{R}\left({\alpha }_1,{\alpha }_2\text{|}\mu \text{,}{\beta }_1,{\beta }_2,{\beta }_3\right)$. The sums of squares printed by PROC GLM can be interpreted in reduction notation most easily under the restrictions

${\Sigma }_i{\alpha }_i={\Sigma }_j{\beta }_j={\Sigma }_i\alpha {\beta }_{ij}={\Sigma }_j\alpha {\beta }_{ij}=0$   (6.1)

that is, by taking an X matrix with full-column rank given by

$X=\begin{array}{cc}\mu {\alpha }_1{\beta }_1{\beta }_2\alpha {\beta }_{11}\alpha {\beta }_{12}& \\ \left[\begin{array}{l}111010\\ ......\\ ......\\ ......\\ 111010\\ 110101\\ ......\\ ......\\ ......\\ 110101\\ 11-1-1-1-1\\ ......\\ ......\\ ......\\ 11-1-1-1-1\\ 1-110-10\\ ......\\ ......\\ ......\\ 1-110-10\\ 1-1010-1\\ ......\\ ......\\ ......\\ 1-1010-1\\ 1-1-1-111\\ ......\\ ......\\ ......\\ 1-1-1-111\end{array}\right]& \begin{array}{l}\\ \\ {\text{n}}_{\text{11}}\text{ rows for observations in cell 11}\\ \\ \\ \\ {\text{n}}_{\text{12}}\text{ rows for observations in cell 12}\\ \\ \\ \\ {\text{n}}_{\text{13}}\text{ rows for observations in cell 13}\\ \\ \\ \\ \\ \\ {\text{n}}_{\text{21}}\text{ rows for observations in cell 21}\\ \\ \\ \\ {\text{n}}_{\text{22}}\text{ rows for observations in cell 22}\\ \\ \\ \\ \\ {\text{n}}_{\text{23}}\text{ rows for observations in cell 23}\\ \\ \\ \\ \end{array}\end{array}$

With this set of restrictions or definitions of the parameters, the sums of squares that result from the following MODEL statement are summarized below:

model y=a b a*b / ss1 ss2 ss3 ss4;

Effect	Type I	Type II	Type III = Type IV
A	$R(\alpha |\mu )$	$R(\alpha |\mu ,\beta )$	$R(\alpha |\mu ,\beta ,\alpha \beta )$
B	$R(\beta |\mu ,\alpha )$	$R(\beta |\mu ,\alpha )$	$R(\beta |\mu ,\alpha ,\mathrm{\alpha \beta })$
A*B	$R(\mathrm{\alpha \beta }|\mu ,\alpha ,\beta )$	$R(\mathrm{\alpha \beta }|\mu ,\alpha ,\beta )$	$R(\mathrm{\alpha \beta }|\mu ,\alpha ,\beta )$

You should be careful when using reduction notation with less-than-full-rank models. If no restrictions had been specified on the model for the two-way structure above, then $\text{R}\left(\text{α}|\text{μ},\text{β},\text{α}\text{β}\right)$ because the columns of the X matrix corresponding to the α_i would be linearly dependent on the columns corresponding to μ and the αβ_ij.

In addition, the dependence of reduction notation on the restrictions imposed cannot be overemphasized. For example, imposing the restriction

α₂ = β₃ = αβ₂₁ = αβ₂₂ = αβ₁₃ = αβ₂₃ = 0    (6.2)

results in a different value for R(α | μ, β, αβ). Although the restrictions of equation (6.6) are those that correspond to the sums of squares computed by PROC GLM, the restrictions of equation 6.2 are those that correspond to the (biased) parameter estimates computed by PROC GLM.

There is a relationship between the four types of sums of squares and the four types of data structures in a two-way classification. The relationship derives from the principles of adjustment that the sums-of-squares types obey. Letting n_ijdenote the number of observations in level i of factor A and level j of factor B, the four types of data structures are

❏ equal cell frequencies: n_ij=common value for all i, j

❏ proportionate cell frequencies: n_ij/ n_il= n_kj/ n_kl for all i, j, k, l

❏ disproportionate, nonzero cell frequencies: n_ij/n_il=n_kj / n_kl for some i, j, k, l, but n_ij>0 for all i, j

❏ empty cell(s): n_ij=0 for some i, j.

The display below shows the relationship between sums-of-squares types and data structure types pertaining to the following MODEL statement:

model y=a b a*b / ss1 ss2 ss3 ss4;

For example, writing III=IV indicates that Type III is equal to Type IV.

Data Structure Type

	1	2	3	4
	(Equal nij)	(Proportionate nij)	(Disproportionate, nonzero nij)	(Empty Cell)
A	I=II=III=IV	I=II, III=IV	III=IV
B	I=II=III=IV	I=II, III=IV	I=II, III=IV	I=II
A*B	I=II=III=IV	I=II=III=IV	I=II=III=IV	I=II=III=IV

6.3.4 Interpreting Sums of Squares in the μ-Model Notation

The μ model for the two-way structure takes the form

y_ijk = μ_ij + ε_ijk    (6.3)

The parameters of the model relate to the parameters of the standard analysis-of-variance model according to the equation

μ_ij = μ + α_i + β_j + αβ_ij

This relation holds regardless of any restriction that may be imposed upon the α, β, and αβ parameters. The advantage of using the -model notation over standard analysis-of-variance notation is that all of the μ_ij parameters are clearly defined without specifying restrictions; thus, a hypothesis stated in terms of the μ_ij can be easily understood.

Speed et al. (1978) give interpretations of the different types of sums of squares (I, II, III, and IV) computed by PROC GLM using the -model notation. It is assumed that all n_ij>0, making Type III equal to Type IV.

Using their results, the sums of squares obtained from the following MODEL statement are expressed in terms of the μ_ij as given in Table 6.1.

model response = a b a*b / ss1 ss2 ss3 ss4;

Table 6.1 Interpretation of Sums of Squares in the -Model Notation

Type	Effect
	Effect A
I	(Σj n1j)/n1. = ... = (Σj najμaj)/na
II	$\begin{array}{l}{\Sigma }_j{n}_{1j}{\mu }_{1j}={\Sigma }_i{\Sigma }_j\left(n_{1j}{n}_{ij}{\mu }_{ij}/n_{.j}\right),...,{\Sigma }_j{n}_{aj}{\mu }_{aj}\\ ={\Sigma }_i{\Sigma }_j\left(n_{aj}{n}_{ij}{\mu }_{ij}/n_{.j}\right)\end{array}$
III & IV	${\mu }_{11}+...+{\mu }_{1b}=...={\mu }_{a1}+...+{\mu }_{ab}$ that is, ${\overline{\mu }}_{1.}=...={\overline{\mu }}_{a.}$ where ${\overline{\mu }}_{i.}={\Sigma }_j{\mu }_{ij}/b$
	Effect B
I & II	${\Sigma }_i{n}_{i1}{\mu }_{i1}={\Sigma }_i{\Sigma }_j(n_{i1}{n}_{ij}{\mu }_{ij}/n_{i.},...,{\Sigma }_i{n}_{ib}{\mu }_{ib}={\Sigma }_i{\Sigma }_j(n_{ib}{n}_{ij}{\mu }_{ij})/n_{i.})$
III & IV	${\mu }_{11}+...+{\mu }_{a1}=...{\mu }_{1b}+...+{\mu }_{ab}$ that is, ${\overline{\mu }}_{.1}=...={\overline{\mu }}_{.b}$ where ${\overline{\mu }}_{ij}={\Sigma }_i{\mu }_{ij}/a$
	Effect A*B
I, II, III, IV	μij − μim − μlj + μlm = 0 for all i, j, l, m

Table 6.1 shows that the tests can be expressed in terms of equalities of weighted cell means, only some of which are easily interpretable. Considering the Type I A effect, the weights n_ij/n_i.attached to μ_ij are simply the fraction of the n_i. observations in level i of A that were in level j of B. If these weights reflect the distribution across the levels of B in the population of units in level i of A, then the Type I test may have meaningful interpretation. That is, suppose the population of units in level i of A is made up of a fraction ρ_i1 of units in level 1 of B, of a fraction ρ_i2 of units in level 2 of B, and so on, where ρ_i1 +…+ρ_ib = 1. Then it may be reasonable to test

H₀ : Σ_jρ_1jμ_1j = ... = Σ_jρ_ajμ_aj

which would be the Type I test in case n_ij / n_i. = ρ_ij..

Practical interpretation of the Type II weights is more difficult—refer to Section 6.3.7.2, “Interpreting Sums of Squares Using Estimable Functions.” Recall that the Type II tests are primarily for main effects with no interaction. You can see from Table 6.1 that the Type II hypothesis clearly depends on the n_ij, the numbers of observations in the cells.

The interpretation of Type III and Type IV tests is clear because all weights are unity. When the hypotheses are stated in terms of the -model, the benefit of the Type III test is more apparent because the Type III hypothesis does not depend on the n_ij, the numbers of observations in the cells. Type I and Type II hypotheses do depend on the n_ij, and this may not be desirable.

For example, suppose a scientist sets up an experiment with ten plants in each combination of four levels of nitrogen (N) and three levels of lime (P). Suppose also that some plants die in some of the cells for reasons unrelated to the effects of N and P, leaving some cells with n_ij <10. A hypothesis test concerning the effects of N and P, which depends on the values of n_ij, would be contaminated by the accidental variation in the n_ij. The scientific method declares that the hypotheses to be tested should be stated before data are collected. It would be impossible to state Type I and Type II hypotheses prior to data collection because the hypotheses depend on the n_ij, which are known only after data are collected.

Note that the Type I and Type II hypotheses are different for effect A but the same for effect B. This occurs because the Type I sums of squares are model-order dependent. Being sequential, the Type I sums of squares are A (unadjusted), B (adjusted for A), and A*B (adjusted for A and B). Thus, the Type I sums of squares for the effects A and B listed in the MODEL statement in the order A, B, A*B would not be the same as in the order B, A, A*B. The Type II hypotheses are not model-order dependent because, for the two-way structure, both Type II main-effect sums of squares are adjusted for each other—that is, A (adjusted for B), B (adjusted for A), and A*B (adjusted for A and B). These Type II sums of squares are the partial sums of squares if no A*B interaction is specified in the model, in which case Type II, Type III, and Type IV would be the same.

Another interpretation of these tests is given by Hocking and Speed (1980), who point out that Type I, Type II, and Type III=Type IV tests for effect A each represent a test of

H₀: μ₁₁ +…+ μ_1b =…= μ_a1 +…+ μ_ab

subject to certain conditions on the cell means. The conditions are

Type I	no B effect, μ.1 = … = μ.k, and no A*B effect, μij – μim – μij + μim = 0 for all i, j, l, m
Type II	no A*B effect, μij – μim – μij + μim = 0 for all i, j, l, m
Type III=Type IV	none, (provided nij > 0 for all ij).

6.3.5 An Example of Unbalanced Two-Way Classification

This example is a two-factor layout with data presented by Harvey (1975). Two types of feed rations (factor A) are given to calves from three different sires (factor B). The observed dependent variable y_ijk (variable Y) is the coded amount of weight gained by each calf. Because unequal numbers of calves of each sire are fed each ration, this is an unbalanced experiment. However, there are observations for each ration-sire combination; that is, there are no empty cells. The data appear in Output 6.7.

Output 6.7 Data for an Unbalanced Two-Way Classification

The analysis-of-variance model for these data is

y_ijk = μ + α_i + β_j + αβ_ij + ε_ijk

where

i	equals 1, 2
j	equals 1, 2, 3.

i and j have elements as defined in Section 6.3.1, “General Considerations.” The model contains twelve parameters, which are more than can be estimated uniquely by the six cell means that are the basis for estimating parameters. The analysis is implemented with the following SAS statements:

proc glm;
   class a b;
   model y=a b a*b / ss1 ss2 ss3 ss4 solution;

The statements above cause PROC GLM to create the following twelve dummy variables:

❏ 1 dummy variable for the mean (or intercept)

❏ 2 dummy variables for factor A (ration)

❏ 3 dummy variables for factor B (sire)

❏ 6 dummy variables for the interaction A*B (all six possible pairwise products of the variables from factor A with those from factor B).

The options requested are SOLUTION, and for purposes of illustration, all four types of sums of squares. The results appear in Output 6.8.

Output 6.8 Sums of Squares for an Unbalanced Two-Way Classification

The first portion shows the statistics for the overall model. The overspecification of the model is obvious: The twelve dummy variables generate only six degrees of freedom (five for the terms listed in the MODEL statement plus one for the intercept).

The next portion of the output shows the four types of sums of squares. Note that Types III and IV give identical results. This is because n_ij>0 for all i, j.

As noted in Section 6.3.3, “Interpreting Sums of Squares in Reduction Notation,” the parameter estimates printed by PROC GLM are a solution to the normal equations corresponding to the restriction in equation (6.2.) The same condition applies to the parameter estimates

$\begin{array}{cc}{\widehat{\text{α}}}_2={\widehat{\text{β}}}_3={\widehat{\text{α}\text{β}}}_{13}={\widehat{\text{α}\text{β}}}_{21}={\widehat{\text{α}\text{β}}}_{22}={\widehat{\text{α}\text{β}}}_{23}=0& (6.4)\end{array}$

These values, which now equal 0, appear in Output 6.8.

Section 6.3.4, “Interpreting Sums of Squares in the -Model Notation,” also shows that the parameters of the model relate to the parameters of the standard analysis-of-variance model (see equation 6.3). A corresponding relation holds between the respective parameter estimates, namely

$\begin{array}{cc}{\widehat{y}}_{ij}={\widehat{\text{μ}}}_{ij}=\widehat{\text{μ}}+{\widehat{\text{α}}}_i+{\widehat{\text{β}}}_j+{\widehat{\text{α}\text{β}}}_{ij}& (6.5)\end{array}$

Putting equations (6.4) and (6.5) together as shown in the table below gives the interpretation of the parameter estimates printed by PROC GLM. The table below also shows the relationship between means and parameter estimates.

	Sire 1	Sire 2	Sire 3
Ration 1	$\begin{array}{l}{\widehat{\text{μ}}}_{11}={\overline{y}}_{11.}\\ =\widehat{\text{μ}}+{\widehat{\text{α}}}_1+{\widehat{\text{β}}}_1+\widehat{{\text{αβ}}_{11}}\end{array}$	$\begin{array}{l}{\widehat{\text{μ}}}_{12}={\overline{y}}_{12.}\\ =\widehat{\text{μ}}+{\widehat{\text{α}}}_1+{\widehat{\text{β}}}_2+\widehat{{\text{αβ}}_{12}}\end{array}$	$\begin{array}{l}{\widehat{\text{μ}}}_{13}={\overline{y}}_{13.}\\ =\widehat{\text{μ}}+{\widehat{\text{α}}}_1\end{array}$
Ration 2	$\begin{array}{l}{\widehat{\text{μ}}}_{21}={\overline{y}}_{21.}\\ =\widehat{\text{μ}}+{\widehat{\text{β}}}_1\end{array}$	$\begin{array}{l}{\widehat{\text{μ}}}_{22}={\overline{y}}_{22.}\\ =\widehat{\text{μ}}+{\widehat{\text{β}}}_2\end{array}$	$\begin{array}{l}{\widehat{\text{μ}}}_{23}={\overline{y}}_{23.}\\ =\widehat{\text{μ}}\end{array}$

---

Note especially the following items in Output 6.8:

❏ The intercept $\widehat{\mu }$ printed by PROC GLM is the cell mean ${\overline{y}}_{23.}$ for the lower right-hand cell (ration 2, sire 3).

❏ The estimate ${\overline{\alpha }}_1=-2.4$ is the difference between the cell means for the two rations fed to sire 3, ${\widehat{\alpha }}_1={\overline{y}}_{13}-{\overline{y}}_{23.}$.

❏ The interaction parameter estimate ${\widehat{\text{α}\text{β}}}_{11}$ = 5.4 is the interaction of ration 1 and ration 2 by sire 1 and sire 3, $\alpha {\beta }_{11}={\overline{y}}_{11.}-{\overline{y}}_{13.}-{\overline{y}}_{21.}+{\overline{y}}_{23.}$. Generally, in a two-way layout with a rows and b columns, the interaction parameter estimates αβ_ij and measures the interaction of rows i and a by columns j and b.

6.3.6 The MEANS, LSMEANS, CONTRAST, and ESTIMATE Statements in a Two-Way Layout

The parameter estimates printed by PROC GLM are the result of a computational algorithm and may or may not be the estimates with the greatest practical value. However, there is no single choice of estimates (corresponding to a particular generalized inverse or set of restrictions) that satisfies the requirements of all applications. In most instances, specific estimable functions of these parameter estimates, like the estimates obtained with the LSMEANS, CONTRAST, and ESTIMATE statements, can be used to provide more useful estimates. The CONTRAST and ESTIMATE statements for balanced data applications are discussed in Chapter 2.

The CONTRAST, LSMEANS, and ESTIMATE statements are similar for one- and two-way models, but principles and interpretations become more complex. Consider the results from the following SAS statements:

proc glm;
   class a b;
   model y=a b a*b;
   means a b;
   lsmeans a b / stderr;
   contrast 'A EFFECT' a -1 1;
   contrast 'B 1 vs 2 & 3' b -2 1 1;
   contrast 'B 2 vs 3' b 0 -1 1;
   contrast 'ALL B' b -2 1 1, b 0 -1 1;
   contrast 'A*B 2 vs 3' a*b 0 1 -1 0 -1 1;
   estimate 'B2, B3 MEAN' intercept 1 a .5 .5 b 0 .5 .5
                                 a*b 0 .25 .25 0 .25 .25;
   estimate 'A in B1' a -1 1 a*b -1 0 0 1;

The MEANS statement provides the raw or unadjusted main-effect and interaction means. The LSMEANS statement produces least-squares (adjusted) means for main effects together with their standard errors. The results of these two statements are combined in Output 6.9. (PROC GLM prints these results on separate pages.)

Output 6.9 Results of the MEANS and LSMEANS Statements for a Two-Way Classification

The raw and least-squares means are different for all levels except B1, which is balanced with respect to factor A.

Quantities estimated by the raw means and least-squares means can be expressed in terms of the μ model. For level 1 of factor A, the raw mean (4.625) is an estimate of (2μ₁₁ + μ₁₂ + μ₁₃)/(2 + 5 + 1), whereas the least-squares mean (4.367) is an estimate of (μ₁₁ + μ₁₂ + μ₁₃)/3. The raw means estimate weighted averages of the μ^ij whose weights are a function of sample sizes. The least-squares means estimate unweighted averages of the μ^ij.

The results of the five CONTRAST statements appear in Output 6.10.

Output 6.10 Contrast in a Two-Way Classification

The first four CONTRAST statements are similar to those presented for the one-way structure. Note that when a contrast uses all available degrees of freedom for the factor (such as the ALL B contrast), the sums of squares are the same as the Type III sums of squares for the factor.

The fifth CONTRAST statement requests the interaction between the factor A contrast and the B2 vs 3 contrast. It is constructed by computing the product of corresponding main-effect contrasts for each AB treatment combination. The procedure is illustrated in the table below:

Construction of Interaction Contrast
			Level of Factor B
		1	2	3
Level of Factor A	Factor A Contrast	Factor B Contrast
		0	1	–1
1	1	0	1	–1
2	–1	0	–1	1

Main-effect contrasts are given on the top and left, and interaction contrasts (products of marginal entries) are given in the body of the table. These are inserted into the CONTRAST statement in the same order of interaction cells as indicated by the CLASS statement (levels of B within levels of A).

In terms of the μ model, the hypothesis tested by the F-statistic for this interaction contrast is

H₀: μ₁₂ − μ₁₃ − μ₂₂ + μ₂₃ = 0

The two ESTIMATE statements request estimates of linear functions of the model parameters. The first function to be estimated is the average of the cell means for levels 2 and 3 of factor B. The other statement requests an estimate of the effect of factor A within level 1 of factor B, or an estimate of μ₂₁ – μ₁₁. Output 6.11 summarizes results from the ESTIMATE statements.

Output 6.11 Parameter Estimates in a Two-Way Classification

Expressing each comparison in terms of model parameters α_i, β_j, (αβ)_ij is the key to filling in the coefficients of the CONTRAST and ESTIMATE statements. Consider the ESTIMATE A in B1 statement, which is used to estimate μ₂₁ – μ₁₁. Writing this expression as a function of the model parameters by substituting

μ_ij = μ + α_i + β_j + αβ_ij

yields

μ₂₁ − μ₁₁ = − α₁ + α₂ − α β₁₁ + αβ₂₁

The −α₁ + α₂ term tells you to insert A − 1 1 into the ESTIMATE statement. There are no β’s in the function, so no B expression appears in the ESTIMATE statement. The −αβ₁₁ + αβ₂₁ term tells you to insert A*B–1 0 0 1 0 0, or, equivalently, A*B–1 0 0 1 into the ESTIMATE statement. The ordering in the statement

class a b;

specifies that the ordering of the coefficients following A*B corresponds to αβ₁₁ αβ₁₂ αβ₁₃ αβ₂₁ αβ₂₂ αβ₂₃. The SAS statement

class b a;

would indicate an ordering that corresponds to αβ₁₁ αβ₂₁ αβ₁₂ αβ₂₂ αβ₁₃ αβ₂₃.

Now consider the CONTRAST A statement. The hypothesis to be tested is

H₀: −(μ₁₁ + μ₁₂ + μ₁₃)/3 + (μ₂₁ + μ₂₂ + μ₂₃)/ 3 = 0

Substituting μ_ij + α_i + β_j + αβ_ij gives the equivalent hypothesis:

H₀: −α₁ + α₂ − (αβ₁₁ + αβ₁₂ + αβ₁₃)/3
  + (αβ₂₁ + αβ₂₂ + αβ₂₃) / 3 = 0

Again the −α₁ + α₂ term in the function tells you to insert A – 1 1. This brings up an important usage note: Specifying A – 1 1 causes the coefficients of the A*B interaction term to be automatically included by PROC GLM. That is, the SAS statement

contrast 'A' a -1 1;

is equivalent to the statement

contrast 'A' a -1 1 a*b -.333333 -.333333 -.333333 .333333 .333333 .333333;

Similarly, the SAS statement

estimate 'A EFFECT' a -1 1;

provides an estimate of

– (μ₁₁ + μ₁₂ + μ₁₃)/3 + (μ₂₁ + μ₂₂ + μ₂₃)/3

without explicitly specifying the coefficients of the αβ_ij terms. However, you should note that specifying the αβ_ij coefficients does not cause PROC GLM to automatically include coefficients for α_i or β_j. For example, the term A – 1 1 must appear in the ESTIMATE A in B1 statement. Similarly, a contrast to test H₀: −μ₁₁ + μ₂₁ = 0 requires the following statement:

contrast 'A in B1' a -1 1 a*b -1 0 0 1;

The A – 1 1 term must be included.

6.3.7 Estimable Functions for a Two-Way Classification

The previous section discussed the application of the CONTRAST statement, which employs the concept of estimable functions. PROC GLM can display the construction of estimable functions as an optional request in the MODEL, LSMEANS, CONTRAST, and ESTIMATE statements. This section discusses the construction of estimable functions and their relation to the sums of squares and associated hypotheses available in the GLM procedure, and to CONTRAST, ESTIMATE, and LSMEANS statements. The presentation of estimable functions consists of results obtained using the unbalanced factorial data given in Output 6.7. For more thorough discussions of these principles, see Graybill (1976) and Searle (1971).

6.3.7.1 The General Form of Estimable Functions

The general form of estimable functions is a vector of elements that are the building blocks for generating specific estimable functions. The number of unique symbols in the vector represents the maximum number of linearly independent coefficients estimated by the model, which is equal to the rank of the X′X matrix. In the GLM procedure this is obtained by the E option in the MODEL statement

proc glm;
   class a b;
   model y=a b a*b / e solution;

Table 6.2 gives the vector of coefficients of the general form of estimable functions for our example. There are only six elements (L1, L2, L4, L5, L7, L8), which correspond to the number of degrees of freedom in the model (including the intercept). The number of elements for an effect corresponds to the degrees of freedom for that effect; for example, L4 and L5 are introduced opposite the effect B, indicating B has 2 degrees of freedom.

Table 6.2 General Form of Estimable Functions

Effect			Parameters*	Coefficients
Intercept			μ	L1
A	1		α1	L3
	2		α2	L1-L2
B	1		β1	L4
	2		β2	L5
	3		β3	L1 – L4-L5
A*B	1	1	αβ11	L7
	1	2	αβ12	L8
	1	3	αβ13	L2 – L7 – L8
	2	1	αβ21	L4 – L7
	2	2	αβ22	L5 – L8
	2	3	αβ23	L1 – L2 – L4 – L5 + L7 + L8

* These are implied by the output but not printed in this manner.

According to Table 6.2, any estimable function Lβ must be of the form

$\begin{array}{ll}\text{Lβ=L1}\text{μ}\text{+L2}{\text{α}}_1+(\text{L1}-\text{L2}){\text{α}}_2+\text{L}4\text{α}{\text{β}}_1+\text{L5}{\text{β}}_2+(\text{L1}-\text{L4}-\text{L5}){\text{β}}_3\hfill & \hfill \\ +\text{L7}\text{α}{\text{β}}_{11}+\text{L8}\text{α}{\text{β}}_{12}+(\text{L2}-\text{L7}-\text{L8})\text{α}{\text{β}}_{13}+(\text{L4}-\text{L7})\text{α}{\text{β}}_{21}\hfill & (6.6)\hfill \\ +(\text{L5}-\text{L8})\text{α}{\text{β}}_{22}+(\text{L1}-\text{L2}-\text{L4}-\text{L5+L7+L8})\text{α}{\text{β}}_{23}\hfill & \hfill \end{array}$

for some specific values of L1 through L8. The various tests in PROC GLM test hypotheses of the form H₀: Lβ =0.

Coefficients for any specific estimable function are constructed by assigning values to the individual L’s. For example, setting L2=1 and all others equal to 0 provides the estimable function α₁ − α₂ + αβ₁₃ − αβ₂₃. It is clear, however, that no estimable function can be constructed in this manner to equal 1 or 2 individually. That is, no matter what values you choose for L1 through L8, you cannot make Lβ = α₁ or Lβ = α₂. This is because α₁ and α₂ are non-estimable functions; without additional restrictions there is no linear function of the data whose expected value is α₁ or α₂.

6.3.7.2 Interpreting Sums of Squares Using Estimable Functions

The coefficients required to construct estimable functions for each effect in the MODEL statement are available for any type of sum of squares requested as an option in the MODEL statement. For example,

model y = a b a*b / e e1 e2 e3;

will provide the general form, the coefficients of estimable functions for Types I, II, and III, and the corresponding sums of squares for each effect listed in the MODEL statement.

Table 6.3 gives the coefficients of the Type I, Type II, and Type III estimable functions associated with factor A. Types III and IV are identical for this example because all n_ij > 0.

Table 6.3Estimable Functions for Factor A

Effect			Type I Parameters	Type II Coefficients	Type III Coefficients
Intercept			0	0	0
A	1		L2	L2	L2
	2		–L2	–L2	–L2
B	1		0.05*L2	0	0
	2		0.325*L2	0	0
	3		–0.375*L2	0	0
A*B	1	1	0.25*L2	0.2697*L2	0.3333*L2
	1	2	0.625*L2	0.5056*L2	0.3333*L2
	1	3	0.125*L2	0.2247*L2	0.3333*L2
	2	1	–0.2*L2	–0.2697*L2	–0.3333*L2
	2	2	–0.3*L2	–0.5056*L2	–0.3333*L2
	2	3	–0.5*L2	–0.2247*L2	–0.3333*L2

All coefficients involve only one element (L2), since the A effect has only 1 degree of freedom. Estimable functions are constructed by assigning specific values to the elements. For factor A, with only one variable, the best choice is L2=1. Application to the Type I coefficients generates the estimable function

$\begin{array}{l}\text{Lβ}={\text{α}}_1-{\text{α}}_2\\ +0.05{\text{β}}_1+0.325{\text{β}}_2-0.375{\text{β}}_2\\ +0.25\text{α}{\text{β}}_{11}+0.625\text{α}{\text{β}}_{12}+0.125\text{α}{\text{β}}_{13}\\ -0.2\text{α}{\text{β}}_{21}-0.3\text{α}{\text{β}}_{22}-0.5\text{α}{\text{β}}_{23}\end{array}$

Thus, using the Type I sum of squares in the numerator of an F-statistic tests the hypothesis Lβ = 0 for this particular Lβ. In addition to α₁ − α₂, this Lβ also involves the function of coefficients of factor B

0.5β₁ + 0.325β₂ − 0.375β₃

as well as a function of the interaction parameters. Actually, this is to be expected, since the Type I function for A is unadjusted—it is based on the difference between the two A factor means $\left({\bar{y}}_{\mathrm{1..}}-{\bar{y}}_{\mathrm{2..}}\right)$.

As explained in Section 6.3.1, the mean for A is

${\overline{y}}_{\mathrm{1..}}=1/8\left[\left(2{\overline{y}}_{11.}+5{\overline{y}}_{12.}+{\overline{y}}_{13.}\right)\right]$

Each cell mean ${\bar{y}}_{ij.}$ is an estimate of the function μ + α_ii + β_j + αβ_ij. Omitting for this discussion the interaction parameters, ${\bar{y}}_{\mathrm{1..}}$ is an estimate of

$\begin{array}{l}1/8\left[2(\text{μ+}{\text{α}}_1+{\text{β}}_1)+5(\text{μ+}{\text{α}}_1+{\text{β}}_2)+(\text{μ+}{\text{α}}_1+{\text{β}}_3)\right]\\ =\text{μ+}{\text{α}}_1+\left(0.25{\text{β}}_1+0.625{\text{β}}_2+0.125{\text{β}}_3\right)\end{array}$

Likewise ${\bar{y}}_{\mathrm{2..}}$ is an estimate of

μ + α₂ + (0.2β₁ + 0.3 β₂ + 0.5β₃)

Hence, $\left({\bar{y}}_{\mathrm{1..}}-{\bar{y}}_{\mathrm{2..}}\right)$ is an estimate of

a₁ − α₂ + 0.05β₁ + 0.325β₂ − 0.375β₃

which is the function provided by Type I.

The coefficients associated with A*B provide the coefficients of the interaction terms in the Type I estimable functions. The coefficients associated with A*B are useful for expressing the estimable functions and interpreting the tests in terms of the model. Recall that

μ_ij = μ + α_i + β_j + αβ_ij

A little algebra shows that any estimable Lβ with coefficients as given in Table 6.3 can also be written as

$\begin{array}{cc}\begin{array}{c}\text{L}\text{β}=\text{L}7{\text{μ}}_{11}+\text{L8}{\text{μ}}_{12}+(\text{L2}-\text{L7}-\text{L8}){\text{μ}}_{13}+(\text{L}4-\text{L}7){\text{μ}}_{21}\\ +(\text{L}5-\text{L}8){\text{μ}}_{22}+(\text{L}1-\text{L}2-\text{L}4-\text{L}5+\text{L}8){\text{μ}}_{23}\end{array}& (6.7)\end{array}$

This is easily verified by starting with Lβ in equation (6.7), replacing each μ_ij with μ + α_i + β_j + αβ_ij, and combining terms to end up with the original expression for Lβ in equation 6.6. For example, after factoring out L2, we see that the Type I estimable function for A is

$\mathbf{\text{L}}\mathbf{\text{β}}=\text{L2(0}\text{.25}{\text{μ}}_{11}\text{+0}\text{.625}{\text{μ}}_{12}\text{+0}\text{.125}{\text{μ}}_{13}-0.2{\text{μ}}_{21}-\text{0}\text{.3}{\text{μ}}_2-\text{0}\text{.5}{\text{μ}}_{23}\text{)}$

Thus the Type I F-test for A tests the hypothesis H₀: Lβ = 0, or equivalently,

H₀: 0.25 μ₁₁ + 0.625 μ₁₂ + 0.125 μ₁₂ = 0.2 μ₂₁ + 0.3 μ₂₂ + 0.5 μ₂₃

This is the hypothesis that is tested in Table 6.1. Since the coefficients are functions of the frequencies of the cells, the hypothesis might not be particularly useful.

Applying the same method to the Type II coefficients for A, we have, after setting L2=1,

$\begin{array}{cc}\mathbf{\text{L}}\mathbf{\text{β}}& =.2697{\text{μ}}_{11}+.5056{\text{μ}}_{12}+.2247{\text{μ}}_{13}-.2697{\text{μ}}_{21}-.5056{\text{μ}}_{22}-.2247{\text{μ}}_{23}\\ & =.2697({\text{μ}}_{11}-{\text{μ}}_{21})+.5056({\text{μ}}_{12}-{\text{μ}}_{22})+.2247({\text{μ}}_{13}-{\text{μ}}_{23})\end{array}$

This expression sheds some light on the meaning of the Type II coefficients. Recall that the Type II SS are based on a main-effects model. With no interaction we have, for example, μ₁₁ − μ₂₁ = μ₁₂ − μ₂₂ = μ₁₃ − μ₂₃. Let Δ denote the common value of these differences. The Type II coefficients are the coefficients of the best linear unbiased estimate $\widehat{\Delta }$ of Δ given by

$\widehat{\Delta }=\Sigma w_j({\bar{y}}_{1j.}-{\bar{y}}_{2j.})$

where

$w_j=\frac{n_{1j}{n}_{2j}/\left(n_{1j}+n_{2j}\right)}{{\displaystyle {\sum }_k\left(n_{1k}{n}_{2k}/\left(n_{1k}+n_{2k}\right)\right)}}$

For example,

$.2697=\frac{(2)(2)/(2+2)}{(2)(2)/(2+2)+(5)(3)/(5+3)+(1)(5)/(1+5)}$

Note that these are functions of cell frequencies and thus do not necessarily generate meaningful hypotheses.

Type III (and Type IV) estimable functions for A likewise (Table 6.2) do not involve the parameters of the B factor. Further, in terms of the parameters of the cell means (μ model)

Lβ = 1/3(μ₁₁ + μ₁₂ + μ₁₃) − 1/3(μ₂₁ + μ₂₂ + μ₂₃)

Thus the Type III F-statistic tests ${\text{H}}_{\text{0}}:{\bar{\mu }}_{1.}={\bar{\mu }}_{2.}$ as stated in Table 6.1. Note again that this hypothesis does not involve the cell frequencies, n_1j.

Table 6.3 gives the coefficients of estimable functions for factor B. There are 2 degrees of freedom for factor B, thus two elements, L4 and L5. Consider first the Type III coefficients because they are more straightforward. The Type III F-test for factor B is testing simultaneously that any two linearly independent functions are equal to 0; functions are obtained by selecting two choices of values for L4 and L5.

The simplest choices are to take L4=1, L5=0 and L4=0, L5=1. This gives the estimable functions

L₂β = β₂ − β₃ + (αβ₁₂ − αβ₁₃ + αβ₂₂ − αβ₂₃)/2

and

L₂β = β₂ − β₃ + (αβ₁₂ − αβ₁₃ + αβ₂₂ − αβ₂₃) / 2

In terms of the μ model, this gives

L₁β = (μ₁₁ − μ₁₃ + μ₂₁ − μ₂₃) / 2

and

L₂β = (μ₁₂ − μ₁₃ + μ₂₂ − μ₂₃) / 2

Thus, the Type III F-statistic tests

$H_0:{\bar{\mu }}_{.1}={\bar{\mu }}_{.3}and{\bar{\mu }}_{.2}={\bar{\mu }}_{.3}$

or, in equivalent form

$\begin{array}{cc}{\text{H}}_{\text{0}}:{\overline{\text{μ}}}_{.1}={\overline{\text{μ}}}_{.2}\text{=}{\overline{\text{μ}}}_{.3}& (6.8)\end{array}$

Another set of choices is L4=1, L5=1 and L4=L5=–1. These lead to

$H_0:{\bar{\mu }}_{.1}={\bar{\mu }}_{.2}and({\bar{\mu }}_{.1}+{\bar{\mu }}_{.2})/2={\bar{\mu }}_{.3}$

which is also equivalent to equation (6.8). Therefore, both sets of choices lead to the same H₀. It is significant that the H₀ in equation (6.8) is independent of cell frequencies and, thus, is desirable for the usual case where cell frequencies are unrelated to the effects of the factors. Table 6.4 gives the coefficients of the Type I & Type II and Type III & Type IV estimable functions associated with factor B.

Table 6.4 Estimable Functions for Factor B

Effect			Type I & Type II Coefficients	Type III & Type IV Coefficients
Intercept			0	0
A	1		0	0
	2		0	0
B	1		L4	L4
	2		L5	L5
	3		–L4–L5	–L4–L5
A*B	1	1	0.401*L4-0.1236*L5	0.5*L4
	1	2	–0.1658*L4+0.3933*L5	0.5*L5
	1	3	–0.2416*L4–0.2697*L5	–0.5*L4–0.5*L5
	2	1	0.5899*L4+0.1236*L5	0.5*L4
	2	2	–0.1685*L4+0.6067*L5	0.5*L5
	2	3	–0.7584*L4+0.7303*L5	–0.5*L4–0.5*L5

 

Recall that since B followed A in the MODEL statement, the Type I SS for B is the same as the Type II SS for B. The coefficients are again a function of the cell frequencies. The nature of the function is not easy to determine but is similar to the Type II coefficients for factor A (see Table 6.1).

As a matter of computational interest, the Type II estimable functions for B are equal to the Type III estimable functions if there is no interaction. For then αβ_ij = 0, and Table 6.4 shows that the coefficients for α_i and β_j are the same for Types II and III. This is not to say that the Type II SS and Type III SS will be equal, but rather that they give tests of the same hypothesis when there is no interaction. If, indeed, there is no interaction, then the Type II F-test is more powerful than the Type III F-test. The assumption of no interaction is, however, probably rarely satisfied in nature.

Table 6.5 Estimable Functions for A*B

Effect		Coefficients for All Types
Intercept		0
A	1	0
0	2	0
B	1	0
	2	0
	3	0
A*B	1	L7
	1	L8
	1	–L7–L8
	2	–L7
	2	–L8
	2	L7+L8

 

Table 6.5 gives the coefficients of the estimable function for the A*B interaction and, again, two elements are available. In this case all types of effects give the same results, since for each type the interaction effects are adjusted for all other effects. The estimable functions can be readily interpreted if the coefficients are recorded in the 2×3 cell format implied by the factorial array. For example, let L7 = –1 and L8 = 0; the resulting function can be illustrated as follows:

		B
		1	2	3
A	1	–1	0	+1
	2	+1	0	–1

This is the interaction in the 2×2 subtable consisting of the columns for B1 and B3, or the interaction of the contrast (α₁ − α₂) with the contrast (β₁ − β₂).

6.3.7.3 Estimating Estimable Functions

Estimates of estimable functions can be obtained by multiplying the vector of coefficients by the vector of parameter estimates using the SOLUTION option. For example, letting L2=1 in Table 6.3 for Type I results in the vector

L = (0 1 −1 .05 .325 −.375 −.375 .25 .625 .125 −.2 −3 −.5)

The vector of parameter estimates (see Output 6.7) is

β̂ = (5.4 −2.4 .0 −2.9 2.933 .0 5.4 −1.33 .0 .0 .0 .0)

The estimate $\mathbf{L}\widehat{\mathbf{\beta }}$ = 1.075 is equal to ${\bar{y}}_{\mathrm{1..}}-{\bar{y}}_{\mathrm{2..}}$, the unadjusted treatment difference. Likewise, using the Type III coefficients gives an estimate of 1.044, which is the difference between the two least-squares means of the A factor (see Table 6.8).

Variances of these estimates can be obtained by the standard formula for the variance of a linear function. The estimated variance of the estimates is s²(L(X′X)⁻ L′), where s², the estimated error variance, is the residual mean square from the overall analysis of variance, and (X′X)– is the generalized inverse of the X′X matrix generated by the dummy variables. The square root of this variance provides the standard error of the estimated function; hence, a t-test is readily constructed.

Since the LSMEANS, CONTRAST, and ESTIMATE statements offer methods of estimating and testing the most desired functions, the preceding technique is seldom employed. However, if these statements produce functions that are nonestimable, the generation of estimates from scratch may provide otherwise unobtainable estimates.

6.3.7.4 Interpreting LSMEANS, CONTRAST, and ESTIMATE Results Using Estimable Functions

Sometimes it may be useful to examine the construction of estimable functions associated with the LSMEANS, CONTRAST, and ESTIMATE statements. Information on the construction of these functions is available by specifying E as one of the options in the statement. (Don’t confuse this with the E=effect option, which specifies an alternate error term.) Output 6.12 shows the results from the E option:

proc glm;
   class a b;
   model y=a b a*b / solution;
   contrast 'B 2 vs 3' b 0 -1 1 / e;
   estimate 'A in B1' a -1 1 a*b -1 0 0 1 / e;
   lsmeans a / stderr e;

Output 6.12 Estimable Functions for the LSMEANS, CONTRAST, and ESTIMATE Statements

The hypothesis tested by the CONTRAST statement is

H₀: − β₂ + β₃ − .5αβ₁₂ + .5αβ₁₃ − .5αβ₂₂ + .5αβ₂₃ = 0

or in the μ-model notation

−.5μ₁₂ + .5 μ₁₃ − .5μ₂₂ + .5μ₂₃

Note that the coefficients of the interaction effects are supplied by the procedure.

The function estimated by the ESTIMATE statement is

Lβ = − α₁ + α₂ − αβ₁₁ + αβ₂₁

or in μ-model notation

Lβ = − μ₁₁ + μ₂₁

The least-squares means for A1 estimates is

μ + α₁ + (β₁ + β₂ + β₃ + αβ₁₁ + αβ₁₂ + αβ₁₃) / 3

or, in μ-model notation

(μ₁₁ + μ₁₂ + μ₁₃) / 3