3.9. Variance and Bias Analyses for Calibration Problems
This section presents an application of the multivariate analysis of variance techniques to test for equality of variances in several measuring devices. The approach is taken from Christensen and Blackwood (1993). Such calibration problems occur frequently in assessing the relative measurement quality of various instruments or of a particular instrument at various times.
Suppose yij i = 1,..., n, j = 1,..., q are the measurements for a random sample of n items distributed around their respective true values, each measured by q instruments or at q different laboratories. If αj is the fixed bias of the jth instrument and ϵij are the independent random errors with zero mean and variance σj2, then the problem of testing the equality of the error variances, namely
H0: σ12 = σ22 = ... = σq2,
can be reduced to a MANOVA problem with the test based on the multivariate regression coefficients.
Let us define, for each of n items, 
as the average measurement for the ith item and 
as the deviation of each measurement on ith item from the corresponding sample mean 
. Then, Christensen and Blackwood (1993) have shown that the testing problem stated above is equivalent to testing
H0: β11 = β12 = ... = β1,q−1=0
in the multivariate linear model
j = 1,...,(q−1); i = 1,...,n. In the context of above model, the null hypothesis H0 can be interpreted as a claim that the observed deviations from the mean do not depend on the mean itself. The above model in matrix form is written as
or
Y = XB + ɛ
and our null hypothesis is that the vector of the slope parameters for the (q−1) variables is equal to zero. This is one of the standard null hypotheses that can be easily tested using PROC GLM or PROC REG.
In the above discussion, 
, i = 1,...,n have not been included in the model. In general, any other set, 
, i = 1,...,n could have been dropped instead. Although the data thus created would differ, the resulting test statistics and hence the conclusions are invariant of any such choice.
EXAMPLE 7
Equality of Variances in Calibration of Thermocouples Christensen and Blackwood (1993) described a study in which 64 measurements on a high temperature furnace were taken by each of five thermocouples that had been bound together and inserted in the furnace. The objective is to test if all five thermocouples have the same precision. In other words, in this example we want to test the hypothesis of the equality of the variances for the five thermocouples.
The data and the corresponding SAS code for analysis are presented in Program 3.8. From the five temperature variables TC1 through TC5 corresponding to these thermocouples and their average TCBAR = (TC1 + TC2 +... + TC5)/5 the variables Y1TILDA through Y5TILDA are defined by taking the differences from their mean TCBAR. For these data, since q=5, we only need to take q−1=4 of these five variables as the response variables to fit a multivariate regression model with TCBAR as the independent variable. We can choose any four of these five as the response variable. As mentioned previously, the values of the test statistics and the corresponding P values are unaffected by any such choice.
Testing the hypothesis of equality of variances of TC1 through TC5 is equivalent to testing the hypothesis that the slope parameters corresponding to the independent variable TCBAR are all zero in the corresponding multivariate linear model. This hypothesis is tested by using the SAS code
proc glm; model y2tilda y3tilda y4tilda y5tilda = tcbar/nouni; manova h = tcbar/printe printh;
or alternatively by using
proc reg; model y2tilda y3tilda y4tilda y5tilda = tcbar; EqualVar: mtest tcbar/print;
The latter choice is used while executing the Program 3.8, and the output is presented in Output 3.8. All the multivariate tests are exact in this case and are also equivalent. Corresponding to the observed value of F (4, 59) as 3.6449, the p value is equal to .0101 leading to the rejection of the null hypothesis of equality of the five variances. The next step may be to determine the equality of various variances in subgroups. The slope coefficients for regressing Y2TILDA through Y5TILDA on TCBAR respectively, obtained from the corresponding outputs of univariate analysis, (not shown), are 0.1288, 0.0242, −0.0697, and −0.0808. Because the five response variables sum to zero in this model the sum of all five coefficients (corresponding to Y1TILDA through Y5TILDA) equals zero. Thus by difference, the slope coefficient for regressing Y1TILDA on TCBAR is −0.0025. Further, the smaller regression coefficients correspond to smaller variances. Hence the five variances can be ordered as
σ52,σ42, σ12, σ32, σ22.
Using the Student-Newman-Keuls test (Kuehl, 1994), Christensen and Blackwood (1993) summarized the grouping as given below.
See Christensen and Blackwood (1993) for further details.
/* Program 3.8 */
options ls=64 ps=45 nodate nonumber;
title1 'Output 3.8';
title2 'Testing the Precisions of Five Thermocouples';
data calib ;
infile 'thermoco.dat' obs=64;
input tc1 tc2 tc3 tc4 tc5 @@ ;
tcbar = (tc1+tc2+tc3+tc4+tc5)/5 ;
y1tilda = tc1 − tcbar ;
y2tilda = tc2 − tcbar ;
y3tilda = tc3 − tcbar ;
y4tilda = tc4 − tcbar ;
y5tilda = tc5 − tcbar ;
/*
proc glm data = calib;
model y2tilda y3tilda y4tilda y5tilda = tcbar /nouni;
manova h = tcbar/printe printh;
*/
proc reg data = calib ;
model y2tilda y3tilda y4tilda y5tilda = tcbar;
EqualVar:mtest tcbar/print;
Bias_Var:mtest intercept, tcbar/print;
run;Example 3.8. Output 3.8
Testing the Precisions of Five Thermocouples
Multivariate Test: EQUALVAR
E, the Error Matrix
0.0475701116 −0.011049524 −0.009544794 −0.013674773
−0.011049524 0.0129013449 0.0015628105 −0.003486742
−0.009544794 0.0015628105 0.0075360191 −0.0005665
−0.013674773 −0.003486742 −0.0005665 0.0173898899
H, the Hypothesis Matrix
0.0036643259 0.0006892739 −0.001983269 −0.002297977
0.0006892739 0.0001296551 −0.000373061 −0.000432258
−0.001983269 −0.000373061 0.0010734184 0.00124375
−0.002297977 −0.000432258 0.00124375 0.0014411101
Multivariate Statistics and Exact F Statistics
S=1 M=1 N=28.5
Statistic Value F Num DF Den DF Pr > F
Wilks' Lambda 0.801854 3.6449 4 59 0.0101
Pillai's Trace 0.198146 3.6449 4 59 0.0101Hotelling-Lawley Trace 0.247109 3.6449 4 59 0.0101
Roy's Greatest Root 0.247109 3.6449 4 59 0.0101
Multivariate Test: BIAS_VAR
E, the Error Matrix
0.0475701116 −0.011049524 −0.009544794 −0.013674773
−0.011049524 0.0129013449 0.0015628105 −0.003486742
−0.009544794 0.0015628105 0.0075360191 −0.0005665
−0.013674773 −0.003486742 −0.0005665 0.0173898899
H, the Hypothesis Matrix
295.33648078 -263.4663775 67.96997654 168.39596614
-263.4663775 235.03968978 -60.63831058 −150.2289007
67.96997654 -60.63831058 15.645075922 38.758738355
168.39596614 −150.2289007 38.758738355 96.021841751
Multivariate Statistics and F Approximations
S=2 M=0.5 N=28.5
Statistic Value F Num DF Den DF Pr > F
Wilks' Lambda 0.000018 3484.1 8 118 0.0001
Pillai's Trace 1.148626 20.237 8 120 0.0001
Hotelling-Lawley Trace 47904.34 347306 8 116 0.0001
Roy's Greatest Root 47904.16 718562 4 60 0.0001
NOTE: F Statistic for Roy's Greatest Root is an upper bound.
NOTE: F Statistic for Wilks' Lambda is exact. |
Another problem of interest in calibration is to test for the equality of both the device variances σi2 and biases αi2, i = 1,..., q. Christensen and Blackwood (1993) showed that with yij = yij, j = 1,...,q − 1 as defined earlier, the null hypothesis of equality of variances and equality of biases is equivalent to testing
in the linear model 
= β0j + β1j +ϵij, j = 1,...,q−1; i = 1,...,n. Thus the hypothesis is on slope coefficients as well as the intercepts. To test this null hypothesis using SAS, we can, as earlier, use the MTEST statement in PROC REG. The SAS code for this is given at the end of Program 3.8. All four multivariate tests lead to very small p values, for example, the Wilks' Λ results in the observed value of exact F(8, 118) as 3484.124, leading to a p value of 0.0001. Consequently, we conclude that for at least two thermocouples either the variances, or the biases, or both of the temperatures are unequal. For further discussion of analysis involving biases and variances, see Christensen and Blackwood (1993).