Design and Analysis of Single-Factor Experiments: The Analysis of Variance

Experiments are a natural part of the engineering and scientific decision-making process. Suppose, for example, that a civil engineer is investigating the effects of different curing methods on the mean compressive strength of concrete. The experiment would consist of making up several test specimens of concrete using each of the proposed curing methods and then testing the compressive strength of each specimen. The data from this experiment could be used to determine which curing method should be used to provide maximum mean compressive strength.

If there are only two curing methods of interest, this experiment could be designed and analyzed using the statistical hypothesis methods for two samples introduced in Chapter 10. That is, the experimenter has a single factor of interest—curing methods—and there are only two levels of the factor. If the experimenter is interested in determining which curing method produces the maximum compressive strength, the number of specimens to test can be determined from the operating characteristic curves in Appendix Chart VII, and the t-test can be used to decide if the two means differ.

Many single-factor experiments require that more than two levels of the factor be considered. For example, the civil engineer may want to investigate five different curing methods. In this chapter, we show how the analysis of variance (frequently abbreviated ANOVA) can be used for comparing means when there are more than two levels of a single factor. We also discuss randomization of the experimental runs and the important role this concept plays in the overall experimentation strategy. In the next chapter, we show how to design and analyze experiments with several factors.

13-1 Designing Engineering Experiments

Statistically based experimental design techniques are particularly useful in the engineering world for solving many important problems: discovery of new basic phenomena that can lead to new products and commercialization of new technology including new product development, new process development, and improvement of existing products and processes. For example, consider the development of a new process. Most processes can be described in terms of several controllable variables, such as temperature, pressure, and feed rate. By using designed experiments, engineers can determine which subset of the process variables has the greatest influence on process performance. The results of such an experiment can lead to

• Improved process yield
• Reduced variability in the process and closer conformance to nominal or target requirements
• Reduced design and development time
• Reduced cost of operation

Experimental design methods are also useful in engineering design activities during which new products are developed and existing ones are improved. Some typical applications of statistically designed experiments in engineering design include

• Evaluation and comparison of basic design configurations
• Evaluation of different materials
• Selection of design parameters so that the product will work well under a wide variety of field conditions (or so that the design will be robust)
• Determination of key product design parameters that affect product performance

The use of experimental design in the engineering design process can result in products that are easier to manufacture, products that have better field performance and reliability than their competitors, and products that can be designed, developed, and produced in less time.

Designed experiments are usually employed sequentially. That is, the first experiment with a complex system (perhaps a manufacturing process) that has many controllable variables is often a screening experiment designed to determine those variables are most important. Subsequent experiments are used to refine this information and determine which adjustments to these critical variables are required to improve the process. Finally, the objective of the experimenter is optimization, that is, to determine those levels of the critical variables that result in the best process performance.

Every experiment involves a sequence of activities:

1. Conjecture—the original hypothesis that motivates the experiment.
2. Experiment—the test performed to investigate the conjecture.
3. Analysis—the statistical analysis of the data from the experiment.
4. Conclusion—what has been learned about the original conjecture from the experiment. Often the experiment will lead to a revised conjecture, a new experiment, and so forth.

The statistical methods introduced in this chapter and Chapter 14 are essential to good experimentation. All experiments are designed experiments; unfortunately, some of them are poorly designed, and as a result, valuable resources are used ineffectively. Statistically designed experiments permit efficiency and economy in the experimental process, and the use of statistical methods in examining the data results in scientific objectivity when drawing conclusions.

13-2 Completely Randomized Single-Factor Experiment

13-2.1 EXAMPLE: TENSILE STRENGTH

A manufacturer of paper used for making grocery bags is interested in improving the product's tensile strength. Product engineering believes that tensile strength is a function of the hardwood concentration in the pulp and that the range of hardwood concentrations of practical interest is between 5 and 20%. A team of engineers responsible for the study decides to investigate four levels of hardwood concentration: 5%, 10%, 15%, and 20%. They decide to make up six test specimens at each concentration level by using a pilot plant. All 24 specimens are tested on a laboratory tensile tester in random order. The data from this experiment are shown in Table 13-1.

This is an example of a completely randomized single-factor experiment with four levels of the factor. The levels of the factor are sometimes called treatments, and each treatment has six observations or replicates. The role of randomization in this experiment is extremely important. By randomizing the order of the 24 runs, the effect of any nuisance variable that may influence the observed tensile strength is approximately balanced out. For example, suppose that there is a warm-up effect on the tensile testing machine; that is, the longer the machine is on, the greater the observed tensile strength. If all 24 runs are made in order of increasing hardwood concentration (that is, all six 5% concentration specimens are tested first, followed by all six 10% concentration specimens, etc.), any observed differences in tensile strength could also be due to the warm-up effect. The role of randomization to identify causality was discussed in Section 10-1.

It is important to graphically analyze the data from a designed experiment. Figure 13-1(a) presents box plots of tensile strength at the four hardwood concentration levels. This figure indicates that changing the hardwood concentration has an effect on tensile strength; specifically, higher hardwood concentrations produce higher observed tensile strength. Furthermore, the distribution of tensile strength at a particular hardwood level is reasonably symmetric, and the variability in tensile strength does not change dramatically as the hardwood concentration changes.

TABLE • 13-1 Tensile Strength of Paper (psi)

FIGURE 13-1 (a) Box plots of hardwood concentration data. (b) Display of the model in Equation 13-1 for the completely randomized single-factor experiment.

Graphical interpretation of the data is always useful. Box plots show the variability of the observations within a treatment (factor level) and the variability between treatments. We now discuss how the data from a single-factor randomized experiment can be analyzed statistically.

13-2.2 ANALYSIS OF VARIANCE

Suppose that we have a different levels of a single factor that we wish to compare. Sometimes, each factor level is called a treatment, a very general term that can be traced to the early applications of experimental design methodology in the agricultural sciences. The response for each of the a treatments is a random variable. The observed data would appear as shown in Table 13-2. An entry in Table 13-2, say y_ij, represents the jth observation taken under treatment i. We initially consider the case that has an equal number of observations, n, on each treatment.

We may describe the observations in Table 13-2 by the linear statistical model

where y_ij is a random variable denoting the (ij)th observation, μ is a parameter common to all treatments called the overall mean, τ_i is a parameter associated with the ith treatment called the ith treatment effect, and _ij is a random error component. Notice that the model could have been written as

where μ_i = μ + τ_i is the mean of the ith treatment. In this form of the model, we see that each treatment defines a population that has mean μ_i consisting of the overall mean μ plus an effect τ_i that is due to that particular treatment. We assume that the errors _ij are normally and independently distributed with mean zero and variance σ². Therefore, each treatment can be thought of as a normal population with mean μ_i and variance σ². See Fig. 13-1(b).

TABLE • 13-2 Typical Data for a Single-Factor Experiment

Equation 13-1 is the underlying model for a single-factor experiment. Furthermore, because we require that the observations are taken in random order and that the environment (often called the experimental units) in which the treatments are used is as uniform as possible, this experimental design is called a completely randomized design (CRD).

The a factor levels in the experiment could have been chosen in two different ways. First, the experimenter could have specifically chosen the a treatments. In this situation, we wish to test hypotheses about the treatment means, and conclusions cannot be extended to similar treatments that were not considered. In addition, we may wish to estimate the treatment effects. This is called the fixed-effects model. Alternatively, the a treatments could be a random sample from a larger population of treatments. In this situation, we would like to be able to extend the conclusions (which are based on the sample of treatments) to all treatments in the population whether or not they were explicitly considered in the experiment. Here the treatment effects τ_i are random variables, and knowledge about the particular ones investigated is relatively unimportant. Instead, we test hypotheses about the variability of the τ_i and try to estimate this variability. This is called the random-effects, or components of variance model.

In this section, we develop the analysis of variance for the fixed-effects model. The analysis of variance is not new to us; it was used previously in the presentation of regression analysis. However, in this section, we show how it can be used to test for equality of treatment effects. In the fixed-effects model, the treatment effects τ_i are usually defined as deviations from the overall mean μ, so that

Let y_i. represent the total of the observations under the ith treatment and _i. represent the average of the observations under the ith treatment. Similarly, let y.. represent the grand total of all observations and .. represent the grand mean of all observations. Expressed mathematically,

where N = an is the total number of observations. Thus, the “dot” subscript notation implies summation over the subscript that it replaces.

We are interested in testing the equality of the a treatment means μ₁, μ₂,..., μ_a. Using Equation 13-2, we find that this is equivalent to testing the hypotheses

Thus, if the null hypothesis is true, each observation consists of the overall mean μ plus a realization of the random error component _ij. This is equivalent to saying that all N observations are taken from a normal distribution with mean μ and variance σ². Therefore, if the null hypothesis is true, changing the levels of the factor has no effect on the mean response.

The ANOVA partitions the total variability in the sample data into two component parts. Then, the test of the hypothesis in Equation 13-4 is based on a comparison of two independent estimates of the population variance. The total variability in the data is described by the total sum of squares

The partition of the total sum of squares is given in the following definition.

The identity in Equation 13-5 shows that the total variability in the data, measured by the total corrected sum of squares SS_T, can be partitioned into a sum of squares of differences between treatment means and the grand mean called the treatment sum of squares, and denoted SS_Treatments and a sum of squares of differences of observations within a treatment from the treatment mean called the error sum of squares, and denoted SS_E. Differences between observed treatment means and the grand mean measure the differences between treatments, and differences of observations within a treatment from the treatment mean can be due only to random error.

We can gain considerable insight into how the analysis of variance works by examining the expected values of SS_Treatments and SS_E. This will lead us to an appropriate statistic for testing the hypothesis of no differences among treatment means (or all τ_i = 0).

There is also a partition of the number of degrees of freedom that corresponds to the sum of squares identity in Equation 13-5. That is, there are an = N observations; thus, SS_T has an − 1 degrees of freedom. There are a levels of the factor, so SS_Treaments has a − 1 degrees of freedom. Finally, within any treatment, there are n replicates providing n − 1 degrees of freedom with which to estimate the experimental error. Because there are a treatments, we have a(n − 1) degrees of freedom for error. Therefore, the degrees of freedom partition is

The ratio

is called the mean square for treatments. Now if the null hypothesis H₀: τ₁ = τ₂ = ··· = τ_a = 0 is true, MS_Treatments is an unbiased estimator of σ² because . However, if H₁ is true, MS_Treatments estimates σ² plus a positive term that incorporates variation due to the systematic difference in treatment means.

Note that the mean square for error

is an unbiased estimator of σ² regardless of whether or not H₀ is true. We can also show that MS_Treatments and MS_E are independent. Consequently, we can show that if the null hypothesis H₀ is true, the ratio

has an F-distribution with a − 1 and a(n − 1) degrees of freedom. Furthermore, from the expected mean squares, we know that MS_E is an unbiased estimator of σ². Also, under the null hypothesis, MS_Treatments is an unbiased estimator of σ². However, if the null hypothesis is false, the expected value of MS_Treatments is greater than σ². Therefore, under the alternative hypothesis, the expected value of the numerator of the test statistic (Equation 13-7) is greater than the expected value of the denominator. Consequently, we should reject H₀ if the statistic is large. This implies an upper-tailed, one-tailed critical region. Therefore, we would reject H₀ if f₀ > f_{α,a−1,a(n−1)} where f₀ is the computed value of F₀ from Equation 13-7.

Efficient computational formulas for the sums of squares may be obtained by expanding and simplifying the definitions of SS_Treatments and SS_T. This yields the following results.

The computations for this test procedure are usually summarized in tabular form as shown in Table 13-3. This is called an analysis of variance (or ANOVA) table.

TABLE • 13-3 Analysis of Variance for a Single-Factor Experiment, Fixed-Effects Model

Computer Output

Many software packages have the capability to analyze data from designed experiments using the analysis of variance. See Table 13-5 for computer output for the paper tensile strength experiment in Example 13-1. The results agree closely with the manual calculations reported previously in Table 13-4.

The computer output also presents 95% confidece intervals (CIs) on each individual treatment mean. The mean of the ith treatment is defined as

A point estimator of μ_i is _i = _i.. Now if we assume that the errors are normally distributed, each treatment average is normally distributed with mean μ_i and variance σ²/n. Thus, if σ² were known, we could use the normal distribution to construct a CI. Using MS_E as an estimator of σ² (the square root of MS_E is the “Pooled StDev” referred to in the computer output), we would base the CI on the t distribution, because

has a t distribution with a(n − 1) degrees of freedom. This leads to the following definition of the confidence interval.

Equation 13-11 is used to calculate the 95% CIs shown graphically in the computer output of Table 13-5. For example, at 20% hardwood, the point estimate of the mean is ₄. = 21.167, MS_E = 6.51, and t_0.025,20 = 2.086, so the 95% CI is

or

It can also be interesting to find confidence intervals on the difference in two treatment means, say, μ_i − μ_j. The point estimator of μ_i − μ_j is _i. − _j., and the variance of this estimator is

TABLE • 13-5 Computer Output for the Completely Randomized Design in Example 13-1

Now if we use MS_E to estimate σ²,

has a t distribution with a(n − 1) degrees of freedom. Therefore, a CI on μ_i − μ_j may be based on the t distribution.

A 95% CI on the difference in means μ₃ − μ₂ is computed from Equation 13-12 as follows:

or

Because the CI includes zero, we would conclude that there is no difference in mean tensile strength at these two particular hardwood levels.

The bottom portion of the computer output in Table 13-5 provides additional information concerning which specific means are different. We discuss this in more detail in Section 13-2.3.

Unbalanced Experiment

In some single-factor experiments, the number of observations taken under each treatment may be different. We then say that the design is unbalanced. In this situation, slight modifications must be made in the sums of squares formulas. Let n_i observations be taken under treatment i(i = 1, 2,..., a), and let the total number of observations N = . The computational formulas for SS_T and SS_Treatments are as shown in the following definition.

Choosing a balanced design has two important advantages. First, the ANOVA is relatively insensitive to small departures from the assumption of equality of variances if the sample sizes are equal. This is not the case for unequal sample sizes. Second, the power of the test is maximized if the samples are of equal size.

13-2.3 MULTIPLE COMPARISONS FOLLOWING THE ANOVA

When the null hypothesis H₀: τ₁ = τ₂ = ··· = τ_a = 0 is rejected in the ANOVA, we know that some of the treatment or factor-level means are different. However, the ANOVA does not identify which means are different. Methods for investigating this issue are called multiple comparisons methods. Many of these procedures are available. Here we describe a very simple one, Fisher's least significant difference (LSD) method and a graphical method. Montgomery (2012) presents these and other methods and provides a comparative discussion.

The Fisher LSD method compares all pairs of means with the null hypotheses H₀: μ_i = μ_j (for all i ≠ j) using the t-statistic

Assuming a two-sided alternative hypothesis, the pair of means μ_i and μ_j would be declared significantly different if

where LSD, the least significant difference, is

If the sample sizes are different in each treatment, the LSD is defined as

The computer output in Table 13-5 shows the Fisher LSD method under the heading “Fisher's pairwise comparisons.” The critical value reported is actually the value of t_0.025,20 = 2.086. The computer output presents Fisher's LSD method by computing confidence intervals on all pairs of treatment means using Equation 13-12. The lower and upper 95% confidence limits are at the bottom of the table. Notice that the only pair of means for which the confidence interval includes zero is for μ₁₀ and μ₁₅. This implies that μ₁₀ and μ₁₅ are not significantly different, the same result found in Example 13-2.

Table 13-5 also provides a “family error rate,” equal to 0.192 in this example. When all possible pairs of means are tested, the probability of at least one type I error can be much higher than for a single test. We can interpret the family error rate as follows. The probability is 1 − 0.192 = 0.808 that there are no type I errors in the six comparisons. The family error rate in Table 13-5 is based on the distribution of the range of the sample means. See Montgomery (2012) for details. Alternatively, computer software often allows specification of a family error rate and then calculates an individual error rate for each comparison.

Graphical Comparison of Means

It is easy to compare treatment means graphically following the analysis of variance. Suppose that the factor has a levels and that ₁., ₂., ..., _a. are the observed averages for these factor levels. Each treatment average has standard deviation σ/, where σ is the standard deviation of an individual observation. If all treatment means are equal, the observed means _i. would behave as if they were a set of observations drawn at random from a normal distribution with mean μ and standard deviation σ/.

Visualize this normal distribution capable of being slid along an axis below which the treatment means ₁., ₂., ..., _a. are plotted. If all treatment means are equal, there should be some position for this distribution that makes it obvious that the _i. values were drawn from the same distribution. If this is not the case, the _i. values that do not appear to have been drawn from this distribution are associated with treatments that produce different mean responses.

The only flaw in this logic is that σ is unknown. However, we can use from the analysis of variance to estimate σ. This implies that a t distribution should be used instead of a normal distribution in making the plot, but because the t looks so much like the normal, sketching a normal curve that is approximately 6 units wide will usually work very well.

Figure 13-3 shows this arrangement for the hardwood concentration experiment in Example 13-1. The standard deviation of this normal distribution is

If we visualize sliding this distribution along the horizontal axis, we note that there is no location for the distribution that would suggest that all four observations (the plotted means) are typical, randomly selected values from that distribution. This, of course, should be expected because the analysis of variance has indicated that the means differ, and the display in Fig. 13-3 is just a graphical representation of the analysis of variance results. The figure does indicate that treatment 4 (20% hardwood) produces paper with higher mean tensile strength than do the other treatments, and treatment 1 (5% hardwood) results in lower mean tensile strength than do the other treatments. The means of treatments 2 and 3 (10% and 15% hardwood, respectively) do not differ.

This simple procedure is a rough but very effective multiple comparison technique. It works well in many situations.

FIGURE 13-3 Tensile strength averages from the hardwood concentration experiment in relation to a normal distribution with standard deviation = 1.04.

13-2.4 RESIDUAL ANALYSIS AND MODEL CHECKING

The analysis of variance assumes that the observations are normally and independently distributed with the same variance for each treatment or factor level. These assumptions should be checked by examining the residuals. A residual is the difference between an observation y_ij and its estimated (or fitted) value from the statistical model being studied, denoted as _ij. For the completely randomized design _ij = _i. and each residual is e_ij = y_ij − _i. This is the difference between an observation and the corresponding observed treatment mean. The residuals for the paper tensile strength experiment are shown in Table 13-6. Using _i. to calculate each residual essentially removes the effect of hardwood concentration from the data; consequently, the residuals contain information about unexplained variability.

The normality assumption can be checked by constructing a normal probability plot of the residuals. To check the assumption of equal variances at each factor level, plot the residuals against the factor levels and compare the spread in the residuals. It is also useful to plot the residuals against _i. (sometimes called the fitted value); the variability in the residuals should not depend in any way on the value of _i. Most statistical software packages can construct these plots on request. A pattern that appears in these plots usually suggests the need for a transformation, that is, analyzing the data in a different metric. For example, if the variability in the residuals increases with _i., a transformation such as log y or should be considered. In some problems, the dependency of residual scatter on the observed mean _i. is very important information. It may be desirable to select the factor level that results in maximum response; however, this level may also cause more variation in response from run to run.

The independence assumption can be checked by plotting the residuals against the time or run order in which the experiment was performed. A pattern in this plot, such as sequences of positive and negative residuals, may indicate that the observations are not independent. This suggests that time or run order is important or that variables that change over time are important and have not been included in the experimental design.

A normal probability plot of the residuals from the paper tensile strength experiment is shown in Fig. 13-4. Figures 13-5 and 13-6 present the residuals plotted against the factor levels and the fitted value _i., respectively. These plots do not reveal any model inadequacy or unusual problem with the assumptions.

TABLE • 13-6 Residuals for the Tensile Strength Experiment

FIGURE 13-4 Normal probability plot of residuals from the hardwood concentration experiment.

FIGURE 13-5 Plot of residuals versus factor levels (hardwood concentration).

FIGURE 13-6 Plot of residuals versus _i.

13-2.5 DETERMINING SAMPLE SIZE

In any experimental design problem, the choice of the sample size or number of replicates to use is important. Operating characteristic (OC) curves can be used to provide guidance here. Recall that an OC curve is a plot of the probability of a type II error (β) for various sample sizes against values of the parameters under test. The OC curves can be used to determine how many replicates are required to achieve adequate sensitivity.

The power of the ANOVA test is

To evaluate this probability statement, we need to know the distribution of the test statistic F₀ if the null hypothesis is false. Because ANOVA compares several means, the null hypothesis can be false in different ways. For example, possibly τ₁ > 0, τ₂ = 0, τ₃ < 0, and so forth. It can be shown that the power for ANOVA in Equation 13-17 depends on the τ_i's only through the function

Therefore, an alternative hypotheses for the τ_i's can be used to calculate Φ² and this in turn can be used to calculate the power. Specifically, it can be shown that if H₀ is false, the statistic F₀ = MS_Treatments/MS_E has a noncentral F-distribution with a − 1 and n(a − 1) degrees of freedom and a noncentrality parameter that depends on Φ². Instead of tables for the noncentral F-distribution, OC curves are used to evaluate β defined in Equation 13-17. These curves plot β against Φ.

FIGURE 13-7 Operating characteristic curves for the fixed-effects model analysis of variance. Top curves for four treatments and bottom curves for five treatments.

OC curves are available for α = 0.05 and α = 0.01 and for several values of the number of degrees of freedom for numerator (denoted v₁) and denominator (denoted v₂). Figure 13-7 gives representative OC curves, one for a = 4 (v₁ = 3) and one for a = 5 (v₁ = 4) treatments. Notice that for each value of a, there are curves for α = 0.05 and α = 0.01.

In using the curves, we must define the difference in means that we wish to detect in terms of . Also, the error variance σ² is usually unknown. In such cases, we must choose ratios of that we wish to detect. Alternatively, if an estimate of σ² is available, one may replace σ² with this estimate. For example, if we were interested in the sensitivity of an experiment that has already been performed, we might use MS_E as the estimate of σ².

13-3 The Random-Effects Model

13-3.1 FIXED VERSUS RANDOM FACTORS

In many situations, the factor of interest has a large number of possible levels. The analyst is interested in drawing conclusions about the entire population of factor levels. If the experimenter randomly selects a of these levels from the population of factor levels, we say that the factor is a random factor. Because the levels of the factor actually used in the experiment are chosen randomly, the conclusions reached are valid for the entire population of factor levels. We assume that the population of factor levels is either of infinite size or is large enough to be considered infinite. Notice that this is a very different situation than the one we encountered in the fixed-effects case in which the conclusions apply only for the factor levels used in the experiment.

13-3.2 ANOVA AND VARIANCE COMPONENTS

The linear statistical model is

where the treatment effects τ_i and the errors ε_ij are independent random variables. Note that the model is identical in structure to the fixed-effects case, but the parameters have a different interpretation. If the variance of the treatment effects τ_i is , by independence the variance of the response is

The variances and σ² are called variance components, and the model, Equation 13-19, is called the components of variance model or the random-effects model. To test hypotheses in this model, we assume that the errors _ij are normally and independently distributed with mean zero and variance σ² and that the treatment effects τ_i are normally and independently distributed with mean zero and variance .^*

For the random-effects model, testing the hypothesis that the individual treatment effects are zero is meaningless. It is more appropriate to test hypotheses about . Specifically,

If = 0, all treatments are identical; but if > 0, there is variability between treatments.

The ANOVA decomposition of total variability is still valid; that is,

However, the expected values of the mean squares for treatments and error are somewhat different than in the fixed-effects case.

From examining the expected mean squares, it is clear that both MS_E and MS_Treatments estimate σ² when H₀: = 0 is true. Furthermore, MS_E and MS_Treatments are independent. Consequently, the ratio

is an F random variable with a − 1 and a(n − 1) degrees of freedom when H₀ is true. The null hypothesis would be rejected at the a-level of significance if the computed value of the test statistic f₀ > f_{α,a−1,a(n−1)}.

The computational procedure and construction of the ANOVA table for the random-effects model are identical to the fixed-effects case. The conclusions, however, are quite different because they apply to the entire population of treatments.

Usually, we also want to estimate the variance components (σ² and ) in the model. The procedure that we use to estimate σ² and is called the analysis of variance method because it uses the information in the analysis of variance table. It does not require the normality assumption on the observations. The procedure consists of equating the expected mean squares to their observed values in the ANOVA table and solving for the variance components. When equating observed and expected mean squares in the one-way classification random-effects model, we obtain

Therefore, the estimators of the variance components are

Sometimes the analysis of variance method produces a negative estimate of a variance component. Because variance components are by definition non-negative, a negative estimate of a variance component is disturbing. One course of action is to accept the estimate and use it as evidence that the true value of the variance component is zero, assuming that sampling variation led to the negative estimate. Although this approach has intuitive appeal, it will disturb the statistical properties of other estimates. Another alternative is to reestimate the negative variance component with a method that always yields non-negative estimates. Still another possibility is to consider the negative estimate as evidence that the assumed linear model is incorrect, requiring that a study of the model and its assumptions be made to find a more appropriate model.

This example illustrates an important application of the analysis of variance—the isolation of different sources of variability in a manufacturing process. Problems of excessive variability in critical functional parameters or properties frequently arise in quality-improvement programs. For example, in the previous fabric strength example, the process mean is estimated by = 95.45 psi, and the process standard deviation is estimated by _y = = = 2.98 psi. If strength is approximately normally distributed, the distribution of strength in the outgoing product would look like the normal distribution shown in Fig. 13-8(a). If the lower specification limit (LSL) on strength is at 90 psi, a substantial proportion of the process output is fallout—that is, scrap or defective material that must be sold as second quality, and so on. This fallout is directly related to the excess variability resulting from differences between looms. Variability in loom performance could be caused by faulty setup, poor maintenance, inadequate supervision, poorly trained operators, and so forth. The engineer or manager responsible for quality improvement must identify and remove these sources of variability from the process. If this can be done, strength variability will be greatly reduced, perhaps as low as _Y = = = 1.38 psi, as shown in Fig. 13-8(b). In this improved process, reducing the variability in strength has greatly reduced the fallout, resulting in lower cost, higher quality, a more satisfied customer, and an enhanced competitive position for the company.

FIGURE 13-8 The distribution of fabric strength. (a) Current process, (b) Improved process.

13-4 Randomized Complete Block Design

13-4.1 DESIGN AND STATISTICAL ANALYSIS

In many experimental design problems, it is necessary to design the experiment so that the variability arising from a nuisance factor can be controlled. For example, consider the situation of Example 10-10 in which two different methods were used to predict the shear strength of steel plate girders. Because each girder has different strength (potentially), and this variability in strength was not of direct interest, we designed the experiment by using the two test methods on each girder and then comparing the average difference in strength readings on each girder to zero using the paired t-test. The paired t-test is a procedure for comparing two treatment means when all experimental runs cannot be made under homogeneous conditions. Alternatively, we can view the paired t-test as a method for reducing the background noise in the experiment by blocking out a nuisance factor effect. The block is the nuisance factor, and in this case, the nuisance factor is the actual experimental unit—the steel girder specimens used in the experiment.

The randomized block design is an extension of the paired t-test to situations where the factor of interest has more than two levels; that is, more than two treatments must be compared. For example, suppose that three methods could be used to evaluate the strength readings on steel plate girders. We may think of these as three treatments, say t₁, t₂, and t₃. If we use four girders as the experimental units, a randomized complete block design (RCBD) would appear as shown in Fig. 13-9. The design is called a RCBD because each block is large enough to hold all the treatments and because the actual assignment of each of the three treatments within each block is done randomly. Once the experiment has been conducted, the data are recorded in a table, such as is shown in Table 13-9. The observations in this table, say, y_ij, represent the response obtained when method i is used on girder j.

The general procedure for a RCBD consists of selecting b blocks and running a complete replicate of the experiment in each block. The data that result from running a RCBD for investigating a single factor with a levels and b blocks are shown in Table 13-10. There are a observations (one per factor level) in each block, and the order in which these observations are run is randomly assigned within the block.

We now describe the statistical analysis for the RCBD. Suppose that a single factor with a levels is of interest and that the experiment is run in b blocks. The observations may be represented by the linear statistical model

where μ is an overall mean, τ_i is the effect of the ith treatment, β_j is the effect of the jth block, and _ij is the random error term, which is assumed to be normally and independently distributed with mean zero and variance σ². Furthermore, the treatment and block effects are defined as deviations from the overall mean, so and . This was the same type of definition used for completely randomized experiments in Section 13-2. We also assume that treatments and blocks do not interact. That is, the effect of treatment i is the same regardless of which block (or blocks) in which it is tested. We are interested in testing the equality of the treatment effects. That is,

FIGURE 13-9 A randomized complete block design.

TABLE • 13-9 A Randomized Complete Block Design

TABLE • 13-10 A Randomized Complete Block Design with a Treatments and b Blocks

The analysis of variance can be extended to the RCBD. The procedure uses a sum of squares identity that partitions the total sum of squares into three components.

Furthermore, the degrees of freedom corresponding to these sums of squares are

For the randomized block design, the relevant mean squares are

The expected values of these mean squares can be shown to be as follows:

Therefore, if the null hypothesis H₀ is true so that all treatment effects τ_i = 0, MS_Treatments is an unbiased estimator of σ², and if H₀ is false, MS_Treatments overestimates σ². The mean square for error is always an unbiased estimate of σ². To test the null hypothesis that the treatment effects are all zero, we use the ratio

which has an F-distribution with a − 1 and (a − 1)(b − 1) degrees of freedom if the null hypothesis is true. We would reject the null hypothesis at the a-level of significance if the computed value of the test statistic in Equation 13-28 is f₀ > f_{α,a−1,(a−1)(b−1)}.

In practice, we compute SS_T, SS_Treatments and SS_Blocks and then obtain the error sum of squares SS_E by subtraction. The appropriate computing formulas are as follows.

The computations are usually arranged in an ANOVA table, such as is shown in Table 13-11. Generally, computer software is used to perform the analysis of variance for a RCBD.

TABLE • 13-11 ANOVA for a Randomized Complete Block Design

Important Terms and Concepts

Analysis of variance (ANOVA)

Blocking

Completely randomized design (CRD)

Error mean square

Fisher's least significant difference (LSD) method

Fixed-effects model

Graphical comparison of means

Least significant difference

Levels of a factor

Linear statistical model

Mean squares

Multiple comparisons methods

Nuisance factor

Operating characteristic curve

Random factor-effects model

Randomization

Randomized complete block design (RCBD)

Replicates

Residual analysis and model checking

Sample size and replication in an experiment

Treatment

Treatment sum of squares

Variance component