Chapter 11

Split-Plot Designs to Accommodate Hard-to-Change Factors

Oftentimes, complete randomization of all test parameters is extremely inefficient, or even totally impractical.

Alex Sewell

53rd Test Management Group, 28th Test and Evaluation Squadron, Eglin Air Force Base

Researchers often set up experiments with the best intentions of running them in random order, but they find that a given factor, such as temperature, cannot be easily changed, or, if so, only at too much expense in terms of time or materials. In this case, the best test plan might very well be the split-plot design. A split plot accommodates both hard-to-change (HTC) factors, e.g., the cavities in a molding process, and those that are easy to change (ETC); in this case, the pressure applied to the part being formed.

How Split Plots Naturally Emerged for Agricultural Field Tests

Split-plot designs originated in the field of agriculture where experimenters applied one treatment to a large area of land, called a whole plot, and other treatments to smaller areas of land within the whole plot, called subplots. For example, when D. R. Cox wrote about the “Split Unit Principle” in his classic Planning of Experiments (John Wiley & Sons, 1958), he detailed an experiment on six varieties of sugar beets (number 1 through 6) that are sown either early (E) or late (L). Figure 11.1 shows the alternatives of a completely randomized design versus one that is split into two subplots.

The split-plot layout made it far easier to sow the seeds because of the continuity in grouping. This is sweet for the sugar beet farmer (pun intended). However, as with anything that seems too easy, there is a catch to the more convenient test plan at the bottom of Figure 11.1: It confounds the time of sowing with the blocks of land, leaving no statistical power for assessing this factor (early versus late). The only way around this is to replicate the whole plot, that is, repeat this entire experiment on a number of fields, as is provided in another agricultural split plot illustrated by Figure 11.2. It lays out a classic field study on three varieties (V) of oats treated with four levels of nitrogen (N) fertilizer described by Sir Fisher’s protégé (DOE pioneer Frank Yates. 1935. Complex experiments. Journal of the Royal Statistical Society Supplement 2: 181–247). The varieties were conveniently grouped into randomized whole plots within six blocks of land (we only show three blocks). It was then fairly easy to apply the different fertilizers chosen at random.

Applying a Split Plot to Save Time Making Paper Helicopters

Building paper helicopters as an in-class design of experiments (DOE) project is well established for a hands-on learning experience (see, for example, George’s Column: Teaching Engineers Experimental Design with a Paper Helicopter, by George Box. 1992. Quality Engineering 4 (3): 453–459). Figure 11.3 pictures an example—the top flyer from the trials detailed in this case study.

Inspired by news of a supreme paper (Conqueror CX22) made into an airplane that broke the Guinness World Record™ for greatest distance flown (detailed by Sean Hutchinson in The Perfect Paper Airplane, Mental Floss, January 14, 2014), engineers at Stat-Ease (Mark and colleagues) designed a split-plot experiment on paper helicopters. They put CX22 to the test against a standard copy paper. As laid out below, five other factors were added to round out the flight testing via a half-fraction, high-resolution (Res VI) two-level design with 32 runs (= 26-1):

1. Paper: 24# Navigator Premium (standard) versus 26.6# Conqueror CX22 (supreme)
2. Wing (technically a rotor in aviation terms) Length: Short versus Long
3. Body Length: Short versus Long
4. Body Width: Narrow versus Wide
5. Clip: Off versus On
6. Drop: Bottom versus Top

Being attributes of construction, the first four factors (identified with lower-case letters) are hard to change, i.e., HTC. The other factors come into play in the flights (operation). They are easy to change (ETC).

To develop the longest flying, most-accurate helicopter, the experimenters measured these two responses (Y):

1. Average time in seconds for three drops from ceiling height
2. Average deviation in centimeters from target

The averaging dampened out drop-to-drop variation, which, due to human factors in the release, air drafts, and so forth, can be considerable.

The experimenters enlisted a worker at Stat-Ease with the patience and skills needed for constructing the paper helicopters. Grouping the HTCs number of flying machines into whole plots, as shown in Table 11.1, reduced the manufacturing time by one half.

After the 16 helicopters were built (numbered by group with a marker), the engineers put each one (in run order) to the test with combinations of the two ETC factors as noted in the table.

Trade-Off of Power for Convenience When Restricting Randomization

As shown in Table 11.2, of the two responses, the distance from target (Y2) produced the lowest signal-to-noise ratio. It was based on a 5-cm minimal deviation of importance relative to a 2-cm standard deviation measured from prior repeatability studies.

Assuming a whole-plot to split-plot variance ratio of 2 (see boxed text below: Heads-Up about Statistical Analysis of Data from Split Plots, for background), a 32-run, two-level factorial design generates the power shown in Table 11.3 at 2.5 signal-to-noise for targeting.

What’s really important to see here is that, by grouping the HTC factors, the experimenters lost power versus a completely randomized design. (It mattered little in this case, but it must be noted that the other factors (the ETCs) gained a bit more power, going from 97.5% to 99.8%.) However, the convenience and cost savings (the CX22 stock being extremely expensive) of only building half the paper helicopters—16 out of the 32 required in the fully randomized design of experiments—outweighed the loss in power, which in any case remained above the generally acceptable level of 80%. Thus, the split-plot test plan laid out in Table 11.1 got the thumbs up from the flight engineers.

Much more could be said about split plots and the results of this experiment in particular (for what it’s worth, CX22 did indeed rule supreme). The main point here is how power was assessed to “right size” the test plan while accommodating the need to reduce the builds.

One More Split Plot Example: A Heavy-Duty Industrial One

George Box in a Quality Quandaries column on Split Plot Experiments (Quality Engineering, 8(3), 515–520, 1996) detailed a clever experiment that discovered a highly corrosion-resistant coating for steel bars. Four different coatings were tested (easy to do) at three different furnace temperatures (hard to change), each of which was run twice to provide power. See Box’s design (a split plot) in Table 11.4 (results for corrosion resistance shown in parentheses). Note the bars being placed at random by position.

The HTC factor (temperature) creates so much noise in this process that in a randomized design it would overwhelm the effect of coating. The application of a split plot overcomes this variability by grouping the heats, in essence, filtering out the temperature differences. The effects graph in Figure 11.4 tells the story.

The combination of high temperature with coating C4, located at the back corner, towers above all others. This finding, the result of the two-factor interaction aB between temperature and coating, achieved significance at the p < 0.05 threshold. The main effect of coating (B) also came out significant. If this experiment had been run completely randomized, p-values for the coating effect and the coating-temperature interaction would have come out to approximately 0.4 and 0.85 p-values, respectively, i.e., nowhere near to being statistically significant. Box concludes by suggesting the metallurgists try even higher heats with the C4 coating while simultaneously working at better controlling furnace temperature. Furthermore, he urges the experimenters to work at understanding better the physiochemical mechanisms causing corrosion of the steel. This really was the genius of George Box—his matchmaking of empirical modeling tools with subject matter expertise.

Our brief discussion of split plots ends on this high note.