Introduction

In my 26 years as a biostatistician at the Rutgers Robert Wood Johnson Medical School in New Jersey, the question I was most frequently asked was, “I’m doing an experiment; how many subjects do I need?” Just about any study, especially one for which you are asking for funding or applying for a grant, will require detailed power and sample size calculations.

As discussed in Chapter 1, the power of a study is the probability that the study will result in a statistically significant finding if the drug or treatment that you are studying is different (hopefully better) than either a placebo or an alternate drug or treatment. For many studies conducted at research labs or universities, powers of 80% or 90% are typical. Very large-scale studies may strive for a power of 95%. Small, exploratory studies might be satisfied with powers closer to 70%. The bottom line is that it is unethical and wasteful to conduct a study with low power. You will have a low probability of demonstrating the superiority of your drug or intervention, and this negative result may dissuade others from investigating the same drug or intervention, when it may actually be beneficial. You are also possibly putting people at risk when there is a small probability that the study will achieve a positive result.

Depending on the type of study (comparing means or comparing proportions, for example), there is a set of questions that needs to be answered before you can determine the number of subjects you will need for a particular study. SAS Studio includes a Power and Sample Size task that performs this analysis for many of the popular study designs.

Computing Sample Size for a t Test

Let’s start with a simple study to compare blood pressure in a group of borderline hypertensive subjects. You want to see whether a low dose of a beta blocker will reduce blood pressure. Because these subjects are borderline hypertensive and the trial will be relatively short, you decide that it is ethical to use a placebo as your control.

What information do you need to decide how many subjects you need to recruit in order to have a power of 80%? Let’s open up the Power and Sample Size tab (Tasks and Utilities ⇒ Tasks ⇒ Power and Sample Size). It looks like this:

Use the t Tests selection to begin the sample size calculation for comparing two means. Double-clicking t Tests brings up the following:

The menu under Type of t test allows you to select from a one-sample test or a two-sample test (either paired or unpaired). Because your study design called for two unpaired groups, select the two-sample option. Next, you can choose to compute power (for a given sample size) or sample size (for a given power). In most studies, you want to know the sample size for a given power. You can request the total sample size or the sample size per group. It is the latter choice that is shown in Figure 15.2.

The next section of the Properties tab asks you to decide if the test is one- or two-sided. Most studies of this type are two-sided. You can also decide if you want to assume equal variances in the two groups. Figure 15.3 shows selections for a two-sided test with equal (pooled) variances.

The calculation for power requires you to either estimate the means of the two groups or the difference between the two means. The screen shot in Figure 15.4 shows the different ways that you can enter this information.

For this example, you are choosing to enter the group means you expect. It is pretty easy to guess the mean for the placebo group because you can take blood pressures on subjects similar to the ones that you will recruit for your study. Choosing one or more means for the treatment group is a bit trickier. Many researchers select values such as a 25% change in the treatment group or a value that they hope to obtain with the treatment, or that is clinically meaningful. With this selection, the menu system opens boxes for you to enter the expected mean for each group (See Figure 15.5).

By clicking the plus sign above the location where you enter your choices for means, you can calculate sample size for different choices of sample means. Figure 15.6 shows that you want to compute sample sizes for means of 130 versus 120 (a 10-point difference) and 130 versus 125 (a 5-point difference).

Your final two decisions are to estimate the standard deviation and the desired power. As with other choices in this task, you can enter several selections for each. In Figure 15.7, you see an estimate of 10 for the standard deviation (estimated from measurements from people similar to the ones that you will recruit for your study) and two powers: .8 and .9. Note that powers are entered as probabilities (values between 0 and 1) and not as percentages.

Before you run the task, click the PLOTS tab and make sure that the box Power by sample size plot is checked. You can let the task scale the power axis, or you can specify minimum and maximum powers by checking the two boxes under Range of values (Figure 15.8).

You are ready to run the task. Figure 15.9 shows the results in table form, and Figure 15.10 shows the results in graphical form.

Notice the large sample sizes necessary to detect a small difference of 5 points and high power (n = 86 per group). The lowest sample size per group (n per group = 17) is for the largest difference (130 versus 120) and the lowest power (80%).

The graph of sample size by power (see Figure 15.10) contains a line for each combination of means, standard deviations, and powers that you entered. The final decision of sample size is sometimes a compromise between how many subjects you can recruit (and pay for) and how large a difference you would like to be able to detect.

Calculating the Sample Size for a Test of Proportions

What information do you need to compute sample sizes for a test to compute the difference between two proportions? The decisions for this calculation are simpler than the decisions that you made for comparing means. The reason is that once you select proportions (or difference between proportions), the standard deviation can be computed.

The information that you need to perform this calculation is listed below:

• Is the test to be conducted as a one-sided or two-sided test? (Usually two-sided)
• What is your alpha level? (Usually ⍺ = .05)
• What is the proportion in the first group (usually a control group)? If unsure, lean toward .5 (maximum variance).
• How large a difference in proportions do you want to be able to detect? (Or the proportion in group two.)
• What power do you want? (You often enter several values such as .8, .85, and .9)

You are now ready to run the Test of Proportions task. Double-click this selection in the Power and Sample Size menu to bring up the following screen. (As before, the Properties screen is shown in pieces.)

For the selection Type of test, select Two independent proportions. Next, decide if you want to test for power or sample size. In this example, you want to know how many subjects you need to obtain a desired power. You have a choice of calculating total sample size or sample size per group. Here, you chose sample size per group.

The next part of the Properties screen asks you to select the statistical test that you plan to use in the analysis. For most studies, especially those with fairly large n’s, the Pearson chi-square test is a good choice. If you believe that you will have small expected values in the study, you might choose Fisher’s exact test. Finally, indicate if the test is to be conducted as a one-sided or two-sided test. In Figure 15.12, you see selections for a Pearson chi-square test conducted as a two-sided test.

Moving farther down the menu, you see a choice for alpha (the default value of .05 is already entered) and whether you want to enter group proportions or other ways of comparing proportions such as odds ratios or relative risk. The choices in Figure 15.13 are alpha = .05 and Group proportions. As with the previous calculation where you were comparing means, you can also enter several choices for proportions in the two groups. You decide to compute sample sizes for two different scenarios: one with proportions of .7 and .8, the other with proportions of .7 and .9.

The last entry in the PROPERTIES tab allows you to enter one or more values for power. In Figure 15.14, powers of .8, .85, and .9 were selected.

Before you run the task, click the PLOTS tab to request a plot of power by sample size.

It’s time to run the task. Click the Run icon to obtain the table and graph displayed in Figure 15.16 and Figure 15.17.

Many researchers are shocked when they see the large sample sizes needed to compare proportions. If you look at the N per Group for comparing proportions of .7 and .8 with a power of 90%, you see that you need 392 subjects per group. The smallest number of subjects per group (62) is for proportions of .7 versus .9 with 80% power.

You might find it more instructive to examine the sample size calculations in graphical form. As with the previous situation where you were comparing means, when you compare proportions, you may have to lower your expectations of detecting small differences and design the study with larger differences in the two proportions and, perhaps, slightly lower power.

Computing Sample Size for a One-Way ANOVA Design

The last example in this chapter describes how to compute sample size or power for a one-way analysis of variance (ANOVA).

In this example, you have made the following decisions:

• There are 3 means estimated to be 20, 25, and 30.
• You have two estimates for standard deviation: 8 and 10.
• You want to compute sample size for powers of 80% and 90%.
• You want to compute the n-per-group (as compared to power for a given sample size).

You would like a plot of Power (X axis) versus sample size, and the axes scaled to show powers from .7 to .9.

Start out by selecting Power and Sample Size for One-Way ANOVA. On the PROPERTIES tab, set the number of groups to 3, select Sample Size per Group, and then enter your hypothesized means, one or more estimates of standard deviation, and one or more values for power. (See Figure 15.18.)

The next step is to click the PLOTS tab. Select a plot of Power by sample size and enter a value for the minimum and maximum power (Figure 15.19).

You are ready to run the task. The first portion of the output is a table showing the required sample size for all combinations of estimated standard deviation and power. Because you entered two values for standard deviation and two powers, the table of sample size per group shows four different (2 times 2) values for sample size per group.

The plot request produced the graph in Figure 15.21.

Inspection of this graph, and the practical constraints of obtaining and possibly paying for subjects, will help you make a final decision on designing your study.

Conclusion

The SAS Studio statistics tasks include power and sample size calculations for several statistical tests, such as comparing means, comparing proportions, or running an analysis of variance. By seeing how to enter data for the three scenarios in this chapter, you should be able to perform calculations for the other designs in the Power and Sample size menu.

Chapter 15 Exercises

1. Compute the sample size for a two-sample t test where you estimate the mean of the two groups to be 200 and 210, respectively. You estimate the standard deviation to be 10, and you want to compute the number of subjects per group for powers of .80, .85, and .9.
2. The proportion of patients cured in under five days without any medication is .80. You want to compute sample size per group under two possible hypotheses. One with a five-day cure rate of .85, the other with a five-day cure rate of .90. Compute the sample sizes for powers of .8 and .9. Also produce a graph showing power on the X axis and sample size on the Y axis. Notice the huge difference in the required sample sizes for the comparison of .8 and .85 versus .8 and .9. In general, you need very large samples to test for small differences in proportions.
3. You are planning a one-way ANOVA design. You have two scenarios that you want to test. In one, the three hypothesized means are 50, 55, and 60. In the second, they are 50, 60, and 70. You also have two estimates of standard deviation: 15 and 20. Compute sample size per group for a power of .8. Include a plot of power versus sample size with the minimum power set at .7 and the maximum power set at .9.