Chapter 10. Sample-Size Analysis for Traditional Hypothesis Testing: Concepts and Issues

Ralph G. O'Brien^[]

^[] Ralph G. O'Brien is Professor, Quantitative Health Sciences, Cleveland Clinic, USA ([email protected]).

John Castelloe^[]

^[] John Castelloe is Senior Research Scientist, SAS Institute, USA ([email protected]).

Sample-size analysis continues to be transformed by ever-improving strategies, methods, and software. Using these tools intelligently depends on what the investigators understand about statistical science and what they know and conjecture about the particular research questions driving the study planning. This chapter covers only the most common type of sample-size analysis—power analysis, i.e., studying the chance that a given hypothesis test will be "statistically significant," p ≤ α. We focus on the core concepts and issues that the collaborating statistician must master and that key investigators must understand.

We begin by reviewing p values and discuss how to conduct sample-size analyses that focus on the classical Type I and Type II error rates, α and β. Then we go further to consider two other error rates, the crucial Type I error rate, α^*, which is the chance that the null hypothesis is true even though p ≤ α, and the crucial Type II error rate, β^*, defined as the chance that the null hypothesis is false in some particular way even though p > α. We argue that α^* and β^* are just as relevant (if not more so) than α and β. These issues are explored in depth through two examples stemming from a straightforward clinical trial.