So, we have estimated that the true population mean is about 65.2; we know the population mean isn't exactly 65.19704—but by just how much might our estimate be off?

To answer this question, let's take repeated samples from the population again. This time, we're going to take samples of size 40 from the population 10,000 times and plot a frequency distribution of the means.

  > means.of.our.samples <- numeric(10000)
  > for(i in 1:10000){
  +   a.sample <- sample(all.us.women, 40)
  +   means.of.our.samples[i] <- mean(a.sample)
  + }

Figure 5.3: The sampling distribution of sample means

This frequency distribution is called a sampling distribution. In particular, since we used sample means as the value of interest, this is called the sampling distribution of the sample means (whew!!). You can create a sampling distribution of any statistic (median, variance, and so on), but when we refer to sampling distributions throughout this chapter, we will be specifically referring to the sampling distribution of sample means.

Check it out: the sampling distribution looks like a normal distribution—and that's because it is a normal distribution.

For a large enough sample size, the sampling distribution of any population will be approximately normal with a mean equal to the population mean, µ, and a standard deviation of:

where N is the sample size and σ is the population standard deviation. This is called the central limit theorem, and it is among the most important theorems in all of statistics.

Look back at the equation. Convince yourself that sample size is proportional to the narrowness of the sampling distribution by noting that the sample size is in the denominator.

The standard deviation of the sampling distribution tells us how variable a sample of a certain size's mean can be from sample to sample. It also tells us how much we expect certain samples' means to vary from the true population mean. The standard deviation of the sampling distribution is called the standard error, and we can use it to quantify our uncertainty about our estimate of the population mean.

If the standard error is small, an estimate from one sample is likely to be closer to the true mean (because the sampling distribution is narrow). If our standard error is big, the mean of any one particular sample is likely to be farther away from the true mean, on average.

Okay, so I've convinced you that the standard error is a great statistic to use—but how do we get it? Up until now, I've said that you can calculate it by either:

However, in practice, this isn't good enough: we don't want to take repeated samples from the population for the same reason that we can't measure the heights of all US women (because it would take too long and cost too much). And, in the case of using the population standard deviation to get the standard error—well, we don't know the population standard deviation—if we did, we would have already had to calculate the population mean, and we wouldn't have to be estimating it with sampling!

Ideally, we want to find the standard error using only one sample. Well, it turns out that for sufficiently large samples, using the sample standard deviation, s, in the standard error formula (instead of the population standard deviation, σ) is a good enough approximation. Similarly, the mean of the sampling distribution is equal to the population mean, but we can use our sample's mean as an estimate of that.

For a concrete example, let's use our sample of 40, our.new.sample:

  > mean(our.new.sample)
  [1] 65.19704
  > sd(our.new.sample)
  [1] 3.588447
  > sd(our.new.sample) / sqrt(length(our.new.sample))
  [1] 0.5673833

Our sample's mean and standard deviation is 65.2 and 3.59 respectively. The standard error of the mean is 0.567.

This means that the sampling distribution of the sample means would look something like this:

Figure 5.4: Estimated sampling distribution of sample means based on one sample