Chapter 12
Deciphering Your Confidence Interval
IN THIS CHAPTER
Correctly interpreting the results of a confidence interval
Avoiding common interpretation mistakes
Knowing what confidence intervals don’t measure
Confidence intervals can be straightforward, black-and-white calculations (with practice), yet their real meaning can still remain a mystery. With the guidance in this chapter, I help you avoid making the mistakes that drive instructors (and, subsequently, you) crazy. You can become one of the enlightened who “get it” and thereby get more points on exams. Because in the end, it’s all about points, isn’t it?
Interpreting Confidence Intervals the Right Way
Because I’ve been a statistics professor for many years, I know how disappointed instructors can get when students don’t completely understand what the results of a confidence interval really mean. (Part of it is our own fault because we have a hard time explaining it clearly ourselves.) Instructors want to harp and wax philosophic on this issue to no end, and we make it a personal challenge to help you understand it. In other words, confidence intervals show up on every quiz or exam that instructors can possibly include them on. The practice problems in this chapter put you through every scenario I can think of to help you avoid misinterpreting a confidence interval. I also address some big picture ideas regarding confidence intervals and how to view them.
Here’s the scoop on what a confidence interval really means: Suppose that you just found a 95 percent confidence interval for the population mean. To interpret it correctly, you say, “We are 95 percent confident that the population mean is in this interval.” That does not mean, however, that there is a 95 percent chance that the population mean is in your interval. After the interval has been calculated, it’s either right or wrong, and you don’t put a probability on it. The 95 percent is how confident you are in the process that led you to the interval you got; in other words, your sampling process. It means that 95 percent of the time, you will get random samples that do represent the situation and result in a correct interval (that contains the population parameter). Five percent of the time you won’t, just by chance. Read this paragraph over a couple of times and really think hard about it before moving on to the problems. You are at the crux of the issue right here, and it may take some time to really grab hold of the idea. Take your time.
See the following for an example of interpreting a confidence interval.
Q. Suppose that survey results say that 69 percent of adult females from a sample taken in the United States registered to vote. The margin of error for the survey that produced these results was plus or minus 3 percent (based on 95 percent confidence). Interpret the results of this confidence interval.
A. Based on your sample, you’re 95 percent confident that the percentage of all females in the United States who were registered to vote was between 66 percent and 72 percent. (This is because 
and 
) The results also mean that if you repeat this survey with many different samples of the same size, 95 percent of the samples would result in intervals that contain the true percentage of females who registered to vote. Better hope that yours is one of those intervals!
1 Based on the results of the survey from the example problem, can you say with 95 percent confidence that your confidence interval contains the true percentage of all adult females registered to vote?
2 Based on the results of the survey from the example problem, can you say that 69 percent of all adult women in the United States registered to vote?
3 Based on the results of the survey from the example problem, can you say that 69 percent of all adult female respondents to this survey registered to vote?
4 Based on the results of the survey from the example problem, can you say that it’s probably true that 69 percent of all female adults in the United States registered to vote?
5 Based on the results of the survey from the example problem, can you say that the percentage of all adult females in the United States who registered to vote is between 66 and 72 percent?
6 Based on the results of the survey from the example problem, can you say with 95 percent confidence that the interval 69 percent plus or minus 3 percent contains the percentage of registered voters for adult females in the sample?
7 Based on the results of the survey from the example problem, can you say that the true proportion of adult female registered voters in the United States has a 95 percent chance of falling somewhere between 66 and 72 percent?
8 Based on the results of the survey from the example problem, can you say that if you repeat this same sampling process over and over with the same sample size, you would get an incorrect confidence interval only 5 percent of the time (that is, it would not contain the true proportion of registered voters for all adult females in the United States)?
9 Is a wide confidence interval a good thing?
10 Will confidence intervals with a high confidence always be wide?
11 Does a larger confidence level lower the chance for bias in the results?
12 Bob thinks that to cut his margin of error in half, he needs to sample twice as many people. Is Bob right?
Evaluating Confidence Interval Results: What the Formulas Don’t Tell You
When data comes from well-designed surveys and experiments, and is based on large random samples, you can feel good about the quality of the information. When the margin of error of any confidence interval is small, you assume that the confidence interval provides an accurate and credible estimate of the parameter. This isn’t always the case, however. Why not? Because not all data come from well-designed surveys and experiments and are based on large random samples.
When it comes to margin of error, less may be more. The formulas don’t realize it when the numbers plugged into them are based on biased data, so you have to spot those situations and disregard the seemingly precise results.
See the following for an example of when a reported margin of error is meaningless.
Q. Suppose that a survey on a popular Internet website receives responses from 50,000 people. The reported margin of error for this survey, according to the formula, is about 0.0045, or 0.45 percent, which is tiny. Is this margin of error correct? Explain. Assume that the calculations are correct.
A. According to the formulas, the mathematics may be correct, but the results are based on biased data and are therefore bogus. Basically, all Internet polls are bogus except those that actually go out and select their participants at random from the population. And that is impossible to do with a general population because many folks don’t use computers or go online.
13 Does margin of error measure bias?
14 Suppose that a margin of error isn’t reported. Should you automatically assume that it has a small value and move on?
Answers to Problems in Confidence Intervals
1 Yes, you can say that you are 95 percent confident that the population parameter is in your confidence interval.
Instructors like to make a big deal out of this interpretation issue. Avoid the trap of saying that a 95 percent confidence interval means that the parameter has a 95 percent chance of being in your interval. After you create the interval, the parameter either falls in or it doesn’t. The 95 percent confidence is in your sampling process, before the fact. After the interval is done, the parameter is either in there or it isn’t.
2 No, the only thing you can be sure of is that 69 percent of the sample registered to vote. You can’t put a one-number guess on where the population percentage lies; you need an interval of possible values (hence, a confidence interval).
You should never give a one-number estimate as to what the value of a population parameter is. You always need an interval, which includes the one-point estimate plus or minus a margin of error.
3 Yes, because this statement is about the sample, which you know everything about. However, the statement doesn’t mean much to you, because ultimately you want to find out about the target population (all females in the United States), not just those in the sample.
4 No, because the figure may be 69 percent, or it may be another percentage. You can hope that the population percentage is close to 69 percent, but you have no guarantee of that.
5 This problem makes you focus on the lower and upper limits of the confidence interval, which are correct, because 69 percent plus or minus 3 percent gives you 66 percent and 72 percent for your limits. However, you can’t say for sure that the actual population percentage falls in between — that you don’t know.
6 Well, you can be 100 percent sure that the percentage falls in between those values. Why? Because this statement concerns the percentage in the sample, you don’t need an interval. You know the percentage is exactly 69. (Again, this kind of statement isn’t relevant, because the purpose of a confidence interval is to give you some idea of where the population parameter may be, using the sample statistic.)
7 No, because this statement again puts a probability on whether or not the parameter is in the interval. The probability should pertain to the intervals, not to the parameters. In 95 percent of the intervals, the parameter is included, because 95 percent of the random samples represent the true population and 5 percent don’t, just by chance.
8 Yes, you can say that. Just hope your interval isn’t one of those 5 percent.
9 No. The goal is to have a narrow confidence interval with a high level of confidence, because that means less chance for error while you (hopefully) zoom in close on the true value.
10 Not necessarily. If the sample size is large, it decreases the margin of error and can offset the larger confidence level that requires a larger 
or t-value.
11 No. Bias isn’t measured by any part of a confidence interval.
12 No, Bob is wrong. If you look at the formula for margin of error for a proportion 
, you see a square root of n in the denominator. That means that 4 times the sample size decreases the margin of error by a factor of 2 (the square root of 4). So, to cut the margin of error in half, you need to quadruple the sample size (multiply it by 4).
13 No. Bias is the amount by which the statistic in the front part of the confidence interval can be systematically off (on the high end or the low end). Bias is nearly impossible to measure, and the equations for margin of error certainly don’t include it.
Margin of error doesn’t measure bias. Biased results cause inaccurate statistics, and no matter how small the added and subtracted margin of error seems to be, the whole interval is inaccurate due to the bias. The best policy is to avoid bias and discount results that seem precise but are based on biased data.
14 No, you can’t assume that it’s a small value. If you have the sample size and the percentage from the sample, you can figure out the size for yourself, but for quantitative data, you need both the sample mean and standard deviation.
Don’t assume that “no margin of error means a good (small) margin of error.” There may be a reason the margin isn’t reported, and that reason may not be good.