Chapter 6

Point and Interval Estimation

Estimation Versus Inference

There are two distinct ways of using statistics: Descriptive statistics and inferential statistics. Descriptive statistics provide summary statistics in the forms of tables, graphs, or computed values. We can use descriptive statistics to describe population data or sample data. Inferential statistics is used to draw conclusions about population parameters using sample statistics. Obtaining sample statistics for inferential statistics is the same as obtaining them for descriptive statistics. The use of the sample statistics determines whether it is descriptive or inferential. The statistics obtained from a sample for the purpose of inference is called estimation, to emphasize the fact that they are estimates for their respective parameters. In the previous chapters, the portions of descriptive statistics that dealt with samples are actually estimations. When we obtain sample mean, proportion, or variance we are calculating estimates of the corresponding population parameters. As we have demonstrated earlier, estimation is important. When sample estimates are used to test population parameters and to indicate how far the estimates are from parameters, then we are in the domain of inferential statistics. Some statisticians believe that the primary objective of statistics is to make inferences about population parameters using sample statistics. Using sample statistics to make deductions about population parameters is called statistical inference. Statistical inference can be based on point estimation or confidence interval, both of which will be covered shortly. They are closely related, and in some cases, they are interchangeable.

Point Estimation

A point estimate is the statistic obtained from a sample. The reason for the name is because the estimate consists of a single value. Examples of point estimates include sample mean , sample proportion , sample variance and sample median. These statistics are used to estimate the population mean (μ), population proportion (π), and population variance (σ2), respectively. Sample median can be used to estimate, both, population median and population mean. In fact, any single-valued estimate obtained from a sample is a point estimate. Good estimates are close to the corresponding population parameter. Proper sampling provides accurate estimates of the unknown population parameter. Descriptive statistics addresses the procedure to obtain and calculate point estimates from the sample. It also explains their properties and uses. As discussed in Chapter 3, a suitable estimate is unbiased, consistent, and efficient.

Although point estimates are useful in providing descriptive information about a population, their usefulness is limited because we cannot determine how far they are from the targeted parameter. In order to provide levels of confidence and a probability for margin of error one needs to know the distribution function of sample statistics. Once a sampling distribution of the sample statistic is known, the probability of observing a certain sample statistic can be calculated with the aid of the corresponding table.

Interval Estimation

Statistics deals with random phenomena. Nothing remains constant in life. Methods of production change. Processes are modified. Machines get out of calibration. New techniques are applied. In all these cases, statistics is used to determine what remains constant and what changes. In estimation theory, sample statistics is used to estimate the population parameter. However, when one uses point estimation, it is not clear how close the estimate is to the parameter and there is no measure of confidence. This does not mean that point estimates are useless or unreliable. With proper sampling the point estimates will be unbiased, consistent, and efficient.

Interval estimation augments point estimates by providing a margin of error for the point estimate. The margin of error is a range, based on the degree of certainty, for the estimate of the population parameter, which is added and subtracted from a point estimate. Confidence intervals are based on the point estimate of the parameter and the distribution function of the point estimate. It is affected by the level of certainty, the variance of data, and the sample size.

Calculating Confidence Intervals

Interval estimation is a simple notion and is defined as:

Point estimate ± Margin of error(6.1)

In order to explain this concept we need to recall things from Chapters 2, 3, and 4. We covered point estimates in Chapter 2, although we did not use the same terminology. Point estimates are sample statistics and are used to estimate the corresponding population parameters. Mean, median, variance, and proportion are some common examples of point estimates.

The margin of error is not totally new either. It is based on the use of Z score, which we covered in Chapter 3. In Chapter 4 we introduced the normal distribution function and demonstrated the use of Z score and standardization in obtaining probabilities of random variables from normal distribution. Also in Chapter 4, we used a Z score to obtain the probability under normal distribution between two points. Previously we pointed out that due to symmetry of the standardized normal distribution, the probability of the area from zero, that is, the center, to a point on the right is equal to the probability of the area from zero to the negative value of that number (see Figure 6.1).

Note that in Figure 6.1, the mean is 0 and the standard deviation is 1. Also recall that the unit of measurement of the Z score is the standard deviation. Conventionally, we do not write 1 but if the standard deviation was different from 1, we must include that in the measure, as in Figure 6.1. Therefore, a point on the normal distribution curve is ZΣ away from the mean.

When dealing with sample statistics, the properties of sampling distribution apply. Therefore, in the case of statistics, that is, estimates obtained from the sample, the Central Limit Theorem states that the standard deviation is given by Replacing σ with the this formula provides

(6.2)

It is very important to realize that the margin of error formula in Equation 6.2 depends on the knowledge of population variance. When the population variance is unknown and the sample variance has to be used, then the formula must be adjusted by replacing the population standard deviation with the sample standard deviation and the Z value must be replaced by the t value as in Equation 6.3:

(6.3)

The concept of margin of error applies to sample statistics but not to the population parameters. Population parameters are constant values and do not have margin of error. However, sample statistics, which are random variables and are estimating their respective population parameters, have a margin of error. As seen in Equation 6.2, margin of error is directly related to variance of the population and the level of confidence, as indicated by the Z score, and inversely related to the square root of the sample mean.

Recall from Chapter 5 that the sampling distribution of sample mean is affected by the knowledge of the population variance. If the population variance is known, the distribution function of the sample mean is a normal distribution. However, if the population variance is not known the distribution function of the sample mean is a t distribution. Therefore, when calculating the margin of error we must use either the normal table or the t table, depending on whether the population variance is known or unknown, respectively.

In Chapter 5, we learned the sampling distribution function of one sample statistics such as mean, proportion, and variance. When calculating margin of error for these statistics we need to use the appropriate distribution function. We learned the sampling distribution of the differences of two sample means and two sample proportions, both of which are a normal distribution. We also learned the sampling distribution of two variances, which is an F distribution. Once the margin of error is calculated, obtaining confidence interval is simple by using Equation 6.1.

Terminology

The probability between –Z and +Z from a standard normal, that is, a normal distribution with mean zero 0 and variance 1, is shown by (1 – a)%. This is because the area outside of the above range is equal to a. In Chapter 7, we will provide more explanation for the naming of these areas and go into more detail on the meaning of the term a.

Customarily, normal distribution tables are calculated for half of the area, because of symmetry. Thus, the Z value, which corresponds to one half of a is shown as

Interval Estimation for One Population Mean

The correct way of writing interval estimation of Equation 6.1 when estimating the population mean is

(6.4)

Definition 6.1

Based on Chapter 5, the (1 – a)% confidence interval for the mean of one population (m) when the population variance is known is given by Equation (6.4).

Note that we need to calculate the following two values to obtain a confidence interval.

(6.5)

(6.6)

Sometimes we refer to the lower bond and upper bond as LB and UB, respectively.

Rule 6.1 Inference with Confidence Interval

The confidence interval covers the true population parameter with (1– a)% confidence.

Note that we assumed the sample variance is equal to the population variance in order to apply the example to this case. If the variance is unknown, which is true more often than it is not, we have to use Equation 6.7. Furthermore, when the value of population variance is not known, we must use a t distribution value rather than a Z distribution value.

Definition 6.2

Based on Chapter 5, the (1 – a)% confidence interval for the mean of one population (m) when the population variance is unknown, is given by the following equation:

(6.7)

In earlier days when t tables were only available for 1%, 5%, and 10% levels of significance, most researchers would use a normal table instead of the t table when sample size was greater than 30. With wide availability of t values for any level of significance, this practice is no longer necessary.

Since t values are somewhat larger, to account for the fact that the population variance is unknown and must be estimated, the corresponding confidence interval is wider than the one calculated using a Z table when we assume to know the population variance.

Definition 6.3

Based on Chapter 5, the (1 – a)% confidence interval for the proportion of one population (π) is given by the following equation:

(6.8)

Definition 6.4

Based on Chapter 5, the (1 – a)% confidence interval for the difference of means of two populations (μ1 – μ2) when the population variances are known and unequal is given by the following equation:

(6.9)

Definition 6.5

Based on Chapter 5, the (1 – a)% confidence interval for the difference of means of two populations (μ1–μ2) when the population variances are known and equal is given by the following equation:

(6.10)

Definition 6.6

Based on Chapter 5, the (1– a)% confidence interval for the difference of means of two populations (μ1–μ2) when the population variances are unknown and unequal is given by the following equation:

(6.11)

Definition 6.7

Based on Chapter 5, the (1 – a)% confidence interval for the difference of means of two populations (μ1 – μ2) when the population variances are unknown and equal is given by the following equation:

(6.12)

Definition 6.8

Based on Chapter 5, the (1 – a)% confidence interval for the difference of two population proportions (π1 – π2) is given by the following equation:

(6.13)

Definition 6.9

Based on Chapter 5, the (1– a)% confidence interval for one population variance (σ2) is given by the following equation:

(6.14)

Note that the chi-squared distribution is not symmetric, therefore we cannot use the ± signs to form the confidence interval. Also, it is important to note that the term refers to the right side of the distribution and, hence, it is larger than which refers to the left side of the distribution. Dividing the same numerator by a larger number provides a smaller result, hence the lower bond; while dividing the same numerator by a smaller value gives a larger result, hence the upper bond.

Definition 6.10

Based on Chapter 5, the (1– a)% confidence interval for the ratio of two population variances is given by the following equation:

(6.15)

Note that since the F distribution is not symmetric, we cannot use the ± signs to form the confidence interval. Also, it is important to note that term refers to the right side of the distribution and, hence, it is larger than which refers to the left side of the distribution. Dividing the same numerator by a larger number provides a smaller result, hence the lower bond, while dividing the same numerator by a smaller value gives a larger result, hence the upper bond.

Since the confidence interval does not cover the value “1,” it is unreasonable to believe that the variances of the two periods are the same. We will address this in more detail in Chapter 7, when we discuss the test of hypothesis. In light of this finding, we should use Equation 6.9 when testing equality of the means for the two periods, as in Example 6.6.

Determining the Sample Size

In Chapter 4 we showed the necessary sample size for estimating the single mean of a population. The sample size in that case was obtained by algebraic manipulation of the margin of error in Equation 6.4, which is repeated below for your reference.

(6.4)

Setting the margin of error equal to a desired margin of error, E, and solving for n results in the following formula:

(6.16)

Similar algebraic manipulations are applied to obtain sample sizes for cases with unknown variances, involving, both, the one or two population means, proportions, and variances. We will only show the results for one population proportions for reference.

(6.17)

Inference with Confidence Intervals

The primary objective of statistics is to make inferences about population parameters using sample statistics. Using sample statistics to make deductions about population parameters is called statistical inference. Statistical inference can be based on point estimation, confidence interval, or test of hypothesis. These are closely related and in some aspects they are interchangeable. The inference can be based on the estimation theory or decision theory. Test of hypothesis is a tool for decision theory. The estimation theory consists of point estimation and interval estimation. This section will deal with the confidence interval.

Population parameters are unknown and constants. Sample statistics, which are random by nature, are used to provide estimates of population parameters. If sampling is random, then the sample statistics is a good estimate of the corresponding population parameter. A good sample statistics has desirable properties, as discussed in Chapter 3. These statistics are called point estimates because they provide a single value as the estimate of the population parameter. If the estimator is “good” then it should be close to the unknown true value of the population parameter. The single estimate does not indicate proximity to the true parameter or probability of being close to the true parameter. Confidence intervals give, both, an idea of actual value of population parameter and also a probability, or a level of confidence, that the interval includes the population parameter.