8.9 Other Useful Hypothesis Tests

The hypothesis tests presented in this chapter are run-of-the-mill techniques that are used in experiments across many disciplines. So far, though, we have only treated techniques for analyzing the mean of one or several samples. Sometimes we are more interested in properties other than the mean, such as variances or proportions. Such analyses require different hypothesis tests. They will not be treated here but to make it easier for the reader to find the appropriate technique for a given situation, a few additional hypothesis tests and the situations in which they are used will be mentioned. Once we have understood the t-test it is quite straightforward to use other techniques, as all tests are based on the same principles:

• Firstly, formulate a question that contains a statistical parameter and a word expressing a difference. It could be “Is the sample mean different from the target value?”, “Are the variances of these two samples different?”, or something similar. In the teatime experiment it would be “Is the proportion of correct classifications higher than what could be expected to occur by chance?”
• Translate the question into a null hypothesis and an alternative hypothesis. These are mathematical relationships between the sample statistic and the population statistic. The null hypothesis assumes that the answer to the question is no – any apparent difference is due to natural, random variation in the data. As it assumes that no difference exists, the null hypothesis always contains an equals sign. The alternative hypothesis assumes that a real difference underlies the variation in the data – a difference that we would like the data to reveal. The alternative hypothesis is always the opposite of the null hypothesis. This means that, if the null hypothesis contains a “=”, the alternative hypothesis contains a “≠”. If the null hypothesis contains a “≤”, the alternative hypothesis contains a “>”, and so on.
• Find the reference distribution associated with the null hypothesis. This is the probability density function that the statistical parameter follows in the case that the process under study is governed completely by chance.
• Determine the critical value of the reference distribution at the desired level of confidence. Also determine the observed value of the test statistic, which is determined by your data. (This involves a variable transformation, for example from a sample mean to a t-value.) If the observed value is more extreme than the critical one the effect is deemed statistically significant, otherwise it is not.

The greatest difference between different hypothesis tests is the reference distribution that is used. At this point it should be clear that the reference distribution of the mean of a small sample is a t-distribution having the same number of degrees of freedom as the sample. This is because the mean can be transformed into the variable t, according to Equation 8.1. Correspondingly, the variance of a sample can be transformed into a variable that follows what is called a χ2-distribution – pronounced “chi-squared distribution”. It is also characterized by the number of degrees of freedom of the sample. If we want to compare the variance of a sample to a target value we use a so-called χ2-test for variance. If we instead want to compare the variances of two samples we use the so-called F-test. We have already been acquainted with the F-distribution in connection with the ANOVA, where we stated that ratios of variances follow this distribution. The F-distribution is characterized by the numbers of degrees of freedom of the two samples that are compared.

Some experiments yield results in the form of proportions. For example, if a team of engineers modifies a manufacturing process to improve product quality, it may be interested in knowing if the failure rate is lower among the items that have gone through the new process compared to those going through the original one. If you want to test proportions there are a number of techniques at your disposal, and the particular situation decides which one is better suited. Box, Hunter and Hunter [3] provide some examples of such tests.