Chapter 2: Study Designs
Double-Blind, Placebo-Controlled Clinical Trials
Whether you are reading a study or designing your own, one of the most important factors that you should consider is the study design. For example, are you going to randomly assign subjects to different treatments, or are you simply going to observe outcomes in people who have some trait in common? This chapter describes some of the commonly used study designs, and this discussion will help you decide the quality of study results and how much faith you should have in the results, regardless of the p-value or size of the effect.
Double-Blind, Placebo-Controlled Clinical Trials
This is the “gold standard” of all study designs. Even though the term “clinical” is in the title, this study design can be used in all fields of study. The caffeine study described in the previous chapter is an example of this study design. The first step is to select a representative sample of people from the population on which you want to make inferences. The population might be limited by age, gender, or ethnicity. For example, for your caffeine study, you would probably not include very young children. After you conduct an analysis to determine how many subjects you should include in your study (discussed in Chapter 15), you randomly assign the subjects into as many groups as necessary. This design does not necessarily have to include a placebo group even though the study design includes the word “placebo” in the title. It might not be ethical to include a placebo group if this could cause harm to the subjects in this group. For example, if you are comparing drugs to lower blood pressure in a population of subjects with high blood pressure, you might choose to have one group take one of the standard medicines and the other groups take one or more new treatments that you want to study.
Next, we need to discuss the term double-blind. Double-blind means that neither the subject nor the person evaluating the subject knows what treatment or drug a subject is receiving. This is quite easy to do if the subject is taking a drug in pill form. For example, the placebo group can be given an inert substance in a capsule. However, what about a surgical technique or acupuncture? How can the subject be blinded in that case? One study, done in the early 1960s, was designed to test if freezing the stomach with a balloon filled with liquid nitrogen would cure stomach ulcers. One group of subjects had their stomachs frozen. For the placebo group, the liquid nitrogen was introduced into the endoscopy tube, but then shunted out before it reached the stomach. Therefore, it looked like the real thing to the subject and to the staff (except for the person who was switching the shunt on or off). Before this study was conducted, stomach freezing was believed to help cure ulcers. However, the results of the study showed it did not work at all.
The placebo effect can be so strong that even different types of placebos can produce different results. For example, placebo injections, capsules, or pills, can produce different results. Even the color of the pill can sometimes make a difference.
A cohort is a group of people who share a common characteristic, such as taking vitamins or getting annual physical exams. A great deal of knowledge can be gained by comparing groups of subjects who differ in their lifestyles or other factors. For example, you could look at people who exercise regularly versus people who are more sedentary and compare endpoints such as blood pressure, cholesterol, or death from cardio-vascular disease. The advantage of studying cohorts is that you can compare groups containing thousands of subjects.
Farmington, Massachusetts is the site of one of most famous series of cohort studies ever conducted. This town was deemed to be a good representation of the US population. All types of surveys were (and still are) administered to people in the town who were willing to participate in the study. Physical exams and blood tests were also performed on a regular basis. Studies such as these gave epidemiologists data to determine how risk factors such as high cholesterol or high blood pressure are related to heart disease.
However, there are several problems with cohort studies. Because the groups are not randomly assigned to a treatment, the groups may differ on many other factors besides the one being studied. One of the most famous cohort studies that demonstrates how bias can affect the results compared women who took hormone replacement therapy (HRT) with women who did not. The results showed that women who took HRT were significantly less likely to experience a heart attack or stroke.
Years later, a large, double-blind, placebo-controlled clinical trial was conducted to compare HRT to a placebo. The results showed that HRT increased the likelihood of heart attacks and strokes. Why were the results of the earlier studies and the new study so different? Who were the women in the early study who received HRT? Most likely, they were more affluent, had better medical care, and might have lived a healthier lifestyle. It was because of those reasons that it looked like HRT was beneficial when it was not. This demonstrates that while cohort studies can be very useful, the results need to be looked at with more skepticism than results from clinical trials.
The last study design to be described in this chapter is a case-control study. Cases are subjects who already have a disease or condition of interest. Controls are subjects chosen to be similar to the cases except for the fact that they do not have the disease or condition in question. Typical case-control studies compare the two groups to see whether there are differences in lifestyle, exposure to chemicals (such as cigarette smoke), or medications. One such study compared women who gave birth to babies with a condition called neural tube defects. This condition causes an abnormal brain, spine, or spinal column. Controls were women who gave birth to babies without this condition. The two groups were compared on many factors. One factor that showed a significant difference was between women who took folic acid supplements and those who did not.
Why not do a clinical trial to test if folic acid is useful in preventing neural tube defects? There are several reasons why a clinical trial would be difficult. First, neural tube defects are quite rare, so you would need to recruit thousands of women for the study. Another problem is an ethical one. If doctors believe that folic acid helps prevent this condition, how can they, in good conscience, deny folic acid to a placebo group?
Case-control studies are very useful when you are studying a rare condition. However, the results from a case-control study have to be looked at with caution. In the neural tube defect study, women who gave birth to babies with this condition may have had more vivid recollections of what they ate and did while pregnant, compared to women who had normal babies. This problem is called recall bias, and it is difficult to control for. Finally, because you start with people with the disease in one group and healthy people in the other group, you cannot compute incidence rates (how many people get the disease or develop the condition in a certain period of time).
The three study designs described in this chapter are all useful in comparing drugs or treatments in groups of people. There are times when you cannot blind the treatment. One example of this is if you are comparing two different methods to teach students to read. Some of the children (depending on age) might realize they are being taught differently than their friends. The results of studies that are not blinded can still provide useful information.