1.6 The Role of Statistics in Critical Thinking and Ethics
According to H. G. Wells, author of such science-fiction classics as The War of the Worlds and The Time Machine, “Statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write.” Written more than a hundred years ago, Wells’s prediction is proving true today.
The growth in data collection associated with scientific phenomena, business operations, and government activities (quality control, statistical auditing, forecasting, etc.) has been remarkable in the past several decades. Every day the media present us with published results of political, economic, and social surveys. In increasing government emphasis on drug and product testing, for example, we see vivid evidence of the need for quantitative literacy (i.e., the ability to evaluate data intelligently). Consequently, each of us has to develop a discerning sense—an ability to use rational thought to interpret and understand the meaning of data. Quantitative literacy can help you make intelligent decisions, inferences, and generalizations; that is, it helps you think critically using statistics.
Biography H. G. Wells (1866–1946)
Writer and Novelist
English-born Herbert George Wells published his first novel, The Time Machine, in 1895 as a parody of the English class division and as a satirical warning that human progress is inevitable. Although most famous as a science-fiction novelist, Wells was a prolific writer as a journalist, sociologist, historian, and philosopher. Wells’s prediction about statistical thinking is just one of a plethora of observations he made about life on this world. Here are a few more of H. G. Wells’s more famous quotes:
“Advertising is legalized lying.”
“Crude classification and false generalizations are the curse of organized life.”
“The crisis of today is the joke of tomorrow.”
“Fools make researchers and wise men exploit them.”
“The only true measure of success is the ratio between what we might have done and what we might have been on the one hand, and the thing we have made and the things we have made of ourselves on the other.”
Statistical thinking involves applying rational thought and the science of statistics to critically assess data and inferences. Fundamental to the thought process is that variation exists in populations of data.
To gain some insight into the role statistics plays in critical thinking, we present two examples of some misleading or faulty surveys.
Example 1.8 Biased Sample—Motorcyclists and Helmets
Problem
An article in the New York Times considered the question of whether motorcyclists should be required by law to wear helmets. In supporting his argument for no helmets, the editor of a magazine for Harley-Davidson bikers presented the results of one study that claimed “nine states without helmet laws had a lower fatality rate (3.05 deaths per 10,000 motorcycles) than those that mandated helmets (3.38)” and a survey that found “of 2,500 bikers at a rally, 98% of the respondents opposed such laws.” Based on this information, do you think it is safer to ride a motorcycle without a helmet? What further statistical information would you like?
Solution
You can use “statistical thinking” to help you critically evaluate the study. For example, before you can evaluate the validity of the 98% estimate, you would want to know how the data were collected. If a survey was, in fact, conducted, it’s possible that the 2,500 bikers in the sample were not selected at random from the target population of all bikers, but rather were “self-selected.” (Remember, they were all attending a rally—a rally, likely, for bikers who oppose the law.) If the respondents were likely to have strong opinions regarding the helmet law (e.g., to strongly oppose the law), the resulting estimate is probably biased high. Also, if the biased sample was intentional, with the sole purpose to mislead the public, the researchers would be guilty of unethical statistical practice.
You would also want more information about the study comparing the motorcycle fatality rate of the nine states without a helmet law to those states that mandate helmets. Were the data obtained from a published source? Were all 50 states included in the study, or were only certain states selected? That is, are you seeing sample data or population data? Furthermore, do the helmet laws vary among states? If so, can you really compare the fatality rates?
Look Back
Questions such as these led a group of mathematics and statistics teachers attending an American Statistical Association course to discover a scientific and statistically sound study on helmets. The study reported a dramatic decline in motorcycle crash deaths after California passed its helmet law.
Now Work Exercise 1.35c
Example 1.9 Manipulative or Ambiguous Survey Questions—Howard Stern on Sirius Radio
Problem
A few years ago, talk-show host Howard Stern moved his controversial radio program from free, over-the-air (AM/FM) radio to Sirius satellite radio. At the time the move was perceived in the industry to boost satellite radio subscriptions. This led American Media Services, a developer of AM/FM radio properties, to solicit a nationwide random-digit dialing phone survey of 1,008 people. The purpose of the survey was to determine how much interest Americans really have in buying satellite radio service. After providing some background on Howard Stern’s controversial radio program, one of the questions asked, “How likely are you to purchase a subscription to satellite radio after Howard Stern’s move to Sirius?” The result: Eighty-six percent of the respondents stated that they weren’t likely to buy satellite radio because of Stern’s move. Consequently, American Media Services concluded that “the Howard Stern Factor is overrated” and that “few Americans expect to purchase satellite radio”—claims that made the headlines of news reports and Weblogs. Do you agree?
Solution
First, we need to recognize that American Media Services has a vested interest in the outcome of the survey—the company makes its money from over-the-air broadcast radio stations. Second, although the phone survey was conducted using random-digit dialing, there is no information provided on the response rate. It’s possible that nonrespondents (people who were not home or refused to answer the survey questions) tend to be people who use cell phones more than their landline phone, and, consequently, are more likely to use the latest in electronic technology, including satellite radio. Finally, the survey question itself is ambiguous. Do the respondents have negative feelings about satellite radio, Howard Stern, or both? If not for Howard Stern’s program, would the respondents be more likely to buy satellite radio? To the critical thinker, it’s unclear what the results of the survey imply.
Look Back
Examining the survey results from the perspective of satellite radio providers, 14% of the respondents indicated that they would be likely to purchase satellite radio. Projecting the 14% back to the population of all American adults, this figure represents about 50 million people; what is interpreted as “few Americans” by American Media Services could be music to the ears of satellite radio providers.
Now Work Exercise 1.33b
Ethics in Statistics
Intentionally selecting a biased sample in order to produce misleading statistics is considered unethical statistical practice.
As with many statistical studies, both the motorcycle helmet study and the satellite radio study are based on survey data. Most of the problems with these surveys result from the use of nonrandom samples. These samples are subject to errors such as selection bias, nonresponse bias (recall Example 1.6), and measurement error. Researchers who are aware of these problems yet continue to use the sample data to make inferences are practicing unethical statistics.
Statistics in Action Revisited
Critically Assessing the Ethics of a Statistical Study
The results from the Pew Internet & American Life Project (PIALP) survey on social networking use led the research center to make conclusions such as “Facebook is the dominant social networking platform.” The survey included a sample of 1,445 adult Internet users. In order to assess the validity of this inference, a critical thinker would consider several issues.
First, is the sample representative of all U.S. adult Internet users? In the PIALP report, the claim is made that the survey results are based on telephone interviews with a nationally representative sample of Internet users. Two samples of adults were selected. The first sample consisted of landline telephone numbers randomly selected (using a random number generator) from all landline numbers listed in the United States. The second sample consisted of cellular telephone numbers randomly selected from all U.S. cell numbers that were not associated with directory-listed landline numbers. As many as seven attempts were made to contact and interview the adult at each number. These contact calls were staggered over times of day and days of the week in an attempt to reduce nonresponse bias. Clearly, there was no attempt to bias the results of the study by intentionally selecting a sample with a certain point of view. In fact, just the opposite occurred. Consequently, the sample of adults appears to be a truly random sample of all adult Internet users in the United States; therefore, the sample is very likely to be representative of the population of interest.
Second, is the sample of 1,445 adults large enough to draw a reliable inference? Although there are more than 250 million Internet users in the United States, we will learn in later chapters that a random sample of size 1,445 will allow the researchers to draw a reliable inference. In fact, the PIALP report states that the margin of sampling error for the study is about 3%. This means that, with a high level of confidence, anywhere from 70% to 76% of all the adult Internet users in the United States have visited a social networking site.
Third, is the wording of the survey questions ambiguous or leading? Such questions are often crafted to elicit an answer that supports a specific point of view. For example, consider a hypothetical question about Twitter: “Twitter is becoming more and more popular with affluent people. Have you ever used or desired to use Twitter in order to keep up with your friends?” The question is meant to elicit a “yes” answer and bias the results of the study. If you examine the questions in the actual study (see p. 2), you will conclude that all are very straightforward and clear. Consequently, the survey results appear to be both ethical and valid.
In the remaining chapters of the text, you’ll become familiar with the tools essential for building a firm foundation in statistics and statistical thinking.