I’d like to wrap up this lecture with a final comment on the bigger question that motivated this topic in the first place. The question was whether there’s a way of defending the superiority of science as a source of knowledge about the world, without resorting to a mythic view of how science works.
I think the answer is “yes,” but it’s a qualified “yes.” I think it’s clear that if we didn’t follow these scientific protocols, our knowledge of the world would be less reliable than it is, and it’s clear why this is so when you think of these protocols as methods for neutralizing the effects of cognitive biases.
As you read the rest of this section, whenever you see the word “science,” remember that marketing research is a social science. And good marketing research uses the same general methods as any science (e.g., experiments), approaches (relying on past research to inform future research); and techniques (e.g., mathematics and statistics) to discover insights about markets.
But saying this doesn’t mean that scientists always follow these protocols. Science is a complex social practice, and there are lots of things that can interfere with or prevent these protocols from being properly implemented. The highest quality studies are often the most expensive studies to conduct, so funding can be a limiting factor. The highest quality studies might also take many years, maybe even decades to conduct, so time constraints can be another factor.
And, in some fields, proper controlled studies might just be impossible to conduct. In genetics, for example, there are experimental ways of measuring the heritability of a trait, which is the percentage of the variation in the trait that can be accounted for by genetic variation in the population. You can do these controlled experiments to directly measure heritability on fruit flies, but you can’t do them on human populations because they would be unethical to conduct. So for humans we’re forced to rely on more indirect methods of estimating heritability that are more limited and more vulnerable to biases.
So I admit that in some ways, this discussion of scientific methods still has an air of mythology about it, in the sense that it sets up an ideal that may never be perfectly realized in practice. But on the other hand, we still retain a notion of what makes science special, namely, that it’s an institutionalized social practice that is committed to these ideals, that it strives to reduce the distorting effects of biases when it can. And in this respect it’s distinctive; there’s no other social institution that functions quite the same way ....
The Takeaway Message
So, the takeaway message of this ... is really fairly limited. It doesn’t imply much for the big philosophical questions about the nature of science. But it does suggest a certain kind of attitude toward science, and the authority of science.
It suggests first of all, that people should be very cautious about relying on their intuitions in judging a scientific (or a marketing) issue. Our intuitions are just not reliable. One kid developing autism after a vaccination does not imply that the vaccine was the cause of the autism. But we all know that—the sample size is too small. What about 6,000 kids? Our intuition tells us that if 6,000 kids develop autism after being vaccinated, that’s at least evidence for a strong correlation, right. wrong. It’s evidence, but it’s lop-sided, it’s an incomplete body of data. Think of the Dr. Jones example.
When you actually look at a more complete body of data, including background rates of autism, the evidence is clear: there is no statistically significant correlation between vaccination and the development of autism. The number of reported cases of autism has certainly shot up over the past 30 years, but part of this is attributable to changes in diagnostic practices; how much of an increase there’s been in the actual prevalence of the condition is still unclear, but there’s no evidence that it’s linked to vaccinations. Multiple studies from different scientific bodies agree on this conclusion.
Now, I know that a lot of people in the anti-vaccine movement resist this conclusion, and there are a lot of conflicted parents who see this as a tug-of-war between equal sides, an anti-vaccine side and a pro-vaccine side. But the takeaway message of this chapter is that the sides are not equal, and we shouldn’t view them as equal. Human beings, left to their own devices, will see correlations where there aren’t any, and attribute meaning to correlations that are actually meaningless. The more invested you are in the outcome, the more likely it is that you’ll be led into error.
Only a proper scientific study can resolve the issue, and when multiple studies converge on the same conclusion, then the rational thing to do, provisionally, is to accept the scientific consensus ...
Chapter Takeaways
Marketing research can be a defense against cognitive biases. When done well, marketing research borrows from science. The scientific way of thinking and the scientific method help marketers avoid cognitive biases and make good decisions.
In the context of marketing, the scientific way of thinking is a method for understanding and explaining marketing phenomena that incorporates objectivity, systematic investigation, clarity of thought (especially regarding the definitions and concepts it seeks to understand), and knowledge gained from all relevant fields of study.
In the context of marketing, the scientific method is a process by which theories are developed and tested for their ability to explain and enhance our understanding of marketing phenomena.
Marketing research (and science) is what we do to keep us from lying to ourselves.