Introduction

1. See Daniel Kahneman, Thinking, Fast and Slow (New York: Farrar, Straus and Giroux, 2011).

2. Aristotle, Politics, trans. E. Barker (London: Oxford University Press, 1972), 123.

3. John Rawls, A Theory of Justice (Cambridge: Belknap Press, 1971), 358–359.

4. Irving L. Janis, Groupthink, 2nd ed. (Boston: Houghton Mifflin, 1982), 7–9.

5. For an overview, see Marlene E. Turner, Anthony R. Pratkanis, and Christina K. Struckman, “Groupthink As Social Identity Maintenance,” in The Science of Social Influence: Advances and Future Progress, ed. Anthony Pratkanis (New York: Psychology Press, 2007), 223–246.

6. Ibid.

7. Kahneman, Thinking Fast and Slow; Dan Ariely, Predictably Irrational: The Hidden Forces That Shape Our Decisions (New York: Harper, 2008); Sendhil Mullainathan and Eldar Shafir, Scarcity: Why Having Too Little Means So Much (New York: Times Books, Henry Holt and Company, 2013); Richard H. Thaler and Cass R. Sunstein, Nudge: Improving Decisions About Health, Wealth, and Happiness (New York: Penguin, 2009).

Chapter 1

1. Chip Heath and Rich Gonzalez, “Interaction with Others Increases Decision Confidence but Not Decision Quality: Evidence Against Information Collection Views of Interactive Decision Making,” Organizational Behavior and Human Decision Processes 61 (1995): 305–326.

2. See Robert S. Baron et al., “Social Corroboration and Opinion Extremity,” Journal of Experimental Social Psychology 32 (1996): 537–560.

3. Huaye Li and Yasuaki Sakamoto, “The Influence of Collective Opinion on True-False Judgment and Information-Sharing Decision,” unpublished manuscript, February 1, 2013, http://ssrn.com/abstract=2210742.

4. Ibid.

5. Robert L. Thorndike, “The Effect of Discussion upon the Correctness of Group Decisions: When the Factor of Majority Influence Is Allowed For,” Journal of Social Psychology 9 (1938): 343–362.

6. Daniel Gigone and Reid Hastie, “Proper Analysis of the Accuracy of Group Judgments,” Psychological Bulletin 121 (1997): 149, 161; Reid Hastie, “Review Essay: Experimental Evidence on Group Accuracy,” in Information Pooling and Group Decision Making, ed. Bernard Grofman and Guillermo Owen (Greenwich, CT: JAI Press, 1986), 129–158.

7. Robert J. MacCoun, “Comparing Micro and Macro Rationality,” in Judgments, Decisions, and Public Policy, ed. Rajeev Gowda and Jeffrey Fox (Cambridge: Cambridge University Press, 2002), 116, 121.

8. J. Scott Armstrong, “Combining Forecasts,” in Principles of Forecasting: A Handbook for Researchers and Practitioners, ed. J. Scott Armstrong (New York: Springer, 2001), 433.

9. James Surowiecki, The Wisdom of Crowds: Why the Many Are Smarter than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations (New York: Doubleday, 2004).

10. Irving Lorge et al., “A Survey of Studies Contrasting the Quality of Group Performance and Individual Performance, 1920–1957,” Psychological Bulletin 55 (1958): 344.

11. Surowiecki, The Wisdom of Crowds, 5 (discussing jar experiment).

12. Ibid., xi–xiii.

13. Some affirmative evidence can be found in Armstrong, “Combining Forecasts,” 417, 419–420, 427, 433–435.

14. Ibid., 16.

15. Theodore C. Sorensen, Kennedy (New York: Harper & Row, 1965), 306.

16. Arthur M. Schlesinger Jr., A Thousand Days: John F. Kennedy in the White House (New York: Houghton Mifflin, 1965), 258–259.

17. Ibid., 255.

18. See the overview in Solomon E. Asch, “Opinions and Social Pressure,” in Readings About the Social Animal, 11th ed., ed. Joshua Aronson and Elliott Aronson (New York: Worth, 2011), 17–26.

19. Reid Hastie, Steven Penrod, and Nancy Pennington, Inside the Jury (Cambridge, MA: Harvard University Press, 1983).

20. Caryn Christensen and Ann S. Abbott, “Team Medical Decision Making,” in Decision Making in Health Care, ed. Gretchen B. Chapman and Frank A. Sonnenberg (New York: Cambridge University Press, 2000), 267, 273–276.

Chapter 2

1. For an overview, see Thomas Gilovich, Dale Griffin, and Daniel Kahneman, Heuristics and Biases: The Psychology of Intuitive Judgment (New York: Cambridge University Press, 2002).

2. Amos Tversky and Daniel Kahneman, “Availability: A Heuristic for Judging Frequency and Probability,” Cognitive Psychology 5 (1973): 208.

3. Paul Slovic, The Perception of Risk (London: Earthscan Publications, 2000), 37–48.

4. Ibid., 40.

5. Amos Tversky and Daniel Kahneman, “Judgment Under Uncertainty: Heuristics and Biases,” in Judgment Under Uncertainty: Heuristics and Biases, ed. Daniel Kahneman, Paul Slovic, and Amos Tversky (Cambridge: Cambridge University Press, 1982), 3.

6. Daniel Kahneman and Shane Frederick, “Representativeness Revisited: Attribute Substitution in Intuitive Judgment,” in Heuristics and Biases: The Psychology of Intuitive Judgment, ed. Thomas Gilovich, Dale W. Griffin, and Daniel Kahneman (Cambridge: Cambridge University Press, 2002), 49.

7. Paul Rozin and Carol Nemeroff, “Sympathetic Magical Thinking: The Contagion and Similarity ‘Heuristics,’” in Heuristics and Biases, Gilovich, Griffin, and Kahneman, eds., 201.

8. Malcolm Gladwell, Blink: The Power of Thinking Without Thinking (New York: Little, Brown, and Co., 2005).

9. Alexander Todorov, Anesu N. Mandisodza, Amir Goren, and Crystal C. Hall, “Inferences of Competence from Faces Predict Election Results,” Science Magazine 308 (2005): 1623–1626.

10. An excellent set of summaries of various biases can be found in Cognitive Illusions, Rüdiger F. Pohl, ed., (New York: Psychology Press, 2012).

11. Don Moore, Elizabeth Tenney, and Uriel Haran, “Overprecision in Judgment,” in Blackwell Handbook of Judgment and Decision Making, ed. Gideon Keren and George Wu (Oxford: Blackwell, forthcoming).

12. Roger Buehler, Dale Griffin, and Johanna Peetz, “The Planning Fallacy: Cognitive, Motivational, and Social Origins,” Advances in Experimental Social Psychology 43 (2010): 1.

13. Scott A. Hawkins and Reid Hastie, “Hindsight: Biased Judgments of Past Events After the Outcomes Are Known,” Psychological Bulletin 107 (1990): 311–327.

14. Hal R. Arkes and Catherine Blumer, “The Psychology of Sunk Cost,” Organizational Behavior and Human Decision Processes 35 (1985): 124–140.

15. Garold Stasser and Beth Dietz-Uhler, “Collective Choice, Judgment, and Problem Solving,” in Blackwell Handbook of Group Psychology: Group Processes, ed. Michael A. Hogg and R. Scott Tindale (Oxford: Blackwell, 2001), 48.

16. Ibid.

17. Janet A. Sniezek and Rebecca A. Henry, “Accuracy and Confidence in Group Judgment,” Organizational Behavior and Human Decision Processes 43 (1989): 1–28. This finding very much bears on excessive risk-taking, including in the context of making war. See Dominic Johnson, Overconfidence and War: The Havoc and Glory of Positive Illusions (Cambridge, MA: Harvard University Press, 2004), 180–183.

18. See Norbert L. Kerr, Robert J. MacCoun, and Geoffrey P. Kramer, “Bias in Judgment: Comparing Individuals and Groups,” Psychology Review 103 (1996): 687, 689, 691–693.

19. Edward L. Schumann and W. C. Thompson, “Effects of Attorney’s Arguments on Jurors’ Use of Statistical Evidence,” unpublished manuscript, 1989.

20. Glen Whyte, “Escalating Commitment in Individual and Group Decision Making: A Prospect Theory Approach,” Organizational Behavior and Human Decision Processes 54 (1993): 430.

21. Ibid., 430–455.

22. Robert J. MacCoun, “Comparing Micro and Macro Rationality,” in Judgments, Decisions, and Public Policy, ed. Rajeev Gowda and Jeffrey Fox (Cambridge: Cambridge University Press, 2002), 116, 121.

23. Mark F. Stasson et al., “Group Consensus Processes on Cognitive Bias Tasks: A Social Decision Scheme Approach,” Japanese Psychological Research 30 (1988): 68–77.

24. See generally Dagmar Stahlberg et al., “We Knew It All Along: Hindsight Bias in Groups,” Organizational Behavior and Human Decision Processes 63 (1995): 46–58.

Chapter 3

1. A. N. Meltzoff and K. Moore, “Imitation of Facial and Manual Gestures by Human Neonates,” Science 198 (1977): 75–78; and see G. Rizzolatti and L. Craighero, “The Mirror-Neuron System,” Annual Review of Neuroscience 27 (2004): 169–192.

2. Elaine Hatfield, John T. Caccioppo, and R. L. Rapson, Emotional Contagion (New York: Cambridge University Press, 1994); Nicholas A. Christakis and J. H. Fowler, “The Spread of Obesity in a Large Social Network over 32 Years,” New England Journal of Medicine 357 (2007): 370–379; J. H. Fowler and Nicholas A. Christakis, “Dynamic Spread of Happiness in a Large Social Network: Longitudinal Analysis over 20 Years in the Framingham Heart Study,” British Medical Journal 337 (2008): 1–9.

3. Matthew J. Salganik, Peter Sheridan Dodds, and Duncan J. Watts, “Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market,” Science 311 (2006): 854–856; Matthew Salganik and Duncan Watts, “Leading the Herd Astray: An Experimental Study of Self-fulfilling Prophecies in an Artificial Cultural Market,” Social Psychology Quarterly 71 (2008): 338–355; Matthew Salganik and Duncan Watts, “Web-Based Experiments for the Study of Collective Social Dynamics in Cultural Markets,” Topics in Cognitive Science 1 (2009): 439–468.

4. Salganik and Watts, “Leading the Herd Astray.”

5. Jan Lorenz et al., “How Social Influences Can Undermine the Wisdom of Crowd Effect,” Proceedings of National Academy of Sciences 108 (2011): 9020–9025.

6. This is a “soft” result, as the conclusion depends on the statistics used to analyze the data, but overall, groups were still wiser than individuals, though not always at conventional levels of statistical significance; Barbara Mellers, Wharton School, interview with authors, Philadelphia, August 14, 2013.

7. The literature here is vast. For a superb summary, see David Hirschleifer, “The Blind Leading the Blind,” in The New Economics of Human Behavior, ed. Marianno Tommasi and Kathryn Ierulli (Cambridge: Cambridge University Press, 1995), 188.

8. We draw here on Hirschleifer, “The Blind Leading the Blind,” 188, 193–194.

9. For a theoretical discussion, see Erik Eyster and Matthew Rabin, “Naïve Herding in Rich-Information Settings,” American Economic Journal: Microeconomics 2 (2010): 221–243.

10. Judith M. Punchoar and Paul W. Fox, “Confidence in Individual and Group Decision Making: When ‘Two Heads’ Are Worse Than One,” Journal of Educational Psychology 96 (2004): 582–591.

11. Cameron Anderson and Gavin J. Kilduff, “Why Do Dominant Personalities Attain Influence in Face-to-Face Groups? The Competence-Signaling Effects of Trait Dominance,” Journal of Personality and Social Psychology 96 (2009): 491–503; Sunita Sah, Don A. Moore, and Robert J. MacCoun, “Cheap Talk and Credibility: The Consequences of Confidence and Accuracy on Advisor Credibility and Persuasiveness,” Organizational Behavior and Human Decision Processes 121 (2013): 246–255.

12. Ibid.; Cass R. Sunstein, Why Societies Need Dissent (Cambridge, MA: Harvard University Press, 2003).

13. Lisa R. Anderson and Charles A. Holt, “Information Cascades in the Laboratory,” American Economic Review 87 (1997): 847–862.

14. Angela A. Hung and Charles R. Plott, “Information Cascades: Replication and an Extension to Majority Rule and Conformity-Rewarding Institutions,” American Economic Review 91 (2001): 1508, 1515.

15. Thus, 72 percent of subjects followed Bayes’s rule in Anderson and Holt, “Information Cascades in the Laboratory,” and 64 percent in Marc Willinger and Anthony Ziegelmeyer, “Are More Informed Agents Able to Shatter Information Cascades in the Lab?” in The Economics of Networks: Interaction and Behaviours, ed. Patrick Cohendet et al. (New York: Springer, 1998), 291, 304.

16. Willinger and Ziegelmeyer, “Are More Informed Agents,” 291.

17. Anderson and Holt, “Information Cascades in the Laboratory,” 847.

18. Jacob K. Goeree, Thomas R. Palfrey, Brian W. Rogers, and Richard D. McKelvey, “Self-Correcting Information Cascades,” Review of Economic Studies 74 (2007): 733–762, found that when chains were longer than twenty members, the groups tended to correct themselves.

19. Hung and Plott, “Information Cascades,” 1515–1517.

20. Ibid., 1516.

21. Timur Kuran and Cass R. Sunstein, “Availability Cascades and Risk Regulation,” Stanford Law Review 51 (1988): 683–768.

22. Robert E. Kennedy, “Strategy Fads and Competitive Convergence: An Empirical Test for Herd Behavior in Prime-Time Television Programming,” Journal of Industrial Economics 50 (2002): 57–84.

Chapter 4

1. Roger Brown, Social Psychology: The Second Edition (New York: Free Press, 1986), 206–207.

2. Ibid., 204.

3. Ibid., 224.

4. For a clear discussion, see ibid.

5. Serge Moscovici and Marisa Zavalloni, “The Group as a Polarizer of Attitudes,” Journal of Personality and Social Psychology 12 (1969): 125–135.

6. Ibid.

7. Ibid.

8. Cass R. Sunstein et al., Are Judges Political? An Empirical Investigation of the Federal Judiciary (Washington, DC: Brookings Institution Press, 2006).

9. David Schkade, Cass R. Sunstein, and Daniel Kahneman, “Deliberating About Dollars: The Severity Shift,” Columbia Law Review 100 (2000): 1139.

10. Brown, Social Psychology, 200–245. An effort to systematize some of these points can be found in Edward L. Glaeser and Cass R. Sunstein, “Extremism and Social Learning,” Journal of Legal Analysis 1 (2009): 263–324.

11. Brown, Social Psychology. It has similarly been suggested that majorities are especially potent because people do not want to incur the wrath or otherwise lose the favor of large numbers of others, and that when minorities have influence, it is because they produce genuine attitudinal change. See Robert S. Baron et al., “Social Corroboration and Opinion Extremity,” Journal of Experimental Social Psychology 32 (1996): 82.

12. Baron et al., “Social Corroboration,” 557–559, shows that corroboration increases confidence and hence extremism.

13. Ibid., 541, 546–547, 557, concludes that corroboration of one’s views has effects on opinion extremity.

14. Brown, Social Psychology, 209–211; John C. Turner, Margaret S. Wetherell, and Michael A. Hogg, “Referent Informational Influence and Group Polarization,” British Journal of Social Psychology 28 (1989): 135–147; Joel Cooper, Kimberly A. Kelly, and Kimberlee Weaver, “Attitudes, Norms, and Social Groups,” in Social Cognition, ed. Marilynn B. Brewer and Miles Hewstone (Oxford: Blackwell, 2004), 259, 269–270.

15. Brown, Social Psychology, 210.

16. Alex Pentland, Social Physics: How Good Ideas Spread—The Lessons from A New Science (New York: Penguin, 2014).

17. Brown, Social Psychology, 211; Cooper, Kelly, and Weaver, “Attitudes, Norms and Social Groups,” 269.

18. Brendan Nyhan, Jason Reifler, and Peter Ubel, “The Hazards of Correcting Myths About Health Care Reform,” Medical Care 51 (2013): 127–132.

19. Brooke Harrington, Pop Finance: Investment Clubs and the New Investor Populism (Princeton, NJ: Princeton University Press, 2008).

Chapter 5

1. Garold Stasser and William Titus, “Hidden Profiles: A Brief History,” Psychological Inquiry 14 (2003): 304–313.

2. Daniel Gigone and Reid Hastie, “The Common Knowledge Effect: Information Sharing and Group Judgment,” Journal of Personality and Social Psychology 65 (1993): 959–974.

3. Ross Hightower and Lutfus Sayeed, “The Impact of Computer-Mediated Communication Systems on Biased Group Discussion,” Computers in Human Behavior 11 (1995): 33–44.

4. Patricia Wallace, The Psychology of the Internet (Cambridge: Cambridge University Press, 1999), 82.

5. Garold Stasser and William Titus, “Pooling of Unshared Information in Group Decision Making: Biased Information Sampling During Discussion,” Journal of Personality and Social Psychology 48 (1985): 1467–1478.

6. Ibid., 1473; see also Stasser and Titus, “Hidden Profiles,” 304.

7. Stasser and Titus, “Pooling of Unshared Information,” 1473.

8. Ibid., 1476.

9. Ibid.

10. Ibid.

11. Stasser and Titus, “Hidden Profiles,” 305.

12. Daniel Gigone and Reid Hastie, “The Common Knowledge Effect.”

13. Ibid., 960.

14. Ibid., 973.

15. Ibid.

16. Susanne Abel, Garold Stasser, and Sandra I. Vaughan-Parsons, “Information Sharing and Cognitive Centrality,” paper ERS-2005-037 (Rotterdam: Erasmus Research Institute of Management, May 2005).

17. Tatsuya Kameda, Yohsuke Ohtsubo, and Masanori Takezawa, “Centrality in Sociocognitive Networks and Social Influence: An Illustration in a Group Decision-Making Context,” Journal of Personality and Social Psychology 73 (1997): 296–309.

18. Garold Stasser, Laurie A. Taylor, and Coleen Hanna, “Information Sampling in Structured and Unstructured Discussions of Three- and Six-Person Groups,” Journal of Personality and Social Psychology 57 (1989): 67, 72–73.

19. Ibid., 78.

20. Ibid; though high-status members also “self-censor” (Melissa C. Thomas-Hunt, Tonya Y. Ogden, and Margaret A. Neale, “Who’s Really Sharing? Effects of Social Expert Status on Knowledge Exchange Within Groups,” Management Science 49 [2003]: 464–477).

21. Cecilia L. Ridgeway, “Social Status and Group Structure,” in Blackwell Handbook of Group Psychology: Group Processes, ed. Michael A. Hogg and R. Scott Tindale (Oxford: Blackwell, 2001), 352, 354 (collecting studies).

22. Gwen M. Wittenbaum, Anne P. Hubbell, and Cynthia Zuckerman, “Mutual Enhancement: Toward an Understanding of the Collective Preference for Shared Information,” Journal of Personality and Social Psychology 77 (1999): 967, 967–978.

23. Stasser and Titus, “Hidden Profiles,” 311.

Chapter 6

1. Carsten K. W. De Dreu, “Minority Dissent, Attitude Change, and Group Performance,” in The Science of Social Influence: Advances and Future Progress, ed. Anthony R. Pratkanis (New York: Psychology Press, 2007), 247–270.

2. Ibid.

3. Caryn Christensen and Ann S. Abbott, “Team Medical Decision Making,” in Decision Making in Health Care, ed. Gretchen B. Chapman and Frank A. Sonnenberg (New York: Cambridge University Press, 2000), 272–276.

4. Ibid; and also consider Sheryl Sandberg, Lean In: Women, Work, and the Will to Lead (New York: Knopf, 2013).

5. Cecilia L. Ridgeway, “Social Status and Group Structure,” in Blackwell Handbook of Group Psychology: Group Processes, ed. Michael A. Hogg and R. Scott Tindale (Oxford: Blackwell, 2001), 354.

6. Harry Kalven and Hans Zeisel, The American Jury (Boston: Little Brown, 1966).

7. Garold Stasser and William Titus, “Hidden Profiles: A Brief History,” Psychological Inquiry 14 (2003): 308.

8. Ibid.

9. Varda Liberman, Steven M. Samuels, and Lee Ross, “The Name of the Game: Predictive Power of Reputations Versus Situational Labels in Determining Prisoner’s Dilemma Game Moves,” Personality and Social Psychology Bulletin, 30 (2004): 1175–1185.

10. Stasser and Titus, “Hidden Profiles,” 309.

11. Jeffrey A. Sonnenfeld, “What Makes Great Boards Great,” Harvard Business Review, September 2002, http://hbr.org/2002/09/what-makes-great-boards-great/.

12. Brooke Harrington, Pop Finance: Investment Clubs and the New Investor Populism (Princeton, NJ: Princeton University Press, 2008).

13. Angela A. Hung and Charles R. Plott, “Information Cascades: Replication and an Extension to Majority Rule and Conformity-Rewarding Institutions,” American Economic Review 91 (2001): 1515.

14. Garold Stasser, “The Uncertain Role of Unshared Information in Collective Choice,” in Shared Cognition in Organization: The Management of Knowledge, ed. Leigh L. Thompson, John M. Levine, and David M. Messick (Mahwah, NJ: Erlbaum, 1999): 49, 56–57.

15. Stasser and Titus, “Hidden Profiles,” 308 (citing studies showing that “when the bearer of unique information was labeled an expert, the group seemingly paid more attention to the information”); Garold Stasser, Dennis D. Stewart, and Gwen M. Wittenbaum, “Expert Roles and Information Exchange During Discussion: The Importance of Knowing Who Knows What,” Journal of Experimental Social Psychology 31 (1995): 244, 248–249, 256 (showing that assigning expert roles led to more discussion of unshared data).

16. Stasser, Stewart, and Wittenbaum, “Expert Roles,” 248–249.

17. The tale is well told in Marlene E. Turner et al., “Groupthink As Social Identity Maintenance,” in The Science of Social Influence: Advances and Future Progress, ed. Anthony R. Pratkanis (New York: Psychology Press, 2007), 223.

18. Andrew S. Grove, Only the Paranoid Survive: How to Exploit the Crisis Points That Challenge Every Company (New York: Doubleday, 1996), 89.

19. Irving L. Janis, Groupthink, 2nd ed. (Boston: Houghton Mifflin, 1982), 268.

20. Gary Katzenstein, “The Debate on Structured Debate: Toward a Unified Theory,” Organizational Behavior and Human Decision Processes 66 (1996): 316–318.

21. Alexander L. George and Eric K. Stern, “Harnessing Conflict in Foreign Policy Making: From Devil’s to Multiple Advocacy,” Presidential Studies Quarterly 32 (2002): 484, 486.

22. Ibid.

23. Ibid.; Stefan Schulz-Hardt, Marc Jochims, and Dieter Frey, “Productive Conflict in Group Decision Making: Genuine and Contrived Dissent As Strategies to Counteract Biased Information Seeking,” Organizational Behavior and Human Decision Processes 88 (2002): 563–586.

24. Stefan Schulz-Hardt et al., “Group Decision Making in Hidden Profile Situations: Dissent As a Facilitator for Decision Quality,” Journal of Personality and Social Psychology 91 (2006): 1080–1093.

25. Brendan Mulvaney, “Red Teams: Strengthening Through Challenge,” Marine Corps Gazette, July 2012, 63–66; Defense Science Board Task Force, The Role and Status of DoD Red Teaming Activities (Washington, DC: 2003), 1–48, www.fas.org/irp/agency/dod/dsb/redteam.pdf.

26. Gene Rowe and George Wright, “The Delphi Technique As a Forecasting Tool: Issues and Analysis,” International Journal of Forecasting 15 (1999): 353–375.

27. Martin Hilbert, Ian Miles, and Julia Othmer, “Foresight Tools for Participative Policy-Making in Inter-Governmental Processes in Developing Countries: Lessons Learned from the eLAC Policy Priorities Delphi,” Technological Forecasting and Social Change 76 (2009): 880–896.

28. Gene Row and George Wright, “Expert Opinions in Forecasting: The Role of the Delphi Technique,” in Principles of Forecasting: A Handbook for Researchers and Practitioners, ed. J. Scott Armstrong (New York: Springer, 2001), 126.

29. Ibid.

30. Ibid., 130; Reid Hastie, “Review Essay: Experimental Evidence on Group Accuracy,” in Information Pooling and Group Decision Making, ed. Bernard Grofman and Guillermo Owen (Greenwich, CT: JAI Press, 1986), 139.

31. Hastie, “Experimental Evidence on Group Accuracy,” 139–145.

32. Ibid., 129.

33. Ibid., 129–130.

34. David H. Gustafson et al., “A Comparative Study of Differences in Subjective Likelihood Estimates Made by Individuals, Interacting Groups, Delphi Groups, and Nominal Groups,” Organizational Behavior and Human Performance 9 (1973): 280–291.

Chapter 7

1. Donald T. Campbell, “Evolutionary Epistemology,” in The Philosophy of Karl R. Popper, ed. P. A. Schilpp (LaSalle, IL: Open Court, 1974), 412–463.

2. Leslie Valiant, Probably Approximately Correct: Nature’s Algorithms for Learning and Prospering in a Complex World (New York: Basic Books, 2013).

3. Tom Kelley and Jonathan Littman, The Art of Innovation: Lessons in Creativity from IDEO, America’s Leading Design Firm (New York: Doubleday, 2001).

4. Barry Nalebuff and Ian Ayres, Why Not? How to Use Everyday Ingenuity to Solve Problems Big and Small (Cambridge, MA: Harvard Business Review Press, 2006).

5. See, for example, Lani Guinier, The Tyranny of the Majority: Fundamental Fairness in Representative Democracy (New York: Free Press, 1994); Anthony J. McGann, “The Tyranny of the Supermajority: How Majority Rule Protects Minorities,” Journal of Theoretical Politics 16 (2004): 53–77.

6. Doug Hall and David Wecker, Jump Start Your Brain, 2nd ed. (Cincinnati: Clerisy Press, 2007).

7. Daniel Kahneman and Don Lovallo, “Timid Choices and Bold Forecasts: A Cognitive Perspective on Risk-Taking,” Management Science 39 (1993): 17–31.

Chapter 8

1. Some affirmative evidence can be found in J. Scott Armstrong, “Combining Forecasts,” in Principles of Forecasting: A Handbook for Researchers and Practitioners, ed. J. Scott Armstrong (New York: Springer, 2001), 417, 419–420, 427, 433–435.

2. See William P. Bottom, Krishna Ladha, and Gary J. Miller, “Propagation of Individual Bias Through Group Judgment: Error in the Treatment of Asymmetrically Informative Signals,” Journal of Risk and Uncertainty 25 (2002): 152–154.

3. Francis Galton, “Vox Populi,” Nature 75 (1907): 450–451.

4. Richard P. Larrick and Jack B. Soll, “Intuitions About Combining Opinions: Misappreciation of the Averaging Principle,” Management Science 52 (2006): 111–127.

5. See also Clintin P. Davis-Stober, David V. Budescu, Jason Dana, and Stephen B. Broomell, “When Is a Crowd Wise?” Decision 2 (2014): 79–101, for a more general mathematical and empirical analysis of the value of diverse estimates.

6. See Scott E. Page, Diversity and Complexity (Princeton, NJ: Princeton University Press, 2011), for another mathematical argument for “diversity’s inescapable benefits.”

7. Bottom, Ladha, and Miller, “Propagation of Individual Bias,” 153.

8. Ibid. See also David M. Estlund, Democratic Authority: A Philosophical Framework (Princeton, NJ: Princeton University Press, 2008), 225: “The mathematical result is beyond dispute, but it applies only under certain conditions. One is that enough of the votes must be statistically independent. This is often misunderstood. On the overly pessimistic side, many have said this cannot be met since there will always be lots of influence on one another. Few will be independent of each other. What the theorem requires, though, is not causal independence but statistical independence.”

9. On some of the technical complexities, see Christian List and Robert E. Goodin, “Epistemic Democracy: Generalizing the Condorcet Jury Theorem,” Journal of Political Philosophy 9 (2001): 277, 283–288, 295–297.

10. Keith Michael Baker, ed. and trans., Condorcet: Selected Writings (Indianapolis: Bobbs-Merrill, 1976), 62. This point is emphasized in Estlund, Democratic Authority.

11. Tali Sharot, The Optimism Bias: A Tour of the Irrationally Positive Brain (New York: Pantheon Books, 2011).

12. Joseph Henrich et al., “Group Report: What Is the Role of Culture in Bounded Rationality?” in Bounded Rationality: The Adaptive Toolbox, ed. Gerd Gigerenzer and Reinhard Selten (Cambridge, MA: MIT Press, 2001), 353–354, for an entertaining outline in connection with food choices.

13. Baker, Condorcet, 56–57.

14. Ibid., 49.

15. Ibid., 61.

Chapter 9

1. Rebecca Greenfield, “The Best and Worst Pundit Predictors of 2012,” The Wire (blog), November 8, 2012, www.theatlanticwire.com/politics/2012/11/best-and-worst-pundit-predictors-2012/58846/.

2. Philip E. Tetlock, Expert Political Judgment: How Good Is It? How Can We Know? (Princeton, NJ: Princeton University Press, 2006).

3. See, for example, James Shanteau, “Competence in Experts: The Role of Task Characteristics,” Organizational Behavior and Human Decision Processes 53 (1992): 252–266.

4. Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t (New York: Penguin, 2012).

5. Greg Sargent, “What Nate Silver Really Accomplished,” The Plum Line (blog), Washington Post, November 21, 2012, www.washingtonpost.com/blogs/plum-line/post/what-nate-silver-really-accomplished/2012/11/21/6f9f10c2-3410-11e2-bfd5-e202b6d7b501_blog.html.

6. Malcolm Gladwell, Blink: The Power of Thinking Without Thinking (New York: Little, Brown, 2005), provides many examples of this form of expertise, although few of these examples actually demonstrate true predictive expertise when subjected to scientific scrutiny.

7. Herbert A. Simon and William G. Chase, “Skill in Chess,” American Scientist 61 (1973): 394–403.

8. Gary Klein, The Power of Intuition: How to Use Your Gut Feelings to Make Better Decisions at Work (New York: Doubleday, 2003), provides a popularized account of some examples of case-based expertise.

9. Lee R. Brooks, Geoffrey R. Norman, and Scott W. Allen, “Role of Specific Similarity in a Medical Diagnostic Task,” Journal of Experimental Psychology: General 120 (1991): 278–287.

10. J. Scott Armstrong, “Combining Forecasts,” in Principles of Forecasting: A Handbook for Researchers and Practitioners, ed. J. Scott Armstrong (New York: Springer, 2001), 419–420. For many factual questions, of course, a little research would be sufficient to identify the correct answers. But for some factual issues, even significant research is inconclusive, and it is best to consult experts.

11. Ibid., 428.

12. Ibid., 428, 430–431.

13. Ibid., 433.

14. Albert E. Mannes, Jack B. Soll, and Richard P. Larrick, “The Wisdom of Small Crowds,” unpublished manuscript, accessed April 2014, http://opim.wharton.upenn.edu/DPlab/papers/workingPapers/Mannes_working_The%20Wisdom%20of%20Small%20Crowds.pdf.

15. Philip Tetlock, “How to Win at Forecasting,” Edge, December 6, 2012, www.edge.org/conversation/how-to-win-at-forecasting.

Chapter 10

1. Thomas L. Friedman, “When Complexity Is Free,” New York Times, September 14, 2013, www.nytimes.com/2013/09/15/opinion/sunday/friedman-when-complexity-is-free.html.

2. Kevin J. Boudreau, Nicola Lacetera, and Karim R. Lakhani, “Incentives and Problem Uncertainty in Innovation Contests: An Empirical Analysis,” Management Science 57 (2011): 843–863.

3. Christian Terwiesch and Yi Xu, “Innovation Contests, Open Innovation, and Multiagent Problem Solving,” Management Science 54 (2008): 1529–1543.

4. William J. Abernathy and Richard S. Rosenbloom, “Parallel Strategies in Development Projects,” Management Science 15 (1969): 486–505; Richard R. Nelson and Sidney G. Winter, An Evolutionary Theory of Economic Change (Cambridge, MA: Belknap, 1982).

5. Stefan Lindegaard, The Open Innovation Revolution: Essentials, Roadblocks, and Leadership Skills (Hoboken, NJ: John Wiley & Sons, 2010); Don Tapscott and Anthony D. Williams, Wikinomics: How Mass Collaboration Changes Everything (New York: Penguin, 2006).

6. Jeffrey D. Zients, “Guidance on the Use of Challenges and Prizes to Promote Open Government,” Executive Office of the President: Office of Management and Budget, March 8, 2010, www.whitehouse.gov/sites/default/files/omb/assets/memoranda_2010/m10-11.pdf.

7. Ibid.

8. “About Challenge.gov,” Challenge.gov, accessed October 4, 2013, https://challenge.gov/p/about.

9. America COMPETES Reauthorization Act of 2010, Pub. L. No. 111-358, 124 Stat. 3982 (2011); Tom Kalil and Tobynn Sturm, “Congress Grants Broad Prize Authority to All Federal Agencies,” Open Government Initiative, December 21, 2010, www.whitehouse.gov/blog/2010/12/21/congress-grants-broad-prize-authority-all-federal-agencies.

10. Cristin Dorgelo, “Challenge.gov: Two Years and 200 Prizes Later,” Office of Science and Technology Policy, September 5, 2012, www.whitehouse.gov/blog/2012/09/05/challengegov-two-years-and-200-prizes-later.

11. For the State Department prize, see Rose Gottemoeller, “Mobilizing American Ingenuity to Strengthen National Security: A Challenge to the Public,” DipNote (US Department of State official blog), August 28, 2013, http://blogs.state.gov/stories/2012/08/28/mobilizing-american-ingenuity-strengthen-national-security-challenge-public. For the air-quality prize, see “The My Air, My Health HHS/EPA Challenge,” last modified June 17, 2013, US Environmental Protection Agency, http://epa.gov/research/challenges/.

12. Office of Science and Technology Policy, Implementation of Federal Prize Authority: Progress Report (Washington, DC: Office of Science and Technology Policy, 2012), 16, www.howto.gov/sites/default/files/implementation-federal-prize-authority.pdf.

13. Challenge.gov, Agency Stories: Challenge and Prize Competitions (Washington, DC: 2011), 5, www.howto.gov/sites/default/files/agency-stories-challenge-prize-competitions.pdf.

Chapter 11

1. Friedrich Hayek, “The Use of Knowledge in Society,” American Economic Review 35 (1945): 519–530, reprinted in The Essence of Hayek, ed. Chiaki Nishiyama and Kurt R. Leube (Stanford, CA: Hoover Institution, 1984), 211–224. A superb treatment of Hayek’s thought is Bruce Caldwell, Hayek’s Challenge: An Intellectual Biography of F. A. Hayek (Chicago: University of Chicago Press, 2004).

2. Nishiyama and Leube, The Essence of Hayek, 212.

3. Ibid., 214.

4. Ibid., 219–220.

5. Ibid., 220.

6. Robert J. Shiller, Irrational Exuberance, 2nd ed. (New York: Broadway Books, 2005).

7. Ibid., 11.

8. Bo Cowgill, “Putting Crowd Wisdom to Work,” Google Official Blog, September 21, 2005, http://googleblog.blogspot.com/2005/09/putting-crowd-wisdom-to-work.html.

9. Justin Wolfers and Eric Zitzewitz, “Prediction Markets,” Journal of Economic Perspectives 18 (2004): 107–126.

10. Saul Levmore, “Simply Efficient Markets and the Role of Regulation: Lessons from the Iowa Electronic Markets and the Hollywood Stock Exchange,” Journal of Corporation Law 28 (2003): 593.

11. Emile Servan-Schreiber et al., “Prediction Markets: Does Money Matter?” Electronic Markets 14 (2004): 243–251.

12. Richard Roll, “Orange Juice and Weather,” American Economic Review 74 (1984): 871.

13. See Wolfers and Zitzewitz, “Prediction Markets,” 113–114.

14. Donald N. Thompson, Oracles: How Prediction Markets Turn Employees into Visionaries (Boston: Harvard Business Review Press, 2012), 105.

15. Ibid., 103.

16. Ibid., 105.

17. Kay-Yut Chen and Charles R. Plott, “Information Aggregation Mechanisms: Concept, Design, and Implementation for a Sales Forecasting Problem,” working paper, Social Science, Division of the Humanities and Social Sciences, California Institute of Technology, March 2002, 3, http://authors.library.caltech.edu/44358/1/wp1131.pdf. (describing variation of this model employed by Hewlett-Packard).

18. Wolfers & Zitzewitz, “Prediction Markets,” 112; Robert W. Hahn & Paul C. Tetlock, “Harnessing the Power of Information: A New Approach to Economic Development,” working paper, AEI-Brookings Joint Center for Regulatory Studies, 2004, 4, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=641444.

19. Wolfers and Zitzewitz, “Prediction Markets,” 112.

20. Thompson, Oracles, 50.

21. For the IEM results, see “They All Got it Right: Polls, Markets, and Models,” PBS NewsHour, November 7, 2012, www.pbs.org/newshour/businessdesk/2012/11/they-all-got-it-right-polls-ma.html. For the final tally, see “2012 Presidential Election Results,” Washington Post, November 19, 2012, www.washingtonpost.com/wp-srv/special/politics/election-map-2012/president/.

22. Joyce Berg et al., “Results from a Dozen Years of Election Futures Market Research,” unpublished manuscript, March 2003, http://tippie.uiowa.edu/iem/research/papers/bergforsythenelsonrietz_2008.pdf; Joyce E. Berg and Thomas A. Rietz, “Prediction Markets as Decision Support Systems,” Information Systems Frontiers 5 (2003): 79–93.

23. Donald Granberg and Edward Brent, “When Prophecy Bends: The Preference-Expectation Link in U.S. Presidential Elections, 1952–1980,” Journal of Personality and Social Psychology 45 (1983): 479.

24. Koleman S. Strumpf, Manipulating the Iowa Political Stock Market, unpublished manuscript, 2004, cited in Wolfers and Zitzewitz, “Prediction Markets,” 118.

25. Robert Forsythe, Thomas A. Rietz, and Thomas W. Ross, “Wishes, Expectations and Actions: A Survey on Price Formation in Election Stock Markets,” Journal of Economic Behavior and Organization 39 (1999): 94.

26. Ibid., 94–95.

27. Charles G. Lord, Lee Ross, and Mark R. Lepper, “Biased Assimilation and Attitude Polarization: The Effects of Prior Theories on Subsequently Considered Evidence,” Journal of Personality and Social Psychology 37 (1979): 2098–2109. See also Muzafer Sherif and Carl I. Hovland, Social Judgment: Assimilation and Contrast Effects in Communication and Attitude Change (New Haven, CT: Yale University Press, 1961), 188 (discussing how individuals filter information to conform to their preexisting positions).

28. Forsythe, Rietz, and Ross, “Wishes,” 94.

29. Berg et al., Election Futures Market Research, 42.

30. Forsythe, Rietz, and Ross, “Wishes,” 99–100. The term “quasi-rational” comes from Richard H. Thaler, Quasi-Rational Economics (New York: Russell Sage Foundation, 1991), xxi.

31. Richard H. Thaler and William T. Ziemba, “Anomalies: Parimutuel Betting Markets: Racetracks and Lotteries,” Journal of Economic Perspectives 2 (1988): 163, explores favorite-longshot bias. See also Charles F. Manski, “Interpreting the Predictions of Prediction Markets,” unpublished manuscript, August 2005, www.aeaweb.org/assa/2006/0106_1015_0703.pdf, which summarizes horse-race data findings.

32. David Forrest and Ian McHale, “Longshot Bias: Insights from the Betting Market on Men’s Tennis,” in Information Efficiency in Financial and Betting Markets, ed. Leighton Vaughan Williams (Cambridge: Cambridge University Press, 2005), 215–230.

33. Interestingly, some sports betting shows the opposite pattern; in English professional football, long shots have been found to be underpriced. See David Forrest and Robert Simmons, “Efficiency of the Odds on English Professional Football Matches,” in Information Efficiency, ed. Williams, 336.

34. The most important evidence can be found on Tradesports’s predictions, where highly unlikely outcomes were overpriced in a number of domains. See Wolfers and Zitzewitz, “Prediction Markets,” 117.

35. See Richard H. Thaler, ed., Advances in Behavioral Finance, vol. 2 (Princeton, NJ: Princeton University Press, 2009); and see Kay-Yut Chen, Leslie R. Fine, and Bernardo A. Huberman, “Eliminating Public Knowledge Biases in Information-Aggregation Markets,” Management Science 50 (2004): 983–994, for one example of a method of correcting prediction market outputs for risk-attitudes and shared-information biases, increasing accuracy over the raw market prices.

36. Shiller, Irrational Exuberance, 2.

37. For much evidence, see Thaler, Advances in Behavioral Finance.

38. Erin Jordan, “Iowa Electronic Markets Yields Near-Accurate Result,” Des Moines Register, November 10, 2004.

Chapter 12

1. Meredith Halama, “NAI Seeking Public Comment on Revised Code of Conduct,” Network Advertising Initiative, March 1, 2013, www.networkadvertising.org/blog/nai-seeking-public-comment-revised-code-of-conduct.

2. Laura Carroll, “Opinions Please! Retailers Seeking Public’s Input,” Las Vegas Review-Journal, July 1, 2013, www.reviewjournal.com/business/retail/opinions-please-retailers-seeking-publics-input.

3. Ibid.

4. Korri Kezar, “Austin Gun Manufacturer Seeking Public Input,” Community Impact Newspaper, June 25, 2013, http://impactnews.com/austin-metro/round-rock-pflugerville-hutto/gun-manufacturer-seeking-public-input/; “The Crowdsourced Smart Rifle,” TrackingPoint, June 21, 2013, https://tracking-point.com/labs/future/.

Chapter 13

1. D. J. Devine and J. L. Phillips, “Do Smart Teams Do Better: A Meta-analysis of Cognitive Ability and Team Performance,” Small Group Research 32 (2001): 507–532; Suzanne T. Bell, “Deep-Level Composition Variables as Predictors of Team Performance: A Meta-analysis,” Journal of Applied Psychology 92 (2007): 595–615.

2. Walter Mischel, Personality and Assessment (New York: Wiley, 1968).

3. Lee Ross and Richard E. Nisbett, The Person and the Situation: Perspectives of Social Psychology (New York: McGraw-Hill, 1991).

4. Malcolm Gladwell, “Personality Plus,” New Yorker, September 2004, 42.

5. See, for example, Kenneth M. Nowack, “Is the Myers Briggs Type Indicator the Right Tool to Use?” Performance in Practice, American Society of Training and Development (fall 1996): 6.

6. Christina A. Rideout and Susan A. Richardson, “A Teambuilding Model: Appreciating Differences Using the Myers-Briggs Type Indicator with Developmental Theory,” Journal of Counseling and Development, 67 (1989): 532. Tricia Varvel et al., “Team Effectiveness and Individual Myers-Briggs Personality Dimensions,” Journal of Management in Engineering 20 (2004): 146, have similar findings: “This research did not conclude that a particular combination of personality type preferences have a direct incidence on team effectiveness.”

7. Bell, “Deep-Level Composition Variables,” 595; Steve W. J. Kozlowski and Bradford S. Bell, “Work Groups and Teams in Organizations,” in Handbook of Psychology: Industrial and Organizational Psychology, 2nd ed., ed. Randy K. Otto (New York: Wiley, 2013), 12:412.

8. Ibid.

9. Anita Woolley et al., “Evidence for a Collective Intelligence Factor in the Performance of Human Groups,” Science 330 (2010): 686–688.

10. Simon Baron-Cohen et al., “The ‘Reading the Mind in the Eyes’ Test, Revised Version: A Study with Normal Adults, and Adults with Asperger Syndrome or High-Functioning Autism,” Journal of Child Psychology and Psychiatry 42 (2001): 241–251.

11. Anita W. Woolley and Thomas W. Malone, “What Makes a Team Smarter? More Women,” Harvard Business Review, June 2011, http://hbr.org/2011/06/defend-your-research-what-makes-a-team-smarter-more-women/.

12. Linus Torvalds and David Diamond, Just for Fun: The Story of an Accidental Revolutionary (New York: HarperCollins, 2001).

13. Andrew Lih, The Wikipedia Revolution: How a Bunch of Nobodies Created the World’s Greatest Encyclopedia (New York: Hyperion, 2009).