How We Know What Isn’t So: The Fallibility of Human Reason in Everyday Life
Recommended by: Bill Roos
Posted in: Social Psychology
Purchase →Explains how cognitive processes of the human brain, built through evolution and usually helpful, can cause us to believe things that are not true.
Thomas Gilovich gives us reason for pause in this book cataloguing a number of cognitive mechanisms in the human brain that often distort our judgment and interfere with our ability to be the knowledgeable, wise, and judicious sages we like to think we are. For example:
“The tendency to impute order to ambiguous stimuli is simply built into the cognitive machinery we use to apprehend the world. It may have been bred into us through evolution because of its general adaptiveness: We can capitalize on ordered phenomena in ways that we cannot on those that are random. The predisposition to detect patterns and make connections is what leads to discovery and advance. The problem, however, is that the tendency is so strong and so automatic that we sometimes detect coherence even when it does not exist.
“The predisposition to impose order can be so automatic and so unchecked that we often end up believing in the existence of phenomena that just aren’t there.
“[T]his willingness to base conclusions on incomplete or unrepresentative information is a common cause of people’s questionable and erroneous beliefs.”
In addition, people tend to see in a body of evidence what they expect to see, which is often what they want to see.
When we want to believe something, we may frame the question as: “Can I believe this?” Since there is almost always some evidence to support our preferred conclusion, we are likely to find supportive evidence if we keep looking. Once we find that evidence we can stop looking. On the other hand, for a conclusion we do not want to accept, we ask, “Must I believe this?” That is a much higher standard.
We can also improve our chances of reaching the “correct” conclusion by carefully choosing the sources we consult, to increase our chances of gathering supportive evidence.
Contributing to our difficulties is the fact that the world does not serve us quality information upon which we can make our judgments:
“But the world does not play fair. Instead of providing us with clear information that would enable us to “know” better, it presents us with messy data that are random, incomplete, unrepresentative, ambiguous, inconsistent, unpalatable, or secondhand. As we shall see, it is often our flawed attempts to cope with precisely these difficulties that lay bare our inferential shortcomings and produce the facts we know that just ain’t so.”
Furthermore, humans have a tremendous ability to invent stories after the fact explaining how something came to be. So,
“[O]nce a person has (mis)identified a random pattern as a “real” phenomenon, it will not exist as a puzzling, isolated fact about the world. Rather, it is quickly explained and readily integrated into the person’s pre-existing theories and beliefs. These theories, furthermore, then serve to bias the person’s evaluation of new information in such a way that the initial belief becomes solidly entrenched.”
Of course, one of the key steps in any analysis is gathering the relevant data. Gilovich spends a good deal of time discussing how we obtain our knowledge:
“Much of what we know in today’s world comes not from direct experience, but from what we read and what others tell us. An ever-higher percentage of our beliefs rest on a foundation of evidence that we have not collected ourselves. Therefore, by shedding light on the ways in which secondhand information can be misleading, we can better understand a common source of questionable and erroneous beliefs.”
Of course, much of our information comes from the news media, and here again human nature contributes to the difficulties:
“Our appetite for entertainment is enormous, and it has a tremendous impact on the tales we tell and the stories we want to hear. The quest for entertainment is certainly one of the most significant sources of distortion and exaggeration in everyday communication.
“As NBC News anchorman Tom Brokaw admits, “It’s tricky, trying to generate understanding and insight while not ignoring the entertainment factor.” Inaccuracies and fabrications propagated by the media are a particularly powerful cause of people’s erroneous beliefs, in part because of the reputation much of the media have for objectivity and accuracy, a reputation that is not always deserved.
“Those who work in the mass media face tremendous pressure to put out a product—to meet a deadline, fill an hour, or generate advertising space. Often the demand for suitable material outstrips the supply of factual stories that are novel and interesting, and the temptation to stretch the truth or lower one’s standards of objectivity and verification can be enormous.”
As an example of how these kinds of cognitive processes can interfere with analysis, Gilovich discusses the effectiveness of the SAT. There is a line of criticism arguing that since the relationship between the SAT and college GPAs is modest (with a correlation coefficient of about 0.2), the SAT is not effective at predicting student success in college. But this is a failure to recognize that we are analyzing a problem without using all the relevant data (the problem of hidden or absent data). Gilovich points out that students with very different SAT scores tend not to enroll in the same schools, so the correlation of SAT scores and GPAs within a university cannot tell us how students with very different SAT scores would perform in the same environment. But by looking at the few schools that have open enrollment but still attract many students with very high SATs, we find a much wider range of SAT scores than is the case in most universities. Examining that wider range (more relevant data), we see that students with high SAT scores are more likely to do well in college than those with low SAT scores (with correlations as high as .6 to .7).
Gilovich sums up the lessons of this valuable book:
“To truly appreciate the complexities of the world and the intricacies of human experience, it is essential to understand how we can be misled by the apparent evidence of everyday experience. This, in turn, requires that we think clearly about our experience, question our assumptions, and challenge what we think we know.”