Confirmation bias
Confirmation bias is the tendency to look for, notice and remember the evidence that fits what you already believe, so that a claim gets tested only in ways that could confirm it. It is not lying and it is not stubbornness. It is a fault in the search: the mind asks "can I find support for this?" when the useful question is "what would show this is wrong?"
What confirmation bias is
Confirmation bias is a systematic distortion in how evidence is gathered and weighed, and it operates at three separate stages: what you go looking for, what you notice when it arrives, and what you recall later. That three stage structure is what distinguishes it from its siblings. Anchoring corrupts a single number. The halo effect corrupts a single rating. Confirmation bias corrupts the whole procedure by which you would have caught either of them, which is why it is the one most worth learning first.
It also has a range that most cognitive bias entries do not. A shopkeeper convinced that a new window display works will remember the busy Saturday and forget the quiet Tuesday. A researcher convinced a material is stronger will run the test again after a bad result and accept the good one. Neither person is dishonest. Both have run a search that could only return one answer.
The one sentence version taught in psychology courses
The standard psychology definition, the one that appears in an introductory course and in AP psych revision, is short: confirmation bias is the tendency to seek and interpret information in a way that supports one's existing beliefs. Two words in that sentence carry the weight. Seek covers the search, and it is the half people forget. Interpret covers what happens to ambiguous evidence, which is that it quietly resolves in favor of the belief already held.
Keith Stanovich and colleagues use the narrower term myside bias for the version that involves a belief you are personally committed to, as opposed to a neutral hypothesis you happen to be entertaining. The distinction is useful: the effect gets stronger the more of your own standing is attached to the answer.
The experiment behind it: Wason's 2-4-6 task
The experiment behind confirmation bias is Peter Wason's 1960 study, published as "On the failure to eliminate hypotheses in a conceptual task", and it is worth walking through because it is cleaner than any everyday example. Wason gave people the number triple 2, 4, 6 and told them it obeyed a rule he had in mind. They could propose their own triples, one at a time, and he would say only whether each one fitted the rule. When they were confident, they announced the rule.
Almost everyone formed a hypothesis quickly: ascending even numbers, or numbers rising in twos. Then they tested it by proposing 8, 10, 12. Yes. 20, 22, 24. Yes. 100, 102, 104. Yes. They announced the rule and were wrong. Wason's actual rule was any three numbers in ascending order, and the only way to find it is to propose a triple you expect to fail, such as 3, 5, 4, or 1, 2, 100. The participants ran dozens of confirmations and almost no attempted refutations.
Wason produced a second demonstration in the mid 1960s, the four card selection task. Four cards show A, K, 4 and 7, and each card has a letter on one face and a number on the other. The rule to test is: if a card has a vowel on one side, it has an even number on the other. Most people turn A and 4. Turning A is right. Turning 4 can only confirm, because the rule says nothing about what is on the back of an even number. The card that can actually break the rule is 7, and very few people turn it. This is where confirmation bias meets deductive reasoning: the logic of the task is trivial once written out, and people still fail it, because the search instinct overrides the structure of the argument.
Where confirmation bias shows up in ordinary work
Confirmation bias shows up wherever a person both holds a view and controls the search, which describes most of professional life. Five familiar settings:
- Hiring. An interviewer forms an impression in the first minute and spends the remaining forty asking questions that would only confirm it.
- Diagnosis of a fault. An engineer decides the problem is the pump, tests the pump repeatedly, and never checks the valve.
- Product claims. A manufacturer runs the durability trial that its material passes and files the one it failed as unrepresentative.
- Reading the news feed. A search engine or a recommender returns more of what you clicked last time, so the search is biased before you make it.
- Reviewing your own past decision. Here confirmation bias hands over to hindsight bias, which rewrites your memory of what you expected so the outcome looks like the one you predicted.
Confirmation bias and the biases it is confused with
Four other biases get filed under this heading and are not the same thing. Keeping them apart makes each one easier to catch.
| Effect | What it distorts | The giveaway |
|---|---|---|
| Confirmation bias | The search for evidence | Every test you ran could only have come out one way |
| Hindsight bias | Memory of what you expected | You are sure you called it, and there is no record |
| Self serving bias | Attribution of causes | Wins are your judgment, losses are the weather |
| The sunk cost fallacy | The decision to continue | The reason to carry on is money already spent |
| The Dunning Kruger effect | Estimation of your own skill | The self rating does not move when the score does |
Confirmation bias is the general member of that family and belongs to the broader category of cognitive bias. The other four each corrupt one specific judgment. This one corrupts the method, so it can feed all of them.
What actually reduces confirmation bias
To reduce confirmation bias, redesign the search so that a disconfirming result is possible and someone is responsible for producing it. Four measures with a decent record:
- State in advance what result would change your mind, in writing, before the data arrives.
- Assign one person the job of arguing the opposite, and judge them on how good the argument is, not on whether it wins.
- Register the analysis before you run it, which is why preregistration became standard practice in several research fields after the replication problems of the 2010s.
- Seek the strongest version of the other case, not the weakest one you can find, since defeating a weak version proves nothing.
What does not work is resolving to be more open minded. Confirmation bias produces no sensation of closed mindedness while it is happening, so there is nothing for the resolution to attach to.
The test to run before you trust your own conclusion
The test to run before you trust your own conclusion is one question, and it is worth answering out loud: what is the single piece of evidence that would make me abandon this, and have I gone looking for it? If no such evidence exists, the belief is not supported by the search you ran, because no search could have gone against it.
Then run the Wason move on whatever you are testing next. Propose the case you expect to fail. If you have only ever proposed cases you expected to pass, you have collected confirmations, and confirmations are cheap.