Ever been in this situation? You have an opinion or an idea and you just know you’re right. You set out to prove it, and all the facts you line up seem to prove your side of the story. This is confirmation bias– the tendency to interpret new evidence as confirmation of one’s existing beliefs or theories.
Even scientists who pride themselves on being objective and evidence-based are susceptible, as physicist Walter Ogburn learned the hard way. In a recent talk given to students in the Berkeley Connect Physics program, Ogburn brought students along with him on a heartbreaking journey from discovery to disappointment.
In the data-heavy slides Ogburn presented, he made his case for his team’s discovery, based on data collected from the BICEP 2 over a three-year period. All the projections looked right and all the signs pointed towards the definitive existence of these primordial B-modes.
Unfortunately, as Ogburn described, the team turned out to be wrong. They had used an inapplicable measurement level for cosmic dust that gave a false positive for primordial B-modes. What the team had announced was likely the presence of cosmic dust that, when viewed through the right lens, appeared to be consistent with primordial B-modes. In a subsequent joint investigation with another team of astronomers, Ogburn and his colleagues confirmed what they had already feared, their findings were not what they seemed. In what felt like a tragic retraction, the evidence critically important to the cosmic inflation theories was snatched away.
Despite this sad turn of events, the researchers at BICEP have not given up hope, and further iterations of BICEP have been deployed in the quest for data.
Ultimately, Ogburn described this turn of events as a learning experience. An “instrumental bias” towards the set of tools that the team was working with, and a tendency to work only with the things with which they were familiar, were two possible areas of weakness that Ogburn identified. These biases led to the critical cosmic dust measurements being overlooked and the data inadvertently manipulated. Ogburn’s advice to aspiring scientists was to check for such biases and fight them at every opportunity to avoid a similar situation.
It is not every day that a momentous-seeming scientific discovery is retracted, and it is even rarer to hear one of the scientists involved in such a retraction speak frankly about the experience. Berkeley Connect students got a unique behind-the-scenes view at the intersection of the scientific method and human fallibility. All students at Berkeley are encouraged to seek out “discovery experiences.” Ogburn’s story was a powerful reminder that the path to discovery is not always straightforward, and can require the ability to learn from one’s mistakes.