Reproducibility Crisis: It’s Time To Revisit The Past

Updated on

Last year, psychologists led by Brian Nosek of the University of Virginia published results of the massive Reproducibility Project. Nosek and his colleagues collectively repeated 100 previously published psychological experiments. But they were able to replicate results of less than 40 of them. The landmark study cast doubt on results of about 60 of published studies.

There is far more evidence than just reproducibility project

Now it has exposed deep divisions in the field of psychology. On one side are psychologists who feel that the field is experiencing a reproducibility crisis. Many of the most well-known results may not be true. On the other side are those who think that no such crisis exists. Now they have openly challenged the statistical analyses and methods of the Reproducibility study.

On Thursday, four psychologists challenged the last year’s Reproducibility Project, saying the study was flawed and its results were wrong. The same day, authors of the replication project, including Brian Nosek, issued a strong rebuttal. Nosek said, “They are making assumptions based on selectively interpreting data and ignoring data that’s antagonistic to their point of view.”

While two sides continue to discuss and debate, the reality is that Reproducibility Project is not the only evidence of problems with psychological studies. There has been publication bias where only studies with positive results are published, while those with negative results are dismissed. Many a times psychologists have failed to replicate the textbook phenomena. Sometimes researchers use questionable practices to conduct their studies. The list is endless.

Future generations need solid foundation, not uncertainty

Michael Inzlicht, a social psychologist at the University of Toronto, said, “Our problems are not small and they will not be remedied by small fixes. Our problems are systemic and they are at the core of how we conduct our science.” Inzlicht has spent almost two decades doing research, and much of his work as been on the concept of ego depletion.

Now Michael Inzlicht wonders whether the topics he chose to study are real and robust. He says our reluctance to dig into the past and ask what needs to be revisited is not going to help us. If there is so much of uncertainty about the results of past studies, how are the students and future psychologists going to decide what research to build upon?

Leave a Comment