Sunday, July 4, 2021

Replication results that strongly contradict an original finding do not necessarily nullify its credibility but we'd at least expect the replication results to be acknowledged & explicitly debated; this happens much less than it should

Hardwicke, Tom E., Dénes Szűcs, Robert T. Thibault, Sophia Crüwell, Olmo Van den Akker, Michele B. Nuijten, and john Ioannidis. 2021. “Citation Patterns Following a Strongly Contradictory Replication Result: Four Case Studies from Psychology.” MetaArXiv. February 9. doi:10.31222/osf.io/wt5ny

Abstract: Replication studies that contradict prior findings may facilitate scientific self-correction by triggering a reappraisal of the original studies; however, the research community's response to replication results has not been studied systematically. One approach for gauging responses to replication results is to examine how they impact citations to original studies. In this study, we explored post-replication citation patterns in the context of four prominent multi-laboratory replication attempts published in the field of psychology that strongly contradicted and outweighed prior findings. Generally, we observed a small post-replication decline in the number of favourable citations and a small increase in unfavourable citations. This indicates only modest corrective effects and implies considerable perpetuation of belief in the original findings. Replication results that strongly contradict an original finding do not necessarily nullify its credibility; however, one might at least expect the replication results to be acknowledged and explicitly debated in subsequent literature. By contrast, we found substantial citation bias: the majority of articles citing the original studies neglected to cite relevant replication results. Of those articles that did cite the replication, but continued to cite the original study favourably, approximately half offered an explicit defence of the original study. Our findings suggest that even replication results that strongly contradict original findings do not necessarily prompt a corrective response from the research community.

 Check also Only 54pct of newspapers than published erroneous research findings published the retraction; the retraction stories were balanced, but shorter than those on the article’s publication and often lacking in context & detail:

Dissemination of Erroneous Research Findings and Subsequent Retraction in High-Circulation Newspapers: A Case Study of Alleged MDMA-Induced Dopaminergic Neurotoxicity in Primates. Brian S. Barnett & Richard Doblin. Journal of Psychoactive Drugs, Nov 26 2020. https://www.bipartisanalliance.com/2020/12/only-54pct-of-newspapers-than-published.html



No comments:

Post a Comment