Mar 04, 2017 Volume 389 Number 10072 p881-982 e3
Research matters: challenges of replication
Scientific progress builds on an ever-accumulating body of evidence gathered over generations, and is deeply rooted in tenets of reliability and reproducibility. Last week, in Nature, Jeffery Mogil and Malcolm Macleod proposed a new framework that aims to instil greater confidence in research, calling for findings from basic research laboratories to be validated by a definitive preclinical trial before testing in human beings.
In their article, Mogil and Macleod highlight concerns of a so-called reproducibility crisis in scientific research. And they are by no means the first. In 2016, results of a brief online questionnaire of over 1500 basic science researchers published in Nature revealed that as many as 90% of those surveyed agreed that there was a reproducibility crisis in science. The Reproducibility Project: Cancer Biology aims to independently reproduce published results of high profile studies in cancer biology. In January this year, eLife published findings from the first five replication studies, some of which showed mixed results.
Such endeavours should be applauded for their intentions to ensure dissemination of robust findings in a culture often driven by fierce competition for publications and funding. But use of the term crisis over-generalises complex issues that are specific to highly intricate biological systems—such terminology could dangerously undermine public confidence in science and researchers. In an era when alternative facts are presented as truth, and lines between fact and fiction are increasingly blurred, trust in research integrity must be reaffirmed and defended.
Replicating studies is important, but to conclude that a failure to reproduce specific findings is failed science oversimplifies the problem. Novel research is, by nature, exploratory and diverse, and variations exist between experimental sites and individuals. Prescriptive regulation of scientific thought and processes that stifle creativity under a guise of enforcing reliability could ultimately impede discovery and advancement. Stakeholders, including funders, publishers, industry, and academics must enter a dialogue to establish nuanced solutions to improve transparency, accountability, and reporting of research.