ScienceResearch
The journals that started checking the code
Five titles now ask authors to run their analysis before publication. The first year of results is humbling.
Five journals now require the analysis code alongside the manuscript. Reviewers run it once, on a clean machine, before the paper goes anywhere near a decision.
In the first year, roughly one submission in three failed to reproduce its own headline number. Most failures were not fraud: they were paths that only worked on the author's laptop, missing dependencies and a spreadsheet read in the wrong order.
What the check catches
The reviewers are not judging the science. They are checking that the figure in the paper is the figure the code produces, and that a stranger can get there without emailing anyone.
“Almost nothing we found was misconduct. It was the ordinary mess of a project that was never meant to be read by anyone else.”
Authors have started keeping notebooks they would once have deleted. The journals say the next step is automated re-runs whenever a paper is corrected.
Priya Raman
Health and science reporter
Priya reports on medicine, research and public health, with a particular interest in how evidence moves from a trial into a clinic. She trained as a pharmacist before becoming a journalist.