|Oakden-Rayner L, Beam AL, Palmer LJ (2018) Medical journals should embrace preprints to address the reproducibility crisis. Int J Epidemiol 47:1363-5.|
Abstract: Preprints can help detect flaws that might otherwise escape the notice of a conventional peer review process. The world moves at an ever-increasing pace, and this is especially true for biomedical research. The expansion of research capacity enabled by high-performance computing and ubiquitous, high-speed internet have created a fertile environment for the rapid dissemination and iteration of new ideas. There has been an explosion of output in many biomedical fields, including -omics of many stripes, ‘in silico’ research, very large cohort studies (‘biobanking’) and medical artificial intelligence.
This rapid pace of progress has not come without a cost. The rush to publication, an unavoidable consequence of the competitive modern research environment and institutional pressures upon researchers,1 has placed a tremendous strain on traditional avenues of research publication. Numerous high-profile retractions have occurred in recent years. It is believed than many, if not most, medical articles describe results that will not be able to be replicated,2 even if published in high-impact journals.3 These concerns have led to suggestions of a ‘reproducibility crisis’,4 although this might be better characterized as a chronic problem of reproducibility that has existed for some time. Research published in this Journal two decades ago demonstrated that long-standing concerns about the reproducibility of epidemiological research were justified.5 Solutions to this complex issue will comprise many parts, and many may have a long time horizon. For instance, others have focused on potential solutions involving strengthening or removing the use of P-value thresholds.6
This editorial focuses on potential changes to the peer review process as one part of the solution to reproducibility issues in biomedical research.
• Keywords: Preprints, reproducibility crisis • Bioblast editor: Gnaiger E
- Most retractions in biomedical journals have been prompted by post-publication review, where the readership of a journal has detected serious flaws in the published research.
- It should not surprise anyone that post-publication review by the readership of a journal (..) can discover methodological flaws that sneak past expert reviewers.
- If we acknowledge that this post-publication model of review will often detect errors that are missed in traditional pre-publication review, the relevant question we have to ask is whether this level of scrutiny should occur before or after publication?
- Some recent editorials8,9 have raised concerns about the lack of quality control inherent in the public release of non-peer reviewed manuscripts, but we instead suggest that the preprint model simply applies community review before traditional peer review, and in doing so strengthens the scientific process. We believe that more eyes will indeed help to make all errors shallow, and in doing so will preserve the role of formal publications as repositories of reliable science.
- The central fear raised by opponents of preprints is that a wave of harmful information will be unleashed on an unsuspecting public and be further amplified by journalists eager for juicy headlines.
- Thankfully the preprint model is self-correcting, as scarce review resources flow to where they are most needed. As Laplace wrote in 1812, ‘The weight of evidence for an extraordinary claim must be proportioned to its strangeness’.12 This axiom is central to the scientific method, and is exemplified by the open review process that occurs around preprints. Manuscripts that make bold claims are more likely to be discussed and shared, and are therefore likely to be more thoroughly reviewed at the preprint stage. Rather than demanding intensive review of every article written, no matter how minor the contribution or negligible the risk associated with incorrect results, we can envision a future where the majority of the review effort is focused on research of potential real impact where the reliability of the claims is truly important.
- Articles published in medical journals should represent the highest quality and most reliable research available, and pre-publication peer review through preprints will enable these standards to be upheld as the pace of research continues to increase.