Some ideas for reducing publication bias and increasing the credibility of published scientific findings, from Brendan Nyhan:
- Pre-accepted articles, based on pre-registered protocol and before findings are known. Brendan reports that this is already happening at AIMS Neuroscience, Cortex, Perspectives on Psychological Science, Social Psychology, and for a planned special issue of Comparative Political Studies.
- Results-blind peer review. A similar idea to pre-accepted articles, this would evaluate submissions on all aspects of a paper (data, methods, import) apart from the actual findings. Brendan notes that this has been attempted at Archives of Internal Medicine.
- Verifying replication data and code, basically providing everything necessary to replicate the study. Already standard at the American Journal of Political Science and American Economic Review.
- Reward higher quality and faster reviews with credits, redeemable for faster review of one’s own manuscripts. I’m not aware of a any journal that has attempted any program aimed at reducing review times, let alone succeeded in doing so.
- Forward reviews of promising manuscripts to section journals. That is, if the flagship journal can’t accept, but recommends publication in an affiliated journal, streamline the process by treating the flagship’s review as the first round. Something like this already happens with JAMA journals and the American Economic Review and its affiliated journals.
- Triple blind reviewing would blind the editor from the authors, not just the authors and reviewers from one another. Already standard at Mind, Ethics, and American Business Law Journal.
As Brendan writes, all of these have limitations and none can remove all potential bias or gaming. Yet it’s hard to argue they’re not worth considering.
