Publication bias is a serious issue, threatening the credibility of medical and social science. Yet, it’s an issue about which something can be done. One thing would be to require all trials to be prospectively registered.* If that were done systematically, at least one could be aware of trials not leading to publication. If trial data were disclosed, one could also have some hope of assessing why.
A natural enforcement mechanism is journals’ uniform refusal to publish studies that were not prospectively registered. In the absence of such steps, we only have the (biased) published record to go on. With so many trials sponsored by parties with a clear financial conflict (e.g., drug and device manufacturers), this is not an insubstantial problem.
In a new study, published in BMJ, Elizabeth Wager and Peter Williams conducted a systematic analysis of a random sample of 200 medical journals’ policy with respect to prospective registration of trials, as well as a qualitative analysis of reasons so few journals require it.
Of the 200 journals sampled, 142 (71%) did not require registration (or at least did not mention this on their website), 55 (28%) required registration, and 3 (2%) encouraged registration but did not make it a requirement for publication. […]
Our study shows that most journals that publish clinical trials do not make prospective registration a requirement for publication or even encourage it in their web based instructions. The editors and publishers we interviewed proposed several reasons why journals might not require trials to be registered, or might not enforce a trial registration policy strictly. These included fear of losing good submissions to other journals, concern about preventing publication of studies from developing countries, and scepticism about the value of insisting on registration for small or exploratory studies. Other reasons why journals may not have a policy on trial registration include not publishing many primary trial reports. […]
Although prospective trial registration is predicted to reduce publication bias and selective reporting of trials and outcomes, only a minority of journals that publish clinical trials make it a requirement for publication and many editors do not seem convinced that they should adopt such a policy. This suggests that journal editors may not believe that the benefits of trial registration are sufficiently important to be a mandatory requirement, or have concerns that requiring registration as a condition of publication would harm their journal, or both. As one editor (referring to small trials) said “it seems hardly worth the effort.” [Emphasis added.]
Among the other results, a collective action problem is clearly identified. From a scientific point of view, we’d all be better off if all journals required trial registration. Yet, any given journal can offer an easier route to publication by not doing so. No doubt some journals can distinguish themselves as more rigorous and prestigious by adopting a registration requirement policy. However, there is heterogeneity in authors’ (and funders’) preferences. No doubt, some just want to publish. They’re willing to pay for it, and don’t care as much about the science or consequences of bias. Indeed, they may not be consciously aware of it. It’s a race to the bottom and, apart from the sponsoring stakeholders and, perhaps, the authors and journals that benefit, we all suffer for it.
* An extension to observational studies is to prospectively disclose analysis plans. This would provide the information needed to assess whether published results are the outcome of “fishing expeditions,” i.e., running regression after regression for outcome after outcome until something pops up as significant, as eventually something will just by chance alone.
