I recommend Lisa Rosenbaum’s three-part NEJM series on financial conflicts of interest (links: part 1, part 2, and part 3). Though it is thought provoking throughout, this single sentence was enough to occupy my mind for several hours:
Once moral intuitions enter the picture, the need to rationally weigh trade-offs is often eclipsed by unexamined convictions about right and wrong.
It is now commonplace for authors to disclose potential financial conflicts of interest (COI) to journals and institutional review boards (IRBs) before paper publication and initiation of research, respectively. You can most easily find COI statements at the end of many published papers, or accompanying them online. Here’s just a part of one COI disclosure for a paper I pulled at random from the NEJM archives:
The paper is about a drug (bevacizumab) manufactured by Genentech (as Avastin), so this particular COI disclosure for this particular author is relevant. (This author is one of 18 or so on the paper. Most of the others have no such disclosed COI, though some do.)
If I’ve ever read any COI disclosures as part of reading or evaluating a published study, it’s only been a few times. I have purposefully avoided them for many years. Why?
I worry about bias: my own. I simply don’t know what to make of COI disclosures. It’s easy to detect a potential or appearance of a COI. It’s much harder to decide how to weigh that when evaluating a study. Sure, it’s a data point that could be meaningful. So could a myriad of “irregularities” that might show up in a full body MRI on a patient with no symptoms of disease. I worry about false positives and emotional harm. How does this author’s prior financial relationship with Genentech affect the published research? Does it affect my head even more?
I do not want to worry about COI (or worry about my worry about it) when evaluating a paper’s methods.
Several years ago I received an email encouraging me to consider the work of a certain author. The work was relevant to whatever I was blogging about at the time. But I knew that author had substantial industry funding for his work, and decided I wasn’t going to read or consider his work on that basis. I emailed back as much.
I regret that decision and that email. I should have considered the work on its merits. My assessment that it could not have been worthwhile was a biased one. I don’t read COI disclosures because I want to protect myself from that bias, acknowledging that I might be blinding myself to the authors’ own biases. There’s no way to win here.
For the same reason, for years I didn’t read authors’ bios. With respect to the quality of the work, why should their institution, titles, or other credentials matter? Either their study is sound or it isn’t. If I can’t assess that from a paper’s text and figures alone (as a blinded reviewer would), then that’s a problem, but it’s not one that can be resolved by knowing an author’s pedigree any more than it can be resolved by knowing her skin color.
In fact, for years I didn’t even read authors’ names on papers. I barely knew who wrote what, until it came time to cite stuff. Then I had to know names. Over the years I came to recognize some, got to know scholars across the country.
Now I’m friends with and colleagues of many. I know where they work. I know their credentials. I consider by lines along with article titles when deciding what to read. There are some authors whose work I never want to miss. Is this a bias? Time being finite, it certainly crowds out reading others’ work.
All this meta data—names, affiliations, degrees, potential COI—can bias. Once it enters my head, I cannot tell the extent to which it does. I could argue that I’m merely being Bayesian when I use prior knowledge of the authors’ work or their institutions. (This one has a well-earned reputation for good work; this other one is from a “lesser” institution widely thought to have an ideological perspective.) And maybe that’s right. But I could also argue that I’m using—even subconsciously—this meta data to unfairly evaluate the work.
Lisa is right that once intuitions—moral and otherwise—like these enter the picture, we’re already in difficult terrain. Problems arise by unexamined convictions, she wrote. But, for me, problems arise by examined ones as well. I do think money influences, as do relationships and beliefs. But when I examine my own feelings about these, I’m no closer to understanding the extent to which I use them in my own biased way, if at all.
Once I gather the meta data, what should I do with it? What have I already done?
