To some people, the funding of a study signals a bias so strong as to raise deep doubts about its findings, if not invalidating them altogether. The story of the promotion and market withdrawal of the painkiller Vioxxcomes to mind. Manufactured by Merck, the once-popular and profitable drug was withdrawn from the market in 2004 over concerns that long-term use increased the risk of heart attacks and strokes. Amid a storm of controversy, a New England Journal of Medicine editorial accused authors of a major Merck-funded study of Vioxx of misleading readers by withholding data that would have further highlighted that risk. The F.D.A. concluded that Vioxx had caused tens of thousands of deaths.
To other people, the disclosure of industry ties might be interpreted assigns of honesty and expertise. For instance, patients might believe that physicians more closely involved in a drug’s development better understand its value or for whom it is most suitable. One study found that patients had more loyalty to doctors who disclosed such ties. Both reactions to disclosure of industry relationships — distrust or trust — are instances of the behavioral economics concept of attribute substitution, a mental shortcut we all employ every day.
When your doctor advises you to take a drug or change your diet for your health, you tend to believe he or she is right without examining the underlying studies for yourself. That’s substituting an attribute you ascribe to the doctor (who’s a trusted and credentialed expert) for your own, very difficult, analysis of the evidence.
Judging a policy as a bad idea just because you had previously voted against the political official promoting it is another example of attribute substitution.
My work, published with colleague Steven Pizer, illustrates this phenomenon. When Medicare’s prescription drug benefit was introduced in 2006, Medicare beneficiaries who lived in areas where the Bush administration was less popular were less likely to enroll in it, even controlling for a host of factors about plans, and the markets in which and populations to whom they were offered. Our interpretation: In making enrollment decisions, many beneficiaries substituted their unfavorable (or favorable) views of the administration that had enacted and promoted the benefit for the more complex and difficult task of assessing its merits directly.
And that’s the problem. Whenever we apply attribute substitution, we’re sidestepping the merits of the thing we’re substituting for. We rely on signs we interpret as markers of quality and merit, even if they’re imperfect substitutes for them. Sure, it’s often a rational and necessary mental shortcut. We lack the time or ability to assess the evidence for every choice we need to make or every view we hold. It may even lead to the right answer more often than not, but it’s unlikely to do so all of the time.
To permit one’s views of an industry-funded study to be substantially colored by that fact is attribute substitution, too. There is evidence that physicians do this. A study published in The New England Journal of Medicine in 2012 found that physicians substantially downgraded their perception of hypothetical industry-funded clinical trials, even when they were reported to be as methodologically rigorous as those funded by other sources.
It’s possible physicians in this study used funding source as a heuristic for sound science. Even research that appears to be methodologically rigorous might conceal a flawed scientific process, which is hard to observe. For instance, without a lot of work, it is hard to assess if researchers selectively withhold unflattering findings about a drug, accentuating the positive and concealing the negative.
When we confront disclosures of potential conflicts of interest in research, we should also confront how we use them and why. We should ask: What information are they substituting for? How can we make that more relevant information more accessible?
Good science is how we avoid fooling ourselves, even when we have incentive to do so, financial and otherwise. The true merits of a study stem from its design and methods, so long as they are fully and transparently reported — and there are many ways we could do a better job of that.
To the extent research design and methods are not up to snuff, that’s the red flag — the door through which conflicts of interest enter and exert undue influence. More rigorous, transparent and reliable research from both industry and nonindustry sources would reduce the need to lean so heavily on mental shortcuts like attribute substitution in judging scientific merit.
@afrakt
