Jerry Avorn, Ameet Sarpatwari, and Aaron Kesselheim have an important article in the NEJM about a lawsuit challenging the FDA’s right to limit what manufacturers can say about the off-label uses of their drugs. The legal and policy theory about the FDA is deep and subtle. I have a half-serious suggestion: Let’s run a controlled trial to find out whether FDA regulation of medications makes a difference.
The manufacturer pursuing the suit claims a First Amendment free speech right to make statements about the effectiveness of its drug, without FDA limits. However, as Avorn et al., note
communication about medications has traditionally been held aside for special legal treatment… legislators and regulators have imposed much tighter communication restrictions on FDA-regulated manufacturers than on other companies. Such measures have been based on the concepts that interpreting statements about medical products such as prescription drugs requires highly specialized knowledge and that the consequences of misleading statements in this domain could have important effects on health or even life itself.
But does FDA regulation of manufacturers actually save lives? Scholars such as Daniel Klein and Alex Tabbarok argue that
FDA control over drugs and devices has large and often overlooked costs that almost certainly exceed the benefits. We believe that FDA regulation of the medical industry has suppressed and delayed new drugs and devices, and has increased costs, with a net result of more morbidity and mortality.
If that is so, where are the bodies resulting from the FDA’s regulations? Writing about the FDA drug approval process, Tabbarok argues that the deaths are real but “statistical”:
the FDA has an incentive to delay the introduction of new drugs because approving a bad drug (Type I error) has more severe consequences for the FDA than does failing to approve a good drug (Type II error). In the former case at least some victims are identifiable and the New York Times writes stories about them and how they died because the FDA failed. In the latter case, when the FDA fails to approve a good drug, people die but the bodies are buried in an invisible graveyard.
So there are — in my view — plausible arguments for either maintaining or loosening FDA regulations. I am a strong advocate of empiricism in policy making: wherever possible, we should build experimental evaluations into our policies and regulations. So if only as a thought experiment — as it were — let’s consider how we could test the rival claims about the FDA.
Suppose we had two FDA regulatory regimes: Regulation-As-Usual (RAU) and Regulation-Lite (RL). When a manufacturer submitted a drug for approval, the FDA administrator would flip a coin to determine whether the drug would be evaluated under RAU or RL. We proceed this way for (say) the next 20 years. Then we look to see: has RL increased the rate of approval of drugs? Did the time from submission to approval decrease? Were more people killed or harmed by RL-approved drugs than by RAU-approved drugs? Did RL-approved drugs produce a greater number of QALYs compared to RAU-approved drugs? These would be difficult quantities to measure and the study would be expensive. But surely the stakes are sufficiently high to justify the cost of the study, if we thought it had a decent chance of succeeding.
What could go wrong? A lot. For example, suppose a manufacturer had a drug that, it believed, had only a marginal chance of getting approved under RAU, but that might get through RL. Then it might submit the drug, but withdraw it if the coin came up RAU. Then despite the randomization, there would be a self-selection bias. Could we design a procedure to defeat this? I’m sure readers can come up with other potential problems.
Would the public be willing to tolerate this? Would they shut the experiment down after the first death of a patient from a side effect of an RL-approved drug? Would the manufacturers, having gotten the door open part way to 50% RL, be willing to wait for two decades to see the results, or would they use Congress to force the door open the rest of the way?
Regardless, we should be looking for every opportunity to transform policy making into an empirically-driven enterprise. Experimental data are more valuable than our theories. Let’s consider evaluating the FDA the same way the FDA evaluates drugs.
