I stand by this with respect to screening mammograms:
If you’re not going to be swayed at all by a randomized controlled trial of 90,000 women with 25 year follow up, excellent compliance, and damn good methods, it might be time to consider that there’s really no study at all that will make you change your mind.
I think it can be slightly amended to fit a lot of hotly debated scientific issues. I hear all the time from people asking for “just one more trial” to be convinced, even when many already exist.
I thought of that when I saw this study in JAMA:
Importance Many patients and physicians assume that the safety and effectiveness of newly approved therapeutic agents is well understood; however, the strength of the clinical trial evidence supporting approval decisions by the US Food and Drug Administration (FDA) has not been evaluated.
Objectives To characterize pivotal efficacy trials (clinical trials that serve as the basis of FDA approval) for newly approved novel therapeutic agents.
Design and Setting Cross-sectional analysis using publicly available FDA documents for all novel therapeutic agents approved between 2005 and 2012.
Main Outcomes and Measures Pivotal efficacy trials were classified according to the following design features: randomization, blinding, comparator, and trial end point. Surrogate outcomes were defined as any end point using a biomarker expected to predict clinical benefit. The number of patients, trial duration, and trial completion rates were also determined.
How many studies are usually needed to approve a drug? I was shocked to find out. The FDA approved 188 new agents for 206 indications between 2005 and 2012. The median number of pivotal trials per indication was 2. Yes, two. Most were randomized, double-blinded, placebo controlled trials. The median number of patients studied per indication was 760.
More than one third of indications were approved with just one pivotal trial supporting them. Almost half of pivotal trials used surrogate end points as the exclusive basis for approval. Most trials were of less than six months duration.
Think about that the next time you dismiss a large RCT, or a number of RCTs, and ask for “just one more”. Ask yourself why you need so little for some things, and so much for others.
