A 1998 paper by Joe Newhouse and Mark McClellan in the Annual Review of Public Health is insightful on the limitations of randomized controlled trials (RCTs) of clinical interventions. They note that a trial can only occur in the
window of time where there is enough belief that a treatment is efficacious so that it is considered ethical to randomize patients to the treatment group but not sufficient belief in the efficacy of the treatment that it would be considered unethical to withhold the treatment.
Even when a trial can be conducted, its results may lack generality because it is conducted in a setting (major teaching hospital, say) that differs from the norm. That is,
the trial may demonstrate that a procedure is efficacious (i.e. obtains desired results under optimal conditions), but it will not necessarily show that it is effective (obtains desired results under typical or standard conditions). Or, somewhat related to this point, in the time since the trial was conducted physicians may have become better at performing a procedure, such that the results of the trial are no longer relevant to current practice. (Bold mine.)
As if that weren’t enough, sometimes the population included in a trial differs from that which will (or is) actually treated generally. Women, children, the elderly, and individuals with comorbidities are among the sub-populations historically excluded from certain types of trials. That is,
the results of the trial may have internal validity (comparisons between the treatment and control groups are unbiased for the population being studied) but not external validity (results do not necessarily apply to other populations). (Bold mine.)
For all these reasons, well-designed observational studies can enhance our understanding of treatment outcomes by examining results over a broader setting and population than might be available in a RCT. Moreover, observational studies are less expensive, can be conducted more quickly, and are applicable in cases for which an RCT would be unethical or impractical, due to problems of recruitment, for example. Thus, as Newhouse and McClellan put it, “[t]he results of a well-designed observational study are useful even if the results of a clinical trial are available.”
