In 1992, as an Applied and Engineering Physics major at Cornell, I had barely given any consideration to health care or the US health system. If you had asked me at the time what proportion of health care delivered is grounded in evidence, I am sure I’d have said “most” or “nearly all” or possibly even “all”. I would bet the vast majority of Americans think that today. If so, they’re wrong, just as I would have been in 1992.
Though some areas of medicine had experienced significant, evidence-driven strides by 1992 (e.g., the treatment of cardiovascular disease), by that year the idea of systematically applying evidence in practice was still a relatively novel one. In a JAMA article that year Guyatt et al. wrote,
A new paradigm for medical practice is emerging. Evidence-based medicine de-emphasizes intuition, unsystematic clinical experience, and pathophysiologic rationale as sufficient grounds for clinical decision making and stresses the examination of evidence from clinical research. […]
We call the new paradigm “evidence-based medicine.”
Twenty years on, it’s astonishing how little progress we’ve made. Could we have done better? Should we have? Even our relative successes at improving the efficiency and effectiveness of US medical practice through research leaves a lot of room for improvement.
Consider, for example, autologous bone marrow transplantation (ABMT) for treating breast cancer. In a scholarly article as riveting as they get, Welch and Mogielnicki (BMJ, 2002) characterize it as
a story of young women dying from aggressive disease, well meaning physicians trying to be equally aggressive in treating it, and lawyers arguing that insurers should pay the bill. It is also a story of professional interests, weak research, financial gain, politics, and fraud.
Used to “rescue” breast cancer patients after they had been treated with super-lethal doses of chemotherapy or radiation, by the late 1980s, ABMT was being touted by some as an effective and important new breast cancer therapy. However, researchers’ enthusiastic public comments to the media were not supported by the evidence since the studies by that time were conducted without control groups.
The media reports had major impact on patients, however. Armed with no compelling evidence but a great deal of hope they inspired, patients demanded that insurers cover ABMT for breast cancer.
[In 1990] Pamela Pirozzi had been advised that her “best chance of surviving more than a year” was a transplant, but her insurance company had refused coverage, stating the procedure was still “experimental.” Armed with a “list of insurance companies in other states that, when challenged, have agreed to pay for the procedure,” the Pirozzis sued.
A federal judge ruled in her favour: “To require that the plaintiff or other plan members wait until somebody chooses to present statistical proof . . . that would satisfy all the experts means that plan members would be doomed to receive medical procedures that are not state of the art.” The same month another federal judge ordered a Massachusetts insurer to pay for a Boston woman to receive a transplant in North Carolina. […]
By 1990 one group of insurers had been sued by over a dozen patients and consumer advocacy groups to cover the treatment.
One plaintiff won the largest award ($8.1M) “ever levied against an insurance company for refusing to provide health coverage benefits.” In 1994, under pressure from Congress, the US Office of Personnel Management ordered all plans offering health benefits to federal employees to include ABMT for breast cancer among the services covered.
In 1992 a JAMA article examined the cost effectiveness of ABMT for breast cancer even though the effectiveness had not yet been established. The authors reported the cost as $115,800 per life year, noting the relatively high price tag may be too much for society. Still, the fact that costs were being raised before effectiveness was established gave the impression that coverage was more a matter of how to pay for the therapy, not whether it should be offered at all.
Finally, in 1995, WR Bezwoda and colleagues reported the results of the first randomized trial. They were impressive. Over half the women receiving ABMT had no subsequent evidence of tumor, compared to only 4% in the control group. Survival time was double (90 weeks vs 45) for those in the treatment group. However, four years later, only Bezwoda’s group could reproduce these results. Four other clinical trials contradicted them. A review team identified problems with Bezwoda’s protocol and was denied access to control patients. “The vice president of the society made the logical inference to the Guardian (London) ‘You could conclude that they might not exist.'”
By 2000 and in light of the clinical trials showing it to be no better than alternatives, ABMT for breast cancer was regarded as ineffective and worthy of abandonment. Welch and Mogielnicki conclude,
For over 10 years desperately ill women had sought bone marrow transplantation as their best chance for survival. Many physicians encouraged this judgment. Fearing bad publicity and lawsuits insurers reluctantly agreed to pay the considerable charges. A strong presumption of benefit and equally strong financial interests impeded progress towards finding an answer.
The obvious lesson from these events was articulated in the New York Times by two of the treatment’s most visible critics. “As a society we have to accept that rigorous evaluation of a new treatment is essential . . . Skipping this step may seem like a compassionate act, but it can have devastating consequences.”
There are other lessons to be drawn from the ABMT experience. Welch and Mogielnicki list them in their conclusion:
- “It is premature to raise the question of cost effectiveness when effectiveness is unknown.”
- “Establishing what is ‘experimental’ is an important role for government.”
- “Public officials should not mandate coverage in the absence of clear data.”
- “The news media watchdog role should be extended to health care.”
To these I would add an additional lesson. As tragic as the ABMT/breast cancer tale is, it’s actually a relative success compared with other harmful aspects of health care delivery that continue, despite the fact that we know them to be harmful. Some of these are very simple, like lack of proper hand washing, or the failure to administer aspirin and a beta-blocker to AMI patients, or over-provision of tests that lead to unnecessary and dangerous procedures. In other cases, we are doing and paying for things for which we do not know the benefits. Some of them are likely harmful and many are costly, like providing angioplasty for heart attack patients beyond a day of onset (more here).
In the case of ABMT for breast cancer, at least we did the science and, eventually, science worked. It worked imperfectly and too slowly, and for that many were harmed. Yet, ultimately, science prevailed. That’s more than we can say for many other therapies. Evidence-based medicine may be 20 years old, but it is still in its infancy. I would have been shocked to know this in 1992, and, quite frankly, I’m shocked to know it today. For what we spend on health care and for what we expect it to do, we ought to be employing every weapon at our disposal to do it better. That we are not is one of the great tragedies of our time.
AF
