• Half of medical treatments of unknown effectiveness

    The British Medical Journal posted on their website, Clinical Evidence, the results of an analysis of randomized controlled trials focusing on harms and benefits of 3,000 medical treatments. The effectiveness of each treatment was rated based on six criteria: (a) beneficial, (b) likely to be beneficial, (c) trade-off between benefits and harms, (d) unlikely to be beneficial, (e) likely to be ineffective or harmful, and (f) unknown effectiveness. The results were striking. Only about a third of the treatments were shown to be beneficial (11%) or likely to be beneficial (23%). Another 7% were rated as trade-offs between benefits and harms, with 6% rated unlikely to be beneficial and another 3% rated likely to be ineffective or harmful. The authors at Clinical Evidence rated the remaining 50% of medical treatments as being of unknown effectiveness. The challenge that evidence ratings like these pose for clinicians is not new.

    That’s the opening paragraph of the introductory paper to the shared decision making-focused February supplement of Medical Care Research and Review (MCRR). Here’s the Clinical Evidence findings in chart form:

    clinical evidence


    In their MCRR intro, Quentin Smith et al. go on to summarize the other papers in the supplement, so I don’t have to:

    Han offers an overview of conceptual, methodological, and ethical problems in communicating uncertainty in clinical evidence, arguing that the key to advancing efforts to manage uncertainty constructively involves greater conceptual clarity and use of consistent representational methods that can be integrated into patient-centered interventions in ways that help patients cope with uncertainties.

    Zikmund-Fisher examines the need for improved precision in patient risk communication, presenting a taxonomy of seven distinct risk concepts and discussing how patient acceptance of a health risk message varies across the seven conceptual levels.

    Llewellyn-Thomas and Crump explore the need for values clarification and preference elicitation (VC/PE) as integral components of the full decision support process, including positing a wider catalogue of approaches to VC/PE.

    Fraenkel focuses on patient values and strategies for bringing them into the decision-making process, including discussions of current methods (e.g., conjoint analysis [CA], best worst scaling [BWS]) for doing so and implications for point-of-care decision support.

    Epstein and Gramling examine issues around clinical decision making when evidence is unclear or inadequate and other factors, such as clinical complexity and nonclinical influences (e.g., family input, religious beliefs), create further challenges that may be amenable to resolution through varied strategies, including health systems reorientation.

    Politi, Lewis, and Frosch offer a discussion of applications of SDM that extend beyond situations in which equipoise between treatment options and/ or clinical outcomes exist to suggest applications of SDM in situations in which evidence is low or conflicting, and ending with recommendations regarding priority areas for future research, including studies designed to assess effective management of uncertainty in clinical settings; a topic largely unexplored through formal research.

    Braddock’s paper looks at SDM and its relevancy to all clinical decision making, emphasizing the potential for using SDM to empower patients to become more fully engaged in understanding the strength of the evidence and what it means in terms of the choices he or she confronts, drawing on the concept of gist knowledge and the role that it can play in the decision-making process.

    McCullough applies principles integral to the medical ethics model to decision making under conditions of clinical uncertainty, emphasizing specific virtues implicit in the roles of the decision makers (i.e., integrity on the part of the clinical professional and prudence on the part of the patient) as these virtues impact decision dynamics and outcomes.


    • Grappling with uncertainty is the central task of being a mortal human being, isn’t it? We keep looking to medicine for certainty, for guaranteed protection against pain, illness, and death. We imagine that screening for disease will somehow keep it at bay.

      Part of this fallacy has to do with the way we use language. We continue to talk about benefits of this or that test or procedure as if they were a sure thing, whereas harms are always conveyed as “risks.” The fact is, there is a risk of benefit, and a risk of harm, and most of the time, neither is certain for a given patient.

      BTW, this is a great graphic. I would like to use it when I give talks. How shall I attribute credit?

    • Is this a study of British medicine or American medicine?

    • ” there is a risk of benefit, and a risk of harm, and most of the time, neither is certain for a given patient.”

      Better: There is a chance of benefit, and a chance of harm, and there is no certainty for a given patient. See: Peanut allergy.”

      Requiring Number Needed to Treat for all treatments would be helpful.

    • This doesnt even address costs. Would be nice to know if drugs that are thought to work cost more or if there is no correlation (my bet).


    • This makes it look like Robin Hanson is correct, that their is very little benefit from healthcare beyond vaccinations,, antibiotics, trauma care and few other things most of which are not expensive.

    • This is an analysis of RCTs, not a survey of all medical treatments. It seems to me that a treatment for which an RCT was conducted has already a level of uncertainty. It would be very interesting to analyze how “tried-and-true” treatments would break out by effectiveness.