• Oh, is that what you think evidence-based medicine is?

    In the most recent EconTalk episode, Russ Roberts interviewed Eric Topol about how technology (mostly genetics) could revolutionize medical care. This post should not be interpreted as a critique of the entire discussion. It’s largely fine. But something struck me as odd.

    Early in the hour, Russ defined “evidence-based medicine” as doctors providing therapies based on tests. The test results are the evidence and the therapies are the medicine. Do many people think that’s what “evidence-based medicine” means? It has never occurred to me to think of it that way. In fact, doing so does a disservice to evidence.

    I always think of “evidence-based medicine” as health care decisions informed by credible evidence of benefits and harms. In fact, evidence doesn’t determine what care is delivered. It’s always a judgement call whether the benefits outweigh the harms and/or the cost (yes, somebody, somewhere is making a cost-based decision even if it is implicit and by default). Ideally, decisions are both informed by evidence and consistent with patient values.

    Later in the program the discussion turns to treatment of ear infections. The thought experiment is that there is a novel gadget that permits the diagnosis of ear infections without a human and with high accuracy — a very good test. Topol then suggests that an antibiotic might be prescribed automatically if the gadget detects infections. Is that evidence based medicine because the (accurate) test detects an infection?

    No! Ear infections almost never require antibiotics. They certainly should not be prescribed routinely. Knowing that is what evidence-based medicine is about. It’s not knowing the results of a test.


    • FWIW, the link above is based on the results of a relatively old Cochrane review. A newer Cochrane review is available and seems to reach modestly more positive conclusions on antiobitics, particularly for children under 2.

      See http://www.ncbi.nlm.nih.gov/pubmed/23440776

      • Thanks. It still doesn’t support routine use in all cases.

      • Topol is the classic case of an idiot who thinks that he is just as good at primary care as he is at his own subspecialty (cardiology).

        Topol hasnt looked inside the ear of a screaming toddler since he was in medical school 40 years ago.

        Cardiologists dont know jack about ear infections.

    • Austin, this is a great example of how we, as health policy analysts and researchers, need to be more careful about the terminology we use and how we describe things and more aware of the fact that the “average” person is not as immersed in the weeds of these issues as we are.

      Like you, I’m pretty sure of what we mean by “evidence-based medicine.” That other people aren’t, especially those who may be influential, tells me that we need to be more careful in how we present issues.

      The ear infection/antibiotic issue reflects a problem I often run across. Conventional wisdom is a powerful thing and not easily refuted, regardless of facts and evidence. For Topol, it’s conventional wisdom that ear infections require antibiotics. That this isn’t true is unknown to him (and many others) or maybe he just doesn’t care.

      I run across this problem all the time in discussing health policy. Conventional wisdom is often wrong (e.g., private insurance always controls costs better than public insurance), but it’s very difficult to refute and even more difficult to be heard after you attempt to refute it. Once you’ve skewered the sacred cow, even if you do it an evidence-based way, people often just tune out. It just can’t be true . . .

    • Russ is a fine economist, but he is not well informed on health care issues AFAICT. Evidence based medicine starts well ahead of the testing process. You need to know when to order the tests and which ones. A discussion of the PSA would have been better.


      • They discussed the PSA. What came up was more correlated with the evidence, but still problematic. For example, Topol said the problem was that the PSA can suggest cancer when there is none. I guess that’s true. But what’s really problematic about prostate cancer is that it can exist and still not be a problem because it is so slowly growing. Most men die with it and not of it.

        • He also spoke of NNS for mammograms and NNT for statins. Critically.

          While I will give Topol credit for citing costs, the genomic tests he advocates for, which will drive necessary (or inhibit unnecessary) therapies, will also expend vast resources in their broad application to the population at large. The very NNS and T’s he criticizes will likely be operative in the genomic, for profit world he envisions.


    • I guess at one level I can understand why Russ was confused on the terminology. But then, what exactly does Russ think is the alternative to “evidence-based medicine?” Does he honestly think there are clinics where the doctor just walks in and says “I honestly don’t know whether theres anything wrong with you so take these pills.”

    • I’m often surprised when apparently knowledgeable physicians misuse the term “EBM.” EBM is about using, well, EVIDENCE from the published literature and applying it to your patient. While we can argue about HOW that evidence is applied, or which evidence is right for a particular circumstance, there has to be a reference to the evidence.

    • I don’t know whether you’re over thinking this. Let’s say Topol’s example was more explicit, if the test found a bacterial infection, it would prescribe an antibiotic. You can then apply your knowledge and be confident that this will happen only very rarely. Would you then be happier about the example?

      • No. Most bacterial infections of the ear, nose, throat don’t require an antibiotic. Automatic use, even in that case, is overuse. Watchful waiting is more appropriate.

    • Physicians are no less biased than social scientists. Some physicians are aggressive, some not so much. For example, for some ophthalmologists, surgery is the first choice, while for other ophthalmologists, it’s the last choice. Why is that? Because the former is greedy? Hardly. It’s bias, bias resulting from many factors, personality being one. No different from the social sciences. For example, there are economists with a libertarian slant who will perform complicated gymnastics to explain why the libertarian way is the only way. The comments about the cardiologist are instructive. To the cardiologist, treating an infection aggressively is a must because it can easily spread and result in death. That an infection in the ear doesn’t have the same potential may be accepted to the ENT and pediatrician but not the cardiologist. In other words, specialists have bias. I’ve spent the past 15 years helping organize endoscopy centers, and now most GI’s have a thriving out-patient practice scoping most every patient after reaching age 50. So thriving in fact that we sometimes have to put the GIs on the clock. To slow them down. Standardized good practices and protocols are great in theory, but work only if we can produce standardized physicians (i.e., human beings).

    • I haven’t heard the episode yet. But instead of criticizing Topol as not getting it, I think we have to acknowledge that if more therapies were based on objective test results, that would actually be a real step forward towards more “true” EBM.

      Consider the treatment of depression in general practice. It is still overwhelmingly common for GPs (and even psychiatrists) to start someone on antidepressants after just a few self-reported symptoms and a couple of questions from the doc about mood. Administering a Beck Depression Inventory, and factoring the score into the decision, would be better than that.

    • It’s evidence all the way down. There are multiple evidentiary components to doing EBM well.

      -Ontologic evidence. Does the condition exist? Prior to Virchow and Pasteur, disease was due to imbalance of humors, sin, or divine predestination. As modern medicine bootstrapped itself out of the muck, it found evidence for congenital, developmental, infectious, degenerative, neoplastic, etc bases for disease. We’re not done- do chronic fatigue syndrome, fibromyalgia, autism, etc actually exist as definable pathologies? Much of CAM seems to be ontologically challenged.

      -Evidence in diagnosis. Using Bayes, the chief complaint is usually enough to establish a prior probabililty, and additional history, exam and testing serve as new evidence to adjust the prior up or down. (Posterior becomes the new prior as you assess whether more evidence is needed to hit your threshold for treatment) This gets back to your skin in the game question. If the diagnosis is X% probable based on history and exam, and I am paying out of pocket, I want evidence presented to me that additional imaging/testing is going to significantly alter the posterior probability. (For example, if you are a healthy 70 year old now six weeks post distal radius fracture, painfree in the cast and with no tenderness over the fracture, an x-ray is of no value in altering the prior probability for the hypothesis of healed fracture)

      -Evidence in treatment. This is where EBM discussions are usually focused. What is the level of evidence that supports the use of a specific treatment? Has there been sufficient filtering of common sources of bias? Where in the epistemic hierarchy (from expert opinion up to randomized, placebo controlled trials) does the evidence sit? Are there patient input factors or externalities that are known to degrade outcomes? Has shared decision making been found to be effective for the diagnosed condition?

      All evidence, all the time.

    • you underestimate the hubris of academic medicine

    • Austin, you may have listened too quickly to the podcast. Russ and Topol clearly understand the sense in which you mean evidence-based medicine, i.e. as prescriptions made based on the informed judgement of the best available knowledge at the time.

      Their point is that “evidence-based” sounds more scientific than it is, and that often it amounts to “eminence-based”, the great line Topol made in the podcast: doctors, even the smartest and most famous, need to be more humble about what it is they really know, because even the most well-designed studies are very often (usually?) fallible.

      Re: the ear infections I think he was just saying that a personalized computer, which knew exactly the genotype/biome of a particular patient might correctly note that an antibiotic is the best course in a particular situation — regardless of what the “evidence” from large populations that may have nothing to do with this particular patient.

      • Topol’s use of eminence based was not novel.

        As for the rest, I cannot know for sure what was in the minds of Russ and Topol. Of course I could have misinterpreted. So could have many others. Thus, it deserves clarification. I stand by my response.

    • Sort of repeating W. Dale – in Behavioral Health the meaning is clear, and not exactly what it seems to be in med/surg health. But there, it’s Evidence Based Practice, or Evidence Based Treatment and means using treatments that have been shown to be effective. (Peer reviewed journal, etc.) There’s been a big push in the last few years (from payors) to demand this, lists of approved treatments for which diagnoses have been published, etc. etc.Check here for more;