• More diagnoses are not always a good thing

    Longtime readers of the blog know that I can easily go off on a rant about how survival rates are not the same as mortality rates. Improvements in one do not necessarily mean improvements in the other. Here’s my now-classic example:

    Let’s say there’s a new cancer of the thumb killing people.  From the time the first cancer cell appears, you have nine years to live, with chemo.  From the time you can feel a lump, you have four years to live, with chemo.  Let’s say we have no way to detect the disease until you feel a lump.  The five year survival rate for this cancer is about 0, because within five years of detection, everyone dies, even on therapy.

    Now I invent a new scanner that can detect thumb cancer when only one cell is there.  Because it’s the United States, we invest heavily in those scanners.  Early detection is everything, right?  We have protests and lawsuits and now everyone is getting scanned like crazy.  Not only that, but people are getting chemo earlier and earlier for the cancer.  Sure, the side effects are terrible, but we want to live.

    We made no improvements to the treatment.  Everyone is still dying four years after they feel the lump.  But since we are making the diagnosis five years earlier, our five year survival rate is now approaching 100%!  Everyone is living nine years with the disease.  Meanwhile, in England, they say that the scanner doesn’t extend life and won’t pay for it.  Rationing!  That’s why their five year survival rate is still 0%.

    You have to understand that not all cancer is fatal. Many cases might never be detected and might never cause death. But if we screen like crazy, we will pick up those cases, too. Those cancers will be treated, and that can cause both mental and physical sequelae. In other words, we may be causing harm without any actual benefit in terms of saving lives.

    There’s a new paper out in BMJ that directly hits this issue. It’s titled, “Preventing overdiagnosis: how to stop harming the healthy“:

    Medicine’s much hailed ability to help the sick is fast being challenged by its propensity to harm the healthy. A burgeoning scientific literature is fuelling public concerns that too many people are being overdosed, overtreated, and overdiagnosed. Screening programmes are detecting early cancers that will never cause symptoms or death, sensitive diagnostic technologies identify “abnormalities” so tiny they will remain benign, while widening disease definitions mean people at ever lower risks receive permanent medical labels and lifelong treatments that will fail to benefit many of them. With estimates that more than $200bn (£128bn; €160bn) may be wasted on unnecessary treatment every year in the United States, the cumulative burden from overdiagnosis poses a significant threat to human health.

    Narrowly definedoverdiagnosis occurs when people without symptoms are diagnosed with a disease that ultimately will not cause them to experience symptoms or early death. More broadly defined, overdiagnosis refers to the related problems of overmedicalisation and subsequent overtreatment, diagnosis creep, shifting thresholds, and disease mongering, all processes helping to reclassify healthy people with mild problems or at low risk as sick.

    The downsides of overdiagnosis include the negative effects of unnecessary labelling, the harms of unneeded tests and therapies, and the opportunity cost of wasted resources that could be better used to treat or prevent genuine illness. The challenge is to articulate the nature and extent of the problem more widely, identify the patterns and drivers, and develop a suite of responses from the clinical to the cultural.

    Here’s the money shot:

    Let me orient you. The blue line represents the number of diagnoses of each type of cancer per 100,000 people. The red line represents the mortality rate for the population for that type of cancer, with deaths per 100,000 people. Data are presented for 1975-2005.

    What you’re seeing is a significant increase in diagnoses of these diseases over the three decades. In some cases, the rate of diagnosis has tripled. Now, unless you think something has changed in the world to make cancer way more common, we can likely attribute this increase in the number of cases to the way we practice medicine; it’s due to increased screening and improved detection of disease. But look at the mortality rates. They’re pretty darn stable. This means that although we’re picking up way more disease, we’re not actually preventing a corresponding amount of death.

    The simplest explanation for this is that many of the cases of cancer we are detecting might not really benefit from early, let alone any, diagnosis. I’m not denying that there is likely some number of people who benefited from early detection and treatment. But, looking at the above, that number is likely small. I can almost guarantee that all of those diagnosed, however, suffered from life-changing anxiety and fear, not to mention surgery, chemo, and/or radiation.

    I won’t even mention the cost.

    We have got to do a better job here. We’ve convinced ourselves that when it comes to screening, it’s got to be more, more, more. It’s getting harder to justify that the benefits outweigh the harms.

    Go read the whole article. It’s ungated.


    • This collection of charts is just amazing — almost unbelievable. You have been beating this drum for a while, to very great effect.

      I have a question though — is there an example of a type of cancer (or a type-within-a-type) where the “early detection saves lives” has been an unqualified success? If so, is there anything about the tech or medicine that we can learn from?

    • I made a similar comment on Don Taylor’s post. First I don’t see how the data presented demonstrate a claim of over diagnosis as opposed to improved outcome based on early diagnosis. I take the point of the “thumb cancer” and I think that there is a real issue there, but the data in the bmj paper don’t show it. Second, I think the graphs are poor as information. The y-axis makes the diagnosis delta seem large when in some cases, like breast cancer, it is about a 50% increase. Meanwhile it makes the mortality look the same when there may be a significant difference- not that any stats are shown. I really do sympathize with the idea and the goal, but weak arguments are worse than no argument….

      • What improved outcomes?

        Moreover, there is a doubling or tripling of diagnoses. The mortality rates aren’t being cut by half in all of these. They’re totally stable. You’d see it if it occurred. Many cancers have barely budged in years.

        And if you require numbers that you don’t see in the charts, you could always go to the trouble of clicking through. I’ll save you a step. Here: http://jnci.oxfordjournals.org/content/102/9/605.full

        • Aaron,
          thanks for the link – I read through the BMJ article but not the one you cite. I think there is a stronger case made in that article that over diagnosis in oncology is an issue. The BMJ article conflates it with a number of other issues, which in my opinion reduces the impact. And I still think those graphs are not really very illuminating. Several issues have percolated in my brain as I thought about this more (instead of working :)). One is captured by this quote from the article you linked

          “The conundrum in overdiagnosis is that clinicians can never know who is overdiagnosed at the time of cancer diagnosis. Instead, overdiagnosis can only be identified in an individual if that individual 1) is never treated and 2) goes on to die from some other cause. Because clinicians do not know which patients have been overdiagnosed at the time of diagnosis, we tend to treat all of them. Thus, overdiagnosis contributes to the problem of escalating health-care costs. But even where there no money involved, overdiagnosis would be a major concern: Although such patients cannot benefit from unnecessary treatment, they can be harmed.”

          There is a real practical problem here. You do not have as far as I can tell a way to distinguish the “real” cases from the “over diagnosed” cases, because you don’t have a good way to distinguish fast tumors from slow or non-progressing tumors when you detect them early. This points in the direction of a solution to the problem (find more specific markers).

          I followed up to the Malmo screening paper, (ref 14 from the link) and I think I found the numbers that speak to the issue. I will set them out here and you can give your opinion. I apologize in advance if this is a well worn topic that others have picked to death!
          So there were 42k women in this town who were divided up into two groups (early screening and control) as I understand it. The two groups were then followed for 15yrs after the experiment ended. About 19k of the women died during that time, and about 2500 were diagnosed with breast cancer (about 10% more in the screened group). So the claim (oversimplified) is that there was about 10% over diagnosis. However, the big issue it seems to me is that of the women who died during the follow up period, the women in the screening group who were diagnosed with breast cancer had a roughly 25% better chance of surviving the breast cancer to die of something else (everyone dies!). The raw numbers are 212/584 (died of bc/diagnosed with bc and died) in the screened group compared to 272/588 (died of bc/diagnosed with bc and died).
          Now please correct me if I’m misunderstanding (seriously, I’m not an epidemiologist or an oncologist!) but this seems to me that if you were in the screened group and you were diagnosed with breast cancer, you had a 10% chance that you were treated for a disease that wouldn’t have killed you, but if you died you were 25% less likely to have died of breast cancer. This implies strongly that the early screening saved some women from dying of breast cancer.
          If I have this right, then the question is whether the risk of over treatment is a price worth paying to save a significant number of lives. That is a big question, and not a simple one.
          This is not to say that all screening tests (or even many or most) are good. But the question is not simple, nor is the answer clear. My vote would be screen early and screen carefully (i.e. work to improve the diagnostic accuracy) and treat as cautiously as possible. But that is a difficult set of parameters to optimize.
          I’ve probably rambled on enough. Interesting topic.

    • Pap smears for cervical cancer is best success story, and probably colonoscopies. These both involve detecting and removing pre-cancerous lesions that may eventually develop into cancer. In contrast a lot of other screening tools are detecting cancer.

    • I completely agree that more diagnoses are not necessarily a good thing, and that earlier detection skews survival statistics. But mortality rates are problematic as well, because we don’t have a proper denominator. Without knowing the true incidence of these cancers (a measure made more difficult by changes in testing patterns and the stage when it is diagnosed) the mortality rate does not tell us very much at all.

      • Incidence should be related to mortality. If incidence drastically rises (or falls for that matter), but mortality does not change, that tells us quite a bit.


        • Well sure, but the problem is defining incidence. If we are diagnosing a lot more cancers because we’re doing more testing, but the actual underlying incidence has not changed, we no longer have a consistent incidence metric.

          In this case how can we tell whether incidence is increasing because of more testing or people actually being more likely to get cancer? And without knowing that, is the mortality rate flat because the true incidence didn’t change, or because we got better at treating cancer?

    • I think this is related to AB’s comment, but what if there was a treatment for thumb cancer that extends life by exactly 3 months for everyone. The numbers would jiggle a bit as the treatment was phased in, but once it was 100% implemented, the mortality rate would be same as before the treatment was introduced, even though there has been a real gain. In general, this will be true of any treatment that isn’t a cure, no? Is there any reliable way of assessing those gains?

      • In fact, clinical trials of many if not most cancer medications show improvements of a few weeks or months in measures like survival time or time to progression.

        I really, really question, though, whether adding a few weeks to someone’s life is actually a benefit. Especially if the quality of life in those weeks is low due to side effects or persistent pain. I have had several family members die from cancer. I don’t believe that it mattered to my Dad whether he lived to 78 or to 78.25.

        This clearly is a case where personal values will lead different people to different conclusion. However, I don’t think oncologists do a very good job of explaining the very modest gains that are available from many treatments. Perhaps more people would forgo the costs and discomforts if they knew.

    • When incidence of a disease goes up per capita with a flat line for morbidity or mortality, thats de facto evidence of TOO MANY DOCTORS.

      We’ve been fed a lie from the federal govt that the US doesnt have enough doctors, but thats not true. Some rural areas are certainly understaffed, but as a whole there is an excess of doctors in this country. When you have extra doctors they run around diagnosing every trivial ailment without adding any true benefit in terms of survival or morbidity.

    • Since I wasted a bunch of time on a comment that you deleted because it was too long (never seen the “delete long comments” rule before). I will summarize. If you look at the mammography study referenced in the paper linked above, you find that there was a 10% “overdiagnosis” and a much better outcome for those diagnosed in the screened population than in the unscreened (about 25% fewer deaths from breast cancer during the followup period). So this is the tradeoff you are proposing (I put all the numbers and caveats in my original comment, I’m not doing that again!). Whether this is a good idea or not is, shall we say, debatable.

    • Would the increased testing to everyone
      1) reduce disparities in use/access to testing and in outcomes?
      2) be worth it to consumers?
      3) result in more competition and eventually decreased costs per test?

    • What would be even more helpful here would be research and the resulting statistics that cull out age and general health at the time of initial diagnosis. I strongly suspect that the “deaths” line is flat because it is being skewed by people who don’t survive cancer because they already are at the end of their lifespans.

      As a nurse in a small ICU in a typical medium sized city, the main cancer patient’s I see are either very elderly or have had serious chronic diseases. These groups don’t tolerate chemo or radiation well at all, and as a result many of them are in the hospital for sepsis, bleeding, organ failure or cardiovascular complications that result in death–not the cancer they were treated for. But only after weeks of aggressive and expensive antibiotic treatment, ventilator support, hemodialysis, L-TAC care… you get the picture.

      I often wonder why we would tell an 82 year old with lung or pancreatic cancer to go through anything other than conservative or palliative treatment, seeing as these cancers are pretty much 100% terminal before 5 years, and that anything else will pretty much kill the patient and ultimately cost hundreds of thousands of healthcare dollars that would be better spent on younger patients who have a chance of tolerating these treatments.

    • I’m neither a statistician nor a medical professional, so my reading of the above charts is likely naive.

      But don’t they demonstrate that, despite overdiagnosis, *most* of our medical and surgical interventions don’t seem to be *killing* people?

      (I say “most” because there’s that odd coincidental “bump” in the case of prostate cancer, which makes it appear as though the increase in diagnosis precipitated an increase in the number of deaths.)