• Maybe we can have a better press corps!

    I was all riled up to get angry over this new paper published in Health Affairs:

    The United States spends more on health care than other developed countries, but some argue that US patients do not derive sufficient benefit from this extra spending. We studied whether higher US cancer care costs, compared with those of ten European countries, were “worth it” by looking at the survival differences for cancer patients in these countries compared to the relative costs of cancer care. We found that US cancer patients experienced greater survival gains than their European counterparts; even after considering higher US costs, this investment generated $598 billion of additional value for US patients who were diagnosed with cancer between 1983 and 1999. The value of that additional survival gain was highest for prostate cancer patients ($627 billion) and breast cancer patients ($173 billion). These findings do not appear to have been driven solely by earlier diagnosis. Our study suggests that the higher-cost US system of cancer care delivery may be worth it, although further research is required to determine what specific tools or treatments are driving improved cancer survival in the United States.

    So much wrong here. First of all, it uses the old “survival rate”/”mortality rate” swap that I’ve discussed here and here and here and here and here. If you want to show that things are better, study the mortality rate, not the survival rate.

    More importantly, there is nothing in this study – nothing – that proves that spending more is what improves things, even if things are better. There’s no causality at all.

    So I braced for the worst in the media. But then Sharon Begley became my new hero:

    Cancer patients in the United States who were diagnosed from 1995 to 1999 lived an average 11.1 years after that, compared with 9.3 years for those in 10 countries in Europe, researchers led by health economist Tomas Philipson of the University of Chicago reported in an analysis published Monday in the journal Health Affairs.

    Those extra years came at a price. By 1999 (the last year the researchers analyzed), the United States was spending an average of $70,000 per cancer case (up 49 percent since 1983), compared with $44,000 in Europe (up 16 percent). Using standard figures for an extra year of life, the researchers concluded that the value of the U.S. survival gains outweighed the cost by an average $61,000 per case. The greater spending on cancer care in the United States, they conclude, is therefore “worth it.”

    Experts shown an advance copy of the paper by Reuters argued that the tricky statistics of cancer outcomes tripped up the authors.

    “This study is pure folly,” said biostatistician Dr. Don Berry of MD Anderson Cancer Center in Houston. “It’s completely misguided and it’s dangerous. Not only are the authors’ analyses flawed but their conclusions are also wrong.”

    Wait…  is that actual reporting, and not a rehashing of a press release? More, please!

    Philipson is a fellow at the conservative American Enterprise Institute and at the Manhattan Institute, served in the administration of President George W. Bush and was a healthcare adviser to Sen. John McCain’s 2008 presidential campaign.

    For the new analysis, Philipson and his colleagues analyzed the survival of cancer patients diagnosed from 1983 to 1999 with any of 13 common cancers, including breast, prostate, colorectal, and leukemias.

    Survival means how long a patient lived after being diagnosed. Philipson’s team focused in particular on survival gains; that is, how long did patients diagnosed in later years live compared with those diagnosed earlier in the period? Such gains, they argued, show what progress countries made in treating cancer.

    While that may seem straightforward, survival data is among the most problematic cancer statistics, Philipson’s team acknowledges. In particular, they are plagued by something called lead-time bias.

    If a tumor is diagnosed very early in its existence – if it has a long “lead time” – the patient may survive, say, two years if the tumor is very aggressive. If an identical tumor is found in that patient’s identical twin later, the twin will survive, say, six months. But the twins die at the same age. The first survived longer with cancer due to lead-time bias, but did not have a longer lifetime.

    Crediting medical care with “improving survival” is therefore misleading, cancer experts have long argued. Lead-time bias makes it seem patients live longer, but the only thing that is longer is the number of years they know they have cancer, not their lifespan.

    The authors of the “worth it?” study nevertheless base their analysis on survival data. They argue that because U.S. cancer mortality rates fell faster than those in Europe, the survival gains must be real and not an artifact of lead-time bias.

    Others call that approach fatally flawed. “Lead-time bias is an issue,” said MD Anderson’s Berry. “I can see no hint of logic in their statement that ‘lead-time bias did not confound our results.'”

    Be still my heart. Please, may I have some more reporting?

    The researchers acknowledge that it is not possible to conclude that improved survival comes from higher spending on cancer care. It might also come from more widespread cancer screening which, experts say, detects more and more “pseudo-disease” — that is, a tumor so nonaggressive it would never have threatened the person’s health or life. That alone would make the survival data look better.

    The Philipson paper was supported in part by Bristol-Myers Squibb Co, whose cancer drugs include Yervoy. A drug for advanced melanoma, it costs $120,000 for a full course of treatment. Clinical trials showed that Yervoy produces a near-miraculous cure for some patients, with a median increase in survival of 3.6 months.

    U.S. spending on cancer care has continued to increase, reaching $72 billion in 2004, the last year for which data is available.

    The new study did not examine the cost-effectiveness of that care. “In the last decade, spending in the U.S. has increased more than in Europe,” said Philipson. “I would be extremely surprised if the survival gains haven’t continued. But it is a much more open question whether that additional spending has been accompanied by an increase in longevity.”

    In the last decade, a number of very expensive cancer drugs were introduced into the United States, including Dendreon Corp’s Provenge for prostate cancer ($93,000 per treatment) and Bristol and Eli Lilly and Co’s Erbitux ($100,000 per year). Their analysis, say Philipson’s team, “does not imply that all treatments are cost-effective.”

    Worse, many are not medically effective. Last week, the American Society of Clinical Oncology released a list of five cancer tests and therapies that do not help patients live longer or suffer less — even apart from how much they cost.

    Please go read the whole thing. And then start a campaign to get Sharon Begley a raise and a promotion.

    @aaronecarroll

    UPDATE: Before commenting or emailing me, please go read this follow-up post.

    Share
    Comments closed
     
    • Great Reuters article.

      I’d also like to point out that even if there wasn’t lead-time bias and they used mortality rates, we couldn’t immediately conclude that the mortality differences are because of the extra costs of the healthcare system/treatment differences. For example, tobacco use is not associated with prostate cancer incidence but it is associated to higher prostate cancer mortality. Europeans are known to have higher tobacco use rates than Americans which could affect mortality differences.

      Ecologic studies are one of the weakest types of studies out there in terms of causal inference, I wish they didn’t get so much press…but at least this article was well done.

    • Will wonders never cease? What are the odds that Health Affairs will post a clarification?

    • Austin:

      Sharon is of that old-fashioned species known as a “beat writer.” Here’s why her story is of particular importance: she writes for Reuters (which you did not mention), a wire service whose stories are picked up by many newspapers and by many health care websites for their newsfeeds. (The rival Associated Press also has a very good beat writer.) It’s not one of the Newspapers of Record, but it’s a very important responsibility.

      Meanwhile, you might want to link your shout-out to the Association for Health Care Journalists website to encourage others who pursue this same kind of approach. Which, as you say, is a lot more than you get from your average blogger. (Of course, your ABOVE average blogger is another story….)

      Finally, while I’m sure Sharon appreciates your praise, I’ll bet she’s wincing at your headline. That’s “press corps” not “press corp.” (A singluar plural? Blame the French.) Not to be confused with a “press corpse,” which would be a reference to the advertising revenue model of most newspapers.

      Regards,
      Michael

    • As a former journalist, I now treat bylines like the general public does — I ignore them. :-)

      Mea culpa.

    • Just a great post, thanks for bringing this to everyone’s attention.

      The whole episode highlights a critical flaw in U. S. health care that causes costs to increase with little or no benefit.

    • Dear Aaron,

      I’ve read your posts on survival rates vs. mortality rates, and I confess I still don’t agree. Survival rates really do capture something important that are lost by mortality rates. Here’s a simple thought experiment: consider a cancer that is basically 100% fatal, like pancreatic cancer or glioblastoma; people who are diagnosed with these diseases typically die very quickly. Now suppose that it really was possible to treat this disease so that patients lived for an extra 5 years, and then died from the disease. Since everyone dies from the disease, the mortality rate remains 100%, and the statistic cannot indicate any clinical improvement. But I think we would all agree that adding a few more years of lifetime would be a big success, even if you still died from the disease at the end. And that’s precisely the insight that survival time captures.

      In one of your earlier posts on this issue, you wrote, “Survival rate, however, can be increased by preventing death, curing the disease, or making the diagnosis earlier.” Yes, those all improve survival time. But increasing the true expected lifetime also improves survival time! And my point is that *everyone* will agree that an increase in the true expected lifetime would be a clinical success.

      The extended quote from Begley misses a very important point: lead-time bias really only occurs when we conduct studies comparing a screening protocol versus non-screening. But if there is no difference in the screening protocol between the two groups that are being studied, then there’s no lead-time bias. Since in America we tend to screen for prostate and breast cancer more aggressively than in Europe, there is a potential for confounding by lead-time bias. I think we can both agree on that point. But for something like leukemia, which we don’t screen for, there’s not going to be a lead-time bias. Philipson et al. reported that they studied 13 different tumors. A close reading of their paper would treat each cancer type separately, and would consider how public health screening policies differ between the countries in the sample. But, absent any strong differences in the respective screening policies, it would be fair to treat any observed differences in survival times as real, and not artifacts of any lead-time bias.

      I don’t have access to the full Health Affairs article, just the abstract, so I currently can’t read through the Methods section of the paper. But it would be interesting to see how the authors dealt with the issue of potential lead-time bias. Until we know that, it’s hard to pass judgement on the paper.

      • You are correct that if we hold the method of diagnosis exactly the same, and then increase survival, we’ve likely prevented death. But that’s almost never the case. We compare ourselves (where we do way more screening) to other countries that do less and then congratulate ourselves on increased survival rates. That’s where the studies usually go wrong.

        See, you have to create a hypothetical to show me I could be mistaken. But that hypothetical is not reality, nor what they (or pretty much anyone) studies.

        I wish they ungated the Appendix (yes, they didn’t even discuss the lead time bias in the actual paper) so that you could read it. I’m not convinced by it.

        • They also didn’t discuss overdiagnosis/pseudodisease, which can come into play and skew survival rates for diseases where screening of asymptomatic patients is prevalent. It is telling that the differences in survival rates were in breast and prostate cancer, two cancers for which people in the US are screened at a much higher rate than most other countries.

          As we know, aggressive screening for breast and prostate cancers has been associated with the identification of abnormalities as “cancer” that were likely never destined to harm the patient. But inclusion of these “cancers” could artificially increase 5-year survival rates without any actual improvement in the treatment.

          It was disappointing to see such shallow and uncritical analysis of this paper in other media outlets, but kudos to Ms. Begley and Reuters for taking a closer look at the data and getting a different perspective.

        • Dear Aaron,

          Thank you for your thoughtful reply.

          I think that the issue of lead-time bias definitely arises with prostate and breast cancer, And the case of colorectal cancer is unclear to me — we screen for this (e.g. performing colonoscopies), but I’m unfamiliar with the standard of care in Europe, and a lead-time bias could arise if there were substantial differences in the protocols. On the other hand, the Philipson group looked at 13 cancers, so that leaves 10 more. I’m not aware of screening for leukemias, so that wouldn’t be susceptible to lead-time bias. But is there a difference in diagnostic techniques between America and Europe? I don’t know, and it would be very interesting to learn about that.

          Eric raises an interesting point — the abstract mentions prostate cancer and breast cancer as the biggest gains, but it doesn’t say anything about the other cancers. If those two were essentially the whole story, and no improvement was observed for the other cancers, then it would very tempting indeed to dismiss the findings as an artifact of lead-time bias. What were the findings for the other cancers? Unfortunately we can’t determine this from the way the abstract is written.

          • What do you think? Why do you think that’s what’s making the press rounds:

            Exhibit 2 shows the estimated value of these survival gains (1983–99) by individual cancer type. The value of additional survival gain was highest for prostate cancer ($627 billion in excess US gains) and breast cancer ($173 billion in excess US gains). But these US gains were partially offset by higher European survival gains for several cancer types, including uterine cancer ($67 billion in excess European gains); colorectal cancer ($46 billion in excess European gains); and melanoma ($2.5 billion in excess European gains).

            I wish I could post the figure, but it’s behind a paywall and I don’t know if they would approve. But it’s striking. Breast and prostate cancers totally dominate.

      • I do have access to the paper. Almost all of the value gained from survival comes from prostate and breast cancer. It is almost nonexistent or negative for other cancers.

        Steve

    • One final technical point . . .

      To estimate a mortality rate, we need count data, while an analysis of survival times will be based on a continuous outcome measure. My suspicion is then that the approach based on survival time would be more powerful, in the sense of requiring fewer study participants to achieve statistical significance, although I’m not certain of this. But if this is in fact the case, then that would be a strong argument in favor of designing experiments with survival time as the outcome measure rather than mortality rate.

      • It’s a very strong argument for doing localized studies to see if changes in care make improvements. Yes, if you can keep diagnosis protocols constant and then change therapy, you can use SR to show improvements. But you can’t do cross-country comparisons with that data unless the diagnosis protocols are the same. That’s what they did, and that’s the mistake,

        Again, you’re looking for ways to use the survival rate that make sense. You’re creating new hypotheticals. I don’t dispute those; I’ve written a post to defend the use of the survival rate properly. But that’s not what this study (nor many others) did.

    • I am a real layman in this discussion, so I will not be offended if no one replies to my following question:

      Why is it assumed that an extra year of life has economic value?

      This was the core of David Cutler’s book a few years ago and I did not understand it then either.

      Whether it is Terri Shiavo or my own father with Alzheimer’s or my friend’s son with a liver transplant at age 2 or my neighbor surviving two stage 4 cancers, all I see in an extra year of life is more cost, to the family and to society as a whole.

      My guess is that a nation does get richer when life expectancy moves from 45 to 75, as happened in America from the 1700’s to today.

      Maybe that is where the assumption comes from about the economic benefit of extra years of life, and I can understand that although it sounds like the old confusion of coincidence vs causation.

      I continue to suspect that if we move life expectancy from 75 to 90 we will get poorer.

      • Economists have used studies to determine what people would be willing to pay to have one more year of life. These are numbers derived from the general population. People might be willing to pay less for one more year if they have Alzheimer’s, but if they are healthy and active dealing with a very self limited illness amenable to treatment, say appendicitis, they are willing to pay much more for that extra year or years. Added to this you can add in the expected economic contributions if treatment extends life while someone is young enough to still be working.

        Steve

      • Remember, most of the gains in US life expectancy result from reductions in infant mortality, not adding years after age 70 or later; the gains at that end of the spectrum have been pretty modest, and they skew heavily in favor of the wealthiest segment of the elderly population.

        The other cost that goes unaddressed (as far as I can tell) is how treatment may actually reduce quality of life not only for the patient with cancer but also for his/her family caregivers. Adding three or six months of treatment-induced misery also has consequences, but it’s not clear that this circumstance has been considered in the study under discussion. Sadly, it often goes unaddressed when MDs propose it to patients and families, too.

    • Any comment on the more rigorous study some of these authors put forth in the Journal of Health Economics in 2010? Is it subject to the same kinds of problems? http://www.sciencedirect.com/science/article/pii/S0167629610000214

    • Thanks for pointing out the Reuters article. Very encouraging to see that kind of work by a wire service.

    • Well, I didn’t go to her boss and demand a raise, but I did buy one of her books. (Train Your Mind, Change Your Brain, despite the poor title, sounds like a decent intro to current neuro, in case I’ve missed anything obvious.)

    • You should have a look at http://www/healthnewsreviews.org
      That site regularly reviews and “grades” the reporting of health/medical news in 65 or so media. The reviews and grades ( 5-star system) are fed back to the media sources to try to bring about improvement. The site is sponsored by the Informed Medical Decisions Foundation (formerly Foundation for Informed Medical Decisions” fimdm.org). Sharon Begley is a favorite of Foundation associates.
      full disclosure: I am a member of the Board of Directors of IMDF.