• Healthcare Triage: Survival vs Mortality Rates

    This is the one I was born to do:

    Almost every time someone wants to proclaim the US to be the “best in the world” in health care, they point to survival rates. Those refer to the percent of people who live a certain amount of time after they’ve been diagnosed with a disease. But there are real problems in using survival rates to compare the quality of care across systems. The metric people should be using is mortality rates. And when we compare mortality rates, we don’t look nearly as good. Why is this important? Glad you asked. We answer in this week’s episode.

    Please watch and share! We need support in terms of subscribers and viewers to keep this going!

    @aaronecarroll

    Share
    Comments closed
     
    • Is there also an apples and oranges comparison happening here? If UK’s NHS detects and treats 100% of various diseases – breast, prostate, lung cancer, etc. – and, due to financial rationing, the U.S. detects and treats only 65% of those same disease populations, is the U.S. selecting the “better” risks in its mortality and 5-year survival statistics? Poor people have more exposure to toxins, have poorer overall health, and don’t live as long; the UK treats them, the US lets them die, often of something else (“heart failure”).

      This is related to my usual rant – how do you calculate your wait time for a medical procedure if you’re excluded from initial diagnosis and then treatment due to financial rationing?

    • If one wishes to promote an idea, one can use only those methodologies that promote the idea. If one wishes to promote science they look at all the methodologies using the best information available.

      I think the video is misleading. No doubt that there are problems with survival rates, but there are also problems with mortality rates and every other method of determining success. Many of the studies account at least in part for those differences. However, when one looks at the major studies even accounting for all the statistical problems one finds the US outcomes to be quite favorable.

      Since the video specifically mentioned cancer one has to think of one of the few good cross national studies ever done. The COCORD study. The US fared better than its western neighbors. Generally the US is hammered because of lifespan and the like which are horrible metrics to determine the quality of a health care system. Perhaps Dr. Carrol would like to give us comparable cross national studies as detailed as that of the CONCORD that demonstrate the CONCORD study is wrong.

      • Really? Please do point me to those major studies that account for all the statistical problems that find the US outcomes to be quite favorable….

        • “do point me to those major studies that account for all the statistical problems that find the US outcomes to be quite favorable”

          Please reread what was written again. I didn’t say every major study accounts for ALL the statistical problems or even that they necessarily accounted for some of them. I wrote: “even accounting for all the statistical problems one finds the US outcomes to be quite favorable.” Try and separate the two phrases.

          While you are at it can you show us any major cross country studies that contradict the conclusions of the CONCORD study?

          • The CONCORD study is, of course, about survival rates. Which I’ve written about extensively.

            I also wrote a 10-part series on quality of care in the US. Feel free to go read it.

            • Yes, I have read a lot that you have written and I don’t disagree that there are problems with regard to all the metrics used or a lot that you say. However that doesn’t mean one metric is better than another and that your preferred metric is the only way to look at things. In the case of the CONCORD study, yes one can raise a bit of criticism, but that is true no matter what study is provided or what set of analysis are done. I am not saying that survival rates are the only method of comparison, rather it is one method of many and the value of the method is dependent upon what is study and the data being used.

              Unfortunately nowhere in your writings that I have seen have you shown any cross country comparative studies that contradict the conclusions of the CONCORD study.

            • The CONCORD study is a comparison of survival rates. Literally, this video, is the evidence I’d provide against it. It uses the one metric I’m saying is flawed IN THIS VERY POST.

              How about this? http://theincidentaleconomist.com/wordpress/the-state-of-us-health-aint-so-good/

          • tara j, I think you are missing the point here. Dr. Carroll is not saying that the conclusions of the CONCORD study are wrong.

            He is instead saying they are *irrelevant*. Survival rates are interesting and useful, but they are neither interesting nor useful when comparing different countries’ health care systems, unless they are somehow adjusted for time of diagnosis. The CONCORD study did no such adjustment. Comparing survival rates between two countries,therefore, doesn’t tell us anything about how good the two health care systems are.

            • The Concord study says “Only invasive malignant tumours (behaviour code 3) were included,” which does actually narrow the field a bit at least. I agree with the point that comparing large populations with wildly different health systems mortality is better than survivability, but I wouldn’t say the survivorship rates are “irrelevant”. They answer two different questions and have two different populations. The video rightfully asserts that a patient who has just been diagnosed is more interested in their personal survival chances than a metric that would include the undiagnosed, for example.

              Both metrics have flaws and uses. The example of thumb cancer being detected earlier due to better processes or more frequent screening is a legitimate concern, but in comparisons can be mitigated by focusing on morphology, size, and indirect metrics (i.e. PSA %) (not that the CONCORD study did this sufficiently, but there are at least mechanisms available). Mortality similarly suffers from reporting on related or unrelated symptoms… if you have cancer but die of heart attack or pneumonia, you may not be certain if the cancer was related to or indirectly caused the death. Survivability is blind to this problem, because it doesn’t particularly care how a patient died, simply whether they survived the duration (one can undoubtedly adjust for this in many ways), and that can either be good or bad depending on how you’re using the data. If mortality is high but survivability is high, you may have an early detection problem. If mortality is high and survivability is low, you more likely have a treatment problem.

              So neither metric is perfect. I agree, again, that mortality is superior for cross country comparisons of overall health, but I’d rather see both values at once. The fact that the way these metrics are reporting can blind the general reader to these nuances is also a troubling issue.

      • I think tara needs to read Dr Caroll’s previous piece where he covered and disproved everything tara stated, a few years ago:

        http://theincidentaleconomist.com/wordpress/how-do-we-rate-the-quality-of-the-us-health-care-system-introduction/

        • Dr. Carroll writes some good stuff, but how does any of that contradict the conclusions of the CONCORD study?

          • @ Tara

            Follow the link Bob provided and click on “Disease Care” and look at the charts. What you find is America is, for the most part, middle-of-the-pack, which would be okay if our spending was middle-of-the-pack, but we spend 17.7% of GDP on health care and the rest of the G8 spends 11-13%.

            Medical bills are the leading cause of bankruptcy and there are concerns about the viability of Medicare and Medicaid because of rising costs.

    • Other relevant statistics would be to look at the average age at death and the average age at diagnose for a given disease.

    • And I’m trying, but so far as I can tell, these is no “CONCORD study” that shows the US coming out ahead in outcomes, contrary to every nearly other study ever done on the subject.

    • When the US appears to be number 1 compared to other countries that have national insurance this is saying something important. We are comparing an “insured population” to a partially insured population. This is very disturbing to those that believe the health care system in the US has bad quality. It is disturbing because uninsured patients should force the numbers way down, but don’t. If the number doesn’t fall that can mean 1) the US health care system is even better than seen or 2) lack of national of national health insurance doesn’t cause the problems being considered.

    • I find the obsession with statistical comparisons between various countries puzzling – at least if the goal is to improve patient care in the United States. If that was the goal, the obvious logical starting point would be to compare survival/mortality rate differentials within the US, control for confounding variables as well as possible, determine what aspects of the treatment regimen for each particular malignancy are generating the optimal outcomes, publicize the findings, and the possibly tether payment incentives to the use of the optimal therapy.

      Once you have the data necessary to understand what aspect of the way that care is delivered in region A differs from region B in a way that actually accounts for the observed differences in clinical efficacy, then you have the data necessary to determine what, if any of the observed international differences are due to variations in treatment in country A and country B, and what differences are driven by factors beyond the scope of medical intervention.

      If the goal is simply to leap from country A has better outcomes than the USA, ergo their health system is better, precisely how will that help any clinician or policymaker improve care in the US?

      Lets assume that the mortality rates for, say, breast cancer in Sweden are better than the US. Are they better than the entire US? What about the counties with the best mortality rates? Of those counties that have worse mortality rates, what aspect of the way we diagnose and treat cancer is driving the differential? How about those counties where survival rates are better? That would be helpful. Simply asserting that a statistical association between having singly payer healthcare and breast cancer mortality rates exists is not.

      • Nor did I make that assertion. But sometimes admitting you have a problem is the first step to working to a solution. In this country, a significant portion of the population believes that we are unassailably awesome, and it’s often because of survival rates.

        • I assume that all the talk about alternate metrics is more diversion and Gish Gallop, by people who would rather have us forget that we spend too much and we get too little.

          I don’t specifically care that if I get cancer, I will be treated super-duper effectively. What I want is to live (heathily) as long as I can. If I do require medical treatment, I don’t want to be bankrupted. I don’t want to be held hostage by a pre-existing condition that forces me to work at a job I hate. I might want to go work for a startup, without having to first be sure that my wife has a stable job with good health insurance. Some of these things are better now, what with years of liberal meddling in health insurance law (bless you, Senator Kennedy, I had two kids born under COBRA insurance).

          One useful thing that single-payer or single-provider health can sometimes (but not always) do is focus attention on the other things that drive medical costs; if, for example, subsidies for door-to-door seated transportation turn out to have an ultimate healthcare cost, then perhaps the secretary of healthcare costs might have a little chat with the secretary of transportation subsidies. Farfetched, I know, but how would that ever happen in our current system? Or perhaps it turns out that our food subsidies are causing people to eat the wrong diet, or to cause some peculiar change in the food itself, and that over time this is bad for us. It’s hard enough to get everyone on the same page when they work for the same organization, but at least with single-payer, they would work for the same organization.

        • I think that you are fundamentally misunderstanding the nature of many (but not all) of the sentiments that you are attempting to disabuse people of with every post in your “Best in the World, My Ass” series. What your erstwhile adversaries are arguing isn’t that people in the US enjoy the best health, or health outcomes in the world. They’re arguing that the clinical efficacy of the therapeutic interventions available here are “the best in the world.”

          They may be wrong about the latter point, and if you have clinical data that shows that the methods that, say, Danish physicians use to treat stage III colon cancer are more effective than the methods that physicians use in the US, or that the standard of interventional neonatal cardiology is superior in France, then go right ahead and present it and perhaps you’ll succeed in refuting the “argument from clinical efficacy.”

          When it comes to the former point, even the people who are the most adamant about the excellence of the care that doctors and hospitals in the US deliver will freely, and often enthusiastically, acknowledge that our health-outcomes often fall short of those achieved in Europe or the wealthy parts of Asia, but go on to argue that the particular set of risk factors associated with the American culture and lifestyle – particularly those that have a statistical association with attending a Baptist church – have far more to do with the poor outcomes than any particular operational deficiency of the US health system.*

          Anyhow – let’s, for the sake of argument, assume that you have succeeded in your goals and persuaded the last person in the US who believed otherwise that the clinical efficacy of cancer care in the US isn’t the best in the world, and has plenty of room for improvement.

          What’s the next step? How, exactly, can we improve the clinical efficacy of the care that we deliver for a particular malignancy by looking at international mortality data instead of survival rate data? We can’t – any more than we can look at the survival rate/mortality differentials between Vermont and Alabama and learn *anything* useful about what’s driving the differentials without adjusting for both variations in how patients are treated, and how they live. Even if the progressive utopia materializes and the US converts to a single payer system, the absence of this data will dramatically hinder our capacity to understand the origins of these differentials, much less address them effectively.

          *You should really read the entire report, but start with figures 11, 16, 17 and Table 1a.
          http://www.healthmetricsandevaluation.org/sites/default/files/policy_report/2013/IHME_GBD_US_FINAL_PRINTED%20070513.pdf

          • Actually, I’m not arguing with that point. I don’t believe the medicines work any less well here. Nor do I believe our physicians are undertrained. I’ve said the opposite many, many times. I think our system is crappy, and that’s the focus of what I write about.

            These studies don’t compare effectiveness of treatment. They compare efficacy in real life. And, when it comes to that, the US falls short.

            We have the potential. That’s why I keep on doing this.

            • Okay – but exactly how is what you are doing with your “Best in the World, My Ass” series actually contributing that would have a meaningful impact on the causes of disparities in “efficacy in real life” that drive the differentials between the US and other countries that so trouble you even under a single-payer regime? Seriously.

              If this is about data – look at the data that tabulates the magnitude of the differentials that drive the massive disparities between regions within the US. Again – figure 11 “Percent of DALYs attributable to the 17 leading risk factors, both sexes, all ages,US, 2010,” What are they: 1)Dietary Risks, 2)Tobacco Smoking 3)High Body Mass Index 4)High Blood Pressure, 5)High Fasting Plasma Glucose 6)Physical Inactivity and Low Physical Activity 7)Alcohol Use 8)High Total Cholesterol 9)Drug Use 10)Ambient Particulate Pollution.

              Then look at table 1a: “Table 1a: Top 10 and bottom 10 counties in terms of life expectancy by sex, 2010,” The best for females and males, respectively: Marin, CA (85-years) and Fairfax VA(82 years). The worst: Perry, KY (73 years) and McDowell, WV (64 years).

              Then ask yourself this question: how would the adoption of single payer meaningfully change *any* of the leading contributors to poor health that are driving the massive differentials in health and longevity between these counties. Seriously. Assume we’ve got single payer modeled precisely on the Canadian single payer system. What specific mechanisms currently at work in Canada would address dietary risk, smoking, high BMI, high blood glucose, low physical activity, etc, and what is your best guess as the magnitude of the change in each that we would see. How much have we moved the needle on any international comparison of health and longevity? How about the differentials in cancer mortality stats? What’s the magnitude of the change we can expect to see in Perry, Kentucky? Work in the findings from the Oregon Medicaid Experiment and the RAND study for extra credit.

          • JayB, I have to say excellent again and then again in the third follow-up. You have targeted the real problems separating them from the fluff and inaccurate conclusions that frequently are associated with studies containing multiple lifestyle choices, behaviors and other things that are not the primary job of the health care sector. Moreover, you have focussed in on the problems that many who believe in outcomes have been unable to verbalize.

      • Excellent

    • Great video. I’m going to bookmark it, and show it to my friends every time the USA! USA! Best in the world! discussions come up.

      This is in the weeds, but when we talk about comparing mortality rates, is there an age-adjustment? It seems like there ought to be. Assuming exactly equal treatment in two countries, the one with the older population would have worse mortality stats for things like cancer, and the one with the younger population would have worse mortality stats for, I dunno, skateboarding deaths or something.

    • Just a few random thoughts.

      In many cancers a 5 year survival without residual cancer is considered a cure. Thus survival rates can even have an end point.

      Race is exceptionally important. Black people are felt to develop more aggressive prostate cancers yet with about 20% blacks and many uninsured blacks the US competes successfully in prostate cancer cure compared to nations without this group even though blacks probably has the highest mortality rate for this disease.

      Early diagnosis does create a statistical problem for certain types of comparisons. We hear that mentioned on the list a lot in that the US picks things up earlier. But, early diagnosis also saves a lot of lives that would die if picked up later so it is a good thing.

      • Yes, but if early detection = cure, then you should see that reflected in the mortality statistics, right? The former cancer patient should go on to die of something else. If there’s no impact on mortality, then there was no cure.

    • I actually like a lot of this study, because it studies the burden of disease. I am not sure of how they abstracted all their numbers. The authors admit to abounding difficulties in doing so. What the study is really looking at is not how well a healthcare system manages a specific disease, but what is happening to society disease wise. It requires a lot of subjective evaluation, but even with that type of evaluation it actually confirms the CONCORD study with regard to breast and colorectal cancer. It also adds other diseases such as stroke and cardiovascular disease to the success.

      A lot of our problems are derived from those things that we pay less attention to. Take diabetes of the young. As the young become more obese diabetes increases and we see an increase in death, disability and chronic disease. It is not that the medical system doesn’t know how to treat diabetes. It does. It is societal habits that are creating much of our problems that we need as a nation to face.

      Take a look at where the study says the biggest increase number of life years is lost. Alzheimer’s, drug use, and kidney disease (a frequent consequence of treating the very ill). Take a look at some of the big items in chronic disability. Musculo-skelatal issues such as low back pain (very subjective) , drug use, and mental health. Take a look at the leading risks in order of ranking, diet, tobacco, obesity.

      Apparently the US is good at treating specific diseases, but the overall health of the nation is not doing as well as it should. It is that portion of society that needs to be better managed.

      • I should have added that the study Burden of Disease does not demonstrate that the US health care system is bad. It demonstrates that certain things in society are causing our overall health to be poorer than expected. The metric that needed to be focused on more is socio economic problems that have a great impact on society and its health.

        Who smokes the most? Who drinks the most? Who uses the most illegal drugs? Who doesn’t follow medical advice the most? Whose diet is the worst? Who is exposed to the most toxins? etc. None of these things have that great a correlation with how well the US provides clinical medical care or care that requires hospitalization.

        Thus let us assume the CONCORD study is 100% correct and the US is best at outcomes. This study (Burden of Disease) would be relatively consistent with it.