• Letting Perfect be the Enemy of Good?

    This is a guest post by J. Michael McWilliams, MD, PhD, assistant professor of health care policy and of medicine at Harvard Medical School and an associate physician in the Division of General Medicine at Brigham and Women’s Hospital. He is also author of the 2009 Milbank Quarterly paper “Health Consequences of Uninsurance among Adults in the United States: Recent Evidence and Implications.” (This post has been cited in the 18 February 2010 edition of Health Wonk Review.)

    An Atlantic Monthly article by Megan McArdle questions whether health insurance coverage saves lives, drawing from a narrow slice of the literature to suggest the beneficial effects of insurance coverage on mortality might be negligible.  While it is true these effects have been challenging for researchers to assess accurately, this question deserves more than a selective reading of the literature to inform the public and policymakers properly.  Indeed, when reviewed comprehensively and with an understanding of key clinical and methodological nuances, the research to date provides consistent and compelling evidence that health insurance coverage significantly improves health outcomes, particularly for adults with treatable conditions (McWilliams 2009).

    Studies on the health consequences of uninsurance can be broadly categorized as observational or quasi-experimental.  Observational studies compare health outcomes between insured and uninsured adults and use statistical techniques to control for differences in other predictors of health between the two groups.  These studies are fundamentally limited because it is usually impossible to control for all possible differences and some differences may be both causes and consequences of insurance coverage.  Consequently, observational results may underestimate or overestimate the true effects of coverage.  From the sizable observational literature, McArdle selects just one negative study to suggest insurance coverage may not affect mortality (Kronick 2009).  Yet several other observational studies that controlled for an equally robust set of characteristics have consistently demonstrated a 35-43% greater risk of death within 8-10 years for adults who were uninsured at baseline and even higher relative risks for older uninsured adults with treatable chronic conditions such as diabetes and hypertension (Baker et al. 2006; McWilliams et al. 2004; Wilper et al. 2009).

    Because these observational studies are not sufficiently rigorous to support causal conclusions, we should look to studies that are more experimental in design for more definitive evidence.  McArdle cites a principal finding of the RAND Health Insurance Experiment (HIE) that more generous coverage led to more health-care utilization but not better health outcomes on average.  However, the set of findings from the RAND HIE that is arguably more salient to this discussion is that more generous coverage did lead to better blood pressure control and lower predicted mortality for low-income adults with hypertension — adults that resemble the uninsured population more closely than the average adult.  Moreover, the RAND study was conducted in the 1970s, prior to numerous advances that have improved the effectiveness of medical care for many acute and chronic conditions.

    From the quasi-experimental literature, McArdle cites evidence of a lack of immediate survival gains with near-universal Medicare coverage after age 65 in the general population (Card et al. 2004; Levy, and Meltzer 2008).  From a clinical perspective, however, we should not expect immediate survival gains for most previously uninsured adults because mortality is such a distal outcome.  Survival gains may not manifest for years after improved chronic disease control and cancer screening are established, suggesting much more complex improvements in mortality trends are likely to evolve after age 65 in response to universal coverage.  Quasi-experiments that rely on abrupt discontinuities occurring with age are not well suited to capturing these complex but potentially large effects.  Consequently, the absence of evidence suggested by these studies is not evidence of absence.  In contrast to the general population, immediate mortality effects might be expected for acutely ill patients for whom coverage may improve access to life-saving procedures and therapies.  Indeed, a more recent study found age-eligibility for Medicare was associated with a substantial and lasting reduction in mortality for patients who were hospitalized for a range of acute illnesses that were amenable to treatment (Card et al. 2009).

    Because many quasi-experimental strategies are geared to capture effects of insurance coverage only if they occur in the short term, they are better suited to examining proximal or intermediate health outcomes.  Therefore, perhaps more can be learned about the effects of insurance coverage on mortality from studies that rigorously examine effects on health outcomes that are highly predictive of mortality.  To date, numerous studies have found consistently beneficial and often significant effects of insurance coverage on health across a comprehensive set of outcomes and a broad range of treatable chronic and acute conditions that affect many adults in the U.S., including hypertension, coronary artery disease, congestive heart failure, stroke, diabetes, HIV infection, depressive symptoms, acute myocardial infarction, acute respiratory illnesses, and traumatic injuries (McWilliams 2009).  In particular, several studies have robustly demonstrated positive effects of near-universal Medicare coverage after age 65 on self-reported health outcomes and clinical measures of disease control, particular for adults with cardiovascular disease or diabetes who make up two-thirds of the near-elderly (Decker and Remler 2004; McWilliams et al. 2007, 2009).  Thus, when rigorous study designs have been coupled with appropriate outcomes and applied to clinical populations for whom medical care is effective, the evidence that insurance coverage improves health and survival is consistent and convincing.

    How many lives would universal coverage save each year?  A rigorous body of research tells us the answer is many, probably thousands if not tens of thousands.  Short of the perfect study, however, we will never know the exact number.  In the meantime, we can let perfect be the enemy of good.  Or we can recognize the evidence to date is sufficiently robust for policymakers to proceed confidently with health care reforms that promise substantial health and financial benefits for millions of uninsured Americans.


    Baker, D. W., J. J. Sudano, R. Durazo-Arvizu, J. Feinglass, W. P. Witt, and J. Thompson. 2006. “Health insurance coverage and the risk of decline in overall health and death among the near elderly, 1992-2002.” Med Care 44:277-82.

    Card, D., C. Dobkin, and N. Maestas. 2004. “The impact of nearly universal insurance coverage on health care utilization and health: evidence from Medicare”. NBER Working Paper Series. Cambridge, MA: National Bureau of Economic Research.

    Card, D., C. Dobkin, and N. Maestas. 2009. “Does Medicare save lives?” Quart J Econ 124(2):531-96.

    Decker, S. L. and D. K. Remler. 2004. “How much might universal health insurance reduce socioeconomic disparities in health? : A comparison of the US and Canada.” Appl Health Econ Health Policy 3:205-16.

    Kronick, R. 2009. “Health insurance coverage and mortality revisited.” Health Serv Res 44:1211-31.

    Levy, H. and D. Meltzer. 2008. “The impact of health insurance on health.” Annu Rev Public Health 29:399-409.

    McWilliams, J. M. 2009. “Health consequences of uninsurance among adults in the United States: recent evidence and implications.” Milbank Q 87:443-94.

    McWilliams, J. M., E. Meara, A. M. Zaslavsky, and J. Z. Ayanian. 2007. “Health of previously uninsured adults after acquiring Medicare coverage.” JAMA 298:2886-94.

    McWilliams, J. M., E. Meara, A. M. Zaslavsky, and J. Z. Ayanian. 2009. “Differences in control of cardiovascular disease and diabetes by race, ethnicity, and education: U.S. trends from 1999 to 2006 and effects of Medicare coverage.” Ann Intern Med 150:505-15.

    McWilliams, J. M., A. M. Zaslavsky, E. Meara, and J. Z. Ayanian. 2004. “Health insurance coverage and mortality among the near-elderly.” Health Aff (Millwood) 23:223-33.

    Wilper, A. P., S. Woolhandler, K. E. Lasser, D. McCormick, D. H. Bor, and D. U. Himmelstein. 2009. “Health insurance and mortality in US adults.” Am J Public Health 99(12):2289-95.

    Comments closed
    • Dr. McWilliams,

      Thanks for a great summary of the topic – but I must confess that I am left to conclude that most of the benefits of expanded health insurance coverage are tied to more people getting treated for two specific chronic conditions – hypertension and diabetes.

      I remain unconvinced that this is sufficient justification for what is on the table – an expensive system that runs the very real risk of making things worse rather than better [lower supply of care and lower quality of care at the margins].

      BTW we have the means to extend coverage for hypertension at least – I am not sure if it works for insulin and diabetes – but Wal Mart through it’s $10 for 90 days by mail perscription drug program keeps the cost of my med cocktail at $120 a year – for another $300 I can go to a walk in clinic and have my BP monitored and deal with any other concerns that might be bothering me.

      A program to provide this type of coverage to all Americans would cost around $50B and should deliver significant measurable benefits in short order.

    • Would you sign Robin Hanson’s petition for a repeat of the RAND experiment? As he said, one could have made the same argument about recent advances (relative to the past) back in the 70s.

    • “However, the set of findings from the RAND HIE that is arguably more salient to this discussion is that more generous coverage did lead to better blood pressure control and lower predicted mortality for low-income adults with hypertension”

      The key word in that sentence is *predicted*. Out of all the health metrics they looked at you might reasonably expect at least one to be “significantly” improved through chance alone. So as it turns out, blood pressure regulation was a bit “better”. However, lower *predicted* mortality arising from this is basically speculative. Had they found lower *actual* mortality (either from this cause or overall) the author might have more of a case. Anyway, one point of doing something like the Rand study is to find out whether you might be reducing mortality from one specific cause only to increase it from some other cause. (A great many treatments are proven to improve one specific observed health metric but not proven to reduce overall mortality.)

      In short, there really is no escape from the Rand findings other than to claim “yeah, but medicine is better now than it was in the 1970s!” Which argues for doing another Rand Experiment, bigger and better than the last. Without having done that – and found different results – I think Megan’s argument stands.

    • Note that nobody has challenged the notion that medical care is effective when “applied to clinical populations for whom medical care is effective” – that’s a tautology. The problem is that medical care is ineffective and dangerous – when applied to clinical populations for whom medical care is *not* effective. Which factor predominates is an empirical question. Will Wilkinson also notes that there are distributional effects here. The populations whose lives will be saved by more access to care will skew towards older sick people in their 60s; the population who will be killed by more access to care will skew towards younger healthy people in their 20s. So even if overall mortality were unchanged – there might still be a reduction in quality-of-life-years.

    • I find this article to be less than convincing.

      McArdle cites the “one” negative study because in her analysis it is the largest study, therefore providing us with the largest sample to infer a conclusion from. Note that the author does not even address the Kronick 2009 article.

      It may be that more of the observational studies that have been published show associations between health insurance coverage and improved health outcomes, but if a large – if not the largest – study shows the possibility of no effect, does this not put the other studies’ results into question?

      I also noticed that a third of the literature cited in this article is lead-authored by the article author.

      Please elaborate on why the Kronic 2009 study does not at least suggest that the effect of health insurance coverage on mortality might be lower than previously estimated, if there is an effect at all.

    • The size of a study is not as important as its design. A more fully elaborated model with a smaller sample is less [more] reliable than a less well developed model with a larger sample size.

      I believe Amy made a typo, which I corrected above. She may contact me through the blog or leave another comment if my edits are incorrect. -Austin Frakt

    • Two points:

      1) All we really care about is whether there is a causal realationship between health insurance and better mortality and morbidity outcomes. Strong evidence of correlation is not strong evidence of causality because there is probably self-selection bias among those who do not have health insurance.

      2) Even if providing free or subsidized health insurance was effective at saving lives, it would not mean that it was COST-effective at saving lives. Poverty causes excess mortality in many different ways, and the government might have a greater impact for less money in other ways.

      See http://randomfinancialthoughts.blogspot.com/2009_11_01_archive.html for a brief discussion.

    • Amy,

      Your point is well taken, however the author explicitly states that the Kronick 2009 study “controlled for an equally robust set of characteristics.” I interpret this as a statement of similarity, if not equivalence, in the design of the studies cited in this article. If the Kronick study is the largest observational study done to date, and there are no major issues or differences in the study design when compared to similar observational studies, the larger sample size gives us more confidence in the results, not less.

      I accept that there are always differences in study designs, particularly the noise inherent in observational studies. My request is that the author – who has clearly concluded that universal healthcare coverage will lead to “substantial health and financial benefits for millions of uninsured Americans” – at the very least explain why the Kronic study demonstrated the possibility of no effect. Without adressing this key point, the article remains unconvincing.

    • I guess what strikes me at the end of the day is how hard it is for the supporters of this argument – that health is improved by access to insurance – to find any measurable effect let alone a large one.

    • On the Rand study, I absolutely totally completely agree that health care in the 70s was so different from health care now that we can’t infer anything about the present from it. For example, it tells us nothing about the effect of insurance on mortality via statins or oral anti-diabetics — they didn’t exist then.

      I also recall a debate about panel attrition in the Rand study. The rate of attrition wsa tiny compared to most panels. However, it was large enough that it could have covered differences in mortality. Again the point is that mortality is rare. Missing the deaths of a fraction of a percent of experimental subjects can totally change the conclusion. More subjects than that vanished- were not tracked — had health outcomes which couldn’t be assessed. Under the implausible assumption that all such subjects died, one would conclude that less generous health insurance had a huge huge effect on mortality (the low rate of attrition was higher for those who received stingy insurance). Under the implausible assumption that they were no more likely to be dead that non drop outs the RAND team concluded that less generous health insurance had approximately no effect on mortality. Given the data, depending on the assumptions, the result could be anywhere in between.

      The best and the good enough. The RAND study might have been the best attempt to prevent panel attrition in human history. Arithmetic says it might also not have been good enough to get a reliable estimate of the effect of the generosity of health insurance on mortality.

      Also there was a definite statistically significant and large effect of generosity of coverage on the rate of dropping out of the financial aspects of the study (returning to pre-study health insurance — this is different from panel attrition ). This probably surprised the Rand team, since it was never economically advantageous to drop out of the financial part.

      It is obvious from that data that subjects delayed health care, dropped out and then obtained the health care (the health care demanded by dropouts was very low especially shortly before they dropped out — this is true for ambulatory care but not hospital care for those who had stingy cover of abulatory care but generous coverage of hospital care and vice versa). The effect of delaying care on both final outcomes (measured) and total health care spending (not evaluated) are minor. The effect on spending while covered by the experimental insurance is large. This pattern, which is very clearly in the data, can explain the RAND results and prove that they are completely uninteresting.

      In response to criticism, the RAND team analysed the drop outs (decades after the study was conducted). They assumed that health care utilization decisions depend only on current insurance — that the fact that I will be insured tomorrow has no effect on whether I seek care today. This assumption is obviously totally utterly insane. Their claims absolutely are as reliable as that assumption.

      The RAND study is the best ever. It is also just not good enough to tell us anything useful.

      I’m just a commenter so i don’t have to provide links. This is a good place to start http://tinyurl.com/yjvcms2 . You have to understand what they are saying to understand that my description ((above) of the absolutely absurd assumption was made by the RAND team is absolutely accurate.

      • The RAND HIE has had its share of criticism. See, for example, Nyman’s concern over attrition bias and the response by study investigators.

        The take away message for me isn’t necessarily that the RAND HIE is flawed or is perfect, but that reasonable people can debate the degree of reliability of the results. That’s true of any study. Just because it had a randomized experimental design does not in and of itself mean that the RAND HIE results are any more or less correct than those from any other study. It all depends on the extent of the known issues with the data and how they’re handled. A well done observational study can be as or more informative than a randomized trial.

    • It is interesting and maybe telling that it is even debatable!

    • Two other features of Rand that deserve comment. First, out of pocket expenditures were capped, so it represents, to some extent, a lower bound estimate of the effect of health insurance, if we assume that “normal” health insurance does not cap costs in a similar way (and it doesn’t to anywhere near the same extent).

      Second, the Rand results represent the effect of health insurance on access to 1980s medical care. Anyone willing to voluntarily accept that care rather than what we have today? So, again, Rand may underestimate the impact of access to 2012 medical care.