• How to tell if the ACA is working

    As provisions of the ACA are implemented — if they are implemented — there will be a lot of competing claims about the extent to which they are working. How should we judge the effects of the law?

    Robert Brook, in JAMA, offers four principal metrics, though admits there are many other sensible ones as well.

    1. “The first measure of the law’s success should be how much preventable mortality that is due to the health care system will be eliminated.”
    2. “The second goal of health care reform should be to drive the avoidable hospitalization rate from ambulatory-sensitive conditions to zero.”
    3. “The third measure of the success of health care reform should be whether it increases to 100% the number of US residents who have access to a system of care.”
    4. “If health care reform is successful, the growth rate of health care costs for all US residents enrolled in a health care system should be reduced to the growth rate of the gross domestic product or less.”
    Doesn’t that all sound perfectly reasonable? It would, indeed, be a wonderful world if success were obtained on all four of these metrics. Yet, I think they’re unfair metrics for the ACA and set it up for failure. Let’s take them in turn.
    1. I agree that there is a relationship between health insurance and mortality, and a vast body of evidence backs me up. However, the relationship is not an immediate one. Many of the uninsured are young and healthy, though some are not. Most of them, thankfully, won’t die for decades even without insurance. Though good coverage will improve their health and increase their odds of living longer, that won’t be measurable for a long, long time. It may one day be possible to credit the ACA with changes in mortality, but we cannot do so in the near term. If we look for mortality effects in the first years or even decade we will be disappointed.
    2. I suppose if by “drive” to zero one means move toward zero, then I’m OK with #2. What I want to caution you about is that we should not expect the rate of “avoidable hospitalizations from ambulatory-sensitive conditions” to actually be zero. These are conditions that, according to administrative and billing data, should be addressed without hospitalization. However, it is well-known that we can’t see everything relevant from administrative and billing data. Some individuals with conditions so labeled really do require hospitalization for reasons unobservable to us. Zero is too low a target. I’d look for a decline, though.
    3. My concern about #3 is that the law isn’t really set up to achieve universal access to a “system of care” (which is more than emergency department care). No question that it should increase such access, but 100% is too high a target. Some classes of individuals — like undocumented immigrants — will not have the same type of access to insurance subsidies and care as others. There will be regions of the country where access is more limited. That should be improved, but the ACA is not designed for improvement to the 100% level.
    4. Though it would be nice to see health care spending achieve a GDP or lower growth rate, I’m not convinced that should be or is a goal of the latest reform. The Medicare target in the law is GDP+1%, after all. Moreover, it is not unreasonable for a nation to spend a greater share of wealth on health as that wealth increases. Naturally, that cannot  be maintained indefinitely, but GDP+0.5% or GDP+0.25% can be maintained for a very long time. Perhaps the right level of spending on health as a proportion of GDP is much higher than we see today. That’s not to say we don’t have waste in the system. We absolutely do! It’s just to note that expecting a GDP or lower growth rate is a very ambitious and possibly inappropriate goal.
    We should definitely establish goals for reform and continue to make reforms until our goals are met. But it is important that those goals be sensible and reasonable. It’s crucial they don’t set up the reform for failure. It is also important that metrics for them are consistent with the intentions of the reform. Not all of the ones suggested by Brook are.
    I’ll bet you can think of some goals for the ACA? How will you judge whether it is successful or not?


    • These are laudable goals but why so ambitious. Would he be willing to give up the reforms in the ACA if these are not achieved?

    • I don’t think he is saying that the ACA needs to achieve all of his points to be deemed successful, just that it should get closer to those goals then the current system. I’m sure he would agree that the ACA is successful if life expectancy stays constant, hospitalization rates stay constant, access increases, and the health-care spending growth rate slows 1-2%.
      I’ll throw in another goal that I think is hard to measure (I’d love a good metric for this…) and one that I think we’ll be hearing a lot about over the coming year, which is: “The ACA should not impose an excessive burden on individuals or businesses”.
      (I know there is significant interpretation possible, what is “excessive”, etc.. But I’d love to hear rational discussion/comparison in regards to this goal.

      • Look, it is not my job to infer what’s in the mind of the author. I go by what he wrote, as will many people who don’t understand the details as well as I do. I think if Brook meant something more nuanced than what he wrote he should have expressed it. If he doesn’t stand by his words, why was it published? I’m happy to have a debate about this, but I’m not going to let someone off the hook for the very words he chose in a prominently published piece. He selected strong normative language and firm numbers like 0 preventable hospitalizations, 100% access, a maximum of GDP growth rate. He wrote that there should be measurable decline in mortality. I stand by my response.

    • I would be interested in your thoughts on the regulations promulgated Jan. 20 by the Department of Health and Human Services that require contraceptive services and sterilizations to be covered by the insurance policies under the Affordable Care Act. without copays, coinsurance, or deductibles, or an exemption for religious organizations.

    • As to the growth of health care costs, we should be extremely careful about how we count them. Current measures sometimes have some very strange structural biases.

      My favorite example is “survivors’ costs” when, for instance, the cost of increasing vaccination rates is assessed to include the other future (and unrelated) health costs of people who would have died from the preventable condition but, because of the vaccine, lived on to use more health care services. So if the ACA somehow accomplishes the first metric mentioned (reducing preventable mortality) it will be, by some measures, responsible for increased health care costs. As the dismal science would estimate, it’s cheaper to die.

      • Tim —

        I don’t see why it’s unreasonable to include “survivor’s costs” in a financial analysis, or how it generates any kind of bias. Let’s use your example: people who are vaccinated against a potentially fatal disease will then go on to increase health care costs because they will use resources that they wouldn’t have if they had died. If the goal is to estimate the impact of the vaccination campaign on the total costs in the system, why isn’t this a very valid point? Of course, to be really fair, we should also estimate the additional revenues that such survivors generate e.g. the premiums that they will pay over their now increased lifespan. But even so, that’s not really an objection to the principle of “survivor’s benefits”, just an issue about the technical details of how to calculate the estimate.

        I think that we are both in agreement that such an analysis should not be used to stop vaccination campaigns. But the argument would be that the social good that such a campaign generates is much more valuable than the net costs incurred. I’m good with that, and I think that most people would be very comfortable with it as well.

        • I’m sorry if I was unclear in my comment. I don’t think this is an inappropriate answer to the question, “How much will this cost?” I just think it’s a really bad answer to the question, “What should our health policy be?” As a metric of whether we have a good health policy, there should be cost AND value included.

          As I understood the original comment, it was (in point four) asking whether costs go up. That’s too thin a measure. Costs can go up in a way that is valuable.


    • With respect to #4, the growth in NHE relative to the growth of GDP:

      US history since at least 1960 suggests a NHE to GDP elasticity of 1.38, and the most recent numbers haven’t fallen off that straight and narrow path. [http://onlinelibrary.wiley.com/doi/10.1002/hec.1765/abstract and discussed in TIE at http://theincidentaleconomist.com/wordpress/most-important-chart-in-health-policy/ ] At a minimum, ACA’s effects on expenditures should be measured by changes in that elasticity.

      I’ve argued that this straight and narrow path is even justifiable because there are several reasons (such as hope) why consumers with higher incomes might want to spend a higher-than-average proportion of their extra incomes on health care. Indeed, rising NHE could have been conceptualized as an engine of economic growth.

      But unfortunately, the ongoing increases in income inequality radically change the insurance dynamics that supported the last 50 years of NHE growth.

    • If it really is difficult to establish success metrics for the ACA, then that suggests that the goals of the law are unclear. You’ve suggested that the metrics should be “consistent with the intentions of the reform”. Perhaps the first step would be to clearly enumerate those.

      I can suggest two metrics that will applied ruthlessly in practice. If people who currently have health insurance see their premiums rise sharply once the law goes into effect, I don’t think that it will be a political success. Similarly, if people see their insurance disrupted (e.g. an employer discontinuing their insurance and sending them onto the exchanges) then again I think the program will be judged a failure.