• Can Risk Adjustment Save the Public Option?

    Ezra Klein has a truly excellent post on adverse selection and the public option. He concludes with, “The most important factor here will be the strength of the risk adjustment in the exchanges, so keep an eye on that.”

    I wonder how optimistic we can be about the degree of variation in spending predicted by risk adjustment models. I think the answer is “not very.” From the literature on health care risk adjustment (via this post):

    Statistical models developed by scholars have relatively low predictive power. Predicting ten percent of the variation in [health] expenditure is considered good (e.g., Medicare Advantage’s risk adjustment model). That means ninety percent of the variation is unexplained by the model or chalked up to random error. An individual ought to be a better predictor of his or her health expenditures than a model that cannot include measures unobservable to the researcher. (How much better? I don’t know.)

    Expenses for some specific services are more predictable. Drug expenses, for example, are persistent because individuals tend to use the same medications year after year. The best statistical models of drug spending can predict about 55% of the variation in next year’s drug expenses, leaving 45% to random error.

    That puts a reasonable cap at 55%, but only for very persistent services, like drugs. Expect the best overall risk adjustment to be no worse than 10% and no where near as good as 55%.

    Private insurers should not be so worried but taxpayers should. The public plan looks game-able.(*)

    (*) A wonky note: It isn’t game-able because the risk adjustment model is of low power. It is game-able because insurers likely have access to information not observable to researchers and omitted from the risk adjustment model, which makes it lower power than it could otherwise be. The risk adjustment model was developed in a political environment in which the insurers were participants.

    Comments closed
    • Well, if it’s clearly gameable, it’s predictable. The same way an insurer can know that a certain population has a different risk profile, a regulator can use them in a risk model. So, if Aetna sees that young men in Chicago are cheaper than the rest of Illinois, a risk adjustment model can less in subsidies to those covering this population based on the average for young men in Chicago. That breaks down somewhat if an insurer has access to private information, or if the model becomes too complex for mass distribution.

      What you and Ezra are confounding is goodness of fit and unbaised estimators. We can’t predict what every individual is going to cost. But we can have a pretty unbiased estimate for individual cost, giving us something closer to what a large population will cost. While you can’t predict any individual with accuracy, you can have a decent idea for a large population. It’s what insurers do in many lines of business. And the best rating variables don’t explain individual behavior at all, but they can predict big differences at the group level. My understanding is that Medicare Advantage and Part D risk adjustors get near the high 90s when predicting total plan costs (I cannot cite). My own research shows qualitatively similar variables in auto insurance.

      • @GrandArch -Your claim is that the risk adjusters I referenced, and those likely to be implemented under health reform, are unbiased when applied to the population that self-selects into one plan or another. For that to be true then there must be no patient or insurer characteristics excluded from the model that are correlated with cost and selection. That is, selection must be exogenous to cost.

        Is it even remotely credible that patients and insurers don’t have additional information that is correlated with plan selection and cost (utilization)? Is there no omitted variable bias? I can’t imagine it to be so. Patient attitudes, preferences, behavior, among other factors influence degree of utilization and plan selection. Providers have access to a wealth of patient and cost information that are correlated with plan design (which has a selection effect) and utilization.

        There are numerous studies that suggest favorable selection into Medicare Advantage plans (and the plans under predecessor programs) based on models that include variables not in the Medicare risk adjustment model. Moreover, it has been discovered that plans have been upcoding to make beneficiaries appear sicker than they are, gaming the risk adjustment system.

        Do we really expect different risk adjustment performance and plan and patient behavior under a reformed health system for the non-elderly? Why? Upon what research ought we base such a conclusion?

        Your claim, for which you provide no citation, is that risk adjustment has 90% predictive power on total plan costs. That would be incredible news, contrary to the literature with which I am familiar. I’d love to see the research that supports it. Do share!

    • Hi Austin
      I have heard numerous interviews from Dutch health care officials re: the progress they have made on risk equalization. I just took it as gospel, but never reviewed the numbers. Did some digging, and at least by this recent Commonwealth report, they are are at 70% with latest adjustments. See Table 1 on page 15 or 16 in linked pdf report.



      • @Brad F – Nice tip! That is very interesting. What is reported is not the same measure as I was using. I had cited the R-squared value and what is given in the table you reference is the percent of actual cost predicted. In fact, if I’m not mistaken the table you cite is revealing a systematic downward bias: average actual costs are always above average predicted costs, at least for the upper end of the distribution that is reported. Given all these reporting issues it is really not possible to make a comparison between the Dutch and US Medicare risk adjustment systems.

        However, from the text of the report it is clear that the Dutch risk adjustment system includes variables that are not included in the Medicare system, such as income. Including such variables can only help performance.

        I’d have to do some more digging and reading to more fully compare the two systems. If you come across any other work that does so, please share it.

    • It was Health Affairs, remembered:

      Dont be getting all economist on us simple country folk. I need the “if you had two tootsie rolls in this pile, and three in this pile”…..(and a diabetic ate all five, which MCO would dump the applicant…)

      • @Brad F – I skimmed it but didn’t see anything in it that could be used to directly compare the Dutch and US risk adjustment systems.

        I presume by the “all economist on us” you mean the reference to R-squared. It’s a measure of the statistical power of a prediction model. If R-squared is 100% then the model predicts all variation in the thing being modeled. Nothing is left to chance. If R-squared is zero it predicts nothing, you have complete randomness. How much of the randomness is predicted by Medicare’s system? As I said ~10%. How much does the Dutch system predict? I haven’t seen that reported in anything you’ve sent, but I’ve read nothing thoroughly. I’m sure there are knowledgeable readers out there who know the answer or where to find it. I just don’t have time to do all the research this second. (It is Friday night after all.)

        Anyway, kudos to you for citing something. (As you can tell, I’m not a fan of folks who make claims that I’ve got something wrong but can’t cite anything to back that up. You did neither of those things and have advanced the debate forward slightly. We now know that we should look into the Dutch system further. But not me, not now.)

      • @Brad F – Thanks. It is a frustrating document. As far as I could tell it did not report an R-squared of the risk adjustment model. And it made no references to the literature. So, I still do not know the extent to which the Dutch system is better than Medicare’s, if at all. No doubt this information is out there and the right paper could be located with Google Scholar. I’ve got some other research to do so I’m not pursuing this now. But if someone puts the right paper in my hands (via URL) I’ll look at it and possibly write about it.