• Oregon Health Study results, part 1

    Officially I’m on vacation, but I’m taking a break to write briefly about the the NBER-published first set of results from the Oregon Health Study, which are out today. The study has a randomized design and examines the difference in health care utilization and some self-reported outcomes for those on Medicaid vs. the uninsured. It is authored Katherine Baicker, Amy Finkelstein, Jonathan Gruber, Joseph Newhouse, Sarah Taubman, Bill Wright, Mira Bernstein, Heidi Allen and members of the Oregon Health Study Group.

    Just to refresh your memory, here’s what I wrote about that study in August 2010,

    Not since the RAND Health Insurance Experiment (HIE) has there been a randomized controlled experiment of the effect of insurance on health outcomes. Finally, a second one is underway, the Oregon Health Study (OHS). It’s being conducted by Heidi Allen, Katherine Baicker, Amy Finkelstein, Sarah Taubman, Bill J. Wright, and the Oregon Health Study Group who report on the study design in the most recent edition of Health Affairs.

    [T]he Oregon Health Study [is] a randomized controlled trial that will be able to shed some light on the likely effects of [Medicaid] expansions. In 2008, Oregon randomly drew names from a waiting list for its previously closed public insurance program. Our analysis of enrollment into this program found that people who signed up for the waiting list and enrolled in the Oregon Medicaid program were likely to have worse health than those who did not. However, actual enrollment was fairly low, partly because many applicants did not meet eligibility standards.

    Get excited! But don’t get too excited. The study runs through 2010 and no outcome results are available yet.

    Some results are available now. They’ve been reported by Ezra Klein, David Leonhardt, and Gina Kolata. Compared to the uninsured group, those in the Medicaid group:

    • received 30% more hospital care,
    • received 35% more outpatient care,
    • were 15% more like to use prescription drugs,
    • received 60% more mammograms,
    • received 20% more cholesterol checks,
    • were 15% more likely to have had a blood tested for high blood sugar or diabetes,
    • were 45% more likely to have had a pap test within the last year (for women),
    • had lower out-of-pocket medical expenditures and medical debt,
    • had a 40% lower probability of needing to borrow money or skip payment on other bills because of medical expenses,
    • incurred $778 more in spending on health care in one year, a 25% increase over the uninsured mean spending level,
    • were 25% percent more likely to report themselves in “good” or “excellent” health,
    • were 70% more likely to have a usual source of care,
    • were 55% more likely to see the same doctor over time,
    • reported better physical and mental health,
    • were 10% percent less likely to screen positive for depression.
    Klein mentions that “there was no evidence of “crowd-out”: Medicaid coverage didn’t make someone more or less likely to purchase private insurance.” One would hope that all of this would also lead to other objective measures of better health. However, a fuller examination of health outcomes are left for part 2 of the study, for which there are no results yet.

    I expect the research team will find that Medicaid does lead to better health. Such a finding would be consistent with some of the results above (better self-reported physical and mental health, less likely to be depressed, the myriad of higher propensity for preventative tests and treatment). It would also be consistent with a body of other evidence summarized on this blog (see the Medicaid-IV tagged posts) and in a NEJM paper by me, Aaron Carroll, Harold Pollack, and Uwe Reinhardt.

    In looking at the NBER paper, I see that the team used an IV method that addresses any potential bias due to unobservable differences between treatment and control groups. They used the selection by lottery as an instrument. It is clearly unrelated to outcomes but tightly related to placement in treatment and control groups. It’s a perfect instrument and an accepted, proven way to handle contamination or other unobservable differences between treatment and control groups in a ranomized design (see Angrist and Pischke).

    I look forward to reading subsequent work from the Oregon Health Study.

    UPDATE: Removed a paragraph on potential treatment-control bias based on feedback from study authors.

    • It’s great to see you took a vacation break to cover this, but I do have one complaint about the post:

      Where’s the excitement!? Not a single exclamation point!

      This is the first in what will be a classic series of papers, to be cited for decades. No other study has been able to examine the effect of receiving insurance on health in a randomized contexts and, to boot, on such a policy-relevant population. I love all the IV papers that have tackled the subject, but this paper towers above.

      I haven’t been in this field long, but I have never felt so excited to read a new health economics paper as I was this morning.

      • Forgive me. I thought my quote of my prior “Get excited!” would be sufficient. 🙂 Also, this was a rushed post. I don’t really have time to work and write while on vacation. My kids need me, and I want to be with them. You might find my Twitter feed more exciting. @afrakt

        • Enjoy your vacation! I didn’t mean to criticize your taking family time, of course. The “complaint” was meant to be tongue-in-cheek, so forgive me 🙂

    • These are all quite positive:

      were 25% percent more likely to report themselves in “good” or “excellent” health,
      reported better physical and mental health,
      were 10% percent less likely to screen positive for depression.

      The rest seem unimportant.

      I think that from my perspective it is good that medicaid seems to work well in Oregon (though it is not proven yet we need the more objective measures). If we could eliminate the high effective marginal tax situation that medicare can cause it an expanded medicaid could be the only Gov. intervention for those below 65.

      I wonder if this “were 10% percent less likely to screen positive for depression” means they were less depressed because they are not worried about medical bills or that they are taking anti-depression meds.

      • What are the magnitudes of these? 0.00001 versus 0.00002? Wow! Twice as many! Statistical significance! Does 40% actually tell you anything?

        were 10% percent less likely to screen positive for depression.

        What’s the *!@#$% magnitude? 10 out of 100 uninsured were depressed, and 9 out of 100 insured were depressed. They probably provided care from psychiatrist and medications to get that one, lone person out of a hundred a microscopic improvement. Even then, it was probably a 1 or 2 point reduction in 60 point MADRS.

        I love it! were 25% percent more likely to report themselves in “good” or “excellent” health, Compared to what? 4 out of 100 without insurance, and 5 out 100 with?

        What is the NNT to get these numbers? How many uninsured must be given insurance before a single benefit from that insurance can be seen?

        What does our host desperately, excitedly, and passionately want the truth to be? I’d be embarrassed to be foaming at the mouth like this, but when you have an agenda and bias that you have no shame expressing, all bets are off.

    • But compared to the uninsured group, do those in the Medicaid group actually have a lower death rate?

      • Michael F. Cannon writes:

        What about health? Though the president has claimed his health-care law will “save lives,” the OHIE detected no evidence that extending Medicaid to 10,000 adults did so in the first year. On one hand, we might not expect to see any effect just one year into the experiment, since mortality rates among adults aged 19 to 64 are relatively low. On the other hand, this finding is consistent with a previous study, coauthored by one of the OHIE researchers, that found no evidence that Medicare (which covers a much older and sicker population) saved any lives even ten years after its introduction. (In future years, OHIE researchers will be able to report on other objective measures of health such as blood pressure and cholesterol levels.)

    • I’m kind of interested to see the paragraph that was removed, as well as what they said to convince you to remove it…

    • Here’s one “surprising” result in case you want one:

      “There is no discernible impact of insurance on emergency room or inpatient hospital use on either margin…” (pg. 22).

      That +35% outpatient care is not coming out of their ER utilization.

      Thanks for the prompt and thoughtful coverage. Excited to see how this develops.

    • Heard Finkelstein interview on way to work this morning. I hate to get excited over just one study, but this does sound like an important well done one. Hope that we can harvest lots of good stuff out of this data base.


    • A rather obvious question that I haven’t seen anyone ask: what was the control group given? If we gave the control group an equivalent amount of cash we could compare the effect of being on Medicaid. If they didn’t give them anything this seems to me to just prove that if you give people some free health care they will use it. But that doesn’t mean it’s an efficient or effective use of the money you spend. Pick people randomly and give them either Medicaid or cash. This feels more like “if you give people $5000 their health care utilization increases” than any kind of proof of the efficacy of Medicaid.

      • I think this is far different from giving people $5,000. It’s giving people access to health care by lowering the cost of care. One could imagine giving cash instead, but one can’t know how that cash will be spent, which to some is a virtue, others a weakness. “Efficient and effective” is not well-defined by you. Even if you define it, someone else need not accept your definition.

        In any event, the average increase in the dollar value of care was $778, as I stated. We’re not talking $5,000, but a small fraction of that. This is a modest cost for significant increases in use of preventative screening and services. $778 is one measure of its value, the cost to taxpayers. Some only see that.

        Another measure of the value will be in its effect on objective health outcomes for a low-income population (beyond depression diagnoses, which were reported in this study). Those results are forthcoming. Past work and the current set of results strongly suggests what they will be, at least qualitatively. We shall see.

      • Thanks, that’s the dollar amount I was looking for, it wasn’t clear to me at first. So give the uninsured group $778. Some if it may get spent on health care, some would not. By “efficient and effective” I simply mean how well does the program meet its goals compared to an alternative. Those goals could just be the measures you cited above. But to properly understand how well it does on those I think you need to give the control group money, otherwise you’ve made the test group wealthier and it’s no longer a fair comparison. What was the average income of those involved? The typical Medicaid recipient is pretty poor, $778 is likely a fairly significant increase in income.

        And have we gotten value for the money? Are the increases in preventive screening a worthwhile cost? Much preventive care is not cost-effective. Was the 60% increase in mammograms among the group over age 50 or those at high risk, following the recommendations of the USPTF? Was the increase in prescription drug usage an increase in necessary and cost-effective drugs? These are the things I’d like to know, just showing an increase in utilization when you give some access to free or subsidized care is a rather obvious result that doesn’t tell us much about whether the spending was wise.

        An illustration: many poor people don’t have reliable access to transportation, which limits their ability to improve their lives by seeking new employment or consistently getting to their current job, getting to a doctor, etc. Let’s say we gave a test group free access to a car-sharing service, and average miles driven increased 50%. Would that on it’s own be a good thing? Wouldn’t we want to know if the test group had an increase in making it to work on time, finding new jobs, or getting to appointments, or were they just using it to drive the movies once a week, or to go to the store that is 5 blocks away?

    • we need studies to prove that it makes sense to provide people with health care?

      the advantages of access to health care is one of those things that do not require a “scientific” study to make sense

      what’s the purpose of this study?

      • yeah, this whole “we still have to study whether providing health care to everybody is a good thing or not” ranks right up there with “evolution is just a theory”.

    • The results from the Oregon study confirm what everyone knows and 100s of studies have shown — providing an uninsured person with insurance will increase their use of health acre services. Yes Virginia, the demand curve does slope down! The “spin” surrounding this article, that it refutes arguments that Medicaid is worse than being uninsured, is just that spin. The real question is whether giving a person the right to unlimited free health care is a cost-effective way to improve health. The Oregon study is completely silent on this issue. Because of some recent research of my own, I found the fact that inpatient care went up by 30% at the same time outpatient care increased dramatically an ominous sign for supporters of recent health care reform. A often heard claim of proponents is that health insurance will result in more and more timely primary care and keep people out of the hospital. Obviously this is not the case, as the Oregon study and several others show. The substantial increase in use of services resulting from insurance will have to result in significant improvements in health to make it cost-effective. Previous evidence suggests this extra use is unlikely to achieve that goal.

    • I hope you enjoyed your vacation. When you return, take a look at the report and see if you still think it

      “… examines the difference in health care utilization and some self-reported outcomes for those on Medicaid vs. the uninsured.”

      In reading the methodology section and Table 1 in particular, it reads to me that the study compares two sets:
      — 30,000 out of a group of 90,000 that were selected to apply for the Medicaid Lite product (something like Massachusetts’ Commonwealth Care), whether or not they actually were among the 10,000 that evenutally enrolled in the product AND EVEN WHETHER OR NOT THEY EVEN QUALIFIED FOR THE PRODUCT. (That is, in Massachusetts’ terms, even if they would not have qualified for Commonwealth Care.)
      — 30,000 out of the remainder of the 90,000 (roughly every other person whose name was not selected in the lottery), whether or not they actually had other insurance AND ALSO WHETHER OR NOT THEY EVEN QUALIFIED FOR THE PRODUCT

      If I’m reading it right, this is not a comparison of Medicaid vs. the uninsured. There appears to be insured individuals in both sets.

      It IS clearly an argument for insurance, taking the authors’ word for it that there is a statistical difference between the number of insured in the two groups. (I can’t find that data yet but I assume it is there?).

      • Keep reading. They do insured vs. uninsured models using the lottery as an instrument to control for unobserved selection and response bias. This is a very strong and standard econometric design fro RTCs.

    • ” The real question is whether giving a person the right to unlimited free health care is a cost-effective way to improve health.”

      Medicaid is cheaper than private insurance and Medicare.


      • It’s cheaper because reimbursements are so low. Many docs won’t accept new Medicaid patients, and a portion of the revenue lost from the low reimbursements is shifted to private insurance. Can you dramatically expand Medicaid and have reasonable quality and availability of care for those people? Could you give the people who you want to add to Medicaid an equivalent amount of cash, or a subsidy to buy insurance in the exchange, and get an equal or better expansion of coverage, as measured by access to quality health care?

        It’s drastically oversimplifying things to just say “Medicaid is cheap”. That cheapness comes at a cost, there is no free lunch.