• Dangerous faux research

    By now you’ve probably heard that McKinsey has refused to release details of its survey methodology that backs the finding that 30% of employers intend to drop coverage as a result of the ACA. By now you know that 30% is a dramatically higher figure than any other credible organization has suggested. It’s not what we’ve seen in Massachusetts. (On all this, see Aaron, Volsky, or Krugman).

    As someone who does research, this really bothers me. It should bother you too. Look, anybody can say what they like on a topic. They can put out a glossy report. They can claim they did a “survey” to make it sound scientifically rigorous. They can talk to the media all about it. They can stand behind their good name and reputation, if they have one. But when what they’re saying runs counter to previous experience and other credible estimates, they’d better have a good explanation.

    But, McKinsey has no explanation. None. They’re stonewalling. You know what would happen to me if I tried that? Suppose I sent my new results to a journal, results that were very different from that of others, and said, “Trust me. They’re good.” Well, my paper would be laughed out of the editorial office.

    And that’s as it should be. That would not be research. That would be the opposite of research. That would be indistinguishable from making things up. Well, anybody can make things up. The difference between making things up and actually doing some sound, rigorous work is the difference between fiction and reality.

    This is where things can get dangerous. If we are willing to accept something indistinguishable from fiction, someone, I am sure, will begin to tell us things that aren’t true. If you want to believe in fiction, fine. But you shouldn’t be surprised to find that the rest of us living in reality find your claims of the existence of fire-breathing, shape-shifting, unicorns to be incredible. You shouldn’t be surprised to be asked to deliver the photograph and the methods used to produce that photograph. If your sources are not willing to deliver that, then please don’t tell me how things will work out here in reality. If you don’t have the ability or willingness to provide the support for your claims, if you don’t even recognize that you should be able to do that, then I’m pretty sure you can’t even see reality from your vantage point.

    Share
    Comments closed
     
    • I think you spelled Fox wrong.

    • McKinsey and Company did a study and said they found that under the new health care law 30% of employers would drop employee health care coverage. McKinsey would not release the methodology, not even to the White House, because as The Dismal Political Economist has learned, McKinsey got its results mixed up and the 30% number is the percentage of future Medicare eligible recipients who would not have health care if the Ryan proposal went into effect.

      from

      http://dismalpoliticaleconomist.blogspot.com/2011/06/perry-to-run-for-president-or-leave.html

    • McKinsey seems to have taken a “postmodern” approach to research: facts are what you can make them.

      As one of George W. Bush’s senior aides (probably Karl Rove) told Ron Suskind:
      “Guys like you are in what we call the reality-based community — people who believe that solutions emerge from your judicious study of discernible reality. That’s not the way the world really works anymore. We’re an empire now, and when we act, we create our own reality. And while you’re studying that reality — judiciously, as you will — we’ll act again, creating other new realities, which you can study too, and that’s how things will sort out. We’re history’s actors…. and you, all of you, will be left to just study what we do.”

    • i read about this on WaPo’s Plum Line blog — and in the comment section there were links that shed more light on McKinsey

      Some of the more questionable strategies of McKinsey:
      http://www.ritholtz.com/blog/2011/03/is-mckinsey-co-the-root-of-all-evil/

      Enron is the house that McKinsey rebuilt.
      http://www.guardian.co.uk/business/2002/mar/24/enron.theobserver

    • actually, the mckinsey report grossly underestimated the number of people employees will drop as a result of the aca. what is uniformly missed by advocates as well as critics is one very important change in the law that will result in wholesale dropping of employee coverage. under current law, companies can only write of employee health care benefits if they cover their entire workforce. but under the new law, companies no longer have to comply with this rule, instead of covering everyone they will be able to cherry pick among their workforce, paying a greatly reduced penalty for dropped workers, without any loss of tax write-off for those still covered. strange (or convenient), that an economics blog missed the basic economics of this important change.

    • Okay, maybe I’m missing something here, but isn’t it sometimes possible to make some simple and non-controversial assumptions about human behavior, read a particular policy (that is based on those same assumptions), and then predict what the outcome will be? I find that in policy analysis, reasoning from non-controversial premises is far more effective than endlessly linking to graphs and statistics and “studies”—all of which can be easily manipulated.

      We know that employers are rationally self-interested, that they seek to maximize profit, and that they are not going to provide health insurance unless it’s in their best interest as a business to do so. We know that at present most businesses find it in their best interest to provide health insurance to their employees. We know that Obamacare (at the very least) radically alters the healthcare insurance landscape. Isn’t it reasonable to assume—completely a priori—that Obamacare could upset the current latticework of incentives employers have to provide health insurance?

      Obamacare places a 2k fine per employee per year on businesses that do not provide health insurance. Why would this have been included in the legislation unless the drafters were worried that other provisions of Obamacare would give employers an incentive to drop health insurance? Again, I don’t think I need an empirical study as “evidence” for this. Why would the fine be in there unless they were worried about employers dropping insurance?

      Then the question is: What parts of Obamacare give employers an incentive to drop insurance? I would say it’s the subsidies. Right now if I make, say, 40k and my employer provides me and my family with insurance that costs it, say, 5k annually, it’s effectively paying me 45k a year. Now if we assume that the government says that if my employer drops my insurance, it will pay a big chunk of that 5k—let’s say 3k of it—what will the employer do? It’ll drop my health insurance, and give me a 2k raise, which means I’ll cost them 42k a year instead of 45k. My compensation remains the same, but my employer saves money. Why wouldn’t they take this option? I don’t think we need some empirical study to show us that employers are rationally self-interested, right?

      So now we see the point of the 2k fine that Obamacare imposes on employers. The drafters knew that the federal subsidies would give employers an incentive to drop health insurance, so they had to include a fine—which was an attempt to create a counter-incentive for employers. But is 2k sufficient? Most health insurance plans cost more than 2k a year—so employers are going to have an incentive to drop the insurance, pay the fine, and then have the federal subsidies make up the difference. The fine would need to be up around 5k (I’m guessing) to have the desired effect.

      Why would the drafters have picked a fine that was obviously too low? Because if this fine is too high it would have made the legislation impossible to pass. As it is, the fine functions as a 2k tax on any job (new or existing) which does not include health insurance. That’s a “job-killing” incentive if there ever were one. You raise that to a 5k or 6k fine for every new low-end job created, and you’ve gone off the deep end. So, I think it’s reasonable to infer, the drafters left in a fine that is far too low for political reasons—just the sort of thing that happens when you have a sprawling committee putting together rules that would be more easily created by a dictator.

      Regardless, I think most first year law students could read a summary of Obamacare and predict that many employers (if not most) are going to drop their insurance. This isn’t difficult policy analysis. At the first least, I think the burden of proof should be on those claiming that employers will not drop health insurance.