By now you’ve probably heard that McKinsey has refused to release details of its survey methodology that backs the finding that 30% of employers intend to drop coverage as a result of the ACA. By now you know that 30% is a dramatically higher figure than any other credible organization has suggested. It’s not what we’ve seen in Massachusetts. (On all this, see Aaron, Volsky, or Krugman).
As someone who does research, this really bothers me. It should bother you too. Look, anybody can say what they like on a topic. They can put out a glossy report. They can claim they did a “survey” to make it sound scientifically rigorous. They can talk to the media all about it. They can stand behind their good name and reputation, if they have one. But when what they’re saying runs counter to previous experience and other credible estimates, they’d better have a good explanation.
But, McKinsey has no explanation. None. They’re stonewalling. You know what would happen to me if I tried that? Suppose I sent my new results to a journal, results that were very different from that of others, and said, “Trust me. They’re good.” Well, my paper would be laughed out of the editorial office.
And that’s as it should be. That would not be research. That would be the opposite of research. That would be indistinguishable from making things up. Well, anybody can make things up. The difference between making things up and actually doing some sound, rigorous work is the difference between fiction and reality.
This is where things can get dangerous. If we are willing to accept something indistinguishable from fiction, someone, I am sure, will begin to tell us things that aren’t true. If you want to believe in fiction, fine. But you shouldn’t be surprised to find that the rest of us living in reality find your claims of the existence of fire-breathing, shape-shifting, unicorns to be incredible. You shouldn’t be surprised to be asked to deliver the photograph and the methods used to produce that photograph. If your sources are not willing to deliver that, then please don’t tell me how things will work out here in reality. If you don’t have the ability or willingness to provide the support for your claims, if you don’t even recognize that you should be able to do that, then I’m pretty sure you can’t even see reality from your vantage point.
