What’s Convincing Evidence?

Robin Hanson’s response to my post is interesting. He writes,

More likely than not, medicine is on average is useless or harmful on the margin; that is my best reading of the evidence.  When I try to persuade folks, I start with our single best data point, the old RAND experiment, and people complain it was too small, short, and long ago (and it let folks leave too easily). …

Without a single very big very clear study there is simply no way to convince most folks that marginal medicine is on average useless, and we should cut way back.

That old RAND health insurance experiment (HIE) was a single, big study, and Hanson is finding it hard to convince people of its merit. But he thinks an even bigger one will be more convincing. I’m very skeptical on that point. Given that, I think the opportunity cost of a half-billion dollar ten-year repeat of the RAND HIE is too high.

Notice I’m not questioning the results of the RAND HIE. Let’s just assume for the sake of this debate that I agree with Hanson on its interpretation. In fact, what we could be debating is how he and I, together and in agreement, could convince others of our jointly held point of view.* He wants to spend a half-billion and ten years on another RAND HIE. I want to fund about 1,000 roughly two-year observational studies.

Anybody want to put up a billion dollars? Hanson and I will split it, fund our studies, and then see how many people we each can convince. I would hope to be able to say things like, “In a review of 1,000  studies conducted over the last few years by different scholars using different data and methods all addressing the marginal value of medical care, the vast majority found X.” He’d be able to say things like, “The largest randomized study ever conducted on the topic found X.” Then critics would begin to attack the studies. He’s got one massive target. I’ve got 1,000. I like my odds.

* Let me be clear. I’m not committing to sharing Hanson’s view on medical care or the RAND HIE. To make that commitment we’d have to have a much longer discussion. But that’s not important for this debate. For this one, let us presume I agree with him except on the question what study/studies to do to convince people of what he believes.

Hidden information below


Email Address*