As a prelude to Aaron’s upcoming series on health care quality (begins Monday), I want to come back to something I wrote in August about descriptive evidence.
It’s not that I […] think qualitative and descriptive analyses answer questions any more definitively than do the complex multivariate approaches I normally consider. It’s that qualitative and descriptive analyses are incredibly helpful in revealing potential relationships, suggesting hypotheses, and relatively quickly and easily revealing the possible broad nature of and context for an issue.
Moreover, less econometrically or statistically complex analyses are more easily communicated to thinkers outside the field, like policymakers, journalists, and beekeepers. That’s important, and I have no problem using results of simpler analytical techniques for policy advocacy so long as the qualitative conclusions they suggest are consistent with more rigorous, but complex and less broadly accessible, approaches.
Now, when it comes to international comparisons of health care quality, I’m aware of no convincing body of evidence–statistically complex or otherwise–that suggests the U.S. generally achieves high quality outcomes relative to other wealthy nations. To my knowledge, and Aaron’s, every serious scholar of our health care system believes that U.S. health care quality, as measured by health outcomes, is poor to mediocre compared to other wealthy nations, particular considering how much we spend.
As evidence that we’re not alone, in a recent literature review of the scholarly work on international comparisons of health quality, Elizabeth Docteur and Robert Berenson concluded,
Taken collectively, the [evidence] from international studies […] does not provide support for the oft-repeated claim that the “U.S. health care is the best in the world.” In fact, there is no hard evidence that identifies particular areas in which U.S. health care quality is truly exceptional.
Instead, the picture that emerges from the information available on technical quality and related aspects of health system performance is a mixed bag, with the United States doing relatively well in some areas — such as cancer care — and less well in others — such as mortality from conditions amenable to prevention and treatment. […]
In the light of the fact that the United States spends twice as much per person on health care as its peers, those who question the value for money obtained in U.S. health expenditures are on a firm footing. […]
[A] less-than-fully informed public comes at a cost in that assertions of excellence divert attention from the need to inspire and foster systematic quality improvement activities. Furthermore, there seems to be a routine genuflection to the widespread belief of U.S. quality excellence. […] In an environment where even insured Americans receive only about half of the services that experts consider necessary, there is a strong argument for placing quality firmly on the health reform agenda.
Since the level of U.S. health care quality is widely misunderstood, it is critical to illustrate the quality of the U.S. health system relative to its peer nations in a fair, yet accessible way. In some sense these objectives are at odds. More sophisticated, multivariate analysis may be “more fair,” but also less accessible. Therefore, as you will see in Aaron’s series on quality, he will display data in a relatively simple way, but he will also go to some pains to control for some of the things that may confound interpretation.
For instance, he’ll show trends over time–as he did in a recent post on infant mortality and another on mortality in general–so that if one believes that countries differ in some important but time-invariant or only slowly-varying ways (e.g. demographics), one can focus on the rate of change of quality measures. That is, countries serve as their own controls. For some outcome measures there are reasons to think effects should differ by sex and/or age, making some sub-analyses more compelling than others. Thus, Aaron will stratify in those dimensions where possible and meaningful.
This won’t satisfy everyone. Even the most thorough analysis with the most sophisticated controls isn’t perfect. If you are desperate to believe our health system is the best in the world, you can find a reason to critique every analysis in this area, whether by Aaron, me, or others. Ultimately, we, and other researchers, are constrained by the data and resources. We do what we can. We wish we could do better. (Send money, lots of it, and we will.) We admit that one cannot draw strong conclusions from just one figure. Thats why in his series on quality Aaron will be showing dozens of them, on many types of measures. He won’t cherry pick. In some cases the U.S. really is the best.
Finally, the charts Aaron will present simply illustrate facts about relationships that exist in the data, independent of whatever conclusions one can or cannot reach. Thus, at a minimum, they raise the idea that the U.S. may not have the best quality health care, even if they cannot prove it. That’s important, particularly because it is so misunderstood. Most who have studied the U.S. health system believe we can do better on quality. If a set of relatively simple descriptive figures helps promote a focus on doing so, that’s good!
