Hospitals that perform a greater volume of a specific procedure do it better, right? Well, actually, that’s believable and supported by evidence. Now for the harder question: it’s the greater volume that causes the better outcomes, right? You know, “learning by doing,” “practice makes perfect,” etc. That’s believable too. But it’s not as empirically verifiable.
In fact, there’s good reason to believe causality runs the other way too. Hospitals that yield better outcomes have higher volume, a referral effect. Ask 100 physicians in your area where to have a CABG and the results won’t be random. They’ll point you to the well-known facility or two that do the best job, perhaps with the lowest mortality. So, more patients will go to those, increasing their volume. Low mortality causes higher volume. That’s not “practice makes perfect.” That’s a referral effect.
So, outcomes (like mortality) and procedure volume are likely to be simultaneously determined (causality runs both ways). How much is “practice makes perfect,” and how much is “selective referral”? We don’t have to guess. Let’s hit the literature.
A recent paper in Health Economics by David Barker, Gary Rosenthal, and Peter Cram (ungated working paper available) provides one route to an answer and includes a literature review that provides others. They examine the relationship between volume and mortality for cardiac revascularization in specialty hospitals versus general hospitals. After controlling for the simultaneity of volume and mortality, they conclude that “specialty hospitals do not have an advantage over general hospitals in mortality rates after cardiac revascularization.” Moreover, they find that mortality rates do influence volume. Therefore, efforts to increase volume may not themselves increase the quality of outcomes. Volume isn’t exogenous.
Here’s their literature review:
The pioneering study of the relationship between volume and outcome (Luft et al., 1979), inspired by the economics literature on industrial learning curves, reported a strong correlation between volume and outcome. It mentioned, however, the possibility that the causal relationship might be from outcome to volume, rather than the other way around. In a second study, Luft (1980) explored the directionality of this relationship using the econometric technique of two-stage least squares and confirmed this concern. He found evidence suggesting that the effect of outcome on volume (the ‘selective referral’ hypothesis) was stronger than the effect of volume on outcome (the ‘practice makes perfect’ hypothesis).
A later study (Hughes et al., 1988) found evidence that the relationship was strong in both directions. Another study (Luft et al., 1990) found that hospital quality affected patient’s hospital choice even before explicit data on quality were available, supporting the relevance of the ‘selective referral’ hypothesis.
Although failure to take account of both hypotheses has the potential to bias the results of studies of hospital volume, very few studies in mainstream medical research journals demonstrate awareness of the bi-directional volume–outcome relationship. Some have addressed the issue using longitudinal data. Hannan et al. (1992, 1995) found evidence supporting both directions of causality. Hamilton and Ho (1998) and Hamilton and Hamilton (1997) found no evidence supporting the ‘practice makes perfect’ hypothesis after controlling for fixed quality effects, supporting the ‘selective referral’ hypothesis.
Studies using cross-sectional data and correcting for possible simultaneity are even less common than longitudinal studies. Other than the work of Harold Luft, we are aware of only three studies that explicitly test or correct for simultaneity. Farley and Ozminkowski (1992), using longitudinal data and two-stage least squares, found that the ‘practice makes perfect’ effect disappeared for coronary artery bypass grafting (CABG) once the ‘selective referral’ effect was controlled for, although the effect remained for some other procedures. Norton et al. (1998) found a t-statistic of only 1.80 in a secondstage regression and concluded that they were unable to reject exogeneity, and hence a correction for simultaneity was not required. This conclusion was criticized as premature by Luft (1998a,b), who argued that a t-value that approached significance should have led the authors to explore additional model specifications to rule out endogeneity. Tsai et al. (2006) used patient distance from hospitals to construct an instrument for volume. After adjusting for simultaneity, the ‘practice makes perfect’ effect became statistically insignificant.
Although ‘selective referral’ is ignored in the vast majority of studies of volume and outcome, there is considerable evidence that it should be taken into account. Failure to take simultaneity into account does not, of course, call into question the correlation between volume and outcome. As Luft (2003) states, if he were injured in an unfamiliar city, he ‘would ask the ambulance driver to avoid taking me to the low-volume facility.’ However, correlation does not imply causation; hence policymakers deciding whether to regionalize care require more sophisticated statistical analysis than do patients deciding where to obtain care.
That final paragraph makes two very good points. First, volume is informative about quality. Second, volume does not necessarily produce quality. Keeping those in mind, one should be very careful about whether and how to influence volume. The authors note elsewhere,
[I]f ‘selective referral’ explains the relationship between volume and outcome, then regionalization could actually worsen outcomes if the wrong hospitals are chosen for expansion. The benefits of regionalization must also be weighed against the cost of transporting patients over long distances (Birkmeyer et al., 2003), and hence, knowing the magnitude of the ‘practice makes perfect’ effect is important. Failure to take account of the ‘selective referral’ effect will tend to exaggeratethe estimated ‘practice makes perfect’ effect, biasing policy recommendations toward regionalization.
References
Omitted because of the existence of an ungated working paper. Most or all references should be found there. Of course, they’re all in the gated version too.
