The following are quotes from Risk Prediction Models for Hospital Readmission A Systematic Review, by Devan Kansagara, et al. (JAMA, 2011):
- Fourteen models were based on retrospective administrative data and could potentially be used for hospital comparison purposes. Most of these included variables for medical comorbidity and use of prior medical services, but a few considered mental health, functional status, and social determinant variables.
- The 9 large population-based or multicenter US studies generally had poor discriminative ability (c statistic range: 0.55-0.65) [*]. The CMS used a methodologically rigorous process to create 3 models for congestive heart failure, acute myocardial infarction, and pneumonia admissions based on hierarchical condition categories, which are groups of related comorbidities.[14-16] All 3 models showed relatively poor ability to predict 30-day all-cause readmissions (c statistics: 0.61 for congestive heart failure, 0.63 for acute myocardial infarction, and 0.63 for pneumonia). A recent study evaluating theCMS heart failure model and an older heart failure model fared similarly (c statistics: 0.59 and 0.61, respectively).[18,23] The other 4 US models have limited generalizability; for example, one model captured readmissions to 1 medical center only,[24] and the other models were developed more than 2 decades ago.[12,22,25]
- [I]mportant gaps in model development in that few studies considered variables associated with illness severity, overall health and function, and social determinants of health.
- Only 1 model attempted to explicitly define and identify potentially preventable readmissions.[46]
- Readmission risk prediction remains a poorly understood and complex endeavor.
- Although the inclusion of [certain] hospital level factors would conceivably improve the predictive ability of models, it would be inappropriate to include them in models that are used for risk standardization purposes. Doing so would adjust hospital readmission rates for the very deficits in quality and efficiency that hospital comparison efforts seek to reveal, and which could be targets for quality improvement interventions.
- Until risk prediction and risk adjustment become more accurate, it seems inappropriate to compare hospitals in this way and reimburse (or penalize) them on the basis of risk-standardized readmission rates. Others have reached similar conclusions, [55] and also have expressed concern that such financial penalties could exacerbate health disparities by penalizing hospitals with fewer resources.[56] Still others have argued that readmission rate is an incomplete accountability measure that fails to consider “the real outcomes of interest—health, quality of life, and value.”[57]
- A recent systematic review of 34 studies found wide variation in the percentage of readmissions considered preventable and estimates ranged from 5% to 79% (median, 27%).[58]
- Even though the overall predictive ability of the clinical models was poor, we did find that high- and low-risk scores were associated with a clinically meaningful gradient of readmission rates.
Comment: There are at least two uses for readmissions prediction models. One is to risk adjust for quality measurement reporting or payment. In this case the question is, what readmission rate should this hospital have given the types of patients it sees and other factors it cannot change? Low predictive power is a concern here, but one should not expect very high predictive power either. A lot of factors relevant to readmission are omitted from such models by design. Specifically, one should not control for those that are modifiable by the hospital. So long as the model is unbiased, it is still of potential use.
Another use of readmissions prediction models is by hospitals to assess the risk of a patient for readmission. Is this patient at high risk? If so, what can we do to lower it? In this case, factors that the hospital can change are important to include in the model and we should expect greater power from it. However, some of that power had better come from modifiable factors. To the extent that factors within hospitals’ control do not alter readmissions much, then there is hardly any value in penalizing them for lower than expected rates.
It is, of course, a judgement call as to whether hospitals can or could modify care and care transitions enough to warrant the types of readmissions payment incentives now in place. Kansagara, et al. suggest that it is not evident they can and that payment incentives are premature.
* The c statistic ranges from 0.5 to 1.0. It is the area under the receiver operating characteristic. Higher values indicate greater predictive power.
