The opioid epidemic is a multifaceted public health crisis characterized by the widespread misuse of and addiction to opioid substances. In the late 1990s and early 2000s, there was a significant rise in the prescription and use of opioid pain relievers. This surge, fueled by pharmaceutical marketing and other factors, contributed to a substantial increase in addiction, overdoses, and fatalities across the United States. To date, more than a million deaths are attributable to opioid overdoses within the last two decades.
With more than 10% of Veterans using opioids at some point in their lives, Veterans Health Administration (VHA) uses risk prediction models to estimate the likelihood of future opioid-related severe adverse events (SAEs). The primary goal of risk prediction scores is to identify Veterans at higher risk of SAEs, initiate case reviews, and take proactive, personalized measures to support their wellbeing.
However, risk prediction models are not perfect and could lead to racially biased recommendations, such as some individuals being more or less likely to receive case reviews.
New Research
In September 2023, health services researchers from the Partnered Evidence-based Policy Resource Center (PEPReC) published a paper titled “Differences in adverse outcomes across race and ethnicity among Veterans with similar predicted risks of an overdose or suicide-related event” in Pain Medicine. The paper assessed prediction disparities within VHA’s risk opioid prediction model called STORM (Stratification Tool for Opioid Risk Mitigation), evaluating differences in SAEs and mortality across race and ethnicity for Veterans with similar STORM risk scores.
Methods
The STORM model is designed to predict the risk of overdose, suicide-related events, and death in Veterans receiving outpatient opioid prescriptions. It incorporates various risk factors such as demographic information, history of overdose or suicide-related events, opioid dosage, and mental health diagnoses.
The study used VHA data from April 2018 to March 2019, focusing on a sample of 84,473 Veterans with STORM risk scores within the top 10% to 5%. (Veterans in the top 5% could have been automatically targeted for a case review, which would have confounded analyses. As such, they were excluded from this study.) The racial and ethnic breakdown of the Veterans in the sample was: 65% White; 20% Black; 8% Hispanic; 1% with more than one race; 0.8% American Indian or Alaska Native; 0.6% Native Hawaiian or Other Pacific Islander; and 0.5% Asian. Of the sample, 3.9% did not have race or ethnicity recorded.
The authors observed Veteran outcomes for six months after entry into the cohort, documenting changes in their STORM scores. They then categorized Veterans based on race and ethnicity while analyzing SAEs and all-cause mortality. Logistic regression models were used to understand the relationship between health outcomes and risk scores, controlling for race and ethnicity, VHA medical facility, age, and sex.
Additional post hoc analyses were conducted to understand potential sources of observed differences. These further explored mortality across different age groups (under 65 versus over 65) and assessed differences in SAEs and healthcare use among racial and ethnic groups in the pre-study time period.
Findings
The authors found racial and ethnic differences in health outcomes despite Veterans having similar STORM risk scores. When examining SAEs or mortality within six months of receiving a STORM risk score, Black Veterans were less likely than White Veterans with similar scores to have a recorded SAE. Black, Hispanic, and Asian Veterans had lower odds of mortality compared to White Veterans with similar STORM risk scores. Even after considering VHA medical facility differences, age, and sex, these differences persisted.
Among Veterans aged 65 and over, both Black and Hispanic Veterans had lower mortality odds than White Veterans with similar STORM risk scores. However, among those under 65, only Hispanic Veterans had significantly lower mortality odds than White Veterans. The authors also explored the impact of other factors that could influence outcomes, such as past SAE rates and past inpatient mental health treatment or emergency department use. White Veterans experienced more SAEs than Black Veterans but fewer than multiracial and Asian Veterans, but emergency department use was highest among Black Veterans and lowest among Native Hawaiian or Other Pacific Islander Veterans.
Conclusion
One limitation of the study is its sole reliance on VHA-provided data. With non-Hispanic White Veterans more likely than other Veterans to seek care outside of VHA, it’s possible that the authors determined a conservative estimation in differences between SAEs. The study is also limited in a full understanding of outcomes. The authors did not evaluate Veteran/provider use or response to risk reduction methods, so they were unable to account for how effective interventions are, or the overall accuracy of the score post-intervention.
The study’s findings of differences in outcomes across race and ethnicity suggest that the STORM model alone may not equitably identify Veterans who could benefit from case reviews. These differences might be linked by a number of factors. Various patterns of substance use, different ways of managing pain and opioid use disorder, and differences in how various racial and ethnic groups access care are all aspects to consider.
The study also highlighted potential built-in biases in predictive models like STORM. As a matter of policy, evaluating calibration bias and label choice bias could improve risk prediction models. VHA is working to combat these issues within STORM, and ongoing efforts will hopefully strengthen the accuracy, responsiveness, and ultimately care Veterans receive.
STORM is an important risk prediction model used by VHA to keep Veterans who use opioids safe and would benefit from continued review and evaluation.
