The following originally appeared on The Upshot (copyright 2019, The New York Times Company) and on page A13 of the print edition on September 10, 2019.
A few years ago, Oregon found itself in a position that you’d think would be more commonplace: It was able to evaluate the impact of a substantial, expensive health policy change.
In a collaboration by the state and researchers, Medicaid coverage was randomly extended to some low-income adults and not to others, and researchers have been tracking the consequences ever since.
Rigorous evaluations of health policy are exceedingly rare. The United States spends a tremendous amount on health care, but very little of it learning which health policies work and which don’t. In fact, less than 0.1 percent of total spending on American health care is devoted to evaluating them.
As a result, there’s a lot less solid evidence to inform decision making on programs like Medicaid or Medicaid than you might think. There is a similar uncertainty over common medical treatments: Hundreds of thousands of clinical trials are conducted each year, yet half of treatments used in clinical practice lack sound evidence.
As bad as this sounds, the evidence base for health policy is even thinner.
A law signed this year, the Foundations for Evidence-Based Policymaking Act, could help. Intended to improve the collection of data about government programs, and the ability to access it, the law also requires agencies to develop a way to evaluate these and other programs.
Evaluations of health policy have rarely been as rigorous as clinical trials. A small minority of policy evaluations have had randomized designs, which are widely regarded as the gold standard of evidence and commonplace in clinical science. Nearly 80 percent of studies of medical interventions are randomized trials, but only 18 percent of studies of U.S. health care policy are.
Because randomized health policy studies are so rare, those that do occur are influential. The RAND health insurance experiment is the classic example. This 1970s experiment randomly assigned families to different levels of health care cost sharing. It found that those responsible for more of the cost of care use far less of it — and with no short-term adverse health outcomes (except for the poorest families with relatively sicker members).
The results have influenced health care insurance design for decades. In large part, you can thank (or curse) this randomized study and its interpretation for your health care deductible and co-payments.
More recently, the study based on random access to Oregon’s Medicaid program has been influential in the debate over Medicaid expansion. A state lottery — which provided the opportunity for Medicaid coverage to low-income adults — offered rich material for researchers. The findings that Medicaid increases access to care, diminishes financial hardship and reduces rates of depression have provided justification for program expansion. But its lack of statistically significant findings of improvements in other health outcomes has been pointed to by some as evidence that Medicaid is ineffective.
