Via Justin Wolfers:
Most people feel that the negative externality that anti-vaxxers impose on society—endangering those who cannot be vaccinated and threatening loss of herd immunity—warrants some government coercion to vaccinate. Yet, in many other instances in which externalities arise, including due to lack of purchase of health insurance, government coercion not as widely accepted. Why the difference? I discuss over on the JAMA Forum.
There’s a federal organization that makes evidence-based, national health care coverage decisions potentially affecting tens of millions of Americans. Its decisions even seem to be related to the presence of cost-effectiveness information (though not the level of cost-effectiveness itself). Can you guess what it is? Check your answer in my new AcademyHealth post.
A nearly identical version of the following originally appeared on The Upshot (copyright 2015, The New York Times Company). In this version, I have made a few, minor stylistic changes.
Do Alcoholics Anonymous participants do better at abstinence than nonparticipants because they are more motivated? Or is it because of something inherent in the A.A. program?
How researchers answered these questions in a recent study offers insight into challenges of evidence-based medicine and evidence-informed policy.
The study, published in the journal Alcoholism: Clinical and Experimental Research, teased apart a treatment effect (improvement due to A.A. itself) and a selection effect (driven by the type of people who seek help). The investigators found that there is a genuine A.A. treatment effect. Going to an additional two A.A. meetings per week produced at least three more days of alcohol abstinence per month.
Separating treatment from selection effects is a longstanding problem in social and medical science. Their entanglement is one of the fundamental ways in which evidence of correlation fails to be a sign of causation. For many years, researchers and clinicians have debated whether the association of A.A. with greater abstinence was caused by treatment or a correlation that arises from the type of people who seek it.
Such confounding is often addressed with an experiment in which individuals are randomly assigned to either a treatment or a nontreatment (or control) group in order to remove the possibility of self-selection. The treatment effect is calculated by comparing outcomes obtained by participants in each group. Several studies of A.A. have applied this approach. For instance, Kimberly Walitzer, Kurt Dermen, and Christopher Barrick randomized alcoholics to receive treatment that strongly encouraged and supported A.A. participation or a control group. The former exhibited a greater degree of abstinence.
In an ideal randomized controlled trial (R.C.T.), everyone selected for treatment receives it and no one in the control group does. The difference in outcomes is the treatment effect, free of bias from selection.
That’s the ideal. However, in practice, randomized controlled trials can still suffer selection problems.
It’s one thing to assign individuals to treatment or control. It’s another to compel them to stick to the group to which they’re assigned. In many studies, researchers cannot. For instance, what’s to stop an individual assigned to the non-A.A. group (the control group) from attending A.A. meetings? Or, what forces those in the treatment group to attend them? Nothing.
A real-world trial has what is known as crossover — people not sticking to their random assignment. It can occur, for instance, if less motivated or sicker people stop adhering to treatment. Or, perhaps, more motivated ones find a way to receive treatment even when assigned to a control group. Because motivation and health can affect switching and be related to outcomes, they can obscure genuine treatment effects. In other words, they inject a selection effect.
For a study with crossover, comparing treatment and control outcomes reflects the combined, real-world effects of treatment and the extent to which people comply with it or receive it even when it’s not explicitly offered. (If you want to toss around jargon, this type of analysis is known as “intention to treat.”) A limitation is that the selection effects introduced by crossover can obscure genuine treatment effects.
To know whether we should do more work to help individuals comply with treatment, it’s important to know if the treatment itself actually works. For that, we need an assessment that’s free of the effects of crossover.
Keith Humphreys, Janet Blodgett, and Todd Wagner provided one for A.A. Though it’s based on study data with crossover, it corrects for it by focusing on the subset of participants who do comply with their random assignment. In a hypothetical example, imagine that 50 percent of the sample receive treatment regardless of which group they’ve been assigned to. And likewise imagine that 25 percent are not treated no matter their assignment. In this imaginary experiment, only 25 percent would actually be affected by random assignment. These are known as “marginal patients”— not marginal because they don’t matter but because they’re the margin affected by randomization.
Analysis of marginal patients yields an estimate of the treatment effect that is free from the bias introduced by crossover. However, it’s not always the case that the resulting treatment effect is the same as one would obtain from an ideal randomized controlled trial in which every patient complied with assignment and no crossover occurred. Marginal patients may be different from other patients.
This is a limitation of such analysis: It provides an estimate of a true treatment effect, but only for those who change behavior due to treatment availability. (This type of analysis — what economists and other social scientists call “instrumental variables analysis” — has been applied in many other studies, including the recent study of Oregon’s Medicaid program that expanded by lottery in 2008.)
Despite the limitation, analysis of marginal patients reflects real-world behavior, too. Not everyone will comply with treatment. But, among those who do, are they made better off? That’s a question worth answering.
The Humphreys study does so and tells us that A.A. helps alcoholics, apart from the fact that it may attract a more motivated group of individuals. With that established, the next step is to encourage even more to take advantage of its benefits.
From “Does Prescription Drug Coverage Increase Opioid Abuse? Evidence from Medicare Part D,” by Rosalie Liccardo Pacula, David Powell, and Erin Taylor:
We find that one particular insurance expansion that focuses exclusively on expansion of prescription drug benefits, i.e. Medicare Part D, has indeed been a significant contributor to the rise in opioid sales and opioid treatment admissions since its implementation in 2006. […] Importantly, however, we find that abuse, as indicated by treatment admissions, rises more for the non-elderly (and in the case of the treatment data), nondisabled population. In other words, we find evidence of diversion from those with legitimate medical need (i.e. the Medicare population) to other individuals. We estimate that an additional 10% increase in Medicare Part D enrollment is associated with a 4% increase in opioid prescriptions in the state, and an 8% increase in opioid treatment admissions.
Artist Sam Van Aken used his Pennsylvania farm background to take grafting to the nth degree and create this Tree of 40 Fruit, an extreme hybrid where every branch, essentially, bears a different variety of stone fruit, and the flowering bud for each creates a brilliant array of colors in the spring.
A paper published last year by Stryjewski et al. found that Massachusetts health reform was not associated with health improvements for patients with certain chronic conditions. A different paper, published earlier in the year by Sommers et al., found that Massachusetts health reform was associated with reductions in mortality. How could both findings be true? Find out in my AcademyHealth post.
Purchasing Medical Innovation, by James Robinson, is a worthwhile account of the forces that promote or constrain medical innovation in the U.S.
It begins with the correct observation that innovation is the source of better, but also very often more expensive health care. This alone is not necessarily problematic as we should be willing to pay for innovation that is sufficiently valuable. What is problematic is that not all innovation that we pay for meets this condition. Health care technology is often used on the wrong patient at the wrong time. These are troubling sources of waste.
From such a beginning, many authors would then consume 150 pages explaining how we might completely revamp the health system so we achieve better and cheaper care, the politics, feasibility, and even desirability of such a thing be damned. To his credit, Robinson does not take this course.
Instead, he explains what shapes the nature and cost of health care innovation in the U.S. with a clear, useful framework. To be supplied and consumed in the system, most technology must pass over four hurdles:
- The FDA must approve it for market access—a hurdle for safety and efficacy.
- Insurers must cover it—a hurdle for clinical and cost effectiveness.
- Physicians and hospitals must offer it—a hurdle for appropriateness and quality.
- Patients must want it—a hurdle sensitive to preferences and risks.
To be sure, there are other hurdles and factors relevant to each of these, and none is ideal in its specified role. But, Robinson does not argue for removal or dramatic reform of any of these hurdles. Implicitly, he seems to accept them as necessary, if imperfect, means of guiding health care innovation.
Importantly, the behavior of each is shaped by organizational structure and financial incentives, which are malleable. Indeed, today they are changing, in some cases in helpful ways. The overriding message of the book is that the era of “cost-unconsciousness” is ending and one emphasizing “value” is underway.
How can we enhance that value? Each of the four hurdles is covered in at least one chapter (the second and third hurdles received two each) that explains its role and address that question.
The FDA chapter was the first thing I’ve read about the agency’s process that I could follow. Perhaps I’ve been reading the wrong things, but most accounts of the FDA I’ve seen either assume I know how it works or only explain a small part of what it does, then move quickly to how it fails. Robinson is more thorough and balanced, specifying the purpose and value of the FDA’s activities, but also indicating where it might make some useful changes to avoid both over- and under-regulating.
The insurance chapters address how Medicare and commercial market insurers make coverage and payment decisions. I suspect most health policy wonks will find much (though perhaps not all) of the material in these chapters familiar: Medicare’s national and regional coverage decision processes, the roles of comparative and cost effectiveness, medical/utilization management, selective- and value-based design and contracting, the variety of reimbursement and payment mechanisms (fee for service, per case, per diem, per episode, capitation, and the like).
My favorite chapters of the book were on hospital purchasing. Robinson doesn’t specify which hurdle plays the most significant role and is most rapidly changing, but my sense is that this is it. Whereas hospitals once competed with one another by offering all of the latest and greatest technology (at high expense) to attract physicians and their referrals, the new paradigm is for hospitals to at least attempt to manage technology purchasing to drive down costs. The basic approaches include technology assessment committees and volume purchasing that forces suppliers to compete more vigorously on price and quality. How do hospitals perform at these?
It’s a big struggle, mostly because it’s difficult to convince a large number of physicians, potentially across disparate facilities in a system, to come to (or accept) consensus on technology. The chapter that looks directly at how Orange County, CA health systems (including Kaiser) do this and the extent to which they succeed is fascinating. I’ve never read a health economist write in such specificity about how hospitals are run. (If I’d read these chapters on hospitals without knowing the author, I’d have guessed he was a physician or hospital administrator, not a health economist. I’d love for doctors who work in hospitals or hospital managers to read chapters 4 and 5 and let me know how accurately it reflects their experiences.)
The book then turns back to more familiar territory: how different varieties of cost sharing and patient engagement affect patient demand and behavior. Robinson concludes with ways in which the FDA, insurers, providers, and consumers can collaborate today to motivate development and use of more valuable technology tomorrow.
For the FDA, this implies lower barriers to initial market access, more extensive postmarket surveillance, and a willingness to retract authorization for products found to be unsafe or ineffective. For insurers, it implies rapid coverage and generous pricing for breakthrough products, thereby allowing evidence to accumulate and products to improve with experience, coupled with price discounting for follow-on therapies and medical management for inappropriate uses. For physicians and hospitals, it implies methods of payment that reward improved product assessment, procurement, and use. For consumers, it implies a structure of cost sharing that encourages adherence to evidence-based care and discourages demand for overpriced services.
I’m grateful and honored Robinson sent me a copy of his book. Though I didn’t have to pay for it, I would have. It’s an innovation worth the price.
From a recent NBER paper by Darius Lakdawalla, Anup Malani, and Julian Reif:
[C]onsider a healthy consumer facing the risk of developing Parkinson’s disease in the years before the discovery of treatments that reduced the disease’s impacts on quality of life. Suppose we measure the quality of one year of life as some percentage of a year spent in perfect health. In the absence of a treatment, contracting Parkinson’s might reduce quality of life from, say, 80% of perfect health to 40%. Consider the introduction of a new medical treatment that costs roughly $5,000 per year and increases quality of life for Parkinson’s patients from 40% to 70%. If the value of perfect health for one year is $50,000, this increase in quality of life is worth $15,000 annually but costs only $5,000 annually. The traditional approach in health economics compares these two numbers to arrive at the net value of the treatment, which in this case would be $10,000 annually.
First of all, I want to flag the use of the term “value” or “net value” here. It’s consistent with what Uwe Reinhardt endorses: the difference—not ratio—of benefit and price. (Click through to the full text of his remarks.) Cost-benefit ratios (or their reciprocols), for instance, are also called “value” by some, but, as Reinhardt noted, that’s weird and inconsistent with the notion of “value” generally used in economics.
Notice that this calculation neglects the way the medical treatment’s introduction also compresses the variance in the quality of life between the Parkinson’s and non‐Parkinson’s states. Prior to the availability of treatment, Parkinson’s was a gamble that lowered quality of life by 40% of a perfectly healthy year, or a loss of approximately $20,000 per year; the treatment transforms the disease into a new gamble that lowers quality of life by just 10% of a perfectly healthy year, or a loss of just $5,000 per year. This compression in quality of life outcomes generates value for consumers who dislike risk.
It is true that the reduction in the variance of health outcomes is mitigated by an increase in the variance of healthcare spending. Before the availability of treatment, the individual may have faced no financial risk from falling ill with Parkinson’s; after its introduction, she faces the risk of a $5,000 per year expenditure. However, if the treatment is priced to generate consumer surplus, the ex post improvement in health outcomes will outweigh its financial cost. Thus, it should come as no surprise that this medical treatment lowers total risk in our example. Prior to the development of treatment, Parkinson’s imposes a risk of losing $20,000 in reduced health. After development, the risk of disease is transformed into a $5,000 financial risk plus a $5,000 health risk. In sum, this medical treatment cut the total risk of Parkinson’s in half. Furthermore, the nascent financial risk associated with purchasing treatment can be mitigated or even eliminated by health insurance.
I’ve put in bold a key assumption (or focus) that the authors apply in their analysis. They are considering only treatments that are priced such that consumer surplus is positive, in the absence of insurance. Given the widespread take-up of insurance, how many treatments are really priced this way? I guess it depends on whose consumer surplus one examines. For many treatments and many patients (but not for all), prices for the uninsured are above that which would generate positive consumer surplus for the uninsured. That fact is the source of moral hazard. Put it this way, at current prices, what’s the market for Sovaldi or proton beam treatments look like without insurance? I think they’re priced precisely to account for insurance. Indeed, I think these products wouldn’t exist without insurance, which is tantamount to saying that there’d be no technology-sustaining consumer surplus positive price.
Am I raising a limitation of the work here? I don’t know. (I will admit to not tracing this through all the math. Think of this as a point I’d raise in a seminar, and then I’d like to hear others who know the work better tell me what’s if what I’ve raised is important. UPDATE: Lead author Darius Lakdawalla responded to this. You’ll find that response below. It’s a good one.)
Even if a consumer has no health insurance, technology can reduce the physical risk she faces. In the Parkinson’s example, she faced a health risk of $20,000 prior to the technology but just a $10,000 risk after it, even if no health insurance is available. Adding health insurance to the analysis would cause the risk to fall even lower, to just $5,000. [… P]roviding consumers with access to better medical technology by encouraging medical innovation may reduce risk more efficiently than providing them with health insurance.
Their conclusion (after analysis),
New medical technologies provide substantial insurance value above and beyond standard consumer surplus. Under plausible assumptions, the insurance value is roughly equal to the conventional value. Accounting for risk thus doubles the value of medical technology over and above conventional calculations.
The ability of medical innovation to function as an insurance device influences not just the level of value, but also the relative value of alternative medical technologies. The conventional framework understates the value of technologies that treat the most severe illnesses, compared to technologies that treat mild ailments. This helps explain why health technology access decisions driven by cost‐effectiveness considerations alone often seem at odds with public opinion. For example, survey evidence suggests that representative respondents evaluating equally “cost‐effective” technologies strictly prefer paying for the one that treats the most severe illness.
I really like this because it aligns how humans tend to feel about the value of medical technologies with economic analysis, explaining why standard cost-effectiveness approaches seem wrong to us. This observation is what gives rise to the rule of rescue.
UPDATE: Here’s Lakdawalla’s response:
In fact, this is not a strong assumption, even for a high-cost drug like your Sovaldi example. To take one example, even the UK’s notoriously stingy health technology assessment agency thinks Sovaldi meets that bar quite easily.
To understand why, it helps to be a bit more literal about the issue. Drugs that generate surplus in the sick state generate a health benefit whose value exceeds the full price of the drug. That is, the gain in quality-adjusted life-years (QALYs) multiplied by the value of a QALY exceeds the full price of the drug. This is the same as saying that the cost-effectiveness ratio of a drug exceeds the value of a QALY. In the case of Sovaldi, the UK concluded that its cost-effectiveness exceeds $50K. Since a QALY is almost surely worth more than that, it follows that Sovaldi generates surplus in the sick state, even when its full price is considered, and even according to the UK.
One caveat is that drugs are priced to hit cost-effectiveness thresholds in markets that perform this analysis — like the UK — but not necessarily in the US. However, most of the time, this ends up being largely a wash. Let’s stick with the Sovaldi example to illustrate. Sovaldi costs $58K in the UK. Large private insurers in the US are probably paying 10-30% more than this, depending on their size and bargaining leverage. This is a pretty typical price differential between UK and US payers. However, the UK’s threshold of $50K/QALY is almost surely much less than 30% below the revealed preference willingness to pay for a QALY in the US. (For example, the labor literature says the value of a statistical life-year is about $200-300K. We have some work showing that metastatic cancer patients are willing to pay about $300K per life year. Etc.) Thus, on balance, Sovaldi is generating surplus in the sick state even at US prices.
Of course, the spirit of your point is still correct, because there are non-trivial numbers of drugs that fail to meet this bar. In addition, if sick people were better insured against the financial risk of illness, more drugs would generate surplus in the sick state, because the willingness to pay for health would go up among the sick. This is the sense in which financial insurance and medical technology are complements.