• The problem with one-size-fits-all health insurance

    The following originally appeared on The Upshot (copyright 2016, The New York Times Company). It is jointly authored by Nicholas Bagley and Austin Frakt and appeared on page A3 of the December 6, 2016 New York Times print edition.

    A co-worker struggling to make ends meet comes to you with a problem. The price of admission to a dear colleague’s retirement party at an upscale establishment is beyond her means, though not yours.

    You both feel obliged to attend. She’d rather bring some refreshments to a conference room than spend what she cannot afford on a lavish event. You like the idea of a grand send-off for your retiring colleague. There can be only one party.

    A similar, underrecognized conundrum arises in health insurance. Both you and your less fortunate co-worker are obliged under the law to obtain coverage (which you both want anyway).

    But you differ in what you’d prefer to pay for. A high-level manager who makes, say, $200,000 per year is probably willing and able to pay more for health care than someone who makes $50,000.

    Unfortunately, neither person really has a choice because the plans all cover “medically necessary” care, meaning any care that offers a clinical benefit. That includes lots of expensive and technologically sophisticated care that is no better, or only slightly better, than cheaper alternatives. You may be just fine paying for high-tech care of marginal value. For your colleague of more modest means, it’s a stretch.

    Consider, for example, treating prostate cancer with proton-beam therapy. It’s more expensive than alternatives like intensity-modulated radiation therapy, but isn’t proven to be any better. If given the choice, many people — especially those with lower incomes — might rather buy health insurance plans that exclude high-cost, low-value treatments.

    The trouble is that insurers rarely sell those sorts of plans. Even insurers that try to exclude a particularly expensive and unproven technology from coverage are often rebuffed by legislatures and the courts.

    This one-size-fits-all approach to insurance coverage disproportionately hurts low-income people, many of whom might reasonably prefer to devote their scarce dollars to housing or their children’s education. To some extent, subsidies and other monetary adjustments can mitigate this problem. Medicare and Medicaid, for example, are financed in large part out of federal income taxes. And within the Affordable Care Act marketplaces, lower-income people receive subsidies that cover some of their costs.

    People who receive coverage through their employers, however, don’t get that kind of help. Perversely, employer-sponsored health insurance is more highly subsidized for the rich than the poor. The subsidy comes in the form of an exclusion of health-insurance premiums from taxation. Since income tax rates are progressive — that is, the rich pay a higher rate of income tax than the poor — lower-income families get less of a benefit.

    But both high-wage and low-wage workers at the same company are effectively forced into the same plans. To qualify for the tax exclusion, federal law requires that companies offer the same plans to all or most of their employees, with no consideration for the variable demand for health care. Employees then pay for their fringe benefits by taking home lower wages — and a flat, across-the-board cut in wages burdens low-wage workers disproportionately.

    In theory, the labor market could adjust in ways that might lessen the problem: Low-income workers, for example, could demand higher wages for being forced into plans that are more expensive than they’d prefer. These would have to be made up by reducing the wages of high-income workers, something it’s not clear they would accept. There’s no evidence that labor markets actually work this way.

    “The notion that labor markets perfectly offset the varying preferences for health insurance among workers by giving higher wages to those who value health insurance less is a comforting but crazy idea,” said Amitabh Chandra, a Harvard economist.

    The problems with one-size-fits-all insurance run deeper. In some insurance markets, like those for small businesses in Massachusetts, employees across companies are pooled together and pay the same premium. A recent report from the Massachusetts attorney general showed that workers in companies that receive health care at less expensive hospitals effectively subsidize those at comparable companies who receive care at more expensive ones.

    The uniformity of insurance plans also affects the pace and composition of technological innovation. To extend our party metaphor, if everyone — even those who preferred simpler events — were effectively forced to pay for any retirement party, regardless of how lavish, we’d see, and pay for, retirement parties of ever-escalating extravagance.

    In much the same way, medical innovators respond to the size of the market for new technologies. The fact that health plans routinely cover all medically necessary care sends an “if you build it, we will pay for it” signal. Innovators are not getting the right signals, the right incentives, to develop high-value or cost-saving treatments. It’s more lucrative, instead, to develop pricey new therapies, even if they offer only marginal clinical benefits. The result is lots of new treatments that don’t provide much bang for the buck.

    Those new treatments pose a continuing challenge to efforts to bend the cost curve. Economists have long known that technology is the primary driver of escalating health expenditures. Indeed, in 2012 medical technologies that were not offered a decade earlier accounted for almost a third of Medicare spending delivered by physicians and outpatient hospital departments.

    What’s to be done? Managing technological innovation would require us to consider policy changes that would have been unthinkable a generation ago. “The tax code could be restructured to make extravagant health insurance less appealing,” Mr. Chandra suggested. Employers might then offer health plans that appealed more to low-income workers.

    The Cadillac tax on expensive health plans, which is scheduled to go into effect in 2020, is a step in that direction, but according to Mr. Chandra doesn’t go far enough. And it is unpopular across the political spectrum. Other ideas — like incorporating cost-effectiveness criteria into Medicare and private plan coverage criteria — are sure to prompt disagreement.

    Contentious solutions they may be. But as the cost of health care and health insurance rises, it won’t be a party for politicians feeling more pressure from consumers.

    Comments closed
  • Will Obamacare be repealed? If so, what then?

    A version of the following originally appeared on The Upshot (copyright 2016, The New York Times Company). It also appeared on page A3 of The New York Times print edition on November 22, 2016.

    The election of Donald J. Trump gives the Republicans in Congress a chance to act on their often-stated desire to get rid of Obamacare, a wish that Mr. Trump mostly says he shares. Aaron E. Carroll and Austin Frakt, our health policy columnists, discuss: Then what?

    Aaron: I think it’s safe to say few in Congress thought they’d have this opportunity. But like the proverbial dog who has finally caught the car, after untold futile attempts, Republicans have finally come within reach of repealing the Affordable Care Act. Now comes the essential question: Will they actually do it? They’ve been promising it forever, but I am still skeptical that it will happen. I believe you disagree. I’m going to let you go first. Why do you think they’ll do it?

    Austin: I think they’ll do it because they so thoroughly own the idea of repeal, having passed bills to repeal, partly repeal, delay or defund the A.C.A. in the House something like 60 times. Just the other day Senator Mitch McConnell endorsed repeal (again). The House and Senate also agreed to do so, in large part, in a budget reconciliation bill earlier this year. The only thing that prevented it was that President Obama vetoed it. I doubt Mr. Trump would do the same if given a similar opportunity.

    Now, I know that a budget reconciliation dismantling of the law is not a full repeal, because according to the rules it can only touch budget-related provisions. This excludes things like requiring insurers to take all comers for premiums that vary only by age and smoking status or preventing them from imposing coverage caps and lifetime limits, among other measures.

    I also must add that I’m much less confident of a repeal (or partial repeal) without agreement on a replacement. But I’ll turn it back to you, Aaron. Do you think the G.O.P. has to offer a full replacement to get its members to sign on to repeal? Or can it offer something that would cover fewer people and with fewer benefits?

    Aaron: I think they can get away with slightly fewer people and somewhat skimpier benefits, but not too much. There’s a part of me that thinks many in Congress were always so willing to vote for a “repeal” because they knew it had no chance of being signed into law. They got credit for the vote without ever having to face the downside. Actually repealing without replacing would mean effectively stripping more than 20 million people of their health insurance, without anything in return.

    This would be an unmitigated political disaster. The stories — of people with cancer, diabetes and more who were suddenly stripped of their insurance and left out in the cold — would very likely dominate our discussion for months. That leaves more than enough time to lead to significant repercussions in the 2018 midterm elections. With no Democratic leaders in any branch of government to blame, I think this would be akin to what happened in the 2010 elections, but in reverse.

    Now, if they can coalesce around a “replace” plan that doesn’t leave too many people out, then I think they could move forward. But in all the years since the A.C.A. was passed, Republicans haven’t been able to do that. Do you think they can? What do you think that plan would look like?

    Austin: One way to get from repeal to replace that minimizes immediate downside political risk is to pass a plan that doesn’t call for repeal for several years, at least after the 2018 midterms, though possibly after the 2020 election. Between now and then, there would need to be some kind of transition to whatever replaces Obamacare that didn’t just dump people off coverage with no alternative.

    But the alternatives could just be not as comprehensive or costly. Absolutely there will be bad stories. But keep in mind, there will be bad stories under Obamacare, too. Rocketing premiums, huge cost sharing and markets with few choices is not a recipe for political success. Republicans now own the task of fixing those things and doing so in a way that does not look as if they’re making Obamacare better.

    They’re actually in a tough policy spot. They’ll get the blame if they don’t fix or repeal the A.C.A., and they’ll get the blame if they don’t replace it with something people like better. Health policy is a very difficult and thankless task. I think they’ll opt for something they can call repeal and replace, but they could also just let Obamacare struggle and die. Neither looks good.

    One other way to get out from under the issue is to kick it to the states. Do you think a Trump administration, working with a G.O.P. Congress, will offer greater flexibility to states to design their own coverage plans that could diverge from Obamacare? If so, what are some ideas states might try?

    Aaron: I think it’s very likely those in Congress could punt Medicaid to the states. For years, they’ve been trying to change Medicaid funding to a block grant that they can then constrain over time. This will be enticing for them because it will allow them to reduce Medicaid spending in the future, while forcing states to make the tough decisions — and take the blame — for cuts in either beneficiaries or services.

    Fixing the markets for those who are getting health insurance through the Obamacare exchanges, though, is a different story. Without some sort of market regulation, which they’ve generally been opposed to, the same problems that existed pre-A.C.A. with respect to pre-existing conditions and individual ratings will exist. Many people will become uninsured. Annual and lifetime limits could reappear. Lots of people will have problems getting insured.

    Moreover, I have yet to be convinced that a significant number of Republicans in the House might coalesce around such a plan. Maybe for Medicaid, but I’m not sure about the exchanges. Even if they could, it’s likely the Democrats in the Senate would try to filibuster either of these plans. Don’t you think?

    Austin: Yes, I think Democrats would filibuster anything they could. The filibuster is not set in stone. A Senate majority can change it, and some are already calling for the G.O.P. to do so. But that doesn’t appearto be what the Senate will do — they’ll retain the filibuster. This could play to their favor, since they can propose things they like, let the Democrats filibuster them and take the blame when repeal kicks in with no replacement. Perhaps that’s another way for Republicans to get out of their political bind.

    Aaron: I’m sure we’ll have more to discuss as President Trump’s administration comes into power.

    Comments closed
  • Does Medicare Advantage save traditional Medicare money?

    The following is co-authored by Austin Frakt and Garret Johnson. Garret is a research assistant for Dr. Ashish Jha at the Harvard T.H. Chan School of Public Health, where he also works with Austin on Medicare Advantage studies. 

    The recent paper by Kate Baicker and Jacob Robbins estimates a spillover effect from Medicare Advantage (MA) to traditional Medicare (TM). The idea is that MA, through care management, influences patterns of care in such a way to reduce costs, which then affects the quality and costs of care for TM beneficiaries as well. (Prior posts on this kind of spillover here and here).

    We wanted to convey the estimates in the paper in a different way, one we find more helpful. Here, using their results, is the answer to the question, for the marginal dollar spent on MA, how much is saved in TM?

    • They find that $1200 in additional payment per MA enrollee per year yields 2.2 pct points higher MA market penetration.
    • They also calculate a $252 per TM beneficiary per year in spillover “savings” for every 10 percentage points in higher MA market penetration. (The savings come through more efficient use of hospital services, in particular shorter lengths of stay, mediated by a concurrent increase in less costly outpatient service use. Therefore, it’s not really immediate savings to Medicare since diagnosis-based Medicare hospital payments aren’t sensitive to shortening lengths of stay. But, longer term, perhaps these savings could be captured through changes in rate increases.)
    • Therefore, combining (1) and (2), for each $1200 per MA enrollee per year in additional payment there’s a $252 × (2.2/10) of spillover savings per TM beneficiary per year, or $55.44. Dividing, that’s $0.0462 of spillover per TM beneficiary for every $1 of payment for an MA enrollee.
    • There are a lot more TM beneficiaries than MA enrollees, so we need to scale these. At the midpoint of the authors’ study window (2005), MA had a market penetration of 13%, leaving 87% in TM. Multiplying these by the figures from (3), we get $0.040 of spillover per $0.13 in MA payment. That’s $0.31 of savings per $1 of payment.
    • The results of (4) is at the particular margin examined. Not every $1 of MA payment is associated with 31 cents of savings. Extrapolating out of sample, today, when the MA/TM enrollment split is 31%/69%, the savings is closer to 10 cents on the dollar. (This is just repeating the calculation in step (4), but with 31%/69% instead of 13%/87%.)

    There are lots of caveats:

    • We mentioned it above, but it should be emphasized that these figures only pertain to the time period (1999-2011) and range of MA payment (from about 5% to 15% above TM costs) and market penetration (9%-11.9%) examined. In particular, our out of sample extrapolation is not likely to be correct. We just did it to illustrate how the results change a lot as market penetration changes.
    • Related, the results above are for the marginal dollar spent on MA. What about all the other dollars? No doubt, some of them contribute toward TM savings, but some do not. In the paper, the authors estimate TM utilization as a quadratic function of MA penetration. For some lower range of penetration, MA is associated with higher per person TM costs (perhaps due to favorable selection into MA). Spillover effects probably taper off at a higher range of penetration, too.
    • What the total cost or savings of MA to TM is today, we cannot tell. We don’t even know if it’s net positive or negative. That estimate would require more estimation work and a bunch of assumptions.

    This is, on the one hand, interesting. Some MA spending is associated with a high degree of TM savings (31 cents on the dollar circa 1999-2011 is HUGE). To the extent that MA growth makes TM “cheaper” for taxpayers, then some amount of payment above TM rates to MA plans may be efficient. On the other hand, this is unsatisfying. We don’t really want to know if the marginal dollar spent on MA in, say, 2005 saved money. We want to know if, in total, MA is saving or costing money today. We just don’t know.

    @afrakt & @garretjohnson22

    Comments closed
  • Just marking the moment. It may not happen again.

    It has all the appearances of bragging, but really we’re just amazed and want to mark this moment. Who knows if it’ll happen again.

    On June 16, Austin and Aaron had the number 1 and 2 most emailed NY Times articles, among those published in the prior week:


    On June 22, Aaron and Austin had the number 1 and 3 most emailed articles, among those published in the prior month:


    What a great ride. Thank you, readers, for all your support.

    @afrakt and @aaronecarroll

    Comments closed
  • Beyond disclosure: How to think about conflicts of interest and the regulation of medical science

    This post is co-authored by Bill Gardner and Austin Frakt.

    The recent controversy about disclosure of conflicts of interest (see Bill here and here; Austin here and here) has called renewed attention to pervasive quality control problems in the scientific literature. We agree with Ian Roberts that

    the challenge is not to describe the flaws in the current system but to create a better one, where decisions about healthcare are informed by valid and reliable evidence.

    We also agree with Mick Watson that

    Open science describes the practice of carrying out scientific research in a completely transparent manner, and making the results of that research available to everyone. Isn’t that just ‘science’?

    How can we get serious about creating an open, valid, and reliable scientific literature?

    We recommend starting by acknowledging our moral response to the problem, and then putting it aside. It’s impeding our thinking. We’re struck by how often we hear that the problem in bias is the “corruption” of some researchers or the “perversion” of the research process. There are many contexts in which it’s important to view science in moral terms. But we doubt that focusing on the virtues or vices of researchers will get us much closer to a solution. Instead, we should think about what institutions and policies will advance scientific learning.

    In an ideal world, peer review of science should concern the evidence—data and methods—and the interpretation of findings in the light of existing knowledge. Facts about the authors ought to be extraneous. Aaron Kesselheim found that reviewers downgraded their ratings of the methodological rigor of clinical trials when they believed that the trials were funded by industry. That seems wrong: Consider how you would react if a study showed that reviewers downgraded their ratings of articles written by women, for example.

    But this is the real world, and you can also make a case that the reviewers in Kesselheim’s study were behaving rationally. We may want reviewers to evaluate a research report based on the data and methods, but authors can only document so much in a paper. Given the limits on what authors can document, there’s reviewer uncertainty about the quality of the evidence. Bayesian inference suggests that the more uncertain you are about the evidence, the more weight you should give to your prior probabilities concerning the credibility of a report’s authors. Therefore, the evidence that studies funded by pharmaceutical companies are biased toward the companies’ products would seem to justify some weight on a prior to distrust their research.

    In practice, though, humans may not compute like perfect Bayesians. We may use real or perceived COIs to over- or under-correct. So the better response, in the long-term, is to reduce our uncertainty about the data and methods. With less uncertainty about the evidence, priors about the authors would matter less and applied (or misapplied) less.

    There are several strategies for reducing this uncertainty that the scientific community has applied (though not uniformly) or could apply going forward (perhaps with some infrastructure development). These strategies include:

    1. Registration of trials and reporting of all registered analysis (or clear metrics of the extent to which they are not reported);
    2. Archiving of trials’ analytical data files (see BMJ‘s Open Data campaign and GlaxoSmithKlein‘s commitment to provide access to anonymized patient-level data);
    3. Archiving of statistical programming (reproducible research);
    4. Expert evaluation of study methods by an individual or individuals without conflicts of interest;
    5. A possible future extension of these strategies is: Archiving of the transformations required to generate the analytic data files.

    Pursuing these strategies would likely increase the transparency and reproducibility of research, the quality of scientific practice, and reduce uncertainty about its credibility and validity. To our knowledge, there are no scientific reasons not to pursue these strategies.

    But there are economic, psychological, and ethical reasons. For example, we can’t make data sets public unless we can make sure that research participants can’t be identified from them. We should also consider the costs in researchers’ time, attention, and resources in complying with more rigorous standards of documentation, with parallel costs to society in possible delay of projects. It is true that science requires a meticulous attention to detail. Nevertheless, humans have finite attention and limited capacities for decision making. More time and attention spent on documentation might mean less time spent thinking and reading.

    We should not take the existence of these potential costs as an excuse to do nothing about improving science and its credibility. We should do something while, reasonably, taking them into consideration.

    There are reasons to believe that improvements in technology and the self-regulation* of research will facilitate our ability to do better science without unduly burdening researchers or endangering research participants. We are more likely to develop those technologies and self-regulations if we frame our considerations more in terms of questions about how to improve the validity, reliability, and transparency of science, as well as the rate of scientific progress, rather than questions about the moral virtues of researchers.

    Disclosure of financial conflicts of interest should be retained as a necessary, though insufficient, tool of scientific integrity. But we must get beyond disclosure, and beyond our outrage over what we think it signals, to tighten up the process of science directly. In a world of competing interests, humans, unfortunately, do not always do good science by accident or because it’s the “right thing to do.” Science is important. We need to treat it as such, and tighten up our regulation of it.

    * “Self-regulation” means regulation by scientists, not the government. The scholarly community must find ways to adequately regulate itself, e.g., through a consensus about the requirements of publication in top (or all) medical journals. Having said this, we acknowledge that NIH requirements on grantees—which we support—are an interesting and important case in which a governmental body can advance open science.

    @Bill_Gardner and @afrakt

    Comments closed
  • In NEJM: Protection or Harm? Suppressing Substance-Use Data

    This post is jointly authored by Nicholas Bagley and Austin Frakt.

    Yesterday evening, the New England Journal of Medicine released a Perspective piece that we co-authored on the recent suppression of Medicare and Medicaid data to researchers. (For our earlier coverage, see the posts collected here.) As we explain, the data suppression is both unnecessary and harmful:

    What if it were impossible to closely study a disease affecting 1 in 11 Americans over 11 years of age—a disease that’s associated with more than 60,000 deaths in the United States each year, that tears families apart, and that costs society hundreds of billions of dollars? What if the affected population included vulnerable and underserved patients and those more likely than most Americans to have costly and deadly communicable diseases, including HIV–AIDS? What if we could not thoroughly evaluate policies designed to reduce costs or improve care for such patients?

    These questions are not rhetorical. In an unannounced break with long-standing practice, the Centers for Medicare and Medicaid Services (CMS) began in late 2013 to withhold from research data sets any Medicare or Medicaid claim with a substance-use–disorder diagnosis or related procedure code. This move—the result of privacy-protection concerns—affects about 4.5% of inpatient Medicare claims and about 8% of inpatient Medicaid claims from key research files (see table), impeding a wide range of research evaluating policies and practices intended to improve care for patients with substance-use disorders.

    The timing could not be worse. Just as states and federal agencies are implementing policies to address epidemic opioid abuse and coincident with the arrival of new and costly drugs for hepatitis C—a disease that disproportionately affects drug users—we are flying blind.

    While NEJM was preparing the piece for publication, ResDAC, released new Medicare data indicating that the suppression is even more extensive than we wrote. For 2013, Medicare suppressed 6.43% of all Medicare inpatient claims; for 2014, that figure rose to 6.8%. (The figures for Medicaid in our piece remain the same.)

    Eric Goplerud, speaking to Alcohol and Drug Addiction Weekly in January, suggested that SAMHSA is planning on proposing a rule change this year that would allow CMS to restore access to the affected data. We hope so. The issue is much too urgent to ignore.

    @afrakt & @nicholas_bagley

    Comments closed
  • Why write for the public about science and research?

    This post is jointly authored by Bill Gardner and Austin Frakt.

    Why do we write for the public about science and research? It’s a lot of work and our day jobs pay better. Last week, Austin and Bill conversed about this on Twitter. We had help from Kristen Rosengren (@RosenKris) and others, including Janet Weiner (@weinerja). We’ve edited the conversation for clarity and expanded some of our answers.

    Austin: I typically do not write more than once about the same topic. A mistake?

    Kristen: Depends on your primary goal. To express yourself, once is enough; to convince or change minds, repetition is helpful.

    Austin: I don’t write to convince, actually. I think that’s a dangerous objective.

    Bill: “I don’t write to convince.” I can see that it might be dangerous to need to change other’s views. But if you don’t want to change others’ views, why offer an argument? Moreover, doesn’t TIE have a goal of science translation?

    Kristen: To educate & inform so others can make decisions? Perhaps biased by our mission, I think that has real value.

    Austin: I write first to convince myself I have a reasonable understanding of the world. And writing for the public changes the quality of my thinking. I find that publishing motivates deeper engagement and care than I would otherwise apply.

    Bill: Excellent motives. One of the things you learn in good science lab meetings or good philosophy seminars is how much deeper you can get when you are pushed by the best critics.

    Nevertheless I feel an obligation to persuade. I had the privilege of a great education leading to a PhD. That incurs a debt because not everyone had those chances. I see some ways in which we could act together to make the world better. This obligates me to take part in public debates about health policy. If my writing gets at the truth, and I write well enough that others can see that truth, maybe we’ll make better choices.

    Austin: I’m delighted to serve a translation role, and even change minds. It’s not my primary motivation, though. Were it so, I worry I’d not be as faithful to the evidence, wherever it may lead. An “obligation to persuade” could (though need not) become a conflict of interest.

    Bill: True. There is a tension between writing to persuade and writing to discover the truth. The danger I fear more is partisanship: that is, writing that serves the needs of your political identity rather than your commitment to the truth. This leads to a risk of motivated reasoning, as Dan Kahan describes so well. One of the great things about social media is that it is easy find smart, well-informed people who disagree with you.

    Austin: I worry about more than partisanship. I’m not sure why that’s the only bias of relevance. Fealty to any set of values would shape how one sees and conveys facts and ideas. Though I think it’s possible for someone to be faithful to evidence and still be principally motivated by an ambition to persuade, I think many minds can’t handle that. It’s very easy to become invested in a position or to take the view that if you’re seen as having been fallible, that weakens your strength of persuasion.

    Whenever I hear or perceive that someone is in it to persuade me I lose a bit of trust. I am much more comfortable if I feel they are just in it to convey truth, as best they can and whatever that means. The way in which one acknowledges limitations and counterpoints offers a clue. It can be done dismissively, or it can be done in a way that shows the writer is really wringing his hands over it. I like to see sweat on the brow. Synthesizing evidence for a firm conclusion is hard and fraught, or should be. That challenge should come through. If it doesn’t, I feel I’m being spun. I try to avoid doing that as a writer.

    Bill: Yes. Cognition is social: we depend on others for access to information. I only know anything about recent physics because I trust Brian Greene, Sean Carroll, and other great science writers. Yet we know this social dependence makes us vulnerable to manipulators. So we are wired to worry about the motivations of our sources. We look for evidence about whether they care about the truth. Of course, caring about the truth can be faked. But I’m not that subtle. As I learned the hard way playing poker, I’m a terrible liar. The only way I can communicate that I care for the truth is to actually care for the truth. So my best shot at persuasion is to be as faithful as I can to the science.

    This is, by the way, the reason why I think TIE has gained a loyal and discerning readership. We are clear about our values but I think we are all first committed to the norms of our disciplines. My sense is that lots of people who disagree with us about both policy and the facts nevertheless trust us to give our best effort at the truth.

    Austin: Trust is a huge topic. I had a high school social studies teacher who convinced me that it’s all we have. He’s right, but to go into that would take another post. (Well, I see I wrote that other post on trust in 2010.)

    Anyway, I’m delighted if we are perceived as trustworthy. I wonder if we’re clear about our values, though. There are degrees of clarity. There are absolutely some elements of my life and upbringing, even professional circumstances, that I have not and likely will not disclose. That’s true of everyone, perhaps to different degrees. Can we ever know why a given person is communicating in a given topic in a given way? What is speaking, the evidence or some tribal value? Or, to what extent does the latter shape presentation of the former? Or, what about subjects that are never raised?

    Bill: Great point and I spoke too quickly in claiming that we are clear about our values. Actually, I find that most of us don’t even fully know our own values. This is part of the value of moral philosophy: it’s a practice of confronting your theories about what is right or good with your judgements about actual cases, with the goal of making them cohere. You get clearer about what you really believe and perhaps you can revise your views for the better. And when you work on the edge between research and policy, getting clear about your values is essential, because policy choices are based on both scientific evidence and goals informed by values.

    To get back to where we started: this is another reason to write for the public. The net is full of smart people with diverse values. They can, perhaps, see something that you are blind to. That valuable exchange can happen whether you convince one another of something or not.

    @Bill_Gardner and @afrakt

    Comments closed
  • Patients overestimate benefits, underestimate harms of treatment. What if they knew the truth?

    The following is co-authored by Austin Frakt and Aaron Carroll. It originally appeared on The Upshot (copyright 2015, The New York Times Company). Click over to that version of the post to see the accompanying chart.

    If we knew more, would we opt for different kinds and amounts of health care? Despite the existence of metrics to help patients appreciate benefits and harms, a new systematic review suggests that our expectations are not consistent with the facts. Most patients overestimate the benefits of medical treatments, and underestimate the harms; because of that, they use more care.

    The study, published in JAMA Internal Medicine and written by Tammy Hoffmann and Chris Del Mar, is the first to systematically review the literature on the accuracy of patients’ expectations of benefits and harms of treatment. They examined over 30 studies that assessed whether patients understood the upsides or downsides of certain treatments. To a great extent, patients didn’t.

    In the 34 studies that assessed understanding of benefits, patients overestimated their potential gain in 22 of them, or 65 percent. For instance, a 2002 study published in the Journal of the National Cancer Institute asked women who had undergone prophylactic bilateral (double) mastectomy to estimate how much the procedure reduced their risk of breast cancer. On average, the women thought they had reduced that risk from 76 percent to 11 percent, an absolute risk reduction of 65 percentage points.

    For the more than 80 percent of the women in the study who did not have a BRCA genetic mutation — which drastically increases the risk of breast cancer — the real risk before surgery of developing breast cancer was 17 percent, meaning they greatly overestimated their risk reduction. Even the women with a BRCA mutation overestimated their risk reduction, but to a lesser extent.

    Another 2012 study published in the Annals of Family Medicine asked patients to estimate the benefits of screening for bowel and breast cancer, and the use of medications to prevent hip fracture and cardiovascular disease. More than two-thirds of patients overestimated the benefits of medications to prevent cardiovascular disease, and more than 80 percent overestimated the benefits of medications to prevent hip fractures.

    Further, 90 percent of patients overestimated the benefits of breast cancer screening, and 94 percent overestimated the benefits of bowel cancer screening. The researchers also asked the patients to estimate the minimum reduction in bad outcomes (like fractures or deaths) they would need to achieve to find the treatment worthwhile. For three of the four studied interventions, the minimum benefit patients would accept was higher than the actual benefit.

    In the 15 studies examined in the systematic review for which harms were a focus, patients underestimated potential downsides in 10 of them (67 percent). For example, a study published in 2012 in the Journal of Medical Imaging and Radiation Oncology asked patients to estimate the risks associated with a CT scan. A single CT scan exposes a patient to the same amount of radiation as 300 chest X-rays, and carries with it a 1-in-2,000 chance of inducing a fatal cancer. More than 40 percent of patients underestimated a CT’s radiation dose, and more than 60 percent of patients underestimated the risk of cancer from a CT scan.

    Why do patients err in assessments of risks and benefits? One reason could be that what they know is driven by the messages they hear. Doctors, direct-to-consumer ads and the media can skew our perceptions. They tend to focus on the benefits, but rarely quantify them. Health care centers, screening advocacy programs and pharmaceutical ads all push us to talk to our doctors about getting treatment without talking about actual gains.

    Doctors also aren’t always good at communicating risks. A 2013 study published in JAMA Internal Medicine found that fewer than 10 percent of patients were told about overdiagnosis and overtreatment associated with cancer screening, even though 80 percent of patients wanted to know about harms.

    This study, and others, indicate that patients would opt for less care if they had more information about what they may gain or risk with treatment. Shared decision-making in which there is an open patient-physician dialogue about benefits and harms, often augmented with use of treatment decision aids, like videos, would help patients get that information. However, a majority of patients still report that they prefer to leave medical decision-making to their doctors.

    It might also be the case that some patients would use more of certain types of care if they had more information. Many chronic conditions remain undermanaged and undertreated in the United States. It’s possible that people with these conditions who had more information would use more care, which could raise spending for these patients but make them better off.

    There’s also an argument to be made that people who overestimate the benefits of medicine to treat some conditions are more likely to take it regularly, which might lead to better outcomes, in some cases, than would occur if these patients were better informed.

    Regardless, even though some patients may benefit somewhat from being ill informed, it seems wrong to argue that we should keep them in the dark. Many of the studies in the systematic review show that people report that they would opt for less care if they better understood benefits and harms. Improved communication could better serve patients and might improve the efficiency of our health system if patients focus on getting the types of care for which the benefit outweighs risk of harm.

    It’s also possible that unrealistic expectations of care help patients cope with disease or provide them with some sense of control. Feeling hopeful about one’s future is not to be dismissed. But those unrealistic expectations don’t come cheap. We should at least consider the price that we pay for being uninformed.

    Comments closed
  • How to Measure a Medical Treatment’s Potential for Harm

    The following is co-authored by Aaron Carroll and Austin Frakt. It originally appeared on The Upshot (copyright 2015, The New York Times Company). Click over to that version of the post to see the accompanying charts.

    As we wrote last week, many fewer people benefit from medical therapies than we tend to think. This fact is quantified in a therapy’s Number Needed to Treat, or N.N.T., which tells you the number of people who would need to receive a medical therapy in order for one person to benefit. N.N.T.s well above 10 or even 100 are common. But knowing the potential for benefit is not enough. We must also consider potential harms.

    Not every person who takes a medication will suffer a side effect, just as not every person will see a benefit. This fact can be expressed by Number Needed to Harm (N.N.H.), which is the flip side of N.N.T.

    For instance, the N.N.T. for aspirin to prevent one additional heart attack over two years is 2,000. Even though this means that you have less than a 0.1 percent chance of seeing a benefit, you might think it’s worth it. After all, it’s just an aspirin. What harm could it do?

    But aspirin can cause a number of problems, including increasing the chance of bleeding in the head or gastrointestinal tract. Not everyone who takes aspirin will bleed. Moreover, some people will bleed whether or not they take aspirin.

    Aspirin’s N.N.H. for such major bleeding events is 3,333. For every 3,333 people, just over two on average will have a major bleeding event, whether they take aspirin or not. About 3,330 will have no bleed regardless of what they do. But for every 3,333 people who take aspirin for two years, one additional person will have a major bleeding event. That’s an expression of the risk of aspirin, complementing the fact that one out of 2,000 will avoid a heart attack.

    Granted, one out of 3,333 is a pretty tiny risk. But remember that the chance of benefit is pretty small, too.

    Sometimes, though, the N.N.H. can be much lower, even lower than that of N.N.T., which suggests the chance of harm is greater than the potential benefit. Consider screening mammograms, which are considered so essential that they are the only screening tests specifically mentioned in the Affordable Care Act, and coverage for them with no cost sharing is required by the law.

    If you look at the data for all randomized controlled trials of breast cancer screening, the N.N.T. for recommending screening to prevent one death from breast cancer after 13 years of follow-up is 1,477. But further analyses show that the one woman would have probably died of other causes anyway. There may be no benefit at all with respect to preventing death from all causes.

    Screening with mammograms can cause harm, though. They lead to overdiagnosis, encouraging the provision of therapies that provide no benefits — but do carry risks, and therefore are considered harms.

    If we look at those same studies, for every 333 women who are assigned to have a screening mammogram, one extra will undergo a lumpectomy or mastectomy as a result. One in every 390 women assigned to have a screening mammogram will undergo an extra course of radiation therapy as a result. (In these randomized controlled trials, patients are either assigned to get screening mammograms or they are not. The study then usually looks at the outcome for all who were assigned to get the mammogram, whether they actually did or not.)

    In other words, for about every 1,500 women assigned to get screening for 10 years, one might be spared a death from breast cancer (though she’d most likely die of some other cause). But about five more women would undergo surgery and about four more would undergo radiation, both of which can have dangerous, even life-threatening, side effects.

    Thus, N.N.H., paired with N.N.T., can be very useful in discussing the relative potential benefits and harms of treatments. As another example, let’s consider antibiotics for ear infections in children. There are many reasons that parents and pediatricians might consider treatment. One commonly cited reason is that we want to prevent serious complication from untreated infections. Unfortunately, antibiotics don’t do that, and the N.N.T. is effectively infinite. Antibiotics also won’t reduce pain within 24 hours. Antibiotics have, however, been shown to reduce pain within two to seven days. Not all children will see that benefit, though. The N.N.T. is about 20 for that outcome.

    Antibiotics can cause side effects, however, including vomiting, diarrhea or a bad rash. The N.N.H. for side effects in this population is 14.

    This means that when a child is prescribed antibiotics for an ear infection, it’s more likely that he will develop vomiting, diarrhea or a rash than get a benefit. When patients are presented with treatment options in this manner, they are sometimes more likely to agree to watchful waiting to see if the ear infection resolves on its own. For most children with ear infections, observation with close follow-up is recommended by the American Academy of Pediatrics.

    A wealth of N.N.T. and N.N.H. data based on clinical trials is available on a website developed by David Newman, a director of clinical research at Icahn School of Medicine at Mount Sinai hospital, and Graham Walker, an assistant clinical professor at the University of California, San Francisco. But it’s important to understand that results from clinical trials do not always reflect what happens in the real world. As criteria for treatment become more permissive beyond those applied in trials, the N.N.T.s can go up. But importantly, N.N.H.s often do not. Healthier people are less likely to see a benefit from antibiotics or an aspirin. But they are not less likely to have a side effect or complication.

    This is because the harms associated with treatment usually have nothing to do with the underlying illness. They are caused by the therapy, regardless of the reason for use. Children will develop diarrhea, vomiting or rashes from antibiotics in the same relative amounts no matter why we are using them. Put another way, clinical trials are designed to target the class of patients that most likely benefits from treatment, but they are not targeted to those more or less likely to experience harm. When treatments are applied in real-world clinical settings, we generally don’t see changes in the proportion of patients harmed by them relative to trials.

    When we stray from recommendations for therapies, and broaden the population given studied treatments, the N.N.T.s often go up, but the N.N.H.s stay the same. Things are often even worse than the data in studies make them look. Fewer people benefit, but just as many are harmed.

    We hope that every therapy has a benefit. The N.N.T. shows us that benefits are often much less likely than many might think. The N.N.H. can show us how likely we are to have a harm compared with a benefit. Considering both, especially in light of how practice often differs from studies, can help us make better decisions about how to care for ourselves and those we love.

    Comments closed
  • Can This Treatment Help Me? There’s a Statistic for That

    The following is co-authored by Austin Frakt and Aaron Carroll. It originally appeared on The Upshot (copyright 2015, The New York Times Company). Click over to that version of the post to see the accompanying charts.

    In his State of the Union address last week, President Obama encouraged the development of “precision medicine,” which would tailor treatments based on individuals’ genetics or physiology. This is an effort to improve medical care’s effectiveness, which might cause some to wonder: Don’t we already have effective drugs and treatments? In truth, medical care is often far less effective than most believe. Just because you took some medicine for an illness and became well again, it doesn’t necessarily mean that the treatment provided the cure.

    This fundamental lesson is conveyed by a metric known as the number needed to treat, or N.N.T. Developed in the 1980s, the N.N.T. tells us how many people must be treated for one person to derive benefit. An N.N.T. of one would mean every person treated improves and every person not treated fails to, which is how we tend to think most therapies work.

    What may surprise you is that N.N.T.s are often much higher than one. Double- and even triple-digit N.N.T.s are common.

    Consider aspirin for heart attack prevention. Based upon both modifiable risk factors like cholesterol level and smoking, and factors that are beyond one’s control, like family history and age, it is possible to calculate the chance that a person will have a first heart attack in the next 10 years. The American Heart Association recommends that people who have more than a 10 percent chance take a daily aspirin to avoid that heart attack.

    How effective is aspirin for that aim? According to clinical trials, if about 2,000 people follow these guidelines over a two-year period, one additional first heart attack will be prevented.

    That doesn’t mean the 1,999 other people have heart attacks. The fact is, on average about 3.6 of them would have a first heart attack regardless of whether they took the aspirin. Even more important, 1,995.4 people would never have a heart attack whether or not they took aspirin. Only one person is actually affected by aspirin. If he takes it, the number of people who remain heart attack-free rises to 1996.4. If he doesn’t, the number remains 1995.4. But for 1,999 of the 2,000 people, aspirin doesn’t make any difference at all.

    Of course, nobody knows if they’re the lucky one for whom aspirin is helpful. So, if aspirin is cheap and doesn’t cause much harm, it might be worth taking, even if the chances of benefit are small. But this already reflects a trade-off we rarely consider rationally. (And many treatments do cause harm. There is a complementary metric known as the number needed to harm, or N.N.H., which says that if that number of people are treated, one additional person will have a specific negative outcome. For some treatments, N.N.T. can be higher than the number needed to harm, indicating more people are harmed than successfully treated.)

    Not all N.N.T.s are as high as aspirin’s for heart attacks, but many are higher than you might think. A website developed by David Newman, a director of clinical research at Icahn School of Medicine at Mount Sinai hospital, and Dr. Graham Walker, an assistant clinical professor at the University of California, San Francisco, has become a clearinghouse of N.N.T. data, amassed from clinical trials. Among them, for example, are those for the effects of the Mediterranean diet.

    The Mediterranean diet, which is heavy in vegetables, fruits, nuts and olive oil; moderate in fish and poultry; and light in dairy, meat and sweets; has long been advocated as a means to avoid heart disease. In people who have never had a heart attack, but who are at risk, the N.N.T. is 61 to avoid a heart attack, stroke or death. And that is for people who adhere to the diet for about five years. For those at higher risk, who have already had a heart attack, to avoid one additional death, the N.N.T. is about 30. That’s the number of people who would have to adhere to the diet for four years so that one extra person survived. About 1.4 people out of 30 such people will die no matter what they eat; 27.6 will not die no matter what they eat. Only one will benefit from sticking to the diet.

    But it’s not easy for everyone to stick to such a diet for that many years. Some — for example, those who enjoy steak and ice cream — will feel that it diminishes their quality of life. When you hear that the diet prevents heart attacks, then it might sound worth it. But does it still sound worth it when you consider that 29 out of 30 people who stick to the diet for several years see no benefit at all? Will you stick to it for years and be the lucky one for whom that matters?

    As treatments go, an N.N.T. of 30 is pretty good. Very few are as low as 10, though some are. For instance, the use of steroids in people having asthma attacks to prevent admission to the hospital has an N.N.T. of eight. This is so obvious, and so powerful a treatment, that there are no commercials and no op-eds preaching steroid use for asthma. (Maybe there should be. It’s likely that this therapy is being underutilized, perhaps because cost-sharing discourages some people with asthma from seeking care when they might need it.) Steroids work very well for asthma attacks — better than many treatments for other conditions. But still, seven of eight people suffering an asthma attack see no benefit at all from steroids with respect to preventing hospitalization.

    Even more concerning, N.N.T.s as calculated from clinical trial data are probably higher than those based on real-world medical care. In clinical trials, treatments are applied to a select population for whom they’re intended. In medical practice, it’s very common for treatments to be applied to a much broader population, including many people for whom they’re less likely to be effective, which increases the N.N.T. This is, perhaps, because doctors would rather offer an explicit treatment — perhaps to harnessplacebo effect — even when it’s not likely to be of additional benefit.

    In fact, as recently reported in The Times, a new study showed that many people who are prescribed aspirin for the primary prevention of cardiovascular disease don’t meet the criteria described above for its use. Because of this use in a population beyond that targeted in clinical trials, the N.N.T. in practice is most likely higher than the 2,000 suggested by those trials. (It’s worth noting that our best estimates of N.N.T.s can rise or fall as more data are collected and as treatments or how or to whom they’re delivered change.)

    Antibiotics are a classic example of overuse. For instance, the N.N.T. for antibiotics to treat radiologically diagnosed acute sinusitis is 15, meaning that 14 out of 15 who take them derive no benefit. But physicians often write prescriptions for antibiotics in situations when the diagnosis of sinusitis is far less assured. This leads to antibiotics being overprescribed and overused, raising their N.N.T. in practice.

    The use of stents to open up clogged arteries in patients who are not actively suffering a heart attack is another treatment that is employed too often. (Stents are considered appropriate in patients who are having a heart attack.) Many more patients believe they extend life than their N.N.T. suggests. The N.N.T is effectively infinite, relative to treatment with medications, for people not suffering a heart attack.

    Until health care technology improves, there’s not a lot we can do about N.N.T.s that are larger than we might hope. It’s just a fact of current medical technology that not everyone benefits from treatment, even when well targeted. President Obama’s push for “precision medicine” is an attempt to change this, by using genomics to focus treatments on people who would most benefit from them. That will take time.

    In the meantime, we would all be better served by a more informed understanding of exactly how much, or how little, benefit is reasonably to be expected by taking a drug, changing our lifestyle or undergoing a procedure. Especially since the chance of benefit, as expressed by N.N.T., might not be worth the risk of harm, as expressed by N.N.H. We’ll discuss that more next week.

    Comments closed