• On Google’s new Inbox

    Google’s new alternative to Gmail is here, and it’s called Inbox, available by invitation only at the moment. You can see Google’s pitch here and read how-to use guides here and here. My tweets on Inbox are here. Below is my review, of sorts, informed by three days of Inbox use.

    What I Like

    For me, Inbox’s killer app is the ability to schedule emails and reminders to return to one’s inbox at a time and/or place of one’s choosing. This is called snoozing. (To date I have only used the time-based reminders, not place-based.) Some smarty-pants at Google has recognized that I, and no doubt gazillions of others, use their inbox like a task list anyway, leaving emails hanging around to remind us to do something, emailing ourselves to remind us to do something, starring things as reminders to do something, and so forth.

    Rather than forcing users to kludge all this onto an email program, Inbox has integrated it more intelligently. This makes sense, and it works. It’s for this feature alone that I have switched to using Inbox for a guestimated 95% of my email use. (More on the other 5% below.)

    I also like that the Done list (the analog to Gmail’s All Mail list) is presented in the order in which the user marked items as done. The most recent done stuff is at the top. In Gmail, the All Mail list is in order of when the last email was received. This is less useful to me, as I’m frequently scrolling my archive for that item I know I put away recently despite having received it weeks ago. The receipt date is less salient and relevant than the archived (“done’d”) date.

    What Google highly promotes is bundling, which is the combining of similar email conversations into groups. This is a bit like Gmail’s tabs, only bundles aren’t stored and presented as tabs, but as blocks of related email. And it’s a bit like tags only with tags, email conversations are only presented as grouped within their tag, not wherever they appear. I never liked Gmail’s tabs (disabled them on day 1) and I’m not a big tagger either. So, though potentially useful in rare instances, I won’t be bundling much. Google hypes bundling, but it’s not where the action is (for me). Still, no harm. I give bundling a mild “like.”

    Google could have done all the above right in Gmail and I’d have been happy.

    What is Not Ready or Not Useful

    Google claims that Inbox isn’t done, and it’s clear a lot of needed features are missing (one example: you can’t edit and manage signatures, which don’t appear on replies and forwards, to my annoyance). Indeed, how-to guides admit you’ll need to go back to Gmail to do some things like selecting multiple emails for bulk deletes and moves, doing anything in settings, emptying Trash, and on and on). Still, over three days, I’m not going back to Gmail much (5% of the time, maybe), yet I cannot completely abandon it. In time, I hope I can.

    Inbox allows you to “pin” stuff to your inbox, which keeps it there. I don’t get why this is useful. I can already keep stuff in my inbox by not removing it from my inbox. And, the whole point of Inbox, as far as I can see, is to help keep your inbox from being cluttered. This is why scheduling things to pop up later — and disappear in the meantime — is useful. Why pin when you can snooze and schedule?

    Google thinks pinning is key because it includes one-click pin buttons on emails and a switch at the top of one’s Inbox to toggle to a pined-only view. I just don’t get why Google thinks pinning is so important. There are almost no other buttons, an attempt at a very clean presentation.

    Meh. Obsessive cleanliness is an overrated design principle. Sure, one doesn’t want too much clutter. But a handful of buttons at the top, for archiving, marking as spam, moving to trash, etc. is a reasonable balance, as achieved in Gmail. In a nearly button-free Inbox, one has to click twice to do some of those things, or use keyboard shortcuts. (There are gestures for the touch-screen implementations on Android and iOS, which are kind of nice.) I also noticed that I have to now click twice to download an attachment. I don’t like this. One click should do it, per Gmail.

    Emails are presented in Inbox with some of their internal contents more visible without opening them, like dates of events, contact info, and thumbnails of attachments. I do not like this because it makes the email look big, take up too much real-estate on my page. Opening the email is not hard. I would toggle this off if I could. Or, I encourage Google to find a way to make it more space-efficient.

    Stars are gone. I don’t mind too much. It took me a bit of time to realize that “snooze until someday (unspecified)” is the equivalent to my use of Gmail stars. Inbox offers a ready way to see all one’s snoozed items (whether scheduled for a specific time or not). And that’s how I used Gmail’s Starred list. Still, the advantage of stars is that, with the couple of dozen or so that Gmail offers, users can flexibly mark emails as they see fit. I don’t see why stars need to go away, except to satisfy some need for cleanliness or to force people into snoozing things. Maybe not everyone wants to snooze!

    To Sum Up

    I like Inbox. I am using it almost exclusively. Scheduling/snoozing is the killer app. It helps me manage my life and my inbox. This is good.

    However, Google needs to bring in or bring back more of Gmails functionality to make Inbox a full-service email app. I believe this is their intention, but I don’t fully trust them. (Sorry, they lost my trust long ago.) Let’s wait and see.

    If you get an Inbox invitation, take it! I wish I could give you one, but Google has not granted me any (yet?). Perhaps my invites have been snoozed.


    Comments closed
  • The latest research on ACOs

    Today in NEJM you’ll find two studies and an editorial pertaining to ACO performance. Below is a brief summary and commentary.

    In Changes in Health Care Spending and Quality 4 Years into Global Payment, Zirui Song et al. examined cost and quality of care for patients served by providers participating in Blue Cross Blue Shield of Massachusetts’ Alternative Quality Contract (AQC). They compared them to the experience of comparable patients enrolled in certain employer-sponsored plans in other Northeastern states over 2009-2012. (If you’re not familiar with what the AQC is and does, read this.)

    In the 2009 AQC cohort, medical spending on claims grew an average of $62.21 per enrollee per quarter less than it did in the control cohort over the 4-year period (P<0.001). This amount is equivalent to a 6.8% savings when calculated as a proportion of the average post-AQC spending level in the 2009 AQC cohort. Analogously, the 2010, 2011, and 2012 cohorts had average savings of 8.8% (P<0.001), 9.1% (P<0.001), and 5.8% (P = 0.04), respectively, by the end of 2012. Claims savings were concentrated in the outpatient-facility setting and in procedures, imaging, and tests, explained by both reduced prices and reduced utilization. Claims savings were exceeded by incentive payments to providers during the period from 2009 through 2011 but exceeded incentive payments in 2012, generating net savings. Improvements in quality among AQC cohorts generally exceeded those seen elsewhere in New England and nationally.

    Here are two charts that illustrate some of the findings:

    AQC cost

    AQC qual

    In Changes in Patients’ Experiences in Medicare Accountable Care Organizations, J. Michael McWilliams et al. considered patients’ experiences with Medicare ACO contracts after one year, relative to before ACOs formed, comparing the change to that of matched Medicare patients not served by ACOs.

    Overall ratings of care and physicians and ratings of interactions with primary physicians did not change differentially in the ACO group, as compared with the control group, from the preintervention period to the postintervention period. In contrast, reports of timely access to care differentially improved in the ACO group. [...]

    Overall ratings of care reported by patients in the ACO group with seven or more [chronic] conditions and HCC scores of 1.10 or higher improved significantly as compared with similarly complex patients in the control group (differential change, 0.11; 95% confidence interval [CI], 0.02 to 0.21; P = 0.02; differential change with adjustment for preceding trends, 0.20; 95% CI, 0.06 to 0.35; P = 0.005).

    In the editorial Accountable Care Organizations — The Risk of Failure and the Risks of Success, Lawrence P. Casalino wrote that

    ACOs represent the best attempt to date to move away from business as usual and toward health care that will improve patients’ health and will not bankrupt the country. If ACOs fail, it may be a long time before a similarly bold concept emerges. [...]

    [Yet, t]he performance of ACOs to date has been promising but not overwhelming. Although some ACOs have gained a substantial return on their investment in improving the health of their patients, many have not. [...]

    The ACO movement is unlikely to succeed unless health insurance plans dramatically increase their number of ACO contracts and unless CMS modifies specifications for its ACO programs — a course that the agency is considering.

    I think Casalino strikes the right tone. There are some encouraging findings about ACOs in the literature, both in the new work by McWilliams, Song, and colleagues, and in prior work. But it’s both early yet and unclear whether the most promising findings from the AQC can be generalized.

    Across public and private ACOs, 18 million Americans receive care from one. Massachusetts is particularly dense in ACOs, as Song et al. write: 85% of physicians in the state have entered the AQC, 72% of Tufts Health Plan commercial managed care enrollees are under global budgets, and five organizations have joined the Medicare Shared Savings Program. This makes Massachusetts a convenient laboratory for ACO-like models, but it also makes Massachusetts unusual and threatens generality of findings from the state. Perhaps other features of Massachusetts are responsible for a tendency for ACO participation and their outcomes.

    I would give ACOs another five or so years before drawing any strong conclusions about what they can do. Even a few years of generally positive results is insufficient to declare victory. It’s reasonable to be optimistic, but cautiously so. A lot could still go wrong.


    Comments closed
  • JAMA Forum: The 2014 Midterm Elections: Is the ACA Still a Political Flashpoint?

    My latest at the JAMA Forum:

    As this year’s national elections near, it’s striking how different the rhetoric around the Affordable Care Act (ACA) is from the last election. Even more noticeable is how different things are from 1 year ago. Parsing the way politicians are talking about the ACA and how data describe how it’s working may give us a clue to what to expect in the coming year.

    Go read!


    Comments closed
  • Want your medical record? It’ll cost you.

    Who owns medical records? Technically, the records themselves are the property of the physicians and hospitals that compile them. But the law has long recognized that patients also have rights to those records.

    Most significantly, under HIPAA’s Privacy Rule, a provider must, upon request, give a patient a copy of her medical records. But, to cover the costs of copying and postage, the Privacy Rule allows providers to charge those patients “a reasonable, cost-based fee.”

    Many providers don’t bother imposing fees at all. But some claim extraordinary costs—often several hundreds of dollars—for copying and releasing patient records. Responding to the potential for abuse, more than two-thirds of the states have imposed caps on fees. The caps vary, but they typically range from $40 to $70 for a 100-page record.

    Excessive charges for patient records are obviously burdensome, especially for the poor. But the charges might be problematic for other reasons as well. Imagine, for example, that you’d like to switch doctors. You’d also like to bring your medical records with you. If it’s going to cost you hundreds of dollars to get those records, might you just sit tight? In other words, could fees for copying medical records impede competition?

    A cool new paper (gated, unfortunately) by Laurence C. Baker, Kate Bundorf, and Daniel Kessler brings some data to answer that question. To my surprise, they “find that patients from states adopting caps on copy fees were significantly more likely to switch doctors.” In any given year, they find, patients were 11% more likely to switch primary care doctors and 13% more likely to switch specialists.

    The authors also wondered whether providers were more likely to adopt electronic medical records in state with caps on copy fees. Allowing providers to charge whatever it costs to copy and send medical records dulls their incentives to improve their record-keeping systems. Might caps encourage them to adopt electronic medical records?

    Again, the data suggest that the answer is yes. “Hospitals from states that imposed a cap,” they write, “were approximately 8 percentage points more likely to adopt an EMR.” Not bad for a simple change in the law.

    At the end of the day, I’m left wondering if there’s any defensible policy reason for not capping copying fees. The caps don’t appear to increase prices—the authors checked that. And it’s hard to fathom why we should tolerate exorbitant charges that make it even harder than it already is to navigate the health-care markets.

    Plus, if we’re prepared to spend a bunch of money through the HITECH Act to spur adoption of electronic medical records, shouldn’t we also impose stringent caps on copying to help move that effort along? It’s distasteful that providers can force patients to bear the costs of their paper-bound inefficiencies. We should put an end to the practice.


    Comments closed
  • AcademyHealth: What hospital leaders think of quality metrics

    I have written, both at AcademyHealth,  here, and even the New York Times about my skepticism about pay for performance measures. One of my main concerns is that the measures picked often don’t translate into actual benefits seen by patients themselves. But that’s just my opinion? What do the people in charge of running the hospitals and programs to improve quality think?

    A recent study in JAMA Internal Medicine helps to answer that question. I talk about that, and more, in my latest post over at the AcademyHealth blog. Go read!


    Comments closed
  • Choosing a health plan is hard, even for a health economist (me)

    The following originally appeared on The Upshot (copyright 2014, The New York Times Company).

    A confession: I am a health economist, and I cannot rationally select a health plan.

    I buy health insurance through the Federal Employees Health Benefits Program, or F.E.H.B.P., which is very similar to the Affordable Care Act’s exchanges. Like the exchanges, the federal employee program runs an online marketplace with a choice of plans, which vary by region.

    Most workers don’t have a lot of choice among plans offered by their employer. But the federal employee program offers me about 20 plans to choose from, and a similar number to almost all other federal employees. This puts me in a position akin to a consumer selecting among many plansin an Affordable Care Act exchange or a Medicare beneficiary selecting among many Medicare Advantage plans.

    I have a lot of sympathy for consumers in these markets. Comparing health plans is hard, even for a health economist like me. (And it’s arguably harder on the Affordable Care Act exchanges, where consumers may also need to report income and apply for subsidies. Federal employees just need to choose a plan.) Each year when I shop for coverage through my employer, I feel like I’m buying myself at least as much grief as I am insurance.

    In one sense, buying health insurance is not different from buying any other product, like a laptop computer or a refrigerator. There are two things to consider: how much you pay (the price) and what you get (the quality). Quality can mean a lot of things for a health plan, and your criteria may differ from mine. For me, the most important aspect is which doctors and hospitals are in its network and, hence, most generously covered. (Some plans cover out-of-network providers less generously; some not at all.)

    A health plan’s price is more amenable to quantitative analysis, but still hard to assess.

    Each laptop has a sticker price, as does each refrigerator. Health insurance has not one but many price-like characteristics. The premium is the most salient price, perhaps. But there are lots of others like co-payments (fixed dollar amounts you pay each time you visit a doctor, get a lab test or pick up a prescription), co-insurance (a percentage of the cost you pay for each visit, test or prescription), and deductibles (how much you pay before your plan pays a single dollar). Complicating matters, deductibles do not apply to every service, and co-payments and co-insurance can vary by service — a different amount for a hospital stay vs. a primary care visit vs. a visit to a specialist, for a brand-name drug vs. a generic, and so forth.

    Given all this, computing something like a sticker price for a plan is daunting. The actual amount an insurance plan will cost me next year is its premium plus a complex interaction of its various other prices with the specific types of health care services my family will use. Fortunately, for federal employee plans, there is an online resource that helps simplify this calculation. Using The Guide to Health Plans for Federal Employees & Annuitants, federal employees can compare the total cost of premiums and cost sharing of plans for low, average and high levels of health care spending. (Low is about $3,000 in annual health care spending, and high is about $30,000.) This guide, which I have used, also includes plan quality ratings. A similar guide exists for the Illinois Affordable Care Act exchange, but for no others.

    One problem is that I don’t know how much or what kind of health care services my family will use next year. But based on past experience, I can make a reasonable guess as to whether it will be low, average or high. Seeing how my total cost for each plan varies across that range helps me understand the consequences of increases or decreases in use.

    But I would find it helpful to supplement that approach with a more precise calculation based on the level of health care my family tends to use. For instance, how much would each plan have cost me last year? The answer is a much closer analog to the type of sticker price one sees for refrigerators and laptops. I’m aware of no online insurance market that provides this type of information.

    Already, the lack of price transparency is enough to make a health economist despair. But it gets worse.

    Some aspects of plan quality are available to most consumers, like consumers’ ratings of customer service and how well doctors in each plan communicate with their patients. But a crucial feature of health plans is not as easily or widely accessible: the extent to which each covers services provided by one’s favorite doctors and hospitals. Except on a handful of Affordable Care Act exchanges and for federal-employee participants in the Washington area, such network information is typically available only on plans’ websites, making gathering and comparing plan networks prohibitively difficult. Moreover, which doctors are in a plan’s network can change over time.

    If I could not precisely price a laptop or assess the size of a refrigerator (or if that size could change after I bought it), I’d have a great deal of difficulty selecting the right one for me and my family. So when I shop for a health insurance plan each year, I have very little confidence that the one I select is the one I would choose were I to have more information available to me.

    And, as a health economist, I have very little confidence that a market with this degree of opacity of prices and quality can serve consumers well. Indeed, research has shown that Medicare beneficiaries have great difficultyin selecting the lowest-cost prescription drug plan. (A Medicare drug-plan pricing and coverage tool is available on the Medicare website, but it’s a fair bet that most beneficiaries do not use it.)

    Insurance markets do not need to be this opaque. We have the technology to track health care use electronically and to create online tools that could tell every health insurance consumer exactly how much each plan would have cost them based on prior-year utilization (as well as for a range of other utilization levels) and to what extent services provided by the doctors they saw and hospitals they visited would be covered next year. This is an effort that some policy experts have called for.

    Though the Affordable Care Act’s exchanges are new, some markets in which consumers can select among many plans are not. The F.E.H.B.P. has existed with plans in competition since 1960. The competitive Medicare Advantage market grew out of predecessor programs that stretch back nearly 30 years.

    Despite this long history, we do not yet offer consumers the tools they would need to become anything like rational market participants. This could change. Companies such as Picwell and Consumers’ Checkbook are working on developing and expanding consumers’ access to such tools. This is welcome news, and it’s about time.

    If even a health economist will confess to needing better access to price and quality information when choosing a health plan, it’s a sure bet that many other Americans need it as well.


    Comments closed
  • Five more big data quotes: The ambitions and challenges

    For this roundup of quotes, I received input from Darius Tahir and David Shaywitz. Prior TIE posts on big data are here. As always, the quotes below do not reflect the views of the authors, but those of the people or community they’re covering. Click through for details.

    1. David Shaywitz in “Why Causation Is (Often) Not Causation – The Retro Humility Of Empiricism,” articulates the “strong version” of the “big data thesis”:

    A strong version of the canonical big data thesis is that when you have enough information, you can make unbiased predictions that don’t require an underlying understanding of the process or context – the data are sufficient to speak for themselves. This is the so-called “end of theory.”

    2. Darius Tahir reports on the content of a Rock Health slide deck:

    Healthcare accelerator Rock Health is predicting big advances for startups and healthcare providers using personalized, predictive analytic tools. The firm has observed $1.9 billion in venture dollars pouring into the subsector since 2011, with major venture capital firms keeping active.

    The use of predictive analytics, essentially looking at historic data to predict future developments to directly intervene in patient care, will only increase as data multiplies, the report argues.

    In 2012, the healthcare system had stored roughly 500 petabytes of patient data, the equivalent of 10 billion four-drawer file cabinets full of information.

    By 2020, the healthcare system is projected to store 50 times as much information, 25,000 petabytes, meaning machine intelligence will be essential to complement human intelligence to make sense of it all.

    See, in particular, pages 18, 19 of the slide deck from Rock Health. I found it interesting that there’s lots of use of “prediction” and “predictive” throughout the deck and no direct language of causality. This is appropriate. I also think that organizations that didn’t understand just this limitation would slip into causal language now and then. In other words, Rock Health, and likely others, know exactly what they’re selling. (I am not disparaging prediction here. It is useful. I am merely distinguishing it from causal inference.)

    3. Tim Hartford has written one of the best pieces on the limitations of big data I’ve read to date. Big data is often also “found data,” hence typically suffers from selection bias. It also invites a multiplicity of hypothesis testing; query the data enough and something (meaningless) will appear statistically significant, eventually. (More by David Shaywitz on this point here.) I recommend reading his piece in full; it includes many examples from Google, Twitter, Target, the city of Boston, and the history of polling. Here’s an excerpt, cobbled from snippets throughout:

    Cheerleaders for big data have made four exciting claims, each one reflected in the success of Google Flu Trends [which Hartford summarizes, as well as its later comeuppance]: that data analysis produces uncannily accurate results; that every single data point can be captured, making old statistical sampling techniques obsolete; that it is passé to fret about what causes what, because statistical correlation tells us what we need to know; and that scientific or statistical models aren’t needed because, to quote “The End of Theory”, a provocative essay published in Wired in 2008, “with enough data, the numbers speak for themselves”. [I quoted from and linked to that Wired article here.]
    Unfortunately, these four articles of faith are at best optimistic oversimplifications. At worst, according to David Spiegelhalter, Winton Professor of the Public Understanding of Risk at Cambridge university, they can be “complete bollocks. Absolute nonsense.” [...]

    A recent report from the McKinsey Global Institute reckoned that the US healthcare system could save $300bn a year – $1,000 per American – through better integration and analysis of the data produced by everything from clinical trials to health insurance transactions to smart running shoes. [...]

    “There are a lot of small data problems that occur in big data,” says Spiegelhalter. “They don’t disappear because you’ve got lots of the stuff. They get worse.” [...]
    The Literary Digest, in its quest for a bigger data set, fumbled the question of a biased sample. It mailed out forms to people on a list it had compiled from automobile registrations and telephone directories – a sample that, at least in 1936, was disproportionately prosperous. To compound the problem, Landon supporters turned out to be more likely to mail back their answers. The combination of those two biases was enough to doom The Literary Digest’s poll. For each person George Gallup’s pollsters interviewed, The Literary Digest received 800 responses. All that gave them for their pains was a very precise estimate of the wrong answer.

    The big data craze threatens to be The Literary Digest all over again. Because found data sets are so messy, it can be hard to figure out what biases lurk inside them – and because they are so large, some analysts seem to have decided the sampling problem isn’t worth worrying about. It is. [...]

    [B]ig data do not solve the problem that has obsessed statisticians and scientists for centuries: the problem of insight, of inferring what is going on, and figuring out how we might intervene to change a system for the better. [...]

    Statisticians are scrambling to develop new methods to seize the opportunity of big data. Such new methods are essential but they will work by building on the old statistical lessons, not by ignoring them.

    4. David Shaywitz in “Turning Information Into Impact: Digital Health’s Long Road Ahead”:

    A leading scientist once claimed that, with the relevant data and a large enough computer, he could “compute the organism” – meaning completely describe its anatomy, physiology, and behavior. Another legendary researcher asserted that, following capture of the relevant data, “we will know what it is to be human.” The breathless excitement of Sydney Brenner and Walter Gilbert —voiced more than a decade ago and captured by the skeptical Harvard geneticist Richard Lewontin – was sparked by the sequencing of the human genome. Its echoes can be heard in the bold promises made for digital health today. [...]

    [T]echnologists, investors, providers, and policy makers all exalt the potential of digital health. Like genomics, the big idea – or leap of faith — is that through the more complete collection and analysis of data, we’ll be able to essentially “compute” healthcare – to the point, some envision, where computers will become the care providers, and doctors will at best be customer service personnel, like the attendants at PepBoys, interfacing with libraries of software driven algorithms.

    5. David Shaywitz in “A Database of All Medical Knowledge: Why Not?” writes about the challenges of finding and assembling big data. Here’s the set-up:

    For scientists and engineers today, perhaps the greatest challenge is the structure and assembly of a unified health database, a “big data” project that would collect in one searchable repository all of the parameters that measure or could conceivably reflect human well-being. This database would be “coherent,” meaning that the association between individuals and their data is preserved and maintained. A recent Institute of Medicine (IOM) report described the goal as a “Knowledge Network of Disease,” a “unifying framework within which basic biology, clinical research, and patient care could co-evolve.”

    The information contained in this database — expected to get denser and richer over time — would encompass every conceivable domain, covering patients (DNA, microbiome, demographics, clinical history, treatments including therapies prescribed and estimated adherence, lab tests including molecular pathology and biomarkers, info from mobile devices, even app use), providers (prescribing patterns, treatment recommendations, referral patterns, influence maps, resource utilization), medical product companies (clinical trial data), payors (claims data), diagnostics companies, electronic medical record companies, academic researchers, citizen scientists, quantified selfers, patient communities – and this just starts to scratch the surface.


    Comments closed
  • You can use a placebo to treat a child’s cold

    I was checking out JAMA Pediatrics, and saw that they’re on placebos, too. “Placebo Effect in the Treatment of Acute Cough in Infants and Toddlers“:

    Importance  Cough is one of the most common reasons why children visit a health care professional.

    Objectives  To compare the effect of a novel formulation of pasteurized agave nectar vs placebo and no treatment on nocturnal cough and the sleep difficulty associated with nonspecific acute cough in infants and toddlers.

    Design, Setting, and Participants  In this randomized clinical trial performed in 2 university-affiliated outpatient, general pediatric practices from January 28, 2013, through February 28, 2014, children 2 to 47 months old with nonspecific acute cough duration of 7 days or less were studied. Surveys were administered to parents on 2 consecutive days, the day of presentation (when no medication had been given the prior evening) and the next day (when agave nectar, placebo, or no treatment had been administered to their child before bedtime) according to a partially double-blind randomization scheme.

    Interventions  A single dose of agave nectar, placebo, or no treatment administered 30 minutes before bedtime.

    Main Outcomes and Measures  Cough frequency, cough severity, cough bothersomeness, congestion severity, rhinorrhea severity, and cough effect on child and parent sleep.

    Colds suck, especially when a small child has one. They’re not very good at “sucking it up”, and there’s so little you can do for them. Most of the over-the-counter medicines have been pulled from the shelves, because they didn’t work and had lots of side effects.

    People are always trying to look for help, even from complementary medicine. This was a randomized controlled trial of an agave nectar formulation versus placebo versus no therapy. Everyone had a baseline measurement the night before they got their randomized “therapy”. On the next night, kids in the trial got a single dose of their “therapy” before bedtime. The main outcomes were cough, congestion, runny nose, and sleep – for both the child and parent.

    The first thing to note is that in all groups, even in the no therapy group, there were improvements from baseline. Even “being studied” seemed to have an effect. The second thing to note is that kids were significantly better on the agave nectar than in the “no therapy” group. But there was no difference between kids in the agave nectar group and the placebo group. The placebo group also did significantly better than the “no therapy” group.

    This is why I tell parents who have kids with colds to “try anything” that I don’t think has a harm. I include cost in the “harm” category. If you want to tell your child you’re giving them a special drink of warm tea, that’s awesome. If you have a special moisturizer that “soothes” their chest, that’s great. If you want to put a little agave nectar in water and tell them it’s medicine, I’m ok with that, too.

    Of course, in this study, the placebo effect was likely working on the parents as well as the kids. Which of them received the bigger benefit isn’t clear. But the effect is there.

    When we tell you that no medicines work for colds, we’re not telling you to do nothing. We’re telling you that nothing works better than placebos. You absolutely should use placebos. They work! Here’s a study that proves it.


    Comments closed
  • Healthcare Triage: Wellness Programs Don’t Seem to Work as Advertised

    The latest Kaiser Family Foundation survey on employer sponsored health insurance focused on the fact that growth in premiums in 2013 was as low as it has ever been in the 16 years of the survey. And that’s awesome. Health insurance premiums have been rising more quickly than we’d like for a long time. But buried in the details of the report were some interesting insights into how employers think about controlling health care costs. One example is that they’re very fond of workplace wellness programs. This is surprising, because while such programs sound great, research shows they rarely work as advertised. Watch and learn!

    This is based on Austin’s and my NYT Upshot piece on wellness programs. Links to references can be found there.


    Comments closed
  • Stories doctors tell: Anecdotes and evidence

    Peter Kramer is a psychiatric educator and the author of the widely-read Listening to Prozac. In a recent New York Times editorial, he argues for the value of stories, including case studies, as a genre of medical literature. Unfortunately, his essay doesn’t convey the difference between anecdotes and evidence.

    But first, Kramer gets important things right.

    A story narrates a sequence of events driven by the actions of one or more characters. We are built — perhaps even wired — to understand the world through stories. Stories can explore how medicine links to larger human concerns. Through stories we can imaginatively connect to the experiences of others, identify with their suffering, and locate their needs for care.

    Stories also matter in the practice of medicine. Doctors tell stories to teach other physicians, to brief colleagues on a case, and to inform patients about a possible course of care so that decision making can be shared. Learning to tell good stories — to be concise yet complete, to be concrete yet vivid, to motivate action while cautioning about uncertainty — is a critical skill for care givers. Kramer describes how stories about cases are the only source of guidance when he reaches the end of medical knowledge and

    …I exhaust the guidance that [clinical] trials can give — and then I consult experts who tell me about this case and that outcome.

    Finally, important clinical advances often begin with an observation about a case.

    [V]ignettes can do more than illustrate and reassure. They convey what doctors see and hear, and those reports can set a research agenda.

    So far, so good. But Kramer goes off the rails when he seeks to rehabilitate stories as a kind of medical evidence. He argues that in the case of modern antidepressants.

    [D]octors had not waited for controlled trials. In advance, the [hypothesis that Prozac made you "better than well"] had served as a tentative fact. Treating depression, colleagues looked out for personality change, even aimed for it. Because clinical observations often do pan out, they serve as low-level evidence — especially if they jibe with what basic science suggests is likely… this approach, giving weight to the combination of doctors’ experience and biological plausibility, stands somewhat in conflict with the principles of evidence-based medicine.

    In the absence of evidence, doctors must decide based on experience and biological plausibility. But Kramer doesn’t convey why evidence-based medicine deprecates anecdotes as evidence. Kramer seems to have forgotten the harms that can occur when clinical stories are taken as evidence.

    Siddhartha Mukherjee’s The Emperor of All Maladies documents the risks associated with medicine by story. The great surgeon William Stewart Halsted (1852-1922) had a biologically plausible belief — confirmed, he thought, by his experience with cases — that extensive removal of tissue surrounding a tumour was necessary to cure breast cancer. He taught the procedure to the next generation of surgeons and for decades, hundreds of thousands of women were radically disfigured by removal of their breasts and extensive tissue in their chests. A clinical trial published in 1985 showed no benefit for radical compared to conservative surgeries.

    This isn’t just a 19th century problem: Read Mukherjee’s history of bone marrow transplantation and high dose chemotherapy for metastatic breast cancer, also covered on TIE. Again, a treatment entered practice far ahead of the completion of clinical trials and again, tens of thousands of women suffered from a painful and expensive treatment that lacked benefit.

    Medicine by case study has been a particular menace in mental health care.

    I began working in mental health after college, as a therapeutic child care worker at a children’s mental health institute that was based on the ideas of Bruno Bettelheim. Bettelheim is now forgotten, but in the 1970s he was a renowned psychoanalyst and public intellectual. There were many therapists like him in that era: charismatics with compelling stories about rescue through psychotherapy.

    Bettelheim published highly influential books reporting cases in which he cured autism through psychotherapy. His theory of autism attributed the disorder to the emotional frigidity of the mother. This made sense in the light of the psychoanalytic theory of the day, itself built on case studies.

    We know now that Bettelheim’s therapy was useless, that his theory was absurd, and that the man himself was a charlatan who may have abused his patients. Most importantly, we now validate psychotherapies through clinical trials, like any other medical intervention. The age of charismatic therapists is over and we have modest but realistic estimates about what psychotherapy can achieve.

    So why do stories fall short as sources of trustworthy medical evidence? Case histories often lack the basic elements of science: objective measurement, public demonstration of effect, and replicability.

    But the most important problem with a case study is epistemic: it cannot carry information about causality. An action is a cause if doing it leads to an outcome and not doing it would not. So to make an inference about causation we need more than a biologically plausible theory. We also need information about would have happened to a patient not only in the factual case where the treatment occurred but also in the counterfactual case where it did not. Valid clinical studies arrange observations so that we can make inferences about both the factual and counterfactual cases, and thereby infer causality. But a case history can present only the factual side. The case study makes sense to us because we supply the belief about the causal linkage. It doesn’t provide independent evidence about that link.

    So, medical humanists and ordinary caregivers must tell stories and yet stay in the realm of science. How? A challenge of writing strong science narratives is to spot the implicit assertions of causality and make sure that they are consistent with scientific evidence, wherever possible. And otherwise, to convey to the reader the uncertainty about causality.


    TIE has many posts on causality.

    Comments closed