• A key ADHD study led to an over-valuing of medications

    From Alan Schwarz in The New York Times:

    Twenty years ago, more than a dozen leaders in child psychiatry received $11 million from the National Institute of Mental Health to study an important question facing families with children with attention deficit hyperactivity disorder: Is the best long-term treatment medication, behavioral therapy or both?

    The widely publicized result was not only that medication like Ritalin or Adderall trounced behavioral therapy, but also that combining the two did little beyond what medication could do alone. The finding has become a pillar of pharmaceutical companies’ campaigns to market A.D.H.D. drugs, and is used by insurance companies and school systems to argue against therapies that are usually more expensive than pills.

    But in retrospect, even some authors of the study — widely considered the most influential study ever on A.D.H.D. — worry that the results oversold the benefits of drugs, discouraging important home- and school-focused therapy and ultimately distorting the debate over the most effective (and cost-effective) treatments. [Emphasis added]

    The study is the Multimodal Treatment of Attention Deficit Hyperactivity Disorder (MTA) study. It is among the best studies ever conducted in child psychiatry. Large samples of children diagnosed with ADHD were randomized into groups that received (a) the usual treatment offered in their community (‘Community Care’), (b) high-quality behavioral treatment, (c) medication following evidence-based guidelines, or (d) behavioral treatment and medication (‘Combined Treatment’). Community Care (a) might have included medication or behavioral treatment, but not the carefully-managed, evidence-based therapies delivered in conditions (b)-(d). Children were followed for 14 months. The Figure below presents the key results.


    Statistical analyses showed that for most of these outcomes, behavioral treatment, medication, and combined treatment showed better results than community care. But although combined treatment was somewhat more successful than medication alone for some outcomes, those differences were not statistically reliable. The researchers concluded that

    Combined behavioral intervention and stimulant medication—multimodal treatment, the current criterion standard for ADHD interventions—yielded no significantly greater benefits than medication management for core ADHD symptoms; this parallels findings reported by others.

    The implication for practice was to medicate, medicate, and medicate. Medication is a much easier health service to deliver than behavioral treatment. Primary care physicians rarely have skills in behavioral treatment and even if they do they do not have the time that behavioral treatment requires. Similarly, it is much easier for a parent to give a child a pill than to change your behavior and your child’s. (Trust me on this: I raised five children.)

    Yet several of the MTA authors are concerned that the study’s apparent validation of a “medication-only” strategy may have harmed children. They — and I — believe that even if there is a short term role for medication, children with ADHD symptoms need training to build the cognitive and social skills to function successfully without medication. Co-author Dr. Lily Hechtman:

    I hope [the MTA] didn’t do irreparable damage. The people who pay the price in the end is the kids. That’s the biggest tragedy in all of this.

    What went wrong?

    It’s not that the MTA researchers were compromised by the pharmaceutical corporations. The study was conducted without drug company support. The constant promotion of medication by Big Pharma created a receptive environment for the message that “drugs are all you need,” but that’s not the authors’ fault.

    The most important problem with the reception of the MTA findings was that we didn’t take a sufficiently developmental perspective in thinking about the problem. Learning to regulate your behavior and focus your attention are among the most important developmental tasks in childhood. A lot of this learning requires training by adults: This is much of what parenting and primary education are about. Some children need more help in this than others. Medication may be part of that help, but it is not a substitute for training. These developmental processes continue throughout childhood. This means that a 14-month study was not a sufficiently long period to draw definitive conclusions about the value of behavioral treatment or combined therapy.

    ADHD has serious, long term effects on children. There hasn’t been nearly enough research on behavioral treatments for it; that has got to change. If you are parent and your child is taking an ADHD medication, I’m not recommending that you stop. But I do recommend that you find out what other treatment options may be available for you.

    Disclosure: Like Aaron, I’ve published on ADHD. I am the statistician for a group seeking to developed an improved method to treat ADHD and I receive funding for this work. Several of the MTA investigators are my friends.


    • “Primary care physicians rarely have skills in behavioral treatment and even if they do they do not have the time that behavioral treatment requires”

      what leads you to believe that they should have the skills and time to do behavioral treatment?

      what is wrong with saying primary care is not the appropriate place for helping patients with ADHD?

      • jamzo,
        Sorry, I didn’t mean to imply that PCPs should be delivering this service.

        But we do need primary care docs to serve as a gateway for these services. That isn’t working now and we need to fix that. Having behavior therapists co-located in the practice might work for a large group practice. In our research we have explored a model for linking many local practices with a central mental health system.

    • How do we know it led to an “over-valuing” of medication? I might agree that “children with ADHD symptoms need training to build the cognitive and social skills to function successfully without medication” (Although… maybe not. We don’t feel that way about children with other chronic conditions, like diabetes or certain mental illnesses), but do we know that kids on the “only medication” treatment do appreciably worse at learning to cope without medication than those in the combination intervention? i don’t see any data about that.

      • PLW,
        I agree that my view needs more evidence. Writing that post is not a project I can take on today.

        I will say, though, that skill training is an essential component of the care of many if not most chronic diseases. There is much to learn about successfully coping with diabetes. Similarly with asthma, osteoarthritis, many cardiovascular diseases, and depression for that matter. “Medication only” would, in my view, be substandard care for each of these disorders.

    • “The constant promotion of medication by Big Pharma created a receptive environment for the message that “drugs are all you need,” but that’s not the authors’ fault.

      The most important problem with the reception of the MTA findings was that we didn’t take a sufficiently developmental perspective in thinking about the problem.”

      I’m curious who “we” is in that second quoted sentence. I completely agree with the conclusion, I’m just wondering to whom the message needs to be communicated, and what can we do to get it right.

      It may not be the authors’ fault, but do you think there is “fault” somewhere (vs. just well-intentioned bad decision making)? I wonder if direct-to-consumer advertising had an adverse affect… I mean, the point at which a parent seeks medical assistance for a child’s behavior must be related to what they consider “normal” and what they consider an appropriate response to ab-“normal”, and I remember that balance changing quite a bit when Ridalin hit the market, but I don’t remember WHY I felt that way.

      I hope the authors aren’t beating themselves up… I think you hit it properly when you said “_reception_” of the MTA findings… if the importance of that piece of the puzzle was overstated (by advertising or any other means), this is again not the authors’ fault.

      • CM,
        “We” meant “Me & a lot of other researchers in the field who promoted this understanding of the MTA results.”

        Glad you liked “reception” — I struggled with what to say at that point.

    • Reading through the study, there’s a more fundamental flaw in how the study has been used. The authors write: “while combined treatment scored numerically “best” on most outcome measures, we did not have statistical power to detect small effects, such as those that might exist between combined treatment and medication management…. lack of significance is never proof of the equivalency of treatments.”

      Unfortunately, while the authors report whether the difference between Medication Management and Combined Treatment is statistically significant, they do NOT report point estimates or confidence intervals (see Table 5).

      The study should properly have been interpreted as “The best-available evidence indicates that Combined Treatment is better than Medication Management, though our confidence intervals are wide.” A Bayesian decision-maker reading this study should 1) believe Combined Treatment is better, 2) be willing to pay more for Combined Treatment than Medication Management, 3) wish the authors had conducted a cost-benefit analysis, and 4) think that the wide confidence intervals calls for further research.

      Nothing in this study should make one believe that Combined Treatment and Medication Management are equivalent.

      The focus on statistical significance over best available evidence (point estimates and confidence intervals) has real costs. I think Deirdre McCloskey’s book is insightful on this point: http://www.amazon.com/The-Cult-Statistical-Significance-Economics/dp/0472050079

    • Disagree Keith, their interpretation is exactly correct. It would be misleading to say “combined was higher” because there was no significant differences between them. If you are just going to say highest is best, and remove any impact of sampling error from your decision making- why bother using a statistical test at all?

      Secondly, point estimates and confidence intervals are going to be equivalent to statistical significance- what would be sample size irrelevant would be effect sizes. WIth no statistically significant differences between the two groups that difference’s CI will include 0.

    • My son is an ADHD patient as well. The Hallowell Center where he is treated has always advocated for a combination approach to him realizing his full potential as he ages and becomes an adult. The article is correct in its observation that other support resources to address behavioral therapy are much more limited partly as a result of the emphasis on medication regimens. We as parents and family members need to push for better study and resources to support a more balanced long-term approach to this disorder.

    • IIRC, in the MTA behavioral interventions did demonstrate statistically significant benefits in specific sub-scales of externalizing (oppositional and aggressive) symptoms, especially in boys.

      This post is a very good analysis of the problem, and I think the conclusion that the 14 month study was too short and did not give enough consideration to development and maturation is right on the money.