• Clinical guidelines that should make you weep

    If you thought I was upset earlier, I was just getting started. The nonsense in some medical societies’ clinical guidelines really makes my blood boil.

    Let’s go back to the beginning. In 1990, the Institute of Medicine recommended the development of clinical guidelines to reduce inappropriate variation in practice. This commenced an explosion of guidelines. For example, the National Guideline Clearinghouse now includes 471 guidelines for hypertension and 276 for stroke alone. For maximal variation reduction, the number of guidelines you need is one. Though it’s possible all of the hundreds of clinical guidelines on a subject say the same thing, that’s not the case. How can variation be reduced with so many guidelines?

    As if that were not bad enough, the recommendations in many guidelines is based on some pretty flimsy evidence, or none at all.

    A recent study of the quality of evidence in cardiology guidelines showed that of more than 7000 recommendations, a median of 11% were based on data from randomized controlled trials (RCTs) and 48% on expert opinion, case studies, or standards of care. In this issue of the Archives, Lee and Vielemeyer report on a similar analysis of guidelines in infectious diseases. Their study shows that of more than 4000 recommendations, 14% were based on data from RCTs and 55% on opinion or case series. Both studies showed that although the number of recommendations increased across time, few of the new recommendations were based on RCT data.

    Are you crying yet? If not, maybe a picture would move you. Here’s one from Lee and Vielemeyer (click to enlarge):

    guideline evidence

    In case you don’t know, Level I evidence is RCTs, Level II is from non-randomized trials, cohort or case-controlled studies, and Level III is, you guessed it, from opinions or descriptive studies. The figure above is showing you all the guidelines in the authors’ data that could be analyzed in this way. They didn’t cherry pick. These are all of them. The amount of Level III evidence (grey) should disturb you.

    It makes sense to reduce variation in cases for which we have good evidence, such as that from RCTs. It does not make sense to reduce variation in cases for which all we have to go on is expert opinion, case studies, or standards of care. Those are the instances in which we should be encouraging doctors to use their best judgement and encouraging researchers to flesh out the body of evidence. And, by the way, there are also documented cases of guidelines going against what the best evidence says, even RCTs. If you listen to the SMART EM podcasts, you’ll hear about some of them.

    Now maybe, in principle, a group of experts can divine an approach that is more likely to do good than harm. After all, following expert opinion is better than nothing, right? No, not always. It depends who the experts are and what their conflicts are. Sadly, we can’t be sure what’s driving the expert opinions that inform many clinical guidelines.

    Major guideline-releasing organizations have recognized the importance of having a rigorous policy regarding conflicts of interest; such policies manage and balance potential conflicts rather than eliminating them. The ACC/AHA’s [American College of Cardiology/American Heart Association’s] code regulating potential conflicts of interest requires the collection and publication of relationships with industry by guideline-writing groups as well as peer reviewers. Relationships are orally disclosed at every meeting, votes are recorded for all recommendations, and members with significant conflicts abstain from voting, although they can participate in the discussion. In addition, the ACC/AHA task force now requires that 30 to 50% of writing group members have no conflicts of interest, and the guideline writing group must be chaired by someone with no conflicts of interest. Finally, there is no industry funding for guideline development, although the ACC and AHA do receive industry support for distribution of guideline derivative products such as pocket guides.

    I’m sorry, the requirement is that only 30-50% of members of the guideline writing committee have no conflicts of interest?! That’s a joke. That leaves the possibility that a majority of members are conflicted. And industry support for guideline derivatives is not much of a firewall. Worse, the process of some guideline development is opaque. What’s being hidden and why?

    All in all, this stinks. In the absence of a transparent, conflict-free process, it’s not unreasonable to throw away recommendations largely based on expert opinion. I think guidelines can play a very helpful role in improving the efficiency of medicine. But they have to be based on something real and they have to be developed in a manner that can be trusted. Is there any sound, patient-centered argument for allowing conflicts and hiding methods?


    • I had no idea that things were so bad.
      First, to have 471 guidelines for hypertension and 276 for stroke is ridiculous! Then, to have a process which is opaque and open to conflict of interest makes the whole thing absurd.
      I assumed that specialty societies would be fair arbiters of the guidelines but it appears not.
      What can be done? Can we have a fair, neutral, expert body vet guidelines?

    • Interesting. You guys should put in some hang time at http://www.sciencebasedmedicine.org

    • Do you have any impressions (or better yet, data) re: how many of these guidelines are actually frequently used? Ie, are most decisions (if they’re made in reference to guidelines at all, which is another problem) made using a narrow set of guidelines? Or are many of these in widespread use?

    • “Is there any sound, patient-centered argument for allowing conflicts…”

      I’m sure the argument put forth is that there aren’t enough non-conflicted experts. People who intensely study PCI vs. CABG, for example, are likely to perform one of the two procedures or otherwise be involved with the industry around them in one way or another. I have no way of knowing how true that might be though.

    • Hello! Very interesting post. As you mentioned cardiology i would like to share my presentation about Clinical guidelines on hypertension in adolescents http://writing-help.com/blog/essay-sample-clinical-guidelines-2/