• Why evidence-informed policy is hard

    I’m pleased to write that an ungated PDF of “‘Developing Good Taste in Evidence’: Facilitators of and Hindrances to Evidence-Informed Health Policymaking in State Government,” by Christopher Jewell and Lisa Bero, is available. Anybody interested in why evidence often fails to penetrate the policymaking process should read it, though it is depressing. Here are some quotes, some of which are from interviews conducted by the authors:

    • One official observed that in assessing the effectiveness of a new medical procedure, “I just did exactly what . . . everyone . . . is hoping I’m not. I talked to my brother-in-law and I Googled it.”
    • In reference to medical malpractice reform, one official discussed the problem with having a legislature with a large contingent of trial lawyers: “Right now you’ve got a bunch of people who could care less about the evidence unless they can manipulate it. . . . They know where their economic interests are . . . and that will trump any kind of evidence.”
    • “And you can always find somebody on the committee that’s had some bad experience and so they’re perfectly willing to tell their story, and then fifteen lobbyists stand up and talk about how different people have been harmed by this sort of legislation. . . . And they aren’t studies . . . the lobbyists even lie . . . I mean they make up stories that don’t exist.”
    • An effort to institute a system to use encrypted state health data to analyze variations in medical practice—an issue that does not affect consumers directly—met with significant opposition on the grounds that “even if you don’t identify the individual, you somehow have violated the person because you were looking at the medical record.”
    • “If the leadership in your governing body believes that government should have as limited a role as possible in providing health services to your state population, then it doesn’t matter what you say. . . . They don’t care about evidence because they don’t believe that government should have a role in providing health care.”

    @afrakt

    Share
    Comments closed
     
    • This report is incredibly discouraging and absolutely true. I don’t know how you initiate change among people who can’t be confused by the facts.

    • Great post and thanks very much for the link. Here’s another good comment about research:

      “In addition, academic researchers generally follow their own interests when choosing what studies to conduct or tailor them to specific requests for grants. Similarly, the synthesis of existing research in the form of systematic reviews is driven by the researchers’ particular interests. As a result, policymakers find that research often “sort of applies . . . but not quite. You know, if we had had this conversation earlier, we could have added this or we could have, you know, fired ten degrees to the west, and we’d have been right on in terms of really helping us all.””

      I have certainly seen this: researchers believing (genuinely) that they are doing good work by doing really meticulous meta-analyses on important topics, but ultimately doing it at a standard that far overshoots the imprecise policy world. Tough to coordinate without giving the impression of meddling in academia or massaging data for political ends.

    • This was an interesting article– glad to see it circulated more broadly. Thought I would share with other readers a model for state to employ that has worked really well in California(!). Yes, California, which I know is not always known as a bastion of evidence-based policymaking, embarked on a bold experiment for considering evidence when it comes to health insurance benefit mandate bills in the Legislature. Ten years ago, California created the California Health Benefits Review Program (CHBRP), to provide unbiased, faculty-driven analysis to the Legislature in nearly real-time (60 days). By law, CHBRP provides independent analysis of the medical, financial, and public health impacts of proposed health insurance benefit mandates and repeals. A small analytic staff in the University of California’s Office of the President works with a task force of faculty from several campuses of the University of California as well as actuarial consultants to complete each analysis during a 60-day period, usually before the Legislature begins formal consideration of a mandate bill.

      Bill sponsors and opponents regularly cite our reports, and it allows debate to consider the relative merits of a bill based upon an accepted and neutral provider of expertise (CHBRP does not make recommendations). Entities like ours (Connecticut has one, created on the California model) might serve as starting points to support states on a wide set of policy topics.