Dan Ho and Becky Elias have a wonderful article in the Boston Review about their new research on the benefits of peer review in restaurant inspections.
Beginning in 2014, we designed a randomized, controlled trial to test the effectiveness of peer review with the food safety staff of King County, where Seattle is located. Half of the inspection staff was randomly assigned to engage in peer review. For sixteen weeks, these inspectors spent one day per week with a randomly selected fellow inspector, taking turns conducting inspections and independently scoring health code violations. We then used information from these peer inspections to identify and train for violations that cause the most confusion.
The results were remarkable. We discovered that, when observing identical conditions in restaurants, health inspectors disagreed nearly 60 percent of the time. Inspectors differed in their assessments of risk magnitude and in interpretations and applications of the health code to particular circumstances, resulting in varying citations for the same condition. Food science is evolving, and the FDA model food code spans nearly 800 pages, so it may not be surprising that implementation varies so much. As one inspector put it, “In the beginning, we [thought] we kn[e]w the code,” but comparing assessments with others provided a “wake-up call.”
A bona fide randomized controlled trial of a government program in a real-world setting? This is rare and exciting stuff, with enormous resonance for public administration and administrative law. (The full research paper will soon be published at the Stanford Law Review.)
It’s also relevant to medicine. If peer review works for restaurant inspections, might it also work for health care? As with restaurants, similar patients are often treated differently depending on who treats them and the norms that prevail in the community. Practice guidelines can moderate that variation, much like the FDA model food code. But guidelines can’t tell you how to deal with atypical cases and they aren’t always followed anyhow. Stitching peer review into medical practice might promote consistency while honoring the clinician’s imperative to attend to the idiosyncrasies of particular patients.
To some extent, peer review has always been a part of medicine—think here of M&M conferences. And it’s starting to get more attention. As Ho and Elias note, Atul Gawande wrote an important New Yorker article in 2011 about surgical coaching. Here at the University of Michigan, Justin Dimick has landed an NIH grant to investigate surgical coaching more generally.
The Ho and Elias study makes me cautiously optimistic about efforts like these. Peer review might or might not work as well for hospitals as it does for restaurants. But it’s certainly worth exploring.
