• Surgical skill and outcomes

    This paper by John Birkmeyer and colleagues in NEJM has received a lot of praise by physicians I follow on Twitter.

    METHODS: We conducted a study involving 20 bariatric surgeons in Michigan who participated in a statewide collaborative improvement program. Each surgeon submitted a single representative videotape of himself or herself performing a laparoscopic gastric bypass. Each videotape was rated in various domains of technical skill on a scale of 1 to 5 (with higher scores indicating more advanced skill) by at least 10 peer surgeons who were unaware of the identity of the operating surgeon. We then assessed relationships between these skill ratings and risk-adjusted complication rates, using data from a prospective, externally audited, clinical-outcomes registry involving 10,343 patients.

    RESULTS: Mean summary ratings of technical skill ranged from 2.6 to 4.8 across the 20 surgeons. The bottom quartile of surgical skill, as compared with the top quartile, was associated with higher complication rates (14.5% vs. 5.2%, P<0.001) and higher mortality (0.26% vs. 0.05%, P=0.01). The lowest quartile of skill was also associated with longer operations (137 minutes vs. 98 minutes, P<0.001) and higher rates of reoperation (3.4% vs. 1.6%, P=0.01) and readmission (6.3% vs. 2.7%) (P<0.001).



    • I’m curious if they have a Cronbach’s alpha (or a similar measure of internal consistency) of the reviewer’s grading/ratings.

      I’m not trying to pick apart the paper as inaccurate, I have no doubt their primary conclusion is accurate. My only problem is the conclusion is kind of boring. Peer-rated “skill” correlates with good outcomes. But how these people define skill is left in a black box and we don’t know how much agreement there is between graders.

      I bring it up because the definition of skill seems like an actionably useful thing. Suggesting we should have more “skilled” surgeons is not really actionable.

    • It seems to me the conclusion seems to be “just do not to go to a surgeon in the bottom quartile”. Are any of the other post hoc comparisons significant?