Hilda Bastian and I previously discussed the extent to which people can comprehend the statistic number needed to treat (NNT). Hilda, who knows this literature far better than I do, helpfully offered a follow-up post that included a lot more studies. At my request, research assistant Jennifer Gilbert looked through all those studies and summarized them in this PDF.
The chart below summarizes findings based on studies suggested in a post by Hilda Bastian and other studies discussed in prior posts by Austin and Hilda. Depending on how one wants to count three studies (Christensen 2003, Gyrd-Hansen 2011, and Cuite 2008—see footnotes of the chart) then out of four to seven studies, two found NNTs no more difficult to comprehend. [See the right-most column of the chart.]
This chart is included in Jennifer’s review, where you’ll find additional detail and hyperlinks to all studies.
Note that in my prior posts on this topic, based on systematic reviews by Akl and Zipkin, I had overlooked the paper by Cuite. That’s because it wasn’t in Akl, and Zipkin didn’t cite it among the NNT comprehension studies, as documented in Jennifer’s review.
From all this, here are my conclusions:
- Many people, myself included, are fond of NNTs. We find them useful for making a point about the surprisingly low rate at which some therapies are helpful.
- That is different than claiming they’re easily understood for the purposes of medical decision making. That’s an empirical question for which we should let research be our guide.
- The research base, to the extent I know it and mostly from Hilda’s posts, suggests NNTs have comprehension challenges. But, I must acknowledge, it doesn’t uniformly point in that direction.
- I would welcome a systematic review that includes all these studies and any others that make the cut. In that review, the quality of the evidence, not just number of studies, should be weighed. Also, any meaningful heterogeneity should be examined. Such a review is a substantial undertaking, and not one I have the resources to do.
(Aside: To those of you who may be disappointed in this post because it didn’t mention that treatments can cause harm too, look here.)
