I’m back from vacation and just trying to keep everyone honest. Or pick a fight. To avoid conflicts of interest, I won’t rate Aaron and Austin’s coverage. But you can in the comments.
UPDATED to add Leonhardt’s NYT article, h/t to Dennis in the comments
| Publication | Author | Correctly identifies the important findings of the study? | Notes the methodological strengths or weaknesses of the study? | Does not improperly extrapolate or interpret the study? | |
| ★★★★ | WSJ | Anna Wilde Mathews | Yes – in the lede and the opening paragraphs | Substantial explanation in the middle of the article | Carefully noted the authors’ caution about extrapolation |
| ★★★★ | Slate | Ray Fisman | Yes – in various paragraphs across the story; story led with policy context | Substantial explanation in the middle of the article | Substantial explanation of limitations in the penultimate paragraph |
| ★★★½ | Wash Po | Ezra Klein | Yes – in detail | Leads with the methodology; substantial discussion | Mentions some limitations, but not fully |
| ★★★½ | The Atlantic | Megan Mcardle | Yes, in the middle, after critiquing improper extrapolation by others | Describes problems with prior studies; great description of “Rorschach effect;” fails to describe the unique methodology of this study | Very careful description of problems extrapolating the study to mortality |
| ★★★ | NYT | David Leonhardt | Yes, with good email quotes from several of the study authors | Yes, but did not explain why the lottery methodology was superior; discussed the political backgrounds of the study authors to suggest bipartisanship | Focused on “health” outcomes in the article without a careful explanation of the authors’ cautionary statements |
| ★★★ | NYT | Gina Kolata | Yes – in the lede and the opening paragraphs | Substantial explanation in the middle of the article + brief mention in the lede | Did not note the authors’ caution about extrapolation |
| ★★ | Think Progress | Matt Yglesias | Yes, but includes “saves lives” which was not in the study | Mentions study design and calls the study “rigorous” but does not tell us why | Fails to mention the authors’ caution and improperly extrapolates to mortality |
| ★★ | Cato | Michael F. Cannon | No – goal of article is to critique Yglesias’ coverage of the study | No – does not mention the methodology | Critiques Yglesias’ coverage of the study and tries to prevent (improper) extrapolation |
| ★ | Huffington Post | No byline | Yes, briefly | No mention at all | No mention at all |
