I was emailing with some journalist-friends about what reporters on policy or social science beats should really know. It’s unreasonable to expect reporters to have the full skill-set of academic empirical research. I may not be a representative voice on this issue, but I think the below set is pretty reasonable. Roughly speaking, these are skills I would expect of the median undergraduate public policy major across the country, especially an undergraduate studying what’s on the reporter’s own beat:
- Knowing how to interpret the size of a linear regression coefficient.
- Understanding what statistical significance means–and doesn’t mean.
- Understanding what r-squared is.
- Having some sense of why regressions go wrong—for example, the possibility of selection bias and reverse causality.
- Understanding the inherent limitations of cross-sectional analyses such comparisons of mortality across states with different levels of inequality.
- Innate suspicion of any complex statistical analysis that lacks a compelling underlying causal story. This would include the fact that most elaborate non-experimental analyses that reach dramatic or counter-intuitive conclusions are simply wrong.
I don’t think it’s reasonable to expect reporters to know anything that requires a derivative, the words “heteroskedastic,” or anything involving the term “instrumental variables.”
If covering health policy or public health, I would add a few more items:
- Knowing what odds ratios and relative risks are, and how these concepts can be misleading when base rates vary.
- Understanding the basic properties of screening tests–e.g. the index card of formulas that link the sensitivity and specificity of screening tests and underlying prevalence of a condition to positive and negative predictive value.
- Familiarity with milestones such as the RAND Health Insurance Experiment, the 5-10 leading articles in the field and the classics such as Paul Starr’s and Ted Marmor’s books that provide the context of current health policy.
- Familiarity with basic vocabulary such as moral hazard and adverse selection.
- Understanding the basic mechanics of a clinical trial, including terms such as “intent to treat” and “effect of treatment on the treated group.”
- Understanding the rudiments of population genetics and basic facts about genes and chromosomes.
- Knowing basic numbers on U.S. health expenditures, overall and within the key categories subject to debate.
- Knowing basic numbers regarding leading causes of mortality.
Unfortunately, we live in a stupidly innumerate society and popular culture. This puts reporters at a tremendous disadvantage, since they come out of this culture. They need to fight the stupid and tool-up.
None of the above items is very hard. You can’t be a good film critic if you’ve never seen Citizen Kane, Star Wars, or the Godfather. You can’t cover a foreign country properly if you don’t know the language and culture. You can’t cover public policy properly if you don’t speak the language and lack bare-bones familiarity with the tools of that trade.
That’s my list, anyway. Is it reasonable?
@haroldpollack
