This post has two points. One is substantive. The other concerns tradeoffs between academic publications and blogging. (Apologies if title is lost on non-NBA-playoff fans.)
First the substance: Drug testing of welfare recipients is a really bad idea
As a policy professional, you like to assume that some policy ideas are dead. Researchers demonstrate that it doesn’t work. Experience demonstrates that it works out poorly when it is put into practice. You think that it’s dead and buried. So you visit the gravesite, nostalgically, carrying the hockey mask it used to wear. Then a bolt of lightning strikes. It pops out of the grave, snatches away the hockey mask, and gets itself enacted into law by various state legislators across the country before you can kill it again.
As I wrote in Wonkblog this week, that’s what’s happening right now. A number of states contemplate suspicionless, population-based screening of welfare recipients for illicit drug use. Measures to do this have passed in eight states and debated in many others, despite an abundance of evidence that such policies will not accomplish their stated purposes.
Go to my Wonkblog column for more details. I should mention that I get a lot of calls from reporters on this matter because I and my co-authors published a series of papers which explored drug use among welfare recipients. We examined Michigan-specific data from the longitudinal Women’s Employment Study (WES). We also examined nationally representative trend data from the hilariously-misnamed National Household Survey on Drug Abuse (NHSDA) and the National Survey of Drug Use and Health (NSDUH).
An advocate called me a few weeks ago, and I realized that my last published analysis used data from more than eight years ago. Maybe things have changed particularly as a smaller fraction of poor family receive TANF aid after recession has hit. So I spent a few hours downloading and examining the latest prevalence data I could find from the 2011 NSDUH data.
NSDUH isn’t ideal. Indeed its defects reflect the silo’d nature of survey data used in public policy. NSDUH does a good job of measuring basic prevalence and substance use disorders (operationalized as abuse or dependence following the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition within the broad U.S. population). None of the other nationally representative surveys come close on these measures. NSDUH also includes terrific information on mental health status and other matters.* NSDUH was not designed to carefully scrutinize socioeconomic variables such as welfare receipt. Datasets such as the Women’s Employment Study or the National Longitudinal Survey of Youth cover these issues with much greater depth.
Whatever NSDUH’s shortcomings, the descriptive epidemiology remained sufficiently clear to convey the basic story. Drug and alcohol disorders are genuine issues within the welfare population. Yet these disorders are not widespread in the welfare population.
In our prior work using both high-quality Michigan data and national samples, roughly one-fifth of TANF recipients reported using some illicit substance over the past year. Roughly 10% of TANF recipients satisfy screening criteria for substance abuse or dependence, with alcohol disorders being far more common than illicit drug dependence or abuse. Ironically, the available chemical testing technologies were most sensitive in identifying casual marijuana users who probably don’t need addiction services.
To update the analysis I examined 2011 NSDUH data for the most pertinent group for welfare policy: Unmarried women between the ages 18 and 49 who live with children in their homes with incomes below 200% of the federal poverty line. I ran some cross-tabs to compare drug-use patterns among women who reported that their families had received welfare assistance for at least one month in the previous year with otherwise comparable women who reported no such aid. Although I didn’t go into detail at Wonkblog, I accounted for NSDUH’s weighting and stratification in the usual ways, using STATA’s SVY routines.
The basic comparisons between welfare recipients and other low-income women are shown below:
To make a short story somewhat shorter, the 2011 NSDUH data proved uncannily similar to what we found before.
Twenty-two percent of welfare recipients reported some form of illicit substance use in the previous year. About half of these users reported only marijuana use. None of the group differences were statistically significant (p< 0.05). Yet point estimates of use and accompanying disorders are slightly higher in the welfare population than in the comparison group on just about every measure. For example, reported prevalence was slightly higher in the welfare population (22.4%) than in the comparison group of low-income single women with children who did not report TANF receipt (18.7%).
Only 3.6% of welfare recipients satisfied criteria for illicit drug abuse or dependence. About 8.6% of TANF recipients satisfied criteria for alcohol disorders—an issue left unaddressed by testing policies. However one runs the numbers, illicit drug use disorders are not common among welfare recipients. These problems are less common than they were twenty years ago. Moreover, other less morally charged physical and mental health problems such as depression are far more prevalent in the TANF population.
Quick ain’t fair: Why am I blogging about this?
This brings me to the second issue. You may be wondering the same thing just because you are bored. I’m thinking about something else. I could have gussied up the above chart, added 95% confidence intervals to the chart, fleshed out the background and data sections, and probably published this in the peer-review literature.
In many ways, that would have been a more substantial product. I could have written more carefully and expertly about NSDUH’s strengths and weaknesses, including those asterisked below. I could have examined more diverse measures and run some multivariate analyses. I could have situated this analysis more carefully in the existing literature. I would have had the benefit of expert reviewers applying additional eyeballs and perspectives to what I did. (I do sometimes run Wonkblog and TIE pieces past experts for informal peer-review, but that’s a different process.)
And–if I’m lucky–my deeper, more-vetted, less readable paper might hit print in 2014, when much of the public and political debate has played itself out.
I’m confident that publication would present the same basic message that I noted within one week at Wonkblog: Drug testing is a bad idea in 2013 for the same basic reasons it was a bad idea in 1993 and 2003. Eighteen months from now, public relations staff from the journal, a funder, or my university might well be emailing Ezra or Austin begging them to cover my important and timely piece.
If this were funded research, if I needed another useful but quotidian peer-review publication, I would have sent some version of this to an academic journal. I would also have done so if this were a more intricate or more methodologically rich analysis. You can’t blog a randomized trial or something requiring technical discussion of X’s and beta’s. That’s now most of my work reaches print. Maybe this NSDUH data allows a more substantial analysis I will run through peer-review.
But I don’t think one needs peer-review to make this simple descriptive point. And it is timely and important. I want people to know this now. Now if I only had written this piece better and more vividly, with more of a human face and a killer Powerpoint chart. There’s a craft to peer-review work. There’s a craft to journalism, too. I’m working my way up the learning curve on that one.
*This methodology is not terrific in reaching the most vulnerable and marginalized populations of great interest in substance abuse policy. I wish that NSDUH and virtually every other national public-use dataset routinely included state identifiers, and that NSDUH would include more biomarker work to verify self-reports. These are all matters are for another day.
@haroldpollack
