Thanks, Austin, Aaron, Don, and Kevin for letting me occasionally share your masthead. As Austin mentioned, I’ll be putting up the occasional weekend blog posts here. Although I have strong political and policy views, my TIE postings will have more of an academic bent. My research concerns public health services to disadvantaged populations. I’ll focus on substance abuse, violence prevention, and disability policy to help round out the great work that others do here. Sometimes I will write about a substantive policy issue. Other times, I will try to unpack a specific article in the research literature.
My long offering today concerns a recent article on a policy experiment. If you want to see how excellent empirical social science is done and why it’s important, devote an hour or so to the article by my SSA colleague Jens Ludwig and his co-authors that just appeared in the October 20 edition of the New England Journal of Medicine.
It’s one of many papers to emerge from the Movement to Opportunity (MTO) study, one of the most important urban policy experiments ever conducted. MTO is a multi-million experiment that been running for many years, with major funding from the Department of Housing and Urban Development, NIH, leading foundations, CDC, and others. (The Center for Health Administration Studies, which I help to run, kicked in a little money to help with this particular analysis. It’s been a good investment.)
To simplify things a bit, MTO families living in high-poverty majority-minority urban communities were randomly assigned across services designed to promote residential mobility. In five cities between 1994 and 1998, 1,312 families were issued traditional Section 8 housing vouchers. Another 1,788 families was offered Section 8 vouchers that were restricted for the first year to be used in neighborhoods with poverty rates below ten percent. This last group also received mobility counseling to help them make this potential move. Meanwhile, a control group of 1,398 families did not receive housing vouchers from MTO. Some members of the control group received housing aid from other sources or otherwise ended up in low-poverty communities.
MTO is one effort to untangle some important and messy questions that fall under the rubric of “neighborhood effects.” One might suspect that a neighborhood with a high concentration of employed college graduates would have good schools. These same employed and educated neighbors might have valuable job contacts. More affluent neighborhoods might have the resources and social cohesion to reduce crime, too.
Neighborhoods influence health in other ways, too. It’s easier to exercise in a safe neighborhood with nice parks and no broken glass on the sidewalks. It’s easier to eat right if local stores stock cheap and nutritious food. Good schools promote health in many ways, as well. So it stands to reason that moving a kid from a high-poverty neighborhood into a more economically stable one might make him more likely to graduate from high school or to get a decent job, and less likely to become obese or end up on a slab in the morgue.
Of course other stories can be told. Moving a 15-year-old from to a more prosperous neighborhood could prove harmful. That youth might face social exclusion or new academic difficulties in school. Middle-class neighbors might have valuable job contacts. Whether these neighbors will actually share these contacts is another matter. A low-income family that moves 10 miles into a low-poverty community might encounter barriers to health and social services and might receive less informal support from family and friends.
These latter possibilities suggest some of the difficulties in translating social science findings into useful policy. Identifying the diverse ways neighborhoods affect their residents might, or might not, provide useful insights to improve people’s lives. Sociologists have demonstrated that crime rates are lower in cohesive communities whose residents are skilled in addressing collective concerns. It’s less clear that we know how to reliably nurture such cohesion through politically and administratively feasible policies.
And of course neighborhoods’ true causal impacts are hard to tease out in the available data. Families are not randomly assigned across communities. Where people choose to live reflects many things about their attitudes, preferences, resources, and personal constraints.
Some of these variables can be captured in statistical models; many can’t be. Suppose, for example, that residents of Skokie, Illinois turn out to be more likely to graduate college than are other youth. Whatever variables one includes in a statistical model, this difference in educational outcomes probably reflects other, hard-to-measure characteristics of families that have chosen to locate there. Moving a randomly selected child to that community would surely have a smaller impact on educational outcomes than one would predict based on a naïve model that ignored these hard-to-measure factors.
These are precisely the kind of questions that provided the impetus for MTO. One intriguing predecessor study was performed right here in greater Chicago. The noted civil rights case called Gautreaux v. Chicago Housing Authority led to a group of low-income African-Americans being provided the opportunity to move into predominantly white, middle-class communities beginning in 1976. Years later, researchers observed that children who had made these moves were more likely to graduate high school and to have other favorable life outcomes than apparently similar peers in communities they had left behind.
MTO sought to rigorously explore whether such benefits were truly a result of moving families into low-poverty neighborhoods, or whether the apparent benefits were the result of some unmeasured characteristics of Gautreaux families that made them more likely to succeed independently of where they lived. Unfortunately the second explanation, selection bias, appears to account for many favorable findings in the original Gautreaux intervention.
Why the difference? Gautreaux attracted a pioneering group of Chicago single moms willing to move to middle-class suburbs. They were poor, but they weren’t typical of their peers. Indeed, they were selected and screened in ways that virtually ensured this would be the case. For example, Gautreaux staff tended to exclude families with problematic credit and tenancy histories. This made sense from the perspective of a program whose managers wanted to avoid problems with landlords or neighbors. This obviously complicates any analysis of the program’s true causal effects.
Sure enough, the observed benefits of residential mobility appeared more modest and varied within MTO’s rigorous experimental design than previous non-experimental studies had led many people to expect. Families offered the opportunity to move into low-poverty communities appeared no more likely to exit welfare or to achieve economic self-sufficiency. Girls displayed improved educational, psychological, and behavioral outcomes in low-poverty communities. Yet for some reason boys seemed to do worse.
MTO highlighted the many obstacles to residential mobility. Only about half of the families offered vouchers to move into low-poverty areas actually did so. Many families moved to higher-income neighborhoods only to subsequently move into higher-poverty neighborhoods. Given the incentives created by MTO’s different treatment arms, families followed a hugely nonrandom self-selection process in deciding whether and for how long to live in specific neighborhoods. As one of Austin’s previous posts describes, an intent-to-treat (ITT) framework is essential to understand the true causal and policy impact of an intervention such as MTO. Josh Angrist and Jorn-Steffen Pischke’s wonderful book, Mostly Harmless Econometrics, goes into these issues in greater depth.
MTO families who actually moved into low-poverty neighborhoods ended up, on average, in neighborhoods that were more cohesive and safer. These families were more likely to report close contact with college graduates, that they lived in a neighborhood where the police actually come when they are called. On average, though, these destination communities were quite racially segregated. MTO also did not appear to move kids to markedly higher-performing schools as measured by standardized test scores, by pupil-teacher ratios, or by statistical measures of “positive climate” in local schools.
Given these patterns, researchers such as Susan Clampet-Lundquist and Douglas Massey argue that MTO does not offer a valid test of the competing hypotheses concerning neighborhoods’ effects on families’ well-being. And of course an intervention to move (say) a teenager into a more prosperous and cohesive communities is not the same as creating the same beneficial conditions within the community that individual lived in from birth.
Clampet-Lundquist and Massey made some errors critiquing MTO’s specific study design. Yet their broader point is well-taken that a more powerful intervention might well yield different results. MTO operates within urban America’s very constrained social and economic landscape. I would certainly like to see the impact of a more powerful intervention that moves low-income families into genuinely integrated, middle-class communities with genuinely good schools. I presume these would produce more favorable effects. Judging by community opposition to MTO in particular cities, the intervention already stresses the limits of what is politically possible. (See, e.g., footnote 63 here.) I would also like to see the impact of powerful interventions that improve existing neighborhoods, as opposed to interventions that offer selected individuals a new place to live. Such neighborhood-improving interventions are few and far between.
Residential mobility is no cure-all for poverty and racial segregation. Yet such mobility can still provide important benefits. In MTO, women reported markedly lower levels of psychological distress, and reported that they were more likely to report feeling “calm and peaceful” in psychological surveys. Families moving to low-poverty communities felt markedly safer at night. They were less likely to report that any household members had been crime victims within the past six months.
Which brings me, at last, to last week’s New England Journal of Medicine article, “Neighborhoods, Obesity, and Diabetes — A Randomized Social Experiment,” by Ludwig, Sanbonmatsu, Gennetian, et al. These researchers re-interviewed MTO families between 2008 and 2010, more than a decade after families had been randomly assigned to the different arms of the MTO intervention. (To grasp this study’s intricacy and care, you might scan the figure here which describes the many steps from initial randomization to the final data used in the present study.)
Researchers assessed adult women’s body mass index (BMI), and women’s glycated hemoglobin (HbA1c) levels. As shown here, women offered opportunities to move to low-poverty neighborhoods were significantly less likely to be morbidly obese (13% less likely to display BMI>35, and 19% less likely to display BMI> 40). Most striking, women offered opportunities to move to low-poverty neighborhoods were about twenty percent less likely to display glycated hemoglobin levels above 6.5%, a standard threshold for a clinical diagnosis of diabetes. Effects were greatest among women who spent the most time in low-poverty communities. (Women offered traditional unrestricted housing vouchers displayed about the same reductions in morbid obesity, though these effects were not statistically significant. No reduction was observed in the HbA1c measure.)
What accounts for these effects? Researchers found no systematic differences in access to routine medical care across the different treatment arms. Women who received help moving into low-poverty communities reported greater personal safety, and reported reduced symptoms of stress and depression. These benefits may have facilitated greater opportunities to exercise or to consume nutritious foods. Low-poverty neighborhoods may create a social environment that facilitates more beneficial health behaviors.
Which of these explanations most mattered? It’s really hard to say. That kind of causal uncertainty is pretty common in empirical research. Many randomized trials convincingly demonstrate that a particular medical or policy intervention made a difference without showing exactly why that intervention worked. That’s frustrating. Yet it gives a terrific starting point for future work.
Over the years, MTO’s mixed findings have disappointed some researchers and policymakers who hoped to find stronger results. This New England Journal study should temper this sense of disappointment. We really can help low-income families by assisting their efforts to move into more prosperous and safer communities.