The Incidental Economist

Ed Leamer’s EconTalk conversation with Russ Roberts this week was among the more interesting episodes and well worth a listen. Much of it focused on issues raised in my summary of Angrist’s and Pischke’s response to Ed Leamer’s 1983 American Economic Review paper titled Let’s Take the Con Out of Econometrics. By way of quick review,

Leamer and others in the early 1980s were distressed by the lack of testing of implications of assumptions in specification and functional form of econometric models. His proposed solution was to analyze the changes in results based on model variations (sensitivity analysis). Angrist and Pischke make a strong case that Leamer was correct in his diagnosis but not necessarily in his prescription. They argue that the “credibility revolution” experienced in empirical microeconomics since Leamer’s critique is due principally to a greater focus on research design not on sensitivity analysis.

Angrist and Pischke argue that methodological innovations that exploit purposeful or natural randomness, including instrumental variables (IV) methods are responsible for taking the con out of econometrics.

That doesn’t mean sensitivity analysis doesn’t have a role. On EconTalk, Leamer made a good argument that it is still relevant, important, and rare. He notes that the model published in an economics paper is just one of the many the authors probably estimated. Did they report the only one that worked, or did many produce similar results? Is the result fragile or robust? These are important questions, and the reader cannot answer them with what is provided in a typical empirical economics paper. (See also Leamer’s written counterpoint to Angrist and Pischke and other responses found on MIT’s website.)

Nevertheless, exploiting randomness does help make econometric models more credible because (1) it removes much of the ambiguity in making causal inferences and (2) it reduces the number of needed control variables. In the case of a randomized design in which subjects are randomly assigned to treatment or control groups without any tilt to selection, no controls are required. The simple difference in means is the estimate of the average treatment effect (or the “intent to treat” effect). There are no other econometric specifications to explore to estimate this quantity. Thus, the result is trivially robust to specification.

Similarly, in IV estimation, one can omit variables without loss of validity, though with some loss of precision, with the exception of any required for the conditional independence of the instrument and potential outcomes (the set of “conditioning variables”, which could be empty). This fact automatically increases the range of robustness of the results if they are significant without any but the conditioning variables included as controls. Of course it relies on the validity of instruments, which is either asserted (with a good argument) and/or tested (where possible). This turns much of the robustness exercise into stipulation of assumptions, which is far more compact and easier to assess.

I believe Leamer agrees with these points because when asked on EconTalk about whether exploiting randomness and IV help address robustness, he did not deny that they did. Instead he turned to the issue of generality. He pointed out that the results of an IV study don’t necessarily generalize to other settings. That’s true. But it’s true of any study, even randomized trials. It isn’t necessarily related to the robustness issue. Leamer makes different points in his written response to Angrist and Pischke, focusing on limitations of asymptotic results. He writes that Angrist and Pischke “persuasively argue [that] either purposefully randomized experiments or accidentally randomized ‘natural’ experiments can be extremely helpful, but [they] … overstate the potential benefits of the approach.”

All of these points–Leamer’s and Angrist’s and Pischke’s–are valid and important. They all relate to the key underlying question: when does a fact reveal a larger truth? Empirical results, no matter how obtained, provide some facts about the data. Can one apply those facts more broadly? Where’s the boundary of validity in doing so? Nobody can answer such questions generally. One hopes (or one should hope) that the range of truth revealed by econometric facts is broad enough to be of use. If not, we’re all in trouble.

Any attempt to increase the range of validity of econometric results should be applauded. Any assertions that all econometrics is not to be trusted is overly broad. Some econometrics may still include some degree of “con,” but with correct application of modern technique a substantial amount can be driven out.

Megan McArdle has intersected with my information sources twice in the last few weeks, this time on the topic of debt. Those of you who read this blog for its personal finance content might be interested in McArdle’s work on this issue.

Her piece in the December 2009 issue of The Atlantic, to which I subscribe, is a close look at Dave Ramsey, his debt reduction and avoidance approach, and McArdle’s own implementation of it.

She was also interviewed on the 7 December 2009 episode of EconTalk, which is among my favorite podcast-delivered programs. The interview begins with a discussion of debt, Ramsey, his system, McArdle’s experience with it, and then turns to other areas in which we struggle with self-restraint (dieting, time-management, organization, and the like). If you’ve never listened to EconTalk and are interested in finance and economics, McArdle’s interview is a good introduction to the program. Give it a try.

This post originally appeared on The Finance Buff.

I’ve never met Russell Roberts, a George Mason University economics professor also affiliated with the Mercatus Center and Stanford University’s Hoover Institution. I know him only through EconTalk, a podcast available from the Library of Economics and Liberty. On each episode of the program, a review of which I recently posted, Roberts discusses a topic in economics with a different guest.

Roberts is also a novelist. His latest, The Price of Everything: A Parable of Possibility and Prosperity (Princeton University Press, 2008), is a story of how prosperity is created and sustained, and the unseen order and harmony that shape our daily lives. He is also the author of The Invisible Heart: An Economic Romance (MIT Press, 2001), a novel discussing an array of public policy issues including corporate responsibility, consumer safety, and welfare. His first novel The Choice: A Fable of Free Trade and Protection (Prentice Hall, 3rd ed., 2006) was named one of the top ten books of the year by Business Week and one of the best books of the year by the Financial Times.

As if that were not enough, Roberts co-blogs at Cafe Hayek, is a frequent commentator on National Public Radio programs, and has authored numerous academic publications, and written for The New York Times and the Wall Street Journal.

As a huge EconTalk fan, I was delighted when Roberts agreed to an interview. Out of respect for his time, I promised him only five questions. But I could easily have asked more. (Actually, I cheated and asked compound questions.) I think we cover a wide range of interesting topics.

The Incidental Economist (TIE): EconTalk began a little over three years ago. What is the story of its origin? What were your expectations and goals for it at the time? How have they changed? Where is it heading?

Russell Roberts (RR): Radio Economics, an economics podcast, invited me to be a host for one of their episodes. People told me it was a waste of time. Reading is more efficient than listening, they said. No one wants to listen to anything for more than two or three minutes, they said. But thousands of people downloaded that episode. I thought, what if there were an auditorium, an arena, really, where thousands of people converged to listen to economics. Wouldn’t I want to show up to talk to that class? So I decided to try it. At first I thought it would be a few times a year. But then I realized that listeners want a weekly dose of economics. So EconTalk became weekly.

We’re on our way to a football stadium of listeners at EconTalk who show up every week, curious to learn something. Knowing they’re out there, I want to show up every week to share a conversation with them. My main goal is unchanged—to help people see how economics is a useful way to think about the world. But the program has changed. I try to talk less now than I did in the early days. At least I think I do.

Where is it heading? I don’t know. I hope to make it better and better. There is still plenty I can do. The recent book club is one example. I’d like to get listeners more actively involved in the process. And I try very hard to get guests with viewpoints that are different from mine. I think those are some of the best episodes.

TIE: You explore economics in several different ways: by podcast, blog, in the classroom, and through your novels. How do these media differ in their capacity to communicate economic concepts? What works well one way but not another?

RR: People are different. Some people like to read. Others don’t. Some have to read. Some have time to listen. Some have great visual imaginations, others are more analytical. Some want a short insight. Some are willing to sit down and listen for an hour or read a whole book. I try to use as many different kinds of media as I can. I’m working on a video right now and hope to do more of that.

TIE: You seem to have made a commitment to spread economic thinking. Why is it important for more people to understand economics concepts? What is at risk without greater understanding? Is progress in this regard being made? How do we know?

RR: I think economics helps you understand the wonder of the world. Understanding the fundamentals of economics (tradeoffs, spontaneous order, incentives, trade) helps you be a fuller human being. It also helps to keep you from being fooled by self-interest cloaked in altruism. That makes you wiser and in theory, a better voter. I wish I knew how to measure progress in economic literacy. One very crude measure, and it is enough for me, is when a listener of EconTalk writes me about something she saw in the news and recognizes it as a “bootlegger and Baptist” story. That is very gratifying.

TIE: Economists seem to have prominent voices and roles these days, even in areas outside economics. Matt Yglesias raised this most recently in a post titled “Prestige Cross-Pollination,” and Ezra Klein called it “a special privilege of economists” in a post titled “The Tyranny of the Economists.” Do you think there is a tendency for economists to overstep the bounds of their expertise? To the extent they do, is it dangerous?

RR: What scares me is hubris backed by the appearance of science. We economists make that leap all the time. It is dangerous but I don’t expect it to stop. I do hope people can learn to be skeptical.

TIE: You’ve interviewed a lot of economists on EconTalk. How has the experience changed you and your view of the profession, its role, and future? Are there any broad lessons you’ve learned that you can share?

RR: Hosting EconTalk has been one of the great intellectual rides of my life, as exhilarating as graduate school. As listeners know, I’ve become much more skeptical of certain kinds of empirical work and much less confident of economics as an empirical science generally. I see it more as a way of thinking, a language, a philosophy for approaching the world. I talk about these issues in this podcast with Robin Hanson. You can also hear it come through in this podcast with Ed Leamer.

This post originally appeared on The Finance Buff and is cited by the 210th Carnival of Personal Finance.

My 75 minute one-way commute by foot and train affords me ample time to continue my education by podcast. My favorite podcast by far is EconTalk, a weekly one-hour interview program affiliated with the Library of Economics and Liberty. (All the podcasts to which I currently subscribe are listed in this spreadsheet.) In a typical episode the host Russell Roberts (George Mason University) interviews an academic economist. His interviewees have also included non-economists with expertise that relate to economics, though often in non-obvious ways. The program, and Roberts, is so good that I’d buy a lifetime subscription to EconTalk. But I don’t have to: it’s free.

I began listening to EconTalk about one year ago on the suggestion of a participant on the Bogleheads Forum. The first episode I heard was an interview with Mike Munger (Duke University) on middlemen. It happened to be a particularly interesting show and is still one of my favorites. Munger has been the most frequent guest and for good reason. He has a talent for making economic concepts accessible and relating them to the everyday. Limiting oneself to Munger’s EconTalk interviews alone would provide an outstanding overview of some of the fundamentals of economic thinking.

While Munger’s interviews may be among the best and most accessible, one can hardly go wrong with any EconTalk interview. Of the approximately 160 shows to date (the program began in March of 2006), I stopped listening to only about five before they ran to completion (a 97% rate of satisfaction). What makes the episodes so good are the guests, of course, but also Roberts. He fills his role well, routinely asking sharp (but not antagonizing) questions, letting guests speak at length, playing skeptic and devil’s advocate where appropriate, taking time to explain concepts, and posing clarifying queries to illuminate subtle points.

Perhaps Robert’s greatest strength is his refreshing awareness of and openness about his own bias. The bulk of one entire episode was devoted to his own exploration and evolution of his bias. He freely acknowledges the influence on his thinking of the Austrian and Chicago schools of economic thought. The work of Hayek has had a particularly strong impact on Roberts, and the Austrian economist is mentioned, if not quoted, in nearly every podcast.

To date, EconTalk has covered a wide range of topics in economics but also some surprising areas one doesn’t immediately associate with the “dismal science.” For example, there are episodes on piracy, ants, seasteading, political polling, baseball, football, ticket scalping, family, happiness, global warming, and many other topics.

Over the past year I’ve kept up with current EconTalk episodes and listened through the entire archived collection. As an experienced listener I can make two suggestions about how not to listen. Do not start with the first March 2006 episode and work forward. Roberts definitely improved with practice and didn’t hit his stride until early 2007. The episodes can be listened to in any order; later ones reference earlier ones but only in passing, not as prerequisites. So, I recommend listening to later shows first or at least some from 2007-2009 before those of 2006.

Second, unless you’re an Adam Smith fanatic you may want to wait on the EconTalk book club’s treatment of Smith’s Theory of Moral Sentiments. In these extra episodes, scheduled between regular EconTalk releases, Dan Klein (George Mason University) and Roberts discuss Smith’s lesser known work in great detail. The style of discussion is less thematic than other EconTalk episodes, which imposes a higher barrier to accessibility in my opinion.

While EconTalk is great, it can always be better. I have two requests. In this time of tremendous attention on health reform I would like to hear some new voices on the health insurance and health care markets. (There has not been a single episode on health care in 2009 and very few, if any, focused squarely on the health insurance market.) My other request is for an interview with an economist with expertise in Medicare, a topic that has not been explored (except possibly in passing) on EconTalk. (Since these are my areas of research I am more aware than others that they have not received much attention to date.)

There may be no such thing as a free lunch, but EconTalk is existence proof of excellent, free education. Over the past year I’ve walked many miles with Roberts and his guests as companions. In the years to come I am sure EconTalk will continue to improve, entertain, and enlighten. I’ll be listening.