The NY Times has a feature up today that compares predicted federal budget surplus (deficit) to what actually occurred (h/t Ezra Klein). Since I have a decade of experience developing and using mathematical simulations for policy evaluation I have a strong reaction to the type of analysis presented by the Times.
From it, many will be tempted to conclude that experts can’t predict the budget accurately so the whole budget prediction exercise is a useless waste of time at best and a political tactic at worst. That type of thinking misses a crucial point of budget predictions.
The point is not necessarily to predict the future accurately, though that would certainly be nice. The point is to use prediction tools to observe the expected changes to future budgets under different policies put into place today. That is, imagine you are a governor of a large state that is facing a budget deficit (know any?). Imagine two options are equally politically viable: (1) Increasing the sales tax by one percentage point and (2) cutting the state Medicaid program by ten percent. One important criterion in deciding between those two would be the difference in future expected budgets. Which one reduces your budget deficit more next year, the year after, in five years, and so forth?
Notice that this exercise has less to do with predicting a single future budget accurately than it does with predicting the difference in future budgets under two regimes. And what’s of great importance is the sign of the difference. Which one does more? As I said, none of this means that accurate absolute prediction isn’t valuable. It is. It’s just not the whole point. And there’s one other reason for that.
Do you think for a second nothing happens between the time of prediction and the time of comparison to actual? I hope not. Lots of things happen. And not all of them are in the model. Budget models typically don’t try to predict the outcomes of elections, the mood of the nation, whether a filibuster-proof majority exists in the Senate, what new legislative ideas will pass and when, terrorist events, and so on. All those things are exogenous to the model. Some of them occur in response to the prediction itself. (Surpluses can initiate a spending spree. Deficits can motivate belt tightening. Etc.)
So, don’t get worked up about inaccuracy of budget predictions. Don’t dismiss the exercise just because the result misses the mark a few years down the road. The best use of budget modeling is as a tool to policy planning. It can lead to better relative decisions even if it is inaccurate in an absolute sense. (The key, of course, is the extent to which the tool is systematically biased, favoring one policy type over another, say. That’s a different story.)
