Aaron did a nice job of describing the trials and frustrations of the grant application and award process. In brief, it can take years for a good idea to finally get funded. His salary depends on grant funding, and so does mine.
But the challenges don’t end with getting funding. You then have to do the work. That, of course, is the fun part. And, when the work is done, you’ve got to publish. Moving a paper from regression output to print can take years too. Just getting the first two or three reviewers to read your paper itself can take the better part of a year or more.
Meanwhile, what the world needs is timely, policy relevant research results. How on earth does one predict what the policy relevant questions will be several years ahead so one’s grant is funded and the paper is published just as the world needs your answer? Well, most people can’t do it most of the time. It’s too hard. The time scales are too long. And policy is too variable.
Fortunately, I work with Steve Pizer, who happens to be amazingly good at reading the policy landscape and seeing a few years ahead. Still, we get tripped up by the grant funding process or reviewers all the time. Sometimes our results are out after the debate has settled. Sometimes we’re too early because the debate shifted and then circles back (that’s OK). I’d have a lot more stuff coming out around now about health reform if only I’d received funding for it a couple of years ago. I tried. It’s hard!
One other challenge to doing policy-relevant research is that it isn’t enough to have a good question or to get funding, one has to have data. It’s a pretty fine needle to thread. If it were only sufficient to have data, or only enough to have a good question, it’d be far easier. But you have to have both and get funding and then get published and try to do it when the policy question is relevant.
I’m not complaining. But all this explains why we researchers defend our work pretty strongly. Once it reaches print, it’s gone through a lot of vetting. The idea and research plan has been worked over by a peer-reviewed merit process to secure funding. Then we’ve toiled for years trying to get the right answer and anticipate all the ways we could have deceived ourselves. Then the paper reviewers get to critique it. Sometimes the work is presented at conferences. We get a lot of feedback and pushback. When it finally gets published, we’ve heard most of the criticisms and dealt with them as best we can given the limitations of data and resources.
So yeah, we’re pretty proud of what we publish. And we think it is right. All that scrutiny by others actually serves a good purpose. It doesn’t guarantee that published results are correct. But it sure does increase the odds.
