There’s a tension in academia between work that has and has not been peer-reviewed. Readers of the blog will know that – in general – I heavily favor peer-reviewed work. I’m not alone in this; most people in research feel the same way.
Because of that, I’m continually faced with questions about the value of what I do here. Many of my colleagues wonder why I don’t write things and send them in for publication in medical journals instead of writing them for a blog.
I’m sensitive to these arguments, but I think they are missing the point.
First of all, let’s own the fact that peer review isn’t perfect. As a friend of mine said repeatedly this last weekend, “look how well peer review worked for Andrew Wakefield.” We can’t pretend that just because it’s had the “stamp” of peer review, that it’s “truth”.
Unfortunately, that’s the perception of many in the general public, though. If it’s peer-reviewed, then it’s “correct”. That’s not the value of the process, however. It’s that in order to be peer reviewed, the methods must have been examined closely and made public. That’s the value of peer-review. If I know a study is peer-reviewed, then I can go look it up, read it for myself, and judge it. I know enough is there for me to make up my own mind as to the validity of the results.
Thus, when one of you sends me a study and tells me it says one thing or another, I can not only tell you that I agree or diagree with it, but I can tell you why.
Using the example above, I could go read the original Andrew Wakefield study in Lancet, come back, and tell you that it didn’t prove that the MMR vaccine caused autism. His “study” was a case series of children with neurological disorders where a number of parents thought that maybe the MMR vaccine was associated with the onset of the disease (assuming that what was written was not a massive fraud).
Peer review doesn’t guarantee quality. It makes it more likely, and additionally guarantees that the means for you to assess quality for yourself are available.
Moreover, I just don’t feel like I could do as much good with this work if I tried to go through medical journals. I could send in a piece, they could take months to review it, and then after having it watered down to please everyone, it might see the light of day in 6 months, long after anyone would care. Traditional peer-review just doesn’t work in this domain.
The down side of this is that although I’ve written more on this blog than the sum total of all my other writings, what I say here doesn’t “count” as part of my real job. The truth is that – to my employers and to many of my colleagues – the least of my manuscripts is more significant than this blog. I’m not saying this out of spite or with any negative emotions at all; it just happens to be true.
I (and I’m sure Austin and Don) would disagree. I think this has tremendous value, not only I hope for all of you, but for me as well. Moreover, I’d argue that my writing is peer-reviewed every day. Other wonks read it, comment on it, and judge it. Health policy experts and journalists pick it up or don’t. And readers come back or go elsewhere. It’s peer-reviewed – just by a much broader group of peers.
