The short answer is no. However, I haven’t been able to get this question out of my mind since attending a session on open notebook laboratory science at the ScienceOnline 2012 conference last week at N.C. State University. Is there something of use from open science techniques that could be brought to the health policy/health economics work that we do?
A bit on definitions. Open science is the practice of immediately making all results of the research enterprise publicly available, while open notebook is the practice of making a lesser amount of data publicly available, with other information withheld for a variety of reasons (volume, usability, legal restrictions, worries about being scooped, etc.). The idea is that immediate disclosure of results increases transparency and allows for the possibility of feedback along the way in a research project (Open Science Federation, @openscience are advocates of the general approach). It is unclear to me if true open science (immediate sharing of everything) actually exists, and I think this movement is really scientists with differing degrees of open notebook approaches (I may be wrong).
My initial reaction upon hearing an example at the conference (research seeking the boiling point of a metal while modifying the context and environment) was to say this doesn’t apply to what I do, since most of my work is based on human subject data that we collect, or on secondary data (like Medicare claims); in both cases, access to the data is expressly limited by legally binding data use agreements and IRB restrictions.
However, the fascinating session, another on peer review, and some extended discussion the next day over coffee with Anthony Salvagno, a graduate student in Physics and Astronomy at the University of New Mexico who works in an open notebook lab (here is his notebook) and Andy Maloney, a post-doc at the University of Texas at Austin who trained in an open notebook lab, has kept me stewing on ways in which the essence or spirit of open notebook science could be brought to good effect to the kind of research that I do, and that most readers of this blog would find of interest.
I am thinking of trying an experiment of sorts and taking a specific paper from a grant, and blogging (on a new blog, not here) a few times a week about the conduct of the research process for that paper. I will tell the IRB ahead of time, and ensure I don’t violate any restrictions. This will really be blogging about how we are thinking about answering a question. Here are the papers we have reviewed, have we missed some? Here is how we are dealing with missing value issues with the explanatory variables. These are the regression models being considered, anyone got a better idea, etc. Does anyone have experience with this type of approach in health policy research, or other work using human subjects data?
DT
