Very, very rarely in science do we achieve a result that we are absolutely, positively sure is correct. We almost always use statistics to give us some estimate of how likely we believe our results to be true, but that answer rarely equals 100%.
But we’d like to think that most of what we find, write up, and publish is correct. One way to define “correct” is as something that someone else can reproduce.
What do I mean by that? I mean that if someone else does the same experiment as me, in another time, in another place, they get the same result. I mean that – over time – people are able to get the same results I do when they perform similar experiments.
By this metric, many areas of science are falling far short of what we’d like.
Many are working on solutions to these issues. Journals are beginning to band together and discuss how to do better reviews. Many trials now need to be registered so that decisions have to be made about how to design, conduct, analyze, and report findings before the research takes place, while researchers are still behind the veil of ignorance.
The NIH has also gotten involved by calling for the creation of training modules to enhance data reproducibility. They are focusing their efforts on four domains:
- Experimental design
- Laboratory practices
- Analysis and reporting
- And the Culture of science
A few years ago, they put out a Request for Applications in this area, and we at Healthcare Triage were funded! We have tackled two of these domains, Experimental Design and Analysis and Reporting. We like to think we’ve got something to say about what makes a good study, well, good. We explored all the key concepts you need to consider in order to ensure that your research is as bias-free as possible.
And, we think we’re pretty good at analysis and reporting as well. Therefore, we talked about what makes a good paper, how to present and discuss your results, and how to avoid the mistakes many make in overselling their findings.
Both of these modules consist of many episodes, or chapters. They’re all short and sweet, they’re all freely available, and they’re open to anyone. If you’re interested in learning about CME for watching these videos, go here for Experimental Design and here for Analysis and Reporting.
We also would appreciate feedback. There’s a short survey you can fill out after you watch the series.
It’s hoped that these modules will help scientists at all levels improve the quality of their experiments and how they report them, and that by doing so, we might improve the problems we’re currently seeing in many areas of research in terms of reproducibility. Please share these videos widely!!!
This training module is part of a series funded by the National Institutes of Health under grant R25GM116146
P.S. Let me add that none of this would ever be possible without Stan Muller and Mark Olsen, my co-pilots at Healthcare Triage, John Green, who executive produces, and Austin Frakt and Jen Buddenbaum, who consulted on all of the scripts!
