Reproducibility of computational research: methods to avoid madness (Session introduction)
2 likes820 views
The session focuses on the reproducibility of computational research, highlighting prevalent methodological, cultural, and funding issues that hinder reproducibility. Key topics include the need for standardized data formats and improved methods for data analysis and sharing. Despite the quantitative nature of computational biology, achieving reproducibility remains a significant challenge.
Reproducibility of computational research: methods to avoid madness (Session introduction)
- 1. Session introduction Reproducibility of computational research: methods to avoid madness Chair: Michael Hucka, Ph.D. Department of Computing and Mathematical Sciences California Institute of Technology Pasadena, CA, USA ICSB 2014, Melbourne, Australia, September 2014
- 2. So, what’s this about reproducibility?
- 4. “In biomedical science, at least one thing is apparently reproducible: a steady stream of studies that show the irreproducibility of many important experiments …” — Wadman. (2013). Nature, 500(7460).
- 6. “We find it utterly unexpected that, overall, it is only a minority of articles that properly describe (in a reproducible way) the computational research performed …” — Hübner, Sahle & Kummer. (2011). FEBS Journal 278(16).
- 7. What is the focus of this session?
- 8. Reproducibility issues Methodological issues Cultural issues Motivations Policies Incentives Funding … Methods Standards Algorithms Infrastructure … Facets of reproducibility
- 9. Reproducibility issues Methodological issues Cultural issues Motivations Policies Incentives Funding … Methods Standards Algorithms Infrastructure … Facets of reproducibility
- 10. Computational research should be easier to get right Have greater control of what is done, and how it’s done ⇒ Greater potential for making our work reproducible Assertion: The methodological issues are amenable to practical interventions Some examples: • Define and adopt standards for data formats, ontologies, protocols • Develop better methods for analysis, simulation, comparison • Develop effective resources for sharing & communicating research
- 11. “… reproducibility in computational biology is aspired to, but rarely achieved. This is unfortunate since the quantitative nature of the science makes reproducibility more obtainable than in cases where experiments are qualitative and hard to describe explicitly.” — Garijo et al. (2013), PLoS One 8(11)
- 12. currently “… reproducibility in computational biology is aspired to, but rarely achieved. This is unfortunate since the quantitative nature of the science makes reproducibility more obtainable than in cases where experiments are qualitative and hard to describe explicitly.” — Garijo et al. (2013), PLoS One 8(11)
- 13. What will be covered in this session?
- 14. Can’t cover all potential topics
- 15. Speaker Subject Relevance Hucka standard formats accurate communication of models Lovell data analysis appropriate inferences from data Kuperstein data curation & visualization software reconciling data from multiple sources Nahid workflow software recreating data analysis procedures