Random Musings on Fixing Data Shambles in Science
2 likes460 views
This document discusses principles and guidelines for reproducible and transparent research. It highlights the importance of rigorous statistical analysis, transparency in reporting standards used, replicates, statistics, randomizations, and inclusion/exclusion criteria. It provides examples of reporting standards like MIAME, which defines minimum information for microarray experiments, and the Gene Ontology, which provides consistent descriptions of genes. Data sharing is also discussed, noting it requires standards, data volume, and bandwidth. While useful standards and best practices exist, support from funders and journals could help further improve reproducibility.
Random Musings on Fixing Data Shambles in Science
- 1. "A Replicable Guide to Reproducible Research" Iddo Friedberg Iowa State University
- 2. “Random Musings on Fixing Data Shambles in Science"
- 5. NIH: Principles and Guidelines for Reporting Research Rigorous statistical analysis Transparency in reporting Data & material sharing Best practices
- 6. Interpretable statistical analysis Effect size and significance “EFFECT SIZE vs. SIGNIFICANCE: smoking increases your relative risk of lung cancer by 2,500 percent; eating two slices of bacon a day increases your relative risk for colorectal cancer by 18 percent. Given the frequency of colorectal cancer, that means your risk of getting colorectal cancer over your life goes from about 5 percent to 6 percent”
- 7. Interpretable statistical analysis Multiple hypothesis testing “Either we have stumbled onto a rather amazing discovery in terms of post-mortem ichthyological cognition, or there is something a bit off with regard to our uncorrected statistical approach.” Craig M. Bennett, et al (2010). Neural Correlates of Interspecies Perspective Taking in the Post-Mortem Atlantic Salmon: An Argument For Proper Multiple Comparisons Correction Journal of Serendipitous and Unexpected Results, 1 (1), 1-5
- 8. Transparency in reporting Standards Replicates Statistics Randomizations Blinding Inclusion & exclusion criteria
- 9. Transparency in reporting Standards Replicates Statistics Randomizations Blinding Inclusion & exclusion criteria
- 10. MIAME (2001) Minimum Information About a Microarray Experiment Experimental conditions Controlled vocabularies for organisms, gene functions
- 11. MIAME (2001) A journal enforced reporting standard enables: Reproducibility Compare data Compare method effectiveness Refutability
- 12. Some more standards Standard Purpose Gene Ontology Protein function Human Phenotype Ontology Human disease MIxS Genomic and Metagenomic data
- 13. The Gene Ontology Pingzhao Hu, Gary Bader, Dennis A. Wigle & Andrew Emili Nature Reviews Cancer 7, 23-34 (January 2007)
- 14. The Gene Ontology Consistent descriptions of genes Mostly Human, mouse, fly & yeast Functional aspects: Molecular function Biological process Cellular component
- 15. Data Sharing Requires standards Requires Volume Requires Bandwidth
- 16. Shape of Things Useful standards & best practices exist in many fields Funders beginning to understand needs for standards / best practices Support for cyberinfrastructure Education / awareness Journal-mandated compliance Funder based compliance(?) Good Could be better
- 17. ISMB/ECCB 2015, July 10-14