Computing Workflows for Biologists: An Overview
- 2. â˘âŻ How many people here plan to analyze data with a computer in their work? â˘âŻ Are you working with other people on this analysis? â˘âŻ Do other people need to understand your analysis? â˘âŻ Do you need to remember and understand your analysis?
- 3. Elements of compu&ng â˘âŻ How data was generated (metadata) â˘âŻ Data â˘âŻ Data cleaning steps â˘âŻ Data analysis steps â˘âŻ Final plots and charts
- 4. Data! â˘âŻ Keep raw data raw â˘âŻ Use meaningful names â˘âŻ Organize your data so computers can read it
- 8. Organizing data in spreadsheets The cardinal rules of using spreadsheet programs for data: â˘âŻ Put all your variables in columns - the thing you're measuring, like 'weight' or 'temperature'. â˘âŻ Put each observa/on in its own row. â˘âŻ Don't combine mul/ple pieces of informa/on in one cell. Some&mes it just seems like one thing, but think if that's the only way you'll want to be able to use or sort that data. â˘âŻ Leave the raw data raw - don't mess with it! â˘âŻ Export the cleaned data to a text based format like CSV. This ensures that anyone can use the data, and is the format required by most data repositories.
- 10. FormaZng problems hTp://www.datacarpentry.org/spreadsheet- ecology-lesson/02-common-mistakes.html
- 11. A Roadmap for the Compu&ng Biologist â˘âŻ Consider the overarching goals of the analysis â˘âŻ Adopt an Itera&ve, Branching PaTern to Systema&cally Explore Op&ons â˘âŻ Reproducibility Checkpoints â˘âŻ Taking Notes for Computa&onal Analysis â˘âŻ Shared Responsibility: The Team Approach to Reproducibility and Data Management Shade and Teal, Compu&ng WorkďŹows for Biologists: A Roadmap hTp://journals.plos.org/plosbiology/ar&cle?id=10.1371/journal.pbio.1002303
- 13. Reproducibility Checkpoints Reproducibility checkpoints are places in a workďŹow devoted to scru&nizing its integrity -⯠the workďŹow (or step in the workďŹow) can be seamlessly used (it doesnât crash halfway or return error messages) -⯠the outcomes are consistent and validated across mul&ple, iden&cal itera&ons -⯠results should make biological sense
- 15. Taking Notes for Computa/onal Analysis â˘âŻ Take notes like you would for experimental work â˘âŻ Comment code â˘âŻ Use version control (Github/Gitlab)
- 16. What needs to go in notes: -⯠Soiware versions used -⯠Descrip&on of what the soiware is doing/goal of that step -⯠Brief notes on devia&ons from default op&ons -⯠WorkďŹows can include diďŹerent soiware (e.g., PANDAseq to QIIME to R), and should also include all âformaZng stepsâ needed to move between tools hopefully you donât need to manually format too much; avoid if possible
- 18. Shared responsibility â˘âŻ Shared storage and workspace can facilitate access to all group data â˘âŻ Using version control repositories can provide access to code and documenta&on (Github, Dropbox) â˘âŻ SeZng expecta&ons for âreproducibility checkpointsâ (team âhackathonsâ: open-computer group mee&ngs dedicated to analysis) â˘âŻ Paper reviews â˘âŻ Looking for help/support outside the lab (bioinforma&cs or user groups, oďŹce hours, StackOverďŹow)