Freenome's Biological Machine Learning Platform
Download as PPTX, PDF0 likes865 views
The document discusses the challenges of applying machine learning to biological data, including noisy data, biases, and confounding factors. It argues that building a machine learning platform can help accelerate model development by allowing researchers to focus on their specialty rather than infrastructure, easily reproduce and build upon each other's work, and uniformly apply robust interpretation techniques. The platform would make common workflows like exploring new preprocessing methods, data types, validation schemes, or models a simple one-step process by leveraging existing shared components.
Freenome's Biological Machine Learning Platform
- 1. Easier, Better, Faster, Stronger: Building ML Platforms to Accelerate Model Development for Biological Data
- 2. © 2018 – Freenome Inc. | Proprietary & confidential.2 What’s in this talk? ● Challenges in applying machine learning (ML) to biological data ● What is a ML platform and why should I build one? ● Speedups for common research workflows
- 3. © 2018 – Freenome Inc. | Proprietary & confidential.3 Challenges with Applying ML to Biological Data
- 4. © 2018 – Freenome Inc. | Proprietary & confidential.4 Time spent doing ML on biological data ● Biological data is noisy and has tons of biases ● Interpretation: Are our results real? Will the model generalize? Diagram inspired by Karpathy’s slide in ScaledML 2018
- 5. © 2018 – Freenome Inc. | Proprietary & confidential.5 Confounding ● What is confounding? ○ Your model learns to predict based on signal that does not generalize to your use case ● This is especially prevalent in biological datasets! ○ Data may not represent the real world ○ Batch effects, lab process changes, sample sourcing, demographics ● Identifying confounding is difficult! Data Model Good cross validation (CV) performanceData Model From institution A, older than 75, and male → cancer, From institution B, younger than 60, and female → healthy, ... Good CV performance Bad real world performance Data Model Upregulation of genes A, B, C and downregulation of genes X, Y, Z → cancer Otherwise → healthy Good CV performance Good real world performance
- 6. © 2018 – Freenome Inc. | Proprietary & confidential.6 Sharing experiments leads to more efficiency In all cases ● Proposed techniques can be tested in a variety of use cases by many people when added to the platform ● Well-established techniques can be used as building blocks in future analysis ● A new technique is used only when we see reproducible increases in performance and/or decreases in confounding. When using bio data ● By default, suspect confounders in any good result ○ A result only stands if run against a suite of shared confounder analysis ● Focus on specific biological hypotheses or measurements which give insight into mechanisms ● Build visualization tools specifically for dataset statistics & low dimensional data representations vs confounders
- 7. © 2018 – Freenome Inc. | Proprietary & confidential.7 What is an ML platform and why should I build one?
- 8. © 2018 – Freenome Inc. | Proprietary & confidential.8 From the user’s perspective Experiment specification ML Platform Trained Model Classification Metrics Defines: ● Data ● Preprocessing/pretraining models ● Classification model ● Validation scheme Reports: ● Aggregate performance ● Performance on different subgroups of the data ● Classification confounding indicators Persists a model that can be reloaded and used on incoming data Visualization Creates plots of: ● Dataset statistics ● Low dimensional data embeddings by suspected confounders Scientists interpret results
- 9. © 2018 – Freenome Inc. | Proprietary & confidential.9 Let’s take a look under the hood Experiment specification ML Platform Visualizations Metrics Experiment runner Data fetcher Experiment saver Preprocessing library Visualizer Pretraining library Model Model library Fold creation Visualization library Metrics library Exploratory analysis/ visualization Prediction Key
- 10. © 2018 – Freenome Inc. | Proprietary & confidential.10 ML Platform makes experimental iteration faster ● Scientists can focus on their speciality instead of infrastructure ● Scientists can easily build and reproduce the findings of fellow scientists ● The most robust interpretation techniques uniformly applied to every finding
- 11. © 2018 – Freenome Inc. | Proprietary & confidential.11 Common workflows
- 12. © 2018 – Freenome Inc. | Proprietary & confidential.12 An ML platform makes idea exploration a 1-step process In the next series of slides, we’ll show that building upon the existing corpus of shared work makes new exploration faster.
- 13. © 2018 – Freenome Inc. | Proprietary & confidential.13 Perform exploratory analysis on a new preprocessing method Experiment specification ML Platform Visualizations Metrics Experiment runner Data fetcher Experiment saver Preprocessing library Visualizer Pretraining library Model Model library Fold creation Visualization library Metrics library Write one new function in the preprocessing library for testing data assumptions Reused Infrastructure
- 14. © 2018 – Freenome Inc. | Proprietary & confidential.14 Perform exploratory analysis on a new data type Experiment specification ML Platform Visualizations Metrics Experiment runner Data fetcher Experiment saver Preprocessing library Visualizer Pretraining library Model Model library Fold creation Visualization library Metrics library Add new data to data the cloud for testing different features
- 15. © 2018 – Freenome Inc. | Proprietary & confidential.15 Create a new validation scheme Experiment specification ML Platform Visualizations Metrics Experiment runner Data fetcher Experiment saver Preprocessing library Visualizer Pretraining library Model Model library Fold creation Visualization library Metrics library Write one new function for testing a new kind of validation
- 16. © 2018 – Freenome Inc. | Proprietary & confidential.16 Create a new model and test how it compares to other models Experiment specification ML Platform Visualizations Metrics Experiment runner Data fetcher Experiment saver Preprocessing library Visualizer Pretraining library Model Model library Fold creation Visualization library Metrics library Write a new model and add it to the model library for testing inference methods
- 17. © 2018 – Freenome Inc. | Proprietary & confidential.17 Takeaways ● ML in biotech needs robust infrastructure for model interpretation ● Building on top of the findings from fellow scientists decreases iteration time and improves results ● This benefit requires a commitment to a scientific platform