dkNET Webinar "Texera: A Scalable Cloud Computing Platform for Sharing Data and Workflow-Based Analyses" 04/26/2024
0 likes35 views
Prof. Chen Li from UC Irvine discusses Texera, a scalable cloud computing platform designed for sharing data and facilitating workflow-based analyses in biological research. The platform aims to address collaboration challenges among biologists, bioinformaticians, and computer scientists while providing accessibility to advanced AI/ML technologies without requiring extensive coding skills. Key features include cloud services, version control, and scalability, alongside efforts to enhance user experience through a GUI interface and support for big data processing.
dkNET Webinar "Texera: A Scalable Cloud Computing Platform for Sharing Data and Workflow-Based Analyses" 04/26/2024
- 1. Prof. Chen Li Computer Science, UC Irvine April 26, 2024 Texera: A Scalable Cloud Computing Platform for Sharing Data and Workflow-Based Analyses 1
- 2. 2 Alice: Biologist (PI) Sally: Bioinformatician Bob: Bioinformatician Chen Li, UCI Example: sequence analysis in biology
- 3. Differential "gene" expression analysis AI/ML analysis Count Matrix Cellranger Quality Control Data Integration Clustering Cluster Annotation Trajectory analysis Sequence analysis pipeline FASTQ Sequences Dimensionality Reduction Step 1 Step 2 Step 4 Step 5 Step 6 Step 7 Normalization Step 3
- 4. - Coding is hard! - Version control of libraries - Needs servers - Slow on large data - Not every lab can afford a bioinformatician 4 Data preparation Coding challenges Data analytics Visualization Sally: Bioinformatician Chen Li, UCI
- 5. Collaboration challenges ● Collaborators of different backgrounds: ○ Biologists ○ Bioinformaticians ○ Computer scientists ● Collaborators from different organizations ○ Same lab: senior students vs new students ○ Other labs 5 Chen Li, UCI
- 6. Limitations ● Only file management, no run-time environment ● Inefficient! Collaboration: existing tools 6 Chen Li, UCI
- 7. ● How to utilize state-of-the-art AI/ML technologies? ● Require advanced coding skills ● Not easily available AI/ML opportunities 7 Chen Li, UCI
- 8. Cloud-computing services for sharing data and workflow-based analyses Benefits: - Cloud services (no installation, software patches) - Version control - Shared editing/execution - Sharing data and workflows - Parallel engine, scalable - … Our solution 8 Chen Li, UCI
- 9. Open source 9 Chen Li, UCI
- 10. Texera example workflow 10 Chen Li, UCI
- 11. Demo! 11 Alice: Biologist (PI) Sally: Bioinformatician Bob: Bioinformatician Chen Li, UCI
- 12. Figures on the entire dataset 12 Quality Control Elbow plot Clustered UMAP Annotated UMAP Chen Li, UCI
- 13. Texera Statistics 13 # of user accounts 332 # of projects 86 # of workflows 2,257 # of executions 31,000 # of workflow versions 357,000 # of publications 23 # of deployed servers 7 # of CPU cores in the largest deployment 400 # of files on GitHub 1,291 # of lines of code on GitHub 101,690 # of pull requests on GitHub 2,096 # of current PhD students 7 # of collaborating professors 17 # of involved undergraduates 80+ # of completed PhD theses 3 # of development years 7 Chen Li, UCI
- 14. Example: analyzing brain images, 256GB 14
- 15. Prof. Chen Li Teaching non-STEM students AI/ML using Texera 15
- 16. Prof. Chen Li High-school students using Texera 16
- 17. Mission: to serve the diabetes, endocrinology, and metabolic diseases research communities through the FAIR sharing of data and knowledge. New NIH award (dkNet) 17 Chen Li, UCI
- 18. Pilot project: inviting users 18 Chen Li, UCI
- 19. - Support a ChatGPT-like interface - Provide more operators and workflows related to sequencing - Make analysis parameters configurable - Parallelize bottleneck steps - Make more AI/ML techniques available - Migrate existing programs to workflows - Support public clouds (e.g., AWS, GCP) - … Open research problems 19 Chen Li, UCI
- 20. - Cloud-computing platform - GUI-based workflows (no coding needed) - Collaboration and sharing of data/analyses - Parallel computing: for big data - Supporting multiple languages: Python, R, Java, … - Supporting AI/ML (training, inference, …) Summary 20 Chen Li, UCI
- 21. Prof. Chen Li Computer Science, UC Irvine Texera: A Scalable Cloud Computing Platform for Sharing Data and Workflow-Based Analyses 21 Acknowledgements: Yicong Huang, Sally Lee, Xinyuan Lin, Xiaozhen Liu, Kun Woo (Chris) Park, Kevin Wu, and the Texera team