Well architected ML platforms for Enterprise Data Science
- 1. • PUBLIC 公開 Well Architected ML Platforms for Data Science Reliable Machine Learning Lifecycle
- 2. Goal Definition Business Understanding Data Understanding Data Preparation Modeling Evaluation Deployment Maintenance Outliers Model Regression Recall Feedback (Re-train) Battle tested CRISP-DM Model for ML Implementations
- 4. Platform Capabilities 1 Autoscaling Cloud-native ML/AI Platform with Identity Access Management and Data Classification 2 Data collection and Curation for Business Analytics Reporting and ML Model Preparation 3 Self-service Training and Experimentation for Forecasting and Simulation 4 Prediction performance and Feedback from drift of forecast Decisions Insights Data Lake IAM PaaS/DbaaS 5 ML as a Service Model Deployment
- 5. • PUBLIC 公開
- 6. • PUBLIC 公開 ML Ops Platform – Technical Architecture1
- 7. • PUBLIC 公開 ML Ops Platform – Technical Architecture2 Data Sources Customer Mart Sales Mart Sales Mart Customer Mart RISKS Mart Sales Mart Account Salesforce SAS Click Stream Speech Telematic s Customer Bus Ops Risk Profitability Click Stream Data Sourcing & Data Wrangling Data Scientists ML Engineers AWS Data and Model Exploration Spark Mlib JupyterHub Notebooks SparkMagic Livy Rest API EMR Spark EMR Presto Tensor flow H2O Sparkling Water Git Dvc ML Flow Model Registry (Versioning) Training Validation S3 S3 Data Collection Data Curation Model Repository Model Inference S3 Model Deployment ML Ops Deployment pipeline Jenkins Advanced Analytics Engine S3 Feature Engineering Api Release Predict Api EKS Jenkin s Click Stream
- 8. • PUBLIC 公開 Heterogenous data sources Interoperability and Integration Schema Management using Parquet Data discovery Catalog Data Lineage and Classification Data Governance and Privacy
- 9. • PUBLIC 公開 •System Appreciation and Discovery •Gap Analysis •Component Design •Team structure and RACI Plan •Milestones and Timelines •Infrastructure •Data Systems •Security •Governance Build •Socialize •Demo •Training •Benchmark •Pilot Evangelize •Transform •Dual Support •Bridge •Decommission Legacy Adopt •Release Management •Warranty •Operational Support •Design Lifecycle •Iterative Releases Run
- 10. • PUBLIC 公開
- 11. • PUBLIC 公開
- 12. • PUBLIC 公開 DOCKERIZED FLASK API EKS CONTAINER REGISTRY REDUCE API BUILD DEPENDENCIES ENABLE FARGATE SERVERLESS GATEWAY
- 13. • PUBLIC 公開 Streamlined Data collection Version controlled Feature Engineering Collaborative Discovery of features Distributed Training with Validation Reliable ML as a service Prediction performance Drift monitoring Model governance and fairness ML Ops Platform Goals
- 14. • PUBLIC 公開 1.Data Science Teams •Conceptualiz ation •Requirement s •Prototype •Design Review 1.ML Platform Team oJAD - Joint Application Design oDesign Approval (JIRA) oModel Development oCoding oTraining oCross Validation oAPI Development Team oAPI Requirements oAPI Security oAPI Catalog oAPI Integration oDevops Team oCICD for Data pipelines oCICD for Training oCICD for Model API oInfra Team oEKS oNetwork oMonitoring o*Governance Team oModel Governance oModel Fairness oModel Monitoring oData Science Teams oModel monitoring oModel feedback
- 15. • PUBLIC 公開 1.Model Conceptualization oBusiness need oMarket Research o Customer Feedback oBusiness value 1.Model Requirements oBusiness potential - Inputs from Model Conceptualization oData sources oCoordinating Customer and Technical requirements oPrototype 1.Model Prototype oData Collection oData Curation oFeature Engineering oTraining and Prediction 1.Model Design Review oJAD - Joint Application Design oDesign Approval 1.Model Development oOperational dependencies oIntegration dependencies oCoding oContinuous Training oFeature Engineering oTraining oCross validation oTracking
- 16. • PUBLIC 公開 1.Model DevOps oUnit tests oCode coverage oCI/CD 1.Model Quality Assurance oModel performance oSystem performance 1.Model Deployment oCloud native oContainerized oData Pipeline oML Service oMonitoring and Logging 1.Model Integration oService oData Feed oDashboard and Visualization 1.Model Management oModel Repository oModel Catalog oModel monitoring oSimulation 1.Model Governance and Fairness oAudit oExplanation oTrace back 1.Model Feedback oRecall and Bias oTraining Outliers oContinuous Training
- 17. • PUBLIC 公開 Monitoring Inputs and decision outputs Model fairness characteristics Overfitting, skew, bias Concept Drift Outliers Revised Training
- 18. • PUBLIC 公開