Azure DevOps AI
- 1. 1
- 2. 2 DevOps for AI Adrian Calinescu (a.k.a Mr. AI DevOps) adcaline@Microsoft.com Andrey Vykhodtsev (just tagging along) anvykhod@Microsoft.com
- 3. 4 Team Data Science process Andrey
- 4. 5 Typical Advanced Analytics project https://xkcd.com/1838/
- 5. 6 Problems with the “pile of scripts” approach • No traceability of work • No clear artifacts created • No governance • What data has been used? • What code? • What code + data resulted in my model? • What was the result for the model X? • No reproducibility • Not possible to hand over 6
- 6. 7 Team data science process https://aka.ms/TDSP
- 8. 9 What makes an ML model (asset examples) 2 3 4 5 5 6 7 8 9 10 11 1 1 2 Source Tests Feature gen / ETL 3 4 Cross validation code Dataset tests 5 6 Training code Inference code 7 Serialized model 8 9 Treatment group Control group 10 Evaluation code 11 Model efficiency tests E T L D ATA S E T M O D E L E VA L
- 9. 10 © Microsoft Corporation Reproducibility / Auditability MLOps Benefits Validation Automation / Observability • Code drives generation and deployments • Pipelines are reproducible and verifiable • All artifacts can be tagged and audited • SWE best practices for quality control • Offline comparisons of model quality • Minimize bias and enable explainability • Controlled rollout capabilities • Live comparison of predicted vs. expected performance • Results fed back to watch for drift and improve model
- 10. 12 Azure ML service Lets you easily implement this AI/ML Lifecycle Azure Machine Learning Workflow Steps
- 11. 13 Azure ML service Workspace Taxonomy
- 12. 15 Demo: a whirlwind tour of Azure ML Service
- 13. 16 “Best kept secret of AI is how it is de-abbreviated. Today we are going to reveal it.” A n d r e y Vy k h o d t s e v
- 14. 17 “AI = Adrian Is going to do it” A n d r e y Vy k h o d t s e v
- 15. 18 Azure Devops and AI (Handled by Adrian) 18
- 16. 19 Sprinkling DevOps on AI Handled by Adrian 19
- 17. 20 © Microsoft Corporation People. Process. Products. What is DevOps? DevOps is the union of people, process, and products to enable continuous delivery of value to your end users. “ ” Build & Test Continuous Delivery Deploy Operate Monitor & Learn Plan & Track Develop @DonovanBrown
- 18. 21 Azure DevOps Deliver value to your users faster using proven agile tools to plan, track, and discuss work across your teams. Build, test, and deploy with CI/CD that works with any language, platform, and cloud. Connect to GitHub or any other Git provider and deploy continuously. Get unlimited, cloud-hosted private Git repos and collaborate to build better code with pull requests and advanced file management. Test and ship with confidence using manual and exploratory testing tools. Create, host, and share packages with your team, and add artifacts to your CI/CD pipelines with a single click. Azure Boards Azure Repos Azure Pipelines Azure Test Plans Azure Artifacts https://azure.com/devops @DonovanBrown
- 19. 22 22
- 21. 24
- 22. 25
- 23. 26 BUILD PIPELINE TRAIN MODEL Compare MSE PROMOTE MODEL (REGISTER) CREATE SCORING IMAGE PUSH TO IMAGE REGISTRY CHECK DEPENDENCIES AGAINST KNOWN CVEs numpy, we’re looking at you. Post to MS Teams (Webhook)
- 24. 27 RELEASE PIPELINE QA CREATE AZURE CONTAINER INSTANCE FROM NEW IMAGE TEST WITH VALIDATION SET PROD MANUAL APPROVAL GATE if (new_image) UPDATE AKS DEPLOYMENT WITH NEW IMAGE TEST WITH VALIDATION SET Post to MS Teams (Webhook)
- 25. 28
- 26. 29 Thank You
- 27. 30
- 28. Data collection and understanding, modeling, and deployment Applications Dashboards Business/custom apps (structured) Logs, files, and media (unstructured) r Sensors and IoT (unstructured) Azure Data Lake store Azure Storage HDInsight Azure Databricks Azure ML ML server Model training Long-term storage Data processing Azure Data Lake Analytics Azure ML Studio SQL Server (in-database ML) Azure Databricks (Spark ML) Data Science VM Cosmos DB Serving storage SQL DB SQL DW Azure Analysis Services Cosmos DB Batch AI SQL DB Azure Data Factory Orchestration Azure Container Service Trained model hosting SQL Server (in-database ML) Advanced analytics pattern in Azure