Evolution of AWS infrastructure for ML: from Zero to Hero
0 likes237 views
The document outlines the evolution of AWS infrastructure for machine learning, detailing the journey from initial automation to containerization and cost reduction. It emphasizes the importance of focusing on business value rather than just technology stack and highlights the significance of defining clear steps and metrics in the process. Future improvements are also discussed, including the use of AWS services for real-time inference and enhanced monitoring.
![About me
{
“First name”: “Taras”,
“Last name”: “Slipets”,
“Occupation”: “Senior Software Engineer”,
“Company”: “Flixbus”,
“Interests”: [
“Software Engineering”,
“Music”,
“Skiing”
]
}
2](https://image.slidesharecdn.com/evolutionofawsinfrastructureforml-190425175722/85/Evolution-of-AWS-infrastructure-for-ML-from-Zero-to-Hero-2-320.jpg)
Evolution of AWS infrastructure for ML: from Zero to Hero
- 1. Evolution of AWS infrastructure for ML from Zero to Hero
- 2. About me { “First name”: “Taras”, “Last name”: “Slipets”, “Occupation”: “Senior Software Engineer”, “Company”: “Flixbus”, “Interests”: [ “Software Engineering”, “Music”, “Skiing” ] } 2
- 3. Disclaimer: my own personal experience 3
- 4. Agenda ● Problem statement ● Infrastructure start point ● Infrastructure evolution ○ First automations ○ Containerisation and cost reduction ○ Reactive approach ● Future improvements ● Summary and Q&A 4
- 5. Agenda ● Problem statement ● Infrastructure start point ● Infrastructure evolution ○ First automations ○ Containerisation and cost reduction ○ Reactive approach ● Future improvements ● Summary and Q&A 5
- 7. Heterogeneous data sources ● Purchase history ● Website activities ● etc... 7
- 8. Agenda ● Problem statement ● Infrastructure start point ● Infrastructure evolution ○ First automations ○ Containerisation and cost reduction ○ Reactive approach ● Future improvements ● Summary and Q&A 8
- 10. Speed up Data Science experiments 10
- 11. Git + SageMaker Notebook Instances 11 AWS SageMaker
- 12. Key takeaways ● K.I.S.S. ● Concentrate efforts on business value, not on technologies stack 12
- 13. Agenda ● Problem statement ● Infrastructure start point ● Infrastructure evolution ○ First automations ○ Containerisation and cost reduction ○ Reactive approach ● Future improvements ● Summary and Q&A 13
- 14. Split up pipeline into clear steps 14
- 15. Input data collection at scale 15
- 16. Features extraction at scale 16 Lambda Redshift Lambda S3Glue Athena
- 17. Model training and inference 17 AWS SageMaker S3S3 CRON + Bash
- 18. We used a VERY big machine 18 AWS SageMaker S3S3 CRON + Bash
- 19. Key takeaways ● Clearly define what you actually need - there might be service ready for use in cloud provider ● Cloud resources are cheaper than long-time manual infrastructure setup and optimisation 19
- 20. MVP in 1 month 20
- 21. Agenda ● Problem statement ● Infrastructure start point ● Infrastructure evolution ○ First automations ○ Containerisation and cost reduction ○ Reactive approach ● Future improvements ● Summary and Q&A 21
- 22. Containerization 22 ECR CI / CD Pipelines
- 24. Costs reduction 24 CRON ECS 24 / 7 On Demand 44%
- 25. Key takeaways ● Get actual practical numbers ● Actual bottlenecks might be completely different from your preliminar gut feelings ● Resolve low-hanging fruits first 25
- 26. Agenda ● Problem statement ● Infrastructure start point ● Infrastructure evolution ○ First automations ○ Containerisation and cost reduction ○ Reactive approach ● Future improvements ● Summary and Q&A 26
- 28. Aligned reactive execution 28 ECS Step Function
- 29. Monitoring and metrics 29 CloudWatch QuickSight
- 31. Key takeaways ● Think about entire e2e flow observability ● Start with simple, but business-relevant KPIs ● Separate business and technical metrics ● Offload knowledge of data engineers into code 31
- 32. Agenda ● Problem statement ● Infrastructure start point ● Infrastructure evolution ○ First automations ○ Containerisation and cost reduction ○ Reactive approach ● Future improvements ● Summary and Q&A 32
- 33. Future improvements 33 ● AWS SageMaker ecosystem usage for model training and inference ● Features extraction optimisation via AWS Athena and Glue ● Custom AWS CloudWatch metrics for pipeline execution monitoring ● Make model training and inference independent ● Enable real-time inference beside batch processing ● ...
- 34. Summary ● Start simple, bring business value ● Cloud resources are cheap and fast to setup ● Preliminary optimisation based on guessing is bull$&it ● Fully-fledged infrastructure automation should go after you have MVP, not before ● Have measurable and observable metrics 34
- 35. Q&A 35