Why AIOps Matters For Kubernetes
This document discusses AIOps and its importance for operating Kubernetes at scale. It begins with an introduction of the speaker and then discusses some of the challenges of monitoring and managing infrastructure and applications as they grow in complexity. Specifically, it notes the explosion of metrics from containers and microservices that make problems harder to identify and isolate. It then introduces AIOps as an approach that can help with both reactive and proactive monitoring through techniques like correlation of metrics, what-if analysis, and optimization of resources. Examples are given of how AIOps has been applied at companies to improve performance and utilization through techniques like scheduling, placement, and controlled oversubscription of resources.
![Oversubscription
Google search + Google brain (deep neural nets)
>90% hardware utilization
No latency violations for search
[Lo et al’15]
>90% HW utilization No tail latency
problems](https://image.slidesharecdn.com/ithome2017aiopsmatters1-170926081133/85/Why-AIOps-Matters-For-Kubernetes-38-320.jpg)
Why AIOps Matters For Kubernetes
- 1. Timothy Chen Co-founder / CTO Why AIOps Matters for Kubernetes
- 2. About me 2 Apache PMC Mesos, Drill Container engineer lead at Mesosphere Help start community effort for Spark on Kubernetes (Google, Redhat, Palantir, Salesforce, etc). @tnachen
- 3. Agenda 3 Operating Kubernetes in production at scale What is AIOps? Operating Kubernetes in production at scale w/ AIOps!
- 4. Adoption
- 10. Container Production Stack Orchestration Security Network Storage Runtime OS Image
- 12. Application Monitoring Kafka Microservice #1 Spark Microservice #2 Microservice #3 Users: SRE DevOps Developers
- 14. Reactive monitoring new challenges Containers != VM Cannot focus on host metrics anymore with dynamic container changes Which metric(s) to track? 10000s of metrics, massive amounts of alert How to identify or isolate the problem (correlate metrics)? HW/Container/Host/Neighbor/Cluster/Dependency…. How to resolve these performance problems? (RCA) Unmanaged resources -> Manual tuning
- 15. Proactive challenges Goal: Maximize performance with minimum cost! Capacity Planning / What-If Analysis Cluster scheduling (interference, heterogenous, etc) Optimal configuration - Orchestration Network Storage Runtime OS Application VM
- 18. AIOps is not a new concept Decades of academic research around managing performance in computing…. - Cluster Scheduling - SLA-driven interference, storage, network, caching...
- 19. Now is the time - Workloads are more complex than before Infrastructure interfaces and data are becoming more standardized (Kubernetes, Prometheus, CTE, etc) All metrics centrally collected -
- 27. The Difficulty of using the Public Cloud Many instance types Reserved, on-demand, spot instances Many instant sizes (10s) Application churn Variability in load Can we guarantee SLOs & minimize cost automatically?
- 28. Choosing the right VM/HW
- 31. Live traffic load testing https://engineering.linkedin.com/blog/2017/02/redliner--how-linkedin-determines-the-capacity-limits-of-its-ser
- 35. Batch scheduling Improve deadline by 5x - 13x, increase utilization 14 - 28%
- 36. Placement
- 37. Variability in user load Bloated reservations due performance variability TwitterGoogle
- 38. Oversubscription Google search + Google brain (deep neural nets) >90% hardware utilization No latency violations for search [Lo et al’15] >90% HW utilization No tail latency problems
- 40. Hyperpilot 40 Help enterprises operate container in production. Using AI to learn how applications are behaving with your infrastructure and actively manage these resources. Processors, Memory, Flash, Disk, NIC Operating system Cluster resource manager Distributed analytics Hadoop, Spark, … IaaS Online, data-intensive (OLDI) Search, social nets, SaaS
- 42. Hiring Engineers in Silicon Valley / Taiwan! tim@hyperpilot.io Christos Kozyrakis (Stanford) Michael Huang (ex-CoreOS) Timothy Chen (ex-Mesosphere)