DevOpsDaysTLV24 - Spot Workload Optimization ML.pdf
0 likes12 views
The document discusses cost optimization techniques in Kubernetes, highlighting the difference between cost reduction and optimization. It outlines various strategies, such as cost visibility, use of spot instances, auto-scaling, and machine learning for prediction of resource usage and application behavior. The challenges associated with these strategies include performance trade-offs, potential downtimes, and the complexities of uncertainty in predictions.
DevOpsDaysTLV24 - Spot Workload Optimization ML.pdf
- 1. Cost Optimization in Kubernetes: Unlocking the Power of Machine Learning DISCLAIMER: The information presented here is for informational purposes only and should not be construed as a product commitment or roadmap statement. Alex Glikson Chief Architect, Spot by NetApp DevOps Days TLV, October 2024 © 2024 NetApp, Inc. All rights reserved.
- 2. © 2024 NetApp, Inc. All rights reserved. 2 • Cost Reduction vs Cost Optimization • Cost Optimization in Kubernetes: Techniques and Trade-Offs • Machine Learning: Uncertainty and The Power of Prediction Outline
- 3. © 2024 NetApp, Inc. All rights reserved. 3 Cost Reduction vs Cost Optimization
- 4. © 2024 NetApp, Inc. All rights reserved. 4 Technique Challenges & Trade-Offs: performance, availability 1. Cost visibility • Fine-grained resource & cost monitoring (per-presource, per-workload) 2. Use spot instances • Risk of application downtime due to node interruptions 3. Leverage RIs/SPs • Balance between Spot instances and RIs/SPs (incl. renewals etc) 4. Cluster auto-scaling • Adding/removing nodes may affect application performance/availability • Distinguish between environments with different “SLA” (e.g., dev/test) 5. Node right-sizing • Changes over time are likely to cause resource fragmentation 6. Node over-committment • Risk of throttling (and possibly application downtime) at peak demand 7. Application auto-scaling • Adding/removing Pods may affect application performance/availability • Risks due to uneven load balancing, non-CPU/memory bottlenecks 8. Application right-sizing • Changing resource ‘requests’ still requires a restart (as of Oct’24) • Throttling might be disruptive for the application (e.g., OOM-killed) 9. Network-aware placement • Balance between application availability and minimizing cross-AZ traffic Cost Optimization in Kubernetes
- 5. © 2024 NetApp, Inc. All rights reserved. 5 Prediction and Machine Learning • Spot markets behavior • Cost, Interruptions • Application behavior • Resource usage patterns, load patterns • Correlations, seasonality, changes, etc Many of the optimization challenges are hard because of uncertainty. Can we predict the future? "Prediction is very difficult, especially about the future.” -- Niels Bohr
- 6. Alex Glikson, Chief Architect, Spot by NetApp alex.glikson@netapp.com Booth G14 Thank You! © 2024 NetApp, Inc. All rights reserved. NETAPP CONFIDENTIAL