The Fine Art of Combining Capacity Management with Machine Learning
2 likes736 views
The document discusses the integration of capacity management with machine learning, emphasizing the importance of using data to enhance IT investment and service performance. It outlines the objectives of capacity management, various machine learning approaches, and provides insights into analytical models and forecasting techniques. Additionally, it highlights the necessity of understanding and trusting data for effective machine learning applications in capacity management.
The Fine Art of Combining Capacity Management with Machine Learning
- 1. The Fine Art of Combining Capacity Management with Machine Learning Charles Johnson Senior Solution Consultant 1
- 2. Housekeeping Webcast Audio • Today’s webcast audio is streamed through your computer speakers. • If you need technical assistance with the web interface or audio, please reach out to us using the chat window. Questions Welcome • Submit your questions at any time during the presentation using the chat window. • We will answer them during our Q&A session following the presentation. Recording and slides • This webcast is being recorded. You will receive an email following the webcast with a link to download both the recording and the slides. 2
- 3. Agenda 1 Capacity Management Overview 2 Machine Learning Overview 3 Algorithms and Analytics 4 Use Case 5 Wrap Up
- 4. “ ” “As one Google Translate engineer put it, "when you go from 10,000 training examples to 10 billion training examples, it all starts to work. Data trumps everything.” Garry Kasparov, Deep Thinking: Where Machine Intelligence Ends and Human Creativity Begins
- 6. • Ensure the right level of ITI investment • Identify and resolve bottlenecks • Evaluate tuning strategies • Improve and report/publish performance • “Right-size” or “consolidate” • Ensure accurate and timely procurements • Ensure effective service level management • Plan for workload growth, new apps / sites • Avoid performance disasters Capacity Management objectives 6
- 7. Capacity Management Inputs and Outputs Inputs Outputs Sub-Process Business Capacity Management Service Capacity Management Resource Capacity Management Technology SLAs Business Plans Operations Budgets… Capacity Plan SLA guidelines Thresholds Charging Audits… 7
- 8. Ensuring adequate capacity • Now (no capacity-related Incidents) • In the future Performance Monitoring • Services • Hardware Resources Tuning • To provide best QOS “now” Forecasting resource demands and service levels • New Applications • Modelling Producing the Capacity Plan • To provide best QOS “in the future” Capacity Management Tasks 8
- 9. • Trends • What-If Trend • Analytical Modelling • Machine Learning • Simulation Modelling • Benchmarking 9 Capacity Management Techniques
- 10. Purpose of Capacity Management Understand your workloads and implement continuous system optimization equals “Stable IT Service” and “Cost Saving” Increase Bad Good Service （Response time） Workload Few ManyDecrease Resource （Cost） 10
- 12. • The ability for a system to take basic knowledge and apply that knowledge to new data • The ability to find unknowns in data • Main points • Learning • Pattern detection • Follow the data • Self-programming What is Machine Learning? 12
- 13. Machine Learning Approaches Supervised Learning • Established set of data • Data is classified • Find patterns in the data Unsupervised Learning • Massive amounts of data • Data is not classified or labeled • Find patterns in the data Other approaches • Reinforcement learning • Neural networks / Deep Learning 13
- 14. Descriptive Analytics • Current reality • Historical context • Aggregates data for insights Predictive Analytics • Anticipate changes by understanding patterns • Constantly needs new data • Looks into the future Prescriptive Analytics • New for machine Learning • Combination of business rules, machine learning and computational modelling 14 Forms of Data Analysis
- 15. Machine Learning Approaches Supervised Based on its color/shape/weight… Unsupervised How the different fruits can be classified inside your grocery store? 15 Is that “fruit” an apple? There is a bunch of different fruits Supervised vs Unsupervised
- 16. Machine Learning Uses Self-Driving Vehicles Automated Recommendations Computer Translation 16
- 17. The Machine Learning Cycle 17
- 20. • Linear Regression • Logistic Regression • Decision Tree • kNN (k-Nearest Neighbors) Common Machine Learning Algorithms 20
- 21. • Predict scores on one variable from the scores on a second variable • Study the relationship between real values based upon continuous variables • Create the best fitting straight line based on the data 21 Linear Regression
- 22. Should I play Golf? 22 Decision Tree Outlook Overcast Rain Sunny Low Humidity Wind High True False Yes Yes Yes No No
- 24. Trending • OK for utilizations, business volumes • Useless for service levels (response time) Analytical models • Quick and easy to set up • Potentially very accurate Simulation models • Time-consuming and difficult to set up • Potentially more accurate Benchmarks and Workload Generators • Perfect, but expensive and complicated (or impossible) • Required depending on industry or question to answer Forecasting Techniques 24
- 25. • Multi-class queuing network theory (QNM) • Modest time, effort & expertise to model a system • Attainable accuracy OK for business decisions • Few metrics, given automatic data collection • Few parameters for What-If changes • Quick scenario evaluations and sensitivity analyses Analytical Models 25
- 26. Join Capacity Management - Machine Learning
- 27. • Need data to make it work • More data the best (Big Data) • Need to understand and trust the data • Remove assumptions and bias • Reduce time to analyze data Join Capacity Management - Machine Learning 27
- 28. Capacity Management Focused Data Sources Data Sources • SMF Record 70 – 79 RMF • SMF Record 14, 15, 17 – Dataset Activity • SMF Record 30 - Job Detail • SMF Record 100, 101, 102 – DB2 • SMF Record 110 – CICS • SMF Record 115, 116 – WebSphere for MQ • Windows Perfmon • VMWare vCenter • UNIX /Linux mpstat, sar, iostat, vmstat • Cloud performance statistics 28
- 29. Use Case
- 30. • What is the problem you are attempting to solve? • What data is available? • Do you have a representative period of time? • What is the “Story” you are attempting to tell? Capacity Management Use Case 30
- 31. Analytical Model
- 32. Baseline LPAR Modeling – Hardware & I/O 32
- 33. Increase ProdOnl 5% LPAR Modeling – Hardware & I/O 33
- 34. Results CPU Change LPAR Modeling – Hardware & I/O 34
- 35. Machine Learning
- 36. Analyze Environment by Region 36
- 38. 38 Show Resource Consumption vs. Trend
- 39. Wrap Up
- 40. • M a c • Machine Learning provides value to Capacity Management • Reduce time spent analyzing data • Follow the data • Understand the data • Trust the data 40 Take Away
- 41. 41 Enhancing CM Process Humans + Computers > Humans Alone Computers Alone
- 42. • Machine Learning for dummies – IBM (Judith Hurwitz & Daniel Kirsch) • M. Asokan - Chief Architect, Distributed Systems & Big Data, Syncsort, Inc. • John Greenwood - Technical Architect, Syncsort, Inc. 42 References