Metrics thatmatter agiledc2016
0 likes284 views
The document discusses the importance of metrics in measuring business, customer, operations, and development success, emphasizing the principles of effective measurement and the need for understandable metrics that drive behavior change. It outlines various key metrics and encourages organizations to monitor a balanced set of metrics over trends to inform decision-making and improve outcomes. Key takeaways include maintaining a small set of metrics, focusing on outcomes rather than individuals, and fostering interaction to encourage exploration of metrics.
Metrics thatmatter agiledc2016
- 1. Whatâs Measured Improves Raj, Rob, and friends Metrics That Matter !
- 2. Agenda ⢠The Why and What of Metrics? ⢠Principles of Measurement ⢠Key Drivers & Metrics ⢠Key Takeaways ⢠Q&A Finally, Pin Your Metric!
- 3. What does mean in todayâs Lean-Agile-DevOps world? 3
- 4. What is Measurement? Measures ⢠I have 5 apples ⢠# incidents Metrics ⢠I have 5 more apples than yesterday ⢠% of Sev1 incidents since feature rollout Observation that reduces uncertainty⌠4
- 6. 6
- 7. What Makes A Good Metric? http://www.flickr.com/photos/circasassy/7858155676/ Understandable Comparative Behavior changing 7
- 8. Measure movement towards your business goals & outcomes Where are you going? 8
- 9. KEY METRICS
- 10. 1. Business Success ⢠Starts with Measurable Goals ⢠Indicators for success ⢠Market share ⢠New business service enablement 10
- 11. 2. Customer/User Success ⢠Frequency of key transactions ⢠Amount of time spent in the application ⢠User satisfaction survey results ⢠A/B test results ⢠Customer ticket volume www.glasbergen.com 11
- 12. 3. Operations Success ⢠Uptime (availability) ⢠Performance (response time) ⢠Resource utilization ⢠Database query times ⢠Mean time to detection ⢠Support 12
- 13. 4. Development Success ⢠Lead time for changes (from development to deployment) ⢠Deployment frequency ⢠% Failed deployments ⢠Incident severity ⢠Outstanding defects
- 14. ⢠Average Revenue per User ⢠Customer Lifetime Value ⢠Daily/Monthly Active Users ⢠Average Session Time ⢠% Change in Customer Volume ⢠Feature Metrics ⢠Development Lead Time ⢠Idle Time ⢠Cycle Time ⢠Work in Progress- Technical Debt ⢠Rework ⢠Idle Time ⢠MTTD ⢠Defects ⢠Deployment Lead Time ⢠MTTR ⢠% of Failed Deployments ⢠Deployment frequency, duration ⢠MTTR ⢠Performance - Response Time ⢠Availability ⢠Security Pass Rate ⢠Customer Ticket Volume ⢠Net Promoter Score ⢠Net Value Score DeployBusiness Success QADevelopment Release Customer Success Operate ⢠Release Frequency ⢠Time/Cost per Release ⢠Predictability Accelerate Value Delivery Balance speed, risk, quality & costReduce time to obtain/respond to customer Now, All Together 14 Operations SuccessDevelopment Success
- 15. Monitoring Framework ⢠Aggregate metrics from all success contexts ⢠See the entire system ⢠Enables visualization, anomaly detection, trending, alerting⌠Source Adaptation: Turnbull, The Art of Monitoring, Kindle edition, chap. 2. Business Success Customer Success Development Success Operations Success 15 Event Aggregator
- 17. #ofStories 0 40 80 120 160 Sprints 78 (6/8-6/21) 80 (7/6-7/19) 82 (8/3-8/16) 84 (8/31-9/13) 86 (9/28-10/11) 88 (10/26-11/8) Scope (Total Stories Planned) Total Completed (Cumulative) Stories Completed Per Sprint Projected (Optimistic) Projected (Pessimistic) Projected (Median) 26 19 483 2019 1116 108106 100 81 77 6966 46 27 16 138138138138 134 128 121118 109 99 94 88 16 27 46 66 69 77 81 100 106 108 88 94 99 109 118 121 128 134 138 138 138 138 Empirical Data Drives Enquiry and Adjustment 17 BUSINESSSUCCESS Q: When are all planned stories estimated to be completed? A: End of Sprint 90 (12/06/16) Q: What is the total number of stories estimated to be completed by 11/08/2016? A: 119 Stories based on average velocity Q: Can we deliver all current planned stories by 10/16/2016? A: Not Likely, given the current trend Fixed Scope Fixed Date Fixed Scope & Date
- 18. Product 1 0 23 45 68 90 7/26/2020 8/2/20208/9/2020 8/16/2020 8/23/2020 8/30/2020 9/6/2020 9/13/2020 9/20/2020 9/27/2020 10/4/2020 10/11/2020 4 11 10 6 11 4 4 8 1 14 6 0 90 79 69 63 52 48 44 36 35 21 15 15 Pending Resolved Product 2 0 75 150 225 300 7/26/2020 8/2/20208/9/2020 8/16/2020 8/23/2020 8/30/2020 9/6/2020 9/13/2020 9/20/2020 9/27/2020 10/4/2020 10/11/2020 18 58 9 2 6 10 20 25 28 19 22 3 229 171 162 160 154 144 124 99 71 52 30 27 Pendi Resol CUSTOMERSATISFACTION âYou Build it, You Own Itâ Improving Responsiveness to Issues
- 19. Improving Deployment Success Rate, Not Just Frequency 19 OPERATIONSSUCCESS Build Daily Success Rate Date Build name 9/12/2016 9/13/2016 9/14/2016 9/15/2016 9/16/2016 Test Suite 1 100% 100% 100% 100% 100% Test Suite 2 100% 100% 100% 100% 100% Test Suite 3 100% 100% 100% 100% 100% Test Suite 4 100% 100% 100% 100% 100% Test Suite 5 100% 100% 100% 85% 100% Test Suite 6 100% 100% 94% 80% 100%
- 20. % Sprint Work Breakdown 0% 25% 50% 75% 100% Sprint83 Sprint84 Sprint85 Sprint86 18% 9%20% 5% 44% 36% 28% 33% 3% 3% 1% 3% 3% 4% 3% 26% 39% 41%52% 6%2%7%2% Other New Capability Major Interface Product Enhancements Production Incidents Technical Debt Time to Fix Arch. improvement Insight into Team Capacity and Work Items 20 DEVELOPMENTSUCCESSBalancing New Feature Work with Maintenance and Improveme
- 21. KEY TAKEAWAYS
- 22. ⢠Keep inventory of metrics small ⢠Measure outcomes, not individuals ⢠Monitor balanced set of metrics ⢠Monitor trends, not data points ⢠Share Widely â Interaction encourages exploration âIf you measure me in an illogical wayâŚdonât complain about illogical behaviorâ - Goldratt 22
- 23. Next Steps Clear Goal Well formed metrics Measure & Improve âMetrics illuminate, not indicateâ â George Dinwiddie
- 25. REFERENCES
- 26. ⢠http://onstartups.com/tabid/3339/bid/96738/Measuring-What- Matters-How-To-Pick-A-Good-Metric.aspx ⢠http://www.slideshare.net/GoAtlassian/understanding-metrics- what-to-measure-and-why-john-custy ⢠http://www.slideshare.net/TeamQualityPro/the-good-the-bad- and-the-metrics ⢠http://devopsenterprise.io/media/ DOES_forum_metrics_102015.pdf ⢠http://devops.com/2014/11/10/devops-scorecard/ ⢠http://www.datical.com/blog/9-metrics-devops-teams-tracking/ ⢠http://devops.com/2015/01/26/metrics-devops/ ⢠https://blog.appdynamics.com/devops/quantified-devops/ ⢠http://www.slideshare.net/jedi4ever/devops-metrics ⢠http://www.slideshare.net/ITRevolution/does15-troy-magennis- and-julia-wester-metrics-and-modeling-helping-teams-see-how- to-improve