How to measure the outcome of agile transformation
20 likes5,090 views
The document outlines the strategies for measuring the outcomes of agile transformations, emphasizing the importance of clear goals, assessment models, and metrics to gauge success. Key elements include evaluating agile readiness, establishing a transformation roadmap, and understanding both qualitative and quantitative benefits of agile practices. The document also highlights the significance of continuous improvement and agile maturity in achieving effective outcomes in organizations embracing agile methodologies.
How to measure the outcome of agile transformation
- 1. How to measure the outcome of Agile transformation? Rahul Sudame CSP, SAFe SPC, PMI-ACP
- 2. Agenda Lets play some game! We need to have clear goals Transformation Roadmap Agile Readiness Assessment Model Current State Assessment Model ‘Agile’ Metrics Agile maturity Model Survey on ROI for Agile Case Study
- 4. Agile Innovation Games for Chapter Strategies Is Agile delivering value to you? Innovation Game: Teams to discuss if Agile is delivering value to you? Are you measuring the outcome of Agile Transformation in your organization? Is Agile implementation providing the desired impact?
- 5. Defining goal of going Agile
- 6. Moving to Agile is not easy! Everyone wants to go Agile! But Agile does not come for free, it needs: ◦ Breaking the bones (waterfall structure) of the organization ◦ Changing mindset ◦ Investment & Commitment So, how to ensure returns of this huge investment? 23 35 41 46 52 0 20 40 60 No Mgmt support Non-willingness Alignment Difficult to Transition No Clear Metrics Response % for challenges in achieving its goals with Scrum Ref: The 2015 State of Scrum Report - Scrum Alliance
- 7. We need to have clear goals Clear identification of Customer & Business benefits Envisioning the goal, based on type of the organization Identification of investment needs Clear identification of ROI parameters Business Case of Agile Transformation Business Case Customer Benefits Business Benefits Project Benefits Investment Needs ROI Parameters
- 8. Building Transformation Roadmap Agile Transformation roadmap in alignment with organizational environment Pilot to try Agile methodology and for the groundwork Inspect & Adapt at all the milestones Build the right environment iteratively Agile Transformation Roadmap
- 10. Agile Readiness Assessment Model Are we ready for Agile? Identification of Agile readiness using objective assessment Team’s interest / enthusiasm to be factored in Best suited teams to initiate pilot Gap analysis can highlight organization’s preparedness Agile Readiness Questionnaire Question Guidelines Points Team size • 5 – Team size of 7 to 9 3 – Team size less than 7 2 – Team with 10 to 15 members • 1 – More than 15 members 2 Team Composition • 5 - Completely cross functional 3 - Cross functional with some dependency on external teams 0 - Not Cross functional 2 Business Value Stream Coverage • 5 - Team covers the entire business value stream 1- Less than 50% of the value stream 0 - Less than 10% of the business value stream 1
- 11. Focus on values, not just practices Coach the team to implement Agile ‘principles’ and not just the ‘practices’ Build an environment of continuous improvement Don’t get trapped in ‘Trough of disillusionment’ Deliver value periodically Sprint Cycle Define Roles Backlog Grooming Agile Estimation Release Planning Sprint Rituals Activities in Sprint Cycles
- 12. Agile Current State Assessment Conduct periodic assessment of team’s Agile practices with involvement of the entire team Assess the effectiveness of different aspects such as Release Planning, Sprint Planning, Product Ownership Identify the Gaps and define a plan of bridging those gaps Current State Assessment Model
- 13. ‘Agile’ Metrics Identify the metrics that provide ‘real’ status of progress Define a mechanism to aggregate required data periodically Do NOT enforce waterfall metrics on Agile teams Don’t get trapped in ‘Perfect’ burn down chart • Earned Business Value / ROI • Release Cycle • Cost of Delay (COD) • Lead / Cycle Time Business Metrics • Velocity / Burn down • Backlog Size • User Stories Done per Iteration • User Stories carried forward to next iteration Process Metrics • Defects count per story / Defect density • Acceptance tests per story • Code Quality & code coverage metrics • Test Automation metrics Quality Metrics
- 14. Adaptability meets Predictability Consistent implementation of Agile and usage of tools can provide release predictability Dynamic data extrapolation can help in analyzing impact of scope changes Real-time visibility to all the stakeholders Release Predictability Chart
- 15. Agile Maturity Model Maturity level of a team can be assessed based on different parameters Organizational benchmarks can be provided as guidelines Agile Maturity Model itself should be lean & aligned with Agile philosophy Level 1 Level 2 Level 3 Level 4 Level 5 Process Group Rating Applicability Score Maturity Index Release Mgmt Effectiveness 6 7 86% 1.79 Sprint Mgmt Effectiveness 7.5 9 83% Product Backlog Mgmt 7 7 100% Frequent Product Delivery 7 8 88% 0.88Quality Assurance Effectiveness 5.5 6 92% Retro & Improvements 1 3 33% 0.33 Customer focus 5 5 100% 1.00 Maturity Index 4.12 Agile Maturity Model
- 16. Reduce Lead Time Optimizing Lead time & Cycle Time would help in reducing ‘waste’ in the system ‘Cost of Delay’ identification would help in delivery of higher value items sooner Focus on business value would help in reducing delays at all levels Deployment 1 month delay for deployment QA 3 weeks delay for signoff Development 2 weeks delay for integration Design 8 days delay for signoff Requirements 2 weeks delay for clarifications Delays at various phases
- 17. Best Agile Team Award Healthy competition amongst the teams can be encouraged Effectiveness of teams on various parameters related with Agile implementations can be considered Team’s approach, Scrum Master / PO’s effectiveness can also be factored in Team Retro Effectiveness Scrum Master Effectiveness Points Team 1 12 24 73 Team 2 18 27 65 Team 3 14 11 62 Team 4 8 14 62 Team 5 10 13 61 Team 6 9 11 46 Assessment for Best Team Award
- 18. Are we making progress? Agile Project Progress can be reviewed periodically Quantitative as well as Qualitative measures to be presented Organization-wide changes across different value- streams to be reviewed Project Progress Report Quantitative Benefits • Delivery cycle reduction • Revenue realization ($$$) Qualitative Benefits • Value delivered to customer • Team efficiency • Customer/Team satisfaction Benefits Not Realized • Value delivered to customer Project Progress Status Repot
- 19. Survey on Outcome of Agile
- 20. Survey on Outcome of Agile Independent Survey Findings – Q2 2015 Highlights parameters used for different people to measure outcome of Agile In Progress: ◦ Details on survey participants ◦ Details on organization, industries, demographics represented ◦ Survey methodology
- 21. Survey on Outcome of Agile Independent Survey Findings – Q2 2015 Highlights parameters used for different people to measure outcome of Agile
- 22. Survey Findings Delivery cycle time - 22% Delivering business value - 19% Working Software / Customer Satisfaction – 14% Not measuring ROI – 11% Reduction in defects, waste & risks – 8% Independent Survey Findings – Q2 2015 Highlights parameters used for different people to measure outcome of Agile
- 23. Survey Findings: Parameters Parameters for measuring outcome of Agile: ◦ Delivery Cycle Time / WIP Reduction ◦ Achievement of business goals/value ◦ Working Software / Customer Satisfaction ◦ Reduction in defects, waste & risks ◦ Improvement in Business/Engineering processes ◦ On time delivery ◦ Delivery within the budget ◦ Break-even point - bang for your buck ◦ Team satisfaction Independent Survey Findings – Q2 2015 Highlights parameters used for different people to measure outcome of Agile
- 24. Survey Findings: Feedback Independent Survey Findings – Q2 2015 Highlights parameters used for different people to measure outcome of Start from what you expect... what value did you expect the transformation to bring? If I could choose just one measure I'd choose Cycle Time - time (in days) Lean up front, big picture planning, determine business goals, and, especially, how those goals interrelate. Its not worth to measure the ROI of Agile. Reducing WiP can have up to a 4x impact on Quality, half your Time- to-Market, and improve your Predictability
- 25. Case Study
- 26. Case Study Product development organization applied Agile enterprise-wide for product development cycles Outcome: ◦ Release Cycle reduced from 1.5 years to 3 months ◦ First release was not appreciated by customers due to low user experience – Agile helped in Failing Faster! ◦ Manual regression testing efforts (for sanity suite): 5 days ◦ Automated regression testing efforts (for sanity suite): 4 hours 90% QA effort reduction ◦ Conversion of a short-term customer contract into a long-term engagement ◦ Time taken for matured environment - 1 year
- 27. Summary Agile Transformation is a journey and involves continuous improvement For effectively presenting ROI, build a baseline and set of parameters/metrics for before and after Agile state ◦ E.g. Pre & Post Agile Delivery Cycles / Durations ◦ Achievement of business / customer goals Identify and present both quantitative and qualitative benefits to the stakeholders periodically ◦ E.g. Customer Satisfaction ◦ Reduction in production hot fixes, change requests Retrospect Agile Maturity at all levels of product development and focus on building effectiveness iteratively E.g. Target: Agile maturity Level 3 or Project Maturity Index > 3.5
- 28. Thank You ! - Rahul Sudame rahulsudame@yahoo.com