Estimate and Measure. Minimize work, maximize value. Part 2
0 likes126 views
The document presents a comprehensive overview of various metrics relevant to measuring the efficiency and impact of work in project management, differentiating between output, outcome, and impact metrics. It emphasizes the importance of understanding and correctly using metrics such as cycle time, throughput, customer satisfaction scores, and conversion rates, while also providing links to resources for further exploration. Additionally, it highlights potential pitfalls in metric usage and encourages a focus on meaningful data to drive continuous improvement.
Estimate and Measure. Minimize work, maximize value. Part 2
- 1. Estimate and Measure Jurgen Appelo @jurgenappelo Minimize work, maximize value Thursday 3 September 2020, 9:00 CEST Part 2
- 2. At the end of this session, you will… Have an overview of popular output metrics Have an overview of popular outcome metrics Have an overview of popular impact metrics Get a link to an estimation exercise
- 3. Work In Progress, Queue Size Velocity, Throughput NPS, Fit for Purpose Conversion Rate, Revenue Work Output Outcome Impact Leading vs Lagging Metrics Warning! Targets for leading metrics may have unintended consequences for lagging metrics. Leading Lagging
- 4. Output Metrics Outcome Metrics Estimate and Measure Impact Metrics Wrapping up
- 5. Output Metrics Photo by Sasikan Ulevik on Unsplash
- 6. Lead Time The time between the appearance of a new work item and its departure. Lead time should be measured from the time of commitment.
- 7. Cycle Time The time between the work item starting and the work item finishing. Cycle time is a lagging indicator of work metrics such as queue size.
- 8. Cycle Time Scatter Plot https://modernkanban.com/how-to-create-a-cycle-time-scatter-plot-in-excel/
- 10. Throughput Run Chart https://kanbanzone.com/resources/kanban/measure-manage-flow-kanban-metrics/ The number of work items finished per unit of time. Always measure end-to-end and then resolve the bottlenecks!
- 11. Burn Chart Shows progress toward a completed to-do list. Burn up charts can more easily reflect a changed scope. Burn-down Burn-up
- 12. Cumulative Flow Diagram A burn up chart per workflow status. Visualizes queue size and wait time.
- 13. Velocity http://site.practiceagile.com/agile-metrics-velocity/ Number of work items (or story points) completed in a sprint. Easily gamed when using story points to “increase velocity”.
- 15. https://en.wikipedia.org/wiki/Confidence_interval Confidence Intervals Benefits Better expresses the uncertainty Forecasting based on min/max limits Drawback Assumes a normal distribution Still not based on real sample data “I feel 95% sure the effort is between 10 and 20 hours of work.”
- 16. Monte Carlo Method https://scrumage.com/blog/2015/09/agile-project-forecasting-the-monte-carlo-method/ Use empirical data points Create random samples Make a simulation Important: Crap in = crap out
- 17. Outcome Metrics Photo by Tengyart on Unsplash
- 18. Ratio of work items / releases rejected by the customer Prevents gaming of cycle time and throughput Encourages customer / team partnership Rejection Rate
- 19. Sean Ellis Test (Product/Market Fit Test) “Very disappointed” “Somewhat disappointed” “Not disappointed” Great if > 40% “Very disappointed” How would you feel if you could no longer use the product/service?
- 20. Net Promoter Score (NPS) How likely is it that you would recommend our company/product/service to a friend or colleague, on a scale of 0 to 10? 9-10: Promoters 7-8: Passives 0-6: Detractors NPS = % Promoters - % Detractors
- 21. Perfection Game What did you like specifically? What could we do better to get closer to a score of 10? How valuable was this on a scale of 1 to 10?
- 22. What was your purpose with this product? How fit for purpose was it, on a scale of 0 to 5? Fitness for Purpose What is the reason for the rating you gave? NFS = % Satisfied - % Dissatisfied 5 - Extremely 4 - Highly 2 - Partially 1 - Slightly 0 - Not at all 3 - Mostly Satisfied DissatisfiedNeutral
- 23. Customer Effort Score (CES) Do you agree with this statement? “Product X made it easy to do Y” 1 2 3 4 5 6 7 Strongly disagree Strongly agree
- 24. Impact Metrics Photo by Jordan McDonald on Unsplash
- 25. Sales funnels Marketing funnels Innovation funnels Recruitment funnels Amount of Entrance Awareness Acquisition Activation Amount of Exit Conversion Rate per stage Duration per stage Overall Conversion Rate Overall Duration Funnel Metrics
- 26. Tracking conversion rates per time cohort or customer segment Funnel stages or time periodsTimecohorts/customersegments Cohort Analysis
- 27. Must-have metrics for SaaS products Common Product Metrics Customer Acquisition Costs (CAC) CAC = Total cost of new customers / Total new customers acquired Churn Number of users that dropped out or canceled / Total number of users Retention Number of users that stayed or returned / Total number of users Customer Lifetime Value (CLV) The total average profit per customer over average customer lifespan.
- 28. Common Product Metrics Daily Active Users (DAU) Number of users actively using the service Monthly Active Users (MAU) Number of users actively using the service Monthly Recurring Revenue (MRR) Total income from customers per month Annual Recurring Revenue (ARR) Total income from customers per year Must-have metrics for SaaS products
- 29. North Star Metric (One Metric That Matters) Engagement Metrics Minutes of Player Activity per Month per User Total Order Size per Customer Transfers or Rides per Customer per Month The North Star Metric is the single metric that best captures the core value that your product delivers to customers.
- 30. Conversion Rates DurationAwareness Acquisition Activation Retention Referral Revenue CAC Pirate (AAARRR) Metrics CLV / MAU / MRR Churn / Retention Engagement Metrics Focus depending on context
- 31. Output Metrics Outcome Metrics Estimate and Measure Impact Metrics Wrapping up
- 36. Q & A