Metrics that Matter
0 likes127 views
The document emphasizes the importance of focusing on outcome-oriented metrics rather than vanity metrics, which can be misleading and lack context. It outlines the 'Pirate Metric Framework' for evaluating customer engagement across acquisition, activation, retention, referral, and revenue. Key takeaways include measuring meaningful outcomes, verifying data accuracy, selecting appropriate visualization methods, and promoting a culture of continuous learning and experimentation.
Metrics that Matter
- 1. Metrics that matter Forget the vanity metrics and measure key outcomes
- 2. Joni Hoadley @JoniHoadley www.jonihoadley.com • 20+ years product management experience • Coach and consultant for companies looking to bring customer-focused products to market more quickly • 12 years as product management leader at Sonos
- 5. What are vanity metrics? Vanity Metrics definition - A standard for measuring something without context, vanity metrics are easily manipulated, and do not lead to actions you can take to grow your business. Examples - app downloads, page views, time on site, twitter followers, facebook likes, etc.
- 6. How to identify vanity metrics Source - https://www.crazyegg.com/blog/mirror-mirror/
- 7. Yes, this entire page is useless Source - https://www.crazyegg.com/blog/mirror-mirror/
- 10. The ugly truth about presenting data
- 11. Where to learn more • Find out which type of chart works best at From Data to Viz • Select which type of data you have (chronological, numeric, categorical) and this site will guide you through the best options.
- 15. Outcome-oriented Metrics Vanity Metrics Outcome-oriented Metrics • Raw numbers such as page views without context • Don’t represent customer value • Don’t provide guidance for future direction • Link actions to results • Focus on delivering customer value • Provide insight into the health of your product and company vs.
- 18. WOW
- 19. Measure what matters Quality, not quantity - Zappos measures call time instead of call volume. They also invented their Happiness Experience Form which is a 100-point scale based on how well reps fulfilled on various goals. http://callcriteria.com/kpi-lessons-masters-customer- service-zappos-case-study/
- 20. Real world example from Sonos
- 21. Pirate Metric Framework A A R R R Acquisition - how well are you getting customers to your site or app? Activation - are your customers having a great ‘first run’ experience? Retention - how often are your customers coming back? Referral - are they telling others about your product? Revenue - are they paying for your service?
- 22. Pirate Metric Framework How many people are signing up for a free trial? What % of people are watching a video from start to finish? What % of people are returning to watch a video again in 30 days? What % of people are sending links of videos to others? What % of people are paying for monthly subscription? A A R R R Acquisition - how well are you getting customers to your site or app? Activation - are your customers having a great ‘first run’ experience? Retention - how often are your customers coming back? Referral - are they telling others about your product? Revenue - are they paying for your service?
- 23. 4 steps to measuring for outcomes 1. Identify the key outcomes. 2. Form a hypothesis. 3. Build your experiment. 4. Measure and analyze
- 24. Experimentation: Building a culture of continuous learning Source: Eric Ries, The Lean Startup Methodology
- 25. Measuring Net Promoter Score Would you recommend <my product> to a friend? https://www.netpromoter.com/know/
- 26. The Problem with NPS Would you recommend <my product> to a friend? https://www.netpromoter.com/know/
- 27. How to derive outcomes from NPS If answer = 0-8, follow up by asking: “What would it take for you to recommend <our product> to someone you know?” https://www.netpromoter.com/know/
- 28. Key Takeaways 1. Measure outcomes, not vanity metrics. 2. Make sure your data is correct. 3. Choose the right type of visualization for the data. 4. Define KPIs based on your company’s strategy and top-line metric. 5. Experiment with small changes and foster a culture of continuous learning.
- 30. References • From Data to Viz - https://www.data-to-viz.com/ • Edward Tufte - https://www.edwardtufte.com/tufte/books_vdqi • Feature Factory by John Cutler - https://hackernoon.com/12- signs-youre-working-in-a-feature-factory-44a5b938d6a2 • Zappos case study - http://callcriteria.com/kpi-lessons-masters- customer-service-zappos-case-study/ • Pirate Metrics/Dave McClure - https://www.slideshare.net/ dmc500hats/startup-metrics-for-pirates-long-version • NPS - https://www.netpromoter.com/know/