Being a product manager - session 4 (metrics)
- 1. Being a Product Manager Session 4: Metrics
- 2. Data is King. Intuition is important, but... Metrics!
- 3. Customer metrics 101 DAU - Daily Active Users WAU - Weekly Active Users MAU - Monthly Active Users Vanity metrics! Why?
- 4. Suppose you’re the PM of LinkedIn NewPosts
- 5. Always give context for your DAU/WAU/MAU
- 6. Churn Rate % of customers who stopped using your product over a specific time period. Calculation is simple. Take the number of customers that you lost last quarter and divide that by the number of customers that you started with last quarter. The resulting percentage is your churn rate. Example: SplitWise had 100 million users on January 1, 2019. It has 97 million users on April 1, 2020. What is the churn rate? 0.03
- 7. Customer metrics contd. ● Product usage/adoption (# of sessions, sign-in frequency, sharing, etc.) ● Percent of users who take a specific action that matters ● Feature usage (usage vs. other features) ● Which customer type is using certain features ● Retention or churn rate ● Quality (e.g. average bugs, net promoter score) Net promoter score measures the customers’ willingness to recommend a product or service to others. Source: https://www.medallia.com/net-promoter-score/
- 8. Types of metrics – Business metrics ● Customer lifetime = 1/Churn Rate ○ For example, 30% of our customers are leaving our business every year. What is the customer lifetime? ● Avg Revenue per User (ARPU) = (Total revenue)/# of users ○ For example, SottaIndustries had a revenue of $250mn with a total customer base of $100mn. What is the ARPU? ● Lifetime Value (LTV) = Monthly Revenue per user x Customer lifetime ○ For example, PrateekNetworks has a per user monthly revenue of $15 and a churn rate of 0.2. What is the customer LTV? ● Customer acquisition cost (CAC) = Marketing and sales expense/New customers ○ For example, HopperIndustries had an MSE of $5mn over the last quarter. It acquired 500,000 new customers over the same time period. What is the CAC? ● Click through rate (CTR) = % of people clicking on a link ● Any many more!
- 10. AARM method Acquisition: Tracking customer signups for a service. The bar for signing up for a service has gotten lower and lower, thanks to the popularity of free signup and pay later “freemium” models. The typical acquisition metric to track is lazy registrations or app downloads. Activation: Getting users that have completed a lazy registration to register fully. For a social networking site like Google+, this may include uploading a photo or completing their profile page. Retention: Getting users to use the service often and behave in a way that helps the user or business. Key metrics include adding more information to their profile page, checking the news feed frequently or inviting friends to try the service. Monetization: Collecting revenue from users. It could include the number of people who are paying for the service or the average revenue per user (ARPU).
- 11. Exercise 1 What are the top metrics you would track for an eCommerce website? Things to consider: ● Examples of top e-commerce websites include: ○ US: Amazon, Walmart, Apple ○ India: Amazon, Flipkart, Snapdeal ○ China: Tmall, JD, Suning ● Consider sales, marketing and website metrics ● Use AARM method: Acquisition, Activation, Retention, Monetization Source: The Product Manager Interview by Lewis C. Lin
- 12. Exercise 1 - Model Answer Acquisition ● Daily sessions ● Cost per acquisition (CPA) ● Cost per click (CPC) ● Cost per impression (CPM) ● Top search engine terms leading to website ● Mailing list click through rate (CTR) ● Mailing list open rate ● Mailing list conversion rate ● Mobile app downloads Activation ● New registered users ● Mobile app opens ● # of searches on app ● New cart started ● User info given (address, credit card, etc) ● New customers with successful purchase Retention ● Conversion rate ● Recommendation engine conversion rate ● Shopping cart abandonment ● Shopping cart size ● Visits from activated users per month Source: The Product Manager Interview by Lewis C. Lin
- 13. Exercise 1 - Model Answer Monetization ● Revenue per customer ● Lapsed customers ● Purchases per year ● Revenue per click ● Cost of sale (ad spend/revenue) ● Customer lifetime value (LTV) ● Cost of shipping Others ● Average listing position on Google ● COGS ● Shipping time ● Stockouts ● Returns ● Checkout errors ● # of reviews ● Market share ● Net margin Source: The Product Manager Interview by Lewis C. Lin Another question? What are the top metrics you would track for a two-sided marketplace? Do it with friends at home.
- 14. A popular (and easy) interview question...
- 15. Root cause analysis Product XYZ witnessed a 10% drop in DAU. Diagnose the issue.
- 16. Why do interviewers ask this question? ● Structure ● Fire-fighting ● Exhaustive list of issues ● Ambiguity
- 17. How to answer? ● Structured ● Insightful questions ● Think from various perspectives ○ Sudden drop vs progressive drop ○ Geography ○ Platform: mobile (iOS, Android), desktop, wearables, etc ○ Demographics: age, gender, etc. ○ Cannibalization: New feature release can hurt existing features ○ PR incident: Facebook political ads ○ Competitors: Released something new? ○ Technical issues
- 18. Exercise 2 ● Live mock ● Need a volunteer...
- 19. More questions related to metrics: 1. What are the metrics you would like to track for xyz feature? 2. Most important metric for xyz feature? 3. Why is it drop a drop and how can we change it?