Building a Data Driven Business
8 likes2,820 views
This document discusses how to build a data-driven business. It provides examples of how collecting and analyzing data can help optimize key performance indicators, measure the impact of initiatives, formulate hypotheses for experiments, and automate business processes. Data is presented as a key asset that can provide competitive advantages when used to personalize experiences, target communications, and inform innovative business experiments through rapid iteration. Building a data-driven culture and optimizing for learning from experiments, rather than just data collection, are emphasized as important factors for success.
Building a Data Driven Business
- 1. Building a data driven business Konstantin Savenkov, CEO, Intento
- 2. Konstantin Savenkov, PhD Chief Research Officer, Zvooq COO, Bookmate and Dream Industries now Founder & CEO
- 3. Turned a number of companies data-driven • B(2B)2C content services • edutainment • ad tech • commercial space management • consulting: online2offline, UGC, biotech Bookmate, Zvooq Theory&Practice, Exchanges Unisound DI Telegraph @ksavenkov
- 4. Without data, people take decisions based on biases and beliefs @ksavenkov
- 5. …data replaces beliefs and provides a competitive advantage @ksavenkov
- 6. If you’re competitor-focused, you have to wait until there’s a competitor doing something. Being customer-focused allows you to be more pioneering. Jeff Bezos Amazon@ksavenkov
- 7. …data helps to be customer-centric @ksavenkov
- 8. Information is the oil of the 21st century, and analytics is the combustion engine. Peter Sondegaard Gartner@ksavenkov
- 9. …check the gas quality and do the maintenance @ksavenkov
- 10. Half of the money spent on advertising is wasted; the trouble is I don’t know which half. John Wanamaker or William Lever @ksavenkov
- 11. …channel attribution and ROI will tell you @ksavenkov
- 12. The average conversion rate in the United States whether you are selling elephants or iPods, will be 2% (с) Авинаш Кошик @ksavenkov
- 13. …data get you above the average @ksavenkov
- 14. Mastery Business Metamorphosis Data Aware Data Monetisation Data-driven Business Optimisation Analytics Business Insights Data-Driven Maturity Index Collect Data Business Monitoring @ksavenkov
- 15. @ksavenkov
- 16. To be data driven, it’s not enough to collect and mine data. You need the culture. @ksavenkov
- 17. Don’t optimise for data! Optimise for learning! @ksavenkov
- 18. All initiatives must improve KPI Formulate hypotheses Run experiments @ksavenkov
- 19. Data validate hypotheses behind your own decisions. Data does not make decisions for you! @ksavenkov
- 22. Looks simple MRR MarginMAU but there are issues @ksavenkov
- 23. IT’S UNCLEAR who’s responsible? how to imrove? how does that relate with resources spent? (besides CEO) (besides “work better”) (when most of the business processes are automatic) @ksavenkov
- 24. KPI Trees MAU New Loyal Returned Traffic Conversion Retention Reactivation @ksavenkov
- 25. KPI Trees MRR MRR stable net new MRR list MRR MRR upgrades MRR downgrades New clients MRR expansion new MRR Reactivation direct sales @ksavenkov
- 27. ADVANCED MODE Ratio Analysis Cohort Analysis @ksavenkov
- 28. HELPS TO Identify bottlenecks Define roles and responsibilities Generate ideas Measure impact of initiatives @ksavenkov
- 30. For planning, groups tasks in Epics that matter Track results of launched Epics A culture of data success learning @ksavenkov
- 31. Case study Marketing project were tracked until launched Study of a half-year long cross-promo campaign discovered a significant loss via COGS. We’e identified key mechanics that led to the loss. We’e adjusted the mechanics, re-negotiated with partners on the ongoing campaigns. PROFIT @ksavenkov
- 33. It doesn’t work this way: marketing campaign @ksavenkov impact!
- 34. product feature launched billing failure pupils back from holidays AppStore featuring people back from vacations @ksavenkov It doesn’t work this way: marketing campaign impact!
- 35. product feature launched billing failure pupils back from holidays AppStore featuring people back from vacations @ksavenkov It doesn’t work this way: marketing campaign impact! DIFFERENT CHANNELS CONVERT WITH A DIFFERENCE IN ORDER OF MAGNITUDE
- 36. ALSO: A lot of Epics that shouldn’t have been launched at all A culture of praising a random success and explaining failures by external causes Impossibility to learn by mistakes @ksavenkov
- 37. formulate measurable hypotheses carefully plan experiments define a condition of success/failure prior to implementation data collection and attribution, split- testing, controlled variables, statistical significance demonstrate explicit risks use models built on past data prioritisation aid and how the failure affects the roadmap ability to prove multiple hypotheses simultaneously @ksavenkov
- 38. Case Studies Estimating mechanics of marketing projects before they launched Product improvement that results in 2.5x conversion, 2x lifetime Split testing of targeting and creative materials for ad campaigns, resulting in conversion 2-3 times higher than organic Immediate increase of “conversion” from all initiatives to successful ones This approach is behind all conclusions in these slides lots of failed experiments success despite of all external factors NO OTHER WAY@ksavenkov
- 39. Case Study: Recommender system for Conversion • Hypotheses to prove: 1. There’re enough users who will use RS output 2. Their conversion will be above average • A/B testing is the only way: – different channels convert with up to 20x difference – current traffic mix is unpredictable and hard to control in case of app installs • Do pilots: – Run with limited resources, then extrapolate and decide if run full-scale
- 40. Case Study: Recommender system for Conversion
- 41. Let’s look at the economics • In case of using a third-party RS on a CPO basis, in this case the CPO is limited by $0.14 (actually, much less) • In case of a flat fee of $1000**/month, this is feasible starting from 7143 new subscribers/month, or $35K of marketing budget. * CAC and marketing budget are model data ** some arbitrary number
- 43. Make decisions on monthly KPI or finished projects is like going backwards @ksavenkov
- 44. Look ahead or at least watch your step, not backwards Make the data work for you @ksavenkov
- 45. Daily indicators Incremental indicators Leading indicators Predictive models spot problems and anomalies just in time example: baremetrics.io a perfect input for inbound marketing accurate goals and perfect financial planning @ksavenkov
- 46. Case Study Some loyal subscribers churned away for 2-3 months, to come back later. It was historically attributed to holidays and other external factors. The daily indicators have shown that all churned users subscribed on weekdays. What a riddle! It turned out that on weekdays the code base is frequently deployed to the production server, flushing the message queue of subscription renewals. The fix increased a lifetime of paying users by 20% @ksavenkov
- 47. Case Study Predicting a lifetime for users that registered right now (10% accuracy) Accurate unit economics for contracts with B2B2C partners, content providers, pricing @ksavenkov
- 48. Case Study A probabilistic model for segmenting users Input data for chained communication (inbound marketing) @ksavenkov
- 49. Case Study Accurate goals loyal new churned guaranteed growth? inevitable stagnation!* *unless KPI are increased@ksavenkov
- 50. Case Study Accurate financial planning Operational model Marketing plan-fact Financial plan-fact Marketing budget CAC, conversion organic forecast deal terms unit economics deals forecast revenue forecasts @ksavenkov
- 52. Hire another employee or train another model? @ksavenkov
- 53. Affects all business processes that scale linearly with a headcount: customer support editorial office content management marketing @ksavenkov
- 54. Case Studies User base grows from 1M to 2M, doubling the headcount in customer support? Implemented auto-reply using our knowledge base and smart templates for support engineers A number of markets increased, adding more editors? Created an algorithm to provide a short-lists based on a user behavior An amount of UGC explodes, more content-managers? Improving reduplication and computer aid based on the collected data @ksavenkov
- 56. Operational analytics - improve the process of KPI improvement (productivity) @ksavenkov
- 57. ITERATION Case Study Improving the Agile process the expectations: @ksavenkov ROADMAPS DESIGN DEVELOPMENT QA SHIPPED PLANNING
- 58. ITERATION Case Study Improving the Agile process the reality: @ksavenkov INFLATING EFFORT ROADMAPS DESIGN DEVELOPMENT QA SHIPPED BACKLOG NEW STUFF TECH. DEBT PLANNING BUGS UX SOFTWARE OPS URGENT STUFF UNCLEAR DESIGN UNCLEAR TECH ITERATING
- 59. Data as an Asset @ksavenkov
- 60. Case Stuies • Compare B2B2C deals through unit economics • Estimate traffic quality for partner ad networks • Data partnerships • Targeted user communication • Personalisation and recommender systems (the next slide) • Bonus track: Investigating a large number of purchase returns for an internet retailer @ksavenkov
- 61. Using Recommender Systems and personalisation CAC LTV Content Costs Marketing Expenses New Customers ARPU Lifetime Consumed Content Mix Conversion Retention Reactivation Exposed Content Mix ÷ × * the recommendation fairy *
- 62. Innovative business experiments, as there’s no recipes To pioneer, you need to iterate quicker than competitors @ksavenkov
- 63. Data, models built over the data and experimental results is the main asset created and exploited by the innovative business @ksavenkov “THE UNFAIR ADVANTAGE”