The Unreasonable Effectiveness of Data
0 likes913 views
Scientific Revenue has developed a machine-learning approach to optimize pricing and user acquisition in mobile games by creating user segments that correspond to different price points. They utilize post-install behavior data to enhance user engagement and retention while predicting key user behaviors related to churn and spending. The document highlights best practices for monetization, centered on data collection, reporting, and predictive analytics, which can significantly increase revenue.
The Unreasonable Effectiveness of Data
- 1. ©2018 Scientific Revenue Confidential and not for redistribution The Unreasonable Effectiveness Of Data January, 2018 Pocket Gamer Connects, London
- 2. ©2018 Scientific Revenue Confidential and not for redistribution Abstract Over the past 4 years, Scientific Revenue has pioneered a machine-learning based approach to pricing. Simply put, Scientific Revenue uses machine learning to create segments of users that are then mapped to different price points. Over the past year, Scientific Revenue has also used the same underlying technologies to improve other aspects of mobile video games. In particular, Scientific Revenue has used post-install behavioral data to improve both user acquisition and engagement. In this talk Scientific Revenue’s Ted Verani will share how artificial intelligence can be used to build profiles for dynamic pricing that are also useful for user acquisition. Included is a case study from a successful customer implementation.
- 3. ©2018 Scientific Revenue Confidential and not for redistribution What if I Told You • Evidence is now clear that machine learning can dramatically improve your revenue • Scientific Revenue does IAP Pricing • But also • Help you acquire high value users • Help you decide whether to monetize a specific user via ads or via IAP • Help you retain high spenders • Increase retention • Optimize your in-game storefronts
- 4. ©2018 Scientific Revenue Confidential and not for redistribution What Scientific Revenue Does: Pricing to the Demand Curve The transition from “one-size fits all” pricing to targeted pricing is a key idea to maximizing IAP Revenue. Machine learning looks at post- install behavioral and purchase data to create a partition of the users, and send users to the right prices.
- 5. ©2018 Scientific Revenue Confidential and not for redistribution (Slightly) More about the Pricing Piece • At Pocket Gamer Connects in Helsinki, we gave a talk on using behavioral economics, big data and machine learning to optimize pricing • https://www.slideshare.net/ScientificRevenue/what-makes-a-price-a-good- price
- 6. ©2018 Scientific Revenue Confidential and not for redistribution The Biggest Takeaway from that Talk • There’s a large, and accessible body of knowledge on payment wall (aka coinstore) design • Many walls have the same monetization across the population (ARPU) • But they induce very different behaviors in the population. • That means that machine learning can target end-users with the appropriate pricing
- 7. ©2018 Scientific Revenue Confidential and not for redistribution User Acquisition and In-Game Ads
- 8. ©2018 Scientific Revenue Confidential and not for redistribution The Machine Learning Pyramid Gathering and Cleaning Data Reporting for Human Consumption Predictive Analytics Changing System Behavior
- 9. ©2018 Scientific Revenue Confidential and not for redistribution The Predictive Questions • Is this user about to churn? • When will this user churn? • How many more minutes will this user play? • Will this user be here a week from now? • Will this user buy an IAP? • Will this user buy more than one IAP? • How much money will the user spend? • Will this be a high value user? Solved, in the literature Solvable, not in the literature (yet)
- 10. ©2018 Scientific Revenue Confidential and not for redistribution The Literature • Churn prediction is solved. • Churn Prediction for High-Value Players in Casual Social Games – Runge et al. • Churn Prediction in Mobile Social Games: Towards a Complete Assessment Using Survival Ensembles – Perianez et all
- 11. ©2018 Scientific Revenue Confidential and not for redistribution Intuitively, You Know Your Data Predicts Outcomes • What is the number one predictor of spend? Engagement over time • What is the number two predictor of spend? A history of previous spend • What is the number three predictor of spend? Repeated interaction with the virtual economy • What is the number four predictor of spend? Did your friends spend? • Which spends more: an iPhone or an Android Device? It depends on which Android device • Does phone storage predict LTV? Yes, weakly – more storage correlates with higher spends
- 12. ©2018 Scientific Revenue Confidential and not for redistribution Facebook Value-based Look-a-likes
- 13. ©2018 Scientific Revenue Confidential and not for redistribution 1 Good Bad Use Machine Learning to Define Distinct Classes of Users
- 14. ©2018 Scientific Revenue Confidential and not for redistribution Empirical Results • Scientific Revenue has now engaged in a systematic exploration of Facebook Look-a-likes for the past year • Scientific Revenue generates a value-based look-a-like set using predictive LTV. • The data set is actually overweighted with low-value users (so that Facebook can focus away from zero-value users) • Example results (from a typical game): • Organic users are lowest value • Acquired users (done by outsourced UA) are worth 2X organic users • Look-a-like users cost 1.5X the professionally acquired users, but generate 6x the ARPU (on a 45 day basis) —Monetize much better, retain much longer.
- 15. ©2018 Scientific Revenue Confidential and not for redistribution Key Point: You Don’t Have to Be Perfect, Just Better • Your systems are already making predictions (you are advertising, and you are setting prices) • Only question is: are you using your data?
- 16. ©2018 Scientific Revenue Confidential and not for redistribution Putting it All Together: Ad Control • Take that red cluster from a few slides ago • Users in that cluster are *never* going to spend • We have a good idea of “never going to spend” within 3 days • We have a great idea of “never going to spend” within 7 days • So .,.. as the predictive models become more certain about a user., we can turn on ads.
- 17. ©2018 Scientific Revenue Confidential and not for redistribution Summary • There is a set of monetization-related best practices which are susceptible to machine learning techniques. • They all rely on the same basic practices: fine-grained data collection and cleaning, accurate and meaningful reporting, and predictive analytics. • They have a 90% overlap in the first three stages of the pyramid • Each practice gives you additional incremental revenue • The cumulative impact is potentially enormous
- 18. ©2018 Scientific Revenue Confidential and not for redistribution Thank You Ted Verani VP, Business Development ted@scientificrevenue.com (415) 999-4190