Predictive Analytics Broken Down
3 likes2,299 views
Conductrics is a cloud-based platform that offers adaptive testing, decision engines, and personalized customer experiences through predictive analytics. The document discusses various methods and requirements for personalization, particularly using linear regression and sequential learning techniques to predict customer behavior. It also highlights the importance of accurate customer data and decision logic to maximize marketing effectiveness and reduce waste.
![How to come up with the Logic?
IF [Customer]
THEN
[Experiences?]](https://image.slidesharecdn.com/predictiveanalyticsbrokendown-160208162328/85/Predictive-Analytics-Broken-Down-14-320.jpg)
![How to come up with the Logic?
IF [Customer]
THEN
[Experiences?]
…in way that Optimizes Performance](https://image.slidesharecdn.com/predictiveanalyticsbrokendown-160208162328/85/Predictive-Analytics-Broken-Down-15-320.jpg)
![How to come up with the Logic?
IF [Repeat and/or
Weekend]
THEN [High/Low Price?]
…in order to be most profitable](https://image.slidesharecdn.com/predictiveanalyticsbrokendown-160208162328/85/Predictive-Analytics-Broken-Down-20-320.jpg)
![twitter: @mgershoff
Targeting Logic as Rule
IF [Weekend]
THEN [Low]
Else [High]](https://image.slidesharecdn.com/predictiveanalyticsbrokendown-160208162328/85/Predictive-Analytics-Broken-Down-57-320.jpg)
![twitter: @mgershoff
Targeting Logic as Rule
IF [Weekend]
THEN [Low]
Else [High]
Expressing the logic as a set of succinct
rules is generally a hard problem](https://image.slidesharecdn.com/predictiveanalyticsbrokendown-160208162328/85/Predictive-Analytics-Broken-Down-58-320.jpg)
Predictive Analytics Broken Down
- 2. CEO / Co-Founder Conductrics www.conductrics.com matt@conductrics.com Past: Database Marketing Education: Artificial Intelligence & Economics twitter:@mgershoff, @conductrics Email:matt@conductrics.com www.conductrics.com/blog Who is this guy?
- 3. Confidential 1. Cloud-based Adaptive Testing and Decision Engine 2. API-Based Testing, Targeting and Optimization • REST API: Compatibility with CMS systems and other platforms • Native Programming Wrappers (iOS, PHP, jQuery, Node, etc.) • New JavaScript API for super fast decisions at scale 3. “WAX” Framework for point-and-click style customers • Client-side, tag-based, “skip IT” style implementation 4. Browser UI • Admin Console • Reporting What is Conductrics? twitter:@mgershoff, @conductrics Email:matt@conductrics.com
- 4. Confidential 1. Experimentation • AB and Multivariate Testing • Adaptive / Bandit Testing 2. Personalization • Targeting with Business Logic • Targeting via machine learning What does Conductrics do? twitter:@mgershoff, @conductrics Email:matt@conductrics.com
- 5. Confidential Promise of Predictive Analytics The Promises: • Help make predictions about the future • Predictions about customer: • Preferences • Intent
- 6. Confidential Benefits of Predictive Analytics The Benefits: • Provide customers with right set of experiences • Eliminate marketing waste
- 7. Confidential Why care how it works? • Better consumer of predictive analytics tools • How to get the most out of it predictive analytics • Help ensure you understand its limitations
- 8. Confidential Scope of talk? All Predictive Analytics Uses Transactional System
- 9. 9 twitter: @mgershoff 1.Data 2.Logic Two Requirements for Personalization
- 10. Data: ‘Sensing’ the World
- 11. Conductrics twitter: @mgershoff Blog: www.conductrics.com/blog • Return Customer • Weekend/Weekday • Mobile/DeskTop • Browser Type • User Age • Geo/Census • Weather • Tenure/RFM Score Observable • Lottery Game • Price • Sales Offers • Shipping Type • Layout/UX • Which Products • Suggested Quantities Intervention Types of Data
- 12. 12 twitter: @mgershoff 1.Data 2.Logic Two Requirements for Personalization
- 13. Confidential Requirements for Personalization Decision logic links Observations to Actions
- 14. How to come up with the Logic? IF [Customer] THEN [Experiences?]
- 15. How to come up with the Logic? IF [Customer] THEN [Experiences?] …in way that Optimizes Performance
- 17. twitter: @mgershoff Example: Lottery Games Show high price games …
- 18. Or show the low price games twitter: @mgershoff Example: Lottery Games
- 19. To keep it simple just look at: • New or Repeat Player • Weekday or Weekend twitter: @mgershoff Example: Lottery Games
- 20. How to come up with the Logic? IF [Repeat and/or Weekend] THEN [High/Low Price?] …in order to be most profitable
- 21. How its Done 1 Learn how Repeat and Weekend customers predict low price games twitter: @mgershoff
- 22. How its Done twitter: @mgershoff 2 Learn how Repeat and Weekend customers predict high price games 1 Learn how Repeat and Weekend customers predict low price games
- 23. How its Done 1 Learn how Repeat and Weekend customers predict low price games twitter: @mgershoff 2 Learn how Repeat and Weekend customers predict high price games 3 Then compare for each customer (Choose the one with the highest value)
- 24. @mgershoff • Deep Learning Nets • Decision Trees • Gaussian Process (is a Bayesian method) • Support Vector Machines • KNN - actually kinda like segmentation • Naive Bayes (is NOT a Bayesian method) • Logistic Regression Predictive Analytics Methods http://conductrics.com/data-science-resources/ http://conductrics.com/data-science-resources-2
- 25. We are going to use Linear Regression @mgershoff
- 26. twitter: @mgershoff Why Linear Regression? Benefits: 1.Has nice Statistical Properties 2.Easy(ish) to interpret 3.In practice, often all you need
- 27. twitter: @mgershoff What is Linear Regression A model of relationships in this form:
- 28. twitter: @mgershoff What is Linear Regression A model of relationships in this form: Prediction = Base + B1*Attribute1 … + Bj*Attributej
- 29. twitter: @mgershoff What is Linear Regression A model of relationships in this form: Prediction = Base + B1*Attribute1 … + Bj*Attributej Just Add up all of the customer ‘attributes’ by the impact (B) of the Feature
- 30. twitter: @mgershoff What is Linear Regression We will learn two models, one for each game:
- 31. twitter: @mgershoff What is Linear Regression We will learn two models, one for each game: Game High = BaseH + WH*Weekend + RH*Return
- 32. twitter: @mgershoff What is Linear Regression We will learn two models, one for each game: Game High = BaseH + WH*Weekend + RH*Return Game Low = BaseL + WL*Weekend + RL*Return
- 34. twitter: @mgershoff Benefits of Sequential Learning 1.Don’t have to wait to collect the data
- 35. twitter: @mgershoff Benefits of Sequential Learning 1.Don’t have to wait to collect the data 2.Constantly updating you can use it real time
- 36. twitter: @mgershoff Benefits of Sequential Learning 1.Don’t have to wait to collect the data 2.Constantly updating you can use it real time 3.Scalable–any real production PA is almost certainly going to use the method
- 37. twitter: @mgershoff Benefits of Sequential Learning 1.Don’t have to wait to collect the data 2.Constantly updating you can use it real time 3.Scalable–any real production PA is almost certainly going to use the method 4.The computations are simple to understand
- 39. twitter: @mgershoff The Sequential Algorithm in words 1) Observe the data for a single customer 2) Using the current parameter values to make a prediction 3) See how far off your predicted value was from the actual value 4) Use how far off you prediction was to update your parameter values 5) Adjust how much you update by something like O(1/n) – this is the Learning Rate, sort of like an average 6) Repeat
- 40. twitter: @mgershoff ParameterNew:= Parameterold - Adjustment The Sequential Algorithm Current Value The Difference (Error) of the actual value and the predicted result Adjustment = (Predicted - Actual) * Learning Rate
- 41. twitter: @mgershoff How it is done: No data yet, high cost game Hidden Base R W Return WkEnd Sales Predict Error 0 0 0 What We Know
- 42. twitter: @mgershoff Hidden Base R W Return WkEnd Sales Predict Error 0 0 0 0 1 1.00 What We Know Observe New Customer on Weekend
- 43. twitter: @mgershoff Hidden Base R W Return WkEnd Sales Predict Error 0 0 0 0 1 1.00 What We Know Observe New Customer on Weekend Prediction= BaseH + WH*Weekend + RH*Return
- 44. twitter: @mgershoff Hidden Base R W Return WkEnd Sales Predict Error 0 0 0 0 1 1.00 0 -1.00 What We Know Plug in values 0 = 0 + 0*0 + 0*1
- 45. twitter: @mgershoff Hidden Base R W Return WkEnd Sales Predict Error 0 0 0 0 1 1.00 0 -1.00 What We Know Updated Base R W 1 0 1 Update Base and Weekend Impact Score
- 46. twitter: @mgershoff Hidden Base R W Return WkEnd Sales Predict Error 1 0 1 0 0 2.00 What We Know Observe New Customer Weekday Prediction= BaseH + WH*Weekend + RH*Return
- 47. twitter: @mgershoff Hidden Base R W Return WkEnd Sales Predict Error 1 0 1 0 0 2.00 1.00 -1.00 What We Know Plug in values 1 = 1 + 0*0 + 1*0
- 48. twitter: @mgershoff Hidden Base R W Return WkEnd Sales Predict Error 1 0 1 0 0 2.00 1 -1.00 What We Know Updated Base R W 1.5 0 1 Update Just the Base Impact Score
- 49. twitter: @mgershoff Hidden Base R W Return WkEnd Sales Predict Error 1.5 0 1 1 0 3.00 What We Know Observe Return Customer on Weekday Prediction= BaseH + WH*Weekend + RH*Return
- 50. twitter: @mgershoff Hidden Base R W Return WkEnd Sales Predict Error 1.5 0 1 1 0 3.00 1.50 -1.50 What We Know Plug in values 1.5 = 1.5 + 0*1 + 1*0
- 51. twitter: @mgershoff Hidden Base R W Return WkEnd Sales Predict Error 1.5 0 1 1 0 3.00 1.50 -1.50 What We Know Updated Base R W 1.75 0.75 1 Update the Base and Return Impact Score
- 52. twitter: @mgershoff Online Regression After 200 Iterations Base R W 2.0 1.0 -1.0 High Price Model Results Sales = 2.0 + 1.0*Return -1.0*Weekend
- 53. twitter: @mgershoff Online Results: 200 Iterations -1.50 -1.00 -0.50 0.00 0.50 1.00 1.50 2.00 2.50 0 50 100 150 Parameter Value Observations Base Level Returning Parameter Weekend Parameter
- 54. 54 matt@conductrics.com; www.conductrics.com Back to Our Task Model: High Price Game High = 2.0 + 1.0*Return -1.0*Weekend
- 55. 55 matt@conductrics.com; www.conductrics.com Back to Our Task Model Low Price Game Low = 1.0 + 1.0*Return + 0.5*Weekend
- 56. twitter: @mgershoff Tabular Targeting Logic Returning Weekend High Price Low Price Selection N N 2.0 1.0 High Y N 3.0 2.0 High N Y 1.0 1.5 Low Y Y 2.0 2.5 Low
- 57. twitter: @mgershoff Targeting Logic as Rule IF [Weekend] THEN [Low] Else [High]
- 58. twitter: @mgershoff Targeting Logic as Rule IF [Weekend] THEN [Low] Else [High] Expressing the logic as a set of succinct rules is generally a hard problem
- 59. How to evaluate our targeting logic? www.conductrics.com/blog
- 60. Conductrics twitter: @mgershoff Blog: www.conductrics.com/blog A/B TEST!!! How do we measure our Targeting?
- 62. www.conductrics.com/blog Random Selection as Control Good: 1. A baseline we can always use 2. Selecting randomly is often good policy when we don’t have any additional information Bad: Often hard to beat (which is Good) Random ain’t too bad
- 63. Users 50/50 Model Evaluation: AB Style Low Model Standard Control Random Play www.conductrics.com/blog
- 64. Users 50/50 Compare the Results of Each Low Model Standard Control Random Play www.conductrics.com/blog
- 65. Random Results: Model ROI/Lift Target - Random Lift = www.conductrics.com/blog
- 66. www.conductrics.com/blog Beware the Faustian Bargain! Description Michał Elwiro Andriolli Date 1895 Source http://www.pinakoteka.zascianek.pl/Andriolli/Index.htm
- 67. www.conductrics.com/blog Customer data needs to be ‘accurate’ AND available at decision time We need to create AND manage experiences and content Need to create AND mange our decision logic Difficult to know what state system is in. Before just one state, now many. Targeting = Complexity
- 68. 𝑽𝒂𝒍𝒖𝒆 𝑪𝒐𝒎𝒑𝒍𝒆𝒙 𝑺𝒚𝒔𝒕𝒆𝒎 − 𝑽𝒂𝒍𝒖𝒆(𝑺𝒊𝒎𝒑𝒍𝒆) 𝑽𝒂𝒍𝒖𝒆(𝑺𝒊𝒎𝒑𝒍𝒆) www.conductrics.com/blog Targeting and the ROI of Complexity
- 70. Image Source: http://en.wikipedia.org/wiki/File:Carl_Sagan_-_1980.jpg; Just 20 Customer Features 1 Million Combinations @mgershoff power(2,20) assuming all non mutually exclusive Segmentation Doesn't Scale