Practical contextual bandits for business
Download as PPTX, PDF3 likes746 views
This document summarizes Yan Xu's presentation on practical applications of multi-armed bandits. Bandits can be used for personalized recommendation, such as recommending news articles, by balancing exploration of new articles with exploitation of known good articles. Amazon's bandit algorithm allows for real-time optimization of multiple variables by modeling interactions between variables. The algorithm was able to increase website conversion by 21% after a single week of optimization.
![CONTEXTUAL BANDITS
[0.1, 0.6]
[0.6, 0.4]
[0.7, 0.1]
[0.4, 0.2]](https://image.slidesharecdn.com/hml-bandits-190623230755/85/Practical-contextual-bandits-for-business-15-320.jpg)
Practical contextual bandits for business
- 1. PRACTICAL BANDITS FOR BUSINESS Yan Xu Houston Machine Learning Meetup June 22, 2019
- 2. OUTLINE - Recap on Bandit Problem - A Contextual-Bandit Approach to Personalized News Article Recommendation http://rob.schapire.net/papers/www10.pdf - An efficient bandit algorithm for realtime multivariate optimization https://www.kdd.org/kdd2017/papers/view/an- efficient-bandit-algorithm-for-realtime-multivariate- optimization
- 4. DILEMMA: EXPLORATION VS. EXPLOITATION The exploration/exploitation trade-off is a dilemma we frequently face in choosing between options. Stay the same route to drive home, or try a new route? Choose your favorite restaurant, or the new one? Listen to your favorite music channel, or try a new artist? Attend a new meetup?
- 5. HOW TO RESOLVE THE DILEMMA https://pavlov.tech/2019/03/02/animated-multi- armed-bandit-policies/ Epsilon Greedy UCB (Upper Confidence Bound) Thompson Sampling
- 15. CONTEXTUAL BANDITS [0.1, 0.6] [0.6, 0.4] [0.7, 0.1] [0.4, 0.2]
- 16. LINUCB ALGORITHM
- 17. LINUCB ALGORITHM
- 21. FEATURE FREE VS LINEAR CONTEXTUAL BANDIT
- 26. DEALING WITH HIGH DIMENSIONALITY ～1000 binary features per user; ~100 binary feature per article
- 29. RESULT: PERSONALIZED NEWS Omniscient: always chooses the article with highest empirical
- 30. CONCLUSION
- 31. AMAZON: BANDITS FOR MULTIVARIATE OPTIMIZATION Published at KDD 2017, KDD 2019 is in Alaska!
- 32. AMAZON: BANDITS FOR MULTIVARIATE OPTIMIZATION
- 35. STEP 1: PROBIT REGRESSION
- 37. STEP 3: HILLING-CLIMBING TO DECIDE
- 39. SIMULATION RESULT Control widget interaction in simulation through alpha_2.
- 40. EXPERIMENT ON REAL TRAFFIC • After only a single week of online optimization, we saw a 21% conversion increase compared to the median layout
- 41. SUMMARY Contextual bandits Linear payoff Add interaction components UCB: Variance estimation of expected rewards Thompson sampling: Sample weights from posterior distribution Applications Recommendation Multi-variate optimization For more details - A Contextual-Bandit Approach to Personalized News Article Recommendation http://rob.schapire.net/papers/www10.pdf - An efficient bandit algorithm for realtime multivariate optimization https://www.kdd.org/kdd2017/papers/view/an-efficient- bandit-algorithm-for-realtime-multivariate-optimization