Improving experimentation velocity via Multi-Armed Bandits
3 likes305 views
Multi-armed bandits are a technique that can improve experimentation velocity by actively choosing which variation or "arm" to test users on, balancing exploring all options and exploiting the best identified so far. In a presentation, Dr. Ilias Flaounas showed that using multi-armed bandits can reduce the number of samples needed in A/B tests by 45-80% compared to randomly assigning users. He also discussed an extension, kernelized contextual bandits, that can locate a target like the city of Bristol from tweets mentioning it using only 300 iterations, much better than searching all 10,000 candidate locations.
Improving experimentation velocity via Multi-Armed Bandits
- 1. Improving experimentation velocity via Multi-Armed Bandits Dr Ilias Flaounas Senior Data Scientist Growth Hacking Meetup, Sydney, 20 June 2016
- 4. Conversion rate PDF•In a classic A/B we pick where to assign the next user randomly. •In MAB we actively choose the cohort. Pick black to exploit Pick green (or red) to explore
- 10. Win for variation “d”.
- 11. Win for variation “d” and estimation of p-values
- 12. Let’s run it for a bit longer… Again, win for variation “d”. Classic A/B/C/D/E: ~2.5K samples Multi-armed bandit: ~1K samples 60% Less samples
- 13. No winner after 1K iterations Classic A/B/C: ~5K samples Multi-armed bandit: ~1K samples 80% Less samples
- 14. No winner after 1K iterations Classic A/B/C: ~2.8K samples Multi-armed bandit: ~1K samples 64% Less samples
- 15. Win for variation “a”. Classic A/B/C: ~1.8K samples Multi-armed bandit: ~1K samples 45% Less samples
- 16. Disadvantages • Reaching significance for non-winning arms takes longer • Unclear stopping criteria - App-specific heuristics • Hard to order non-winning arms and assess reliably their impact Advantages • Reaching significance for the winning arm is faster • Best arm can change over time • There are no false positives in the long term
- 18. • How can we locate the city of Bristol from tweets? • 10K candidate locations organised in a 100x100 grid • At every step we get tweets from one location and count the number of mentions of the word “Bristol” • Challenge: find the target in sub-linear time complexity!
- 19. • Contextual bandits can tackle this problem • We proposed the KernelUCB, a non- linear & contextual flavour of MAB. • The last few steps of the algorithm before it locates Bristol. Technical description: M. Valko, N. Korda, R. Munos, I. Flaounas, N. Cristianini, “Finite- Time Analysis of Kernelised Contextual Bandits”, UAI, 2013.
- 20. Target is the red dot. KernelUCB Matlab code: http://www.complacs.org/pmwiki.php/CompLACS/KernelUCB KernelUCB with RBF kernel converges after ~300 iterations (instead of >>10K).
- 22. Thank you! Yes, we are hiring Dr Ilias Flaounas Senior Data Scientist